Hi and welcome to the first part of this ‘Introduction to AI Engineering’ tutorial series! In this part we’ll first take a look at ChatGPT, the different options and models available, and how to use it via the API with Python so we can automate our ChatGPT calls.
What you’ll need
To follow along with this tutorial series, you’ll need a code editor. I recommend using Visual Studio Code, but you can use any code editor you like. If you don’t have a code editor installed, you can download Visual Studio Code from here.
You’ll also need to have Python installed on your computer. If you don’t have Python installed, you can download it from here. If you’re asked whether you want to add Python to your PATH, make sure to check yes.
The last thing we’ll need is a ChatGPT account. If this is your first time using it, don’t worry, you will get a bunch of free credits after signing up for your first account so you should be able to follow along without having to pay any money.
Go to https://platform.openai.com/ and log in. If you already use OpenAI and already have an account set up with a free or paid API key, you can use that. If you don’t have an account just log in with your Google account. It will ask you something simple, like to fill in your birthday, and ta-da, you have an account!
When you log in on a brand new account you will see something like this (navigate to the dashboard if you land on a different page):
Find API keys in the left sidebar and click on it. If this is a new account it will ask you to verify your phone in order to create a new API key:
The reason they do this is to prevent bots from creating loads of free accounts and abusing their system. Just give them a phone number and they will send you a verification code to enter. You will also get a bunch of free credits from them to follow along with this tutorial, so it’s a win-win!
Find the green button to + Create new secret key in the top right corner and click on it:
In the next window, you can leave everything as is. You don’t need to give it a name or select a project. You can do these things if you want to, but I’ll just create a nameless general key for now by accepting everything as is and clicking the green Create secret key button:
You will now see your new API key:
So make sure you press the Copy button and save it somewhere safe, maybe a password manager. You won’t be able to see this key again, though you can always generate a new one if you lose it. Make sure not to share your key as anyone with your key can use your credits!
Now that we have our API key, we can get started!
Models available and pricing
If we go to the pricing page on the OpenAI website, we can see that several models are available for the newest GPT-4o model. At the time of writing this tutorial, models and their costs look like this:
As you can see, there are currently three versions of GPT-4o available, so what is going on here? gpt-4o is the original and first version of the model whereas gpt-4o-2024-05-13 is an updated version thereof, and gpt-4o-2024-08-06 is the latest update. To further complicate matters simply inputting the generic gpt-4o will tend to automatically use the latest version available, which is gpt-4o-2024-08-06 in this case, so you don’t have to worry about the specific version.
As you can see from the pricing of $5 per 1m input tokens and $15 per 1m output tokens for the other model, the latest gpt-4o-2024-08-06 version is considerably cheaper at $2.50 per 1m input tokens and $10 per 1m output tokens. This is nice as it encourages us to use the latest and most up-to-date model. Now this may sound very expensive but 1 million tokens is a very large amount. If we look at the same prices but per 1000 tokens it looks a lot different:
So what is a token exactly? A token in the context of a Large Language Model (LLM) like GPT-4o is a piece of text that the model processes. The model uses these tokens to understand and generate text, with each token representing a chunk of the input or output data.
As an LLM does not think in words like we humans do, a token in the English language is typically about 3-4 text characters long. So as a rough generalization, you can think of 1000 input tokens as about 3000-4000 characters of text you send to the model, and 1000 output tokens as about 3000-4000 characters of text the model sends back to you. (Note that this can vary depending on the specific text and language used.)
The GPT-4o model shown above is the currently most powerful and capable flagship model. But as LLMs are quite powerful in general and we often ask them to do simple tasks or answer simple questions, we often don’t need quite this much firepower.
It would be kind of like sending a tank to pick up groceries at the store. It can do it, but it’s a bit overkill. Imagine you’re coming up with your tank and brraaawhh and all the roads get destroyed, traffic signs falling over and the store is in ruins and you’re like “I just wanted some milk and bread…”.
Luckily OpenAI has more suitable budget models for this purpose. Where this used to be gpt-3.5-turbo in the past, the new GPT-4o model has its own budget version called gpt-4o-mini which replaces the aforementioned 3.5-turbo. If we look at the pricing we can see it is over 15 times cheaper both input and output token-wise compared to the flagship model: