OpenAI Fine-Tuning (4/4): Training and Running ChrisGPT

Hi and welcome back to part 4, where we’ll be training and running ChrisGPT. In this part, we’ll finally be using the OpenAI fine-tuning API endpoints, which are fairly simple!

There are two ways to use the fine-tuning API, both of which are very simple. The first way to do this is programmatically, using Python code just like we do when sending normal calls to ChatGPT. We’ll be looking at this first. The second way is to use the web interface for the fine-tuning API.

Using the fine-tuning API programmatically

Go ahead and create a new file called chris_gpt_training.py in the root directory of your project:

📁Finx_Fine_Tuning
    📁data
        ...
    📁utils
        ...
    📄.env
    📄chris_gpt_dataset_generator.py
    📄chris_gpt_dataset_validation.py
    📄chris_gpt_training.py ✨
    📄constants.py
    📄Pipfile
    📄Pipfile.lock

So let’s start with our imports and basic setup:

from constants import CLIENT, DATA_DIRECTORY


JSONL_FILE = DATA_DIRECTORY / "Finx_completed_dataset.jsonl"
MODEL = "gpt-3.5-turbo-1106"
SUFFIX = "chris_gpt"

We import the OpenAI Client we stored in CLIENT and DATA_DIRECTORY. Then we quickly set up a path to the JSONL data for training (make sure you don’t accidentally use the json instead of jsonl one). We also set the model to the 1106 version as this is the newest one that has fine-tuning. My testing showed 1106 to be significantly better for fine-tuning than the older 0613 version.

Finally, the SUFFIX part will allow us to choose a part of the fine-tuning model’s name ourselves. This suffix will become part of the name you use to call your model, which is quite useful for identification as the model names are a bit long and all extremely similar if you don’t have a suffix. An example:

# Example model name without suffix
ft:gpt-3.5-turbo-1106:personal::8ot8ZLJR

# Example model name with suffix
ft:gpt-3.5-turbo-1106:personal:chris-gpt:8ot8ZLJR

Now let’s have a look at the file related methods in the openai client. Make sure you don’t run the file yet:

# File related methods
file = CLIENT.files.create(file=open(JSONL_FILE, "rb"), purpose="fine-tune")

CLIENT.files.list(purpose="fine-tune")

CLIENT.files.delete(file.id)

These methods are all fairly self-explanatory. The create method will upload your file, make sure to use the rb (read-binary) mode for the file and provide the purpose as fine-tune so that the OpenAI servers know what this file is for. This returns an object we catch in the file variable. It looks like this and contains some basic file data, most importantly the id:

# Example of a file object

FileObject(
    id="file-DamWAnhgpnRvEyMZ3dOdHpvG",
    bytes=865053,
    created_at=1708303339,
    filename="Finx_completed_dataset.jsonl",
    object="file",
    purpose="fine-tune",
    status="processed",
    status_details=None,
)

The list method will list all files, allowing us to filter on a purpose, so we’ll filter for files with the purpose of fine-tune here. It just returns a list of the FileObject objects you see above. The delete method will delete a file by its id, using whatever ID you pass in to delete.

Fine-tuning methods

Now let’s take a look at the fine-tuning-job related methods. Again, don’t run this file yet, let’s just have a quick look first:

# Fine-tuning-job related methods
fine_tuning_job = CLIENT.fine_tuning.jobs.create(
    model=MODEL,
    training_file=file.id,
    hyperparameters={"n_epochs": 3},
    suffix=SUFFIX,
)

CLIENT.fine_tuning.jobs.list()

CLIENT.fine_tuning.jobs.retrieve(fine_tuning_job.id)

CLIENT.fine_tuning.jobs.cancel(fine_tuning_job.id)

The most important method is of course the create method, which will create a fine-tuning job. You pass in the model and the file.id of the file you want to do the training on. You can optionally pass in hyperparameters such as the number of epochs we discussed earlier, and the suffix we talked about. This method returns a FineTuningJob object, which looks like this:

FineTuningJob(
    id="ftjob-1OATxnQAgdY4yjPNmSBai95f",
    created_at=1708318423,
    error=Error(code=None, message=None, param=None, error=None),
    fine_tuned_model=None,
    finished_at=None,
    hyperparameters=Hyperparameters(
        n_epochs=3, batch_size="auto", learning_rate_multiplier="auto"
    ),
    model="gpt-3.5-turbo-1106",
    object="fine_tuning.job",
    organization_id="org-oMYMXpp7Cr9pG1rG5Z8a1T2w",
    result_files=[],
    status="validating_files",
    trained_tokens=None,
    training_file="file-EX13iLyISBZcreRCH3Fm51Pn",
    validation_file=None,
)

We can see that the FineTuningJob object also has an ID that we can use to refer to it, and some basic info, such as the fact that this one has not finished yet. We can see the hyperparameters, including the batch_size and learning_rate_multiplier which are set to auto by default.

The batch_size is the number of examples in each training batch. Batch size is the amount of data the model looks at before it learns something new. It’s like reading a few pages of a book, and then stopping to think about what you’ve read before continuing. At the end of the batch, the predictions are compared to the desired output, and the error is calculated and used to update the model. We’ll just leave this set to auto.

The learning_rate_multiplier is a value that multiplies the learning rate of the model. The learning rate is a hyperparameter that controls how much to change the model in response to the estimated error each time the model weights are updated. If this is set to high the model may very easily overfit to the specific data it has been trained on. We’ll also leave this set to auto.

The list method, again, simply returns a list of all the FineTuningJob objects for your account. The retrieve method will return a single FineTuningJob object by its id and the cancel method will cancel a fine-tuning job by its id. These methods are extremely simple and there really is nothing more to them.

Again, don’t run this fine yet. The last method we have is for deleting a completed fine-tuned model:

# Fine-tuned-model related methods
CLIENT.models.delete("model_id_here")

For this, you need to have a fully trained model to delete obviously, and then just pass in the ID. Before we actually go ahead and run this file, I’ll comment out several of the methods to leave them in for our reference, as we naturally don’t want to delete the file we just uploaded nor cancel the fine-tuning-job:

from constants import CLIENT, DATA_DIRECTORY


JSONL_FILE = DATA_DIRECTORY / "Finx_completed_dataset.jsonl"
MODEL = "gpt-3.5-turbo-1106"
SUFFIX = "chris_gpt"


# File related methods
file = CLIENT.files.create(file=open(JSONL_FILE, "rb"), purpose="fine-tune")

print(CLIENT.files.list(purpose="fine-tune"))

# CLIENT.files.delete(file.id)


# Fine-tuning-job related methods
fine_tuning_job = CLIENT.fine_tuning.jobs.create(
    model=MODEL,
    training_file=file.id,
    hyperparameters={"n_epochs": 3},
    suffix=SUFFIX,
)

# CLIENT.fine_tuning.jobs.list()

print(CLIENT.fine_tuning.jobs.retrieve(fine_tuning_job.id))

# CLIENT.fine_tuning.jobs.cancel(fine_tuning_job.id)


# Fine-tuned-model related methods
# CLIENT.models.delete("model_id_here")

I’ve gone ahead and added print statements around the files.list and the fine_tuning.jobs.retrieve calls so we can see the output in our terminal. Now go ahead and run this file to start your fine-tuning job!

You should see the FineTuningJob object printed to the console. From here on we’ll be switching to the web interface for the fine-tuning API, as it’s much easier to work with and has real-time progress.

Using the web interface for the fine-tuning API

As developers we sometimes think we need to do everything programmatically, but there really is no need if we have an easy web interface that is much simpler to use. There really is no point in coding up some complex programmatic solution for something you’re only going to be doing once in a while.

If you have a look at https://platform.openai.com/files, you will see the web interface for managing the files you uploaded to OpenAI:

File API:

This interface is much nicer to read than the long file object list in your terminal window, and you can also upload new files here directly.

Now let’s switch to the fine-tuning tab at https://platform.openai.com/finetune, to see the fine-tuning job that we have started:

Fine-tuning jobs API:

As you can see I’ve done quite some testing so I have a whole load of models here, but you should see your model either “Validating files…” or maybe already in the training stage. You can also create new fine-tuning jobs on this page or cancel a running fine-tuning job using the cancel button. We can also see a lot of details on our fine-tuning job as it progresses:

Fine-tuning details:

Please ignore all the “Cancelled” fine-tunes in my list, I tend to double-check the code when writing these things so I triggered a lot of fine-tunes that were not needed and therefore canceled them straight after. The important thing here is first of all, when the model is done training, you will see the full model name for your fine-tune. In this case, I have ft:gpt-3.5-turbo-1106:personal:chris-gpt-full:8ot8ZLJR as my model name here, but yours is probably not done training yet.

It can take anywhere from a couple of minutes to potentially even hours for OpenAI to train your model, depending on the size of the data and how busy the servers are. Mine typically finished within 10 to 30 minutes, and OpenAI will send you an email when the model training is done. In the meantime, you can see the progress update in real-time in the right-side panel here, where you can see the training loss go down in real-time. You can also see the messages at the bottom keeping you posted on the progress.

So go ahead, it’s time again for your well-deserved coffee break☕ with your favorite snack🥨. Have a small break and give your model time to train, and I’ll see you back here when it’s done!

Running our own fine-tuned ChrisGPT model!

Ok, so I assume you’re back and your model is done training, which means you now have the name of your personal fine-tuned model from the https://platform.openai.com/finetune fine-tuning page. The name will also be in the email you receive when the training is done. First, go and open the constants.py file in your root directory, as we want to make some updates to the file:

from pathlib import Path

from decouple import config
from openai import OpenAI


CLIENT = OpenAI(api_key=str(config("OPENAI_API_KEY")))
DATA_DIRECTORY = Path(__file__).parent / "data"
## Add the below variables
OUTPUT_DIRECTORY = Path(__file__).parent / "output"
CHRIS_GPT = "ft:gpt-3.5-turbo-1106:personal:chris-gpt-full:8ot8ZLJR"

We added an output directory for our generated files, and I stored the name for my ChrisGPT model in the CHRIS_GPT variable. Make sure you replace the CHRIS_GPT string with the name of your own model and do not copy mine, as you will not be able to access my personal model. Make sure you save and close this file.

Now create the empty folder for the output:

📁Finx_Fine_Tuning
    📁data
        ...
    📁output ✨
        (empty)
    📁utils
        ...
    📄.env
    📄chris_gpt_dataset_generator.py
    📄chris_gpt_dataset_validation.py
    📄chris_gpt_training.py
    📄constants.py
    📄Pipfile
    📄Pipfile.lock

And then create a new file in your root directory called chris_gpt.py:

📁Finx_Fine_Tuning
    📁data
        ...
    📁output
        (empty)
    📁utils
        ...
    📄.env
    📄chris_gpt_dataset_generator.py
    📄chris_gpt_dataset_validation.py
    📄chris_gpt_training.py
    📄chris_gpt.py ✨
    📄constants.py
    📄Pipfile
    📄Pipfile.lock

Now let’s start with our imports for chris_gpt.py:

from constants import CLIENT, OUTPUT_DIRECTORY, CHRIS_GPT
import time

We import the CLIENT, OUTPUT_DIRECTORY, and