💡 Full Course with Videos and Course Certificate (PDF): https://academy.finxter.com/university/openai-api-function-calls-and-embeddings/
Welcome back to the final part of this tutorial series. In this part, we’ll be looking at simple sentiment analysis using embeddings. For most text classification tasks, fine-tuned machine learning models will do better than embeddings, because they have been meticulously tuned and trained on problem-specific data. There is training data, with the correct answers and classifications, and the model is trained to predict the correct answer by seeing lots of correct answers. But what if we don’t have any training data? We can use zero-shot classification to classify with zero labeled training data using ChatGPT embeddings.
In this last part, we’ll be working with a Jupyter notebook, as this will allow us to easily display the graphs in line with the code, and have a nice visual representation of our Pandas DataFrames. If you don’t like to use Jupyter notebooks you can just use a regular Python file and insert the same code, but you’ll occasionally need to insert a print statement in your file to see what we’re doing, and your print output will look a little bit less pretty is all.
I won’t go into depth on Jupyter notebooks here, but I will explain the bare basics you need to know, so if you’ve not used Jupyter notebooks before I would encourage you to follow along and take this opportunity to explore them.
For those new to Jupyter notebooks
Assuming you’re working with VS Code, you’ll need two things. If you’re already using Jupyter notebooks you can obviously skip these two steps.
1. pip install jupyter (just run the command in your console window) 2. Install the Jupyter extension in VS Code by selecting the extensions icon on the left side and searching for Jupyter, by Microsoft.
Once you’ve done that you should be good, depending on the configuration of your system.
A Jupyter notebook very very basically just allows us to chop our code up into blocks, which we can run one at a time. Unless we restart our notebook the kernel executing our code will be kept alive between running cells, also keeping our variables in memory. So in one cell, we could define ‘variable = “Hi this is some text”‘, and run that cell, and then in the next cell we could ‘print(variable)’ and it would print “Hi this is some text”. In fact, we can often skip the print statement altogether as you’ll soon see.
Ok let’s get started!
For this part, we’ll be using the same database we’ve used for part 4 of our tutorial where we had ChatGPT generate SQL queries to answer our questions about the database. You can download the file for free from https://www.kaggle.com/datasets/joychakraborty2000/amazon-customers-data and extract the zip file anywhere. The file has 2 versions of the data inside, one called database.sqlite which we used for part 4 of the tutorial series, and one called Reviews.csv. For this part, we’re going to be using the CSV version, and I’m going to rename it to ‘Gx_reviews_database.csv’ and put it in the base directory of my project.
> Gx_reviews_database.csv (renamed from Reviews.csv)
This CSV file has exactly the same customer reviews data as the SQLite version we used for part 4. Now let’s create a new file called ‘Ga_data_preparation.ipynb’ in the base directory of our project.
> Ga_data_preparation.ipynb
The .ipynb extension is the extension for Jupyter notebooks, and VS Code will automatically recognize and open it in the Jupyter notebook editor. If you’re using a regular Python file you can just call it ‘Ga_data_preparation.py’ instead. In the top left you can click +Code to add more code blocks to your notebook. Go ahead and just add like 5 or 6 before we get started.
In the first code cell, we’ll put our imports:
import openai
import pandas as pd
import decouple
config = decouple.AutoConfig(" ")
openai.api_key = config("CHATGPT_API_KEY")
EMBEDDING_MODEL = "text-embedding-ada-002"
INPUT_DB_NAME = "Gx_reviews_database.csv"
OUTPUT_DB_NAME = "Gx_review_embeddings.csv"
Note that the decouple and config part where we load the API key is slightly different than you’re used to. This is needed to make it work in Jupyter notebooks. Use the old method from the previous parts if you’re using a regular Python file. The other imports are all familiar by now and we define a couple of constants up top like the embedding model and the name of the input database and the output file name we’ll use to store the embeddings. (This output file does not have to exist yet, it will be auto-created).
*For those new to Jupyter notebooks (the very basics you need to know):
- On the left side of each cell you'll see an arrow, if you click it this particular cell will be executed.
- The variables will stay in memory and be available amongst different cells.
- If you want to start fresh you can restart your notebook by pressing the 'Restart' button at the top, which will restart the kernel and clear all variables. You then have to run each block again, or you can also press the 'Run All' button up top.
In the next cell, we’ll read up some data for us to work with:
df = pd.read_csv(INPUT_DB_NAME, usecols=["Summary", "Text", "Score"], nrows=500) df = df[df["Score"] != 3] df["Summ_and_Text"] = "Title: " + df["Summary"] + "; Content: " + df["Text"] df.head(5)
In the first line, we use Pandas to read data from a CSV file like the previous tutorial. We specify the database name as the first argument, then the columns we want to use, which means we will ignore all other columns in the data except for summary, text, and score, and the final argument is the number of rows we want to read. I’m going to read only 500 rows from this massive dataset. But if you’re very worried about tokens you can read even less and set it to 100.
The next line “df = df[df[“Score”] != 3]” may look a bit confusing at first glance if you’re not familiar with Pandas, so let’s read it from the inside out. df[“Score”] != 3 will return a boolean array of True and False values, with each row being tested for a True or False evaluation, where True means the score is not equal to 3. Then we use this boolean array to index our DataFrame, which means we only keep the rows where the score is not equal to 3. Any rows where the statement df[“Score”] != 3 evaluates to True will be retained in our dataset and any rows where this same statement evaluates to False will be filtered out. This is because we want to do binary classification, and we only want to classify positive and negative reviews, so we’ll remove all reviews with a score of 3, which is a neutral review.
In the third line, we add a new column to our DataFrame called “Summ_and_Text” which is just a concatenation of the summary and the text of each review, with a little bit of text added in between to separate the two. Finally, we print the first 5 rows of our DataFrame to see what it looks like. Note we can just declare df.head(5) whereas in a normal Python file, we have to use print(df.head(5)).
Go ahead and run this cell (make sure you run cell number 1 first with the imports). You should see a pretty representation where each row has 4 columns, prefixed by an id that Pandas generated, making for a data structure that looks like this:
Score Summary Text Summ_and_Text 0 5 Summary here.. Review here... Title: Summary here; Content: Review here 1 1 Summary here.. Review here... Title: Summary here; Content: Review here 2 4 Summary here.. Review here... Title: Summary here; Content: Review here 3 2 Summary here.. Review here... Title: Summary here; Content: Review here 4 5 Summary here.. Review here... Title: Summary here; Content: Review here
Generating the embeddings
Now that we have a DataFrame with only the data we want, we will need to generate embeddings again and save them somewhere, before we can start analyzing the sentiment and doing stuff with it. In a new cell of your Jupyter notebook, write the following function:
total_token_usage = 0
embeddings_generated = 0
total_data_rows = df.shape[0]
def get_embedding(item):
global total_token_usage, embeddings_generated
response = openai.Embedding.create(
model=EMBEDDING_MODEL,
input=item,
)
tokens_used = response["usage"]["total_tokens"]
total_token_usage += tokens_used
embeddings_generated += 1
if (embeddings_generated % 10) == 0:
print(
f"Generated {embeddings_generated} embeddings so far with a total of {total_token_usage} tokens used. ({int((embeddings_generated / total_data_rows) * 100)}%)"
)
return response['data'][0]['embedding']
This is mostly the same, we define the global variables for the number of tokens used, the number of embeddings generated, and the total number of data rows in our dataset. Then we define a function called get_embedding which takes an item as input and returns the embedding for that item. Inside the function we use the global keyword to access the global variables and increment them as appropriate and just like in the previous tutorial, we also print a progress message for every 10 embeddings generated.
Go ahead and run this cell so the function will be stored in memory and available for us to use. Now we can use this function to generate embeddings for our dataset. In a new cell, write the following code:
df["Embedding"] = df.Summ_and_Text.apply(lambda item: get_embedding(item))
df.to_csv(OUTPUT_DB_NAME, index=False)
print(
f"""
Generated {embeddings_generated} embeddings with a total of {total_token_usage} tokens used. (Done!)
Successfully saved embeddings to {OUTPUT_DB_NAME}.
"""
)
df.head(10)
We add a new column to our DataFrame named ‘Embedding’ and set its value to the Summary and Text column after a function has been applied to each item inside using the apply method. This function takes each item and runs the get_embedding function, passing in each item one by one and returning the embedding, thus filling the ‘Embedding’ column in our DataFrame with the embeddings.
We then use Pandas to save the DataFrame to a CSV file again, skipping the index (the ID numbers auto-generated by Pandas). Finally, we print a message to the console and print the first 10 rows of our DataFrame to see what it looks like. Go ahead and run this cell and wait until it’s done running.
Generated 10 embeddings so far with a total of 680 tokens used. (2%)
Generated 20 embeddings so far with a total of 1531 tokens used. (4%)
Generated 30 embeddings so far with a total of 2313 tokens used. (6%)
Generated 40 embeddings so far with a total of 3559 tokens used. (8%)
Generated 50 embeddings so far with a total of 4806 tokens used. (10%)
Generated 60 embeddings so far with a total of 5567 tokens used. (12%)
...
Generated 463 embeddings with a total of 45051 tokens used. (Done!)
Successfully saved embeddings to Gx_review_embeddings.csv.
Score Summary Text Summ_and_Text Embedding
0 5 Summary here.. Review here... Summ_and_text... [numbers...]
1 1 Summary here.. Review here... Summ_and_text... [numbers...]
2 4 Summary here.. Review here... Summ_and_text... [numbers...]
3 2 Summary here.. Review here... Summ_and_text... [numbers...]
4 5 Summary here.. Review here... Summ_and_text... [numbers...]
You’ll see your progress as it’s running and finally, your success message and a representation of the DataFrame printed out, representing a structure like above. You’ll also have a file named Gx_review_embeddings.csv with the data stored in CSV format. We now have our data prepared and we’re ready to do some sentiment analysis!
Sentiment analysis
To keep things organized, I’m going to be doing this in a separate file. Go ahead and save and close this one and create a new Jupyter notebook called ‘Gb_classification.ipynb’ in the base directory of our project.
> Gb_classification.ipynb
Open it up and press the ‘+ Code’ button in the top left a couple of times to give us a few cells to work with. In the first cell, place the following imports and setup variables:
import pandas as pd
import numpy as np
import openai
import decouple
from sklearn.metrics import classification_report, PrecisionRecallDisplay
from openai.embeddings_utils import cosine_similarity, get_embedding
config = decouple.AutoConfig(" ")
openai.api_key = config("CHATGPT_API_KEY")
EMBEDDING_MODEL = "text-embedding-ada-002"
CSV_DB_NAME = "Gx_review_embeddings.csv"
THRESHOLD = 0
Pandas and Numpy are familiar, and naturally, we also import openai and the decouple module to use our config and then set the openai key. Note we have to use the alternative config = decouple.AutoConfig call again as this is required for Jupyter notebooks over the way we used in our regular Python files before.
We also import the classification_report and PrecisionRecallDisplay from sklearn.metrics, which we’ll use to evaluate our model. Sklearn will make it easy for us to see how many correct versus incorrect classifications our model is making, and what its precision is. We also import cosine_similarity to calculate the similarity between two embeddings, and get_embedding which is just a built-in shortcut method to get the embedding for a given text.
Below we declare our embedding model, database name, and a threshold as constant variables so we can use them throughout this file. The threshold refers to the threshold we’ll use to classify a review as positive or negative. We’ll be able to play around with this value later to find the sweet spot for the greatest accuracy.
In the next cell, we’ll read up our data:
df = pd.read_csv(CSV_DB_NAME)
df["Embedding"] = df.Embedding.apply(eval).apply(np.array)
df["Sentiment"] = df.Score.replace(
{1: "Negative", 2: "Negative", 4: "Positive", 5: "Positive"}
)
df = df[["Sentiment", "Summ_and_Text", "Embedding"]]
df.head(5)
First, we read the csv file and load the data to a Pandas DataFrame. Then we select the ‘Embedding’ column and evaluate the string values back to arrays and then Numpy arrays for greater efficiency just like we did in the last tutorial. Then we add a new column called ‘Sentiment’ which is just a copy of the ‘Score’ column, but with the values 1 and 2 replaced with ‘Negative’ and 4 and 5 replaced with ‘Positive’. This is because we want to do binary classification between either positive or negative reviews.
Finally, we set the df variable equal to the DataFrame but with only the ‘Sentiment’, ‘Summ_and_Text’, and ‘Embedding’ columns selected, effectively filtering out all other columns. Then we print the first 5 rows of our DataFrame to see what it looks like using the .head method. Go ahead and run this cell, but of course make sure you ran the first cell with the imports and constants first. Your data structure will look something like this:
Sentiment Summ_and_Text Embedding 0 Positive Title: Summary here; Content: Review here [numbers...] 1 Negative Title: Summary here; Content: Review here [numbers...] 2 Positive Title: Summary here; Content: Review here [numbers...] 3 Negative Title: Summary here; Content: Review here [numbers...] 4 Positive Title: Summary here; Content: Review here [numbers...]
Testing different classification labels
Now let’s move on to the next cell. It will contain a single function, which we’ll go over in parts. This function will test the accuracy of classification labels, outputting a Precision-Recall curve, which is just a graph showing the accuracy of our predictions. This will allow us to test labels such as ‘Positive’ and ‘Negative’, or more complex labels such as ‘Positive product review’ and ‘Negative product review’ to see which best match positive/negative review embeddings. The idea of this is that we test the embedding for a term like ‘Positive product review’ against the embeddings of the actual reviews in the database. If a particular review’s embedding has a high similarity to the embedding for the string ‘Positive product review’, we can assume there is a high similarity in meaning, as in this is likely a positive product review.
Our function will have the ability to take any labels we pass in, so we can test different sets of labels and see which gives us the highest accuracy. We also made the sentiment column in our dataset (see above), which contains the correct answers. Therefore we’ll be able to compare our predictions based on the embeddings with the correct answers in the sentiment column and see how good our accuracy is.
So let’s get started on this function in a new code cell: