Welcome back to part two, where we’re going to ‘chat’ with an entire book! We’ll be able to ask a textual question and we’ll receive a textual response based on the information inside the book. Besides being really cool, what are the practical real-world use cases for this?
Imagine you’re a writer, and you are on book number 7 of a series. You want to make sure you don’t contradict yourself or create some kind of continuity problem with the contents of the last 6 books, so you just ‘chat’ with your previous books to quickly find out. Just ask your books and get a natural-language answer back!
Say you have a large amount of documentation on your product or software suite explaining exactly how to do this and that. You can store the entire documentation and instead of keyword searches, people will be able to semantically search for how to do something and find exactly whatever part of your documentation explains how to do that exact thing!
This is extremely helpful and I personally find this extremely cool! The last-named use case is already being used in professional production environments. The point is that people can ask a natural language question and get a natural language response back in return. It’s just like chatting with ChatGPT except it has access to the entire book or even multiple books or an entire documentation base and will answer your questions based on that knowledge.
So how can we do this? We will use embeddings! To avoid boring people who already took my ‘Function calls and embeddings’ course with repetition, I won’t go too deep into what embeddings are exactly. Basically an ’embedding’ is a way for the computer to represent a word or sentence as a vector, which is just a long list of numbers. These numbers represent the meaning of the text, but not the exact words used! The embedding for the string ‘happy’ and ‘feel-good’ would be very similar for example, even though the characters used in both strings are completely different, the meaning is in a similar direction. This is the whole idea behind embeddings.
This allows computers to compare vectors for different pieces of text and see how similar they are in meaning. This is how we can compare a user’s question to the contents of a book and find the most similar parts of the book to the user’s question. We can then return the most similar parts of the book as the answer to the user’s question. Again, if you want more information on embeddings and how they work, please refer back to my ‘Function calls and embeddings’ course.
OpenAI has not just the ChatGPT API, but also another API endpoint that allows us to send a piece of text and get one of these embeddings in return. We’ll be using this OpenAI embeddings API for our embeddings.
The process
Our basic steps to achieve a full book or documentation chat are as follows:
- Load our data and split it up into small parts
- For each small part, send it to the OpenAI embeddings API and get the embedding back
- Store the embedding in a vector database, specifically designed to store and compare/search embeddings
- When a user asks a question, convert the question to an embedding and search the vector database for the closest embeddings, indicating that their meaning is related to the user’s question.
- Take over the world for total domination! (optional)
You can do this with a single huge document, like a book, or with a large collection of smaller documents, like the documentation for a software library. It doesn’t matter and the underlying process is basically the same. We will create a database of vectors and compare the user query converted to a vector to the vectors in the database. The closest vectors will be related to the user query and be returned as the answer to the user.
In the previous tutorial series we used a .csv file to store our embeddings. While this is fine for smaller projects, if you’re going to be building a large production project you will need a vector database. That’s why for this tutorial we’ll be looking at PineCone for storing our embeddings. PineCone is a cloud-based vector database that is extremely easy to use and has a free tier that is more than enough for most projects.
That’s enough introduction, let’s get started on world domination… I mean, let’s get started on building our book chat!
Installing dependencies and getting a book
Before we get started, let’s install some dependencies and get a book we want to chat with. First, run the following command in a terminal window:
pip install pypdf pinecone-client
We’ll use the pypdf library to read our book (which is in pdf format) and the pinecone-client library to connect to our PineCone vector database (we’ll create a free account later).
Next is our book. I will be using “How to succeed or, Stepping-stones to fame and fortune” by Orison Swett Marden. It’s a book from 1896 and is in the public domain, so we can use it for free. You can download the book from the following link:
🔗 https://manybooks.net/titles/mardenor2051320513-8.html#google_vignette
Go ahead and create a new folder in your root project folder named ‘2_Chat_with_large_documents’ and put the book inside in a data folder, I will be renaming it to ‘How-to-succeed.pdf’. Your folder structure should look like this:
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁data
📄How-to-succeed.pdf
📄.env
Preparing our data / creating a vector database
The code for this tutorial part will be relatively short, but it is important that we know and keep track of what is happening and what LangChain is abstracting away for us, so that this doesn’t turn into some kind of magical code soup that we don’t understand. The less code there is, the harder it can often be to understand what is going on, so we’ll take a look under the hood as we go.
In your '2_Chat_with_large_documents' folder, create a file called '1_book_chat_setup.py'
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁data
📄How-to-succeed.pdf
📄1_book_chat_setup.py
📄.env
Inside this file we’ll get started with our imports:
import re import pinecone from decouple import config from langchain.document_loaders import PyPDFLoader from langchain.embeddings.openai import OpenAIEmbeddings from langchain.schema import Document from langchain.vectorstores import Pinecone
The re library is the built-in regular expression library, we’ll use it to clean up our texts. Pinecone is for the database we’ll be using, decouple should be familiar by now for loading our API key. PyPDFLoader allows us to load up PDFs in Langchain, and OpenAIEmbeddings simply makes a call to the OpenAI embeddings API, just like we used ‘ChatOpenAI’ in the previous tutorial series. You’re well familiar with the Document data structure by now and Pinecone from langchain.vectorstores is another LangChain helper that will make it easier to interact with our pinecone vector database in the cloud.
Speaking of Pinecone, let’s create a free account. Go to pinecone.io and press the ‘Sign Up Free’ blue button on the top right of the webpage. You can just use your Google account to sign up for a free plan. Now you’ll need two things. First, you’ll need your API key. You can find this by clicking on ‘API Keys’. Copy the key and paste it into your .env file below your OpenAI key as follows:
OPENAI_API_KEY=your_openai_api_key_here PINECONE_API_KEY=your_pinecone_api_key_here
Again, make sure not to use any spaces anywhere in the .env file.
Second, you will need to create an index to store the embeddings (vectors) in. Back to the Pinecone website, in the left-hand menu above ‘API Keys’ should be a menu item named indexes. Click this and then click the ‘Create Index’ button. For the name use "langchain-vector-store". Set the number of dimensions to 1536 as this is what OpenAI embeddings use (it refers to the number of dimensions in the vector that represents our text, which basically means each vector we’ll be storing is a list of 1536 floating point numbers).
For the metric, use cosine, as this is the standard mathematical way to compare vectors. Finally set the Pod Type to starter, as this is free to use in our free account. Then click the ‘Create Index’ button to create your pinecone database. You’ll see something like this, letting you know the vector database is up and running:
langchain-vector-store Free Tier
METRIC DIMENSIONS POD TYPE HOST
cosine 1536 starter https://.......pinecone.io
PROVIDER ENVIRONMENT MONTHLY COST
$0
Loading our book into the vector database
Back to our '1_book_chat_setup.py' file, let’s do some setup:
embeddings_api = OpenAIEmbeddings(openai_api_key=config("OPENAI_API_KEY"))
pinecone.init(api_key=config("PINECONE_API_KEY"), environment="gcp-starter")
pinecone_index = "langchain-vector-store"
First, we prepare the embeddings API by initializing a new OpenAIEmbeddings and passing in our openai_api_key from the .env file. The embeddings endpoint uses the same API key as the ChatGPT endpoint. Next, we initialize pinecone by calling pinecone.init and loading in our pinecone_api_key.
The environment for the second argument can be found by clicking the ‘Indexes’ menu item on the left-hand side of the Pinecone website. This will show your langchain-vector-store and among the information will be ‘Environment’. In my case, this is ‘gcp-starter‘, and it will probably be so for you as well.
Finally, we initialize a simple string variable and set its value to the name of our pinecone index, which is “langchain-vector-store“.
Time to load our book!
loader = PyPDFLoader("data/How-to-succeed.pdf")
data: list[Document] = loader.load_and_split()
The first line just creates a loader object linked to our file path but does not load it yet. We then call the .load_and_split method on the loader object which will return a list of Document objects with each Document being one page.
Remember that Document objects have a “page_content” and a “metadata” property. We won’t be using the metadata property in this one. So let’s simplify our data to a simple list of strings each containing the text of one page:
page_texts: list[str] = [page.page_content for page in data] print(page_texts[6][:1000])
We use a list comprehension and inside, for every page in the data variable, we get only the page_content property. Then we print the first 1000 characters of the 7th page just to have a look at what we have. Go ahead and run this Python file so far. Make sure you cd into your …/Finx_LangChain/2_Chat_with_large_documents folder in your terminal before running the file. It will take a minute and then you should see something like this:
One great need of the world to-day is for men and women who are good animals. To endure the strain of our concentrated civilization, the .....
Let’s get rid of all these unnecessary tabs that appeared in the loading process. Remove the previous print statement and then add the following below the page_texts variable:
page_texts_fixed: list[str] = [re.sub(r"\t|\n", " ", page) for page in page_texts] print(page_texts_fixed[6][:1000])
For each page in the page_texts, we will run a regex substitution, replacing any ("\t" tab) (| or) ("\n" newline) characters with a simple " " space character for this page. The list comprehension will run this on every page and return a new list. Let’s print a small snippet again and see if it’s fixed:
One great need of the world to-day is for men and women who are good animals. To endure the strain of our concentrated civilization, the coming man and woman must have....
That’s a lot better. Go ahead and get rid of the print statement. Now all we need to do is send each page to the OpenAI embeddings API and store the resulting embedding in our pinecone vector database. LangChain makes this very easy for us. (For more info on what these API calls do and what vectors are, refer to my ‘Function calls and embeddings’ course available here at the Finxter Academy). Add the following code to finish up our setup file:
vector_database = Pinecone.from_texts(
page_texts_fixed, embeddings_api, index_name=pinecone_index
)
We call the .from_texts method on the Pinecone helper class that comes with LangChain. We pass in our list of strings containing the page texts, the embeddings_api we want to use to get the embeddings, and the name of our pinecone index to store the embeddings in. LangChain will do the rest for us behind the scenes. It will send each page to the OpenAI embeddings API and store the resulting embedding in our pinecone vector database.