Welcome to this tutorial series on LangChain. My name is Dirk van Meerveld, and it is my pleasure to be your host and guide for this tutorial series!
We’re going to be using LangChain to further improve our ChatGPT superpowers and explore more cool ways in which we can put these powerful technologies to practical use. This tutorial series assumes a basic knowledge of ChatGPT and how to use it. It will also help if you understand ChatGPT function calls and embeddings as it will help you really understand what is going on below the hood. While we will cover these at a basic level as they come up, it may seem a bit magical and abstract if you don’t fully understand what is going on.
For basic ChatGPT you can check out my ‘Giggle Search’ tutorial series (also teaches Django) and for function calls and embeddings you can check out my ‘ChatGPT function calls and embeddings’ tutorial series, both available here on the Finxter Academy.
So what is LangChain?
LangChain is a library that makes it easier to work with large language models (LLMs). Anything we can do with LangChain we can technically also do without it, it’s just that it makes our lives a lot easier. LangChain helps us set up our prompts, conversation history, reusable chains of LLM calls, and even agents that use the LLM to reason and make decisions. One of the other advantages is that our code will not be dependent on a specific LLM, so in most cases, we can actually use exactly the same code with a different LLM without changing much about the code.
LangChain also provides a large amount of tools for us to use to load data such as document loaders, transform data, such as text splitters, and store data, for example in a vector database. It also contains loads of tools that we can give to our LLM agent for the LLM to use, kind of like function calls. An example would be to give our LLM a Wikipedia tool and then our LLM agent will be able to search Wikipedia for more information on any subject if it needs such information to adequately answer our questions.
Let’s get started with a basic example
We can talk a lot about what LangChain is in theory, but it’s much better to get a feel for it by actually using it. So let’s get into the practical examples and as you go through each tutorial section you will get a better feel for what LangChain is and how to use it.
Before we get into the challenging examples, let’s do a very basic LangChain LLM call just to get our feet wet and get familiar with LangChain. Create a new project folder and open it in your favorite code editor (I’ll be using VS Code throughout this series). Then create the folder for part 1 inside. I’m going to name it ‘1_Summarizing_long_texts‘ as we’re going to be focussing on summarizing in a little bit. Inside this folder create a new python file named ‘1_basic_call.py‘. My folder structure will look as follows:
📁Finx_LangChain (root project folder)
📁1_Summarizing_long_texts
📄1_basic_call.py
As we will be covering different subjects I will not combine our modules as a single project but keep them in separate folders, so you can easily refer back to any parts later if you need. Naturally, this is not a good practice for general software development, so keep that in mind.
Now first open a new terminal window and run the following command to install the needed libraries/dependencies:
pip install openai tiktoken langchain python-decouple
You’ll probably have openai already installed for working with the openai APIs. Tiktoken will help us calculate the amount of tokens later and the langchain library is the focus for this whole tutorial series! Python-decouple will allow us to load our API keys from a .env file.
A ChatGPT call in LangChain
Now open your 1_basic_call.py file and let’s get started by adding the following two imports up top:
from langchain.prompts import ChatPromptTemplate from langchain.chat_models import ChatOpenAI
First, we import the ChatPromptTemplate class from the LangChain library, which will allow us to create the prompt we want to feed to ChatGPT (or another LLM). Secondly, we import ChatOpenAI from the LangChain library, as we will be using ChatGPT for our calls in this course. To use this class we need to have the ‘openai‘ library installed (it was in the install command earlier).
Back to our basic_call.py file. Supplement the code as follows:
from langchain.prompts import ChatPromptTemplate
from langchain.chat_models import ChatOpenAI
prompt = ChatPromptTemplate.from_template(
"Please tell me which foods are famous in {place}"
)
model = ChatOpenAI()
chain = prompt | model
First, we define the prompt we will give to LangChain. For this, we will use the ChatPromptTemplate.from_template method, and feed it a simple string for now. Notice the {place} in between brackets, we will be able to insert variable input in here to dynamically change our prompt on each call.
We set the model as ChatOpenAI, since LangChain can work with other LLMs, so we need to specify which we want to use. Then we create our first simple ‘chain’ by using the | pipe operator to connect our prompt to our model. Think of it as piping the output of the first variable as input into the second, like Bash scripting, connecting them together in a chain. These chains are a major concept in LangChain and will keep coming up in the future.
Setting our API key
Before we start running any code we must provide our openai API key, as LangChain will call the openai library under the hood which will of course need our API key to make requests to ChatGPT on our behalf. Feel free to use an environment variable named OPENAI_API_KEY if you’re familiar with this, and the openai library will pick up on it automatically in the background.
As environment variables can be a bit tricky with differences in setup for different systems and OS combinations, and I don’t want anybody to get stuck in endless system-specific debugging, I’ll take a simple approach using the python-decouple package that will work universally and still keep our API key out of our source code. We already installed it in the install command earlier.
Now create a .env, which is simply a file by the name of '.env' with no name but only the extension of .env, in the base directory and add the following line to it:
OPENAI_API_KEY=yoursuperdupersecretapikeygoeshere
Make sure to insert your own API key from Openai.com, or sign up for an account if you do not have one yet. Also, do not use any spaces as it will not work for .env files.
💡Tip: If you're using GIT on this project, make sure to add '.env' to your .gitignore file so you don't commit your API key to your repository. If you're not yet familiar with GIT/version control, don't worry about this message.
Make sure the .env file is saved in the base directory with your folder structure as follows:
📁Finx_LangChain
📁1_Summarizing_long_texts
📄basic_call.py
📄.env
Making our first call
Now we can finish up our basic_call.py file as follows:
from langchain.prompts import ChatPromptTemplate
from langchain.chat_models import ChatOpenAI
from decouple import config
prompt = ChatPromptTemplate.from_template(
"Please tell me which foods are famous in {place}"
)
model = ChatOpenAI(openai_api_key=config("OPENAI_API_KEY"))
chain = prompt | model
All we added was an import, importing config from the decouple package, and when defining the model variable we passed in our API key as argument by calling the config function and passing in the name of our variable in the .env file as a string.
Our chain variable is now a ‘RunnableSequence‘ object returned to us by LangChain, we can run this sequence by calling its .invoke method, adding the following line below our code:
result = chain.invoke({"place": "France"})
print(result.content)
Notice that we pass in a dictionary that has the keys that correspond with the {place} we had in our template. Now if you go ahead and run the file in your terminal you will get something along the following lines:
France is known for its rich culinary traditions and has several famous foods. Some of the most renowned dishes and food items in France include: 1. Baguette: The iconic French bread, characterized by its long, thin shape and crispy crust. 2. Croissant: A buttery, flaky pastry often enjoyed for breakfast or as a snack. 3. Escargots de Bourgogne: Snails cooked in garlic butter and typically served as an appetizer. 4. Coq au Vin: A classic French dish made with chicken, red wine, mushrooms, onions, and bacon. 5. Ratatouille: A flavorful vegetable stew consisting of eggplant, zucchini, bell peppers, tomatoes, onions, and herbs. 6. Bouillabaisse: A traditional Provençal fish stew made with various types of fish, shellfish, vegetables, and aromatic herbs. 7. Foie Gras: A luxury food product made from the liver of a specially fattened duck or goose. 8. Quiche Lorraine: A savory pie filled with a mixture of eggs, cream, cheese, and bacon or ham. 9. Crêpes: Thin pancakes that can be filled with sweet or savory ingredients, such as Nutella, fruits, cheese, or ham. 10. Macarons: Delicate, colorful, and almond-based cookies with a sweet filling, typically made in various flavors. These are just a few examples, and France has an extensive culinary repertoire with numerous regional specialties.
Summarizing a large piece of text – the challenge
So now that we made a simple call, let’s look at a real use case for LangChain. We’re going to be using ChatGPT for summarizing a large piece of text. So the problem with ChatGPT here is that it has a context limit. This context is the maximum total amount of tokens (each token is basically a couple of text characters) we can send to ChatGPT in a single API call and differs between models. This limits us in the amount of data we can send to ChatGPT in a single request. We can’t just send a whole book and request a summary for instance.
For normal ChatGPT this limit is at 4096 tokens, but there are special ChatGPT 4 8k and 32k context versions out there, and also a 3.5 turbo 16k model. The problem is still the same though, if we need to summarize a very large piece of text, eventually we will run out of context ‘space’ to send all these tokens. For this tutorial series, we will be using ChatGPT turbo 3.5, so everyone will be able to follow along without needing access to ChatGPT 4.
We’ll also stick to the 4k context model for purposes of explanation, as it will save you some tokens as the learner versus going past the context limit on the 16k model. We could send 80.000 tokens to a model with a 16.000 token limit, or send 6.000 tokens to a model with a 4.000 token limit. The principle is exactly the same so we’ll be doing the latter in this tutorial to save you some tokens and make this tutorial extremely cheap to follow in terms of API costs.
So what do we do if we want a summary of a document that is larger than our context limit? We cannot just send the whole document and ask for a summary as we’ll get a token limit error:
⚠️openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 4175 tokens. Please reduce the length of the messages.
So what we need to do is split up the document into smaller chunks and send these chunks to ChatGPT one by one. Then we can take all the summaries and combine them into one big summary. Say we make 5 calls with 4000 tokens each, splitting the text into 5 parts and asking for a summary each time. We save all 5 small summaries in memory and then we make a final ChatGPT call number 6 where we send all these summaries back to ChatGPT again and ask for a final summary of all the summaries it has made for us!
Now while we can program all of this manually in a chain of ChatGPT calls that will handle the sub-summaries, save the results, and send the texts for the final summary, this is a lot of work. We’d also have to account for edge cases where all the summaries combined are still larger than the context limit and things get pretty complex. LangChain is going to make our lives a lot easier here. Defining these kinds of ‘chains’ of LLM calls is exactly what LangChain is good at!
LangChain names this strategy Map-reduce as it will first ‘map’ over all our smaller text chunks individually and then ‘reduce’ the resulting summaries into a single final summary result.
Let’s do it!
Create a new file in your '1_Summarizing_long_texts' folder and name it '2_summarize_txt.py'. We’ll get to the code in here later, but before we do, to keep things organized let’s create two sub-folders inside the ‘1_Summarizing_long_texts’ folder. One will be named 'prompt_templates' and one will be named 'utils'. This will just keep our main code more readable and easier to explain by not cramming everything into one file.
📁Finx_LangChain
📁1_Summarizing_long_texts
📁prompt_templates
📁utils
📄1_basic_call.py
📄2_summarize_txt.py
📄.env
Creating our prompt templates
Let’s first get our prompt templates out of the way. Inside the prompt templates folder make a file called prompts.py. Inside this new prompts.py file we’ll set up the two prompts we need.
map_prompt = """
The following is a part of a speech
{text_chunk}
Based on this particular part of the speech, please write a concise summary
Helpful Answer:
"""
This prompt will be used for each part of the cut-up text. I’ve changed the prompt to say 'speech' instead of text as I’ll be using a long speech for th