Intro to AI Engineering with Python (3/6) – LangChain 101

Hi and welcome to part 3 of this course! In this part, we’ll be looking at LangChain, which is a framework designed to make it easier to build applications that use large language models (LLMs). Think of it as a set of tools that helps bridge the gap between LLMs and the applications you might want to build with them.

LangChain helps us:

  • Provide a unified interface: Any code you write can be used with different LLMs with little modification, and you can use the same code to write prompts or tools for different LLMs. This means you could write code and use the GPT-4o model by OpenAI that we have used so far, and then decide you actually want to work with Google Gemini instead, and you could do so with minimal changes.
  • Prebuilt tools for common tasks: Langchain includes tools for common tasks you might want to do with LLMs, such as building chatbots, summarizing documents, or analyzing code. Besides just building our own tools and functions, we can also import community pre-built tools.
  • Memory and Context: Langchain makes it easy to incorporate memory and context into our LLM applications. This means our application can remember past interactions and use that information to inform future responses, like having a real conversation.

We’ll be looking at the very basics in this introductory lesson, but if you like what you see we have a whole lot more for you on this topic, and we’ll point you there at the end of this lesson.

Installing LangChain

Ok, time to get started with LangChain! We’ll start with some installs. Run the following command in your terminal to install LangChain:

pip install langchain

This will install LangChain and a bunch of other stuff you need with it. I’m using LangChain version 0.3.1 at the time of writing this, so you can also run pip install langchain==0.3.1 to install this specific version if you want to make sure you have the same experience as me.

The next thing we need to install is the langchain-openai library. This is a library that acts as a bridge between LangChain and the OpenAI API. Run the following command in your terminal to install it:

pip install langchain-openai

I’m using version 0.2.1 at the time of writing, so you can once again also run pip install langchain-openai==0.2.1 to install this specific version if you want.

You also need the openai library, but we already installed that in part one of the series.

Getting started

Now create a new file named langchain_basics.py in the root folder of your project:

📁 Intro_to_AI_engineering
    📁 output
    📄 .env
    📄 chat_gpt_request.py
    📄 generate_image.py
    📄 generate_speech.py
    📄 langchain_basics.py     ✨ New file
    📄 test_audio.mp3
    📄 text_to_summarize.txt
    📄 transcribe_audio.py

Inside this new langchain_basics.py file, let’s get started with the following imports:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv

load_dotenv()

So let’s take a look at all these wacky imports here:

  • StrOutputParser is a class that will help us parse the output from the LLMs into a string format. Normally when you get the return from ChatGPT, we have to index into the response.choices[0].message.content to get the response. Just think of this as a convenience class that will help us with this. (It has more functionality than that but that’s the basic idea).
  • ChatPromptTemplate is a class that will help us create a template for our chat prompts. This will make it easier to create prompts for the LLMs (any LLM you want to work with).
  • ChatOpenAI is a class that will basically just allow us to create an instance of OpenAI and use it with LangChain.

Like last time we import and use load_env to read the environment variable from our .env file.

So as stated, the big value here of these output parsers and prompt templates is that they are a unified interface that we can use in the same manner without changes even if we change the LLM we are using halfway through our project or in the future, or want to use multiple at the same time.

Prompt templates

Now let’s continue our code by creating a prompt template. You’ll see how this works as we go along:

french_german_prompt = ChatPromptTemplate.from_template(
    "Please tell me the french and german words for {word} with an example sentence for each."
)

For this simple example prompt, I’ll create an example that asks for the French and German words for a given word, along with an example sentence for each. This is just a simple example to show the parts of LangChain and how they work together.

The {word} part is the template variable that we can replace with any word we want to ask about. We then create a ChatPromptTemplate using the from_template method and pass in our prompt string. The ChatPromptTemplate class will help us create prompts for the LLMs in a more convenient way and basically deals with formatting message history like this:

## Example of a ChatPromptTemplate
template = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful AI bot. Your name is {name}."),
    ("human", "Hello, how are you doing?"),
    ("ai", "I'm doing well, thanks!"),
    ("human", "{user_input}"),
])

We need only a single message here though, which is why we use the from_template method. In this case, LangChain will assume this to be a human message so this will result in something like this:

## Example of a ChatPromptTemplate
template = ChatPromptTemplate.from_messages([
    ("human", "Please tell me the french and german words for {word} with an example sentence for each.")
])

Creating a chain

Now that we have a prompt template to create our prompts, let’s continue:

llm = ChatOpenAI(model="gpt-4o-mini")
output_parser = StrOutputParser()

french_german_chain = french_german_prompt | llm | output_parser

First, we define our LLM instance using the ChatOpenAI class and pass in the model we want to use. I’ll be using gpt-4o-mini as it is more than enough for the simple test we’re doing here, but you can also use gpt-4o if you want to use the larger model. Think of this llm object we just defined kind of like the CLIENT object we used in the previous parts of this series.

You may notice we’re not passing in an API key here, even though we did so in the previous parts. This is because load_dotenv() has read the .env file and set the OPENAI_API_KEY environment variable for us, and LangChain is set up to read this value from the environment automatically, so we don’t need to manually pass it in.

We then create an instance of the StrOutputParser class by calling it using the () brackets, saving the new instance in a variable named output_parser. This will help us to parse the output from the LLMs into a string response as discussed earlier.

Now that we have three building blocks, it is time for one of LangChain’s important concepts, “chains”. We can simply use the | operator to chain these building blocks together. This operator is taken from the pipe operator in Unix, which is used to chain commands together.

In this case, we take the french_german_prompt as the entry point of our chain, and we pipe the resulting prompt into our llm, making an LLM call. We then pipe the output into our output_parser to get the string response. Notice how easy and readable the chain is, like a simple step 1, step 2, step 3.

In LangChain We use chains to build stuff with large language models, hence the name LangChain. This piping style of syntax above is referred to as LCEL or LangChain Expression Language.

Running the chain

You can use an if __name__ == "__main__": block here if you want, and ask for user input, inserting that, but for simplicity’s sake, I’ll just hardcode words in here for now as I want to focus on LangChain, and you already know how to get user input.

So let’s actually run this chain. To do this we can simply use the invoke method on our chain:

result = french_german_chain.invoke({"word": "blueberries"})
print(result)

We can technically also just pass in the string "blueberries" as we only have a single variable, but it’s better practice to use a dictionary like this as you may have multiple variables in your prompt. So go ahead and run this Python file and you should get something like the following:

In French, blueberries are called **"myrtilles."**      

Example sentence:
"J'adore ajouter des myrtilles à mon yaourt le matin."  
(I love adding blueberries to my yogurt in the morning.)

In German, blueberries are called **"Heidelbeeren."**

Example sentence:
"Ich mache einen Kuchen mit Heidelbeeren."
(I am making a cake with blueberries.)

The order or structure may be slightly different as we didn’t specify any specific desired output structure, but that’s not the point here, it works! You’ll notice LangChain is very easy to read and understand, and this exact same code can be used with other LLMs with little modification.

We can also very easily stream the response instead. Edit your code like this, commenting out the previous invoke call and calling stream instead:

# result = french_german_chain.invoke({"word": "blueberries"})
# print(result)

for chunk in french_german_chain.stream({"word": "blueberries"}):
    print(chunk, end="", flush=True)

So for every chunk in the stream that results from calling french_german_chain.stream with the word “blueberries”, we print the chunk to the console. The end="" and flush=True are just to make sure there are no line breaks in between print messages (insert an empty string "", basically nothing, at the end of each printed chunk) and that the output is printed immediately to the console.

Now if you run it again, you’ll see the tokens being streamed and written to your console in real time.

Another useful method provided for us is batch, so let’s give that a spin as well:

# for chunk in french_german_chain.stream({"word": "blueberries"}):
#     print(chunk, end="", flush=True)

print(
    french_german_chain.batch(
        [{"word": "squirrels"}, {"word": "invade"}, {"word": "earth"}]
    )
)

This time we pass in a list of dictionaries with one entry for each run in the batch. Running this will give the responses in a list, one for each entry in the batch:

['Sure! \n\nIn French, the word for squirrel is **"écureuil."**  \nExample sentence: *L\'écureuil grimpe rapidement aux arbres en quête de nourriture.*  \n(Translation: The squirrel quickly climbs the trees in search of food.)\n\nIn German, the word for squirrel is **"Eichhörnchen."**  \nExample sentence: *Das Eichhörnchen springt von Ast zu Ast im Park.*  \n(Translation: The squirrel jumps from branch to branch in the park.)', 'Sure! Here are the French and German words for "invade," along with example sentences:\n\n**French:** \n- **Word:** envahir\n- **Example Sentence:** Les troupes ennemies ont décidé d\'envahir le pays.  \n  *(Translation: The enemy troops decided to invade the country.)*\n\n**German:** \n- **Word:** einmarschieren\n- **Example Sentence:** Die Armee plant, in der Nacht einmarschieren zu lassen.  \n  *(Translation: The army plans to invade during the night.)*', 'Certainly! \n\nIn French, the word for "earth" is **"terre."**  \nExample sentence: "La terre est notre planète, et nous devons la protéger."  \n(Translation: "The earth is our planet, and we must protect it.")\n\nIn German, the word for "earth" is **"Erde."**  \nExample sentence: "Die Erde dreht sich um die Sonne."  \n(Translation: "The earth revolves around the sun.")']

So here you could index into the list’s index [0], [1], [2] to get the responses for each word in the batch. This is very useful if you want to run some kind of LLM transformation on a larger set of data.

Now go ahead and comment that one out as well and let’s check the properties of our chain:

# print(
#     french_german_chain.batch(
#         [{"word": "squirrels"}, {"word": "invade"}, {"word": "earth"}]
#     )
# )

print("input_schema: ", french_german_chain.input_schema.model_json_schema())
print("output_schema: ", french_german_chain.output_schema.model_json_schema())

And if we run that we get a JSON schema that shows the in and outputs of our chain:

input_schema {'title': 'PromptInput', 'type': 'object', 'properties': {'word': {'title': 'Word', 'type': 'string'}}}
output_schema {'title': 'StrOutputParserOutput', 'type': 'string'}

We can see that the input takes a single object variable that needs to have a key word with a string value, which is what we have been providing for every call we have made so far. If we add more variables to our prompt, we’ll see them in the schema as well. The output schema is a simple string because we used the StrOutputParser to parse the output into a string in the end.

Adding complexity

That is the basics of an extremely simple chain in LangChain. So let’s make it a bit more complex here. In this same file let’s declare a second chain and let’s say for the sake of a simple demonstration that this second chain is supposed to check if the output of the first chain is correct or not. (We’re just using simple examples here to save time and get to the good stuff faster).

So down below the other stuff in the langchain_basics.py file, let’s define the prompt template for our second chain:

# print("input_schema: ", french_german_chain.input_schema.model_json_schema())
# print("output_schema: ", french_german_chain.output_schema.model_json_schema())


check_if_correct_prompt = ChatPromptTemplate.from_template(
    """
    You are a helpful assistant that looks at a question and its given answer. You will find out what is wrong with the answer and improve it. You will return the improved version of the answer.
    Question:\n{question}\nAnswer Given:\n{initial_answer}\nReview the answer and give me an improved version instead.
    Improved answer:
    """
)

This time we have two variables in our prompt, question and initial_answer. We ask it to give an improved version of the first answer. The first answer is likely to be perfect already but again this is just for the sake of a quick demonstration.

We can reuse the llm and output_parser instances we created earlier, so let’s just create a new chain with the new prompt:

check_answer_chain = check_if_correct_prompt | llm | output_parser

Now we will need to run the input through the first chain, and then we need to keep both the original prompt from the first chain and the answer we get back from the first chain to pass them into the second one. So let’s do that:

def run_chain(word: str) -> str:
    initial_answer = french_german_chain.invoke({"word": word})
    print("initial answer:", initial_answer, end="\n\n")
    answer = check_answer_chain.invoke(
        {
            "question": f"Please tell me the french and german words for {word} with an example sentence for each.",
            "initial_answer": initial_answer,
        }
    )
    print("improved answer:", answer)
    return answer

So we define a function run_chain that takes a word as string input and will return a string. The initial answer is our return after we invoke the french_german_chain with the word.

We then print this answer using end="\n\n" to add a double line break. Now we invoke the second check_answer_chain, passing in a dictionary as we’ve always done so far. The question key holds the original question we asked in the first chain, with the word we used passed into the string, and the initial_answer key holds the answer we got back from the first chain.

We print the ‘improved answer’ and return it.

Now let’s run this function with a word:

run_chain("strawberries")

I apologize if I suddenly gave you a craving for strawberries! 🍓🍓🍓 Run it and your output will be something like this:

initial answer: Certainly! 

In French, the word for strawberries is **"fraises."**
Example sentence: *J'adore manger des fraises avec un peu de crème.* (I love eating strawberries with a little cream.)

In German, the word for strawberries is **"Erdbeeren."**
Example sentence: *Im Sommer esse ich gerne Erdbeeren mit Joghurt.* (In the summer, I like to eat strawberries with yogurt.)


improved answer: Certainly!

In French, the word for strawberries is **"fraises."**
Example sentence: *J'adore manger des fraises avec un peu de crème.* (I love eating strawberries with a little cream.)

In German, the word for strawberries is **"Erdbeeren."**
Example sentence: *Im Sommer esse ich gerne Erdbeeren mit Joghurt.* (In the summer, I enjoy eating strawberries with yogurt.)

Additionally, for context, in both languages, strawberries are often associated with summer and are popular in desserts and snacks.

Now both answers are pretty much the same, as the question was super simple and the second chain realized there was nothing wrong, so it just added some extra context to the answer in an effort to improve it, but that is not the point here. The point is that you have successfully fed one chain into another chain to create your first small LLM system. A chain of chains!

So that works fine, but you can see passing the values around to the second chain is a bit cumbersome. Now imagine we want to add a 3rd step to the chains above or even a 4th one. A conditional split path perhaps? If x then call chain a and else call chain b.

All of this is possible, including far more complex structures, writing our own tools to call functions, detailed logging, and much more. You will learn how to do all of these in the Building Complex Multi-Agent Teams and Systems with LangGraph course, where you will take this knowledge to the next level several times over, and learn how to automate almost anything LLM and AI related!

At the end of this course, you will know how to create the above team of AI agents that work together to solve your problems for you, and rearrange it in any other structure you desire to accomplish complex tasks effortlessly, all using LangChain and its extension LangGraph!

That’s it for part 3 of this broad intro to AI tutorial series. I’ll see you in part 4 where we’ll take a look at building an interface for our AI app using Gradio. See you there!

Leave a Comment