Welcome back to part 6! In this part, we’re going to look at the LangChain Expression Language (LCEL) for chaining together elements to create an LLM chain that we can run. You’ve seen this syntax briefly in part 1 but we haven’t gone much into it after that. In this part, you will learn how to build a complex LLM chain with several layers interwoven with each other. Despite this complexity, it will still be very readable and easy to understand thanks to the LangChain expression language.
Before we get started though, as we’re going to be building an RCI chain, let’s talk about what an RCI chain is and why it is useful. RCI stands for:
Recursive Criticism and Improvement
What this basically means is that we’re going to ask a question to the language model, ChatGPT in our case. Then we’re going to get a criticism on this answer, looking for any problems, errors, or areas where this answer can be improved. Then we call ChatGPT again and ask it to improve its answer based on the critique.
So who is going to be doing the critiquing? Well, it turns out that Large Language Models are surprisingly good at critiquing themselves! So we’re going to ask ChatGPT to critique its own answer and then improve its answer based on the critique. This is the basic idea behind RCI.
Why is this useful? As you can read in this paper (https://arxiv.org/pdf/2303.17491.pdf), which I think is the origination of the term RCI itself, it is very effective at solving computer tasks and more tricky reasoning problems. The future of AI is clearly going to involve more and more AI doing things for us, especially computer tasks. So these researchers seem to be on to something with this RCI idea, and I’ve actually already seen this used in real-life applications as well, but more on that later.
Let’s build stuff!
So let’s get started with a practical example, before we get too stuck in theory, and create a new folder called ‘6_RCI_and_LCEL‘ and inside we’ll create a new file named ‘1_RCI_chain.py‘ like this:
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁3_Agents_and_tools
📁4_Custom_tools
📁5_Understanding_agents
📁6_RCI_and_langchain_expression_language
📄1_RCI_chain.py
📄.env
The file structure for this part of the tutorial will be pretty simple, to give you a break after that last one 😉!
Inside our ‘1_RCI_chain.py‘ file we’ll start by importing what we’ll need, as usual:
from dataclasses import dataclass
from typing import Optional
import langchain
from decouple import config
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
ChatPromptTemplate,
HumanMessagePromptTemplate,
SystemMessagePromptTemplate,
)
from langchain.schema.output_parser import StrOutputParser
We’re going to use Python’s built-in dataclasses and Optional type to define a simple data structure, we import langchain so we can discuss the debug feature later, config as always, and ChatOpenAI. We also import some prompt template convenience classes which will make it easier to create our prompts from the templates, and a StrOutputParser which will basically just return the string output of the LLM back to us from the JSON object ChatGPT sends back.
Now let’s set up our ChatGPT API:
chatgpt_api = ChatOpenAI(
model="gpt-3.5-turbo", temperature=0, openai_api_key=config("OPENAI_API_KEY")
)
RCI_log dataclass
Before we dive into the chain, let’s use a simple dataclass to define our own simple data structure. We will use this type of data structure to represent and store all the stages of an RCI call. The question, initial answer, critique, and final answer. We’ll call this data class an RCI_log and define it below:
@dataclass
class RCI_log:
question: str
initial_answer: Optional[str] = None
constructive_criticism: Optional[str] = None
final_answer: Optional[str] = None
def dict(self):
return self.__dict__.copy()
We use Python’s @dataclass decorator to define a dataclass, which is basically just a class that is used to store data. We can define the fields of the dataclass in the class definition, and then we can create instances of this class and store data in them. We name our dataclass RCI_log and define the fields question, initial_answer, constructive_criticism, and final_answer.
The question is of type string and required, whereas the other three fields use the Optional type with [str] in the brackets. This means that these three fields should either be a string, or they should be ‘None‘, which makes them optional. We set the default value for these three fields to None, as this will allow us to create an RCI_log object passing in just the question, and then we can fill out the other fields later.
We can also define methods in the class, which we do here with the ‘dict‘ method. This method just returns a copy of the .__dict__ attribute. The .__dict__ attribute is a special attribute in Python that returns a dictionary containing the attributes and their values of an object. It is used to access the internal dictionary that holds the instance attributes of an object and basically exposes what’s actually stored in memory. We can use this to get a dictionary representation of our RCI_log object, which will be useful later.
One benefit of this dataclass over just using a normal dictionary is that your type-checker (if you have one turned on), will complain if we accidentally mistype a property or try to set one that doesn’t exist and we also get IntelliSense autocompletion because we have predefined the properties. This dataclass is a bit overkill for just this small tutorial but if you’re working with large projects passing loads of types of data around this can really help keep things organized.
Creating our RCI chain
Anyway, that was just a little detour, let’s get cracking on our RCI LLM chain! Let’s declare a function that will run our RCI chain, below and outside of the dataclass indentation:
def run_rci_chain(question: str) -> RCI_log:
log: RCI_log = RCI_log(question)
Our function named run_rci_chain takes a question as a string and returns an RCI_log, which is the type we just defined above using our dataclass. We create an instance of this dataclass and store it in a variable named log, passing in the question as the first argument.
Now define an inner method inside this method:
def run_rci_chain(question: str) -> RCI_log:
log: RCI_log = RCI_log(question)
def combine_system_plus_human_chat_prompt(
sys_template: str, human_template: str
) -> ChatPromptTemplate:
return ChatPromptTemplate.from_messages(
[
SystemMessagePromptTemplate.from_template(sys_template),
HumanMessagePromptTemplate.from_template(human_template),
]
)
We define a function called combine_system_plus_human_chat_prompt which takes a system template and human template as strings and will return a ChatPromptTemplate object. What is a ChatPromptTemplate object? Go ahead and hover over the object name in your IDE and you’ll see that it is basically just a list of tuples with a message history.
This function then returns a ChatPromptTemplate created with the .from_messages method, which takes a list of messages as an argument. We create this list of messages by using the .from_template method for both a system message and a human message passing in our system and human templates (we haven’t created these yet, but whatever was input into the function).
This will basically just return an object like this, with whatever system and human prompt templates we passed in:
[
("system", "You are a helpful AI bot.... blabla setup instructions"),
("human", "Whatever we want the LLM to do for us on this ChatGPT call."),
]
# Do not put in your file #
That’s all a ChatPromptTemplate object is, a combination of several prompt templates into a chat history type list of tuples, with a role like “system” or “human” assigned for each message. So now we will need to create several of these ChatPromptTemplate objects for every call we will make to ChatGPT.
Still inside the run_rci_chain function, but outside the inner function:
def run_rci_chain(question: str) -> RCI_log:
log: RCI_log = RCI_log(question)
def combine_system_plus_human_chat_prompt():
.....
initial_chat_prompt = combine_system_plus_human_chat_prompt(
"You are a helpful assistant that provides people with correct and accurate answers.",
"{question}",
)
We create our initial_chat_prompt, by using the function we just created to combine a system prompt of "You are a helpful assistant that provides people with correct and accurate answers." and then a human prompt of whatever the user’s question or input was by replacing the {question} placeholder. Again, this will just return a list of tuples with the first tuple holding the ("system": "instructions") system role and message and the second tuple holding the ("human": "question") human role and message.
After we ask the initial question we will need to get a critique on the answer we got, so let’s set up that prompt as well:
critique_chat_prompt = combine_system_plus_human_chat_prompt(
"You are a helpful assistant that looks at a question and it's given answer. You will find out what is wrong with the answer and give a critique.",
"Question:\n{question}\nAnswer Given:\n{initial_answer}\nReview the answer and find out what is wrong with it.",
)
We run our function again to create the second ChatPromptTemplate object but this time the system prompt with ChatGPT’s instructions is completely different. We ask it to find out what is wrong and critique the first answer. (If there’s nothing wrong with it, it will tell us). We then feed it the original question and the answer that was given.
Now, we just need another ChatPromptTemplate with a system and user message for the third and final call:
improvement_chat_prompt = combine_system_plus_human_chat_prompt(
"You are a helpful assistant that will look at a question, its answer and a critique on the answer. Based on this answer and the critique, you will write a new improved answer.",
"Question:\n{question}\nAnswer Given:\n{initial_answer}\nConstructive Criticism:\n{constructive_criticism}\nBased on this information, give only the correct answer.\nFinal Answer:",
)
This time we ask it for a new and improved answer, based on the question, the initial answer, and the constructive criticism which we all feed into the template using {placeholders}.
LangChain Expression Language
So what is this LangChain Expression Language that we’ll be using? It’s actually very simple, and you’ve already seen it in part 1 of the tutorial.
chain = prompt | model # Do not put in your file #
Expression language allows you to compose chains in LangChain by simply using the | pipe operator. So the above simply means that the prompt feeds into the model. It’s kind of like the pipe operator in Bash, where the output of the first item is ‘piped’ into the input of the second, creating a ‘chain’ for Large Language Models, hence the name ‘LangChain’.
So let us try this out, and create a chain to run our first initial_chat_prompt through ChatGPT, still continuing inside the run_rci_chain function:
def run_rci_chain(question: str) -> RCI_log:
.....
.....
.....
initial_chain = initial_chat_prompt | chatgpt_api | StrOutputParser()
So we declare a new LangChain chain named initial_chain, and we use the | pipe operator to pipe the initial_chat_prompt containing our initial system message and the human message containing the user query into the chatgpt_api and then pipe the output of that into the StrOutputParser. Again, the StrOutputParser will simply return the LLM’s final answer to us.
So if we call this first chain, we will get our first answer, but there is one last thing we need to call a chain like this. We’ve actually also seen this in part 1. The initial chat prompt has the {placeholder} values in there, which need to be replaced by our values. We pass these in using a simple dictionary. So we could invoke our initial chain like this:
# Example, do not keep in your code #
answer = initial_chain.invoke({"question": "What is an elephant?"})
And this would totally work. (Note that if you insert this into your file and run it you will return nothing as the run_rci_chain function is not called anywhere yet, we’ll do that later). However, remember that we create the RCI_log data type that just so happens to contain entries for all the variables our prompt templates will need! (Which is of course no coincidence.) We also gave it a convenient function to output all its data to a dictionary.
So the above can be replaced by:
answer = initial_chain.invoke(log.dict(