Hi and welcome back! In this part, we’re going to be building our own custom agent from scratch. So far the whole agent may have seemed a bit magical as it just runs off reasoning back and forth on its own. In this part we’re going to really understand how an agent works and how it’s built by making our own. This part will be a bit longer than usual so buckle up and get comfortable!
In this part we’re going to be reinventing the wheel a little bit but, without going too deep down the rabbit hole, this will greatly increase your understanding of how an LLM tool that can basically only generate text can become a powerful AI agent that makes decisions and takes actions.
Make a new folder named '5_Understanding_agents' to begin with, and inside make another folder named 'tools'. We’re going to be using 2 different tools in this part and one of them is going to be our internet tool from part 4. So copy the 'internet_tool.py' file from '4_Custom_tools/tools' into '5_Understanding_agents/tools'. Your folder now looks like this:
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁3_Agents_and_tools
📁4_Custom_tools
📁5_Understanding_agents
📁tools
📄internet_tool.py (copy of the internet tool from part 4)
📄.env
Just in case you need the contents of the internet_tool.py file here it is again:
######################################################################################################################
##### This file is just a copy of the one from part 4, do not make changes here, make them in the original file ######
######################################################################################################################
import requests
from bs4 import BeautifulSoup
from langchain.tools import BaseTool
class InternetTool(BaseTool):
name: str = "internet_tool"
description: str = (
"useful when you want to read the text on any url on the internet."
)
def _get_text_content(self, url: str) -> str:
"""Get the text content of a webpage with HTML tags removed"""
response = requests.get(url)
html_content = response.text
soup = BeautifulSoup(html_content, "html.parser")
for tag in ["nav", "footer", "aside", "script", "style", "img"]:
for match in soup.find_all(tag):
match.decompose()
text_content = soup.get_text()
text_content = " ".join(text_content.split())
return text_content
def _limit_chars(self, text: str) -> str:
"""limit number of output characters"""
return text[:10_000]
def _run(self, url: str) -> str:
try:
text_content = self._get_text_content(url)
return self._limit_chars(text_content)
except Exception as e:
return f"The following error occurred while trying to fetch the {url}: {e}"
def _arun(self, url: str):
raise NotImplementedError("This tool does not support asynchronous execution")
if __name__ == "__main__":
tool = InternetTool()
print(
tool.run("https://en.wikipedia.org/wiki/List_of_Italian_desserts_and_pastries")
)
Obviously, in a real coding project never ever ever copy code like this! Having code in multiple places is bad. But for the sake of this tutorial, we’re going to keep everything segregated per folder so that you can easily reference anything you want to later on.
Moby Duck, Moby Dick??
Now inside the '5_Understanding_agents/tools' folder make another file named 'moby_duck_search.py'. (Not Moby Dick from the novel about the whale, I’ll explain the name in a second):
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁3_Agents_and_tools
📁4_Custom_tools
📁5_Understanding_agents
📁tools
📄internet_tool.py (copy of the internet tool from part 4)
📄moby_duck_search.py
📄.env
Inside this file, we’ll write the second tool our custom agent will use. Open up 'moby_duck_search.py' and start with our imports:
from json import dumps from langchain.tools import BaseTool from langchain.utilities import DuckDuckGoSearchAPIWrapper
We will use the Python built-in JSON library’s dumps or dump-to-string method to convert objects to string format. We import the BaseTool because we need to inherit from it as always. We also import DuckDuckGo because we’re going to be using DuckDuckGo to search for articles but limiting the search to the MobyGames gaming website. Hence this tool being named Moby-Duck-Search.
I chose this example website randomly and you can also use a different similar website about a specific topic if you so desire. This one will be pretty easy, so let’s get to implementing our tool:
class MobyDuckSearch(BaseTool):
name: str = "moby_duck_search" # Pun intended.
description: str = (
"A tool that uses DuckDuckGo Search to search the MobyGames game website. "
"Useful for when you need to answer questions about games. "
"Input should be a search query. "
)
api_wrapper = DuckDuckGoSearchAPIWrapper()
We start our class declaration by inheriting from the BaseTool and setting a default value for the name and description. We also create an instance of the DuckDuckGoSearchAPIWrapper which comes with LangChain which we store in a variable named ‘api_wrapper‘. We can simply reuse this but alter the input slightly. While still inside the MobyDuckSearch class block add:
def _run(self, query: str) -> str:
"""Just call the DuckDuckGoSearchAPIWrapper.run method, but with the edited query."""
targeted_query = f"site:mobygames.com {query}"
results_with_metadata: list = self.api_wrapper.results(
targeted_query, num_results=3
)
return dumps(results_with_metadata)
def _arun(self, query: str):
raise NotImplementedError("This tool does not support asynchronous execution")
We implement the ._run() method that takes a query as a string and returns a string from the function. We first create a targeted_query which is just "Mario-Kart" in becomes "site:mobygames.com Mario-Kart" out. You’re probably familiar with this syntax from Google as it will allow you to search for results only on that specific website.
We then call the .results() method on our api_wrapper instance (which is just the DuckDuckGoSearchAPIWrapper we imported from LangChain and has the .results method already built-in for us) and pass in the targeted_query and the number of results we want. Note that we save this in a variable of type list named ‘results_with_metadata‘ as the result will also include metadata which contains the link or URL to the search results page. It’s good to be explicit with variable naming instead of just saying ‘results‘ so it’s very clear and readable exactly what kind of data this variable holds.
We then return the results_with_metadata variable but we first convert it to a string using the json.dumps() method. This method converts it to a stringified JSON object because LLMs only work with strings so we need a string return to our method, else we won’t be able to feed it back into ChatGPT or another LLM. Finally, we declare the unused ._arun() method just to be complete.
Our MobyDuckSearch class now looks like this:
class MobyDuckSearch(BaseTool):
name: str = "moby_duck_search" # Pun intended.
description: str = (
"A tool that uses DuckDuckGo Search to search the MobyGames game website. "
"Useful for when you need to answer questions about games. "
"Input should be a search query. "
)
api_wrapper = DuckDuckGoSearchAPIWrapper()
def _run(self, query: str) -> str:
"""Just call the DuckDuckGoSearchAPIWrapper.run method, but with the edited query."""
targeted_query = f"site:mobygames.com {query}"
results_with_metadata: list = self.api_wrapper.results(
targeted_query, num_results=3
)
return dumps(results_with_metadata)
def _arun(self, query: str):
raise NotImplementedError("This tool does not support asynchronous execution")
A quick test run
Let’s add a quick test below and outside our class to see if it works:
if __name__ == "__main__":
moby_duck_tool = MobyDuckSearch()
print(moby_duck_tool.run("lego star wars"))
Remember this will only run if we run the file directly, and not if it’s imported inside another file. We just create a new instance of our moby duck tool and then run it with a query of 'lego star wars'. Go ahead and run the file to see if your tool is working, the structure of your output should look like this:
[
{"snippet": "Snippet here", "title": "Title here", "link": "https://link.com/"},
{"snippet": "Snippet here", "title": "Title here", "link": "https://link.com/"},
{"snippet": "Snippet here", "title": "Title here", "link": "https://link.com/"}
]
Note that even though it kind of looks like a list of dictionaries it’s just a string type variable we can feed to ChatGPT.
Go ahead and save this file and now create an __init__.py file in the '5_Understanding_agents/tools' folder.
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁3_Agents_and_tools
📁4_Custom_tools
📁5_Understanding_agents
📁tools
📄__init__.py
📄internet_tool.py (copy of the internet tool from part 4)
📄moby_duck_search.py
📄.env
Import both the moby duck and internet tool inside this file:
from .moby_duck_search import MobyDuckSearch from .internet_tool import InternetTool
Now go ahead and save and close your __init__.py file.
Setting up our Agent’s prompt
We’re done with our tools for now. The first thing we’re going to build for our agent is the prompt. Our instructions to the agent on what we want it to do and how. To keep our file structure and our final agent code cleaner and readable we’re going to create one more folder. Create a folder named ‘prompts‘ inside the '5_Understanding_agents' folder and inside create an empty __init__.py file and a file named 'base_agent_template.py'. Your folder structure should now look like this:
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁3_Agents_and_tools
📁4_Custom_tools
📁5_Understanding_agents
📁prompts
📄__init__.py (empty file)
📄base_agent_template.py (empty file)
📁tools
📄__init__.py
📄internet_tool.py
📄moby_duck_search.py
📄.env
Open your 'base_agent_template.py' file and declare the following variable containing a prompt inside:
base_agent_template = """
Answer the following questions as best you can, but speaking as fanatic gaming enthusiast. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Remember to speak as a fervent gaming enthusiast when giving your final answer.
Question: {input}
{agent_scratchpad}
"""
This is just a prompt template as we’ve seen several of in the past, notice the {tools} and {tool_names} variables. We’ll be replacing these with the actual tools we want to use and their names. We also have a {agent_scratchpad} variable which we’ll use to store the agent’s thoughts and actions as it goes through the process of answering the question. We’ll see how this works in a bit.
Note how most of this template is a standard structure that defines how we want the LLM to structure its thought process and answer our question. Now that we can see a basic template form for this ReAct reasoning style agent it makes a lot of sense and is actually quite simple. We’re merely asking the LLM to output text as that is all it knows how to do and all it can do in the end. We’re just asking it to output it in a specific format and order so that we can work with it, that’s all! This should make the whole reasoning agent from the previous parts look a lot less magical and a lot more understandable.
Go ahead and save and close this file, then open your __init__.py file, making sure to open the one in this folder which is still empty, and import the base_agent_template:
from .base_agent_template import base_agent_template
Ok go ahead and save and close that as well.
Let’s build our Agent
Let’s move on to our agent. Create a file called '1_building_an_agent.py' inside your '5_Understanding_agents' folder:
📁Finx_LangChain
📁1_Summarizing_long_texts
📁2_Chat_with_large_documents
📁3_Agents_and_tools
📁4_Custom_tools
📁5_Understanding_agents
📁prompts
📄__init__.py
📄base_agent_template.py
📁tools
📄__init__.py
📄internet_tool.py
📄moby_duck_search.py
📄1_building_an_agent.py <------New file
📄.env
Inside the '1_building_an_agent.py' file we’ll start with our imports. There are going to be quite a lot of them as in this part we’ll really dig into building an agent from its parts, so let’s get started:
import re
from decouple import config
from langchain import LLMChain
from langchain.agents import (
AgentExecutor,
AgentOutputParser,
LLMSingleActionAgent,
Tool,
)
from langchain.chat_models import ChatOpenAI
from langchain.prompts import StringPromptTemplate
from langchain.schema import AgentAction, AgentFinish
from prompts import base_agent_template
from tools import MobyDuckSearch
I’ll go over these imports in a broad sense, and we’ll explain them in more detail when we use each import. That makes more sense as we’ll actually get to see what the import does instead of just talking about it theoretically. We import the “re” module for regular expressions, as we want to test if the model output matches certain patterns later on. Decouple is of course for our API key, and all the other parts are bits and pieces we will combine together to build our agent.
Finally, we import our own MobyDuckSearch tool and prompt template. (We will get the internet tool involved later on in the tutorial, I haven’t forgotten about it!). Again, we’ll see how each of these imports works when we use them.
Setup
Let’s set up our ChatGPT API and our tools:
chat_gpt_api = ChatOpenAI(
temperature=0, model="gpt-3.5-turbo-0613", openai_api_key=config("OPENAI_API_KEY")
)
moby_duck_tool = MobyDuckSearch()
tools = [
Tool(
name=moby_duck_tool.name,
func=moby_duck_tool.run,
description=moby_duck_tool.description,
)
]
We set up our ChatGPT API as always. We then create a new instance of the MobyDuckSearch class and then create a list of tools. Inside we only create a single Tool object for now, using the name and description we have defined inside the class and passing the .run method as the func argument.
The prompt template (formatter)
We’re going to be writing custom versions of most things so you can really see and understand what is going on. We’ll go to the prompt-template formatter next. We have the prompt we defined in the prompts folder, but it has the variables that need to be plugged into it before it can be used. Let’s start on our new class:
class MobyDuckPromptTemplate(StringPromptTemplate):
template: str
tools: list[Tool]
We define a new class that inherits from StringPromptTemplate. This is a class that is used to format the prompt which is sent to the LLM. The format method must return a string. New instances of this class will take a template and a list of Tools as input. The template is a template string like we have in our prompts folder with {variables} in brackets in the string. We now need to add a .format method:
class MobyDuckPromptTemplate(StringPromptTemplate):
template: str
tools: list[Tool]
def format(self, **kwargs) -> str:
# Method implementation will go here.
The .format method is defined as an abstract method in the BasePromptTemplate, which is a parent of the StringPromptTemplate we inherited from. An abstract method in the parent class means the child classes should implement it. This method should return the formatted prompt as a string and will contain our prompt formatting logic.
Our .format method takes self, which is this particular instance of the class itself, and **kwargs, which is a dictionary of whatever other arguments were passed in when the .format method was called, basically just catching all the other arguments that were passed in.
So, before we start building our format method, what exactly is going to be inside this kwargs dictionary? At this point in the format function, our kwargs dictionary will look like this:
# Do not put this in your code #
{'input': 'The user input query', 'intermediate_steps': [list of steps taken so far]}
Where did these two key-value pairs come from? Who or what passed them into our format method? The AgentExecutor class which we imported from LangChain and will take a look at later on takes care of passing in the input and intermediate_steps if any have taken place, so on the first call this will be an empty list, but after the agent calls a tool or does something else, the AgentExecutor will add the action and observation to the intermediate_steps list and pass it back into the format method.
As we now know our format method will receive an intermediate_steps variable, let’s pop it off into a variable called ‘intermediate_steps‘.
def format(self, **kwargs) -> str:
intermediate_steps = kwargs.pop("intermediate_steps")
So in order to format our prompt, which is the whole purpose of this .format method, we need to pass in the variables we left open in our prompt template, namely {tools}, {tool_names}, {input}, and {agent_scratchpad}. As we saw a moment before, we already have the input key in our kwargs dictionary, so that one’s taken care of.
Next, we need to provide the {agent_scratchpad} This scratchpad is the LangChain name for basically the notes of the agent. As stated above, with each step the AgentExecutor will return the intermediate_steps to our format method, so we can prep the prompt for the next step by adding the agent’s actions and observations so far to the prompt of the next call. So before each ChatGPT call a new fresh prompt will be generated using our format method, inputting the actions and observations so far into the prompt before asking ChatGPT for its next step.
This acts as a sort of memory. Remember that a ChatGPT call is just a text completion based on whatever text you put in. So we feed back whatever the agent has thought and done so far, otherwise, it has no idea or memory of what has happened. We’ll start with an empty string for our scratchpad.
def format(self, **kwargs) -> str:
intermediate_steps = kwargs.pop("intermediate_steps")
scratchpad = ""
Now what exactly is in our intermediate_steps variable? The intermediate_steps variable is a list of tuples (value1, value2) with two values each. The first value is an AgentAction object, which is a dataclass with three attributes: