Welcome back to the final part of this tutorial series. This part was not actually supposed to exist, but after completing the curriculum for the Gemini tutorial series Google suddenly dropped some new and awesome features, so here we are.
Please bear with me as this might be a bit more spontaneous and unscripted compared to normal and I’ll use very basic and quick examples to explain the new features, but they are just too cool not to cover at all, so this is like an extra bonus part if you will.
Code Execution
So the first new feature I want to talk about is code execution. This might seem similar to function calling, which we’ve been using in the course, but it is actually very different. When doing function calling we write functions on our end and then Gemini merely gives us the arguments and the function it wants us to call. The function exists on our computer and gets called on our computer.
The new code execution feature allows the code to run on Gemini’s servers. They basically give you a small containerized environment on the Gemini servers that will execute the code on the API side. What’s more is you don’t have to write the function here, in this particular case it is Gemini that writes the code, and executes it on their own servers, based on your instructions!
So let’s look at a quick example to see this in action. First, make sure you are in your project root folder, and that your pipenv is activated. Then run the following command to update the google-generativeai package to the latest version:
pipenv update google-generativeai
This is needed so we can run the new syntax (including the code_execution option). Now create a new file in your project folder and name it code_execution.py. In this file write the following code:
from load_env import configure_genai from utils import safety_settings genai = configure_genai() model = genai.GenerativeModel( model_name="gemini-1.5-pro", safety_settings=safety_settings.low, tools='code_execution', ) response = model.generate_content(( "Please create a Python script that will print an integer in reverse order. " "The script should take an integer as input and return the reversed integer. " "For example, if the input is 123, the output should be 321. " "Make sure you execute the script to test if it really works. " )) print(response.text)
We use the easy configure_genai setup again to save us some time loading in the API key and whatnot, and then get the genai object.
When setting up the model make sure you use the pro version here, as code generation is definitely one of the more advanced features that will test the limits, so having the strongest model here is a must.
We use the safety settings from our utility, and now we add the tools argument like we’ve seen before. Instead of passing in a list with our own functions, we can simply pass in the string code_execution this time to enable the code execution functionality. Note that this will not work if you skipped over the pipenv update step.
Then we call the generate_content method on the model, which is basically just the single request version of the chat method we’ve been using, but everything you see here can also be applied in the chat method.
I’ll ask it to create a Python script to reverse integers, and to make sure to test if the script actually works. Now if we run this we’ll see something really cool. This is all one single output, from one single API call, but I’ll split it up here for readability.
The first code it came up with:
def reverse_integer(n):
"""
This function takes an integer and returns the reversed integer.
Args:
n: An integer.
Returns:
The reversed integer.
"""
reversed_n = 0
while n > 0:
digit = n % 10
reversed_n = reversed_n * 10 + digit
n //= 10
return reversed_n
# Get input from the user
n = int(input("Enter an integer: "))
# Reverse the integer
reversed_n = reverse_integer(n)
# Print the reversed integer
print(f"The reversed integer is: {reversed_n}")
Now we can see that it ran this code:
print(reverse_integer(123))
Traceback (most recent call last): File "<string>", line 2, in <module> NameError: name 'reverse_integer' is not defined
And now it realized its mistake:
It seems I forgot to include the function definition in the code sent to the interpreter. Let's fix that!
It comes up with new code:
def reverse_integer(n):
"""
This function takes an integer and returns the reversed integer.
Args:
n: An integer.
Returns:
The reversed integer.
"""
reversed_n = 0
while n > 0:
digit = n % 10
reversed_n = reversed_n * 10 + digit
n //= 10
return reversed_n
# Get input from the user
n = int(input("Enter an integer: "))
# Reverse the integer
reversed_n = reverse_integer(n)
# Print the reversed integer
print(f"The reversed integer is: {reversed_n}")
And now it tries to run again:
print(reverse_integer(123))
Traceback (most recent call last): File "<string>", line 2, in <module> NameError: name 'reverse_integer' is not defined
It seems here, that the Gemini API is writing the function in one file and tries to run it in a different file, which is why it can’t find the function. Keep in mind that the code execution feature has literally been released yesterday so I’m sure this will be fixed soon.
Nevertheless, the model does finally realize what is going wrong here:
I apologize for the continued error. It appears I am still not correctly executing the code within the same context. I need to find a way to persist the function definition between code blocks. Let me try a different approach. I will define the function within the execution block itself:
def reverse_integer(n): """ This function takes an integer and returns the reversed integer. Args: n: An integer. Returns: The reversed integer. """ reversed_n = 0 while n > 0: digit = n % 10 reversed_n = reversed_n * 10 + digit n //= 10 return reversed_n print(reverse_integer(123))
321
It seems this approach worked! The code successfully reversed the integer 123 and printed the output 321. Let me know if you have any other tasks or code you'd like me to execute! I am still learning to interact with the code execution environment effectively.
So finally it realized it should just test a simplified version and have the print statement in the same file. While this doesn’t look very impressive yet, I’m sure this will improve rapidly. The one thing that is very cool to see is that it started to reason iteratively and learn from its mistakes to fix them over time.
Gemini actually generated code 3 times and sent it to its on-server code execution environment 3 times to test if it worked. That’s pretty darn cool and kind of like having a mini AI Agent already without any setup work like you would have in LangGraph or Autogen, if you’ve worked with either of those.
Context caching
The next cool newly released feature I really want to highlight before we go is context caching. I believe this is going to be very big in the future. The problem with LLMs right now is that they are becoming more powerful, and as such new uses such as analyzing videos are coming up, which are really cool, but also very expensive.
As the possible amount of input tokens increases the cost of the API calls increases as well, and where images or text make no major difference, a video is multiple orders of magnitude more expensive because the file size is so much bigger.
Now if we want to have a proper conversation with multiple turns to analyze a video and ask several questions about it, for each call we have to send that entire video in the context again, which is super expensive. Remember, you don’t actually have a conversation with an LLM, this is all an illusion.
You just have a history that saves the conversation so far and gives the illusion that the LLM remembers what you talked about because you send the history along with each new call, but each call to the LLM is in fact completely isolated and has no relationship to the previous ones.
To solve this problem, Google came up with Context Caching. This is for large sets of input tokens like a video. You pass the video in context once, the normal way, and then cache those input tokens. Now for the next follow-up question you ask about this same video, you don’t pass in the video again, but refer to the cached tokens on the Gemini servers instead.
This is a really cool idea, so let’s look at the prices. Now there are different prices between the flash and pro models here, and since this feature has literally be