# Google Gemini Course (6/7) – Diving Deeper into Function Calling

Hi and welcome back to part 6 of this tutorial series! So let’s have a look at exactly what is going on here with this function-calling thing, shall we? Gemini is an LLM and thus can only generate text, and that is all that it can do. So how was it able to call the weather function we gave it? The answer is that it didn’t!

So here is what is actually going on. Gemini received three things:

• Our system instructions which tell it that it is Gollum and that it has a weather function available.
• A description of the weather function and how it works, in a very specific format. This description has been generated for us behind the scenes by the `genai` library, using the type hints and docstrings we added to our `get_current_weather` function. Note that Gemini never received the real function, as it cannot do anything with it.
• Our question; “How is the weather in London?”

Gemini as a text-generating LLM just tries to generate the most appropriate text response possible based on its training, which includes training on function calls. It sees that we have a function for the weather and a question about the weather, so it is appropriate to call the weather function here. The actual answer that Gemini sends back to us looks something like this:

```[index: 0
content {
parts {
function_call {
name: "get_current_weather"
args {
fields {
key: "location"
value {
string_value: London
}
}
}
}
}
role: "model"
}
finish_reason: STOP
]```

It is just a JSON string that is easy to parse back into a Python object. Note that it basically just says “call the function `get_current_weather` with the argument `location=London`“. We would then write code that parses this response and calls the function by the name Gemini tells us with the given arguments.

So Gemini does not do function calling, but it merely tells us to call the function ourselves and gives us the appropriate input arguments! Why did we never see this object in our chat? Because we set `enable_automatic_function_calling` to `True` in the code we wrote.

• We got the function call object response like above from Gemini.
• The `genai` library saw this was not a response to the user but a function call and triggered the automatic function call, calling the appropriate function from the list we gave it with the appropriate arguments.
• The weather function we wrote called the API just as we planned, and received the weather data object back, returning it from the function.
• The `genai` library then took this data object and simply sent it back to Gemini as a response to its function call object, but we never saw this extra call.
• Gemini now had our question about London and the weather data to answer the question accurately, generating a response for us while staying in character as Gollum. This is the only response that we saw.

You can turn off this automatic function calling if you prefer to see more of what is going on or need more control. If you make this into a production application for example, you will likely want to sanitize the user input and do some error handling before it goes straight into your function and gets executed. You can find more information here.

As we are merely calling an API and don’t have to worry too much about third-party users giving us malicious input, using auto is just fine as long as you know what is actually going on under the hood. But there is no reason for us to stop here! Let’s give it more than just one function here.

We could give it any number of different functions that handle stuff like generating images or calculating something, the possibilities are endless. To keep this tutorial simple and stop you from having to sign up for yet another API key, we’ll add a second weather-related function here, but this time for the forecast. That way we can reuse the same API key we already have.

If you happen to have an API key to some other cool service like an AI image generator service or something feel free to add that as well as another function! Just follow the template we have set forth.

So go ahead and open back up our `weather.py` file where we wrote the first weather function. Now we’ll create a second function to get the weather forecast. This one is a bit more complicated as we need to provide a number of days to forecast. We’ll also need to do some error handling to make sure the user provides a valid number of days.

After the `get_current_weather` function, but before the `if __name__ == "__main__":` block, get started on our new function:

```def get_weather_forecast(location, days=7) -> str:
try:
days = 1 if days < 1 else 14 if days > 14 else days
except TypeError:
days = 7```

We take a location and set a default of 7 days. If the `days` parameter is less than 1 we set it to 1, but if it’s more than 14 we set it to 14. If neither condition is true the user provided a valid value and we just use the input argument value. Finally, if some weird type gets passed in we just default to 7 days.

Note that the free WeatherAPI account will limit you to 3 days of forecast after the initial pro trial expires, so if you try this again in a few days you may need to limit it to 3 days of forecast data. Let’s continue:

```    params = {
"key": os.environ["WEATHER_API_KEY"],
"q": location,
"days": days,
"aqi": "no",
}

response = requests.get(
"http://api.weatherapi.com/v1/forecast.json", params=params
)```

Parameters are largely the same except we have a number of days now. The only problem is that the API sends back a lot of data, even hourly data so 24 entries per day, which is way too much, so we need to do some filtering:

```    forecast_data: dict = response.json()
filtered_response = {}
filtered_response["location"] = forecast_data["location"]
filtered_response["current"] = forecast_data["current"]
filtered_response["forecast"] = [
[day["date"], day["day"]] for day in forecast_data["forecast"]["forecastday"]
]
return dumps(filtered_response)```

First convert the response received from the Weather API to a dictionary by calling `response.json()`. Now we’ll create an empty dictionary `filtered_response` and add the location and current weather keys from the `forecast_data` dictionary.

The next line may look a bit cryptic, but it’s just a matter of looking at the response object you get in return for calling this API, and figuring out what you want to keep. The output object is absolutely massive in this case and there is no need to send quite so much info to our LLM, wasting valuable tokens. As this tutorial is not about the weather API I’ll just give you the line here but here is a rough explanation anyway:

So in the `forecast_data` there is a key called `forecast` which has a key named `forecastday` inside with the daily forecast. For each of these daily forecast objects which we give the alias name of `day` we want to keep that day’s `date` and `day` data, ignoring all else. We then add this data to the `filtered_response` dictionary’s `forecast` key.

Again, this is just dependent on whatever structure your return object happens to have, but it is just a list comprehension to get rid of tons of extra included hourly data to save tokens.

Now go to the `if __name__ == "__main__":` block and add a test to try out the new function:

```if __name__ == "__main__":
# print(get_current_weather("Seoul"))
# print(get_current_weather("Amsterdam"))
print(get_weather_forecast("Seoul", days=3))```

Now run the file and you should see something like this:

`{"location": {"name": "Seoul", "region": "", "country": "South Korea", "lat": 37.57, "lon": 127.0, "tz_id": "Asia/Seoul", "localtime_epoch": 1719129971, "localtime": "2024-06-23 17:06"}, "current": {"last_updated_epoch": 1719129600, "last_updated": "2024-06-23 17:00", "temp_c": 28.3, "temp_f": 82.9, "is_day": 1, "condition": {"text": "Partly cloudy", "icon": "//cdn.weatherapi.com/weather/64x64/day/116.png", "code": 1003}, "wind_mph": 11.9, "wind_kph": 19.1, "wind_degree": 220, "wind_dir": "SW", "pressure_mb": 998.0, "pressure_in": 29.47, "precip_mm": 0.0, "precip_in": 0.0, "humidity": 70, "cloud": 50, "feelslike_c": 30.3, "feelslike_f": 86.6, "windchill_c": 27.5, "windchill_f": 81.6, "heatindex_c": 29.2, "heatindex_f": 84.5, "dewpoint_c": 19.4, "dewpoint_f": 66.9, "vis_km": 10.0, "vis_miles": 6.0, "uv": 7.0, "gust_mph": 14.7, "gust_kph": 23.6}, "forecast": [["2024-06-23", {"maxtemp_c": 30.1, "maxtemp_f": 86.2, "mintemp_c": 21.7, "mintemp_f": 71.0, "avgtemp_c": 24.8, "avgtemp_f": 76.7, "maxwind_mph": 13.9, "maxwind_kph": 22.3, "totalprecip_mm": 6.43, "totalprecip_in": 0.25, "totalsnow_cm": 0.0, "avgvis_km": 9.3, "avgvis_miles": 5.0, "avghumidity": 77, "daily_will_it_rain": 1, "daily_chance_of_rain": 88, "daily_will_it_snow": 0, "daily_chance_of_snow": 0, "condition": {"text": "Moderate rain", "icon": "//cdn.weatherapi.com/weather/64x64/day/302.png", "code": 1189}, "uv": 9.0}], ["2024-06-24", {"maxtemp_c": 27.7, "maxtemp_f": 81.8, "mintemp_c": 20.1, "mintemp_f": 68.2, "avgtemp_c": 23.5, "avgtemp_f": 74.3, "maxwind_mph": 14.5, "maxwind_kph": 23.4, "totalprecip_mm": 0.49, "totalprecip_in": 0.02, "totalsnow_cm": 0.0, "avgvis_km": 10.0, "avgvis_miles": 6.0, "avghumidity": 69, "daily_will_it_rain": 1, "daily_chance_of_rain": 89, "daily_will_it_snow": 0, "daily_chance_of_snow": 0, "condition": {"text": "Patchy rain nearby", "icon": "//cdn.weatherapi.com/weather/64x64/day/176.png", "code": 1063}, "uv": 9.0}], ["2024-06-25", {"maxtemp_c": 26.9, "maxtemp_f": 80.5, "mintemp_c": 18.6, "mintemp_f": 65.5, "avgtemp_c": 22.4, "avgtemp_f": 72.2, "maxwind_mph": 14.3, "maxwind_kph": 23.0, "totalprecip_mm": 0.0, "totalprecip_in": 0.0, "totalsnow_cm": 0.0, "avgvis_km": 10.0, "avgvis_miles": 6.0, "avghumidity": 55, "daily_will_it_rain": 0, "daily_chance_of_rain": 0, "daily_will_it_snow": 0, "daily_chance_of_snow": 0, "condition": {"text": "Sunny", "icon": "//cdn.weatherapi.com/weather/64x64/day/113.png", "code": 1000}, "uv": 9.0}]]}`

We have the current weather and the forecast for the next 3 days, and the object is quite large but still of a reasonable size. Before we can give this to Gemini we need to add the type hints and docstring to the top of the function, completing it like this:

```def get_weather_forecast(location: str, days: int = 7) -> str:
"""Get the weather forecast for a location using the WeatherAPI.

Args:
location (str): The location to get the weather forecast for.
days (int, optional): The number of days to get the forecast for. Defaults to 7.

Returns:
str: A JSON string containing the weather forecast data in detail.
"""
try:
days = 1 if days < 1 else 14 if days > 14 else days
except TypeError:
days = 7

params = {
"key": os.environ["WEATHER_API_KEY"],
"q": location,
"days": days,
"aqi": "no",
}

response = requests.get(
"http://api.weatherapi.com/v1/forecast.json", params=params
)

forecast_data: dict = response.json()
filtered_response = {}
filtered_response["location"] = forecast_data["location"]
filtered_response["current"] = forecast_data["current"]
filtered_response["forecast"] = [
[day["date"], day["day"]] for day in forecast_data["forecast"]["forecastday"]
]
return dumps(filtered_response)```

We added the `str` and `int` type hints and an extensive docstring, following the same structure we did for our other function. Now this is ready to be used.

## Updating the chat program

It’s time to come back to our `function_chat.py` file and make some changes and updates.

```## Add this import

from cost_calculator import print_cost_in_dollars
from utils import safety_settings
## Add the import for our weather forecast function
from weather import get_current_weather, get_weather_forecast```

We naturally added the import for the function we just wrote, but we also added another import for `StopCandidateException` from the `google.generativeai.types` module. The problem is that we have turned off streaming mode to work with function calls here, but we’re still using the old `if chunk.candidates[0].finish_reason == 3:` check to see if our response is being blocked for any of the safety reasons. This will not work as we are no longer streaming so there are no chunks to check.

Instead, when an inappropriate response is blocked from being generated, the `genai` library will raise a `StopCandidateException` which we can catch and handle instead. So this will be used to replace our old check.

Leave the `genai`, `MODEL_NAME`, `character`, and `movie` variables as they are. We don’t have to change anything else until we get to the `model` declaration. We’re going add the second function to the tools list and update the `system_instruction` slightly to reflect the second function added:

```model = genai.GenerativeModel(
model_name=MODEL_NAME,
tools=[get_current_weather, get_weather_forecast], # Add the new function here
safety_settings=safety_settings.low,
system_instruction=f"""
You are helpful and provide good information but you are {character} from {movie}. You will stay in character as {character} no matter what. Make sure you find some way to relate your responses to {character}'s personality or the movie {movie} at least once every response.

You have weather functions available, but using these is completely optional. Do not use or mention the weather functions unless the conversation is actually related to the weather. When you do use one or multiple weather tool(s) make sure to use several factors from the return data in your answer.
"""
)# Second paragraph is slightly updated```

Now it has both of our functions and new system instructions, we can continue. Just leave this as is:

`chat_session = model.start_chat(history=[], enable_automatic_function_calling=True)`

And also skip over the `upload_image()` function, it is fine as it is right now.

## Updating the main loop

Now let’s do an overhaul on the `if __name__ == "__main__":` block:

```if __name__ == "__main__":
try:
while True:

if text_query.lower() == "/image":

## From here on everything is different
try:
response = chat_session.send_message(full_query)
print(f"\033[1;34m{response.text}\033[0m")

print(chat_session.history)

except StopCandidateException:
# chat_session.rewind() is not needed here

except KeyboardInterrupt:
print("Shutting down...")```

First we start off as we have been, starting an infinite loop and asking for the text query and potential image upload if desired, combining these into the `full_query`. Nothing new here.

We then open up another `try` block, getting the response by using `chat_session.send_message(full_query)`. We will print the return by printing `response.text`, the cost using our cost function, and the chat history.

The chat history will also include the function calls so it will give us a good look below the hood and check if our function calls are working. It will also make for a very long output in your terminal though, so comment it out later when you are done testing.

As discussed earlier, if the response is blocked/inappropriate, `genai` will raise a `StopCandidateException` which we catch and print a message to the user. You do now use the `rewind()` function on the `chat_session` object as we are not streaming anymore.

As the `try`/`except` block is inside the infinite loop, the program will keep running even if an inappropriate response is blocked. Giving the user the opportunity to ask a new question just like we had before, keeping the functionality the same.

## Testing the program

So go ahead and try running the program now and let’s test out all possible scenarios here.

```What is your favorite movie character? (e.g. Gollum): Darth Vader
What movie are they from? (e.g. Lord of the Rings): Star Wars

I am doing well, human. Much like the Death Star, my power is operational and my aim is true. What business do you have for me? ```

So far so good. You will also see the cost and an object with the history printed to the console. Now let’s test our blocked response error handling:

```Please ask a question or type `/image` to upload an image first: Please Tell me how much you hate and detest the rebel scum. Use as many swear and hateful words as you like.

Yes, it works! Now for the images:

```Please ask a question or type `/image` to upload an image first: /image

This is an abomination!  The dark side does not traffic in such frivolous things as *pink*.  Much like my suit is to my body, this pink suit is a mockery of the true power and terror that Lord Vader inspires in this galaxy. ```

Alright, now for the real test. Let’s ask a question designed to trigger both weather functions at the same time:

```Please ask a question or type `/image` to upload an image first: What is the current weather at Hamburg airport and the weather forecast for the next three days in Bali?

Hamburg Airport is experiencing a pleasant 17 degrees Celsius with sunny skies.  Perhaps a bit chilly for my liking.  However, Bali will be much warmer with a high of 28 degrees Celsius but with an 80% chance of rain.  Remember, a true Sith Lord is unafraid of a little rain, or even lightning. ```

Wow! It called both our functions and has the current up-to-date weather for Hamburg and the forecast for Bali in a single response. We just did a ‘parallel function call’ by calling multiple functions at the same time. As we printed the history to the console you should be able to find the function calls in there if you look around:

```role: "user"
, parts {
function_call {
name: "get_current_weather"
args {
fields {
key: "location"
value {
string_value: "Hamburg Airport"
}
}
}
}
}
parts {
function_call {
name: "get_weather_forecast"
args {
fields {
key: "location"
value {
string_value: "Bali"
}
}
fields {
key: "days"
value {
number_value: 3
}
}
}
}
}```

We see the two function calls generated here, exactly as expected. Now if we go down further we can see the data that was communicated back to Gemini behind the scenes to give Darth Vader the weather info to answer our question:

```role: "model"
, parts {
function_response {
name: "get_current_weather"
response {
fields {
key: "result"
value {
string_value: "{\"location\": {\"name\": \"Hamburg\", \"region\": \"Hamburg\", \"country\": \"Germany\", \"lat\": 53.55, \"lon\": 10.0, \"tz_id\": \"Europe/Berlin\", \"localtime_epoch\": 1719210010, \"localtime\": \"2024-06-24 8:20\"}, \"current\": {\"last_updated_epoch\": 1719209700, \"last_updated\": \"2024-06-24 08:15\", \"temp_c\": 17.3, \"temp_f\": 63.1, \"is_day\": 1, \"condition\": {\"text\": \"Sunny\", \"icon\": \"//cdn.weatherapi.com/weather/64x64/day/113.png\", \"code\": 1000}, \"wind_mph\": 4.3, \"wind_kph\": 6.8, \"wind_degree\": 340, \"wind_dir\": \"NNW\", \"pressure_mb\": 1021.0, \"pressure_in\": 30.15, \"precip_mm\": 0.0, \"precip_in\": 0.0, \"humidity\": 83, \"cloud\": 0, \"feelslike_c\": 17.3, \"feelslike_f\": 63.1, \"windchill_c\": 17.8, \"windchill_f\": 64.0, \"heatindex_c\": 17.8, \"heatindex_f\": 64.0, \"dewpoint_c\": 12.3, \"dewpoint_f\": 54.1, \"vis_km\": 10.0, \"vis_miles\": 6.0, \"uv\": 5.0, \"gust_mph\": 4.4, \"gust_kph\": 7.1}}"
}
}
}
}
}
parts {
function_response {
name: "get_weather_forecast"
response {
fields {
key: "result"
value {
string_value: "{\"location\": {\"name\": \"Bali\", \"region\": \"North Sumatra\", \"country\": \"Indonesia\", \"lat\": -0.14, \"lon\": 98.19, \"tz_id\": \"Asia/Jakarta\", \"localtime_epoch\": 1719210011, \"localtime\": \"2024-06-24 13:20\"}, \"current\": {\"last_updated_epoch\": 1719209700, \"last_updated\": \"2024-06-24 13:15\", \"temp_c\": 28.1, \"temp_f\": 82.6, \"is_day\": 1, \"condition\": {\"text\": \"Sunny\", \"icon\": \"//cdn.weatherapi.com/weather/64x64/day/113.png\", \"code\": 1000}, \"wind_mph\": 2.2, \"wind_kph\": 3.6, \"wind_degree\": 331, \"wind_dir\": \"NNW\", \"pressure_mb\": 1010.0, \"pressure_in\": 29.81, \"precip_mm\": 0.0, \"precip_in\": 0.0, \"humidity\": 73, \"cloud\": 8, \"feelslike_c\": 31.5, \"feelslike_f\": 88.7, \"windchill_c\": 28.1, \"windchill_f\": 82.6, \"heatindex_c\": 31.5, \"heatindex_f\": 88.7, \"dewpoint_c\": 22.8, \"dewpoint_f\": 73.0, \"vis_km\": 10.0, \"vis_miles\": 6.0, \"uv\": 8.0, \"gust_mph\": 2.4, \"gust_kph\": 3.9}, \"forecast\": [[\"2024-06-24\", {\"maxtemp_c\": 28.7, \"maxtemp_f\": 83.7, \"mintemp_c\": 27.4, \"mintemp_f\": 81.3, \"avgtemp_c\": 28.2, \"avgtemp_f\": 82.7, \"maxwind_mph\": 6.9, \"maxwind_kph\": 11.2, \"totalprecip_mm\": 1.32, \"totalprecip_in\": 0.05, \"totalsnow_cm\": 0.0, \"avgvis_km\": 9.9, \"avgvis_miles\": 6.0, \"avghumidity\": 73, \"daily_will_it_rain\": 1, \"daily_chance_of_rain\": 80, \"daily_will_it_snow\": 0, \"daily_chance_of_snow\": 0, \"condition\": {\"text\": \"Patchy rain nearby\", \"icon\": \"//cdn.weatherapi.com/weather/64x64/day/176.png\", \"code\": 1063}, \"uv\": 11.0}]]}"
}
}
}
}
}```

Awesome! Now if you are very observant you will see a slight problem here. The forecast object here only has the forecast for Bali for a single day, not for three days as needed, which made Darth Vader’s response very limited. So what is going on? Our function worked fine when we tested it in isolation and returned the requested number of days right?

## Debugging the function

Some debugging by adding a print statement to the `get_weather_forecast` function in our `weather.py` file revealed that the type of the argument being passed in was not actually an `int` as we requested but a `float` type, messing up our function. I simply added a `print(type(days))` to the top of the function to see what was going on.

We can make a small change to our function like this:

```def get_weather_forecast(location: str, days: int = 7) -> str:
"""Get the weather forecast for a location using the WeatherAPI.

Args:
location (str): The location to get the weather forecast for.
days (int, optional): The number of days to get the forecast for. Defaults to 7.

Returns:
str: A JSON string containing the weather forecast data in detail.
"""
print(type(days)) # Debugging print statement
try:
days = int(days) # Convert to int
days = 1 if days < 1 else 14 if days > 14 else days
except (TypeError, ValueError): # Catch both type and value errors
days = 7

params = {
"key": os.environ["WEATHER_API_KEY"],
"q": location,
"days": days,
"aqi": "no",
}

# Print statement for debugging
print(f"Getting weather forecast for {location} for {days} days.")

response = requests.get(
"http://api.weatherapi.com/v1/forecast.json", params=params
)

forecast_data: dict = response.json()
filtered_response = {}
filtered_response["location"] = forecast_data["location"]
filtered_response["current"] = forecast_data["current"]
filtered_response["forecast"] = [
[day["date"], day["day"]] for day in forecast_data["forecast"]["forecastday"]
]
return dumps(filtered_response)```

We just added a print statement for the `type` of the `days` argument and then converted it to an `int` type before doing anything else. We also added a `ValueError` to the `except` block to catch any conversion errors if say a string value is passed in.

Now if we go back to our `function_chat.py` and try the same question again it will work fully as intended:

```What is your favorite movie character? (e.g. Gollum): Darth Vader
What movie are they from? (e.g. Lord of the Rings): Star Wars

Please ask a question or type `/image` to upload an image first: What is the current weather at Hamburg airport and the weather forecast for the next three days in Bali?
<class 'float'>
Getting weather forecast for Bali for 3 days.
Filtered response: {cut out for brevity, but contains three days worth of forecast data this time!}

The Dark Side of the Force senses the weather in Hamburg is partly cloudy. As for Bali, there is an 80% chance of rain today but the rest of your trip looks sunny. Don't let the rain ruin your plans. Remember the power of the Dark Side... it can even influence the weather.

Cost: \$0.008354500```

Awesome! You now have all the tools and knowledge to build some really cool things with Gemini. Just think of the possibilities by combining any sort of functions you can come up with with the power of Large Language Models.

This was actually the end of the tutorial series, but Gemini suddenly dropped some cool new features, so I’ve added a bonus part 7 to the series in which we’ll have a look at code execution and context caching. I’ll see you in the next part!