The LLM Series #2: Function Calling in OpenAI Models: A Practical Guide

Author(s): Muhammad Saad Uddin

Originally published on Towards AI.

“Artificial Intelligence is the new electricity” as stated by Andrew Ng. Indeed, there’s a spark in the air, and it’s not just static. Welcome to the second article of our LLM Series, where we’re turning up the voltage on OpenAI models U+1F329️U+26A1. I’ll pull back the curtain on OpenAI’s functional calling capabilities, crunching analytics like a math whiz, and answering user queries at lightning speed. It’s about making Language Learning Models not just efficient but supercharged. Get ready for an electrifying journey, and remember, while AI might be the new electricity, I promise no shocking jargon, just illuminating insights U+1F4A1.

We’re going to use Azure OpenAI again. If you’re curious about why we’re doing this, you can find a detailed explanation in the previous article of this series here. We will extract endpoint details; in my case, I use gpt-4 for this

openai.api_type = "azure"
openai.api_version = # API key Version 
openai.api_base = # Azure OpenAI resource's endpoint value
openai.api_key = # Azure OpenAI api key

Function calling is one of the strongest and unique capabilities of open models, which offers enhanced efficiency, structured responses, flexibility, and the ability to handle complex tasks. This feature allows developers to define custom functions, enabling the model to call these directly for streamlined processes. The model can generate structured JSON outputs for easier data manipulation, and it can summarize and return these results to the user for better interaction. Simply put, you can use your LLM to do complicated analysis through defined functions and explain the results to the user in a way that’s easy to understand.

We will start by defining some basic functions, read_data and calculate_sales . The first function reads a simple sales CSV file, and the second one, based on the product name, filters the data and returns the output as JSON.

def read_data():
 df = pd.read_csv('base_data.csv', sep=';')
 return df

For the simplicity of this guide, I’ve included just a basic function. However, as you begin to explore this on your own, I’m confident that you’ll be able to tackle more complex tasks!

def calculate_sales(product: str):
 df = read_data()

 vals = list(df['Product Name'].unique())

 if product not in vals:
 return f"Invalid Product Name. Please choose from: {vals}"

 df_filter = df[df['Product Name']==product]

 # Convert df to dictionary 
 df_dict = df_filter.to_dict('records')

 return json.dumps(df_dict)

Next, we explicitly define a JSON format that OpenAI models need to recognize the functions available for them to call. We provide a logical name for the function and a description of what the function does. It’s crucial to make this description as simple and comprehensive as possible, as a vague description might lead to misinterpretation by the model. We also define inputs for the model and their type, whether it’s a string, integer, float, or another datatype. If you want to limit choices, an enum can be useful in recognizing the correct keyword in user queries. we define all this in function_options and function_to_use is for runtime script when we extract and call the function.

function_options = [
 {
 "name": "calculate_sales",
 "description": "Get the sales data for a given product name",
 "parameters": {
 "type": "object",
 "properties": {
 "product": {
 "type": "string",
 "enum": ['product A','product B','product C','product D','NOT LISTED']},
 },
 "required": ["product"],
 }, 
 }
]

functions_to_use = {
 "calculate_sales": calculate_sales
 } 

Designing a prompt is an art in itself, as a well-designed prompt can significantly improve results and minimize unusual behaviors. As long we have to cater hallucinations in LLMs, this will remain a critical factor in any application designed to incorporate LLMs.

I strongly recommend everyone to read a recent paper about prompts, which can be incredibly beneficial in creating effective prompts.

We then design our system prompt carefully to ensure minimal hallucinations or information leakage and also create a sample user query for input into our setup.

system_prompt = """You are an expert sales bot which provides deep dive insights and analysis related to products. \
 You will be given some information about product as context and you will analyze the given data and only answer to queries when context or data \
 about the specific product asked is given else response with: :I don't have enough knowledge about these products please contact a sales rep:. """

user_query = """provide me a summary of yearly sales for product A"""

Building upon the components we’ve created, we now define our API call, which runs mainly via openai.ChatCompletion.create which require name of the model, an input message which include system message, context and user query, as well as the JSON schema of functions and how the model behaves around these functions.

I have set function_call to ‘auto’ here granting the model autonomy to choose between generating a message or calling a function.

input_message = [{"role": "system", "content": f"{system_prompt}"},
 {"role": "user", "content": f"{user_query}"}]
response = openai.ChatCompletion.create(
 engine="gpt-4",
 messages=input_message,
 functions=function_options,
 function_call='auto'
 )
model_response = response["choices"][0]["message"]
print(model_response)

For our test case, we execute a basic query with the prompt and function information. Here’s what the model responded with:

{
 "role": "assistant",
 "function_call": {
 "name": "calculate_sales",
 "arguments": "{\n \"product\": \"product A\"\n}"
 }
}

So, based on the sample question, the model understands that it can answer the query by calling a function. As a first step, it responds with the name of the function it deems most logical for the query and provides the required input for that function

before going in details about what to do with this output, let's see how many tokens we have used:

response['usage']

<OpenAIObject at 0x2> JSON: {
 "prompt_tokens": 155,
 "completion_tokens": 16,
 "total_tokens": 171
}

Since this was a simple query and we only had the schema of a single function, the token usage is fairly low.

This usage can abruptly change for more advance systems, Especially, if you have multiple functions and your user query requires calling several functions. In some cases the combination of function calls and context length may exceed limit of tokens and result in “Token limit exceeded” error.

This implies that even if your model is capable of correctly routing functions and extracting relevant data, the input token limit might not be sufficient to accommodate all this as input. In this case, we have to devise a strategy on how to ensure the model will not exceed the token limit as this, from my experience, is one of the most common errors occurring in LLM’s development pipelines whether it is for RAG, function calling or a combination of both

I will explain this in more details in next part of this series, where I’ll work with multi-function calling capabilities of OpenAI.

response

Now, let’s take a brief look at the complete response message we receive from the API call:

"object": "chat.completion",
 "created": ,
 "model": "gpt-4",
 "prompt_filter_results": [
 {
 "prompt_index": 0,
 "content_filter_results": {
 "hate": {
 "filtered": false,
 "severity": "safe"
 },
 "self_harm": {
 "filtered": false,
 "severity": "safe"
 },
 "sexual": {
 "filtered": false,
 "severity": "safe"
 },
 "violence": {
 "filtered": false,
 "severity": "safe"
 }
 }
 }
 ],
 "choices": [
 {
 "finish_reason": "function_call",
 "index": 0,
 "message": {
 "role": "assistant",
 "function_call": {
 "name": "calculate_sales",
 "arguments": "{\n \"product\": \"product A\"\n}"
 }
 },
 "content_filter_results": {}
 }
 ],
 "usage": {
 "prompt_tokens": 155,
 "completion_tokens": 16,
 "total_tokens": 171
 }
}

How about we redefine the user query and ask a bit more specific questions?

user_query = """Provide me a summary of sales for product C for year 2019"""

input_message = [{"role": "system", "content": f"{system_prompt}"},
 {"role": "user", "content": f"{user_query}"}]
response = openai.ChatCompletion.create(
 engine="gpt-4",
 messages=input_message,
 functions=function_options,
 function_call='auto'
 )
model_response = response["choices"][0]["message"]
#print(model_response)

model_response.get("function_call")
print("Function call Recommended by Model:")
print(model_response)
print()

As you can see below, if we add another ‘year’ parameter to our function, the model might be able to detect that too. However, as I stated earlier, this is purely for demonstrating the capabilities of function calling. How you choose to maximize its use is entirely up to you.

Function call Recommended by Model:
{
 "role": "assistant",
 "function_call": {
 "name": "calculate_sales",
 "arguments": "{\n \"product\": \"product C\"\n}"
 }
}

With the model’s response ready, the next question is how we actually call the function. To do this, we add the following code: We identify the function name and retrieve the relevant function from the functions_to_use object, load the parameters identified by the model, and execute the function:

function_name = model_response["function_call"]["name"]

function_to_call = functions_to_use[function_name]

function_args = json.loads(model_response["function_call"]["arguments"])
function_response = function_to_call(**function_args)
 
print("Output of function call:")
print(function_name)
print(function_to_call)
print(function_args)
print()
print(function_response)

We can see the details as well as the JSON-style output of the function call. This output can now be used as part of the input context for the final answer

Output of function call:
calculate_sales
<function calculate_sales at 0x0000029056DF3490>
{'product': 'product C'}

[{"Product Name": "product C", "Year": 2019, "Sales": 377}, {"Product Name": "product C", "Year": 2020, "Sales": 854}, .....]

To achieve this, we now append the output from the function call to our initial message. This serves as the context for our next call.

# adding model and function response to message
input_message.append(
 {
 "role": model_response["role"],
 "function_call": {
 "name": model_response["function_call"]["name"],
 "arguments": model_response["function_call"]["arguments"],
 },
 "content": None
 }
 )

input_message.append(
 {
 "role": "function",
 "name": function_name,
 "content": function_response,
 }
 )
print("Messages before next request:")
for message in input_message:
 print(message)
print()

We can now see how our input message looks for the next call:

Messages before next request:
{'role': 'system', 'content': "You are an expert sales bot which provides deep dive insights and analysis related to products.You will be given some information about product as context and you will analyze the given data and only answer to queries when context or data about the specific product asked is given else response with: :I don't have enough knowledge about these products please contact a sales rep:. "}
{'role': 'user', 'content': 'Provide me a summary of sales for product C for year 2019'}
{'role': 'assistant', 'function_call': {'name': 'calculate_sales', 'arguments': '{\n "product": "product C"\n}'}, 'content': None}
{'role': 'function', 'name': 'calculate_sales', 'content': '[{"Product Name": "product C", "Year": 2019, "Sales": 377}, {"Product Name": "product C", "Year": 2020, "Sales": 854},....]

Using this new information, we make another call to the model:

response2 = openai.ChatCompletion.create(
 engine="gpt-4",
 messages=input_message,
 functions=function_options,
 function_call='auto'
 )
model_response_2 = response2["choices"][0]["message"]
print(model_response_2)

and recieve a precise answer:

{
 "role": "assistant",
 "content": "In 2019, the sales for Product C were 377 units."
}

Token usage is still quite low as we only have a single function and a very small dataset. However, don’t be misled; this can reach up to thousands of tokens per call. At that point, you’ll need to find a balance between function complexity, dataset size, and other technical considerations.

response2['usage']

<OpenAIObject at 0x2> JSON: {
 "prompt_tokens": 289,
 "completion_tokens": 16,
 "total_tokens": 305
}

Ok, now you might be wondering: Do I need to execute this step by step?

No, absolutely not!

You can consolidate the script as shown below. This allows the model to handle everything once you input the query.

input_message = [{"role": "system", "content": f"{system_prompt}"},
 {"role": "user", "content": f"{user_query}"}]
response = openai.ChatCompletion.create(
 engine="gpt-4",
 messages=input_message,
 functions=function_options,
 function_call='auto'
 )
model_response = response["choices"][0]["message"]
print(model_response)

if model_response.get("function_call"):
 print("Function call Recommended by Model:")
 print(model_response.get("function_call"))
 print()
 
 function_name = model_response["function_call"]["name"]

 function_to_call = functions_to_use[function_name]

 function_args = json.loads(model_response["function_call"]["arguments"])
 function_response = function_to_call(**function_args)
 
 print("Output of function call:")
 print(function_response)
 print()


input_message.append(
 {
 "role": model_response["role"],
 "function_call": {
 "name": model_response["function_call"]["name"],
 "arguments": model_response["function_call"]["arguments"],
 },
 "content": None
 }
 )
 
input_message.append(
 {
 "role": "function",
 "name": function_name,
 "content": function_response,
 }
 ) 
print("Messages before next request:")
for message in input_message:
 print(message)
print()

response2 = openai.ChatCompletion.create(
 engine="gpt-4",
 messages=input_message,
 functions=function_options,
 function_call='auto'
 )
model_response_2 = response2["choices"][0]["message"]
print(model_response_2)

Now It’s time to take the spotlight!, leverage your newly learned expertise to automate some complex tasks. I‘d love to hear about your victories and success stories in the comments. Also, If you haven't read the previous article of this LLM series yet, don’t worry — here’s the link for you!

That’s it for today, But don’t worry, the adventure continues! In the next chapter of the LLM series, we will have a look at the amazing multiple functional calling capabilities of GPT models. If this guide has sparked your curiosity and you wish to explore more exciting projects in this LLM series, make sure to follow me. With each new project, I promise a journey filled with learning, creativity, and fun. Furthermore:

• U+1F44F Clap for the story (50 claps) to help this article be featured
• U+1F514 Follow Me: LinkedIn U+007CMedium U+007C Website U+007C Got AI queries?

Delighted by the above piece? these additional recommendations will surely pique your interest:

The LLM Series #1: The Fast Track to Fine-Tuning Mastery with Azure

Hassle-free with minimal coding

pub.towardsai.net

The Future of Resumes: Craft Your Interactive Resume Web App

Welcome to the digital age, where paper resumes are as outdated as dial-up internet! If you’re still sending out PDF…

pub.towardsai.net

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI