This guide discusses calling custom functions as tools. Writer also offers prebuilt tools that models can execute remotely:
You need an API key to access the Writer API. Get an API key by following the steps in the API quickstart.We recommend setting the API key as an environment variable in a
.env file with the name WRITER_API_KEY.Overview
To use tool calling, follow these steps:- Define your functions in code
- Pass the functions to the model in a chat completion request
- Append the assistant’s response (containing tool calls) to the message history
- Check to see which functions the model wants to invoke and run the corresponding functions
- Append the tool results to the message history
- Pass the updated messages back to the model to get the final response
- Append the final response to maintain complete conversation history
Tool calling overview
Example: Calculate the mean of a list of numbers
Example: Calculate the mean and standard deviation of a list of numbers
Define your custom functions
First, define the custom functions in your code. Typical use cases for tool calling include calling an API, performing mathematical calculations, or running complex business logic. You can define these functions in your code as you would any other function. Here’s an example of a function to calculate the mean of a list of numbers.def calculate_mean(numbers: list) -> float:
return sum(numbers) / len(numbers)
function calculateMean(numbers) {
if (numbers.length === 0) {
throw new Error("Cannot calculate mean of an empty array");
}
return numbers.reduce((sum, num) => sum + num, 0) / numbers.length;
}
Describe functions as tools
After you’ve defined your functions, create atools array to pass to the model.
The tools array describes your functions as tools available to the model. You describe tools in the form of a JSON schema. Each tool should include a type of function and a function object that includes a name, description, and a dictionary of parameters.
Tool structure
Thetools array contains an object with the following parameters:
| Parameter | Type | Description |
|---|---|---|
type | string | The type of tool, which is function for a custom function |
function | object | An object containing the tool’s description and parameter definitions |
function.name | string | The name of the tool |
function.description | string | A description of what the tool does and when the model should use it |
function.parameters | object | An object containing the tool’s input parameters |
function.parameters.type | string | The type of the parameter, which is object for a JSON schema |
function.parameters.properties | object | An object containing the tool’s parameters in the form of a JSON schema. See below for more details. |
function.parameters.required | array | An array of the tool’s required parameters |
function.parameters.properties object contains the tool’s parameter definitions as a JSON schema. The object’s keys should be the names of the parameters, and the values should be objects containing the parameter’s type and description.
When the model decides you should use the tool to answer the user’s question, it returns the parameters that you should use when calling the function you’ve defined.
Example tool array
Here’s an example of atools array for the calculate_mean function:
tools = [
{
"type": "function",
"function": {
"name": "calculate_mean",
"description": "A function that calculates the mean (average) of a list of numbers. Any user request asking for the mean of a list of numbers should use this tool.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers"
}
},
"required": ["numbers"]
}
}
}
]
const tools = [
{
type: "function",
function: {
name: "calculate_mean",
description: "A function that calculates the mean (average) of a list of numbers. Any user request asking for the mean of a list of numbers should use this tool.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers",
},
},
required: ["numbers"],
},
},
},
];
To help the model understand when to use the tool, follow these best practices for the
function.description parameter:- Indicate that the tool is a function that invokes a no-code agent
- Specify the function’s purpose and capabilities
- Describe when the tool should be used
“A function that calculates the mean of a list of numbers. Any user request asking for the mean of a list of numbers should use this tool.”
Pass tools to the model
Once the tools array is complete, pass it to the chat completions endpoint along with the chat messages.Tool choice control
The chat completion endpoint has atool_choice parameter that controls how the model decides when to use the tools you’ve defined.
| Value | Description |
|---|---|
auto | The model decides which tools to use, if any. |
none | The model doesn’t use tools and only returns a generated response. |
required | The model must use at least one of the tools you’ve defined. |
calculate_mean tool, you can set tool_choice to {"type": "function", "function": {"name": "calculate_mean"}}.
In this example, tool_choice is auto, which means the model decides which tools to use, if any, based on the message and tool descriptions.
import json
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
messages = [{"role": "user", "content": "what is the mean of [1,3,5,7,9]?"}]
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto"
)
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
let messages = [{"role": "user", "content": "what is the mean of [1,3,5,7,9]?"}];
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto"
});
Process tool calls
When the model identifies a need to call a tool based on the user’s input, it indicates it in the response and includes the necessary parameters to pass when calling the tool. You then execute the tool’s function and return the result to the model.Proper conversation history management requires appending the assistant’s response, tool results, and final response to the message history.
The examples below demonstrate processing a single tool call. For handling multiple tool calls in one request, see the multiple tool calls section.
- Streaming
- Non-streaming
Streaming
When using streaming, the tool calls come back in chunks inside of the delta object of the choices array. To process the tool calls:- Stream and collect tool calls from the response chunks
- Reconstruct the assistant’s response and append it to the message history
- Execute functions and append tool results to the message history
- Get the final response from the model and append it to the message history
Step 1: Stream and collect tool calls from the response chunks
First, stream and collect tool calls from the response chunks.streaming_content = ""
function_calls = []
for chunk in response:
choice = chunk.choices[0]
if choice.delta:
# Collect tool calls as they stream in
if choice.delta.tool_calls:
for tool_call in choice.delta.tool_calls:
if tool_call.id:
# Start a new function call
function_calls.append({
"name": "",
"arguments": "",
"call_id": tool_call.id
})
if tool_call.function:
# Append to the most recent function call
if function_calls:
function_calls[-1]["name"] += tool_call.function.name or ""
function_calls[-1]["arguments"] += tool_call.function.arguments or ""
# Collect regular content (for cases where no tools are called)
elif choice.delta.content:
streaming_content += choice.delta.content
let streamingContent = "";
const functionCalls = [];
for await (const chunk of response) {
const choice = chunk.choices[0];
if (choice.delta) {
if (choice.delta.tool_calls) {
for (const toolCall of choice.delta.tool_calls) {
if (toolCall.id) {
functionCalls.push({
name: "",
arguments: "",
call_id: toolCall.id
});
}
if (toolCall.function) {
if (functionCalls.length > 0) {
functionCalls[functionCalls.length - 1].name += toolCall.function.name || "";
functionCalls[functionCalls.length - 1].arguments += toolCall.function.arguments || "";
}
}
}
} else if (choice.delta.content) {
streamingContent += choice.delta.content;
}
}
}
Step 2: Check finish reason and reconstruct the assistant’s response
Check thefinish_reason to determine if tools were called, then reconstruct and append the assistant’s response to the conversation history.# Inside the streaming loop, check for finish_reason
if choice.finish_reason:
if choice.finish_reason == "stop":
# No tools were called, just regular response
messages.append({"role": "assistant", "content": streaming_content})
break
elif choice.finish_reason == "tool_calls":
# Reconstruct and append assistant message with tool calls
tool_calls_for_message = []
for func_call in function_calls:
tool_calls_for_message.append({
"id": func_call["call_id"],
"type": "function",
"function": {
"name": func_call["name"],
"arguments": func_call["arguments"]
}
})
assistant_message = {
"role": "assistant",
"content": None,
"tool_calls": tool_calls_for_message
}
messages.append(assistant_message)
// Inside the streaming loop, check for finish_reason
if (choice.finish_reason) {
if (choice.finish_reason === "stop") {
// No tools were called, just regular response
messages.push({ role: "assistant", content: streamingContent });
break;
} else if (choice.finish_reason === "tool_calls") {
// Reconstruct and append assistant message with tool calls
const toolCallsForMessage = [];
for (const funcCall of functionCalls) {
toolCallsForMessage.push({
id: funcCall.call_id,
type: "function",
function: {
name: funcCall.name,
arguments: funcCall.arguments
}
});
}
const assistantMessage = {
role: "assistant",
content: null,
tool_calls: toolCallsForMessage
};
messages.push(assistantMessage); // Append assistant response
}
}
Step 3: Execute functions and append tool results
Execute each function in thefunction_calls list and append the results to the messages array.# Inside the tool_calls finish_reason block
for function_call in function_calls:
function_name = function_call["name"]
if function_name == "calculate_mean":
try:
arguments_dict = json.loads(function_call["arguments"])
function_response = calculate_mean(arguments_dict["numbers"])
# Append tool result to conversation history
messages.append({
"role": "tool",
"content": str(function_response),
"tool_call_id": function_call["call_id"],
"name": function_name,
})
except Exception as e:
# Handle errors gracefully
messages.append({
"role": "tool",
"content": f"Error: {str(e)}",
"tool_call_id": function_call["call_id"],
"name": function_name,
})
// Inside the tool_calls finish_reason block
for (const functionCall of functionCalls) {
const functionName = functionCall.name;
if (functionName === "calculate_mean") {
try {
const argumentsDict = JSON.parse(functionCall.arguments);
const functionResponse = calculateMean(argumentsDict.numbers);
// Append tool result to conversation history
messages.push({
role: "tool",
content: functionResponse.toString(),
tool_call_id: functionCall.call_id,
name: functionName,
});
} catch (error) {
// Handle errors gracefully
messages.push({
role: "tool",
content: `Error: ${error.message}`,
tool_call_id: functionCall.call_id,
name: functionName,
});
}
}
}
Step 4: Get and append the final response
After appending tool results, get the final response from the model and append it to maintain complete conversation history.# Inside the tool_calls finish_reason block, after processing all functions
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=True
)
final_content = ""
for chunk in final_response:
choice = chunk.choices[0]
if choice.delta and choice.delta.content:
final_content += choice.delta.content
# Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
// Inside the tool_calls finish_reason block, after processing all functions
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: true
});
let finalContent = "";
for await (const chunk of finalResponse) {
const choice = chunk.choices[0];
if (choice.delta && choice.delta.content) {
finalContent += choice.delta.content;
process.stdout.write(choice.delta.content);
}
}
// Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
Complete streaming code example
import json
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
def calculate_mean(numbers: list) -> float:
return sum(numbers) / len(numbers)
tools = [
{
"type": "function",
"function": {
"name": "calculate_mean",
"description": "Calculate the mean (average) of a list of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers"
}
},
"required": ["numbers"]
}
}
}
]
messages = [{"role": "user", "content": "what is the mean of [1,3,5,7,9]?"}]
# Step 1: Initial request with tools
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto",
stream=True
)
# Step 2: Process streaming response to collect tool calls
streaming_content = ""
function_calls = []
for chunk in response:
choice = chunk.choices[0]
if choice.delta:
# Collect tool calls as they stream in
if choice.delta.tool_calls:
for tool_call in choice.delta.tool_calls:
if tool_call.id:
# Start a new function call
function_calls.append({
"name": "",
"arguments": "",
"call_id": tool_call.id
})
if tool_call.function:
# Append to the most recent function call
if function_calls:
function_calls[-1]["name"] += tool_call.function.name or ""
function_calls[-1]["arguments"] += tool_call.function.arguments or ""
# Collect regular content (for cases where no tools are called)
elif choice.delta.content:
streaming_content += choice.delta.content
# Check if streaming is complete
if choice.finish_reason:
if choice.finish_reason == "stop":
# No tools were called, just regular response
print(f"Response: {streaming_content}")
messages.append({"role": "assistant", "content": streaming_content})
break
elif choice.finish_reason == "tool_calls":
# Step 3: Reconstruct and append assistant message with tool calls
tool_calls_for_message = []
for func_call in function_calls:
tool_calls_for_message.append({
"id": func_call["call_id"],
"type": "function",
"function": {
"name": func_call["name"],
"arguments": func_call["arguments"]
}
})
assistant_message = {
"role": "assistant",
"content": None,
"tool_calls": tool_calls_for_message
}
messages.append(assistant_message) # Append assistant response
# Step 4: Execute each function and add results to messages
for function_call in function_calls:
function_name = function_call["name"]
if function_name == "calculate_mean":
try:
arguments_dict = json.loads(function_call["arguments"])
function_response = calculate_mean(arguments_dict["numbers"])
# Add tool response to messages
messages.append({
"role": "tool",
"content": str(function_response),
"tool_call_id": function_call["call_id"],
"name": function_name,
})
except Exception as e:
# Handle errors gracefully
messages.append({
"role": "tool",
"content": f"Error: {str(e)}",
"tool_call_id": function_call["call_id"],
"name": function_name,
})
# Step 5: Get the final response from the model
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=True
)
final_content = ""
for chunk in final_response:
choice = chunk.choices[0]
if choice.delta and choice.delta.content:
final_content += choice.delta.content
print(choice.delta.content, end="", flush=True)
# Step 6: Add final response to message history
messages.append({
"role": "assistant",
"content": final_content
})
break
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
function calculateMean(numbers) {
if (numbers.length === 0) {
throw new Error("Cannot calculate mean of an empty array");
}
return numbers.reduce((sum, num) => sum + num, 0) / numbers.length;
}
const tools = [
{
type: "function",
function: {
name: "calculate_mean",
description: "Calculate the mean (average) of a list of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers",
},
},
required: ["numbers"],
},
},
},
];
async function main() {
let messages = [
{ role: "user", content: "what is the mean of [1,3,5,7,9]?" },
];
// Step 1: Initial request with tools
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto",
stream: true
});
// Step 2: Process streaming response to collect tool calls
let streamingContent = "";
const functionCalls = [];
for await (const chunk of response) {
const choice = chunk.choices[0];
if (choice.delta) {
// Collect tool calls as they stream in
if (choice.delta.tool_calls) {
for (const toolCall of choice.delta.tool_calls) {
if (toolCall.id) {
// Start a new function call
functionCalls.push({
name: "",
arguments: "",
call_id: toolCall.id,
});
}
if (toolCall.function) {
// Append to the most recent function call
if (functionCalls.length > 0) {
functionCalls[functionCalls.length - 1].name += toolCall.function.name || "";
functionCalls[functionCalls.length - 1].arguments += toolCall.function.arguments || "";
}
}
}
}
// Collect regular content (for cases where no tools are called)
else if (choice.delta.content) {
streamingContent += choice.delta.content;
}
}
// Check if streaming is complete
if (choice.finish_reason) {
if (choice.finish_reason === "stop") {
// No tools were called, just regular response
console.log(`Response: ${streamingContent}`);
messages.push({ role: "assistant", content: streamingContent });
break;
} else if (choice.finish_reason === "tool_calls") {
// Step 3: Reconstruct and append assistant message with tool calls
const toolCallsForMessage = [];
for (const funcCall of functionCalls) {
toolCallsForMessage.push({
id: funcCall.call_id,
type: "function",
function: {
name: funcCall.name,
arguments: funcCall.arguments,
},
});
}
const assistantMessage = {
role: "assistant",
content: null,
tool_calls: toolCallsForMessage,
};
messages.push(assistantMessage); // Append assistant response
// Step 4: Execute each function and add results to messages
for (const functionCall of functionCalls) {
const functionName = functionCall.name;
if (functionName === "calculate_mean") {
try {
const argumentsDict = JSON.parse(functionCall.arguments);
const functionResponse = calculateMean(argumentsDict.numbers);
// Add tool response to messages
messages.push({
role: "tool",
content: functionResponse.toString(),
tool_call_id: functionCall.call_id,
name: functionName,
});
} catch (error) {
// Handle errors gracefully
messages.push({
role: "tool",
content: `Error: ${error.message}`,
tool_call_id: functionCall.call_id,
name: functionName,
});
}
}
}
// Step 5: Get the final response from the model
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: true
});
let finalContent = "";
for await (const chunk of finalResponse) {
const choice = chunk.choices[0];
if (choice.delta && choice.delta.content) {
finalContent += choice.delta.content;
process.stdout.write(choice.delta.content);
}
}
// Step 6: Add final response to message history
messages.push({
role: "assistant",
content: finalContent,
});
break;
}
}
}
}
main();
Non-streaming
If you setstream to false, the tool calls come back in one object inside of the message object in the choices array. To process the tool calls:- Get the assistant’s response and append it to the message history
- Check for tool calls and execute the corresponding functions, appending tool results to the message history
- Get the final response from the model and append it to the message history
Step 1: Get and append the assistant’s response
First, get the assistant’s response message and append it to your conversation history. This is crucial for maintaining context.response_message = response.choices[0].message
messages.append(response_message) # Always append the assistant's response
const responseMessage = response.choices[0].message;
messages.push(responseMessage); // Always append the assistant's response
Step 2: Execute functions and append tool results
Next, check for tool calls and execute the corresponding functions with the arguments provided by the model. Then, append the tool results to the conversation history.tool_calls = response_message.tool_calls
if tool_calls:
for tool_call in tool_calls:
tool_call_id = tool_call.id
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "calculate_mean":
function_response = calculate_mean(function_args["numbers"])
# Append tool result to conversation history
messages.append({
"role": "tool",
"tool_call_id": tool_call_id,
"name": function_name,
"content": str(function_response),
})
const toolCalls = responseMessage.tool_calls;
if (toolCalls && toolCalls.length > 0) {
for (const toolCall of toolCalls) {
const toolCallId = toolCall.id;
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
if (functionName === "calculate_mean") {
const functionResponse = calculateMean(functionArgs.numbers);
// Append tool result to conversation history
messages.push({
role: "tool",
tool_call_id: toolCallId,
name: functionName,
content: functionResponse.toString(),
});
}
}
}
Step 3: Get and append the final response
After appending tool results, get the final response from the model and append it to maintain complete conversation history:final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=False
)
final_content = final_response.choices[0].message.content
print(f"Final response: {final_content}")
# Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: false
});
const finalContent = finalResponse.choices[0].message.content;
console.log(`Final response: ${finalContent}`);
// Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
Complete non-streaming code example
import json
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
def calculate_mean(numbers: list) -> float:
return sum(numbers) / len(numbers)
tools = [
{
"type": "function",
"function": {
"name": "calculate_mean",
"description": "Calculate the mean (average) of a list of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers"
}
},
"required": ["numbers"]
}
}
}
]
messages = [{"role": "user", "content": "what is the mean of [1,3,5,7,9]?"}]
# Step 1: Initial request with tools
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto",
stream=False
)
# Step 2: Get and append assistant response
response_message = response.choices[0].message
messages.append(response_message)
tool_calls = response_message.tool_calls
# Step 3: Process tool calls if any
if tool_calls:
for tool_call in tool_calls:
tool_call_id = tool_call.id
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "calculate_mean":
function_response = calculate_mean(function_args["numbers"])
# Append tool result to conversation history
messages.append({
"role": "tool",
"tool_call_id": tool_call_id,
"name": function_name,
"content": str(function_response),
})
# Step 4: Get final response
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=False
)
final_content = final_response.choices[0].message.content
print(f"Final response: {final_content}")
# Step 5: Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
else:
# No tool calls, just use the original response
print(f"Response: {response_message.content}")
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
function calculateMean(numbers) {
if (numbers.length === 0) {
throw new Error("Cannot calculate mean of an empty array");
}
return numbers.reduce((sum, num) => sum + num, 0) / numbers.length;
}
const tools = [
{
type: "function",
function: {
name: "calculate_mean",
description: "Calculate the mean (average) of a list of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers",
},
},
required: ["numbers"],
},
},
},
];
async function main() {
let messages = [
{ role: "user", content: "what is the mean of [1,3,5,7,9]?" },
];
// Step 1: Initial request with tools
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto",
stream: false,
});
// Step 2: Get and append assistant response
const responseMessage = response.choices[0].message;
messages.push(responseMessage); // Append assistant response
const toolCalls = responseMessage.tool_calls;
// Step 3: Process tool calls if any
if (toolCalls && toolCalls.length > 0) {
for (const toolCall of toolCalls) {
const toolCallId = toolCall.id;
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
if (functionName === "calculate_mean") {
const functionResponse = calculateMean(functionArgs.numbers);
// Append tool result to conversation history
messages.push({
role: "tool",
tool_call_id: toolCallId,
name: functionName,
content: functionResponse.toString(),
});
}
}
// Step 4: Get final response
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: false
});
const finalContent = finalResponse.choices[0].message.content;
console.log(`Final response: ${finalContent}`);
// Step 5: Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
} else {
// No tool calls, just use the original response
console.log(`Response: ${responseMessage.content}`);
}
}
main();
Multiple tool calls
When the model uses multiple tools in a single request, you’ll receive multiple tool calls in the response. This section shows you how to handle multiple tool calls for both streaming and non-streaming approaches.Non-streaming: The processing logic for multiple tool calls is identical to single tool calls - the same code handles both scenarios. The only difference is that the loop for checking the response for tool calling processes multiple tool calls instead of one.Streaming: Multiple tool calls require different logic because tool calls stream in chunks. You need to collect and reconstruct multiple tool calls from streaming chunks before processing them.
Define multiple functions
First, define the functions in your code.import math
def calculate_mean(numbers: list) -> float:
if not numbers:
raise ValueError("Cannot calculate mean of an empty list")
return sum(numbers) / len(numbers)
def calculate_standard_deviation(numbers: list) -> float:
if len(numbers) < 2:
raise ValueError("Cannot calculate standard deviation with fewer than 2 numbers")
mean = calculate_mean(numbers)
variance = sum((x - mean) ** 2 for x in numbers) / len(numbers)
return math.sqrt(variance)
function calculateMean(numbers) {
if (numbers.length === 0) {
throw new Error("Cannot calculate mean of an empty array");
}
return numbers.reduce((sum, num) => sum + num, 0) / numbers.length;
}
function calculateStandardDeviation(numbers) {
if (numbers.length < 2) {
throw new Error("Cannot calculate standard deviation with fewer than 2 numbers");
}
const mean = calculateMean(numbers);
const variance = numbers.reduce((sum, num) => sum + Math.pow(num - mean, 2), 0) / numbers.length;
return Math.sqrt(variance);
}
Define tools array with multiple functions
Create a tools array that includes both functions:For more information about describing functions as tools, see describe functions as tools section
tools = [
{
"type": "function",
"function": {
"name": "calculate_mean",
"description": "Calculate the mean (average) of a list of numbers. Use this when asked for the average or mean of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers to calculate the mean for"
}
},
"required": ["numbers"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate_standard_deviation",
"description": "Calculate the population standard deviation (N) of a list of numbers. Use this when asked for standard deviation, variance, or statistical spread of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers to calculate the standard deviation for"
}
},
"required": ["numbers"]
}
}
}
]
const tools = [
{
type: "function",
function: {
name: "calculate_mean",
description: "Calculate the mean (average) of a list of numbers. Use this when asked for the average or mean of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers to calculate the mean for"
}
},
required: ["numbers"]
}
}
},
{
type: "function",
function: {
name: "calculate_standard_deviation",
description: "Calculate the population standard deviation (N) of a list of numbers. Use this when asked for standard deviation, variance, or statistical spread of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers to calculate the standard deviation for"
}
},
required: ["numbers"]
}
}
}
];
Process multiple tool calls
When the model identifies a need to call multiple tools based on the user’s input, you’ll receive multiple tool calls in the response. The processing differs between streaming and non-streaming approaches.- Streaming
- Non-streaming
Streaming
When using streaming with multiple tool calls, the tool calls come back in chunks inside of the delta object of the choices array. To process multiple tool calls:- Stream and collect multiple tool calls from the response chunks
- Reconstruct the assistant’s response with all tool calls and append it to the message history
- Execute all functions and append tool results to the message history
- Get the final response from the model and append it to the message history
Step 1: Stream and collect multiple tool calls from the response chunks
First, stream and collect multiple tool calls from the response chunks. This is where multiple tool calls differ from single tool calls:streaming_content = ""
function_calls = []
for chunk in response:
choice = chunk.choices[0]
if choice.delta:
# Collect tool calls as they stream in
if choice.delta.tool_calls:
for tool_call in choice.delta.tool_calls:
if tool_call.id:
# Start a new function call
function_calls.append({
"name": "",
"arguments": "",
"call_id": tool_call.id
})
if tool_call.function:
# Append to the most recent function call
if function_calls:
function_calls[-1]["name"] += tool_call.function.name or ""
function_calls[-1]["arguments"] += tool_call.function.arguments or ""
# Collect regular content (for cases where no tools are called)
elif choice.delta.content:
streaming_content += choice.delta.content
# Check if streaming is complete
if choice.finish_reason:
if choice.finish_reason == "stop":
# No tools were called, just regular response
messages.append({"role": "assistant", "content": streaming_content})
break
elif choice.finish_reason == "tool_calls":
print(f"Tool calls collected: {len(function_calls)}")
break
let streamingContent = "";
const functionCalls = [];
for await (const chunk of response) {
const choice = chunk.choices[0];
if (choice.delta) {
// Collect tool calls as they stream in
if (choice.delta.tool_calls) {
for (const toolCall of choice.delta.tool_calls) {
if (toolCall.id) {
// Start a new function call
functionCalls.push({
name: "",
arguments: "",
call_id: toolCall.id
});
}
if (toolCall.function) {
// Append to the most recent function call
if (functionCalls.length > 0) {
functionCalls[functionCalls.length - 1].name += toolCall.function.name || "";
functionCalls[functionCalls.length - 1].arguments += toolCall.function.arguments || "";
}
}
}
}
// Collect regular content (for cases where no tools are called)
else if (choice.delta.content) {
streamingContent += choice.delta.content;
}
}
// Check if streaming is complete
if (choice.finish_reason) {
if (choice.finish_reason === "stop") {
// No tools were called, just regular response
messages.push({ role: "assistant", content: streamingContent });
break;
} else if (choice.finish_reason === "tool_calls") {
console.log(`Tool calls collected: ${functionCalls.length}`);
break;
}
}
}
Step 2: Reconstruct and append assistant message with multiple tool calls
Reconstruct the assistant message with all collected tool calls:# Step 3: Reconstruct and append assistant message with multiple tool calls
tool_calls_for_message = []
for func_call in function_calls:
tool_calls_for_message.append({
"id": func_call["call_id"],
"type": "function",
"function": {
"name": func_call["name"],
"arguments": func_call["arguments"]
}
})
assistant_message = {
"role": "assistant",
"content": None,
"tool_calls": tool_calls_for_message
}
messages.append(assistant_message)
// Step 3: Reconstruct and append assistant message with multiple tool calls
const toolCallsForMessage = [];
for (const funcCall of functionCalls) {
toolCallsForMessage.push({
id: funcCall.call_id,
type: "function",
function: {
name: funcCall.name,
arguments: funcCall.arguments
}
});
}
const assistantMessage = {
role: "assistant",
content: null,
tool_calls: toolCallsForMessage
};
messages.push(assistantMessage);
Step 3: Execute all functions and append results
Execute each function and append the results to the conversation history:# Step 4: Execute all functions and append results
for function_call in function_calls:
function_name = function_call["name"]
print(f"Processing tool call: {function_name}")
if function_name == "calculate_mean":
try:
arguments_dict = json.loads(function_call["arguments"])
function_response = calculate_mean(arguments_dict["numbers"])
except Exception as e:
function_response = f"Error: {str(e)}"
elif function_name == "calculate_standard_deviation":
try:
arguments_dict = json.loads(function_call["arguments"])
function_response = calculate_standard_deviation(arguments_dict["numbers"])
except Exception as e:
function_response = f"Error: {str(e)}"
else:
function_response = f"Unknown function: {function_name}"
print(f"Function response: {function_response}")
# Append tool result to conversation history
messages.append({
"role": "tool",
"content": str(function_response),
"tool_call_id": function_call["call_id"],
"name": function_name,
})
// Step 4: Execute all functions and append results
for (const functionCall of functionCalls) {
const functionName = functionCall.name;
let functionResponse;
console.log(`Processing tool call: ${functionName}`);
if (functionName === "calculate_mean") {
try {
const argumentsDict = JSON.parse(functionCall.arguments);
functionResponse = calculateMean(argumentsDict.numbers);
} catch (error) {
functionResponse = `Error: ${error.message}`;
}
} else if (functionName === "calculate_standard_deviation") {
try {
const argumentsDict = JSON.parse(functionCall.arguments);
functionResponse = calculateStandardDeviation(argumentsDict.numbers);
} catch (error) {
functionResponse = `Error: ${error.message}`;
}
} else {
functionResponse = `Unknown function: ${functionName}`;
}
console.log(`Function response: ${functionResponse}`);
// Append tool result to conversation history
messages.push({
role: "tool",
content: functionResponse.toString(),
tool_call_id: functionCall.call_id,
name: functionName,
});
}
Step 4: Get the final response
Get the final response from the model after all tool calls are processed:# Step 5: Get the final response
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=True
)
final_content = ""
for chunk in final_response:
choice = chunk.choices[0]
if choice.delta and choice.delta.content:
final_content += choice.delta.content
print(f"Final response: {final_content}")
// Step 5: Get the final response
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: true
});
let finalContent = "";
for await (const chunk of finalResponse) {
const choice = chunk.choices[0];
if (choice.delta && choice.delta.content) {
finalContent += choice.delta.content;
}
}
console.log(`Final response: ${finalContent}`);
Step 5: Append final response to conversation history
Append the final response to maintain complete conversation history:# Step 5: Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
// Step 5: Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
Complete streaming code example
import json
import math
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
def calculate_mean(numbers: list) -> float:
if not numbers:
raise ValueError("Cannot calculate mean of an empty list")
return sum(numbers) / len(numbers)
def calculate_standard_deviation(numbers: list) -> float:
if len(numbers) < 2:
raise ValueError("Cannot calculate standard deviation with fewer than 2 numbers")
mean = calculate_mean(numbers)
variance = sum((x - mean) ** 2 for x in numbers) / len(numbers)
return math.sqrt(variance)
tools = [
{
"type": "function",
"function": {
"name": "calculate_mean",
"description": "Calculate the mean (average) of a list of numbers. Use this when asked for the average or mean of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers to calculate the mean for"
}
},
"required": ["numbers"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate_standard_deviation",
"description": "Calculate the standard deviation of a list of numbers. Use this when asked for standard deviation, variance, or statistical spread of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers to calculate the standard deviation for"
}
},
"required": ["numbers"]
}
}
}
]
messages = [{"role": "user", "content": "What are the mean and standard deviation of [10, 20, 30, 40, 50]?"}]
# Step 1: Initial request with tools
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto",
stream=True
)
# Step 2: Process streaming response to collect multiple tool calls
streaming_content = ""
function_calls = []
for chunk in response:
choice = chunk.choices[0]
if choice.delta:
# Collect tool calls as they stream in
if choice.delta.tool_calls:
for tool_call in choice.delta.tool_calls:
if tool_call.id:
# Start a new function call
function_calls.append({
"name": "",
"arguments": "",
"call_id": tool_call.id
})
if tool_call.function:
# Append to the most recent function call
if function_calls:
function_calls[-1]["name"] += tool_call.function.name or ""
function_calls[-1]["arguments"] += tool_call.function.arguments or ""
# Collect regular content (for cases where no tools are called)
elif choice.delta.content:
streaming_content += choice.delta.content
# Check if streaming is complete
if choice.finish_reason:
if choice.finish_reason == "stop":
# No tools were called, just regular response
print(f"Response: {streaming_content}")
messages.append({"role": "assistant", "content": streaming_content})
break
elif choice.finish_reason == "tool_calls":
print(f"Tool calls collected: {len(function_calls)}")
break
# Step 3: Reconstruct and append assistant message with multiple tool calls
tool_calls_for_message = []
for func_call in function_calls:
tool_calls_for_message.append({
"id": func_call["call_id"],
"type": "function",
"function": {
"name": func_call["name"],
"arguments": func_call["arguments"]
}
})
assistant_message = {
"role": "assistant",
"content": None,
"tool_calls": tool_calls_for_message
}
messages.append(assistant_message)
# Step 4: Execute all functions and append results
for function_call in function_calls:
function_name = function_call["name"]
print(f"Processing tool call: {function_name}")
if function_name == "calculate_mean":
try:
arguments_dict = json.loads(function_call["arguments"])
function_response = calculate_mean(arguments_dict["numbers"])
except Exception as e:
function_response = f"Error: {str(e)}"
elif function_name == "calculate_standard_deviation":
try:
arguments_dict = json.loads(function_call["arguments"])
function_response = calculate_standard_deviation(arguments_dict["numbers"])
except Exception as e:
function_response = f"Error: {str(e)}"
else:
function_response = f"Unknown function: {function_name}"
print(f"Function response: {function_response}")
# Append tool result to conversation history
messages.append({
"role": "tool",
"content": str(function_response),
"tool_call_id": function_call["call_id"],
"name": function_name,
})
# Step 5: Get the final response
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=True
)
final_content = ""
for chunk in final_response:
choice = chunk.choices[0]
if choice.delta and choice.delta.content:
final_content += choice.delta.content
print(choice.delta.content, end="", flush=True)
print(f"\nFinal response: {final_content}")
# Step 6: Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
function calculateMean(numbers) {
if (numbers.length === 0) {
throw new Error("Cannot calculate mean of an empty array");
}
return numbers.reduce((sum, num) => sum + num, 0) / numbers.length;
}
function calculateStandardDeviation(numbers) {
if (numbers.length < 2) {
throw new Error("Cannot calculate standard deviation with fewer than 2 numbers");
}
const mean = calculateMean(numbers);
const variance = numbers.reduce((sum, num) => sum + Math.pow(num - mean, 2), 0) / numbers.length;
return Math.sqrt(variance);
}
const tools = [
{
type: "function",
function: {
name: "calculate_mean",
description: "Calculate the mean (average) of a list of numbers. Use this when asked for the average or mean of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers to calculate the mean for"
}
},
required: ["numbers"]
}
}
},
{
type: "function",
function: {
name: "calculate_standard_deviation",
description: "Calculate the standard deviation of a list of numbers. Use this when asked for standard deviation, variance, or statistical spread of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers to calculate the standard deviation for"
}
},
required: ["numbers"]
}
}
}
];
async function main() {
let messages = [{ role: "user", content: "What are the mean and standard deviation of [10, 20, 30, 40, 50]?" }];
// Step 1: Initial request with tools
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto",
stream: true
});
// Step 2: Process streaming response to collect multiple tool calls
let streamingContent = "";
const functionCalls = [];
for await (const chunk of response) {
const choice = chunk.choices[0];
if (choice.delta) {
// Collect tool calls as they stream in
if (choice.delta.tool_calls) {
for (const toolCall of choice.delta.tool_calls) {
if (toolCall.id) {
// Start a new function call
functionCalls.push({
name: "",
arguments: "",
call_id: toolCall.id
});
}
if (toolCall.function) {
// Append to the most recent function call
if (functionCalls.length > 0) {
functionCalls[functionCalls.length - 1].name += toolCall.function.name || "";
functionCalls[functionCalls.length - 1].arguments += toolCall.function.arguments || "";
}
}
}
}
// Collect regular content (for cases where no tools are called)
else if (choice.delta.content) {
streamingContent += choice.delta.content;
}
}
// Check if streaming is complete
if (choice.finish_reason) {
if (choice.finish_reason === "stop") {
// No tools were called, just regular response
console.log(`Response: ${streamingContent}`);
messages.push({ role: "assistant", content: streamingContent });
break;
} else if (choice.finish_reason === "tool_calls") {
console.log(`Tool calls collected: ${functionCalls.length}`);
break;
}
}
}
// Step 3: Reconstruct and append assistant message with multiple tool calls
const toolCallsForMessage = [];
for (const funcCall of functionCalls) {
toolCallsForMessage.push({
id: funcCall.call_id,
type: "function",
function: {
name: funcCall.name,
arguments: funcCall.arguments
}
});
}
const assistantMessage = {
role: "assistant",
content: null,
tool_calls: toolCallsForMessage
};
messages.push(assistantMessage);
// Step 4: Execute all functions and append results
for (const functionCall of functionCalls) {
const functionName = functionCall.name;
console.log(`Processing tool call: ${functionName}`);
let functionResponse;
if (functionName === "calculate_mean") {
try {
const argumentsDict = JSON.parse(functionCall.arguments);
functionResponse = calculateMean(argumentsDict.numbers);
} catch (error) {
functionResponse = `Error: ${error.message}`;
}
} else if (functionName === "calculate_standard_deviation") {
try {
const argumentsDict = JSON.parse(functionCall.arguments);
functionResponse = calculateStandardDeviation(argumentsDict.numbers);
} catch (error) {
functionResponse = `Error: ${error.message}`;
}
} else {
functionResponse = `Unknown function: ${functionName}`;
}
console.log(`Function response: ${functionResponse}`);
// Append tool result to conversation history
messages.push({
role: "tool",
content: functionResponse.toString(),
tool_call_id: functionCall.call_id,
name: functionName,
});
}
// Step 5: Get the final response
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: true
});
let finalContent = "";
for await (const chunk of finalResponse) {
const choice = chunk.choices[0];
if (choice.delta && choice.delta.content) {
finalContent += choice.delta.content;
process.stdout.write(choice.delta.content);
}
}
console.log(`\nFinal response: ${finalContent}`);
// Step 6: Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
}
main();
Non-streaming
The processing logic for multiple tool calls is identical to single tool calls - the same code handles both scenarios. The only difference is that the loop for checking the response for tool calling processes multiple tool calls instead of one.Step 1: Get and append the assistant’s response
First, get the assistant’s response message and append it to your conversation history. This is crucial for maintaining context.response_message = response.choices[0].message
messages.append(response_message) # Always append the assistant's response
tool_calls = response_message.tool_calls
const responseMessage = response.choices[0].message;
messages.push(responseMessage); // Always append the assistant's response
const toolCalls = responseMessage.tool_calls;
Step 2: Execute functions and append tool results
Next, check for tool calls and execute the corresponding functions with the arguments provided by the model. Then, append the tool results to the conversation history.tool_calls = response_message.tool_calls
if tool_calls:
for tool_call in tool_calls:
tool_call_id = tool_call.id
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
# Execute the appropriate function based on the tool call
if function_name == "calculate_mean":
function_response = calculate_mean(function_args["numbers"])
elif function_name == "calculate_standard_deviation":
function_response = calculate_standard_deviation(function_args["numbers"])
else:
function_response = f"Unknown function: {function_name}"
# Append tool result to conversation history
messages.append({
"role": "tool",
"tool_call_id": tool_call_id,
"name": function_name,
"content": str(function_response),
})
else:
# No tool calls, just use the original response
print(f"Response: {response_message.content}")
const toolCalls = responseMessage.tool_calls;
if (toolCalls && toolCalls.length > 0) {
for (const toolCall of toolCalls) {
const toolCallId = toolCall.id;
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
// Execute the appropriate function based on the tool call
let functionResponse;
if (functionName === "calculate_mean") {
functionResponse = calculateMean(functionArgs.numbers);
} else if (functionName === "calculate_standard_deviation") {
functionResponse = calculateStandardDeviation(functionArgs.numbers);
} else {
functionResponse = `Unknown function: ${functionName}`;
}
// Append tool result to conversation history
messages.push({
role: "tool",
tool_call_id: toolCallId,
name: functionName,
content: functionResponse.toString(),
});
}
} else {
// No tool calls, just use the original response
console.log(`Response: ${responseMessage.content}`);
}
Step 3: Get and append the final response
After appending tool results, get the final response from the model and append it to maintain complete conversation history: final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=False
)
final_content = final_response.choices[0].message.content
print(f"Final response: {final_content}")
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: false
});
const finalContent = finalResponse.choices[0].message.content;
console.log(`Final response: ${finalContent}`);
Step 4: Append final response to conversation history
Append the final response to maintain complete conversation history: # Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
// Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
Complete non-streaming code example
import json
import math
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
def calculate_mean(numbers: list) -> float:
if not numbers:
raise ValueError("Cannot calculate mean of an empty list")
return sum(numbers) / len(numbers)
def calculate_standard_deviation(numbers: list) -> float:
if len(numbers) < 2:
raise ValueError("Cannot calculate standard deviation with fewer than 2 numbers")
mean = calculate_mean(numbers)
variance = sum((x - mean) ** 2 for x in numbers) / len(numbers)
return math.sqrt(variance)
tools = [
{
"type": "function",
"function": {
"name": "calculate_mean",
"description": "Calculate the mean (average) of a list of numbers. Use this when asked for the average or mean of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers to calculate the mean for"
}
},
"required": ["numbers"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate_standard_deviation",
"description": "Calculate the standard deviation of a list of numbers. Use this when asked for standard deviation, variance, or statistical spread of numbers.",
"parameters": {
"type": "object",
"properties": {
"numbers": {
"type": "array",
"items": {"type": "number"},
"description": "List of numbers to calculate the standard deviation for"
}
},
"required": ["numbers"]
}
}
}
]
messages = [{"role": "user", "content": "What are the mean and standard deviation of [10, 20, 30, 40, 50]?"}]
# Step 1: Initial request with tools
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto",
stream=False
)
# Step 2: Get and append assistant response
response_message = response.choices[0].message
messages.append(response_message)
tool_calls = response_message.tool_calls
# Step 3: Process tool calls if any
if tool_calls:
for tool_call in tool_calls:
tool_call_id = tool_call.id
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
# Execute the appropriate function based on the tool call
if function_name == "calculate_mean":
function_response = calculate_mean(function_args["numbers"])
elif function_name == "calculate_standard_deviation":
function_response = calculate_standard_deviation(function_args["numbers"])
else:
function_response = f"Unknown function: {function_name}"
# Append tool result to conversation history
messages.append({
"role": "tool",
"tool_call_id": tool_call_id,
"name": function_name,
"content": str(function_response),
})
# Step 4: Get final response
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=False
)
final_content = final_response.choices[0].message.content
print(f"Final response: {final_content}")
# Step 5: Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
else:
# No tool calls, just use the original response
print(f"Response: {response_message.content}")
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
function calculateMean(numbers) {
if (numbers.length === 0) {
throw new Error("Cannot calculate mean of an empty array");
}
return numbers.reduce((sum, num) => sum + num, 0) / numbers.length;
}
function calculateStandardDeviation(numbers) {
if (numbers.length < 2) {
throw new Error("Cannot calculate standard deviation with fewer than 2 numbers");
}
const mean = calculateMean(numbers);
const variance = numbers.reduce((sum, num) => sum + Math.pow(num - mean, 2), 0) / numbers.length;
return Math.sqrt(variance);
}
const tools = [
{
type: "function",
function: {
name: "calculate_mean",
description: "Calculate the mean (average) of a list of numbers. Use this when asked for the average or mean of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers to calculate the mean for"
}
},
required: ["numbers"]
}
}
},
{
type: "function",
function: {
name: "calculate_standard_deviation",
description: "Calculate the standard deviation of a list of numbers. Use this when asked for standard deviation, variance, or statistical spread of numbers.",
parameters: {
type: "object",
properties: {
numbers: {
type: "array",
items: { type: "number" },
description: "List of numbers to calculate the standard deviation for"
}
},
required: ["numbers"]
}
}
}
];
async function main() {
let messages = [{ role: "user", content: "What are the mean and standard deviation of [10, 20, 30, 40, 50]?" }];
// Step 1: Initial request with tools
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto",
stream: false
});
// Step 2: Get and append assistant response
const responseMessage = response.choices[0].message;
messages.push(responseMessage);
const toolCalls = responseMessage.tool_calls;
// Step 3: Process tool calls if any
if (toolCalls && toolCalls.length > 0) {
for (const toolCall of toolCalls) {
const toolCallId = toolCall.id;
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
// Execute the appropriate function based on the tool call
let functionResponse;
if (functionName === "calculate_mean") {
functionResponse = calculateMean(functionArgs.numbers);
} else if (functionName === "calculate_standard_deviation") {
functionResponse = calculateStandardDeviation(functionArgs.numbers);
} else {
functionResponse = `Unknown function: ${functionName}`;
}
// Append tool result to conversation history
messages.push({
role: "tool",
tool_call_id: toolCallId,
name: functionName,
content: functionResponse.toString(),
});
}
// Step 4: Get final response
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: false
});
const finalContent = finalResponse.choices[0].message.content;
console.log(`Final response: ${finalContent}`);
// Step 5: Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
} else {
// No tool calls, just use the original response
console.log(`Response: ${responseMessage.content}`);
}
}
main();
When processing multiple tool calls, ensure that:
- All tool calls are executed before getting the final response
- Each tool result is properly appended to the conversation history
- Error handling is implemented for each tool call
- The final response includes all tool results for context
Example: External API call
The following example covers a common use case for tool calling: calling an external API. The code uses a publicly available dictionary API to return information about an English word’s phonetic pronunciation. This example is using non-streaming; for streaming, refer to the multiple tool calls streaming example to adjust the code.Define function calling an API
First, define the function in your code. The examples below take in a word, call the dictionary API, and return the phonetic pronunciation of the word as a JSON-formatted string.import requests
import json
def get_word_pronunciation(word):
url = f"https://api.dictionaryapi.dev/api/v2/entries/en/{word}"
response = requests.get(url)
if response.status_code == 200:
return json.dumps(response.json()[0]['phonetics'])
else:
return f"Failed to retrieve word pronunciation. Status code: {response.status_code}"
async function getWordPronunciation(word) {
const url = `https://api.dictionaryapi.dev/api/v2/entries/en/${word}`;
try {
const response = await fetch(url);
if (response.ok) {
const data = await response.json();
return JSON.stringify(data[0]['phonetics']);
} else {
return `Failed to retrieve word pronunciation. Status code: ${response.status}`;
}
} catch (error) {
return `Error fetching word pronunciation: ${error.message}`;
}
}
Define tools array
Next, define a tools array that describes the tool with a JSON schema.tools = [
{
"type": "function",
"function": {
"name": "get_word_pronunciation",
"description": "A function that will return JSON containing the phonetic pronunciation of an English word",
"parameters": {
"type": "object",
"properties": {
"word": {
"type": "string",
"description": "The word to get the phonetic pronunciation for",
}
},
"required": ["word"],
},
},
}
]
const tools = [
{
type: "function",
function: {
name: "get_word_pronunciation",
description: "A function that will return JSON containing the phonetic pronunciation of an English word",
parameters: {
type: "object",
properties: {
word: {
type: "string",
description: "The word to get the phonetic pronunciation for"
}
},
required: ["word"]
}
}
}
];
Pass the tools to the model
Call thechat.chat method with the tools parameter set to the tools array and tool_choice set to auto.
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
messages = [{"role": "user", "content": "what is the phonetic pronunciation of the word 'epitome' in English?"}]
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto",
stream=False
)
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
let messages = [{role: "user", content: "what is the phonetic pronunciation of the word 'epitome' in English?"}];
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto",
stream: false
});
Check response for tool calling
Loop through thetool_calls array to check for the invocation of the tool. Then, call the tool’s function with the arguments the model provided.
response_message = response.choices[0].message
messages.append(response_message)
tool_calls = response_message.tool_calls
if tool_calls:
tool_call = tool_calls[0]
tool_call_id = tool_call.id
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "get_word_pronunciation":
function_response = get_word_pronunciation(function_args["word"])
const responseMessage = response.choices[0].message;
messages.push(responseMessage); // Append assistant response
const toolCalls = responseMessage.tool_calls;
if (toolCalls && toolCalls.length > 0) {
const toolCall = toolCalls[0];
const toolCallId = toolCall.id;
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
if (functionName === "get_word_pronunciation") {
const functionResponse = await getWordPronunciation(functionArgs.word);
}
}
Append the tool result to the conversation history
Finally, pass the result back to the model by appending it to the messages array, and get the final response.messages.append({
"role": "tool",
"tool_call_id": tool_call_id,
"name": function_name,
"content": function_response,
})
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=False
)
final_content = final_response.choices[0].message.content
# Append final response to conversation history
messages.append({
"role": "assistant",
"content": final_content
})
print(f"Final response: {final_content}")
# Final response: The phonetic pronunciation of the word "epitome" in English is /əˈpɪt.ə.mi/...
messages.push({
role: "tool",
tool_call_id: toolCallId,
name: functionName,
content: functionResponse,
});
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: false
});
const finalContent = finalResponse.choices[0].message.content;
// Append final response to conversation history
messages.push({
role: "assistant",
content: finalContent
});
console.log(`Final response: ${finalContent}`);
// Final response: The phonetic pronunciation of the word "epitome" in English is /əˈpɪt.ə.mi/...
Complete external API call example
import requests
import json
from writerai import Writer
# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()
def get_word_pronunciation(word):
url = f"https://api.dictionaryapi.dev/api/v2/entries/en/{word}"
try:
response = requests.get(url)
if response.status_code == 200:
return json.dumps(response.json()[0]['phonetics'])
else:
return f"Failed to retrieve word pronunciation. Status code: {response.status_code}"
except Exception as e:
return f"Error fetching word pronunciation: {str(e)}"
tools = [
{
"type": "function",
"function": {
"name": "get_word_pronunciation",
"description": "A function that will return JSON containing the phonetic pronunciation of an English word",
"parameters": {
"type": "object",
"properties": {
"word": {
"type": "string",
"description": "The word to get the phonetic pronunciation for",
}
},
"required": ["word"],
},
},
}
]
messages = [{"role": "user", "content": "what is the phonetic pronunciation of the word 'epitome' in English?"}]
# Step 1: Initial request
response = client.chat.chat(
model="palmyra-x5",
messages=messages,
tools=tools,
tool_choice="auto",
stream=False
)
# Step 2: Append assistant response
response_message = response.choices[0].message
messages.append(response_message)
tool_calls = response_message.tool_calls
# Step 3: Process tool calls
if tool_calls:
for tool_call in tool_calls:
tool_call_id = tool_call.id
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "get_word_pronunciation":
function_response = get_word_pronunciation(function_args["word"])
# Append tool result
messages.append({
"role": "tool",
"tool_call_id": tool_call_id,
"name": function_name,
"content": function_response,
})
# Step 4: Get final response
final_response = client.chat.chat(
model="palmyra-x5",
messages=messages,
stream=False
)
final_content = final_response.choices[0].message.content
# Step 5: Append final response
messages.append({
"role": "assistant",
"content": final_content
})
print(f"Final response: {final_content}")
# The conversation history is now complete and ready for additional turns
import { Writer } from "writer-sdk";
// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();
async function getWordPronunciation(word) {
const url = `https://api.dictionaryapi.dev/api/v2/entries/en/${word}`;
try {
const response = await fetch(url);
if (response.ok) {
const data = await response.json();
return JSON.stringify(data[0]['phonetics']);
} else {
return `Failed to retrieve word pronunciation. Status code: ${response.status}`;
}
} catch (error) {
return `Error fetching word pronunciation: ${error.message}`;
}
}
const tools = [
{
type: "function",
function: {
name: "get_word_pronunciation",
description: "A function that will return JSON containing the phonetic pronunciation of an English word",
parameters: {
type: "object",
properties: {
word: {
type: "string",
description: "The word to get the phonetic pronunciation for"
}
},
required: ["word"]
}
}
}
];
async function main() {
let messages = [{
role: "user",
content: "what is the phonetic pronunciation of the word 'epitome' in English?"
}];
// Step 1: Initial request
const response = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
tools: tools,
tool_choice: "auto",
stream: false
});
// Step 2: Append assistant response
const responseMessage = response.choices[0].message;
messages.push(responseMessage);
const toolCalls = responseMessage.tool_calls;
// Step 3: Process tool calls
if (toolCalls && toolCalls.length > 0) {
for (const toolCall of toolCalls) {
const toolCallId = toolCall.id;
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
if (functionName === "get_word_pronunciation") {
const functionResponse = await getWordPronunciation(functionArgs.word);
// Append tool result
messages.push({
role: "tool",
tool_call_id: toolCallId,
name: functionName,
content: functionResponse,
});
}
}
// Step 4: Get final response
const finalResponse = await client.chat.chat({
model: "palmyra-x5",
messages: messages,
stream: false
});
const finalContent = finalResponse.choices[0].message.content;
// Step 5: Append final response
messages.push({
role: "assistant",
content: finalContent
});
console.log(`Final response: ${finalContent}`);
}
}
main();