This guide explains how to use another Writer model as a tool in chat sessions using the tool calling capability of the Chat completion endpoint.

With model delegation, you can use another Writer model as a tool in a chat application with a separate Palmyra model. Writer has a predefined LLM tool that allows you to delegate specific tasks to other models during a chat session.

For example, in a chat application using Palmyra-X-004, you can delegate image generation tasks to the palmyra-vision model.

This guide will help you understand how to perform model delegation using the Writer API.

You need an API key to use LLM tool delegation. Follow these steps to get an API key.

​
Overview

You can use the LLM tool to delegate specific tasks to another model when using the chat endpoint. Using tool calling, you can specify the Writer model you want to use for a given task. When the primary chat model calls the LLM tool based on the user’s input, it signals it in the chat API response.

The LLM tool works especially well with domain-specific Writer models, such as palmyra-fin, palmyra-med, or palmyra-creative.

To use the LLM tool, add it to the tools array in your chat-completion endpoint request.

The LLM tool supports Palmyra-X-004 and later and supports streaming and non-streaming responses.

​
Tool object

The LLM tool object is defined as follows:

ParameterTypeDescription
typestringThe type of tool, which is llm for LLM tool
functionobjectAn object containing the tool’s description and model
function.descriptionstring(optional) A description of what the model will be used for
function.modelstringThe ID of the Writer model to be used for this tool
"tools": [
    {
        "type": "llm",
        "function": {
            "description": "A model specialized for financial analysis and reporting",
            "model": "palmyra-fin"
        }
    }  
]

​
Usage example

Here’s an example of how to use the LLM tool in your application.

1

Create a tools array containing an LLM tool

To use the LLM tool, create a tools array that specifies the Writer model you want to use:

"tools": [
    {
        "type": "llm",
        "function": {
            "description": "A model specialized for financial risk assessment",
            "model": "palmyra-fin"
        }
    }
]
2

Send the request using chat completions

Then, add the tools array to the chat method or endpoint call along with your array of messages.

curl --location 'https://api.writer.com/v1/chat' \
    --header 'Content-Type: application/json' \
    --header 'Authorization: Bearer <token>' \
    --data '{
        "model": "palmyra-x-004",
        "temperature": 0.7,
        "messages": [
            {
                "role": "user",
                "content": "Analyze the risk factors for our Q4 investment portfolio."
            }
        ],
        "tool_choice": "auto",
        "tools": [
            {
                "type": "llm",
                "function": {
                    "description": "A model specialized for financial risk assessment",
                    "model": "palmyra-fin"
                }
            }
        ],
        "stream": true
    }'
3

Process the response

Finally, process the response from the chat endpoint. The text results will be the first option in the choices array within the response object.

The code samples below demonstrate how to process the response depending on whether streaming is enabled or not. Learn more about processing tool calling responses in the tool calling guide.

response_text = ""
for chunk in response:
    if chunk.choices[0].delta.content is not None:
        response_text += chunk.choices[0].delta.content

print(response_text)

The response contains the generated content from the specialized model you specified in the tool configuration, as well as an llm_data object that provides information about the delegated model execution:

  • prompt: The prompt that was sent to the delegated model
  • model: The model that was used to generate the response
{
    "id": "1234",
    "object": "chat.completion.chunk",
    "choices": [
        {
            "index": 0,
            "finish_reason": "stop",
            "delta": {
                "content": "Based on the Q4 investment portfolio analysis...",
                "role": "assistant",
                "tool_calls": null,
                "llm_data": {
                    "prompt": "Analyze risk factors for Q4 investment portfolio",
                    "model": "palmyra-fin"
                }
            }
        }
    ],
    "created": 1715606709,
    "model": "palmyra-x-004"
}

By following this guide, you can use specialized, fine-tuned Writer models for specific tasks within your chat applications.

Was this page helpful?

Knowledge Graph chat supportTools API