Skip to main content

Overview

The chat endpoint supports structured outputs for chat completions with palmyra-x5 and palmyra-x4 models. Structured outputs allow you to specify the response schema that the model should follow. You can use structured outputs in one of two ways:
  1. Pass in a json_schema object in the response_format parameter when making a request to the chat endpoint.
  2. Use the parse method in the Python and JavaScript SDKs and pass in a Pydantic or Zod object.

Sample request and response

The sample request and response below shows how to use structured outputs with the chat endpoint and the response_format parameter. The json_schema object must be in JSON Schema format. To guarantee that the model includes all the fields in the response, include the required field in the json_schema object.
The response from the chat endpoint includes a message object with the content field set to the raw response from the model.

SDK parse methods

The Python and JavaScript SDKs provide helper parse methods for working with structured outputs. You can define a Pydantic or Zod model for the response schema, and pass it as the response_format parameter to the parse method. The result from the parse method includes a message object with the parsed field. This field’s value is an instance of the Pydantic or Zod model you passed for the response_format parameter.
If you include tool calls in your request, the parse method only works with custom functions where the tool calls are of type function. It does not work with built-in tools with other types such as web_search, vision, or translate. For those cases, you can pass a json_schema object in the response_format parameter for the chat endpoint.