Writer AI module
This module leverages the Writer Python SDK to enable applications to interact with large language models (LLMs) in chat or text completion formats. It provides tools to manage conversation states and to dynamically interact with LLMs using both synchronous and asynchronous methods.
Getting your API key
To utilize the Writer AI module, you’ll need to configure the WRITER_API_KEY
environment variable with an API key obtained from AI Studio. Here is a detailed guide to setup up this key. You will need to select an API app under Developer tools
Once you have your API key, set it as an environment variable on your system:
You can manage your environment variables using methods that best suit your setup, such as employing tools like python-dotenv.
Furthermore, when deploying an application with writer deploy
, the WRITER_API_KEY
environment variable is automatically configured with the API key specified during the deployment process.
Chat completion with the Conversation class
The Conversation
class manages LLM communications within a chat framework, storing the conversation history and handling the interactions.
import writer as wf
import writer.ai
def handle_simple_message(state, payload):
# Update the conversation state by appending the incoming user message.
state["conversation"] += payload
# Stream the complete response from the AI model in chunks.
for chunk in state["conversation"].stream_complete():
# Append each chunk of the model's response to the ongoing conversation state.
state["conversation"] += chunk
# Initialize the application state with a new Conversation object.
initial_state = wf.init_state({
"conversation": writer.ai.Conversation(),
})
Initializing a conversation
A Conversation
can be initialized with either a system prompt or a list of previous messages. It can also accept a default configuration dictionary that sets parameters for all interactions.
# Initialize with a system prompt for a Financial Analyst specializing in balance sheets
conversation = Conversation("You assist clients with analyzing and understanding their balance sheets")
# Initialize with a history of messages related to balance sheet queries
history = [
{"role": "user", "content": "Can you explain the liabilities section?"},
{"role": "assistant", "content": "Certainly! Liabilities are legally binding obligations payable to another entity."}
]
conversation = Conversation(history)
# Initialize with a configuration suitable for financial analysis discussions
config = {'max_tokens': 200, 'temperature': 0.5}
conversation = Conversation("You provide detailed insights into balance sheet components", config=config)
Adding messages to conversation
Messages can be added to a Conversation
instance using the +
operator or the add
method.
# Using the `+` operator to add a balance sheet-related query
conversation += {"role": "user", "content": "Can you break down the assets section of the balance sheet?"}
# Using the `add` method to add a balance sheet-related query
conversation.add(role="user", content="How should I interpret the equity section?")
Completing and streaming Conversations
The complete
and stream_complete
methods facilitate interaction with the LLM based on the accumulated messages and configuration. These methods execute calls to generate responses and return them in the form of a message object, but do not alter the conversation’s messages
list, allowing you to validate or modify the output before deciding to add it to the history.
Instance-wide configuration parameters can be complemented or overriden on individual call’s level, if a config
dictionary is provided to the method:
# Overriding configuration for a specific call
response = conversation.complete(config={'max_tokens': 200, 'temperature': 0.5})
Text generation without a conversation state
These complete
and stream_complete
methods are designed for one-off text generation without the need to manage a conversation state. They return the model’s response as a string. Each function accepts a config
dictionary allowing call-specific configurations.