Initiate Async Chat Stream | PrivateGPT

Initiate an asynchronous chat completion stream.

This endpoint starts an asynchronous chat completion process that streams events. Unlike synchronous chat, this endpoint returns immediately with a message_id that can be used to observe the stream progress.

Key Features:

Asynchronous Processing: Non-blocking request handling with immediate response
Stream Observation: Use returned message_id to observe real-time events
Works exactly like synchronous chat, but in an async manner

Notes:

Optional message_id query parameter for custom stream identification
Stream events follow the same format as synchronous chat responses
Stream status can be monitored via status endpoint

Initiate an asynchronous chat completion stream. This endpoint starts an asynchronous chat completion process that streams events. Unlike synchronous chat, this endpoint returns immediately with a message_id that can be used to observe the stream progress. Key Features: * Asynchronous Processing: Non-blocking request handling with immediate response * Stream Observation: Use returned message_id to observe real-time events * Works exactly like synchronous chat, but in an async manner Notes: * Optional message_id query parameter for custom stream identification * Stream events follow the same format as synchronous chat responses * Stream status can be monitored via status endpoint

Query parameters

message_idstring or nullOptional

Optional custom identifier for the stream. If not provided, a unique ID will be generated automatically.

Request

Request body for initiating asynchronous chat completion with multi-turn conversation support.

Contains message history, tool definitions, system prompts, and response configuration. The request initiates a background process that streams events, which can be observed via the stream endpoint.

The request body defines the complete conversation context and AI behavior parameters for generating asynchronous responses.

Request body for initiating asynchronous chat completion with multi-turn conversation support. Contains message history, tool definitions, system prompts, and response configuration. The request initiates a background process that streams events, which can be observed via the stream endpoint. The request body defines the complete conversation context and AI behavior parameters for generating asynchronous responses.

modelstringRequiredDefaults to default

Model identifier or alias.

messageslist of objectsRequired

Conversation messages for the request.

max_tokensintegerRequired>=1

Maximum number of tokens to generate in the response.

systemlist of objectsOptional

System prompt input. Accepts str, list[str], System, list[System], or null. It is normalized internally to list[System].

toolslist of objects or nullOptional

Optional tool definitions.

thinkingobjectOptional

Thinking configuration.

tool_choiceobjectOptional

Tool selection policy.

output_configobjectOptional

Optional output configuration options.

cache_controlobject or nullOptional

Optional request-level cache control.

streambooleanOptionalDefaults to false

Whether to stream the response back to the client.

tool_contextlist of objects or nullOptional

Context to provide to the tools, such as documents, databases connection strings, or data relevant to tool usage.

mcp_serverslist of objectsOptional

List of MCP servers to use for tool retrieval. Each server can have its own configuration.

containerstring or nullOptional

Container identifier for reuse across requests.

response_formatobjectOptional

Deprecated response format. Use output_config.format instead.

priorityinteger or nullOptional

Priority of the request, used for prioritizing responses.

seedinteger or nullOptional

Random seed for reproducibility.

min_pdouble or nullOptional

Minimum probability threshold for token selection. Tokens with probability below this value are filtered out.

top_pdoubleOptional0-1

Nucleus sampling parameter. Only tokens with cumulative probability up to this value are considered.

temperaturedoubleOptional0-1

Controls randomness in generation. Higher values make output more random, lower values more deterministic.

top_kintegerOptional>=0

Limits token selection to the top K most likely tokens at each step.

repetition_penaltydouble or nullOptional

Penalty applied to tokens that have already appeared in the sequence to reduce repetition.

presence_penaltydouble or nullOptional

Penalty applied based on whether a token has appeared in the text, encouraging topic diversity.

frequency_penaltydouble or nullOptional

Penalty applied based on how frequently a token appears in the text, reducing repetitive content.

stop_sequenceslist of stringsOptional

Custom stop sequences that stop generation when matched.

metadataobjectOptional

Request metadata (for example, user_id).

service_tierenumOptionalDefaults to auto

Service tier preference (for example, “auto” or “standard_only”).

inference_geostring or nullOptional

Geographic region hint for inference processing.

correlation_idstring or nullOptional

Correlation ID for tracking the request across systems.

maximum_loaded_skillsinteger or nullOptional>=1

Optional cap for concurrently loaded skills in a conversation. When exceeded, the oldest loaded skill is evicted.

Response

Chat stream initiated successfully

message_idstring

Unique identifier for the initiated stream

statusenum

Initial status of the stream (typically ‘pending’)

messagestringDefaults to Request initiated successfully

Confirmation message for successful stream initiation

Errors

401

Unauthorized Error

422

Unprocessable Entity Error

$	curl -X POST /v1/messages/async \
>	-H "Content-Type: application/json" \
>	-d '{
>	"model": "default",
>	"messages": [
>	{
>	"role": "user",
>	"content": "How do you fry an egg? Choose the best method."
>	}
>	],
>	"max_tokens": 1000,
>	"system": [
>	{
>	"text": "You are a helpful cooking assistant. Provide clear, step-by-step instructions.",
>	"citations": {
>	"enabled": true
>	}
>	}
>	],
>	"tools": [
>	{
>	"name": "egg_fryer",
>	"description": "A tool to fry eggs with precise temperature control",
>	"input_schema": {
>	"properties": {
>	"temperature": {
>	"description": "Temperature in degrees Celsius",
>	"type": "number"
>	},
>	"time": {
>	"description": "Time in minutes to fry the egg",
>	"type": "number"
>	}
>	},
>	"required": [
>	"temperature",
>	"time"
>	],
>	"type": "object"
>	}
>	}
>	],
>	"thinking": {
>	"enabled": false
>	},
>	"tool_choice": {
>	"type": "auto",
>	"disable_parallel_tool_use": false
>	},
>	"stream": false,
>	"response_format": {
>	"type": "text"
>	},
>	"temperature": 0.7
>	}'

1	{
2	"message_id": "msg_async_12345",
3	"status": "pending",
4	"message": "Request initiated successfully"
5	}