Skip to main content

​
Create chat completion

import maitai

client = maitai.MaitaiAsync()

messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Say hello in one sentence."},
]

response = await client.chat.completions.create(
    messages=messages,
application="demo_app",
intent="GREETING",
    session_id="YOUR_SESSION_ID",
# model="llama3-70b-8192",  # Optional: omit to use Portal config
)

​
Maitai parameters

​
application
string
required
The Application reference name (shown in the Portal). Maitai uses this to group your traffic.
​
intent
string
required
The Intent / action type for this request (also called action_type). This is used for organization, configuration, and quality tooling.
​
session_id
string
Optional but recommended. Groups related requests into a session in the Portal. If omitted, the SDK will generate one.
​
metadata
map<string, any>
Optional metadata tags stored with the request for filtering/debugging.

​
Evaluations and corrections (optional)

​
evaluation_enabled
boolean
Enable/disable Sentinel evaluations for this request (Portal config also controls this).
​
apply_corrections
boolean
When enabled, Maitai may apply an automatic correction if evaluations find a fault.
Corrections require server-side inference.
​
safe_mode
boolean
Prioritize accuracy over latency when applying corrections.
​
callback
function(EvaluateResponse)
Optional callback invoked with the evaluation results. See Evaluate.

​
Inference routing

​
server_side_inference
boolean
Controls where inference runs.

​
Fallbacks (optional)

​
fallback_config
object
Optional fallback configuration used when the primary model fails or times out.

​
OpenAI parameters

Maitai accepts the standard OpenAI chat.completions.create parameters (for example: model, temperature, tools, tool_choice, response_format, stream, etc.). If you set them in the request, they override Portal defaults. Related guides:

​
Response fields

Maitai returns an OpenAI-compatible chat.completion response, plus a few extra fields:
​
request_id
string
The Maitai request identifier.
​
evaluate_response
object or null
Evaluation results (when evaluations are enabled).
​
correction_applied
boolean
Whether a correction was applied.
​
input_safety_score
number or null
Optional input safety score (when enabled). A number between 0 and 1 where lower values indicate a potential jailbreak attempt.
​
first_token_time
number
Time to first token (milliseconds), when available.
​
response_time
number
Total response time (milliseconds), when available.
​
fallback_reason
string or null
Present when a fallback model was used (explains why).
​
usage
object
Usage statistics for the completion request.

​
Streaming usage

​
usage
object

An optional field that will only be present when you set stream_options: {'"include_usage": true} in your request. When present, it contains a null value except for the last chunk which contains the token usage statistics for the entire request.