Sending requests

Formats

POST /v1/chat/completions and POST /v1/completions, OpenAI's formats
POST /v1/messages, Anthropic's Messages format
POST /v1/responses, OpenAI's Responses format

All of them stream. Chat and text completions also take stop, seed, logprobs, n and the usual sampling settings.

Capabilities

Each model lists what has passed on real hardware, such as tools and json_schema, under capabilities in GET /v1/models.

Thinking

Models with the reasoning capability return their thinking in message.reasoning. It counts toward max_tokens and is billed as output, so leave room for the answer, or turn it off:

"chat_template_kwargs": {"enable_thinking": false}

Tools and json_schema on these models were tested with thinking off.

Prompt caching

Repeated prompt prefixes such as a long system prompt are cached automatically and kept separate for each account. Responses show the cached part as usage.prompt_tokens_details.cached_tokens, and it's billed at the cached input price.

Response headers

Header	Meaning
`X-Request-Id`	Quote this if you contact us
`X-Cost-USD`	What this request cost, fees included
`X-Token-Cost-USD`	The token part of that cost
`X-Cold-Start-Fee-USD`	The start fee, when this request paid one
`X-Idle-Fee-USD`	The idle fee, when this request paid one
`X-Prompt-Tokens`, `X-Cached-Tokens`, `X-Completion-Tokens`	Tokens counted
`X-Execution-Ms`	GPU time used
`X-Queue-Ms`	Time spent waiting for a worker

A stream reports its cost after it finishes, at GET /v1/usage/{request_id}, using the X-Request-Id header it started with.