OpenAI compatibility

Use any OpenAI SDK or tool by changing the base URL and the key.

Deference accepts OpenAI Chat Completions, legacy text Completions, Responses and Embeddings requests. Set the base URL and key, and keep the rest of your code.

from openai import OpenAI

client = OpenAI(
    base_url="https://deference.si/v1",
    api_key="sk-df-...",
)

Supported endpoints

Endpoint	Notes
`POST /v1/chat/completions`	Streaming, tools, structured outputs and images where the model supports them
`POST /v1/completions`	Legacy text completions, including streaming. Requires `prompt` and a compatible model
`POST /v1/responses`	Stateless: send the whole conversation in `input` each time
`POST /v1/embeddings`	Embedding models only
`POST /v1/images`	Deference's image endpoint. The OpenAI SDK's `images.generate` calls a different path, so call this one directly
`GET /v1/models`	Public, no key needed
`GET /v1/models/{id}`	One model
`GET /v1/key`	The calling key's limit, spend and available credit

For legacy text completions, send prompt as a string or array and use max_tokens to cap output. A model that supports chat does not necessarily support legacy text completions.

How requests are handled

Request fields pass through to the provider, which ignores parameters a model does not support. Chat Completions and legacy Completions that omit the output cap and use no server tools can have that cap lowered so the hold fits available credit. A cap you set is kept.
Model ids are author/name, for instance anthropic/claude-sonnet-5.5. Choose an endpoint that matches the model's output: chat or Responses for text, Embeddings for embeddings, and Images for image generation. See Choose a model.
Each request runs on the one model it names, at that model's price. models, fallbacks, route and provider return 400 unsupported_parameter, and so does a plugins entry outside the free list — response-healing, file-parser on the cloudflare-ai or native engine, and context-compression with enabled: false. On Images, an n that is not a whole number from 1 to 10 returns 400 invalid_parameter.
On chat completions, legacy completions, embeddings and images, user is replaced with a stable account tag that cannot be reversed. A model id that is not the catalog id, including a trailing [1m], is replaced with the catalog id. Otherwise Responses keeps the body.
Every response carries x-request-id, errors included. A successful non-streamed response also carries x-deference-cost, the cost in micro-dollars. It is provisional when the body has a generation id. On Images, which never have one, a reported cost is final. A success that reports no cost stays unresolved, and the header is 0.

Files and server tools

Models that list file input read files themselves. For other models, Chat Completions parses files with the free cloudflare-ai parser. Responses and Messages return 400 unsupported_parameter unless the request sets "plugins": [{ "id": "file-parser", "pdf": { "engine": "cloudflare-ai" } }].
Every file part, inline or given as a URL, and every image or video given as a URL is held at the larger of the text estimate and the model's context window, or 1,000,000 tokens when that window is unknown.
Tools the provider runs, such as web search, are held step by step. max_tool_calls lowers the number of steps, up to 30 — but not when stop_server_tools_when is set, which keeps the allowance at 30.
Search through X (x_search) and audio input (input_audio) return 400 unsupported_parameter. Models that bill per search, such as perplexity/sonar-deep-research, return 400 unsupported_model.

Streaming

A chat stream ends with a chunk that carries usage. With stream_options.include_usage, that chunk has an empty choices array, as in OpenAI's API. Without it, the chunk keeps one choice with an empty delta.

Not available

Audio, files, batches, assistants, fine-tuning and moderation are not part of Deference, and neither are OpenAI's image paths (/v1/images/generations, edits and variations). Calls to them return 404.

Errors

Errors use OpenAI's envelope: error.message, error.type, error.code and error.param. See Errors.