# OpenAI compatibility

> Use any OpenAI SDK or tool by changing the base URL and the key.

Deference accepts OpenAI Chat Completions, legacy text Completions, Responses and Embeddings requests. Set the base URL and key, and keep the rest of your code.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://deference.si/v1",
    api_key="sk-df-...",
)
```

## Supported endpoints

| Endpoint                    | Notes                                                                                                            |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `POST /v1/chat/completions` | Streaming, tools, structured outputs and images where the model supports them                                    |
| `POST /v1/completions`      | Legacy text completions, including streaming. Requires `prompt` and a compatible model                           |
| `POST /v1/responses`        | Stateless: send the whole conversation in `input` each time                                                      |
| `POST /v1/embeddings`       | Embedding models only                                                                                            |
| `POST /v1/images`           | Deference's image endpoint. The OpenAI SDK's `images.generate` calls a different path, so call this one directly |
| `GET /v1/models`            | Public, no key needed                                                                                            |
| `GET /v1/models/{id}`       | One model                                                                                                        |
| `GET /v1/key`               | The calling key's limit, spend and available credit                                                              |

For legacy text completions, send `prompt` as a string or array and use `max_tokens` to cap output. A model that supports chat does not necessarily support legacy text completions.

## How requests are handled

* Request fields pass through to the provider, which ignores parameters a model does not support. Chat Completions and legacy Completions that omit the output cap and use no server tools can have that cap lowered so the hold fits available credit. A cap you set is kept.
* Model ids are `author/name`, for instance `anthropic/claude-sonnet-5.5`. Choose an endpoint that matches the model's output: chat or Responses for text, Embeddings for embeddings, and Images for image generation. See [Choose a model](https://deference.si/docs/models).
* Each request runs on the one model it names, at that model's price. `models`, `fallbacks`, `route` and `provider` return 400 `unsupported_parameter`, and so does a `plugins` entry outside the free list — `response-healing`, `file-parser` on the `cloudflare-ai` or `native` engine, and `context-compression` with `enabled: false`. On Images, an `n` that is not a whole number from 1 to 10 returns `400 invalid_parameter`.
* On chat completions, legacy completions, embeddings and images, `user` is replaced with a stable account tag that cannot be reversed. A model id that is not the catalog id, including a trailing `[1m]`, is replaced with the catalog id. Otherwise Responses keeps the body.
* Every response carries `x-request-id`, errors included. A successful non-streamed response also carries `x-deference-cost`, the cost in micro-dollars. It is provisional when the body has a generation id. On Images, which never have one, a reported cost is final. A success that reports no cost stays unresolved, and the header is 0.

## Files and server tools

* Models that list `file` input read files themselves. For other models, Chat Completions parses files with the free `cloudflare-ai` parser. Responses and Messages return 400 `unsupported_parameter` unless the request sets `"plugins": [{ "id": "file-parser", "pdf": { "engine": "cloudflare-ai" } }]`.
* Every file part, inline or given as a URL, and every image or video given as a URL is held at the larger of the text estimate and the model's context window, or 1,000,000 tokens when that window is unknown.
* Tools the provider runs, such as web search, are held step by step. `max_tool_calls` lowers the number of steps, up to 30 — but not when `stop_server_tools_when` is set, which keeps the allowance at 30.
* Search through X (`x_search`) and audio input (`input_audio`) return 400 `unsupported_parameter`. Models that bill per search, such as `perplexity/sonar-deep-research`, return 400 `unsupported_model`.

## Streaming

A chat stream ends with a chunk that carries `usage`. With `stream_options.include_usage`, that chunk has an empty `choices` array, as in OpenAI's API. Without it, the chunk keeps one choice with an empty delta.

## Not available

Audio, files, batches, assistants, fine-tuning and moderation are not part of Deference, and neither are OpenAI's image paths (`/v1/images/generations`, edits and variations). Calls to them return `404`.

## Errors

Errors use OpenAI's envelope: `error.message`, `error.type`, `error.code` and `error.param`. See [Errors](https://deference.si/docs/api-reference/errors).