---
title: "Vision and files"
subtitle: "Send images and documents to chat models as URLs or base64 data."
---

Chat models that read images or documents take them as content parts in a user message. You write the parts in OpenAI's format, and the gateway translates them into each provider's format, so the same request works for OpenAI, Anthropic and Gemini models. Every call is logged as a `chat` row with its cost.

**Pick a model:** filter by input type `image` or `file` on the [model catalog](https://www.respan.ai/models), or read [Models](/docs/ai-gateway/models/overview).

## URL or base64

Every media part carries its content in one of two forms:

| Form | Example | Use when |
| --- | --- | --- |
| Public URL | `https://example.com/receipt.png` | The file is already hosted and reachable from the internet. The provider fetches it. |
| Base64 data URL | `data:image/png;base64,iVBORw0...` | The file is local or private. The bytes travel in the request body. |

Which form each part takes:

| Part | URL | Base64 data URL |
| --- | --- | --- |
| `image_url` (images) | Yes | Yes |
| `file` (PDF, plain text) | No | Yes, in `file_data` |
| `input_audio` ([audio](/docs/ai-gateway/modalities/audio#audio-in-chat)) | No | Yes, raw base64 in `data` |

Base64 makes the request body about a third larger than the file, and every body counts against the gateway's request size budget, so prefer URLs for large images. To turn a local file into a data URL:

```python
import base64

with open("receipt.png", "rb") as f:
    data = base64.b64encode(f.read()).decode()

url = f"data:image/png;base64,{data}"
```

## Images

Add `image_url` parts to a user message. Send several images by adding more parts to the same message.

<Tabs>
<Tab title="cURL">
```bash
curl https://gateway.respan.ai/v1/chat/completions \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What is the total on this receipt?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/receipt.png"}}
      ]
    }]
  }'
```
</Tab>
<Tab title="Python">
```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.respan.ai/v1",
    api_key=os.environ["RESPAN_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What is the total on this receipt?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/receipt.png"}},
        ],
    }],
)
print(response.choices[0].message.content)
```
</Tab>
<Tab title="TypeScript">
```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.respan.ai/v1",
  apiKey: process.env.RESPAN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gpt-5.5",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "What is the total on this receipt?" },
      { type: "image_url", image_url: { url: "https://example.com/receipt.png" } },
    ],
  }],
});
console.log(response.choices[0].message.content);
```
</Tab>
</Tabs>

## Files

Add a `file` part to send a PDF (`application/pdf`) or plain text (`text/plain`) as a base64 data URL.

```python
with open("contract.pdf", "rb") as f:
    data = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Summarize this contract in three bullets."},
            {"type": "file", "file": {
                "filename": "contract.pdf",
                "file_data": f"data:application/pdf;base64,{data}",
            }},
        ],
    }],
)
```

| Field | Type | Description |
| --- | --- | --- |
| `file.file_data` | string | The document as a data URL: `data:application/pdf;base64,...` or `data:text/plain;base64,...`. |
| `file.filename` | string | Optional file name, passed to providers that take one. |

The gateway translates `file_data`. An uploaded file id (`file.file_id`) works only on the provider that issued it; it is not translated to other providers.

## Provider support

| Provider | Images | Files |
| --- | --- | --- |
| OpenAI | Yes, natively | Yes, natively |
| Anthropic | Yes, mapped to image blocks | Yes, mapped to document blocks |
| Gemini | Yes, mapped to inline data | Yes, mapped to inline data in Gemini's own format |

A given model may still not read images or files. Check its input types (next section).

{/* REVIEW: confirm gateway-next calls Gemini chat in native generateContent (file parts are refused on Gemini's OpenAI-compatible endpoint). */}

## Model input types

Every model in the catalog lists what it accepts in `input_modalities` and what it returns in `output_modalities`:

| Field | Values |
| --- | --- |
| `input_modalities` | `text`, `image`, `audio`, `video`, `file` |
| `output_modalities` | `text`, `image`, `audio`, `video`, `embeddings`, `speech`, `transcription`, `rerank`, `decisions`, `realtime` |

A model that reads images lists `image` in `input_modalities`; one that reads PDFs lists `file`. Filter on these fields on the [model catalog](https://www.respan.ai/models), or in the models list API ([List models](/docs/apis/models/list-models)).

{/* REVIEW: confirm which public models API returns input_modalities/output_modalities and accepts them as filters (the family list endpoint does; check /api/models/public/). */}

## Routing

When a request can go to several models, such as through a [route](/docs/ai-gateway/routing/routes-and-aliases) with [fallbacks](/docs/ai-gateway/routing/fallbacks) or [load balancing](/docs/ai-gateway/routing/load-balancing), the gateway drops the candidates whose `input_modalities` don't include `image` when the messages carry an image. The first model is always tried as you named it.

{/* REVIEW: this capability filtering comes from the old gateway; confirm gateway-next filters a route's attempts by input modality. */}

## Respan parameters

Respan parameters work as on any chat request: in the body (or `extra_body`) or the `X-Data-Respan-Params` header. See [Respan parameters](/docs/ai-gateway/customers/respan-params).

## Pricing and logs

- Images and documents are billed as part of the chat call, from the input tokens the provider reports, at the model's catalog price.
- The trace's `chat` row keeps the request's messages, including the media parts.

## Next steps

- [Structured outputs](/docs/ai-gateway/requests/chat-completions#structured-outputs): extract fields from a document as JSON.
- [Images](/docs/ai-gateway/modalities/images): generate and edit images.
- [Audio](/docs/ai-gateway/modalities/audio): send audio to chat models.
- [Text](/docs/ai-gateway/modalities/text): chat request basics.