Point an OpenAI-compatible client at the gateway. Give it your client key and a route.
1. Put the key in an environment variable
Every example on this page reads the key from this variable. Do not paste the key into a configuration file.
bash
export HOOSHYAR_API_KEY="<your client key>"
2. Select a route
A route is a named type of task. The administrator sets the routes and the model behind each route.
Send auto as the model to let the gateway select the route. To select a route yourself, send its name as the model, or send the x-task-type header.
Call /v1/models to see the routes that your key can use.
3. Configure your client
Claude Code
Where: your shell profile (~/.zshrc, ~/.bashrc) or a per-project .envrc
bash
export ANTHROPIC_BASE_URL="https://gateway.example.com"
export ANTHROPIC_AUTH_TOKEN="${HOOSHYAR_API_KEY}"
# Optional: pin every request from this shell to one route instead of
# letting the classifier choose per message.
# export ANTHROPIC_CUSTOM_HEADERS="x-task-type: <ROUTE_NAME>"
Claude Code sends the Anthropic Messages API, which the gateway serves natively — no model rewriting on your side. Unset ANTHROPIC_API_KEY in the same shell, or it takes precedence over the token above.
model is the ROUTE name — the gateway advertises each route through /v1/models, so switching model in Continue switches route.
OpenAI SDK (Python)
Where: anywhere you construct the client
python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.example.com/v1",
api_key=os.environ["HOOSHYAR_API_KEY"],
max_retries=2, # 3 tries in total, with backoff and jitter
)
resp = client.chat.completions.create(
model="auto", # a ROUTE name; use "auto" to let the classifier choose
messages=[{"role": "user", "content": "Hello"}],
)
OpenAI SDK (TypeScript)
Where: anywhere you construct the client
typescript
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://gateway.example.com/v1",
apiKey: process.env.HOOSHYAR_API_KEY!,
maxRetries: 2, // 3 tries in total, with backoff and jitter
});
const resp = await client.chat.completions.create({
model: "auto", // a ROUTE name; use "auto" to let the classifier choose
messages: [{ role: "user", content: "Hello" }],
});
App or bot (fetch, with retries)
Where: the module that calls the gateway
typescript
// Retry a refused or reset connection and a 502, 503 or 504. 3 tries,
// about 30 s at most. Every other status is the answer: return it.
const RETRY_STATUS = new Set([502, 503, 504]);
const BASE_DELAY_MS = [2_000, 8_000]; // before try 2, before try 3
export async function complete(body: object, sessionId: string): Promise<Response> {
for (let attempt = 0; ; attempt++) {
const last = attempt >= BASE_DELAY_MS.length;
try {
const res = await fetch("https://gateway.example.com/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.HOOSHYAR_API_KEY}`,
"Content-Type": "application/json",
"x-session-id": sessionId, // the same id on every try
},
body: JSON.stringify(body),
});
if (last || !RETRY_STATUS.has(res.status)) return res;
await res.body?.cancel();
await sleep(delayMs(attempt, res.headers.get("retry-after")));
} catch (err) {
if (last) throw err; // refused, reset, DNS: no answer at all
await sleep(delayMs(attempt, null));
}
}
}
// Retry-After when the gateway sends one (max 15 s), else 50-100 % of the base.
function delayMs(attempt: number, retryAfter: string | null): number {
const seconds = Number(retryAfter);
if (seconds > 0) return Math.min(seconds, 15) * 1_000;
const base = BASE_DELAY_MS[attempt]!;
return base / 2 + Math.random() * (base / 2);
}
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
// One session per conversation. A stream that breaks after the first token
// is NOT retried: fetch returns at the headers, so the read error reaches you.
// Show the partial answer, or tell the user and start the turn again.
const res = await complete({ model: "auto", messages }, `chat-${chatId}`);
Retries only what has no answer yet: a refused or reset connection, 502, 503 and 504. A 4xx (a guardrail block, a scope refusal, a rate limit) is the answer. Keep the session id at 64 characters or less: the gateway does not record a longer one.
The fastest way to confirm a key works. x-task-type names the route explicitly; with model: "auto" and no header, the classifier picks one.
Images, audio, batches and rerank
These surfaces do not use the chat route. Replace <MODEL_ID> with a model name from your administrator.
Images
Where: a terminal, or the same SDK client
bash
curl https://gateway.example.com/v1/images/generations \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"model":"auto","prompt":"a wide shot of a lighthouse at dusk","n":1}'
# Editing an existing image is multipart, and the model is a FORM FIELD —
# send it BEFORE the file part (see the note below).
curl https://gateway.example.com/v1/images/edits \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-F "model=<MODEL_ID>" \
-F "prompt=make the sky clearer" \
-F "image=@photo.png"
Generation is ROUTED — it goes to the image_generation route and is billed per image RETURNED, so a partial failure costs less, not more. Edits and variations are multipart and are NOT routed: you name a concrete model, because rewriting a field inside a body carrying image bytes would mean re-encoding it. Put model before the file part — a scoped key whose model arrives after the file is refused rather than allowed through unchecked.
Audio
Where: a terminal, or the same SDK client
bash
# Speech (text in, AUDIO BYTES out) — routed to the `tts` route,
# priced per character of input.
curl https://gateway.example.com/v1/audio/speech \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"model":"auto","input":"Hello from Hooshyar.","voice":"alloy"}' \
--output speech.mp3
# Transcription (multipart) — name a concrete model, `model` first.
curl https://gateway.example.com/v1/audio/transcriptions \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-F "model=<MODEL_ID>" \
-F "file=@meeting.m4a"
Speech returns raw audio, forwarded byte-for-byte — pipe it to a file, don't parse it as JSON. Transcription is multipart, so the same model-before-file rule applies. Note that a transcription's audio_seconds is NOT recorded: the duration is inside the file, which the gateway deliberately never reads.
Batches
Where: a terminal, or the same SDK client
bash
# 1. Upload the input file (one JSON request per line).
curl https://gateway.example.com/v1/files \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-F "purpose=batch" \
-F "file=@requests.jsonl"
# 2. Submit the batch, naming the file id from step 1.
curl https://gateway.example.com/v1/batches \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"input_file_id":"file-…","endpoint":"/v1/chat/completions","completion_window":"24h"}'
# 3. Poll it. When the provider finishes, this is also what records the
# cost — so polling your own batch is doing everyone a favour.
curl https://gateway.example.com/v1/batches/batch_… \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}"
Use the SAME key for both steps: a batch may only name an input file that key uploaded, so a file uploaded with another key is refused as if it did not exist. Results arrive within the completion window (up to 24 h), and cost appears on My batches once the batch is reconciled — priced at interactive rates for now, so treat it as an over-estimate.
Rerank
Where: a terminal, or the same SDK client
bash
curl https://gateway.example.com/v1/rerank \
-H "Authorization: Bearer ${HOOSHYAR_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"model":"auto",
"query":"how do I rotate a client key",
"documents":["key rotation guide","billing FAQ","routing overview"]
}'
Routed to the rerank route (textless, like images and speech). Documents are COUNTED but not priced per document — providers bill per search — so the report shows the measure without inventing a rate for it.
Streaming
Streaming works as in the OpenAI API. The gateway sends the stream to your client unchanged.