Reliability and failover

Routing selects eligible providers for the same public model, subject to protocol, modality, access and health constraints. Multiple providers reduce interruption risk but cannot guarantee uninterrupted service.

Retry boundaries

The gateway may retry an unsent connection failure or an eligible upstream error before output starts. It does not blindly retry transport failures or timeouts after a request has been sent. Once a stream starts, the gateway does not replay it to the user.

A client timeout can leave upstream generation running. Retrying a creation request may duplicate work and charges; no general idempotency guarantee is provided. Query a known video task before creating another.

Diagnose failures

Record X-Gateway-Trace-ID, status, timing and retry metadata. Check model permissions, budget and rate limits before treating a failure as a provider outage. Circuit breaking and capacity checks may leave no eligible route; back off rather than amplifying the outage with parallel retries.

See streaming, errors and observability.