Skip to main content

Model Routes

Named, ordered provider+model failover lists an agent can generate through.

Overview​

A model route is a project-scoped, ordered list of provider+model targets. Target 0 is tried first; a retryable failure (a provider error, a timeout, a rate limit — whichever classes retry_on lists) falls through to the next target. Deterministic failures — a 400-class error, an auth failure, a content policy rejection — never fail over: they fail the generation immediately.

This module is a verbatim mirror of the runtime: every field, method, status code and error shape is the runtime's own, re-rooted under the project in the path. Errors raised by the route machinery arrive in the runtime's envelope.

See the OpenAPI spec for the full endpoint and schema reference, or browse it rendered under API Reference → Model Routes.

Data Model​

ModelRoute​

FieldTypeDescription
idstring
project_idstring
namestring
targetsobject[]
retry_onarray of provider_error | timeout | rate_limited
failure_thresholdinteger
cooldown_secondsinteger
created_atstring (date-time)
updated_atstring (date-time)

name is unique per project. targets are tried in order: position is priority. retry_on lists the failure classes eligible for failover; after failure_threshold consecutive retryable failures a target is skipped for cooldown_seconds, then probed again.

ModelRouteTarget​

FieldTypeDescription
ai_provider_idstringAI provider in the route's project
modelstringModel name to call on that provider
timeout_secondsintegerPer-attempt deadline, enforced with an AbortSignal composed with the caller's signal. A timeout classifies as timeout; the caller's own signal firing aborts the run without failover. Omitted means no per-target deadline.
max_retriesintegerRetries on this target before falling through to the next one. The route is the only retry authority — routed calls pass maxRetries: 0 to the AI SDK so its own retry loop cannot multiply these.

max_retries defaults to 0.

Key Concepts​

Ordered failover with a bounded budget​

Targets are tried in array order, and the total attempt budget — the sum of 1 + max_retries over all targets — may not exceed 10; POST /v1/projects/{project_id}/model-routes rejects a route over the cap with a 400 naming the computed total. Every target must reference an AI provider in the same project, and a duplicate name in the project is a 409.

Circuit breaking​

After failure_threshold consecutive retryable failures a target is skipped for cooldown_seconds before being probed again. Breaker state is keyed by (provider, model), so every route pointing at the same backend shares it.

A managed target is checked when it is written​

A target on a naturali provider is refused before the route is stored:

  • 400 MODEL_NOT_OFFERED when its model has no price, or is not a catalog model at all, such as the vendor's own id rather than its name; meta names the model. See A managed model must carry a price.
  • 400 VALIDATION_FAILED when its model is served by a source its managed provider does not serve; meta names the model, its source and the provider_source. The shared naturali provider serves every source.

A target on a provider of your own is never checked.

This holds however the write arrives: a call to this API, a formation deploy or an agent's tool call.

Deleting a referenced route​

DELETE /v1/projects/{project_id}/model-routes/{route_id} answers 409 while an agent still references the route — a routed agent has no pinned provider to fall back on, so repoint or delete the agent first.

Examples​

Create a route with a fallback​

naturali create-model-route \
--project-id proj_V1StGXR8Z5jdHi6B \
--name primary-with-fallback \
--targets '[
{ "ai_provider_id": "aip_V1StGXR8Z5jdHi6B", "model": "gpt-4o-mini", "max_retries": 1 },
{ "ai_provider_id": "aip_9fJk2LmNpQrStUvW", "model": "claude-haiku-4-5" }
]'

List a project's routes​

naturali list-model-routes --project-id proj_V1StGXR8Z5jdHi6B