Relay: environments and policies

Why environments

Production and a data team's sandbox should not share a budget, a model list or a retention setting. In Relay every control hangs off an environment, and every key belongs to exactly one. An organisation on the Team plan has one environment; Business has five; Enterprise as many as needed.

Providers and channel order

Each environment lists the provider connections it may use, in order. A typical production order is: your own Anthropic key → your own OpenAI key → Vertex AI in your Google Cloud project. Relay tries the first healthy channel that serves the requested model; a channel that errors or times out is cooled down for sixty seconds and skipped by the next request. The channel used is returned in X-Relay-Channel.

Environments marked Google-resident use Vertex AI only and never route to an external provider, even as fail-over; a model Vertex does not serve is refused with region_unavailable.

Model allow-list

A list of model ids (exact match, or a prefix such as claude-*). Requests for other models are refused with 403 model_not_allowed and the error names the list, so a developer can fix the call without asking anyone. An empty list means "any model the connected providers serve" — recommended only for sandboxes.

Budget mode

Each environment has a monthly budget in US dollars and a mode:

• Hard — at the limit Relay returns 402 budget_exceeded until the calendar month rolls over or an admin raises the budget. Use for anything that is not production-critical, and for production when a stopped service is cheaper than an unbounded bill.
• Soft — traffic continues; alerts go out at 80 % and 100 % (email to environment owners, webhook if configured).

How the dollar figure is computed, and why it is accurate on your own keys too, is in Relay: budgets.

Guardrail policy

Per environment, for requests and for responses separately: off, flag (record the verdict, let the call through) or block (422 guardrail_blocked). Detectors in the first release are deterministic — PII patterns (e-mail, phone, national ID formats, card numbers) and prompt-injection markers — so the same input always gets the same verdict. See Relay: guardrails.

Keys

Keys are created per environment with a requests-per-minute limit and an optional expiry. Rotation: create the new key, deploy it, revoke the old one — both are valid in the overlap. Revocation takes effect within seconds across all Relay instances.

Retention

Trace retention is set per environment within the plan's maximum (30 / 90 / 365 days). Whether prompt and completion bodies are stored is a separate per-environment switch, off by default; when on, bodies follow the same retention and are included in exports.

Who can change what

Organisation owners manage billing, SSO and environments. Environment admins manage that environment's providers, policies and keys. Viewers see dashboards and traces. Every change is written to the audit log with actor, time and before/after values.