Always-on model reliability.
Automatic failover, circuit breakers, and real-time latency tracking keep your AI workflows running even when individual providers go down.
Capabilities
Automatic Failover
Per-provider health tracking detects failures in real time and automatically reroutes traffic to healthy providers. Streaming responses are preserved across all failover scenarios — users experience no interruption.
Circuit Breakers
Configurable circuit breakers prevent cascading failures. When a provider exceeds error thresholds, it is removed from the rotation automatically and re-evaluated after a ~5-minute recovery window.
Latency Monitoring
Real-time per-provider latency tracking surfaces degradation before it becomes a failure. Route away from slow providers with policy-based latency thresholds — no manual intervention required.
Instant Model Controls
Platform operators can disable any model or provider with a single action — no deployment required. Emergency lockout takes effect across all routing in seconds, giving operations teams immediate control when something goes wrong.
How it works
Health checks run continuously
Arbitex Gateway monitors every connected provider in real time. Error rates, latency percentiles, and availability are tracked per-provider and surfaced on the operations dashboard. No configuration required — health monitoring is always on.
Failures trigger automatic failover
When a provider exceeds error thresholds, the circuit breaker activates and traffic is rerouted to the next healthy provider in the fallback chain. Streaming connections are maintained — the end user sees no interruption. The affected provider enters a ~5-minute recovery window before being re-evaluated.
Operators stay in control
Review health events in the audit log and latency trends on the dashboard. Use instant model controls to disable any provider or model immediately — no deployment, no code change. Re-enable with a single action when ready.