Staff Engineer, Routing
Lucas works on failover, load-shaping and cost-aware routing across the provider fleet. He writes about the parts of running an AI gateway that only show up in production: the tail latencies, the mid-stream failures, the bills nobody forecast.
Areas of focus
Most fallback configurations are written once, never exercised, and quietly wrong. Provider outages are a when, not an if. Here is how to design a chain that degrades gracefully instead of failing over into a wall.
In a multi-turn conversation, every earlier message is re-sent on every new turn. Cost grows with the square of conversation length. Here is the math, and four ways to bend the curve back down.
See it in your stack
Browse the catalogue, ask Echo what fits your workload, or get started and we'll help you onboard the right models and gateway configuration.