I spent over a decade working on hyperscaler infrastructure, and part of that time leading Google's Capacity Safety team. When you operate at that scale, you learn two lessons early and you never forget them. First, latency is user experience. A slow system doesn't feel "a bit slow" to the person using it; it feels broken. Second, reliability and cost efficiency are the same discipline. Systems that degrade politely under load are also the systems you can run economically.
Those lessons shaped how I designed the Meilynx proxy. The hot path comes first, always. Traffic interception runs the simplest code that can do the job, and anything optional or expensive stays off the critical path or runs on an explicit time budget. No clever algorithms standing between your request and your model provider. Simplicity wins big here, and this post is about proving that with numbers rather than asserting it with adjectives.
Every team that puts a governance layer in front of their AI traffic asks the same question first: what does this cost me in latency? It's the right question. An inline proxy sits in the critical path of every model call, and a slow one taxes every user of every application behind it.
So in August 2026 we ran a full load and stress campaign against a Meilynx managed proxy in its production configuration and pushed roughly 1.5 million requests through it: ramps to saturation, sustained soaks, burst storms, and a matrix of governance configurations. Here's what we found, and exactly how we found it.
The headline
Four milliseconds, flat
The number we care most about is proxy overhead: the time Meilynx adds on top of what your model provider takes. The proxy computes this per request (total time minus upstream time) and exports it as a histogram, so the measurement comes from the data path itself, not from a marketing spreadsheet.
Across the entire campaign, the 95th percentile of proxy overhead held at 4 ms. Not 4 ms at idle. It was 4 ms at 60 requests per second, 4 ms at 300 requests per second, 4 ms during a 90-minute sustained soak, and 4 ms while the instance was saturated and queueing. Every payload size we tested (1 KB, 12 KB, 100 KB), every governance configuration, every load level. The processing pipeline never budged.
Here's the shape of the whole distribution, from one representative high-load window of 554,498 requests:
Practically the entire distribution lives under 5 ms, and a third of requests complete their governance pass in under a millisecond. Our internal engineering target of 25 ms sits far to the right of anything the proxy actually produced.
That flatness is the finding we were most interested in. When you overload a single proxy instance, the latency your clients see does grow. But it grows as queueing in front of the instance, the same way any saturated server behaves, while the governance work itself stays constant. Overload degrades politely. And it never once became an error: zero of the ~1.5 million requests in this campaign failed.
Guardrails
The detection stack costs single-digit milliseconds
Governance rules are configurable, so "what does governance cost" depends on what you turn on. We measured three configurations at a steady rate, across three payload sizes, and read the validator time straight from the proxy's own histograms:
Validator cost scales with payload bytes. These are pattern scanners doing real work over your prompt, and the cost tops out at about 7 ms at p95 with six content validators enabled against 100 KB payloads. Worst case measured, everything on, biggest payload: 9 ms of total proxy overhead at p95.
Two things we deliberately did not measure here: externally hosted guard models and customer webhooks. Those run on your own endpoints with configurable time budgets, so their latency belongs to the endpoint you point them at.
Streaming: 22 ms to first byte
Agent traffic streams. In live pass-through mode, the streaming path used when no output-rewriting rule is active, client-observed time to first byte was 22 to 26 ms. The stream starts flowing essentially as fast as the upstream produces it, with governance evaluated inline.
Methodology
How we measured, so you can argue with us
We take reliability seriously. Availability is one of the Trust Service Criteria covered in our SOC 2 examination, where an independent auditor examined the monitoring, capacity, and recovery machinery behind it. Publishing our load-test methodology and numbers, in enough detail for anyone to scrutinize or reproduce, is the same posture applied to performance. Here's the setup:
- Real production shape. The target was a managed proxy provisioned by the same automation that provisions customer proxies. Same instance size, same always-on configuration, same audit pipeline writing tamper-evident records for every request. Not a stripped-down lab build.
- A stub upstream with fixed latency. We pointed the proxy at a model-API simulator with a constant 200 ms response time. That isolates our overhead from provider variance, removes rate limits, and gives an independent cross-check: client-measured end-to-end time minus 200 ms should equal proxy plus network overhead. It did.
- Open-model load generation. We used k6 with arrival-rate executors from a same-region VM. The load keeps arriving on schedule whether or not the target slows down, so tail latency can't hide behind coordinated omission.
- Two vantage points. Every scenario was measured from the client and from the proxy's own exported histograms. When the two disagreed (client tails versus a flat server-side pipeline), the disagreement itself was informative: it located the queueing at the instance boundary rather than inside the pipeline.
- Stress included. Ramps until saturation, instant zero-to-full bursts, and a 90-minute soak at 150 requests per second. That soak alone was 808,268 requests, with no drift in any percentile from the first window to the last.
The harness (load scripts, upstream simulator, and runbook) lives in our repository and runs as a recurring regression check. These numbers get re-earned, not framed and hung on a wall.
What it means
Governance you can afford to leave on
The practical takeaway: with a full content-detection stack enabled, Meilynx adds single-digit milliseconds to calls that typically take your model provider hundreds to thousands of milliseconds. That's well under one percent of a typical LLM round trip. The "governance tax" argument for running ungoverned AI traffic doesn't survive contact with the measurement.
These are measured results from our August 2026 campaign on a production-shape deployment, and we'll keep publishing as the numbers evolve. If you want to see the machinery behind them, the audit chain, the fail-closed posture, and the examination-grade evidence the proxy produces while adding those 4 ms, that's what the platform is about.