Why Your Fraud Model Needs to Decide in 45 Milliseconds

You've spent three weeks training a fraud model. Terabytes of transaction history, 94% recall on the test set. Now a card gets tapped at a coffee shop, and your model has 45 milliseconds to decide whether the transaction is legitimate.

Forty-five milliseconds. Less than half the time it takes to blink.

The obvious explanation is that fraud happens fast, so the model needs to be fast. But fraud doesn't happen in 45 milliseconds. A stolen card number gets used over days or weeks. The speed constraint comes from the payment network.

Where the milliseconds go

When you tap a card, the payment terminal sends an authorization request through the card network to the issuing bank. The bank runs a fraud check, makes an approve-or-decline decision, and sends the response back. That entire round trip, from tap to terminal beep, takes well under a second.

Merchants expect the full authorization round trip in under 200 milliseconds. That's your budget. Start spending it.

Network transit takes roughly 10 to 15 milliseconds. The transaction travels from the merchant's terminal through the payment processor to the issuer's systems, limited by geography and the speed of light.

Feature retrieval takes another 5 to 10 milliseconds. Your model needs context about this specific cardholder, like how often they shop here and what their average transaction amount looks like over the last 30 days. Those values have to come from a low-latency store, pre-calculated and waiting.

After rules evaluation, response serialization, and safety margins, your model gets maybe 10 to 25 milliseconds for inference. That's the window everything has to fit inside.

Why nobody runs a transformer in the hot path

An XGBoost model scores a transaction in roughly 1 to 5 milliseconds on a single CPU core. No GPU required. Visa's fraud scoring system, Advanced Authorization, processes each transaction in about one millisecond. The network has been running neural networks optimized for this constraint since 1993.

A transformer-based model needs tens to hundreds of milliseconds, and it typically needs a GPU. That alone can exceed the fraud model's entire inference window.

So the budget picks your architecture before you write a line of training code. Gradient-boosted trees dominate production fraud scoring because they fit inside the clock. A recent benchmark found GBDTs outperform LLM-based scorers by roughly 8,000 to 1 on latency in the payment authorization path. Deep learning works for offline fraud analysis, but the hot path doesn't have room for it.

A restaurant with a 90-second ticket window works the same way. The menu isn't limited by what the kitchen could make. A two-hour braise might taste better, but it can't hit the plate before the ticket expires.

The feature problem is worse than the model problem

Your model's inference takes 2 milliseconds. But the features it needs have to be ready before inference starts.

During training, you compute "average transaction amount over 30 days" from a data warehouse query. It takes four seconds, perfectly accurate, but in production you need that same number in 5 milliseconds, and no warehouse query runs that fast.

So you precompute. A feature store holds pre-calculated values in a low-latency cache, usually Redis or a similar store. When a transaction arrives, you look up the features instead of computing them. Feature stores like Feast and Tecton exist specifically because this problem is universal in real-time ML.

But precomputation creates its own failure mode. You now have two implementations of the same feature: the warehouse version you trained on and the cached version you serve with. Different code paths running against different data sources, occasionally with different rounding. They drift apart, and your model's accuracy in production silently degrades because the features it sees at inference don't match what it learned from.

This training-serving skew is where most production fraud systems quietly break, even when the model itself works exactly as designed.

Forty-five milliseconds picks your architecture

Visa processes over 65,000 transactions per second at peak capacity. Each one gets a fraud decision inside that narrow window. Forty-five milliseconds picks your model, forces precomputation over on-the-fly feature engineering, and explains why your training cluster and your serving stack look nothing alike.