Latency: the Race to Zero...Are We There Yet?

Physics says zero latency is impossible, and Amir Langer opens by conceding the point immediately. The interesting question is not whether the race can be won but what is actually left to optimise once the easy wins are gone. His central claim is that the remaining wins do not come from faster hardware or from throwing threads and memory at the problem — they come from architecture, and specifically from separating concerns so aggressively that you gain the freedom to choose the communication mechanism between components, all the way down to an inlined function call.

Langer has been a software developer for a long time and joined the fintech industry in 2007 at a startup called Tradefair, which later renamed itself LMAX and became widely known for open-sourcing the Disruptor. He was the first developer there and a team lead for many years. He is now a principal developer at Adaptive, which he describes as "the home of Aeron", and much of the forward-looking material in the talk comes from a future project called Aeron Sequencer that he is developing with Martin Thompson — whom he calls the father of the Disruptor and Aeron — and others. The talk was recorded at QCon London (InfoQ files it under its QCon London 2025 topic).

These notes report what Langer presented. Where background has been added for readers who have not worked on trading systems, it is explicitly labelled as writer-added context.

What You Will Learn

• Why latency, and specifically predictable latency, is a direct revenue lever in trading systems rather than a general engineering virtue.
• Why the "buy faster hardware" era ended and what replaced communication cost as the dominant term in a modern distributed system's latency budget.
• What actually made the LMAX Disruptor fast, according to the person who was there, and why the answer is less exotic than the folklore.
• The latency ladder from cloud UDP through kernel bypass, IPC and finally a function call, and what each rung costs you.
• How replicated state machines, a totally ordered log, checkpoints and logical groups combine into a system where replication, high availability and near-zero recovery time fall out of the model rather than being bolted on.
• Why Aeron's cluster implements Raft specifically, and what its egress fan-out, scaling and recovery problems are.
• What a sequencer architecture is, which of those three problems it removes, and why it currently has no published numbers.

Why Latency Is Worth Paying For

Langer grounds the motivation in money before touching any technology. In fintech, latency links directly to profit: if you are faster than the competition, you reach the better deals first. He offers a concrete observation from a previous trading system where the team could see exactly which market makers had the lowest latency — they were the ones quoting the smallest spreads, because they knew they could get in fast and reprice when the market moved. This is his reported experience from operating a specific trading system, not a general market-structure claim.

The subtler point is that average latency is close to useless as a target. "If I send an order to a trading system, I really don't care what's the average latency of that trading system. I only care about the latency of my order." Predictability is therefore a first-class requirement alongside speed, and this distinction recurs throughout the talk: several of the techniques he presents are sold as much on tail-latency stability as on the median.

There is a third motivation that is easy to overlook: recovery. While a component is recovering it is unresponsive, and unresponsive is unavailable, so recovery time is latency too. This framing matters later, because the replicated-state-machine architecture he builds towards drives mean time to recovery towards zero almost as a side effect.

The scope he sets is a mission-critical system such as a trading system — highly distributed, but also required to be scalable and resilient. In such a system most of the time is spent on communication between components, those messages must be fast, and none of them may be lost.

The Past: Old Solutions, Repeatedly Rediscovered

Langer's historical section carries a thesis rather than being decoration. He starts with the Roman cursus publicus — not merely the famous roads, but a system of horses, wagons and relay points where officials could swap out the horse, the wagon and the rider, letting the state move messages and goods at much lower latency than private citizens could.

The electrical telegraph gets a human origin story: Samuel Morse was away on a business trip when his wife died unexpectedly, and he did not get the message in time. Langer says Morse was so devastated that he set out to invent something that would give future generations a better chance of hearing in time. The invention is, in that telling, directly motivated by latency.

The Pony Express is the pivot of the argument. It cut message delivery from Missouri to California from three to five weeks down to ten days, which Langer calls a huge success story. But the mechanism — relay points where the horse or rider is replaced — was already more than a thousand years old. Nothing was invented. And the company went bankrupt in 1861, only eighteen months after that success, because the telegraph reached the west coast and a fundamentally better solution existed. The lesson Langer draws and then applies repeatedly is that the biggest latency wins usually come from recombining known ideas, and that an impressive optimisation of the wrong mechanism is fragile.

The Present: Why Hardware Stopped Rescuing Us

Until roughly two decades ago, Langer says, reducing latency in software was simple: replace the hardware and take the boost. That no longer works, and he lists what changed. Modern processor designs are far more complex, with many more caching layers, and they occupy more physical space; throughput keeps increasing but latency is getting higher. There is far more shared memory available across cores, but having it is not the same as exploiting it. The cloud is everywhere and hides a great deal of complexity, but nothing is free — the cost is more layers of abstraction to fight through. And distributed systems are larger than ever, so where a decade or so ago you might still reason about transaction locks or CPU clocks, communication is now the bottleneck. "That's the real problem for latency."

Against that, he characterises the naive developer response as turning the volume "all the way up to 11": out-of-memory error, give it more memory; high latency, give it more threads and more CPUs, then hope the framework and the operating system will magically find an optimal schedule for your workload. His verdict is blunt — that doesn't work.

He then states the honest version of the easy answer. The genuinely best way to reduce latency is to ignore every other quality attribute: strap yourself to the rocket and go, without caring whether you explode midway or where you land. Sending messages fast is easy if you do not care about losing any. That is unacceptable in his domain, which sets up the real question of the talk: can you design a low-latency system that does not compromise the other quality attributes, and what are the trade-offs?

What Actually Made the Disruptor Fast

LMAX open-sourced the Disruptor in 2010. Langer describes it as a very efficient way of passing objects between threads in Java — in effect a very efficient queue. It had, in his words, all the tricks in the world: no memory allocation, precise knowledge of what the system did with memory barriers, and a ring buffer at its heart, a structure hardware designers had known since the 1970s. Again, nothing was invented.

The correction he wants to make is about attribution. All of those tricks contributed, but "the one big thing about this project that gave us a huge latency boost was separation of concerns". The Disruptor let LMAX separate the work streams: journaling an incoming message was decoupled from decoding it, and decoding was decoupled from the business logic. The result was threads that did exactly one thing, were never interrupted, and never waited. That is where the real improvement came from.

Writer-added context: a thread that neither blocks nor is interrupted keeps its instruction and data working set resident and avoids context-switch and lock-contention costs. Langer does not spell out this mechanism in the talk; he asserts the architectural cause and the observed result.

The Latency Ladder: From the Cloud to a Function Call

Martin Thompson and Todd Montgomery open-sourced Aeron in 2014. Langer describes its starting point as a very efficient, low-latency, reliable way of sending messages between processes. It can run over UDP — unicast or multicast — and can also use IPC, inter-process communication.

His measurement harness throughout is a simple echo test: a source sends a message to a target, the target sends it back, and the round-trip time is measured at a specific throughput rate. He frames what follows as a single scenario walked down "quite a few orders of magnitude" rather than a comparison of unrelated systems. The concrete numbers appeared on his slides rather than in the spoken narration, and the transcript preserves only a few of them, so the figures below are limited to what he actually stated aloud. What he stresses is that the actual numbers do not matter so much — it is the difference between the rungs that carries the argument.

The first comparison is a small surprise he poses to the audience. His cloud figures were from GCP, with other numbers for other providers. Asked whether the C version of Aeron is higher or lower latency than the Java version, his answer is "it's both higher and lower. It's about the same."

The next rung is kernel bypass. Aeron integrates with DPDK, which lets it bypass the layers of abstraction — including the operating system and the sockets — and write Ethernet packets directly into the network interface card. DPDK works on both GCP and AWS. Moving off the cloud onto their own hardware gave numbers better than plain cloud but not as good as cloud with DPDK; applying kernel bypass again on their own PerfLab hardware, this time via ef_vi, another project Aeron integrates with, got them to single-digit microseconds.

IPC is the rung below the network entirely, using the shared memory modern machines now have. The trade-off he names explicitly is that both source and target need access to that same shared memory, which in practice means they must be on the same host.

Then he pushes one step further, and this is the conceptual hinge of the talk. IPC is still message passing: you encode the message, put it on shared memory, and decode it. You have already accepted the same-host constraint. If you tighten that constraint to the same process, you can replace the message with a function call. A virtual function call is tens of nanoseconds, and if the compiler inlines the function, "this really is zero."

Writer-added note on the bottom rung: collapsing a link into a function call buys speed with the process boundary. Components in the same process can no longer be deployed, upgraded, or fail independently — a much heavier architectural commitment than the same-host constraint IPC imposes. Langer presents this rung as the logical endpoint of the ladder, not as a default.

The question this table poses is the one Langer wants to answer: can you design a distributed system with separation of concerns and enough control over communication that you can swap the channel per link — a function call where that is worth the coupling, a network hop where it is not? To answer it he goes back to academic research.

The Research Lineage

Langer presents an explicitly incomplete list of prior work, again to reinforce that nothing here is being invented. He names Virtual Synchrony and Ken Birman around 1987, Viewstamped Replication and Barbara Liskov around 1988, and then Paxos: Leslie Lamport attempts to publish the consensus protocol in 1989, it is too vague for anybody to understand, it slowly gathers pace and is finally published in 1998, and by 2001 people realise it is still vague, so Paxos Made Simple appears. In 2013 Diego Ongaro publishes Raft, which attempts to simplify further — a raft, as Langer puts it, to take us from the island of Paxos to a much more understandable consensus protocol.

Writer-added context: the transcript runs the Birman and Liskov attributions together, so the exact year-to-author pairing above follows the speaker's spoken ordering rather than an independently verified citation. A consensus protocol is the mechanism by which a set of nodes agrees on a single value or a single ordering of values even when some of them fail.

Replicated State Machines as the Unit of Computation

Every one of those projects starts from the same very basic computation model, and Langer treats it as the essence of the whole design: the replicated state machine. Input events arrive; for each input event the state machine deterministically modifies its state and then generates one or more output events. It must be deterministic, it knows only events, and it is asynchronous. Its power comes from its simplicity.

His worked example is a matching engine. Orders come in, the state is the order book, the order book gets modified, and execution reports are the output events.

Virtual Synchrony, he says, already had the two key ingredients his team believes such a system needs. The first is a totally ordered sequence of messages, which he calls the log. The idea is powerful because determinism plus a total order gives replication for free: hand the same log to different instances of the state machine and they arrive at identical states. Two further properties fall out of it. Checkpoints let a state machine declare that it has processed message two and not yet reached message three, so every component knows exactly where every other component is and can compare positions. And the log supplies time: the timestamp on the messages is the time of the system, so no separate synchronisation mechanism is needed and clock drift stops being a concern.

The second ingredient is logical groups. State machines can be grouped and then managed as a group, and the group membership is itself replicated in the log — so every component knows who the members are at any given point in time, which in this model means at any given point in the log, including who joined and who left.

Put together, all replicas in a group hold exactly the same state at the same checkpoint. Langer then enumerates what that buys you in latency terms. Background work can be divided across instances: assign one instance to do backups and another to serve queries while the rest keep processing, since they all have the same state. High availability comes from designating one member active and letting it publish its output events while the others keep consuming the log, sitting at the same state and ready to take over if the active member dies.

The more aggressive option is multiple active members. Because the state machines are deterministic, it does not matter which instance produces a given output message — the messages will always be the same. So you let all of them publish, the fastest one wins, and you discard the duplicates. The trade-off he names is bandwidth: you pay for redundant transmission and get the fastest possible delivery. The same property drives mean time to recovery for an instance to effectively zero, because by the time a recovering instance would have produced a message, the other active members already produced it.

Composing State Machines

Separation of concerns appears here in two distinct forms, and Langer is careful to distinguish them.

The first is between state machines and state machine groups, which arises because they communicate via the log and therefore have to define a very clear protocol between them. He considers that a good thing, but notes it may be slow, which motivates state machine composition — more than one state machine inside the same group member. He describes two composition patterns. In the dependency pattern, both state machines consume the log and are therefore at the same checkpoint, and at that point one can safely query the other precisely because they are at the same point in time. In the pipelining pattern, the output messages of one state machine are the input events of the next.

The second form of separation is between the business logic — the state machine implementations — and the rest of the system's concerns. The state machines do not care how the log is persisted or how it is distributed across instances. They care only about the events they consume, the events they emit, and their state. This is what makes the earlier latency ladder usable: because the business logic is indifferent to the transport, composition inside a process can collapse a link down to a function call without the state machine knowing.

Making the Log Fault Tolerant

Wonderful state machines are not enough; the log itself must survive failure. Distributed systems research supplies the answer — a quorum of three, five or seven nodes running a consensus protocol among themselves to agree on what the log is.

Langer acknowledges the reputation problem directly: consensus "has a history of considered very slow", and his team believes it can be implemented pretty fast, but many past solutions avoided it because it looked slow and settled for a primary and secondary instead. Those solutions then need mitigations, and he gives one concrete example: the "one in flight" technique, where you allow only a single message in flight to the primary so that if the primary fails and you cut over to the secondary, there is only one message to worry about.

Aeron's open-source cluster implements Raft, and the reason given is specific and latency-motivated rather than ideological: Raft has the concept of a strong leader, and a leader that is not swapped around gives more predictable latency. "It's strong until it dies, but it is strong."

The cluster echo benchmark sends a message into the cluster, runs the consensus protocol, and returns the response after consensus is reached. Because the underlying transport is the same Aeron transport, the relationships hold proportionally: C and Java come out the same, and DPDK gives a large boost — not only much lower latency but, he emphasises, more predictable latency. On the PerfLab with kernel bypass, the figure he states aloud is a P999 of less than 29 microseconds for the full path of sending a message in, reaching consensus, and sending the response back. That number is his team's own lab hardware with kernel bypass, not a general guarantee about consensus.

IPC is deliberately absent here, and the reasoning is worth internalising. The entire point of running a consensus protocol is fault tolerance; if all the nodes are on the same host you get no fault tolerance regardless, so consensus over IPC makes no sense.

Architecture And Data Flow

Langer describes two architectures in sequence. The Mermaid diagrams below are this writer's rendering of the two designs as he described them verbally; the component relationships are his, the visual layout is not from his slides.

The first is the cluster architecture his customers run today. Applications send messages in, the cluster agrees on the log, the log is fed to the state machines that live inside the cluster nodes, and their output events become messages on an egress channel back to the applications.

flowchart LR
  A1[Application] -->|ingress| C
  A2[Application] -->|ingress| C
  subgraph C[Fault-tolerant cluster - Raft]
    L[(Agreed log)] --> SM1[State machine]
    L --> SM2[State machine]
  end
  SM1 -->|output events| E[Egress channel]
  SM2 -->|output events| E
  E --> A1
  E --> A2

He identifies three problems with it. The first is fan-out: in fintech, one message in nearly always produces considerably more than one message out, so merely managing the volume of output events turns the egress channel into a bottleneck. The second follows from the first — a scaling problem, because you cannot keep adding state machines and handling more data when everything is condensed into that one egress channel. The third is recovery: upgrading a state machine means taking down one of the cluster nodes, which has a cost for the consensus protocol, so you are left running a hot standby or finding some other workaround.

The sequencer is the answer, and Langer is explicit that it is not new either. The architecture has existed in fintech for some time; the first company to talk about it publicly was Island ECN in 1996, and their sequencer ended up being the sequencer in NASDAQ. The idea is to have a component that runs no business logic at all — it only decides the log, sequences the messages and timestamps them — while the state machines move out into the applications.

flowchart LR
  A1[Application with composed state machines] -->|ingress| C
  A2[Application with composed state machines] -->|ingress| C
  subgraph C[Fault-tolerant cluster]
    S[Sequencer - ordering and timestamps only]
  end
  S -->|distributed log| A1
  S -->|distributed log| A2
  S -->|distributed log| A3[Application group]

Because the cluster now publishes the log rather than the output messages, the fan-out problem disappears — the log is a condensed version of the input events only — and the scaling problem goes with it. Recovery cost also drops, because upgrading a state machine is now upgrading an application: take it down, bring it back up, and let it resume consuming the log from the same checkpoint. The end state he envisions is a sequencer inside a fault-tolerant cluster distributing the log to many groups of composed state machines, giving a distributed system whose separation of concerns lets you fit the communication mechanism to your particular problem.

Trade-offs Raised in Q&A

The sequencer has no published performance numbers, and Langer was asked about this directly. His answer has two parts. It is a future product that does not exist yet outside "our laptops, or a very crude version of it". More interestingly, he argues an echo benchmark would not show anything: an echo version of the sequencer would look very similar to the cluster numbers because it is barely doing anything. The sequencer's value is a tighter fit to a real distributed system, so only real scenarios will show the difference. That is stated as intent, not as a measured result — everything in the sequencer section should be read as a design argument rather than validated performance.

Kernel bypass couples you to hardware. Asked about the downsides, Langer said it is very low level and very tightly bound to the network interface card. Replace your hardware and you may find you need a different integration, or that the new card has no support at all. The performance is real; the portability cost is real too.

Business dependencies cannot be architected away. An audience member asked whether slow work in one component — say, slow queries — drags on the rest of the system, given that everything synchronises on the same sequence. Langer's answer was candid: components in a distributed system do communicate, and if one sends a message to another and needs a response back, that is a real dependency with no way around it. "If that's the business problem, then that's the business problem." What the architecture removes is the accidental dependencies. His prescription for the real ones is to never block on them: decouple, separate concerns, work with queues, send the message and react to the response later while doing other work in the meantime. Synchronously waiting for a response is, in his words, "really bad".

Java is not the bottleneck, but it does require discipline. An audience member argued that fast IPC is undermined by Java — the transcript records the questioner saying "10 milliseconds" and immediately self-correcting to "10 microseconds" — because loading a class into memory could take on the order of fifty microseconds. Langer's response was that Java is genuinely fast, that the JIT compiler's optimisations change the game, and that the fact Java and C reach comparable numbers is itself the evidence. He conceded the general point about language cost — "if you have an interpreted language versus C, there's no battle here, really" — and argued that the JIT is precisely why Java does not belong in that category. The caveat is that you must not allocate memory freely, because the garbage collector will eventually run and you will lose your predictable latency; allocate in advance and never on the hot path and you are fine. He frames this as ordinary language-specific discipline rather than a Java handicap: "This fight has ended. Java is not slow by any means."

Low latency in the cloud is genuinely hard. Asked which of these techniques should reach mainstream software, Langer's short answer was "I don't know". His longer answer was that low-latency reliable message protocols could plausibly move lower in the stack and become more widely used. But the cloud brings its own challenges — availability zones, and frequently no visibility into the hardware underneath — and some customers care about latency enough that they still buy their own hardware. What he would like to see is more customised low-latency reliable messaging offered by the cloud providers themselves.

Where the Race Stands

Langer's closing answer to his own title is that we are not there and will not be. The race to zero latency "still goes on and will still go on", and he says so with no expectation that it ends. What has changed is where the remaining ground is won. A distributed system that is virtually synchronous — the 1987 idea — combined with state machine composition gets much closer to zero than tuning any single component does, because it lets you fit the communication mechanism to the actual shape of your problem instead of accepting whatever the architecture happened to impose.

Practical Takeaways

Start by measuring the percentile that matches your business, not the mean. Langer's order-latency argument generalises: if individual requests carry individual value, a P999 target is the honest target and an average is a comfortable fiction. Recovery time belongs in the same budget, since an unresponsive component is an unavailable one.

Before reaching for exotic techniques, look for work streams that can be decoupled. The Disruptor's headline gain came from giving each thread exactly one job so it never waited and was never interrupted — journaling, decoding and business logic as separate stages. That refactoring is available in most systems and requires no special hardware.

Treat the communication mechanism as a design variable rather than a fixed property of your architecture. Once business logic only knows about input events, output events and its own state, you are free to move a link between network, IPC and in-process function call as requirements change, and to choose differently per link.

If you need determinism-based replication, enforce it ruthlessly. Every benefit in the replicated-state-machine model — replication, hot standbys, duplicate discarding, near-zero recovery — depends on identical inputs producing identical outputs. A stray call to the system clock or a random number generator inside the state machine silently voids all of it. This last consequence is writer-added context; Langer states the determinism requirement but does not enumerate the ways it can be broken.

Given the NIC coupling described above, pin down which cards you support before adopting kernel bypass, and plan a hardware refresh as an integration project rather than a procurement exercise. If you are on Java, pre-allocate everything and keep the hot path allocation-free.

Key Terms

Reference: Latency: the Race to Zero...Are We There Yet?