Beyond pipeline replication: Rethinking FPGA packet processing for the 400G era
For years, increasing network bandwidth followed a relatively predictable path. Ethernet speeds doubled, FPGA devices became larger and faster, and packet-processing architectures evolved incrementally. A design that successfully handled 10G traffic was extended to 25G, then 40G, and eventually 100G with relatively modest architectural changes.
Many engineers naturally assume that the transition from 100G to 400G is simply another step along the same path.
It is not.
The move to 400G represents one of the biggest architectural shifts programmable networking has experienced in more than a decade. Many design assumptions that worked perfectly at 100G begin to fail, not because FPGA technology has stopped advancing, but because the underlying physics and architecture have fundamentally changed.
A fully utilized 400G Ethernet link can transmit nearly 600 million minimum-sized packets every second (calculation based on 400 Gb/s Ethernet and 64-byte minimum Ethernet frames). At that scale, even seemingly insignificant inefficiencies become major design bottlenecks. Routing delays measured in picoseconds, additional buffering stages, or a few extra clock cycles suddenly determine whether a design succeeds or never reaches timing closure.
This is precisely why surprisingly few production-ready 400G FPGA SmartNIC platforms exist today despite several years of 400G Ethernet standardization.
Building a true 400G FPGA solution is far more than increasing bandwidth. It requires rethinking how packets move through programmable hardware.
From faster links to different architectures
One common misconception is that 400G networking is simply a faster version of 100G networking.
The physical interface may look familiar. Ethernet packets remain Ethernet packets. Software APIs such as DPDK or RTE Flow continue to expose similar programming models. Applications such as firewalls, intrusion detection systems, telecom infrastructure or packet brokers still perform many of the same operations.
However, beneath this familiar software layer, almost every hardware assumption changes.
The transition from earlier Ethernet generations to 400G required a fundamental evolution of the physical layer. Instead of relying on mature NRZ signalling used for many years in 10G, 25G and 100G systems, modern 400G interfaces depend on PAM4 modulation and 50G or 100G per-lane SerDes technology.
Although the first IEEE 400G Ethernet specification (802.3bs) appeared in 2017, the technologies necessary for robust production deployment matured much later. The IEEE 802.3ck standard, defining 100G electrical interfaces, was ratified only in 2022, while the first 800G Ethernet standard followed in 2024.
Only recently have FPGA vendors introduced devices with sufficiently capable transceivers, integrated Ethernet MACs, PCI Express Gen5 interfaces and high-bandwidth memory to make practical 400G SmartNIC implementations possible.
Yet solving the physical interface is only the beginning.
Receiving 400 Gbps is relatively straightforward.
Processing 400 Gbps deterministically inside an FPGA is a completely different engineering problem.
The hidden bottleneck is not the Ethernet port
When discussing high-speed networking, conversations often focus on the optical transceivers, Ethernet MACs or PCI Express bandwidth. These components are certainly critical, but they are rarely the primary obstacle.
The real challenge lies inside the FPGA itself.
Every incoming packet must pass through multiple processing stages before it reaches host memory or leaves the network interface. A typical packet-processing pipeline may include parsing Ethernet, IP, and transport headers, performing classification and lookup/matching operations, applying forwarding or security policies, modifying packet headers, collecting statistics, and finally transferring processed data through DMA into system memory.
At 100G, modern FPGA architectures handle these tasks efficiently using deeply pipelined processing blocks operating at a few hundred megahertz.
At 400G, however, the required throughput increases fourfold while the achievable FPGA clock frequency does not.
This simple observation drives almost every architectural decision that follows.
Unlike CPUs or ASICs, FPGA performance cannot be increased indefinitely by raising the operating frequency. Configurable routing, programmable logic, and flexible interconnects provide tremendous adaptability, but they also introduce physical timing limitations that become increasingly difficult to overcome as designs grow in complexity.
Eventually, designers discover that the problem is no longer bandwidth.
The problem becomes architecture.
Why this matters beyond networking
These architectural limitations are not merely academic.
Modern data centres increasingly rely on programmable networking infrastructure to perform tasks that were once handled entirely by software. Firewalls inspect traffic at line rate. Telecom systems process user-plane packets with deterministic latency. AI clusters exchange enormous datasets across high-speed fabrics. Financial trading platforms depend on predictable packet timing measured in microseconds – or even nanoseconds.
All of these applications share one common requirement:
Packets must be processed at line rate without introducing excessive latency, packet loss or unpredictable behaviour.
As bandwidth continues growing toward 400G and eventually 800G, traditional FPGA packet-processing techniques begin approaching their practical limits.
This raises a fundamental engineering question:
If increasing clock frequency is no longer feasible, how should packet-processing architectures evolve?
The answer is less obvious than many engineers initially expect.
And it begins by examining why the most intuitive solution – simply replicating existing 100G processing pipelines – turns out to create an entirely new set of problems.
Key Takeaways
- 400G is not simply “four times 100G”. It fundamentally changes the architectural constraints of FPGA packet processing.
- Modern FPGA devices can now support 400G physical interfaces, but moving packets through programmable logic at line rate remains the real engineering challenge.
- The limiting factor is no longer Ethernet technology – it is the architecture of the packet-processing pipeline itself
Why traditional FPGA architectures stop scaling
At first glance, the solution seems almost obvious.
If a single FPGA pipeline can process 100G of traffic, why not simply instantiate four identical pipelines and distribute packets between them?
For many engineering teams, this is the natural starting point. It minimizes redesign, reuses existing IP blocks, and appears to provide a straightforward path toward 400G throughput.
Unfortunately, this intuition collides with the realities of modern FPGA architecture.
What appears elegant on a whiteboard quickly becomes one of the most difficult engineering challenges in high-speed programmable networking.
The first instinct: Increase the clock frequency
Before discussing parallel pipelines, it is worth addressing another common assumption.
Why not simply run the existing design four times faster?
This approach works well in software. CPUs continuously increase clock frequencies, and ASIC designers often push critical datapaths well into the gigahertz range.
FPGA devices, however, obey very different rules.
Unlike ASICs, where every transistor and interconnect is optimized for a single purpose, FPGAs derive their flexibility from a vast programmable routing network. Every logic block can be connected to almost any other logic block through configurable interconnects. This programmability is what makes FPGAs so powerful – but it also introduces unavoidable routing delays.
As designs become larger, signals travel longer distances through the device. Critical paths become increasingly difficult to optimize, and timing closure turns into one of the dominant engineering challenges.
In practice, modern packet-processing pipelines typically operate somewhere between 250 and 400 MHz, depending on design complexity and FPGA family.
While the latest FPGA generations continue to improve timing performance, increasing a complex networking design toward one gigahertz simply is not a realistic option. Physics becomes the limiting factor. Instead of increasing frequency, designers must find different ways to increase throughput...
The obvious alternative: Replicate everything
If increasing clock frequency is no longer feasible, another solution immediately suggests itself. Keep the clock frequency unchanged.
Simply duplicate the entire processing pipeline.
Incoming packets are distributed across multiple parallel pipelines using a hash function based on Ethernet, IP or transport-layer headers. Each pipeline processes its own subset of traffic independently before all streams are merged again at the output.
Conceptually, the architecture looks attractive. This strategy is widely used in networking systems because it scales well for many traffic patterns.
However, FPGA implementations expose several hidden costs that become increasingly difficult to ignore.
Resource consumption grows faster than expected
A packet-processing pipeline is far more than a sequence of combinational logic.
Each pipeline typically contains:
- packet parser
- metadata extraction
- lookup/matching engines
- match-action tables
- statistics collection
- packet editing buffering
- DMA interfaces
- control logic
Replicating four pipelines therefore does not simply quadruple throughput. It also replicates nearly every hardware resource inside the FPGA. Logic utilization increases dramatically. Memory consumption rises. Routing congestion grows. Power consumption increases.
Most importantly, the placement and routing tools now need to close timing across a significantly larger and more fragmented design. Ironically, many projects discover that adding more hardware actually makes timing closure more difficult rather than easier.
Packet ordering suddenly becomes a system-level problem
Resource utilization is only the beginning. A much more subtle problem emerges once packets are distributed across multiple independent pipelines.
Imagine arriving at a supermarket with your family: Everyone enters the store together. At the checkout, however, each person chooses a different cashier. Although everyone arrived simultaneously, they leave the store at different times.
Network packets behave exactly the same way. Once traffic is split across several pipelines, each packet experiences a slightly different processing time.
Some packets require only simple forwarding. Others trigger multiple memory lookups, security checks or header modifications. Long packets occupy processing resources longer than short packets. As a result, packets naturally leave the FPGA in a different order than they entered. For many applications, this is unacceptable.
Why packet ordering matters
One possible solution is to introduce a reordering buffer that restores the original packet sequence before packets leave the FPGA.
While conceptually straightforward, this solution introduces several new problems. Large buffers consume valuable Block RAM and UltraRAM resources.
Additional bookkeeping logic increases design complexity. Most importantly, buffering inevitably adds latency.
In low-latency networking systems, every additional clock cycle matters. Some applications simply cannot tolerate packet reordering at all. Telecommunications infrastructure depends on deterministic packet delivery to minimize jitter and maintain service quality.
Intrusion detection systems and firewalls often reconstruct packet streams before inspection. Receiving packets out of order increases CPU load and may even open opportunities for traffic evasion attacks.
Financial trading platforms rely on highly deterministic latency where even microsecond-level variations can directly impact trading performance.
At 400G, packet ordering is no longer an implementation detail. It becomes a system requirement.
Load balancing is never perfect
Even if packet ordering could be solved, another challenge remains. Traffic itself is rarely balanced.
Hash-based load balancing assumes that traffic consists of many independent flows distributed evenly across all processing pipelines. But real data centres rarely behave this way.
Large storage transfers. AI training traffic. Database replication. NVMe-over-TCP. Backup operations. These workloads often generate a handful of extremely large flows – commonly referred to as elephant flows.
Because hash-based distribution relies on flow entropy, packets from the same flow need to end up in the same pipelines to be processed with the same context and potentially distributed to the same CPU cores in the host system (flow-aware packet processing). Parsing and hashing must run at line-rate; otherwise it is not possible to perform this type of distribution, which leads to round-robin or similar distribution schemes.
When flow-aware distribution implemented correctly, high-volume ‘elephant’ flows end up pinned to a single pipeline, leaving other resources idle while the hot pipeline drops traffic. Unlike thousands of small network sessions, an elephant flow cannot simply be divided across multiple pipelines without breaking packet ordering.
The consequence is predictable. One pipeline becomes saturated. The remaining pipelines remain partially idle.
Overall FPGA utilization may appear low, yet packets are already being dropped because one processing path has reached its limit.
The system fails long before reaching its theoretical aggregate throughput.
The real problem is architectural
By this point, a pattern begins to emerge. Increasing clock frequency is constrained by FPGA physics. Replicating pipelines dramatically increases resource utilization. Multiple pipelines introduce packet-ordering challenges. Hash-based distribution struggles with real-world traffic.
None of these problems are bugs. They are natural consequences of scaling architectures originally designed for much lower bandwidths. This realization leads to an important question.
What if increasing throughput did not require replicating packet-processing pipelines at all?
Instead of processing one packet at a time across many parallel engines, could an FPGA process multiple packets simultaneously within a single, deterministic pipeline?
That question became the starting point for an entirely different architectural approach – one that abandons traditional pipeline replication in favor of multi-packet processing.
Key Takeaways
- The primary limitation of 400G FPGA networking is architectural, not physical connectivity.
- Replicating 100G pipelines introduces hidden costs in routing, memory usage, timing closure, and packet ordering.
- Real-world traffic patterns, especially elephant flows, expose the limitations of hash-based load balancing far earlier than synthetic benchmarks suggest.
- Scaling future FPGA networking architectures requires rethinking packet processing itself – not simply adding more pipelines.
Rethinking packet processing: A different architectural approach
By this point, the limitations of traditional scaling strategies become increasingly clear.
Increasing clock frequency quickly reaches the physical limits of FPGA technology. Replicating packet-processing pipelines consumes enormous hardware resources while introducing routing complexity, packet reordering and load-balancing challenges.
The obvious question becomes:
What if the problem is not how many pipelines we build, but how we think about packet processing itself?
The architectural decomposition described here is conceptually aligned with modern programmable data-plane models such as P4, which separate packet processing into parser, match-action, and deparser stages. The focus of this article, however, is one level lower: how these abstractions can be implemented efficiently in FPGA hardware at 400G and beyond.
Instead of asking how to process one packet faster, perhaps we should ask a different question:
Can an FPGA process multiple packets during a single clock cycle without replicating the processing pipeline?
Thinking in width instead of frequency
For decades, improving networking performance generally meant increasing clock frequencies or adding more parallel processing engines. Both approaches eventually hit practical limits inside programmable logic.
A third option exists.
Rather than scaling performance through frequency or replication, throughput can be increased by widening the datapath itself.
Instead of designing a pipeline that processes a single packet every clock cycle, the pipeline is designed to process several packets simultaneously within the same cycle. Conceptually, the processing pipeline still behaves as a single deterministic engine.
Internally, however, it operates on an extremely wide data word capable of carrying multiple independent packets at once.
The philosophy can be summarized in one sentence:
Scale throughput by processing width rather than processing frequency.
This idea sounds deceptively simple. Its implementation is anything but.
A single pipeline. Multiple packets.
This is the design philosophy behind the DYNANIC architecture.
Rather than dividing incoming traffic across multiple independent packet-processing engines, DYNANIC keeps a single logical pipeline and increases its processing width.
Internally, the architecture operates on a 2048-bit-wide datapath, allowing multiple packets to be processed during the same clock cycle.
Instead of four independent 100G pipelines, there is one deterministic processing pipeline capable of handling the equivalent throughput.
This immediately changes several characteristics of the system. Packet ordering is naturally preserved because packets continue travelling through a single processing path.
There is no need to reconstruct packet order using large reordering buffers. Metadata remains synchronized throughout the pipeline.
State management becomes significantly simpler. Most importantly, the architecture avoids duplicating complex processing blocks that would otherwise consume valuable FPGA resources.
It is important to emphasize that this is not merely a wider bus. It requires redesigning almost every processing stage.
The real challenge begins here
Processing multiple packets simultaneously sounds straightforward until one considers how packets actually arrive. Ethernet packets have variable lengths. Some are only 64 bytes long. Others approach the maximum transmission unit.
As packets arrive continuously from the Ethernet MAC, they become densely packed onto the internal datapath. Packet boundaries rarely align with convenient word boundaries.
A single 2048-bit data word may contain:
- the end of one packet,
- two complete packets,
- and the beginning of yet another packet.
Every clock cycle therefore presents a completely different alignment. Unlike traditional architectures, there is no guarantee that packet headers begin at predictable bit positions.
This creates one of the most challenging aspects of multi-packet processing.
The pipeline must continuously detect packet boundaries, identify headers and maintain packet context regardless of where individual packets begin or end.
Nothing is naturally aligned.
Every processing block must be redesigned
Once multiple packets occupy the same datapath, conventional packet-processing IP blocks are no longer sufficient.
Every stage of the processing pipeline must become multi-packet aware.
The parser must identify and decode multiple protocol headers simultaneously, regardless of their alignment within the datapath.
Classification logic must perform multiple parallel lookups while ensuring that memory resources remain efficiently shared and free from contention.
Packet modification engines must edit headers independently without corrupting neighbouring packets that occupy the same data word.
Metadata generation becomes considerably more complex because each packet carries its own processing context despite sharing the same physical datapath.
Even seemingly simple functions such as checksum updates or statistics collection require fundamentally different implementations.
The challenge is therefore not widening the bus itself. The challenge is redesigning an entire networking architecture around that wider bus.
Working with the FPGA instead of against it
One interesting aspect of modern FPGA devices is that their internal hardware has gradually evolved in the same direction. Recent FPGA families increasingly expose wide internal interfaces for high-performance subsystems.
PCI Express Gen5 controllers exchange data using very wide internal buses. 400G Ethernet MACs deliver data in wide parallel words. HBM memory controllers are optimized for massive parallel bandwidth rather than extremely high clock frequencies.
In other words, modern FPGA silicon increasingly favors width over frequency. Designing packet-processing architectures around wide datapaths therefore aligns naturally with how the hardware itself operates.
Rather than continuously converting between narrow and wide interfaces, a wide-pipeline architecture minimizes unnecessary adaptations and allows data to flow more efficiently through the device. This reduces latency, simplifies integration, and improves overall resource utilization.
Looking beyond 400G
Perhaps the most interesting consequence of this architectural philosophy is its scalability.
Traditional approaches typically require redesigning the architecture every time network bandwidth doubles. A pipeline optimized for 100G becomes four pipelines for 400G. Eight pipelines for 800G. Eventually, complexity grows faster than throughput.
A width-oriented architecture follows a different path.
As FPGA technology evolves and wider interfaces become available, throughput increases primarily by extending processing width rather than multiplying independent processing engines.
While future generations will certainly introduce their own engineering challenges, the underlying architectural principles remain unchanged.
In that sense, multi-packet processing is not simply a solution for today’s 400G networking. It represents an architectural direction for future programmable networking platforms.
Key Takeaways
- Scaling throughput by increasing datapath width fundamentally changes the packet-processing architecture.
- Multi-packet processing preserves deterministic packet ordering without relying on complex reordering buffers.
- The real innovation lies not in using a wider datapath, but in redesigning every processing stage to operate on multiple packets simultaneously.
- Modern FPGA devices increasingly favor wide internal interfaces, making this architectural approach a natural fit for future generations of programmable networking.
Engineering principles for the 400G era
Throughout the evolution of programmable networking, one observation has remained remarkably consistent:
Successful architectures rarely emerge from simply adding more hardware. They emerge from questioning the assumptions that previous generations took for granted. The transition to 400G represents exactly such a moment.
It is tempting to view higher bandwidth as a problem that can be solved by increasing parallelism, duplicating processing engines or pushing FPGA devices harder. In practice, these approaches quickly expose architectural limitations that become increasingly difficult to manage as systems grow in complexity.
After examining the challenges of traditional scaling approaches, several engineering principles begin to emerge. While they are presented here in the context of 400G networking, they are likely to remain relevant well beyond the current generation of programmable hardware.
First principle – Scale useful work, not clock frequency
Historically, increasing performance often meant increasing operating frequency.
For FPGA-based networking, this approach offers only limited headroom.
Modern devices already operate close to practical timing limits for large packet-processing designs. Every additional megahertz comes at the cost of increasingly complex routing, longer implementation cycles and reduced design portability.
A more sustainable strategy is to increase the amount of useful work completed during each clock cycle.
Instead of asking “How can we make the FPGA run faster?”, the more relevant question becomes:
“How can we accomplish more work without increasing the clock?”
This shift in perspective fundamentally changes how packet-processing pipelines are designed.
Second principle – Preserve determinism by design
Every time packets are divided across independent processing paths, the architecture inherits a new responsibility.
Eventually, those packets must be synchronized again. Additional synchronization stages introduce buffering. Buffering introduces latency. Latency introduces uncertainty.
As networking applications increasingly depend on deterministic behavior rather than simply maximum throughput, preserving packet ordering throughout the processing pipeline becomes an architectural objective rather than a post-processing task.
The most elegant synchronization mechanism is often the one that never becomes necessary.
Designing architectures that naturally preserve packet order reduces complexity throughout the entire system.
Design for determinism instead of correcting non-determinism.
Third principle – Complexity should solve the right problem
Every high-performance architecture contains complexity. The important question is where that complexity is placed.
Duplicating processing pipelines often appears straightforward during the early stages of a project. Over time, however, the architecture accumulates additional mechanisms for synchronization, buffering, load balancing, and state management.
These mechanisms are not part of the packet-processing problem itself.
They exist primarily to manage the side effects of architectural replication. An alternative approach is to invest engineering effort directly into more capable processing blocks.
A parser that understands multiple packets. Lookup/matching engines capable of parallel operation. Metadata management designed around wider datapaths.
Although these components are individually more sophisticated, they eliminate much of the surrounding infrastructure that replicated architectures eventually require.
Complexity is not removed. It is focused where it contributes directly to useful packet processing.
Fourth principle – Design for the next bandwidth increase
Networking history offers one remarkably consistent lesson.
Bandwidth always grows. Clock frequency grows much more slowly.
Architectures optimized exclusively for today’s performance target often require significant redesign when the next Ethernet generation arrives.
A more sustainable philosophy is to develop architectural principles that remain valid as bandwidth continues increasing.
Whether future networking platforms operate at 400G, 800G or beyond, the fundamental challenge remains the same:
Move more data.
Without multiplying architectural complexity at the same rate.
Scalable architectures are therefore those that require the fewest conceptual changes as technology evolves.
Fifth principle – Let the hardware work with you
One of the most interesting developments in recent FPGA generations is that the silicon itself increasingly favors parallel data movement.
High-performance Ethernet MACs exchange wide data words. PCI Express controllers expose wide internal interfaces. HBM architectures prioritize bandwidth over clock frequency.
Modern FPGA devices are already optimized for moving large amounts of data in parallel.
Packet-processing architectures that embrace these characteristics align naturally with the hardware they execute on.
Rather than continuously adapting between narrow and wide interfaces, they allow data to flow through the device using the mechanisms the FPGA already provides most efficiently.
Good architecture rarely fights the underlying hardware. It exploits it.
A different way to think about packet processing
Perhaps the most important conclusion is not a technical one.
For many years, packet-processing performance was measured primarily by throughput.
Today, throughput alone is no longer sufficient.
Architectural efficiency.
Deterministic behavior.
Scalability.
Resource utilization.
Maintainability.
These characteristics increasingly define whether a programmable networking platform remains practical as bandwidth continues growing.
The transition to 400G therefore represents more than another performance milestone.
It encourages FPGA architects to reconsider some of the assumptions that have guided packet-processing design for decades.
And that may ultimately prove more important than the bandwidth increase itself.
Key Takeaways
- The most scalable architectures increase useful work per clock cycle rather than relying solely on higher frequencies or additional processing pipelines.
- Deterministic behavior should be an architectural property – not something restored later through additional synchronization.
- Engineering complexity is unavoidable; the goal is to invest it where it directly improves packet processing rather than managing the side effects of the architecture itself.
- The best networking architectures evolve alongside FPGA technology instead of fighting its physical constraints.
Conclusion – Beyond 400G: A shift in architectural thinking
The transition to 400G Ethernet is often described in terms of bandwidth.
More gigabits. More lanes. Faster transceivers. Larger FPGA devices.
While all of these advances are important, they do not capture the fundamental change taking place inside programmable networking systems.
The real challenge of the 400G era is no longer transporting packets across the network.
It is processing them efficiently once they arrive.
As FPGA technology continues to evolve, adding more logic or increasing interface bandwidth alone will no longer guarantee better system performance.
Architectural efficiency is becoming the defining factor.
How data moves through the device. How processing stages interact. How complexity scales as bandwidth doubles.
These questions increasingly determine whether an architecture remains practical beyond a single product generation.
Architecture will define the next generation
Every major technological transition forces engineers to reconsider assumptions that once seemed obvious.
For programmable networking, one of those assumptions has been that higher throughput naturally requires more processing pipelines.
At 400G, that relationship begins to break down.
Future packet-processing architectures will likely be defined less by the number of pipelines they contain and more by how effectively they utilize each clock cycle.
Rather than continuously multiplying processing engines, the focus shifts toward increasing useful work, preserving deterministic behaviour and aligning packet-processing architectures with the capabilities of modern FPGA devices.
This is not simply an optimization for today’s networking speeds.
It is an architectural direction that will remain relevant as the industry moves toward 800G and beyond. While the 100G era was about “more of the same,” the post-400G world is about “better architectural utilization.”
Lessons learned
Looking back at the evolution of programmable networking, one lesson becomes clear.
Bandwidth increases are inevitable. Architectural complexity is optional. Every unnecessary synchronization stage. Every duplicated processing block. Every workaround introduced to compensate for architectural limitations becomes part of the system engineers must maintain for years to come.
Conversely, architectures that preserve determinism, minimize unnecessary data movement and scale through efficient parallelism establish a stronger foundation for future generations.
The challenge is no longer building faster packet processors. It is building architectures that continue scaling as networking itself evolves.
Our perspective
The principles discussed throughout this article have directly influenced the design of the DYNANIC architecture.
Rather than viewing 400G as simply another bandwidth milestone, we approached it as an opportunity to rethink packet processing around deterministic execution, efficient use of FPGA resources and architectures designed to evolve with future hardware.
We do not see this as the final answer to programmable networking. We see it as one possible direction – one that reflects where modern FPGA technology is already heading.
As networking continues advancing, we believe success will depend less on how much hardware can be added and more on how intelligently that hardware is used.
Ultimately, sustainable performance is not achieved by building larger packet-processing systems. It is achieved by building better architectures.