Let’s take for example just a simple L3 forwarding case: there are two essential lookups, one for the route to yield the next-hop, and another for the link-layer address of the next-hop. When you have thousands of routes in hundreds of VRF contexts, you have no semblance of assured data availability (even if you were to implement some intelligent pre-fetching).

There is a misconception (which befuddles me) that multi-threading only benefits multi-tasking at an OS level; this is in fact far from the truth. Multi-threading actually benefits packet processing environments in a much larger scale, because you cannot avoid latency specifically in run-to-completion models.

Conversely, if you presume that there is never any idle cycle, then a 3 GHz core can conceptually outperform 3 individual 1 GHz cores because a 3 GHz core could finish the job in a third of the time.

Clearly, that is never the case because in packet processing that today involves upwards of 10 lookups/packet with hundreds of thousands, if not millions of entries per table, controlling (or predicting) pipeline behavior on a cycle-by-cycle basis is impossible.

The one postulate I can agree with, is that the demands on all cores or threads are not symmetrical: this is exactly a source of advantage for a hardware thread with a single core where even if all threads are executing the same instruction stream, they are not executing the exact same instruction (or operating on the same object) at a given cycle.

You can also surely run a lightweight run-to-completion model with a multi-threaded core, and certainly is the better way to accomplish ideal performance. Linux is horrible as a packet processing framework, so it generally doesn’t behoove associating it with any performance or throughput related discussion.

I would be curious to see a multi-core vendor backup the hypothesis that the run-to-completion model somehow allows them to reach 100% pipeline efficiency in packet processing environments.

-TG

On the data plane, a much simpler executive can be used. In a run-to-completion model an individual core (which I’ll call a Network Acceleration Engine, NAE) can poll for packets(incoming our outgoing), perform the necessary processing, queue for dispatch, and return to polling state. In this case, multiple threads are not necessarily going to make the NAE run more efficiently. If the NAE is I/O or bus bound, the system performance will be poor regardless of the processing power. Likewise cache stalls will kill throughput performance, even if a second thread is available to prevent the core from going idle.

I agree that multithreading is a brilliant and useful concept for most OS environments. But in the particular case of network packet processing, the demands on all cores are not symmetrical.

All multi-threaded processors in fact support threading on a multi-core architecture: the duality in choice allows native selection and concurrent use of both ILP and TLP. The original article appears to address core performance within a single-core, not across multiple cores. Clearly, one could make a collective argument about “Disadvantages of multi-core in next generation packet processing” and that would also be jaded.

I encourage readers to visit http://www.cs.washington.edu/research/smt/ for an unbiased approach to multi-threading, in the specific context of where it could be a favorable design choice in concordance with multi-core selection.

Multi-threading is not panacea and neither is multi-core; multi-threading can be a distinct benefit in a variety of use cases. Many modern core architectures (MIPS-MT, POWER5, UltraSPARC, Nehalem etc.) and many vendors (MIPS, NetLogic, IBM, Sun Microsystems, Intel) all support multi-threading simultaneously with multi-core.

Multi-core is here to stay. And so is multi-threading.

You need not pick between the two: you can have both, have your cake and eat it too.

-TG

There are some other factors which reduce deterministic performance with multi-threading. First, if the processor hardware decides when to switch thread, then software developer does not control when a thread is executed and for how long, before hardware switches the execution to another thread. Second, even if hardware tries to run all the threads at the same time, these threads also compete for the same execution units. It is not always clear to the software developer how the hardware allocates the execution units among multiple threads being executed. As a result, performance determinism gets impacted.

Performance determinism is an important performance attribute for packet processing. Throughput is not the only important factor.