Episode Details

We explore whether a BSD can replicate Cisco router performance; RETGUARD, OpenBSDs new exploit mitigation technology, Dragonfly’s HAMMER2 filesystem implementation & more!

This episode was brought to you by

Headlines

Can a BSD system replicate the performance of a Cisco router?

• Short Answer: No, but it might be good enough for what you need

Traditionally routers were built with a tightly coupled data plane and control plane. Back in the 80s and 90s the data plane was running in software on commodity CPUs with proprietary software. As the needs and desires for more speeds and feeds grew, the data plane had to be implemented in ASICs and FPGAs with custom memories and TCAMs. While these were still programmable in a sense, they certainly weren't programmable by anyone but a small handful of people who developed the hardware platform. The data plane was often layered, where features not handled by the hardware data plane were punted to a software only data path running on a more general CPU. The performance difference between the two were typically an order or two of magnitude. source

Except for encryption (e.g. IPsec) or IDS/IPS, the true measure of router performance is packets forwarded per unit time. This is normally expressed as Packets-per-second, or PPS. To 'line-rate' forward on a 1gbps interface, you must be able to forward packets at 1.488 million pps (Mpps). To forward at "line-rate" between 10Gbps interfaces, you must be able to forward at 14.88Mpps.
Even on large hardware, kernel-forwarding is limited to speeds that top out below 2Mpps. George Neville-Neil and I did a couple papers on this back in 2014/2015. You can read the papers for the results.

However, once you export the code from the kernel, things start to improve. There are a few open source code bases that show the potential of kernel-bypass networking for building a software-based router. The first of these is netmap-fwd which is the FreeBSD ip_forward() code hosted on top of netmap, a kernel-bypass technology present in FreeBSD (and available for linux). Full-disclosure, netmap-fwd was done at my company, Netgate. netmap-fwd will l3 forward around 5 Mpps per core. slides

The first of these is netmap-fwd which is the FreeBSD ip_forward() code hosted on top of netmap, a kernel-bypass technology present in FreeBSD (and available for linux). Full-disclosure, netmap-fwd was done at my company, Netgate. (And by "my company" I mean that I co-own it with my spouse.). netmap-fwd will l3 forward around 5 Mpps per core. slides

Nanako Momiyama of the Keio Univ Tokuda Lab presented on IP Forwarding Fastpath at BSDCan this past May. She got about 5.6Mpps (roughly