Open-Sourcing the CoreNIC Firmware

Programmability is everywhere! Gone are the days when hardware components would be entrusted with a single task. Nowadays, even network cards run low-level software, also known as “firmware,” usually distributed as binary images by vendors. But what if we were to open these programs to the users? Since we love to create new opportunities, Netronome just published the source code for the CoreNIC firmware, used with the Agilio SmartNICs. This write-up provides some context and details about this release, because opening our firmware is something important – not only for Netronome, but also for the community at large, on many levels.

The CoreNIC Firmware

Let's answer one question before we start: What did we release, exactly?

The repository published by Netronome contains the source code for the CoreNIC firmware, which controls the Network Flow Processor (NFP) at the heart of our Agilio SmartNICs. It supports a number of features including (but not limited to) SR-IOV and basic BPF offload.

Figure: Diagram of the primary components of the CoreNIC software architecture. Please refer to the documentation from the GitHub repository for details.

How does it work? At a high level the NIC firmware is logically separated into two component groupings: control plane and programmable datapath. The control plane configures a pool of “workers” and passes them a list of “actions” to execute in response to receiving network packets on a given ingress queue. These actions can be optimized packet processing primitives (RSS queue selection, VLAN tagging, etc.) or more complex tasks such as running an offloaded BPF program on the packet.

The repository also contains documentation about the firmware, which will prove an invaluable resource to those willing to understand the internals of manycore packet processing devices. It also features step-by-step instructions and guidance for compiling the code into a binary that can be loaded onto the SmartNICs. Remember that the code we uploaded is the very one we use to produce the firmware images we distribute: It is fully functional, optimized, and 100% ready to accelerate your traffic!

Going Open Source

The decision to open source the CoreNIC firmware was not taken overnight. It falls in line with our long-standing strategy to publish as much of our software as we can. In the past, we have published our nfp driver and had it merged upstream in the Linux repository. We also released software tools, as we did for bpftool, the go-to utility for managing BPF objects on the system, or more recently for Libkefir, a library we use to help translate network filtering rules into BPF programs.

Why do we place our trust in open source? First of all, we believe it is essential for customers and partners to be able to look into the code they deploy on their infrastructure. My colleague, Edwin Peer, will give a presentation on the topic at the next LinuxConf [ZA]. Because network processors have a role of paramount importance in any modern data center architecture and are sometimes entrusted with cipher keys, it is essential that people can access the source code used to program such devices.

Another advantage of making the code available to the public is the new relationship established between the engineers and the users. We believe that when the sources come under scrutiny, engineers feel even more responsible for the quality of their code, and give the best of themselves to produce flawless software. And the very few bugs they might miss may well be caught and reported by external eyes!

But opening the source code leads to more than increased quality on our side. Network developers all around the world can now study the documentation, and play and hack the CoreNIC code. We are extending the community we have been building over the last few years, centered around the Open-NFP platform – as always, feel free to come aboard. We are eager to see what kind of research and collaboration this may lead to. Remember that there are developers who implement their own, optimized drivers for high-performance switching for example. Opening and documenting the CoreNIC firmware should provide tremendous assistance to such projects!

This is a Big Deal

Contrary to drivers or software utilities, it is worth noting that open sourcing the firmware running directly on the devices is not a common decision within the data center NIC market. Overall, there is a certain tendency that the closer you get from the hardware, the less control you manage to retain on your infrastructure. Understanding, debugging or even validating closed-source firmware is next to impossible. This is an additional reason why we wanted to publish it: It felt important to give control to our customers, even at the lowest level. To the best of our knowledge, Netronome is the only actor in the field who has published code for modern SmartNICs at this time. We hope that this will encourage other companies to do the same. In the meantime, the publishing was even acclaimed by the Linux networking maintainer: “This is a big deal,” he emphasized. Do we feel proud of our release? Absolutely!

We also believe that open sourcing software helps the industry at large, by providing reference implementations that developers can copy and improve. If there is no open NIC firmware available in the wild, everyone willing to support a new device has to start from scratch again! We certainly do not claim that CoreNIC should be considered a universal reference, but we do hope that the best parts of the software can prove helpful to others. The BSD 2-clause license used for the code makes it possible to reuse it in many projects.

Working Together

As of this writing, only a few days have passed since we released the firmware, so we cannot draw any kind of conclusion yet. But the early reactions on social media were extremely positive. Developers started to analyze and comment the code, in particular the parts related to BPF offload, or even to contribute! This all sounds promising.

Obviously, everyone is encouraged to have a look at our CoreNIC firmware on its GitHub repository. As a reminder, it is absolutely not necessary to build it to use the Agilio SmartNICs, as we regularly build binaries and distribute them through the linux-firmware project, which means the compiled firmware is already available from all major Linux distributions. But now we can also work together, as a community, to review, improve, and optimize the code that processes all our packets. Let’s go fast.