Libkefir: All Your Rules in One Bottle

Netronome is releasing libkefir, a library for converting network filtering rules into BPF programs, in a simple and efficient way! But how does it work? What is it for? Fear not, for all your questions will [hopefully!] be addressed in this blog.

All the Filters in the World

When it comes to filtering packets on a Linux server, many options are available. Firewalls (iptables/nftables) and Traffic Control (TC) filtering are different hooks that can be used in the networking stack. Higher up on the system, filtering on sockets is done for programs like tcpdump or Wireshark. In the opposite direction, directly on the hardware, some NICs support hardware “n-tuple” filters set up with ethtool.

All of these mechanisms come with pros and cons. They offer a certain degree of flexibility, but in the end they all come down to comparing packet headers with rule patterns, and applying a given action when the result matches. It may be very difficult to use them for complex custom use cases, or to support new protocols for example.

The recent rise of eBPF, of course, has the world of Linux filtering shaken upside down. Now filters are fully programmable. XDP programs, at the driver level, become super-efficient. Hardware offload capabilities can be used in full cooperation with the kernel, allowing flexible programs to be run directly on SmartNICs such as Netronome's Agilio products. BPF is fast, and powerful. Let's just switch everything to BPF!

One Filter to Rule Them All

Many tools are actually moving to BPF. Take bpfilter, for example: This is a work in progress that implements an alternative back-end to Netfilter. Based on BPF, it should remain compatible with the iptables utility, and be able to translate the legacy rules into BPF bytecode. But moving to a new framework is never that easy. Switching to BPF often comes at a cost: It is more complex to write BPF programs, even as C code, than simple command lines as for iptables or TC. Furthermore, in spite of the enhanced performance, moving a complete filtering setup to BPF involves translating large sets of rules, which may represent considerable engineering efforts.

At the latest FOSDEM event a couple of months ago I spoke about some leads for convergence between the different filtering mechanisms on Linux. One of them, only at the early stages of development at that time, was a library that could be just the solution to this problem: A set of functions designed to help convert network filtering rules into BPF programs, with all the performance – but no sweat.

All Your Rules in One Bottle

The recipe is simple: Take that library mentioned at FOSDEM. Add some more hours of development, an ounce of bug chasing, a handful of documentation pages, stir. Mix with a cup of tests, dress with a pack of git commits, add a pinch of salt. And voilà... We are now releasing libkefir!

Libkefir’s logotype

The name comes from KErnel FIltering Rules (also, kefir is good). The C library is designed to work with network filters made of rules. These rules can be expressed in a variety of ways:

• It is possible to craft them “from scratch,” in a C application, by filling the struct kefir_rule exposed by the library. This allows for very fine customization of the rules.
• It is possible to load rules from expressions used with some other filtering mechanisms. For example, as of this writing, it is possible to load rules from the strings one would pass to configure:
  • TC flower filters
  • Ethtool hardware filters
• Syntax for additional tools should be added in the future. Libpcap filters or iptables rules are excellent candidates.

Once a filter has been created, the library offers functions to convert it into a C program (and possibly to dump the generated C code), then to compile that program into BPF bytecode, to load this BPF program into the kernel, and to attach it to a hook such as XDP.

Initial design diagram for the library – iptables rules and pcap-lib expressions not yet supported

C code generation, compilation to BPF, loading and attachment can all be done in a single function call. But of course, those tasks can also be dissociated. This makes the library flexible enough, for instance, to build a filter, convert it into a C program and handle the C code to the user, so that they can edit it as needed before compiling the program to BPF. Then it can be simply loaded into the kernel, ready to be attached by a later call to the library (or to the ip link utility). All kinds of workflows are possible!

Say you built a libkefir filter programmatically but want to save it for later use. You are covered; it is possible to export those filter objects as JSON and to restore them from a file afterwards. You want to change the values the library stores in a BPF map when loading the BPF filter? You can dump the equivalent command lines for updating the rules with bpftool, just edit and run. We tried to adapt the library to all your needs.

Give It a Try!

With libkefir, writing a tool to translate TC flower rules from standard input into a single BPF program should take but a few lines of code. And this is just one example. We feel confident that it will help various users move their filtering rules to new, fast, shiny BPF programs. Do you want to give it a try? The library is hosted on GitHub.

Feel free to compile it, to run some of the example applications. Then feel free to use it in your own applications, and to build software to create and manage filters. Do not hesitate to send us feedback, via GitHub issues or any other communication channel. We do hope you will enjoy libkefir!