Linux kernel WiFi stack basics
or how my sanity was cured through callbacks
In my first post I mentioned that I would dive into the Linux kernel details on BIP and how it works in the Linux kernel WiFi stack. To do this we need to have an understanding of the architecture of the Linux kernel WiFi stack.
The following article is based on the 6.6.18 LTS kernel release1, and the hostap_2_10 tag of the hostapd/wpa_supplicant source code2.
Overview
The WiFi stack inside Linux consists of multiple modules running in userspace and kernelspace. Together, all of these modules implement the functionality as specified in the various IEEE 802.11 standards for the Station Management Entity (SME) and the MAC Layer Management Entity (MLME). The latest approved and rolled-into-one version of 802.11 is the 2020 revision.3
A rough outline of the major modules in the Linux WiFi stack is shown in the following diagram. These modules are described in the following sections.
Note that this diagram is showing the control plane. The data plane is not as complicated or extensive as the control plane.
hostapd and wpa_supplicant
The hostapd and wpa_supplicant userspace daemons implement the SME and parts of the MLME as defined in the 802.11 standard. The hostapd and wpa_supplicant daemons support a wide variety of different chipsets and interfaces through many different methods, including netlink (nl80211), wireless extensions (WEXT), as well as platform drivers (NDIS, OpenBSD), and chipset specific drivers (Atheros).
The overall architecture of hostapd and wpa_supplicant is nicely summarised in the online documentation here. At the core of both daemons is an event loop which hooks into other subsystems within the program. Events are injected into the event loop from control interfaces (wpa_cli), from various different state machines (WPA, EAPOL etc.), from drivers events (nl80211, NDIS etc.) as well as from other inputs.
For the purpose of this discussion I’ll be looking at ‘nl80211’, which is the driver used to interface with the cfg80211 kernel module.
Communication between userspace and kernelspace for the nl80211 driver is via netlink sockets, which are Linux specific socket types dealing with control and status information. These sockets are bi-directional between kernelspace and userspace.
nl80211 driver interface
Diving into the source/drivers/driver_nl80211.c file, we find a very long and moderately complicated piece of code. The key part of this code which is interesting to me is on line 12114, which is a structure containing many callback functions. This is the interface which the higher level authenticator and supplicant modules use to communicate with the cfg80211 kernelspace module via nl80211, which is heavily used inside the driver_nl80211.c file. A small sample of the driver defined callbacks is shown in the following screensnip.
The callbacks registered by the driver support configuration and control, as well as SME/MLME operations.
Most of the callbacks defined in this file follow the same pattern: take in some arguments, allocate a netlink buffer, fill in the buffer, send to the kernel, free the buffer. The return path for events from the kernel (stimulus from outgoing netlink messages or from the hardware itself) is discussed in the following section.
nl80211 driver events
Given the asynchronous nature of the netlink message interface, the return path for the response messages (such as authentication responses, probe responses etc.) is via a second code path (src/drivers/driver_nl80211_event.c).
The interesting area inside this source file is around line 2858, with the ‘do_process_drv_event’ function. This is the main receive loop which parses netlink events and calls out to different modules within the rest of the system.
…
The nl80211 driver interface is the userspace to kernel direction, the nl80211 event interface is the kernel to userspace direction.
By the way, do_process_drv_event is called by process_global_event, which itself is registered as a callback using the nl_cb_set function which is part of libnl.
cfg80211 kernel module
The cfg80211 kernel module is the first layer on the kernel side of the WiFi stack, and takes inputs from userspace to execute commands (such as get scan results, start AP etc.) and set attributes (such as MAC addresses, encryption keys etc.)
This module has a set of callbacks registered via the cfg80211_ops array (include/net/cfg80211.h), a screensnip of which can be seen below.
These callbacks are registered by either the mac80211 kernel module (SoftMAC implementations), or by a device specific driver (FullMAC implementations).
The registration of the mac80211 module into these callbacks is in net/ma80211/cfg.c.
An example of a FullMAC driver - the Quantenna FullMAC driver - registering these callbacks is in drivers/net/wireless/quantenna/qtnfmac/cfg80211.c.
The callbacks which are registered with cfg80211 are called from many places, and in the example I am discussing (hostapd/wpa_supplicant using nl80211), the file net/wireless/nl80211.c is the ‘other end’ of the netlink socket as used by the userspace daemons.
If we take an example from wpa_supplicant, when it sends the ‘NL80211_CMD_AUTHENTICATE’ command via netlink, using the wpa_driver_nl80211_authenticate function from the src/drivers/driver_nl80211.c file, the corresponding logic which executes in kernelspace (after various validations are done) is in the nl80211_authenticate function in the net/wireless/nl80211.c file.
mac80211 kernel module
The mac80211 kernel module registers callbacks with cfg80211 as described in the previous section. A screensnip of this code is shown below.
On the lower end of the mac80211 kernel module the callbacks (struct ieee80211_ops) which the device specific driver registers against are in the include/net/mac80211.h file, line 4273 as per the screensnip below.
The struct ieee80211_ops is filled in by the hardware specific driver, for example, with the Ath11k driver, this is how the structure is filled in:
SoftMAC? FullMAC?
At a high level, SoftMAC uses the facilities of the mac80211 kernel module to implement parts of the 802.11 MAC - that is - software implements various pieces of functionality. The areas implemented by mac80211 include software based crypto routines, AMSDU and AMPDU aggregation, beacon frame generation, power save buffering and more.
Given the extensive and robust functionality in the mac80211 module, many chipset vendors opt to implementing a SoftMAC solution as it gives a significant time advantage during development and bringup. However, the performance of the final WiFi device using SoftMAC is more reliant on the host CPU running the mac80211 module than a FullMAC device.
FullMAC on the other hand are systems where major blocks of the 802.11 MAC are offloaded into the chipset or chipset firmware of the given device.
In terms of the Linux kernel modules, SoftMAC uses cfg80211, mac80211 and a chipset specific driver with callbacks into mac80211. FullMAC on the other hand uses cfg80211 and a chipset specific driver with callbacks into cfg80211 - ie, bypassing mac80211. Therefore FullMAC drivers lose the benefit of the significant functionality within the mac80211 kernel module - this functionality must be implemented on the device firmware/hardware itself.
SoftMAC and FullMAC devices
The following table shows a (non-comprehensive!) sample of both SoftMAC and FullMAC drivers. Note that the IPW2X00 Intel chipsets are marked FullMAC* as they seem to use an early version of mac80211 (see config LIBIPW)
Conclusions
This article has skimmed the surface of the high level architecture and dived into some of the implementation details of the Linux WiFi stack. We see that function callbacks are heavily used in both userspace and kernelspace software, giving flexibility for different driver architectures at userspace (nl80211, WEXT, NDIS etc.) as well as in kernelspace (SoftMAC vs. FullMAC) to support a wide variety of hardware platforms.
My next article will dive into the specifics of BIP as it is implemented within the Linux WiFi stack, and I’ll try and verify the potential attack vectors as outlined in my first BIP article.
As usual, please let me know if you see any errors or major gaps in this article. I’m always open to constructive feedback.
Linux Stable source code online git tree, tag v6.6.18 https://gitlab.com/linux-kernel/stable/-/tree/v6.6.18?ref_type=tags
hostapd source code online git tree, tag hostap_2_10 https://w1.fi/cgit/hostap/tree/?h=hostap_2_10
IEEE Standard for Information Technology — Telecommunications and Information Exchange between Systems Local and Metropolitan Area Networks — Specific Requirements.
Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications (IEEE Std 802.11™️‐2020) - https://ieeexplore.ieee.org/document/9363693
Thank you for providing the high-level architecture of the Wi-Fi stack.
Could you please share your understanding of the data path flow from user space -> Mac80211 -> driver -> firmware?
Hi Richard,
Thank you very much for these clear explanation on these topics.
Its really helpful and much appreciated the effort you put it into this.
Thank you