03 - Misc (Character) Device

In this lab you will write a kernel module in Rust that exposes a character device to userspace. Character devices are one of the oldest abstractions in Unix: they appear as files under /dev and let userspace interact with the kernel through the familiar open, read, write, ioctl, and close system calls. The miscellaneous device subsystem is a lightweight layer on top of character devices that removes the need to allocate and manage a dedicated major number. Instead, every misc device shares major 10 and receives a dynamically assigned minor, making it the natural starting point for new drivers that do not belong to an established subsystem.

The lab uses the kernel::miscdevice module, which provides safe Rust wrappers around the C struct miscdevice and struct file_operations APIs. By the end of the lab you will have a working driver that userspace can open, control through ioctl commands, and exchange data with through read and write.

Objectives

Understand what miscellaneous devices are, how they relate to character devices, and how they are registered and unregistered from the kernel.
Understand how IOCTL numbers are structured — the type, nr, size, and direction fields — and how to define, dispatch, and decode them using the kernel::ioctl API.

Misc devices

The miscellaneous device subsystem is the simplest way to expose a character device node under /dev. Rather than allocating a dedicated major number, every misc device shares major 10 and receives a dynamically allocated minor. The kernel documentation for the subsystem lives at driver-api/misc_devices, and the Rust API is provided by the kernel::miscdevice module, whose C counterpart is include/linux/miscdevice.h.

Registration

A misc device is registered by creating a MiscDeviceRegistration<T> value. The generic parameter T is your driver type — the type that implements MiscDevice. Registration is done through MiscDeviceRegistration::register, which takes a MiscDeviceOptions struct (currently just a name) and returns a PinInit that must be stored in a pinned location for the lifetime of the device. Deregistration happens automatically when the MiscDeviceRegistration is dropped.

note

Pinning and PinInit are covered in detail further on in this guide.

use kernel::{
    c_str,
    miscdevice::{MiscDevice, MiscDeviceOptions, MiscDeviceRegistration},
    prelude::*,
};

#[pin_data(PinnedDrop)]
struct MyModule {
    #[pin]
    dev: MiscDeviceRegistration<MyDeviceInstance>,
}

impl kernel::InPlaceModule for MyModule {
    fn init(_module: &'static ThisModule) -> impl PinInit<Self, Error> {
        let opts = MiscDeviceOptions { name: c_str!("my-device") };
        try_pin_init!(Self {
            dev <- MiscDeviceRegistration::register(opts),
        })
    }
}

#[pinned_drop]
impl PinnedDrop for MyModule {
    fn drop(self: Pin<&mut Self>) {}
}

The minor number is always assigned dynamically (MISC_DYNAMIC_MINOR). After a successful register call, MiscDeviceRegistration::device returns a reference to the underlying struct device, which is useful for logging and devres allocations.

File permissions

MiscDeviceOptions currently only exposes the name field. The mode field of the underlying C struct miscdevice is not yet wired up, so the device node is created with the kernel default permissions (0600). Use a udev rule to set different permissions until this is addressed upstream.

Implementing `MiscDevice`

All driver behaviour is defined by implementing the MiscDevice trait and annotating it with #[vtable]. The only required method is open; all others have default implementations that return ENOTTY (or do nothing for release).

use kernel::{
    alloc::KBox,
    fs::File,
    miscdevice::{MiscDevice, MiscDeviceRegistration},
    prelude::*,
};

struct MyDeviceInstance {
    // per-open-instance state
}

#[vtable]
impl MiscDevice for MyDeviceInstance {
    type Ptr = KBox<Self>;

    fn open(_file: &File, _reg: &MiscDeviceRegistration<Self>) -> Result<KBox<Self>> {
        Ok(KBox::new(MyDeviceInstance {}, GFP_KERNEL)?)
    }

    fn release(device: KBox<Self>, _file: &File) {
        drop(device);
    }

    fn ioctl(
        device: <KBox<Self> as kernel::types::ForeignOwnable>::Borrowed<'_>,
        _file: &File,
        cmd: u32,
        arg: usize,
    ) -> Result<isize> {
        match cmd {
            MY_GET_STATUS => { /* ... */ Ok(0) }
            MY_SET_CONFIG => { /* ... */ Ok(0) }
            _ => Err(ENOTTY),
        }
    }
}

The associated type Ptr determines how per-open-instance data is owned and passed between callbacks. KBox<Self> is the most common choice, giving heap-allocated per-open state. Arc<Self> is appropriate when all open file handles should share the same data. The Ptr type must implement ForeignOwnable + Send + Sync.

Available callbacks

The #[vtable] macro only links a function pointer into the kernel's file_operations table when you actually provide an implementation. Providing a method for a callback you do not use wastes nothing.

Method	`file_operations` field	Notes
`open`	`open`	Required. Returns the `Ptr` stored as file private data.
`release`	`release`	Called when the last reference to the file is dropped.
`read_iter`	`read_iter`	Streaming read via `IovIterDest`.
`write_iter`	`write_iter`	Streaming write via `IovIterSource`.
`ioctl`	`unlocked_ioctl`	See `kernel::ioctl` for building command numbers.
`compat_ioctl`	`compat_ioctl`	32-bit userspace on 64-bit kernel. Defaults to `compat_ptr_ioctl` if `ioctl` is provided.
`mmap`	`mmap`	Receives a `VmaNew` to configure the mapping.
`show_fdinfo`	`show_fdinfo`	Writes extra lines into `/proc/<pid>/fdinfo/<fd>`.

32-bit compatibility

If your ioctl data structures have the same layout on 32-bit and 64-bit userspace — meaning no bare long, unsigned long, or pointer members — you do not need to implement compat_ioctl. The kernel will automatically use compat_ptr_ioctl as a fallback when ioctl is provided, which adjusts pointer-sized arguments and then calls your ioctl handler directly.

If your structures differ by ABI (e.g. you have a 32-bit pointer field represented as __u64 on 64-bit), implement compat_ioctl explicitly and handle the conversion there. See the kernel's guidance on structure layout for compat mode for the full list of problematic member types.

Module reference counting

The current Rust miscdevice abstraction does not set the owner field of file_operations. This means the kernel does not automatically increment your module's reference count when a file is opened, and rmmod can unload the module while file handles are still live.

BUG

This workaround can be removed once the owner field is wired up through a type-level ThisModule associated type, which is currently proposed upstream.

Until the upstream fix lands, the safest mitigation is to call try_module_get / module_put manually in open and release:

fn open(file: &File, reg: &MiscDeviceRegistration<Self>) -> Result<KBox<Self>> {
    // Prevent the module from being unloaded while this file is open.
    let ok = unsafe { kernel::bindings::try_module_get(kernel::THIS_MODULE.as_ptr()) };
    if !ok {
        return Err(ENODEV);
    }
    Ok(KBox::new(MyDeviceInstance {}, GFP_KERNEL)?)
}

fn release(device: KBox<Self>, _file: &File) {
    drop(device);
    unsafe { kernel::bindings::module_put(kernel::THIS_MODULE.as_ptr()) };
}

IO Control (IOCTL)

ioctl() is the primary way userspace applications communicate with device drivers for operations that don't fit the standard read/write model. The second argument to every ioctl() call — the command number — is not an arbitrary integer. It is a structured 32-bit value whose fields encode everything the kernel needs to route and validate the call.

The full specification lives in include/uapi/asm-generic/ioctl.h and is documented in the kernel's Ioctl Numbers and ioctl based interfaces pages. The Rust API is provided by the kernel::ioctl module.

Bit Layout

An IOCTL number packs four fields into a single 32-bit integer, from most-significant to least-significant:

Field	Width	Purpose
`dir`	2 bits	Data direction: none (0), write (1), read (2), read+write (3)
`size`	14 bits	`sizeof` the userspace data type (max 16 383 bytes)
`type`	8 bits	Magic number identifying the driver or subsystem
`nr`	8 bits	Sequential command number within that driver

Architecture differences

Most architectures use this layout, but some differ. PowerPC, for example, uses 3 bits for direction and 13 bits for size. Always check include/ARCH/ioctl.h for your target.

All four fields are combined by the internal _IOC(dir, type, nr, size) primitive:

// From include/uapi/asm-generic/ioctl.h
#define _IOC(dir, type, nr, size) \
    (((dir)  << _IOC_DIRSHIFT)  | \
     ((type) << _IOC_TYPESHIFT) | \
     ((nr)   << _IOC_NRSHIFT)   | \
     ((size) << _IOC_SIZESHIFT))

The Four Builder Macros

Rather than calling _IOC directly, drivers use four higher-level helpers. The naming convention is from the user's point of view — "write" means the user writes data to the kernel, "read" means the user reads data from the kernel:

Macro / Function	Direction	Use case
`_IO(type, nr)`	None	No data transfer; a pure command
`_IOW(type, nr, T)`	User → kernel	e.g., `SET_FOO` — user supplies a value
`_IOR(type, nr, T)`	Kernel → user	e.g., `GET_FOO` — user receives a value
`_IOWR(type, nr, T)`	Both	Bidirectional; often used to multiplex sub-commands

The Rust API

The kernel::ioctl module provides direct Rust equivalents. The key difference from C is that the data type is expressed as a generic type parameter rather than a macro argument, so the compiler enforces the type at the call site and sizeof can never be computed incorrectly:

use kernel::ioctl::{_IO, _IOR, _IOW, _IOWR};

const MY_MAGIC: u32 = b'J' as u32;  // registered type for your driver

// No data transfer — a bare command
const MY_RESET:      u32 = _IO(MY_MAGIC, 0);

// Kernel writes a u32 status value to userspace
const MY_GET_STATUS: u32 = _IOR::<u32>(MY_MAGIC, 1);

// Userspace writes a config struct to the kernel
const MY_SET_CONFIG: u32 = _IOW::<MyConfig>(MY_MAGIC, 2);

// Both directions (e.g., update-in-place)
const MY_XFER:       u32 = _IOWR::<MyData>(MY_MAGIC, 3);

The module also provides decoder functions for use inside the driver's ioctl handler:

use kernel::ioctl::{_IOC_DIR, _IOC_TYPE, _IOC_NR, _IOC_SIZE};

fn ioctl(..., cmd: u32, arg: usize) -> Result<isize> {
    match cmd {
        MY_GET_STATUS => { /* ... */ }
        MY_SET_CONFIG => { /* ... */ }
        _ => {
            pr_warn!("Unknown ioctl: type={:#x} nr={} size={}",
                _IOC_TYPE(cmd), _IOC_NR(cmd), _IOC_SIZE(cmd));
            Err(ENOTTY)
        }
    }
}

Return ENOTTY for unknown commands

When your driver receives an unrecognised command number, it must return -ENOTTY. Returning -EINVAL or -ENOSYS is incorrect (though common in older drivers) — ENOTTY is the standard signal that the command is not supported on this file descriptor.

Decoding a Number by Hand

Given the raw hex value 0x82187201:

Bits 31–30 (0b10) → direction = read (kernel → user)
Bits 29–16 (0x218 = 536) → size = 536 bytes
Bits 15–8 (0x72 = 'r') → type magic = 'r'
Bits 7–0 (0x01) → nr = 1

That decodes to _IOR('r', 1, struct dirent[2]), which is VFAT_IOCTL_READDIR_BOTH — exactly as shown in the kernel's ioctl decoding guide.

Choosing a Type (Magic Byte)

The type field (magic byte) groups all ioctls belonging to a single driver or subsystem. The kernel maintains a registry of allocated magic bytes in Documentation/userspace-api/ioctl/ioctl-number.rst.

When adding a new driver:

Check the table for a free byte with room to grow. 32–256 sequence numbers (nr values) is a typical target range.
Prefer an unused ASCII letter — they are easier to spot in source and strace output. The letters 'J' and 'Y' are currently unregistered.
Avoid heavily conflicted bytes such as 'H', 'F', 'M', 'P', 'V', and 'T', which are shared by many sound and video drivers.
Register your allocation by submitting a patch to ioctl-number.rst through the normal kernel patch process.

Globally unique command numbers make debugging easier: if userspace accidentally calls an ioctl on the wrong device, the driver returns ENOTTY instead of silently doing the wrong thing. Tools like strace can also decode registered numbers back to their symbolic names.

A Note on the Size Field

The 14-bit size field is informational only — the kernel does not automatically copy size bytes for you. Its primary value is as a debugging and versioning aid. Be aware that many legacy ioctls have incorrect size fields due to a historical mistake: passing sizeof(arg) (the size of the pointer) instead of the data type to the C macro. The Rust API avoids this entirely by using a generic type parameter, so size_of::<T>() is always used.

Do not version your ioctl interface by overloading the size field with different struct layouts. Instead, assign a new nr value for each interface revision and keep the old one working as a compatibility shim.

Transferring numbers with `UserSlice`

When an IOCTL handler receives a _IOR, _IOW, or _IOWR command, the arg parameter is a raw userspace pointer to a buffer. You must never dereference it directly. Instead, use UserSlice from kernel::uaccess, which wraps the kernel's copy_from_user and copy_to_user APIs and guarantees that the address range is validated and that accesses stay within the user mapping.

All methods on UserSlice and its reader/writer types are safe. Accessing a bad or unmapped address returns EFAULT rather than causing undefined behaviour.

Construction

A UserSlice is created from the raw pointer and length passed to your IOCTL handler:

use kernel::uaccess::{UserPtr, UserSlice};

fn ioctl(
    device: ...,
    _file: &File,
    cmd: u32,
    arg: usize,
) -> Result<isize> {
    match cmd {
        MY_IOCTL => {
            let user = UserSlice::new(
                UserPtr::from_addr(arg),
                core::mem::size_of::<u32>(),
            );
            // ...
        }
        _ => Err(ENOTTY),
    }
}

Constructing a UserSlice performs no checks. Validity is only checked when you actually read or write. The arg from an IOCTL call is always a usize; pass it through UserPtr::from_addr to get the typed pointer UserSlice::new expects.

Obtaining readers and writers

UserSlice is consumed when you call one of its three factory methods. This design enforces that you cannot accidentally build multiple independent readers over the same memory, which is the primary source of TOCTOU bugs.

UserSlice::reader consumes the slice and returns a UserSliceReader for read-only access:

let reader = UserSlice::new(UserPtr::from_addr(arg), size).reader();

UserSlice::writer returns a UserSliceWriter for write-only access:

let mut writer = UserSlice::new(UserPtr::from_addr(arg), size).writer();

UserSlice::reader_writer returns both at once, sharing the same address range. Use this for _IOWR commands where you read the input and then write the result back into the same buffer:

let (reader, mut writer) = UserSlice::new(UserPtr::from_addr(arg), size).reader_writer();

Reading from userspace

UserSliceReader::read::<T> reads exactly size_of::<T>() bytes and returns a value of type T. The type must implement FromBytes, which is implemented for all plain-integer types (u8, u16, u32, u64, i32, etc.) and for types you annotate with #[derive(FromBytes)]:

// _IOW: userspace is sending us a u32
fn handle_set_value(arg: usize) -> Result<isize> {
    let mut reader = UserSlice::new(
        UserPtr::from_addr(arg),
        core::mem::size_of::<u32>(),
    ).reader();

    let value: u32 = reader.read()?;
    pr_info!("received value: {}\n", value);
    Ok(0)
}

Writing to userspace

UserSliceWriter::write::<T> copies size_of::<T>() bytes to userspace. The type must implement AsBytes:

// _IOR: userspace is reading a u32 from us
fn handle_get_version(arg: usize) -> Result<isize> {
    let mut writer = UserSlice::new(
        UserPtr::from_addr(arg),
        core::mem::size_of::<u32>(),
    ).writer();

    let version: u32 = 1;
    writer.write(&version)?;
    Ok(0)
}

A complete `_IOWR` example

A read-write IOCTL reads data from userspace, transforms it, and writes the result back. Use reader_writer to get both handles over the same memory:

fn handle_transform(arg: usize) -> Result<isize> {
    let size = core::mem::size_of::<u32>();
    let (reader, mut writer) = UserSlice::new(
        UserPtr::from_addr(arg),
        size,
    ).reader_writer();

    let input: u32 = reader.read()?;
    let output: u32 = input.wrapping_mul(2);
    writer.write(&output)?;
    Ok(0)
}

TOCTOU safety

The API enforces that each byte of userspace memory is read at most once. Reading advances the reader's internal cursor, so subsequent reads start from the new position. You cannot reuse a UserSlice once a reader or writer has been created from it.

If you genuinely need to read the same address twice — for example, to validate the data before acting on it — use UserSliceReader::clone_reader to get a second reader at the same position. Be aware that the data may change between the two reads, so you must treat the second read as the authoritative copy and act on it, not on the first:

let reader = UserSlice::new(UserPtr::from_addr(arg), size).reader();
let validator = reader.clone_reader(); // reads from the same address

// validate using `validator`
// act using `reader` — NOT `validator`, which was already consumed by validation

warning

Do NOT construct two separate UserSlice instances over the same address and read from each. That is the classic TOCTOU bug. Use clone_reader when a second read is unavoidable, and always act on the last copy you read.

Setting up tmux

Having tmux available in the initramfs lets you split your terminal while the kernel is running — useful for tailing dmesg in one pane while running test commands in another. Because the initramfs is a minimal environment, you need a statically linked binary, two extra mount points, and the terminfo entries that tmux uses to drive the terminal.

Enable Unix socket support in the kernel

tmux communicates between its server and client processes over a Unix domain socket. This requires CONFIG_UNIX to be enabled in the kernel configuration. In a minimal custom kernel it is often left out, so you need to add it explicitly.

Open your kernel config with make LLVM=1 menuconfig and navigate to:

Networking support
  └── Networking options
        └── Unix domain sockets

warning

Enable it as built-in (*, not M) — the initramfs has no module loading infrastructure so a module will not be found at boot:

warning

If CONFIG_UNIX is missing or built as a module, tmux will appear to start but immediately exit with a socket error. You will see nothing in the terminal and dmesg will not help because the failure happens entirely in userspace.

Recompile the kernel

Make sure you recompile the kernel using make LLVM=1 -jN., where N is the number of cores that your processor has available.

Download a static tmux binary

Download a pre-built static binary for your host architecture from the tmux static builds releases page. Choose the tmux-*-x86_64.tgz archive for x86 machines or tmux-*-aarch64.tgz for arm64, then extract it and copy the binary into your initramfs:

tar -xzf tmux-*.tgz
cp tmux $INIT_RAM_FS/bin/tmux
chmod +x $INIT_RAM_FS/bin/tmux

tip

$INIT_RAM_FS is the root of your unpacked initramfs tree, as defined on the initramfs setup page.

Create the required mount points

tmux needs /tmp for its socket and /dev/pts for the pseudo-terminal devices it allocates for each pane. Create both directories inside the initramfs:

mkdir -p $INIT_RAM_FS/tmp
mkdir -p $INIT_RAM_FS/dev/pts

Update the init script

The init script needs to mount tmpfs on /tmp and devpts on /dev/pts before tmux is invoked. Add the two mount lines shown below, after the existing devtmpfs mount:

#!/bin/busybox sh

# Install the busybox commands and set the PATH variable
/bin/busybox --install -s

# Mount kernel filesystems
mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs devtmpfs /dev

# Mount filesystems required by tmux
mount -t tmpfs tmpfs /tmp         # tmux socket directory
mkdir /dev/pts
mount -t devpts devpts /dev/pts   # pseudo-terminal devices

# Write a banner
cat << !

Welcome to the Rust Kernel Development Minimal Linux!
Press CTRL+a x to exit QEMU

!

# Run a shell
exec setsid /bin/cttyhack /bin/sh

devpts provides the /dev/pts/N devices that the kernel allocates whenever a program calls openpty() or posix_openpt(). tmux opens one of these for each pane it creates. Without this mount, tmux starts but immediately fails to open a pseudo-terminal and exits.

tmpfs on /tmp gives tmux a writable directory for its server socket (/tmp/tmux-0/default by default). Without it, the socket creation fails and no session can be attached.

Copy the terminfo entries

tmux needs terminfo descriptions for the terminal type it emulates and for the outer terminal it is running inside. In a QEMU serial console the outer terminal is typically linux; inside a tmux pane it is xterm or xterm-256color. Copy both entries from your host system.

mkdir -p $INIT_RAM_FS/usr/share/terminfo/l
mkdir -p $INIT_RAM_FS/usr/share/terminfo/x

cp /usr/share/terminfo/l/linux          $INIT_RAM_FS/usr/share/terminfo/l/linux
cp /usr/share/terminfo/x/xterm          $INIT_RAM_FS/usr/share/terminfo/x/xterm
cp /usr/share/terminfo/x/xterm-256color $INIT_RAM_FS/usr/share/terminfo/x/xterm-256color

note

If the copy commands fail, your distribution may store terminfo files under /etc/terminfo or /lib/terminfo instead of /usr/share/terminfo. Run the following to locate them on your host:

find /usr /etc /lib -name linux -path '*/terminfo/*' 2>/dev/null

tip

The terminal window in which you run QEMU must be at least 80 columns × 25 rows. tmux enforces a minimum size and will refuse to create a session if the reported dimensions are smaller. If tmux exits immediately without drawing anything, resize your terminal and try again.

Exercises

tip

Download, build and copy to $INIT_RAM_FS/bin the ioctl utility.

Define an argumentless IOCTL command — Using 'J' as your type, define a command PING with sequence number 0 using _IO. Implement it in your driver's ioctl handler so that it returns Ok(0) on success. Verify from the command line that issuing the command does not return an error.
Return a value through an IOCTL — Define a GET_VERSION command using _IOR::<u32> with sequence number 1. When the command is issued, copy a version number (e.g. 1) into the userspace buffer provided by the caller. Verify from the command line that the correct value is printed.
Accept a value through an IOCTL — Define a SET_VALUE command using _IOW::<u32> with sequence number 2. In the handler, read the u32 from userspace and store it in your device's per-instance state. Print the received value with pr_info! and confirm it appears in dmesg.
Round-trip with _IOWR — Define a TRANSFORM command using _IOWR::<u32> with sequence number 3. Read the u32 sent by userspace, double it, and write the result back into the same buffer. Verify from the command line that the value returned is twice the value sent.
Reject unknown commands — Issue an IOCTL command number that your driver does not recognise. Confirm that the driver returns ENOTTY and that the command line tool reports the appropriate error. Check that your match arm's catch-all is returning Err(ENOTTY) and not Err(EINVAL).
Decode a command number by hand — Print the raw value of your GET_VERSION command number (e.g. with pr_info! at module init time). Using the bit layout from the IO Control page, manually decode the direction, size, type, and sequence number fields and verify they match what you defined.
Extend state across calls — Define both a SET_VALUE and a GET_VALUE command. Store the value set by SET_VALUE in your device's per-instance state (protected by a Mutex), then return it via GET_VALUE. From the command line, set a value, then retrieve it and confirm the round-trip is consistent.
Implement multiple open instances — Open your device file twice from two separate processes. Set a different value in each instance using SET_VALUE, then read it back. Confirm that the values are independent, demonstrating that per-open state is correctly isolated in your Ptr type.

tip

Install tmux so you can run several commands at the same time.

Objectives​

Misc devices​

Registration​

Implementing MiscDevice​

Available callbacks​

Module reference counting​

IO Control (IOCTL)​

Bit Layout​

The Four Builder Macros​

The Rust API​

Decoding a Number by Hand​

Choosing a Type (Magic Byte)​

A Note on the Size Field​

Transferring numbers with UserSlice​

Construction​

Obtaining readers and writers​

Reading from userspace​

Writing to userspace​

A complete _IOWR example​

Setting up tmux​

Enable Unix socket support in the kernel​

Recompile the kernel​

Download a static tmux binary​

Create the required mount points​

Update the init script​

Copy the terminfo entries​

Exercises​

Objectives

Misc devices

Registration

Implementing `MiscDevice`

Available callbacks

Module reference counting

IO Control (IOCTL)

Bit Layout

The Four Builder Macros

The Rust API

Decoding a Number by Hand

Choosing a Type (Magic Byte)

A Note on the Size Field

Transferring numbers with `UserSlice`

Construction

Obtaining readers and writers

Reading from userspace

Writing to userspace

A complete `_IOWR` example

Setting up tmux

Enable Unix socket support in the kernel

Recompile the kernel

Download a static tmux binary

Create the required mount points

Update the init script

Copy the terminfo entries

Exercises