Linux Kernel Programming

This is the home page for the "Linux Kernel Programming" course. Here, you will find (in a badly-organised form) all the material, slides, and information needed to attend the course

For the previous editions of the "Kernel Programming" course, check the old websites: 2018 and 2019.

Lessons:

This is a tentative schedule for the course:

First lesson: 2020/10/12, 9:30 -> 12:00
- Introduction to the course: why kernel programming is "special", ...
- Introduction to the Linux kernel. Bring your laptop with you, we will compile a kernel, and maybe test a simple "hello world" in kernel land!
Second lesson: 2020/10/19, 9:30 -> 12:00 (video of the second part of the lesson, in italian)
- Booting a Linux-based OS with the kernel compiled last week
- Introduction to Linux kernel modules
Third lesson: 2020/10/26, 9:30 -> 12:00 (videos of the first part, second part and third part of the lesson, in italian)
- Interactions between user-space and a kernel module
- Some theory about OS kernels
Fourth lesson: 2020/11/09, 9:30 -> 12:00 (videos of the first part and second part of the lesson, in italian)
- Blocking a task, again
- Competitive synchronization vs cooperative synchronization
- Examples about mutexes and "handmade" cooperative synchronization
- Interruptible sleep and signal handling
Fifth lesson: 2020/11/16, 9:30 -> 12:00 (videos of the first part, second part, and third part of the lesson, in italian)
- Cooperative synchronization done right (with waitqueues or completions)
- Coping with signals, again
- Linux kernel lists, with an example
Sixth lesson: 2020/11/23, 9:30 -> 12:00 (videos of the first part and second part of the lesson, in italian)
- Discussion of a possible solution for the exercize assigned last week
- Some experiments with kernel threads
- Something more about kernel locking
Seventh lesson: 2020/11/30, 9:30 -> 12:00 (videos of the first part and second part of the lesson, in italian)
- Again on kernel threads: example mixing kernel threads and devices
- Mutex debugging with lockdep
- The CPU scheduler
Eighth lesson: 2020/12/07, 9:30 -> 12:00 (videos of the first part and second part of the lesson, in italian)

Slides:

Introduction to kernel programming
Few words about OS kernels
The Linux kernel modules
Something about locking in the Linux kernel
The Linux CPU scheduler
Introduction to virtio

More Information:

This is some documentation about the programming interface provided by the Linux kernel.

And here are few notes about kernel memory allocation flags. Some more details about the allocator can be found here (for the curious).

Experiments:

How to compile a Linux kernel from source
To boot the generated kernel with a minimal filesystem, use qemu-system-x86_64 -kernel linux-5.8.14/arch/x86_64/boot/bzImage -initrd tinyfs.gz
- NEW! To use an italian keyboard with the minimal filesystem, download keyb.gz, then concatenate it with tinyfs.gz (NOTE: you need to download the new tinyfs.gz!!!) using "cat tinyfs.gz keyb.gz > test.gz" and use the new "code.gz" as initramfs. When the VM is booted, type "sudo /sbin/loadkmap < /etc/keymaps/italian.kmap". If you do not have the "loadkmap" command, then you need to download tinyfs.gz again; if you do not find "italian.kmap", then you need to concatenate keyb.gz with the initramfs you are using.
The minimal filesystem has been generated with https://gitlab.retis.sssup.it/l.abeni/BuildCore.git. See these instructions.
The simplest possible kernel module (hello, world!!!)
First experiments with a misc device: how to implement some simple user-space/kernel-space communication
- Simple exercize: periodically blocking on read
First examples about task synchronization
- Coping with signals
Improved examples about task synchronization with waitqueues and completions
Example about using the Linux kernel lists
Using kernel lists to buffer data written to a device
Example about kernel threads
Example with kernel threads and devices
Example about bugs in using mutexes

Project Ideas:

Here are some ideas for possible projects for the exam. Of course, if you already have some idea for a project, we can discuss about it (these ideas are just suggestions) Keep in mind that some projects are more complex than others and remember to contact me before starting to work on a project, so that I can send you some clarifications and details.

Write some kind of module implementing kernel-space/user-space communication through a misc device and synchronizing user-space processes and threads. For example, you can extend the example showed during the lessons, to implement synchronous communication (a "write" operation blocks until the whole data has been received by a "read" operation). Or you can implement some mechanism to direct the data to specific processes, maybe using capabilities (see https://www.slideshare.net/mdecky/ipc-in-microkernel-systems-capabilities around slide 10. Or you can implement some kind of security mechanism in the message-passing module
Implement a module creating a "periodic device": one ore more processes or thread can open the device, select a period (by writing to the device or using an ioctl()), and sleep periodically on the device (for example, by reading data from it: when a read() is performed on the device, the caller is blocked until the beginning of the next period --- the period size is set as mentioned above)
- Improvement: when a task sets its period (by writing on the device, or something else) its priority is automatically set according to RM
Implement a "cyclic executive" scheduling module: a static schedule (specifying when to schedule some tasks, and specifying the task ID for each task) can be set by writing to a misc device, then when the schedule is started the module will schedule/deschedule each thread at the right time
Run some experiments with the threaded NAPI patchset and virtio network devices (measuring the network performance with and without the patchset). Optionally, try to use the IRQ threads as NAPI threads when threaded interrupt handlers are used (see the "threadirqs" kernel boot parameter)