Linux Kernel Programming

This is the home page for the "Linux Kernel Programming" course. Here, you will find (in a badly-organised form) all the material, slides, and information needed to attend the course

For the previous editions of the "Kernel Programming" course, check the old websites: 2018, 2019, and 2020.

Lessons:

• First lesson: 2023/12/04, 9:00 -> 11:00
  • Introduction to the course
  • The Linux kernel
• Second lesson: 2023/12/12, 9:00 -> 11:00, White room
  • Configuring and building the Linux kernel
  • Using the Linux kernel in a VM: first simple experiments
  • A minimal filesystem; the boot process
• Third lesson: 2023/12/19, 9:00 -> 11:00, White room
  • Introduction to Linux kernel modules
• Fourth lesson: 2024/01/08, 9:00 -> 11:00
  • Interactions between kernel space and user space: misc devices
  • Blocking and waking up a task: something about OS Kernels
• Fifth lesson: 2024/01/15, 9:00 -> 11:00 (Video - in italian)
  • Something more about OS kernel, task states, scheduling, etc...
  • First examples with competitive and cooperative synchronization: see http://retis.santannapisa.it/luca/KernelProgramming/Src/SynchExamples.git (to be cloned with git)
• Sixth lesson: 2024/01/22, 9:00 -> 11:00
  • Cooperative synchronization done right (with waitqueues or completions)
  • Coping with signals, again
• Seventh lesson: 2024/01/29, 9:00 -> 11:00
  • Some details about schedule(): blocking a task
  • Linux kernel lists, with an example
  • Something about kernel threads
• Eighth lesson: 2024/02/05, 9:00 -> 11:00
  • Kernel threads, again
  • Another example: kernel threads and devices
  • Some details about kernel locking and similar
• Nineth lesson: 2024/02/13, 9:00 -> 11:00
• Tenth lesson: 2024/02/19, 9:00 -> 11:00

Slides:

• Introduction to kernel programming
• The Linux kernel modules
• Few words about OS kernels
• Something about locking in the Linux kernel
• The Linux CPU scheduler
• Introduction to virtio

More Information:

This is some documentation about the programming interface provided by the Linux kernel.

And here are few notes about kernel memory allocation flags. Some more details about the allocator can be found here (for the curious).

Experiments:

• How to compile a Linux kernel from source
• To boot the generated kernel with a minimal filesystem, use qemu-system-x86_64 -kernel arch/x86_64/boot/bzImage -initrd tinyfs.gz
  • NEW! To use an italian keyboard with the minimal filesystem, download keyb.gz, then concatenate it with tinyfs.gz (NOTE: you need to download the new tinyfs.gz!!!) using "cat tinyfs.gz keyb.gz > test.gz" and use the new "code.gz" as initramfs. When the VM is booted, type "sudo /sbin/loadkmap < /etc/keymaps/italian.kmap". If you do not have the "loadkmap" command, then you need to download tinyfs.gz again; if you do not find "italian.kmap", then you need to concatenate keyb.gz with the initramfs you are using.
• The minimal filesystem has been generated with https://gitlab.retis.sssup.it/l.abeni/BuildCore.git. See these instructions.
• The simplest possible kernel module (hello, world!!!)
• First experiments with a misc device: how to implement some simple user-space/kernel-space communication
  • Simple exercize: periodically blocking on read
• First examples about task synchronization: git clone http://retis.santannapisa.it/luca/KernelProgramming/Src/SynchExamples.git
  • Then run git checkout -b fix_bug origin/fix_bug (and then try the coop1, coop2, coop3, coop4, allow_signals, allow_signals1 branches)
• Improved examples about task synchronization with waitqueues and completions
• Example about using the Linux kernel lists
• Using kernel lists to buffer data written to a device
• Example about kernel threads
• Example with kernel threads and devices
• Example about bugs in using mutexes
• Program to schedule threads/processes as SCHED_DEADLINE
• Test application to start multiple real-time threads

Project Ideas:

Here are some ideas for possible projects for the exam. Of course, if you already have some idea for a project, we can discuss about it (the following ideas are just suggestions). Keep in mind that some projects are more complex than others and remember to contact me before starting to work on a project, so that I can send you some clarifications and details.

• Write some kind of module implementing kernel-space/user-space communication through a misc device and synchronizing user-space processes and threads. For example, you can extend the example showed during the lessons, to implement synchronous communication (a "write" operation blocks until the whole data has been received by a "read" operation), or you can implement a fixed-size queue (using either a kernel list or a circual buffer), blocking writers when the queue is full. Or you can implement some mechanism to direct the data to specific processes, maybe using capabilities (see https://www.slideshare.net/mdecky/ipc-in-microkernel-systems-capabilities around slide 10. Or you can implement some kind of security mechanism in the message-passing module
• Implement some kind of periodic real-time application using kernel threads (created when a module is inserted and destroyed when it is removed)
• Implement a module creating a "periodic device": one ore more processes or thread can open the device, select a period (by writing to the device or using an ioctl()), and sleep periodically on the device (for example, by reading data from it: when a read() is performed on the device, the caller is blocked until the beginning of the next period --- the period size is set as mentioned above)
  • Improvement: when a task sets its period (by writing on the device, or something else) its priority is automatically set according to RM
• Implement a "cyclic executive" scheduling module: a static schedule (specifying when to schedule some tasks, and specifying the task ID for each task) can be set by writing to a misc device, then when the schedule is started the module will schedule/deschedule each thread at the right time
• Run some experiments with the threaded NAPI patchset and virtio network devices (measuring the network performance with and without the patchset). Optionally, try to use the IRQ threads as NAPI threads when threaded interrupt handlers are used (see the "threadirqs" kernel boot parameter)