Linux Kernels

A computer system can be divided into three pieces:

1. [Hardware] Motherboard (CPU, RAM, and other chips), various daughter cards (e.g., a graphics card, a wireless network card), keyboard, mouse, screen, hard disks, other IO devices, and power supply.
2. [Software] Operating system, and applications.
3. [Firmware] A basic input/output subsystem (BIOS) preloaded into a ROM

A typical PC fits this classification very well; but, there are many classes of computers that do not. E.g., some computer systems may not have a key board or a hard disk.

A program is a sequence of CPU instructions (also called machine code) structured in a very specific way. Software is a generic synonym for programs. Applications are a subclass of software.

The CPU can execute its instructions only when they are in RAM. We have so many programs that it is infeasible to leave them all sitting in RAM. The OS stores them on a HDD and loads them into RAM as needed.

We will explain in the lectures the "Fetch, Decode, and Execute cycle" of a CPU.

CPU can generate traps and faults.

Devices raise interrupts. Interrupt handling is done by the OS.

The moment a computer system is powered on, the firmware is in control of every piece of the hardware. On a PC, this firmware is called the BIOS (basic input output subsystem).

A computer system must not be shut down "ungracefully". All the running processes need to be informed of the impending shut down, and wait politely for them to terminate. This is known as graceful termination of the OS. Most systems can now detect a power failure, and initiate a shutdown. However, un-responding processes not withstanding, a shutdown cannot be postponed indefinitely, and the OS forcibly terminates.

There are many definitions of OS based on its typical functionality. From the book by Silbershatz and others: "What are the three main purposes of an operating system? Answer: (i) To provide an environment for a computer user to execute programs on computer hardware in a convenient and efficient manner. (ii) To allocate the separate resources of the computer as needed to solve the problem given. The allocation process should be as fair and efficient as possible. (iii) As a control program it serves two major functions: (a) supervision of the execution of user programs to prevent errors and improper use of the computer, and (b) management of the operation and control of I/O devices."

A very useful definition of an OS is based on its components. If we were to spread out all the code that constitutes an OS on a piece of cloth, we can then cut it up into the following components:

OS = VM + FileM + ProcM + Net + IO + UserM + (Boot) + [SysPrg]

Strictly speaking, an OS does not include GUI, nor CLI. In the world of Linux, the GUI is provided by X11, Gnome and KDE software subsystems, and CLI by bash, csh, zsh or other such shells. These are not strictly part of the Linux OS. In the Windows world, the GUI is provided by (plain) Explorer, and the CLI by cmd or PowerShell. These are not strictly part of the Windows OS.

Programs like Word, Internet Explorer, Konqueror are applications. Visual Studio, g++, Active Python and such are (sub-) categorized as development tools.

The above components can also be combined and then re-divided into a kernel + initial ram disk components + drivers + interrupt handlers.

The code of system calls is not disjoint from the code of previously mentioned components. It is made of different pieces from these components. Linux and Windows have around 350 system calls each. The details of Linux syscalls can be found in the files:

-rw-r--r--  1 root root  1403 2010-01-03 06:57 /usr/include/sys/syscall.h
-rw-r--r--  1 root root 22041 2010-01-28 00:56 /usr/include/asm/unistd_64.h

Most of the OS stays RAM resident and dormant until (i) a system call is made by a process, (ii) a hardware interrupt causes a handler in the OS to run, or (iii) a software triggered even causes a handler in the OS to run. Consequently, viewing an OS only as a collection of processes is going to be incomplete.

The following is a severely pruned list of processes on a Linux PC as generated by the ps command. The process-id shown (except for 1) may change.

For a list of Windows processes, invoke Task Manager and then select Show processes from all users.

1. Powered on. BIOS->POST. ROM and RAM. CPU fetch-decode-execute cycle.
2. Discover boot device. Boot sectors. Boot device priority order.
3. Bring in and give control to the boot loader. Bring in v. "load"
4. Bring in and give control to OS loader. NTLDR, GRUB, etc. (http://en.wikipedia.org/wiki/GRUB)
5. OS kernel.
6. Cold versus warm boot. Cold as if power is just applied. Warm skips first few steps.

1. Linux init TBD 6. For Linux: process init is started. Process init (in Linux, and its counterpart in Windows) is in control of "normal" operation of a computer.
2. Ubuntu and other distributions use UUID (Universally Unique Identifier) to indicate the root device. From the Wiki: In its canonical form, a UUID consists of 32 hexadecimal digits, displayed in 5 groups separated by hyphens, for a total of 36 characters. For example: 550e8400-e29b-41d4-a716-446655440000 .
3. Linux distributions have many independent designs that can serve as init. Currently (2019), systemd is widely used. Among others are runit and openrc.

1. https://en.wikipedia.org/wiki/Windows_startup_process

1. See the list of processes shown in a previous section.
2. The init process is expected to be alive during this phase. All other processes are descendents of init.
3. We usually configure init so that certain processes are re-generated should they die.

1. Properly shutting down is part of the functionality of the first process. It requests all process to voluntarily terminate immediately. Finally, it kills them all.
2. Various IO operations may be on-going at the time of shut down. If we terminate such operations arbitrarily, the file volume will become corrupted.
3. The first process can also sense an imminent power outage.

1. A computer system can be divided into three pieces:
   1. [Hardware] Motherboard (CPU, RAM, and other chips), various daughter cards (e.g., a graphics card, a wireless network card), keyboard, mouse, screen, hard disks, other IO devices, and power supply.
   2. [Software] Operating system, and applications.
   3. [Firmware] A basic input/output subsystem (UEFI/BIOS) preloaded into a ROM
2. OS Kernel Definition by Components: OSK == VirtualMem + FileM + ProcessM + Net + IO + UserM + (Boot) + [SysPrg] ;; M stands for Management.
3. http://cecs.wright.edu/~pmateti/Courses/2350/Labs/OS-on-USB/OSonUSBLab.html Recommended Reading.

1. In this course, we are focused on the security aspects of the kernel.
2. Preventing Security Exploits is an expectation of all OS kernels, and system software. There is no consensus on the definition of what "exploits" are, and what we mean by "prevention".
3. A few specific classes of security exploits are describe below.
4. Underneath all these exploits is the technique known as privilege escalation.

Kernel should

1. check the integrity of init.
2. track the provenance (of processes)
3. track network activity

1. As a collection of files: vmlinuz, initrd, /lib/modules/*.
2. Kernel is NOT a process.
3. Kernel, like processes, uses pages, and segments of virtual memory.
   1. The "text" (machine code) of a kernel is expected to remain unchanged during execution. Just like a process.
   2. Kernel uses a stack. Has private and public methods. The public methods are offered to user processes as system calls.
4. Kernel discovers the pathname of init program, and invokes it.
5. Except for init, all other processes are created at the request of init or its children. man pstree
6. Several psuedo file volumes: /proc, /sys, /dev Browse!
7. /etc/modules

1. All OS kernels, of Linux, MacOS, or Windows, are buggy. OS kernels have not yet reached a state of being bug free. A security issue in a bug makes it vulnerable. An exploit presents an actual use of this vulnerability in demonstrating the security issue.
2. See Coverity articles for Linux kernel bugs. Over the years thousands of bugs have been discovered through code audits.
3. https://scan.coverity.com/projects/linux "See how defect density for 'Linux' compares with defect density for other … arch/x86/.*, No, 179,049, 0.54. Kernel .*/kernel/.*, No, 207,512, 0.70 …… The root cause of each defect is clearly explained, making it easy to fix bugs." Lines of code analyzed‎: 15,650,125. Required Visit.

1. 2019 Total number of vulnerabilities : 662 https://www.cvedetails.com/vulnerability-list/vendor_id-33/product_id-47/cvssscoremin-7/cvssscoremax-7.99/Linux-Linux-Kernel.html Required Visit.
2. https://www.cvedetails.com/product/47/Linux-Linux-Kernel.html Linux Kernel Vulnerability Statistics 1999 - 2018. Required Visit.

1. The Main Linux Kernel repository https://www.kernel.org/;
   1. stable: 5.3.6 As of 2019-10-11 104M Oct 11 linux-5.3.6.tar.xz
   2. du -sh linux-5.3.6/ before build gives 987M
   3. du -sh linux-5.3.6/ afetr build gives 21G
   4. Here is a top level build log example: https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.4-rc3/BUILD.LOG.amd64 from Ubuntu
2. https://www.kernel.org/doc/html/latest/ The Linux Kernel documentation
3. http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/ These binary packages represent builds by Ubuntu of the mainline or stable Linux kernel tree.
4. git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git etc.
5. https://elixir.bootlin.com/linux/v5.4-rc2/source Interlinked, and browseable source code.

1. Hardening depends on this how-to.
2. Source code repositories
3. Build Tools
4. Build Overview
5. Configure the build: make xconfig or make menuconfig
6. make bzImage about 030 mins on my machine.
7. make modules about 150mins on my machine.
8. make install few seconds
9. create a new entry in Grub
10. Deploy a new Linux Kernel

1. Guide to follow https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel for Ubuntu Linux kernel. For a generic Linux kernel, follow https://wiki.ubuntu.com/KernelTeam/GitKernelBuild
2. Software Tools/ Packages Needed in the build: sudo apt install the following
   1. git build-essential kernel-package fakeroot libssl-dev
   2. install libncurses5-dev libqt5*-dev ccache bison flex libelf-dev
   3. They amount to some 300+ MB.

1. Create a Linux kernel configuration: What components to include/ exclude / build-as-modules. There are two main ways to configure. xconfig uses GUI, menuconfig uses text-console.
2. % make xconfig This will take only a few seconds. It generates a GUI program that presents a menu.
3. % xconfig Invoke it and answer "thousand questions". Select the various kernel components carefully. (This can take an hour or more depending on your familiarity.) Requires your interaction and understanding. Make an effort to understand the QAs, but do not get discouraged. There are many to choose by answering y/n/m; y = yes make it part of the kernel, n = no omit it altogether, m = build it as a module. Make sure that everything needed for boot is in the kernel image and not as a module. You may have to web search for details.
   1. The configuration, a text file, is saved to a file named .config Click on Save before quitting xconfig. The config file you produced is the plain ascii text file (not recommended to edit this by hand): linux/.config

   2. An example config produced: ./dot-config.txt About 9600+ lines.

      -rw-rw-r-- 1 pmateti pmateti 218282 Oct  9 22:08 .config
      
      

1. % make bzImage modules This can take 30+ minutes on an i7. You can insert extra flags for make (e.g., -j32) to speed up the build. Does not require your interaction.

1. Copy the vmlinuz, modules to /boot etc. Modules are stored in /lib/modules/ kernel-version-number.
2. # make install This (i) locates and copies the bzImage, config and System.map files into the /boot directory; (ii) locates and copies all the modules (*.ko) also onto the LXU into /lib/modules/ version-number-of-kernel.

1. Prepare a new menu item in the Grub OS boot loader.
2. # grub-update Revise the /boot/grub/menu.lst on the Linux-on-USBD appropriately so that the machine can be booted with the newly built kernel.
3. Lab Task: Get a kernel package and deploy it. http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/linux-image-unsigned-4.19.0-999-lowlatency_4.19.0-999.201810082201_amd64.deb This is a built .deb Debian package. This can be installed by dpkg

1. A patching program identifies an area in the given file, replaces it with another.
2. Patching can be done either in source code or in binary.
3. Suppose we have src code file fg.c. We discovered bugs and improvements that can be made. We did such changes by editing fg.c and saved it as fh.c. There is a standard program named diff that takes fg.c and fh.c and generated a editor script called a patchfile that can transform fg.c to fh.c
4. The program named patch takes the patchfile and fg.c automates the generation of fh.c. From the man page: SYNOPSIS patch [options] [originalfile [patchfile]]

1. The kernels deployed by distributors are almost always patched differently from each other. Each distributor documents what patches they have applied. Unfortunately, they do not document which patches they considered but did not apply, and the reasons for the choice.
2. https://kernel.ubuntu.com/~kernel-ppa/mainline/daily/2019-10-12/ lists the following src code patches:

0001-base-packaging.patch
0002-UBUNTU-SAUCE-add-vmlinux.strip-to-BOOT_TARGETS1-on-p.patch
0003-UBUNTU-SAUCE-tools-hv-lsvmbus-add-manual-page.patch
0004-debian-changelog.patch
0005-configs-based-on-Ubuntu-5.4.0-0.1.patch

1. https://www.kernel.org/ lists patches that transform your earlier downloads to the latest one.

1. This is a highly knowledge-based operation.
2. Moved into "Hardening" a kernel.

1. Fetch the kernel source.
2. Install the kernel source.
3. Rebuild the kernel source.
4. Copy the kernel source.
5. Clean old module & config files.
6. Apply the patch.
7. Recompile Kernel.
8. Compile the kernels modules.

1. http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/ Get the src dode, apply the listed patches.
2. Build the kernel from the resulting source.
3. How will you verify that the newly built kernel offer the "same" semantics as the one that Ubuntu published.
4. Deploy the new build.