Compiling a custom FreeBSD kernel for Parallels VirtualMachine May 13 2020
I use Parallels on macOS to spin virtual machines for learning purposes. Lately, I've been interested in FreeBSD, so I wanted to create a slimmed-down version of the FreeBSD Kernel for my VM. And that is the topic of this post, how to create a custom FreeBSD Kernel for a Parallels virtual machine. It is not limited to Parallels tho. You can easily use the content of this note to build any custom Kernel.
So let's get started.
Table of Contents
The Kernel source code
You can find the kernel source on:
1
/usr/src/
If you didn't select the option to download the source during the installation of FreeBSD, you'll have to download it from the Internet. There is a GitHub mirror https://github.com/freebsd/freebsd that you could clone. Or you can use SVN and download it following the instructions here using Svnlite.
If you are used to building projects using Makefiles, you'll feel right at home. I encourage you to have a look at the Makefile. It has a lot of useful information. But to save you time, the compilation of the kernel will expect your configuration, either by default searching for /sys/<arch>/conf/GENERIC or if you specify it.
You can get your computer architecture by using the uname(1) command:
1
2
3
$ uname -m
#In my case
amd64
Before we attempt to compile our kernel, we'll need to define the options that our custom kernel will support. So let's do that.
Basing our setup on the GENERIC configuration
FreeBSD supports many architectures. You can take a gander at the /sys directory. This directory includes configurations for each of the supported architectures. My computer has an Intel processor, so I'll base my configuration on the amd64 architecture. Inside that directory, you can find the generic configuration, and I'll use that as our starting point:
1
/sys/amd64/conf/GENERIC
I'll make a copy called PARALLELS.
1
cp GENERIC PARALLELS
If you have a look at that file or the one for your architecture, you'll find options, devices, and variables that specify the configuration that will define your custom Kernel once you compile it.
The first lines of my PARALLELS (The copy I made form the GENERIC config), looks like this:
1
2
cpu HAMMER
ident GENERIC
I'll leave the same CPU identifier, and will change the name that will identify my custom kernel. I'll change it to PARALLELS. The identifier helps when I want to verify which kernel I'm using at the moment. I can identify it as PARALLELS using sysctl(8):
1
$ sysctl kern.ident
Ok, we continue down the file commenting out all the options that you don't need. If you are unsure about an option, you have a few choices:
- Safe option - leave it there, your current Kernel has it and is not breaking anything.
- Learning by experimentation - remove it and see what breaks. The problem with this approach is that it might not break during compilation, but when some software tries to use a feature related to that option. Not recommended, but I can see how this might be exciting.
- Investigating - You can use
man -k <option>and check the documentation, or use google to check if you can safely remove the option. Boring but safe, and sometimes more efficient than spending days afterwards figuring out what went wrong. This option is the one I recommend.
After all the configuration options, you come to the device section. Here is where it gets interesting, and we can chop a lot of drivers that we don't use. If we follow the advice on this link by Allan Jude, we should be able to remove SCI, FireWire, and USB device drivers, also all network adapters but ed and miibus. Well, that wouldn't work for me. When removing devices, you should make sure that you are not removing drivers that you use. You should check /var/run/dmesg.boot, where you can see all the detected hardware. For me, I had an em network card:
1
2
3
4
5
6
7
...
em0: <Intel(R) PRO/1000 Network Connection> port 0xd240-0xd25f mem 0xd0000000-0xd001ffff at device 5.0 on pci0
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Ethernet address: 00:1c:42:ed:e6:2e
em0: link state changed to UP
em0: netmap queues/slots: TX 1/1024, RX 1/1024
....
Do a quick scan in your dmesg.boot and check that you are leaving all the drivers that your hardware requires. Also, make sure that your driver doesn't depend on another driver:
1
2
3
# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure
device iflib
device em # Intel PRO/1000 Gigabit Ethernet Family
If I comment out iflib I won't be able to compile because em depends on iflib.
If you keep scrolling down, you'll reach the virtualisation section. Because I'm explicitly using Parallels then I can remove the other virtualisation devices and options:
1
2
HyperV - Microsoft
Xen - Xen HyperVisor HVM - https://xenproject.org/
So what's left is Virtio and from https://kb.parallels.com/4948
1
Virtio network adapter is the fastest card. However, it works only in Linux and BSD guest operating systems. It is a default adapter for Linux-based OSes.
So I'll leave virtio and comment out the rest:
1
2
3
4
5
6
7
# HyperV drivers and enhancement support
#device hyperv # HyperV drivers
# Xen HVM Guest Optimizations
# NOTE: XENHVM depends on xenpci. They must be added or removed together.
#options XENHVM # Xen HVM kernel infrastructure
#device xenpci # Xen HVM Hypervisor services driver
Alright, after we have all our configuration ready in our PARALLELS configuration file, we can compile our kernel!
But before, let's be smart and create a backup of our current kernel. I'm confident that I did a good job researching the options and devices, but I've made mistakes in the past (Trust but verify).
Creating a backup of our current kernel
Our kernel is located at /boot/kernel so a simple copy should be fine:
1
2
cd /boot
cp -a kernel kernel.2020-05-13
I named mine kernel.2020-05-13 but you can name it whatever you want(e.g. kernel.good).
We are going to add our backup to the list of possible kernels to load for the system loader. If anything goes wrong, we can go back to our working kernel.
1
2
#We can edit /boot/loader.conf manualy or use syrc(8)
sysrc -f /boot/loader.conf KERNELS="kernel kernel.old kernel.2020-05-13"
When we compile and install the kernel, the script creates an automatic backup as kernel.old, but if we'll be in a cycle of compile/install until we find the correct configuration, kernel.old can be rewritten with any of our attempts and not our working kernel.
Ok, finally let's compile and install.
Compiling and installing our custom Kernel
Now is when everything comes together. Go to the kernel source, and run buildkernel:
1
2
cd /usr/src
make KERNCONF=PARALLELS buildkernel
If we forego the KERNCONF variable the script will search for GENERIC.
If you got any errors, read the message carefully and go back and fix your PARALLELS configuration file accordingly.
If everything went fine and the compilation completes successfully, we install the kernel:
1
2
make KERNCONF=PARALLELS installkernel
#This moves the current /boot/kernel to /boot/kernel.od and moves our newly created one to /boot/kernel
And that's it! We can reboot and test.
If the kernel doesn't work, you can go to the boot menu select the working version of your kernel and figure out what went wrong.
Final Thoughts
Creating a custom Kernel is not a complicated task, it mostly takes time to research the correct options and drivers for your particular case. You might be wondering "how much space I saved?" after creating my custom kernel specially tailored for Parallels. Well, here is the answer:
1
2
3
$ du -sh /boot/kernel*
67M kernel
75M kernel.good
12M! The things we do for 12M. If we go only for size is not that much, but it was fun. And the less code we have in our system, the smaller the surface area for bugs. If you want to have a look at the PARALLELS configuration file I used, you can find it in this GitHub Gist.
Remember that your hardware might be different, so have a look at /var/run/dmesg.boot to check your hardware.
Alright, that's it for today.