Compiling a custom FreeBSD kernel for Parallels VirtualMachine May 13 2020
I use Parallels on macOS to spin virtual machines for learning purposes. Lately, I've been interested in FreeBSD, so I wanted to create a slimmed-down version of the FreeBSD Kernel for my VM. And that is the topic of this post, how to create a custom FreeBSD Kernel for a Parallels virtual machine. It is not limited to Parallels tho. You can easily use the content of this note to build any custom Kernel.
So let's get started.
Table of Contents
The Kernel source code
You can find the kernel source on:
If you didn't select the option to download the source during the installation of FreeBSD, you'll have to download it from the Internet. There is a GitHub mirror https://github.com/freebsd/freebsd that you could clone. Or you can use SVN and download it following the instructions here using Svnlite.
If you are used to building projects using Makefiles, you'll feel right at home. I encourage you to have a look at the Makefile. It has a lot of useful information. But to save you time, the compilation of the kernel will expect your configuration, either by default searching for
/sys/<arch>/conf/GENERIC or if you specify it.
You can get your computer architecture by using the
$ uname -m
#In my case
Before we attempt to compile our kernel, we'll need to define the options that our custom kernel will support. So let's do that.
Basing our setup on the GENERIC configuration
FreeBSD supports many architectures. You can take a gander at the
/sys directory. This directory includes configurations for each of the supported architectures. My computer has an Intel processor, so I'll base my configuration on the
amd64 architecture. Inside that directory, you can find the generic configuration, and I'll use that as our starting point:
I'll make a copy called
cp GENERIC PARALLELS
If you have a look at that file or the one for your architecture, you'll find
devices, and variables that specify the configuration that will define your custom Kernel once you compile it.
The first lines of my
PARALLELS (The copy I made form the GENERIC config), looks like this:
I'll leave the same CPU identifier, and will change the name that will identify my custom kernel. I'll change it to
PARALLELS. The identifier helps when I want to verify which kernel I'm using at the moment. I can identify it as
$ sysctl kern.ident
Ok, we continue down the file commenting out all the options that you don't need. If you are unsure about an option, you have a few choices:
- Safe option - leave it there, your current Kernel has it and is not breaking anything.
- Learning by experimentation - remove it and see what breaks. The problem with this approach is that it might not break during compilation, but when some software tries to use a feature related to that option. Not recommended, but I can see how this might be exciting.
- Investigating - You can use
man -k <option>and check the documentation, or use google to check if you can safely remove the option. Boring but safe, and sometimes more efficient than spending days afterwards figuring out what went wrong. This option is the one I recommend.
After all the configuration options, you come to the device section. Here is where it gets interesting, and we can chop a lot of drivers that we don't use. If we follow the advice on this link by Allan Jude, we should be able to remove SCI, FireWire, and USB device drivers, also all network adapters but
miibus. Well, that wouldn't work for me. When removing devices, you should make sure that you are not removing drivers that you use. You should check
/var/run/dmesg.boot, where you can see all the detected hardware. For me, I had an
em network card:
em0: <Intel(R) PRO/1000 Network Connection> port 0xd240-0xd25f mem 0xd0000000-0xd001ffff at device 5.0 on pci0
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Ethernet address: 00:1c:42:ed:e6:2e
em0: link state changed to UP
em0: netmap queues/slots: TX 1/1024, RX 1/1024
Do a quick scan in your
dmesg.boot and check that you are leaving all the drivers that your hardware requires. Also, make sure that your driver doesn't depend on another driver:
# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure
device em # Intel PRO/1000 Gigabit Ethernet Family
If I comment out
iflib I won't be able to compile because
em depends on
If you keep scrolling down, you'll reach the virtualisation section. Because I'm explicitly using Parallels then I can remove the other virtualisation devices and options:
HyperV - Microsoft
Xen - Xen HyperVisor HVM - https://xenproject.org/
So what's left is
Virtio and from https://kb.parallels.com/4948
Virtio network adapter is the fastest card. However, it works only in Linux and BSD guest operating systems. It is a default adapter for Linux-based OSes.
So I'll leave
virtio and comment out the rest:
# HyperV drivers and enhancement support
#device hyperv # HyperV drivers
# Xen HVM Guest Optimizations
# NOTE: XENHVM depends on xenpci. They must be added or removed together.
#options XENHVM # Xen HVM kernel infrastructure
#device xenpci # Xen HVM Hypervisor services driver
Alright, after we have all our configuration ready in our
PARALLELS configuration file, we can compile our kernel!
But before, let's be smart and create a backup of our current kernel. I'm confident that I did a good job researching the options and devices, but I've made mistakes in the past (Trust but verify).
Creating a backup of our current kernel
Our kernel is located at
/boot/kernel so a simple copy should be fine:
cp -a kernel kernel.2020-05-13
I named mine
kernel.2020-05-13 but you can name it whatever you want(e.g.
We are going to add our backup to the list of possible kernels to load for the system loader. If anything goes wrong, we can go back to our working kernel.
#We can edit /boot/loader.conf manualy or use syrc(8)
sysrc -f /boot/loader.conf KERNELS="kernel kernel.old kernel.2020-05-13"
When we compile and install the kernel, the script creates an automatic backup as
kernel.old, but if we'll be in a cycle of compile/install until we find the correct configuration,
kernel.old can be rewritten with any of our attempts and not our working kernel.
Ok, finally let's compile and install.
Compiling and installing our custom Kernel
Now is when everything comes together. Go to the kernel source, and run
make KERNCONF=PARALLELS buildkernel
If we forego the
KERNCONF variable the script will search for
If you got any errors, read the message carefully and go back and fix your
PARALLELS configuration file accordingly.
If everything went fine and the compilation completes successfully, we install the kernel:
make KERNCONF=PARALLELS installkernel
#This moves the current /boot/kernel to /boot/kernel.od and moves our newly created one to /boot/kernel
And that's it! We can reboot and test.
If the kernel doesn't work, you can go to the boot menu select the working version of your kernel and figure out what went wrong.
Creating a custom Kernel is not a complicated task, it mostly takes time to research the correct options and drivers for your particular case. You might be wondering "how much space I saved?" after creating my custom kernel specially tailored for Parallels. Well, here is the answer:
$ du -sh /boot/kernel*
12M! The things we do for 12M. If we go only for size is not that much, but it was fun. And the less code we have in our system, the smaller the surface area for bugs. If you want to have a look at the PARALLELS configuration file I used, you can find it in this GitHub Gist.
Remember that your hardware might be different, so have a look at
/var/run/dmesg.boot to check your hardware.
Alright, that's it for today.