Host naming organisation for your local lab Jan 28 2020
There are many fields in Computer Science, and tech in general, where you’ll want a lab with multiple VMs, containers, or even physical devices running various services. When learning networks, it’s useful to have a local lab. Another example is when doing Reverse Engineering, we sometimes want to work on an ARM processor instead of our desktop x86 processor, so we create a VM.
You get the idea. At some point in your career, you’ll find yourself surrounded by many hosts running services in different ports. Not having a system to reference your host can become a source of confusion. For example, if you stop for the weekend, the following Monday you have to go through your history to figure out if you were using port 2222 or port 2223 to ssh to the VM you were working on last week. In this short post, I’ll show you some techniques I use to keep my lab organised.
We’ll be using loopback aliases and port forwarding to use default ports on every VM, container, physical device, and easily keep track of each host.
First, let’s get some basics out of the way.
Port forwarding - Running VMs and Containers
When we publish a service it usually runs on a specific port, it doesn’t matter if we use a VM, a container, or a physical device, all of them will have services running on specific ports. For VM’s and Containers, the standard procedure is to do a port forwarding. We match a port on the host to a port on the VM or containers. For example, when defining a docker service (using
docker-compose file), we can define the port mapping:
1 2 3 4 5 6 7 8 9 10 nginx: image: nginx build: context: ./DockerContexts/nginx/ dockerfile: ./files/nginx.dockerfile ports: - 8080:80 - 4443:443 networks: - net-frontend
In this example, we are mapping the port
80 (HTTP) of the container to our host’s port
8080, and the port
443 (HTTPS) on the container to the host’s port
4443. That allows us to access the container’s Nginx server using the local ports. In the host, we can go on our browser to the URL
http://localhost:8080 and that port is going to show us what the container is exposing on port
The same can be done with VirtualBox (There are many tutorials explaining how to do this, here is one). And the same can be achieved when using QEMU. If you followed last’s week post on Running Raspbian OS on QUEMU to learn ARM assembly, you saw the command:
1 2 3 4 5 6 7 8 9 10 11 12 $ qemu-system-arm \ -M versatilepb \ -cpu arm1176 \ -m 256 \ -hda ./2019-09-26-raspbian-buster-lite.img \ -net nic, \ -net user,hostfwd=tcp::5022-:22 \ -dtb ./qemu-rpi-kernel/versatile-pb.dtb \ -kernel ./qemu-rpi-kernel/kernel-qemu-4.19.50-buster \ -append 'root=/dev/sda2 panic=1' \ -no-reboot \ -nographic
The relevant argument for us now is:
1 -net user,hostfwd=tcp::5022-:22 \
With that argument, we are telling QEMU to make a port forward from port
22 on the virtual machine to the hosts port
5022. With the port forwarding in place, we can
ssh -p5022 email@example.com to login via ssh to the VM.
As you can see, it starts to get a little bit tricky keeping track of which port is for which VM. Let’s say I have a VirtualBox VM where we run our network analysis tools, and also have a QEMU VM running my Raspbian OS to do some Revers Engineering on ARM. I want to be able to ssh into both VMs, so I assign a port to each:
Everything clear there, but now every time I have to log in, I have to remember:
1 2 3 4 # I'm going to connect to VirtualBox $ ssh -p 2222 127.0.0.1 # Now to QEMU $ ssh -p 5022 127.0.0.1
The only differentiator is the port, but it’s hard to remember numbers. What if I stop working on the network project for two weeks? When I come back, will I remember which port I used without having to go to the configuration of the VM? Probably not.
So what to do?
Before answering that question, let’s see what happens with physical devices.
When we have physical devices in our network, we assign them an IP address, and that works for the first few days. We use the IP address as the identifier. Let’s say I have a Raspberry Pi in my network with IP
192.168.0.24. Easy, I want to log into the RPi, I just:
1 $ ssh 192.168.0.24
But what happens when I also have another computer I use as a server? I assign the server a different IP, let’s say
188.8.131.52. I think you begin to see the problem.
With the VM’s and containers, the problem was the ports, and with the physical devices, the problem is the IPs. Numbers are hard to remember, so what will any sane person do? Use mnemonics.
As you probably know, we already solved that problem for the Internet by using DNS(Domain Name System). With DNS, we map the IP of servers on the internet to easy to remember names. But that is quite a complex solution for us. So unless you want to run a full DNS system on your local computer let’s take the easy way of editing our
/etc/hosts file :).
On Linux or macOS you’ll be able to use the
/etc/hosts file to map IPs to names. This names will be resolved before querying the DNS servers. The syntax is quite easy:
1 127.0.0.1 rderik.local
We write the IP followed by a string. In the example, we are mapping
rderik.local to my localhost’s IP address
127.0.0.1. Let’s extend the example and add our RPi and local file server (To edit the
/etc/hosts file you need admin privileges so use
1 2 3 127.0.0.1 rderik.local 192.168.0.24 mypie 192.168.0.25 alexandria
I can easily remember those two names (Alexandria, in honour of the Great Library of Alexandria, to keep all my documents).
1 2 3 4 # to connect to my Raspberry Pi $ ssh mypie # to connect to my file server $ ssh alexandria
Alright, that solves the problem with the IPs, but what to do with the VMs and containers?
iptables (Linux) and
pf (macOS, and BSDs).
Port forwarding with
We know that we can use the
/etc/hosts file to map IP addresses to names, so you might think, “alright, we just need to add the port to the IP, and we are done!” Good idea! But alas, no. It doesn’t work like that. It’ll be great, but no.
So what can we do?
Make use of all your knowledge
First, let’s make clear what we want. We want to be able to reach any of our services through an easy to remember name. For example, we would like to ssh directly to our QEMU VM, and not have to specify and remember a port.
We know we can assign a name using the
/etc/hosts file, but that would only save me from typing
localhost. I would like to go directly to the VM, with an instruction similar to:
1 $ ssh myemu
The problem is that ssh will use the default port (
22), and the port we want to connect is
5022 was the port being used by the VM we created using QEMU.
Alright, let’s pause for a moment and think of what we currently have.
We had a VM that runs open ssh server on port
22, and we did a port forward to access it through our localhost but using the port
5022. Why did we change the default port
We did it because we would like to run many VMs or containers, and we could only have one process running on a specific port. So in our example, we mapped QEMU VM’s port
22 to our localhost on port
5022, and VirtualBox VM’s port
22 to port
2222 on our localhost.
If there were a way to distinguish which service we want to connect, we could do the correct port forwarding. Well, there is, but it’ll require some clever use of network interface aliases.
Using network interface aliases to our advantage
What we are going to do is we are going to create an alias for our loopback interface and use that alias to identify the service we want to access and then do the correct port forwarding. It’ll make more sense once we’ve done it, so stay with me.
First, let’s define what a loopback interface is:
“The loopback device is a special, virtual network interface that your computer uses to communicate with itself”.
Now, let’s create an alias to the loopback interface:
1 2 3 4 5 6 # on macOS $ sudo ifconfig lo0 192.168.0.101 alias # on Linux $ sudo ifconfig lo:1 192.168.0.101 # if you want to add another increment the index for example: $ sudo ifconfig lo:2 192.168.0.102
We created an alias for our loopback interface that we can access via IP
192.168.0.101. You can check using
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # on macOS $ ifconfif lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP> inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 192.168.0.101 netmask 0xffffff00 nd6 options=201<PERFORMNUD,DAD> ... # on Linux $ ifconfig ... lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 92 bytes 6940 (6.9 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 92 bytes 6940 (6.9 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo:1: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 192.168.0.101 netmask 255.255.255.0 loop txqueuelen 1000 (Local Loopback) ...
As you can see we access the
lo on Linux) via
Now we can modify our
/etc/hosts to point to that address.
1 192.168.0.101 mypie
(Assuming our QEMU VM’s ssh port is
Let’s test our configuration. We can log in to our QEMU using ssh and specifying the port:
1 $ ssh -p5022 pi@mypie
To avoid using the port, we’ll do a port forward.
1 2 3 4 5 6 7 8 # on macOS $ echo ' rdr inet proto tcp from any to 192.168.0.101 port 22 -> 127.0.0.1 port 5022 rdr inet proto tcp from any to 192.168.0.101 port 5022 -> 127.0.0.1 port 5022' | sudo pfctl -ef - # on Linux $ sudo iptables -A OUTPUT \ -t nat -d 192.168.0.101 \ -p tcp --dport 22 -j REDIRECT --to-port 5022
We are redirecting
22 on IP
192.168.0.101 to port
5022(and also keeping the
5022 available), so now we can go directly to:
1 2 3 $ ssh pi@mypie # and it works, even if we want to specify the port for any reason $ ssh -p5022 pi@mypie
We should be able to login now without a problem.
That’s it. You can see now how we redirect to the specific port for each service using the loopback aliases to differentiate each service. Also, using the
/etc/hosts to map it to the IP and everything ties up nicely.
This setup won’t persist through reboots, so we have to do some more configuration to make it persistent. It was important to do it manually first so we can test and have the option to reboot if we make some big mistake that we prefer to reset everything than revert step by step.
Alright, let’s see how to make it persistent.
Making the changes persistent
Good news is that we already have done the hard part, figuring out how to distinguish our services and access them via an easy to remember name. The bad news is that making the changes persistent will depend on your system. I can’t list every possible configuration for each system. What I’ll do is show you how to do it for two of the most common systems, Ubuntu and macOS. And using them as a base, you can figure it out for your specif case.
Let’s start on Ubuntu.
Ubuntu is using Netplan instead of
ifconfigupdown, so I’ll be using Netplan. First, create a file to configure the loopback with the IPs we want to use. I’ll create a file in
/etc/netplan/ with the name
loopback-init.yaml. Add the content:
1 2 3 4 5 network: ethernets: lo: addresses: - 192.168.0.101/32
You can apply the changes(or reboot the whole system):
1 $ sudo netplan apply
Let’s check the interface:
1 $ ifconfig
If you check the output, you won’t be able to see the IP address but if you use
ip addr you’ll see the additional address added to the loopback interface. We are not creating a new loopback interface, we are only assigning a new IP address so you can’t see it with the
ifconfig command, but you can see it with the
1 2 3 4 5 6 7 8 9 10 $ ip addr # You'll see something similar to this: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 192.168.0.101/32 scope global lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever
If you haven’t run our
iptable configuration, do it now:
1 $ sudo iptables -A OUTPUT -t nat -d 192.168.0.101 -p tcp --dport 22 -j REDIRECT --to-port 5022
We can now save our current
iptables configuration to a file:
1 $ sudo sh -c "iptables-save > /etc/iptables.rules"
We need a script to run the
iptables configuration every time we reboot, so we are going to take advantage of
networkd-dispatcher. Per the instructions in netplan.io documentation, we’ll add a script inside
/etc/networkd-dispatcher/routable.d/. We’ll name the script
loopbach.sh, and add the following content:
1 2 3 4 #!/bin/sh iptables-restore < /etc/iptables.rules exit 0
Let’s make it executable:
1 $ sudo chmod +x /etc/networkd-dispatcher/routable.d/loopback.sh
Now every time we reboot the
iptable rules will be restored, and we would be able to access our example VM from a nice and easy to remember name. For our example:
1 $ ssh pi@mypie
Now for macOS.
Remember, on macOS, we are going to use port forwarding. We want to be mindful of other configuration previously set on the system and also follow the same configuration patterns. So we are going to put our configuration on
/etc/pf.anchors/, let’s create a file called
com.rderik.loopback.redirect. You can name it however you want, but make it clear that it’s a custom configuration. Add our rules (A trailing new line seems to be required):
1 2 3 rdr inet proto tcp from any to 192.168.0.101 port 22 -> 127.0.0.1 port 5022 rdr inet proto tcp from any to 192.168.0.101 port 5022 -> 127.0.0.1 port 5022
The second rule allows us to keep access to the port
5022, so we can keep using it if we so desire. The first rule provides the redirect from port
Now let’s add our configuration to
/etc/pf.conf, my file looks like this (A trailing new line seems to be required):
1 2 3 4 5 6 7 8 9 scrub-anchor "com.apple/*" nat-anchor "com.apple/*" rdr-anchor "com.apple/*" rdr-anchor "com.rderik.loopback.redirect" dummynet-anchor "com.apple/*" anchor "com.apple/*" load anchor "com.apple" from "/etc/pf.anchors/com.apple" load anchor "com.rderik.loopback.redirect" from "/etc/pf.anchors/com.rderik.loopback.redirect"
Now we can test our configuration:
1 $ sudo pfctl -vnf /etc/pf.conf
If everything was successful, we load the rules:
1 $ sudo pfctl -F all -ef /etc/pf.conf
We can now ssh to our VM:
1 $ ssh pi@mypie
Alright, now is where it gets hairy. You need to decide if it’s worth it for you to set it up to be persistent through reboots.
First, let’s create a small script that you could run on every reboot. It won’t be complicated. You can make it as complex as you want, I’ll just show a basic version but I encourage you to improve on it (I’ll call it
1 2 3 4 5 6 7 #!/usr/bin/env bash set -e #Create the interface aliases ifconfig lo0 192.168.0.101 alias # Load the pf configuration pfctl -F all -ef /etc/pf.conf
You could run that every time you reboot your computer and that should be fine. Now, if you want it to run automatically, we need to work on creating a
If you want to learn more about
launchd agents, you can check my post on Creating a Launch Agent that provides an XPC service on macOS using Swift, especially the section “Understanding Launch Agents”.
In general, you’ll need a
plist that defines the launch agent. Our agent will only run the script we created before, Again, I’ll make the most basic version, and you can improve on it. Here is the
plist I’ll call it
1 2 3 4 5 6 7 8 9 10 11 12 <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.rderik.loopback.infrastructure</string> <key>RunAtLoad</key> <true/> <key>Program</key> <string>/Users/derik/loopback_infrastructure.sh</string> </dict> </plist>
And we can add it to
It’s not super complicated, but you’ll be the one to decide if you want to spend the time creating and testing the agent or just use the script.
And that’s it.
That should give you enough information to build the organisation for your VMs, containers, and devices.
We saw a possible lab naming organisation that we could use to reduce the memorisation of IPs and ports for your hosts (virtual machines, containers or devices). As always, there is a trade-off. Every time we add a new host or services, and we want it to follow the same infrastructure, we need to take it into account, to keep everything consistent. We have to see which of the following steps are necessary:
- Add the loopback interface alias (Make sure we are not clashing with any other IPs on the network).
/etc/hoststo add a new name to the IP
- Configure the port forwarding
- In our anchor file, if we are using macOS.
- Using iptables, if we are using Linux.
This process generates some overhead, and you’ll be the judge in making sure it makes sense for your lab.
The increase in complexity is also something that I’ve seen in DevOps. Sometimes we get carried away and end up with a massive infrastructure that could easily be replaced by a couple of well-thought shell scripts. And we end up investing a lot of time and resources in maintaining the infrastructure instead of building the product.
But it’s also your lab, so have fun, use clever names. Maybe choose your favourite book and use characters from there. Or from a fantasy universe(e.g. Cosemere if you like Brandon Sanderson). Find what works for you, naming is hard, but make it fun.
Anyways, I tried to keep the configuration as simple as possible and assumed there was no other previous configuration. I hope you got a general idea, but you’ll still have to adapt it to your specific case. Also, the configurations shown in this post don’t take into account security. If you are running a malware analysis lab, please review the rules correctly and restrict access to only the necessary hosts, so there is no leakage to your host system.
So that’s it for this post. I hope you find it useful. Let me know what you think and how you keep your hosts organised.
Related topics/notes of interest
- pfcl - cheat sheet
- A StackOverflow answer on how to set up an agent that calls ifconfig
- If you are using a Netplan on Ubuntu, the FAQ has a good reference on how to “simulate” hooks as we did it before using ifupdown for loading the iptables rules.
- If you want to check how we used to do it with ifupdown, you can check this post.
- You could also check Ubuntu’s help on using iptables-persistent.
- And if you want to have a look at how to use Netplan, you can check their examples.
- Useful iptables patterns and how to list and delete iptables rules.
- Port forwarding with pfctl.
- A couple of examples on how to do some NAT and port forwarding using
pfctl, On stack overflow: NAT, port forwarding and how to maintain the original port available.