HostingGeek

Ever since the first time I heard about Xen and its ability to run any OS side by side on the same server I have had the urge to run a BSD based OS with a Linux OS. Today I have sucessfully achieved my goal, and this is how I did it.

First some background on the server itself. The server is a Dell PowerEdge 1750 with Dual Xeon processors and 3GB of RAM and 500GB of RAID storage. The server is running the Xen 3.0.2 hypervisor kernel (the main kernel that handles the paralization, or virtualization, of the hardware). The Dom0 system is running Debian 3.1 with some patches to the kernel to work with the LSI based RAID 5 card in the server. Each virtual OS installed on the server is given its own partition and is managed using LVM in Dom0.

The vast majority of information about NetBSD running under Xen as a DomU seems to be either Xen 2.0 specific, or assumes you are running NetBSD as Dom0. Unfortunetly, the Xen 2.0 information is no going to work on a Xen 3.0 machine, and more so our Dom0 is Debian, so we needed to come up with our own.

Here is how I did it, and what sort of problems I encountered.

The entire process is pretty easy, but finding the actual information can be tough, and finding the files you need can be even tougher. Here is kind of a rough over view of the process…

1) Set up your partition that will hold NetBSD. We are using a LVM partition named vg00-netbsd.
2) Set up the xen domU config file.
3) Boot the netbsd install kernel for Xen 3.0.
4) Follow the sysinstall steps like you normally do to install NetBSD. I had to use an FTP based installation, because I could never get the CDROM to work correctly.
5) Complete the install and shutdown NetBSD.
6) Edit the domU config file and change the kernel from the install kernel to the normal NetBSD kernel.
7) Boot NetBSD DomU and enjoy.

So here are the specifics.

Step 1: You need to download the NetBSD Xen 3.0 kernels (install and normal) and put them some place on your Dom0. I put mine in the /boot of the server, because it sort of made sense to me, but they can be almost anywhere. You can download the DomU kernels from NetBSD’s FTP servers from the daily build areas. The kernels for Xen 3.0 are not in the release versions of NetBSD so you have to find them. I would post links to them, but most likely the would go stale over time. Go to ftp://ftp.netbsd.org/pub/NetBSD-daily/ and navigate through to either the NetBSD 3.1 tree or the NetBSD 4.0 tree. You are looking for the directory i386/binary/kernel/ in that directory you will find the two kernels you need. The install kernel is called netbsd-INSTALL_XEN3_DOMU.gz and the normal kernel is named netbsd-XEN3_DOMU.gz. Download both of those kernels as you will need them later.

Step 2: Once you have downloaded your kernels you will need to create a xen config file for your NetBSD DomU. Here is an example of the one I used:

kernel = “/boot/netbsd-INSTALL_XEN3_DOMU.gz”
memory = 128
name = “netbsd”
vif = [ ” ]
disk = [ ‘phy:/dev/mapper/vg00-netbsd,0x01,w’ ]
root = “/dev/wd0d”

You will need to change the disk = line to match where you are installing NetBSD to on your server. After you have created that file in your xen config directory (our was /etc/xen/).

Step 3: We are ready to boot NetBSD for the first time. To boot NetBSD we run the command:

xm create -c netbsd

“netbsd” is the name of the DomU config file we created in step 2, so change that to match what you used in that step.

A couple of times we noticed that Xen didn’t attach us to the console of the booting NetBSD DomU, so you may need to connect to it manually. To do so do the following:

xm list

Which will print out a list of running Xen instances like this:

Name ID Mem(MiB) VCPUs State Time(s)
debian 0 1374 4 r—– 2354.5
plesk 10 1024 1 -b—- 161.5
netbsd 50 128 1 -b—- 1.2
qmail 8 128 1 -b—- 953.4

We will need to know the ID of the instance we want to attach to. In the example above this is 50. Then we attach to the console of that DomU by typing:

xm console 50

To break out of the console at any time simple press CTRL+] at the same time.

Step 4: Once you are in the console you should see the sysinstall application. You can follwo the prompts and install NetBSD like you would normally do. One problem I did encounter was that for what ever reason the server would stop talking to the FTP server due to some sort of DNS lookup failure. It did this no matter which kernel I tried. I eventually resorted to using the IP address instead, and the installation worked perfectly.

Step 5: Once the install is completed, break out of the server and shut it down via the command:

xm shutdown 50

Again replace 50 with the ID of the DomU of your NetBSD install.

Step 6: Edit the DomU file and change the kernel line to point to your normal NetBSD kernel. So your DomU config file should look somethnig like this now:

kernel = “/boot/netbsd-XEN3_DOMU.gz”
memory = 128
name = “netbsd”
vif = [ ” ]
disk = [ ‘phy:/dev/mapper/vg00-netbsd,0x01,w’ ]
root = “/dev/wd0d”

Step 7: Reboot your NetBSD DomU via the command:

xm create -c netbsd

Enjoy your NetBSD running under a Debian/Linux Dom0 in Xen 3.0!

Gotchas:

Having used the NetBSD system only breifly I have noticed that there is something “funky” with the networking and the way it behaves. I noticed over a sustained ping that the network interface starts to drop packets, every other packet it seems. Modifying the vif = line in the DomU config to read:

vif = [ ‘bridge=xenbr0’ ]

seems to have cleared up the issue. This line bridges the ethernet interface inside the DomU to the xenbr0 interface in the Dom0. It seems to have cleared up the issue to date.

And there
you have it! NetBSD running under a Linux Dom0 on top of Xen 3.0. The world just got a whole lot smaller.