HostingGeek

Recently I did a Xen 3.0 install on a Dell 1750 server with RAID5. By default the Xen 3.0 installation (from source) doesn’t compile in the LSI drivers for the RAID card, so I had to enable the option in the Dom0 kernel in Xen.

So I went in to the xen0 directory under my Xen source (/usr/src/xen-3.0.0/linux2.6.12-xen0) and ran:

make ARCH=xen menuconfig

I then went to Device Drivers -> SCSI Device Support -> SCSI low-level drivers -> and enabled the “LSI Logic New Generation RAID Device Drivers”, because after all… Newer is better right?

I recompiled Xen and the Xen kernels (cd ../; make kernels; make install) and reboot the machine and went on with my day.

I got a call from the client later in the day that the server was reporting a problem with the RAID array. Long stroy short, the customer was running a NAGIOS plugin to monitor the RAID array via the /proc/megaraid entry. Little did I know that the new generation RAID device drivers in the 2.6 kernel no longer make use of the /proc/megaraid proc entry, so it was breaking the NAGIOS plugin.

To correct this problem use the “LSI Logic Legacy MegaRAID driver” instead of the “LSI Logic New Generation RAID Device Drivers” in the 2.6 kernel (this is not an option in the 2.4 kernel). That will give you back the /proc/megaraid entry and you should be all set.

Additionally, since you are already compiling a kernel, you should enable the kernel option “Use register arguments” under “X86 Processor Configuration”. This option will allow you to use the Dell OMSA device drivers for additional control over your Dell PowerEdge server. You can read more about it here: http://www.uta.fi/~pauli.borodulin/dellomsa/omsa44.html

I have many servers that I have to watch on a regular basis. We use a KVM to switch between the consoles of the servers and generally we only see the console for any given server for a couple of seconds.

The KVM we use has this great feature where if it detects any activity on the console, the KVM will swtich to that console and display the output for us.

This is how we use that functionality to quickly display any problems a server might be having. It is by no means a fool proof and accruate, but it does help to bring things to our attention on a regular basis, doesn’t require you to install anything special (all programs should come with a basic install of a linux OS), and its cool… What else do you really need?

What we do is one turn off the console’s screen saver. It is evil, and we don’t like it. Actually it serves a valid purpose, but not for our needs so lets turn it off:

/usr/bin/setterm -blank 0

Now you can put that into /etc/rc.local for RedHat based distros (Fedora, Redhat Enterprise Linux, CentOS, etc).

For Debian you can create a file in /etc/init.d/ and name it blank_screen and chmod 755 the file. Then just create a symbolic link to the startup dir:

ln -s /etc/init.d/screen_blank /etc/rcS.d/S61blank_screen

Now that we have the console set not to turn off, we can do the next part of our monitoring mojo. We will setup to monitor files that are important to us, such as /var/log/messages, and /var/log/syslog. These files might not be the files you want to monitor, so you can modify the line below to match the files of your choosing.

We use tail, because it displays the lines as they are added to the file we are monitoring, and it is simple and lightweight.

/usr/bin/tail -v –follow=name /var/log/messages –follow=name /var/log/syslog > /dev/tty2

What this does is runs tail to monitor the files /var/log/messages and /var/log/syslog. We monitor the filename, rather than the descriptor (which is the default behavior of tail). Monitoring the descriptor means that if the filename is changed — for example logrotate, then we will follow the file to the new name. Since once a file is rotated by logrotate it won’t be updated, we don’t want this behavior. We add the -v (verbose) so that tail will tell us the name of the file that corresponds with the output. Then we redirect the ouput to /dev/tty2 (virtual console 2).

This will output the information directly on the screen so that we can see it via the KVM.

Additionally you might want to turn off the login prompt on the virtual console 2. We can do this by editing the /etc/inittab file and commenting out the following line:

2:23:respawn:/sbin/getty 38400 tty2

You can just put a # in front of that line and then have init re-examine the file by running:

kill -HUP 1

So there you go… you now have the output of any file you want to the console of your server. Set your KVM to autoscan, and enjoy your easy to view logs.