|Notes to Self|
|Serial Over LAN|
See also PXE boot and IPMI
IPMI allows you to configure the BMC on Dell machines (probably others) to talk to the computer via its serial port and relay the input, and output to another machine, over the LAN interface. E.g. Serial over LAN support. SOL requires IPMI version 2 which is only available on a few Dells at this time (see here for details. This is quite some ways from a replacement for a non-graphical console (e.g. KVM), but it's close enough to get excited about it. If it works.
This interface requires that everything related to the console that you are interested in seeing and responding to must use the serial console. That is:
All of these must be configured to write to the serial port, rather than (or in addition to) the normal console. Sun machines have been doing this for ages. This sounds like an really long complex list, but fortunately, Dell and probably some other manufacturers offer an option to forward some console traffic over the serial port. This means that you don't need to configure all of the above, just a couple. Read on for my experience with Dells.
My goal is just to get through a basic boot - in particular, boot the machine, be able to invoke BIOS or much more interestingly, invoke PXE boot.
This sequence is a necessity if you have dozens or more machines at a remote site and need to install a new system (e.g. move from i386 to amd64). If just one of the steps requires you to press enter, you're screwed - you need a KVM (keyboard and VGA monitor, mouse not needed). Many places have invested in a serial console switch like those at http://www.kvms.com/ to name just one place. Yikes, they are expensive!
For this you need a console because SOL is not enabled yet. I'd hoped this would work on all my PowerEdge machines (SC1425, PE1850, PE1950 and PE2950). SOL is new enough that older machines don't support it at all, or enough. For this to have a chance to work, your machine's BIOS must support IPMI 2.0. In my world that means only 1950s and 2950s work. Boot into Dell's BIOS and choose the following:
Move to 'Serial Communication' Set Serial Communication ....... On with Console Redirection via COM2 External Serial Connector .. COM2 Failsafe Baud Rate ......... 57600 Remote Terminal Type ....... VT100/VT220 Redirection After Boot ..... Enabled Save and reboot
Note - enabling 'Redirection After Boot' works wonders. You do not need to configure Grub or Syslinux to use the serial console, since Dell's BIOS is handling that for you. In fact, setting up serial console for GRUB (as described in the URL above) actually messes things up. Don't configure Syslinux either... Dell's redirection is all the magic you need for that part.
You must tell the kernel to generate messages to the serial port as well as the console. Almost certainly your kernel was compiled with serial console support. Mine was, so I don't have any notes on what to enable, sorry. Enabling serial console consists of simply passing some parameters to the kernel at boot time.
vi /boot/grub/menu.lst # Add 'console=tty1 console=ttyS1,57600n8' after 'ro' kernel /vmlinuz-2.6.24-mos root=/dev/sda3 ro console=tty1 console=ttyS1,57600n8 # In addition add this to kopt= keyword too so it is propagated # when grub-update is done (probably not wanted) # Note kopt is a COMMENT in your grub menu file. # kopt=root=/dev/sda3 ro console=tty1 console=ttyS1,57600n8
The URL above has notes about changing getty. I did not need to do anything with this, Debian seemed to have it all set up. Lucky me. As a final step you need to reboot this machine so the kernel gets these parameters.
At this point you should have a machine (I'll call it HOSTNAME) which has the BIOS changes made and has been booted with the serial console parameters. On another machine do:
ipmitool -I lanplus -H HOSTNAME -U ROOTUSER sol activate Password: [SOL Session operational. Use ~? for help]
This just invokes ipmitool to communicate with HOSTNAME using IPMI. The 'lanplus' option means to use IPMI v2.0. This is where you'll find out if your HOSTNAME actually supports the protocol you need. If you get the SOL Session prompt, you're set. Now reboot HOSTNAME (ipmitool can powercylce the machine), you'll see something reassuringly familiar - those messages from BIOS.
F2 = Setup Phoenix ROM BIOS PLUS Version 1.10 1.0.1 F10 = Utility Mode Copyright 1985-1988 Phoenix Technologies Ltd. F11 = Boot Menu Copyright 1990-2006 Dell Inc. F12 = PXE Boot All Rights Reserved Dell System PowerEdge 1950 www.dell.com Testing memory. Please wait. etc
Realize this SOL is a pretty ugly terminal interface. There will likely be lots of funny looking characters etc, but the good news is that it can be used. You just need to know how to enter some special characters - press the escape key once and then one more character. ESC + 1 sends F1 to HOSTNAME. ESC + 2 sends F2. You can guess F1 to F9 easily enough. ESC+0 is F10. ESC+! is F11 and ESC+@ is F12 (for PXE boot). The arrow keys work in Setup as you'd hope. The last thing you need to know is how to get out of this:
~. # Yes, tilde + period > # Now back at your normal shell prompt # Now deactivate the interface ipmitool -I lanplus -H HOSTNAME -U ROOTUSER sol deactivate
All it all, SOL is pretty easy to get going, your BIOS willing. I'm not interested in doing much with the kernel after it gets booted, so I've not played with that at all. I just wanted access to those things before the kernel - so I can control what gets booted (or installed).
Once SOL is working for console support, you might want to set up a serial TTY so you can get in when SSH or the network is not working. Once the machine is booted, the serial port can be used for a conventional login. Here are some notes on what to do for Debian systems. Other systems will be similar.
Modify /etc/inittab to enable the serial getty by adding this line:
T1:23:respawn:/sbin/getty -L ttyS1 57600 vt100
The speed setting may need to be changed for your hardware. Most recent systems can handle 57600, but some now default to 115200 (although I bet you can set this in the BIOS). The device name (ttyS1) could differ on your system too. Anyway, after changing inittab, do this to get the getting working:
kill -HUP 1
On Ubuntu systems you'll find the file /etc/event.d/ttyS1 and you should modify the line
exec /sbin/getty 57600 ttyS1
with the correct speed etc. Then issue the command start ttyS1 to get it going. Now you're ready to open the serial console connection with:
ipmitool -I lanplus -H IPMIAddress -U IPMIUser sol activate
You may need to hit 'enter' to get a login prompt. If there is already and active connection elsewhere deactivate the serial connection as mentioned above.
Pretty slick. This set of tricks has made us decide to NOT purchase a serial KVM, avoiding all that cabling. A clear win for us.