Protocol analysts of DRAC remote console

As you know by now, I have been playing with Dell's remote consoles in hope that I will be able to connect from my Linux to Dell's RAC reliably. Currently, I have to run Windows XP with Internet Explorer and Java in kvm to have access to my servers, and that's clearly not reliable combination.

DRAC is PCI card which is presented to system as VGA which then transfers screen updates over the network to client. It also allows virtual media, but in a sense, it's mix-up of http over ssl and few propitiatory protocols:

  • 443 - https interface
  • 3368 - virtual media (proprietary)
  • 5900 - keyboard and mouse (ssl encrypted)
  • 5901 - video redirection (optionally ssl encrypted)
It's very strange that all documentation calls 5900 video redirection port and 5901 keyboard/mouse redirection when all traces of traffic between client and server clearly show that ports are swapped in implementation.

Did you notice ssl encrypted keyboard/mouse channel? I first decided to tackle this problem with well known SSL man in the middle approach. I decided to use simpliest possible approach first using something like: 

apt-get install stunnel

openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout cert.pem

# https mitm
stunnel -p cert.pem -d 443 -r 5443
stunnel -c -d 5443 -r 10.60.0.100:443

# 5900 mitm
stunnel -p cert.pem -d 5900 -r 5999
stunnel -c -d 5999 -r 10.60.0.100:5900
and than recoding all output using wireshark: 
sudo tshark -w /tmp/drac.pcap -i any 'port 5999 or port 5901 or port 5443'
This allowed me to capture all unencrypted traffic into single pcap file which proved very useful for initial protocol analysis using wireshark. In short, you have to do following:
  1. make https connection to https://drac/cgi-bin/webcgi/winvkvm?state=1 and acquire vKvmSessionId console redirection authentication key
  2. connect to keyboard/mouse port 5900 forcing SSL_cipher_list to supported RC4-MD5 cipher and send vKvmSessionId
  3. connect to video port 5901
Finding supported cipher for communication between us and server was a real problem. They are using openssl-0.9.7f and I had to downgrade all the way to Debian woody to make stunnel work. Same problem is visible with latest firmware update for DRAC where Active X plugin doesn't have old enough configuration in SSL handshake and doesn't work any more. Java plugin, on the other hand, provides much more cipher options, so one of them still works. ssldump was very useful for finding such problems.

Fortunately, kost was much more persistent than me, and he found out that adding 'SSL_cipher_list' => 'RC4-MD5' will force supported cipher. Armed with that new finding, I was able to modify kost's ssl mitm script up to the point where I can see decrypted key-presses, mouse movements and video settings. Hack, I even wrote drac-vkvm.pl async client which does steps outlined above.

All is not well, unfortunately. When sending authentication request, we need vKvmSessionId which we get from web server, but packet which is sent contains also two bytes which change with session. I haven't been able to figure this part out, and since same two byte sequence is needed to open video channel (to see VGA output) so I'm stuck.
Bytes don't look like crc16, and source code doesn't provide any hints about secondary 16-bit auth info. It seems that client calculates it somehow, since both connections close when I try to send different values for it.

I could write session recorder, but that isn't terribly useful, because it still forces me to use Windows+Java setup to access my console. I will collect usefull snippets about Dell's RAC protocol on wiki.