Slashdot Mirror
Low-Bandwidth, Truly Remote Management?
kaiser423 writes "I'm looking to integrate some highly critical solutions into what would essentially be a remote, moving datacenter. No operators will be allowed at the site, and we may be able to have a high-speed INMARSAT data link. As a backup, we're planning to have multiple redundant low-speed Iridium data links. Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs. We're willing to write the terminal interfaces necessary for our programs, and possibly do the remote desktop thing with some of our 3rd-party programs. But what is out there that would give us this type of access, work robustly over a high-latency, low-bandwidth stream, and would be tolerant to intermittent network outages? Please hold the pick 2 of the 3 jokes, I know they're contradictory goals; I'm looking for a compromise here! These boxes would regrettably nearly all be running Windows (with some VxWorks). Does anyone out there remember those days, and have any solutions that they preferred?" Read on for a few more details of this reader's requirements.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
There's a surprising amount you can do from the command line within Windows these days. For UI access, RDP beats the common alternatives hands down, even if you log in just to use a command prompt remotely and thus have console state stored between sessions if the connection goes down. Have you actually tried this?
I wonder if anybody can put some numbers on the latency and bandwidth? I spent four months in China maintaining Windows servers in California via RDP. With latency often around 600-750ms and packet loss, it was painful but still usable. I was even contending with nested RDP sessions (RDP over the VPN to a machine in an office in CA, and then RDPed from there to a colocation facility).
"Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs."
Dial in using the telephone system and use a sequence of DTMF tones on your telephone keypad to carry out a task. This will be low bandwidth (about 2,700 Hz) and low cost.
Ganty
Good ol' RS-232 let's you do a lot. Run one very low power board that can sit there listening to RS-232 input and act on commands. It can then toggle the power of other equipment plus route messages from them however you choose.
The world is made by those who show up for the job.
Tivoli Management Framework had configurations designed to work with satellite links as slow as 16k. That solution was for monitoring and configuration management though -- not what you want.
Your big problem here is your expectations. Remote Desktop over a slow-speed, high latency link just isn't viable. Anyone paying the megabucks required to support a field-deployed solution will not be happy with the crappy service you'll ultimately provide.
You need to extensively model how your application works and develop appropriate procedures, runbooks for your remote operators and a toolset of programs or script to provide support for this "critical" solution.
Conformity is the jailer of freedom and enemy of growth. -JFK
Two words will get you far in this situation, Command Line. Low bandwidth, latency tolerant, and generally asynchronous. If you can get any tools with a command line option, embrace them.
GUIs suck, and they suck more over the conditions you describe. Avoid them like the plague. Also, think about mirroring the files you need to manage and editing them locally, then uploading them when you are done. Not always possible, but if it is doable, it can make your life a lot easier.
Scripting is your friend here.
-Charlie
Perhaps I am not understanding what the issue is with using the standard console servers and PDUs out there? All serial access is pretty darn low bandwidth.
http://www.avocent.com/products/serial-consolemanagement.aspx
http://www.raritan.com/products/serial-console-switches/
Plug one of these in, then connect a serial cable to your servers. Many include a modem if you have a pair of copper wires for a phone line so you can keep it out of band.
http://www.bitvise.com/winsshd It does the job connecting all kinds of platforms/client implementations. It does PKI too.
HP's Compaq line of servers has **excellent** remote admin capabilities.
Push the whole thing over an stunnel and you are good to go.
Implementation is another issue. Publish an email if your budget supports consultants. Errmm. Well, it looks like slashdot is taking the place of a qualified expert, so good luck with that.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
We use VNC and NetOp with our satellite sites. It works decent. It is slow (maybe 2 minutes) to authenticate with AD when you initially log on (if you log in locally, it is faster), but once you are authenticated, things work pretty good. There is definate latency between when you click the mouse and action happens, but it is definately usable.
As for the iridum setup, you might want to check those speeds again. When we looked into it, they were only able to offer a 2400 baud serial connection over the Iridium system. That is REALLY slow, and with high latency. We decided against going with the backup.
I'm not sure where you are planning on deploying these setups, but maybe a cellular modem would fit your needs more. They are fast, low latency, and comparable in price. The only hitch with them is that there is no SLA; just best effort. If you are out of coverage area, try to hook up a high gain antenna to the cell modem and try again.
I would just use remote desktop tools to manage your servers. VNC especially works quite well with low bandwidth, high latency connections.
One thing to let you know though. Make sure that you have someone that can go to the site and has access to the network equipment. We have almost 20 sites like this, and about 2 go down a month. The sat-modems or sat-controllers sometimes need to be rebooted, and having someone near the site to do that can save you guys A LOT of money. When we have to send out a tech, it costs around $2,000.00 due to driving charges.
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
We've been extremely happy with the eRIC remote management board from Peppercon for all our lights-out remote-datacenter Windows machines. We use it over landline dialup modem (33.6) but it will work fine over any serial link you can throw at it, as long as you're patient. Full graphical remote console, remote IPMI, remote reboot, remote poweron, indepdent power supply (optional), it's great. They're a little hard to track down in the US, but I believe Raritan distributes them now. They're not cheap, but if you're paying for INMARSAT and Iridium, you don't care.
The folks at RealWeasel have a cute little device. Plugs into a PCI slot and emulates a VGA card. It then outputs, over rs-232, a serial console approximation of whatever the system is displaying on the VGA device. Also has watchdog, manual reset, and keyboard functions. Those, plus a bog standard serial terminal server, and you are all set.
Agreed.
I remotely manage 50+ ProLiant servers this way, mostly via SSH to the iLO. Unfortunately since you want/need to run Windows on them you'll have to go graphic mode (via web interface and a Java app) to the OS. (I manage Linux servers so I can do it all via CLI.) You can even do remote installs via virtual media that mounts your local CD/DVD drive (or ISO image) on the remote, although that'd be painful at your speeds.
-- Alastair
Start with your server hardware. Most Dell servers have a Dell Remote Access Card which allows you to get a full console (including BIOS and power on options) via web page. Performance of the full GUI over a slow link is marginal however. I'm sure other server vendors have similar options.
Also or alternatively, look for BMC controllers (http://en.wikipedia.org/wiki/Baseboard_management_controller) and IPMI interfaces (http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface).
From there address further needs with RS232, Telnet, SSH, etc. Step up to RDP and VNC for GUI needs.
I'm going to go back in my box and will think within the limits of my box: MS Sucks Linux Good I read too much Slashdot.
He's one of Santa's elves. Santa's trying to get into the 21st century here. He's tired of having to do all this work. He's deploying unmanned present delivery machines. This elf is just freaking now because santa gave him all year to work on it but he spent it drinking cocoa and snorting candycane and now he's got less than a month left before he needs to demo it.
You are using English. Please learn the difference between loose and lose; they're, there, and their; your and you're.
Windows 2003 server actually has serial console.
Just do /EMS ON /PORT COM1 /BAUD 19200 /ID 1 /r /t 0
BOOTCFG
and reboot:
shutdown
Painfully untrue, especially in Server 2008 (for which the core install doesn't even have a GUI). There are scripts, tools, and other things that make remote administration of windows possible in many ways that were much harder, previously. No GUI needed.
Painfully untrue, especially in Server 2008 (for which the core install doesn't even have a GUI). There are scripts, tools, and other things that make remote administration of windows possible in many ways that were much harder, previously. No GUI needed.
Seconded.
The real question is how much bandwidth you need for the WMI calls/data that all the new MMC's use... could potentially give him a GUI to work with over his slow connection if he so desires.
Boot Windows, Linux, and ESX over the network for free.
As someone who works for a Microsoft Gold Partner I suppose I ought to defend Windows Server 2008 but the Core version *DOES* have half a GUI (the command line is in a window and it uses notepad for text editing for instance). What it does lack is .NET Framework support - apparently that needs a full GUI to even install and therefore PowerShell is NOT currently available on Server Core!
Using TightVNC (high compression) and the DFMirage driver helps a lot, as does forcing your VNC viewer into 256-color mode (something I also do for RDP). I don't know about constrained network bandwidth, but on a LAN these things make VNC just as fast as RDP IMHO.
Tip for using 800x600 -- if you set the Taskbar to auto-hide, you will still have just enough room to click OK/Cancel on tall dialogs.
Back to the submitter -- seriously, Telnet/SSH command line is really going to be your main option. I really doubt you're going to be able to do anything useful over a 9.6Kbps GUI. You should grab a Linux box with two bridged NICs and set up NetEm to do some bench testing and see how slow you can go before you blow a blood vessel in your head.
DRM 'manages access' in the same way that a prison 'manages freedom'
Powershell can be installed on Core, it just isn't supported. Also 2008 R2 will have powershell.
That being said, installing in full mode, you can use powershell in a supported config and manage it using that. It is awesome.
Powershell (Which you can install on 2003 as well) + HP ILO and you should be set.
I would like to say stay away from Dell's DRAC if possible. I've worked with DRAC III, IV, and V and they all *suck* compared to ILO 1/RILOE/RILOE II/ILO2
...only I used BGAN instead of standard INMARSAT (which may actually be what you're referring to here). In instances where I needed GUI access on Windows boxes I found a very workable solution: Installed 1 Linux box with FreeNX server and put RDP client on that box. I'd NX into the Linux box and then RDP into each Windows box from there. Absolutely workable over even a crappy connection.