Slashdot Mirror
Low-Bandwidth, Truly Remote Management?
kaiser423 writes "I'm looking to integrate some highly critical solutions into what would essentially be a remote, moving datacenter. No operators will be allowed at the site, and we may be able to have a high-speed INMARSAT data link. As a backup, we're planning to have multiple redundant low-speed Iridium data links. Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs. We're willing to write the terminal interfaces necessary for our programs, and possibly do the remote desktop thing with some of our 3rd-party programs. But what is out there that would give us this type of access, work robustly over a high-latency, low-bandwidth stream, and would be tolerant to intermittent network outages? Please hold the pick 2 of the 3 jokes, I know they're contradictory goals; I'm looking for a compromise here! These boxes would regrettably nearly all be running Windows (with some VxWorks). Does anyone out there remember those days, and have any solutions that they preferred?" Read on for a few more details of this reader's requirements.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
you can do an aweful lot with IPMI, i.e. power cycle, and a remote access card that supports ssh can do the rest, alternatively a TTY terminal and pipe your consoles to serial ports
oh, windows? nevermind
There's a surprising amount you can do from the command line within Windows these days. For UI access, RDP beats the common alternatives hands down, even if you log in just to use a command prompt remotely and thus have console state stored between sessions if the connection goes down. Have you actually tried this?
I wonder if anybody can put some numbers on the latency and bandwidth? I spent four months in China maintaining Windows servers in California via RDP. With latency often around 600-750ms and packet loss, it was painful but still usable. I was even contending with nested RDP sessions (RDP over the VPN to a machine in an office in CA, and then RDPed from there to a colocation facility).
It's no joke. Embedded remote access cards like iLO from HP or DRAC from Dell are the only real solutions.
It would be painful to use their console viewing features over a low speed satellite link but, it would be no problem to power cycle, collect statistics and more. You can even forgo the web interfaces and use ssh on many of them.
"Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs."
Dial in using the telephone system and use a sequence of DTMF tones on your telephone keypad to carry out a task. This will be low bandwidth (about 2,700 Hz) and low cost.
Ganty
Good ol' RS-232 let's you do a lot. Run one very low power board that can sit there listening to RS-232 input and act on commands. It can then toggle the power of other equipment plus route messages from them however you choose.
The world is made by those who show up for the job.
Tivoli Management Framework had configurations designed to work with satellite links as slow as 16k. That solution was for monitoring and configuration management though -- not what you want.
Your big problem here is your expectations. Remote Desktop over a slow-speed, high latency link just isn't viable. Anyone paying the megabucks required to support a field-deployed solution will not be happy with the crappy service you'll ultimately provide.
You need to extensively model how your application works and develop appropriate procedures, runbooks for your remote operators and a toolset of programs or script to provide support for this "critical" solution.
Conformity is the jailer of freedom and enemy of growth. -JFK
Two words will get you far in this situation, Command Line. Low bandwidth, latency tolerant, and generally asynchronous. If you can get any tools with a command line option, embrace them.
GUIs suck, and they suck more over the conditions you describe. Avoid them like the plague. Also, think about mirroring the files you need to manage and editing them locally, then uploading them when you are done. Not always possible, but if it is doable, it can make your life a lot easier.
Scripting is your friend here.
-Charlie
http://www.bitvise.com/winsshd It does the job connecting all kinds of platforms/client implementations. It does PKI too.
HP's Compaq line of servers has **excellent** remote admin capabilities.
Push the whole thing over an stunnel and you are good to go.
Implementation is another issue. Publish an email if your budget supports consultants. Errmm. Well, it looks like slashdot is taking the place of a qualified expert, so good luck with that.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
You may be able to run RDP across even the low speed links - choose the bare minimum screen resolution and color depth possible. It will still be pretty frustrating and slow, but you could use it with enough patience. Or you can run VNC, though I believe its performance will be less than RDP.
For CLI access, install an SSH server on the Windows box. If your code runs as a service, you can interface to it through a CLI client. It's some development work, but possible.
For power, I like APC's smart power strips. They support HTTP and SSH access.
Whatever solution I used, it would have to be run over an encrypted satellite link.
We use VNC and NetOp with our satellite sites. It works decent. It is slow (maybe 2 minutes) to authenticate with AD when you initially log on (if you log in locally, it is faster), but once you are authenticated, things work pretty good. There is definate latency between when you click the mouse and action happens, but it is definately usable.
As for the iridum setup, you might want to check those speeds again. When we looked into it, they were only able to offer a 2400 baud serial connection over the Iridium system. That is REALLY slow, and with high latency. We decided against going with the backup.
I'm not sure where you are planning on deploying these setups, but maybe a cellular modem would fit your needs more. They are fast, low latency, and comparable in price. The only hitch with them is that there is no SLA; just best effort. If you are out of coverage area, try to hook up a high gain antenna to the cell modem and try again.
I would just use remote desktop tools to manage your servers. VNC especially works quite well with low bandwidth, high latency connections.
One thing to let you know though. Make sure that you have someone that can go to the site and has access to the network equipment. We have almost 20 sites like this, and about 2 go down a month. The sat-modems or sat-controllers sometimes need to be rebooted, and having someone near the site to do that can save you guys A LOT of money. When we have to send out a tech, it costs around $2,000.00 due to driving charges.
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
Without knowing what you're trying to accomplish, it's hard to give any real advice. Automated spy van? Unmanned laser plane? Knight Rider style service truck? Continually running train with nuke launcher? There are only so many things you can do with an intelligent roving unmanned platform.
The folks at RealWeasel have a cute little device. Plugs into a PCI slot and emulates a VGA card. It then outputs, over rs-232, a serial console approximation of whatever the system is displaying on the VGA device. Also has watchdog, manual reset, and keyboard functions. Those, plus a bog standard serial terminal server, and you are all set.