Slashdot Mirror
Low-Bandwidth, Truly Remote Management?
kaiser423 writes "I'm looking to integrate some highly critical solutions into what would essentially be a remote, moving datacenter. No operators will be allowed at the site, and we may be able to have a high-speed INMARSAT data link. As a backup, we're planning to have multiple redundant low-speed Iridium data links. Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs. We're willing to write the terminal interfaces necessary for our programs, and possibly do the remote desktop thing with some of our 3rd-party programs. But what is out there that would give us this type of access, work robustly over a high-latency, low-bandwidth stream, and would be tolerant to intermittent network outages? Please hold the pick 2 of the 3 jokes, I know they're contradictory goals; I'm looking for a compromise here! These boxes would regrettably nearly all be running Windows (with some VxWorks). Does anyone out there remember those days, and have any solutions that they preferred?" Read on for a few more details of this reader's requirements.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
We've been looking at remote in-band and out-of-band management solutions, and really have found a ton of products. However, the "low-bandwidth" solutions still exceed our potential Iridium bandwidth (~10Kbps). Even if we have the INMARSAT link (192Kbps sustained, higher burst), a number of these solutions would hit that limit. We're starting to look at going old-school with some terminal-style applications, but haven't found much of a market for it; it seems to be a market that died with 56k modems. PC Weasel looks kind of like it might work, but the demo doesn't work for Windows.
you can do an aweful lot with IPMI, i.e. power cycle, and a remote access card that supports ssh can do the rest, alternatively a TTY terminal and pipe your consoles to serial ports
oh, windows? nevermind
There's a surprising amount you can do from the command line within Windows these days. For UI access, RDP beats the common alternatives hands down, even if you log in just to use a command prompt remotely and thus have console state stored between sessions if the connection goes down. Have you actually tried this?
I wonder if anybody can put some numbers on the latency and bandwidth? I spent four months in China maintaining Windows servers in California via RDP. With latency often around 600-750ms and packet loss, it was painful but still usable. I was even contending with nested RDP sessions (RDP over the VPN to a machine in an office in CA, and then RDPed from there to a colocation facility).
If you would be willing to reconsider your choice of OS, you might be able to get what you need very easily. Nearly everything on a Linux server can be managed via a SSH connection.
If Windows is a must, you might still be able to get the advantage of the unix terminal interface via CygWin, and manage the servers via SSH.
It's no joke. Embedded remote access cards like iLO from HP or DRAC from Dell are the only real solutions.
It would be painful to use their console viewing features over a low speed satellite link but, it would be no problem to power cycle, collect statistics and more. You can even forgo the web interfaces and use ssh on many of them.
"Essentially, we're looking to be able to power up/down and reboot some computers, and be able to start/stop some programs."
Dial in using the telephone system and use a sequence of DTMF tones on your telephone keypad to carry out a task. This will be low bandwidth (about 2,700 Hz) and low cost.
Ganty
The main name in terminal emulator programs if I understand what you're saying (only had time to skim, sorry) is TerraTerm or VersaTerm, one UNIX one Windows. I don't think they died with 56k modems. I would be really interested in any progress you make in this.
I do not have a sig. You are hallucinating.
Good ol' RS-232 let's you do a lot. Run one very low power board that can sit there listening to RS-232 input and act on commands. It can then toggle the power of other equipment plus route messages from them however you choose.
The world is made by those who show up for the job.
You will need to develop your own hardware to power on and off the devices. Just make a little embedded device who's only job is to stop/start the machines. Make all of the machines wol and use that to start them. Just cut the power when you want to shut things off in case of unresponsive systems. Everything else is straight forward.
how about some of the remote power management hardware that offers telnet/ssh admin access?
Tivoli Management Framework had configurations designed to work with satellite links as slow as 16k. That solution was for monitoring and configuration management though -- not what you want.
Your big problem here is your expectations. Remote Desktop over a slow-speed, high latency link just isn't viable. Anyone paying the megabucks required to support a field-deployed solution will not be happy with the crappy service you'll ultimately provide.
You need to extensively model how your application works and develop appropriate procedures, runbooks for your remote operators and a toolset of programs or script to provide support for this "critical" solution.
Conformity is the jailer of freedom and enemy of growth. -JFK
Two words will get you far in this situation, Command Line. Low bandwidth, latency tolerant, and generally asynchronous. If you can get any tools with a command line option, embrace them.
GUIs suck, and they suck more over the conditions you describe. Avoid them like the plague. Also, think about mirroring the files you need to manage and editing them locally, then uploading them when you are done. Not always possible, but if it is doable, it can make your life a lot easier.
Scripting is your friend here.
-Charlie
Perhaps I am not understanding what the issue is with using the standard console servers and PDUs out there? All serial access is pretty darn low bandwidth.
http://www.avocent.com/products/serial-consolemanagement.aspx
http://www.raritan.com/products/serial-console-switches/
Plug one of these in, then connect a serial cable to your servers. Many include a modem if you have a pair of copper wires for a phone line so you can keep it out of band.
a: Remote management cards often have command line interfaces for resetting, system health, etc, through SSH. True, SSH with 800ms RTT times is a pain-in-the-ass, but if scripted, should work fine.
b: Once you can power cycle/machine health remotely, now you use SSH to connect to a command line shell on the system itself (yes, even windows) and do all further tasks from the command line.
Test your net with Netalyzr
I think it is important that you have all the equipment you can on IP addressable Ethernet Power strips so you can physically cycle the power remotely independent of higher level computer control. Something like this: Power Strip
There is no substitute for the ability to toggle the most significant bit--for sure.
It sound like this is for Science in Antarctica.
http://www.bitvise.com/winsshd It does the job connecting all kinds of platforms/client implementations. It does PKI too.
HP's Compaq line of servers has **excellent** remote admin capabilities.
Push the whole thing over an stunnel and you are good to go.
Implementation is another issue. Publish an email if your budget supports consultants. Errmm. Well, it looks like slashdot is taking the place of a qualified expert, so good luck with that.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
You may be able to run RDP across even the low speed links - choose the bare minimum screen resolution and color depth possible. It will still be pretty frustrating and slow, but you could use it with enough patience. Or you can run VNC, though I believe its performance will be less than RDP.
For CLI access, install an SSH server on the Windows box. If your code runs as a service, you can interface to it through a CLI client. It's some development work, but possible.
For power, I like APC's smart power strips. They support HTTP and SSH access.
Whatever solution I used, it would have to be run over an encrypted satellite link.
We use VNC and NetOp with our satellite sites. It works decent. It is slow (maybe 2 minutes) to authenticate with AD when you initially log on (if you log in locally, it is faster), but once you are authenticated, things work pretty good. There is definate latency between when you click the mouse and action happens, but it is definately usable.
As for the iridum setup, you might want to check those speeds again. When we looked into it, they were only able to offer a 2400 baud serial connection over the Iridium system. That is REALLY slow, and with high latency. We decided against going with the backup.
I'm not sure where you are planning on deploying these setups, but maybe a cellular modem would fit your needs more. They are fast, low latency, and comparable in price. The only hitch with them is that there is no SLA; just best effort. If you are out of coverage area, try to hook up a high gain antenna to the cell modem and try again.
I would just use remote desktop tools to manage your servers. VNC especially works quite well with low bandwidth, high latency connections.
One thing to let you know though. Make sure that you have someone that can go to the site and has access to the network equipment. We have almost 20 sites like this, and about 2 go down a month. The sat-modems or sat-controllers sometimes need to be rebooted, and having someone near the site to do that can save you guys A LOT of money. When we have to send out a tech, it costs around $2,000.00 due to driving charges.
Copyright 2010. All rights reserved. This comment may not be copied in any way including, but not limited to caching.
It's kind of unclear whether you want to actually access these machines or run tasks on them (or both). However, Windows supports remote scripting and you can install an SSH server for interactive terminal access (or perhaps tunnel the script execution over the SSH connection if you're ultra paranoid).
Apart from whatever control systems you choose - or are forced into, you can improve the odds by having redundant servers.That way, if your data links are too slow, or suffer an outage your primary will fail-over to the redundant system. Of course, you will need to test the possible failure modes first and assure yourself that you've thought of everything, but if the systems really are highly critical you will be willing to spend the time and money to do this.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Get an IP aware KVM.
Using these you are separated from the operating system completely. You can see the machine boot, get into its BIOS, do a power reset (with a compatible power strip).
They have encryption and use a Web interface. Some have a fat client.
And try to run things from a command line as much as possible. Have the machine start a full screen command session upon boot, and hide the task bar. That should minimize the initial screen scrape.
Its the next best thing to being there...
- - - - - - - - - - -
I am a programmer. I am paid to produce syntax not grammar. Deal with it.
We've been extremely happy with the eRIC remote management board from Peppercon for all our lights-out remote-datacenter Windows machines. We use it over landline dialup modem (33.6) but it will work fine over any serial link you can throw at it, as long as you're patient. Full graphical remote console, remote IPMI, remote reboot, remote poweron, indepdent power supply (optional), it's great. They're a little hard to track down in the US, but I believe Raritan distributes them now. They're not cheap, but if you're paying for INMARSAT and Iridium, you don't care.
Right idea but if he's using sat links? Then that tells me that phone lines may not be an option. Now shortwave or some other form of wireless however...
Now for the OP I'd recommend asking himself what exactly needs to go over the link and what can stay local? Power up/down and reboot can stay local. Service start/stop? That depends.
Shai Schticks:"You don't make peace with friends, you make peace with enemies"
Without knowing what you're trying to accomplish, it's hard to give any real advice. Automated spy van? Unmanned laser plane? Knight Rider style service truck? Continually running train with nuke launcher? There are only so many things you can do with an intelligent roving unmanned platform.
www.baytech.net (".net" not ".com") has remote terminal servers, which include banks of rs232 ports and remotely controlled Power outlets. I think they are what you are looking for. Their terminal servers can either be dialed into, or network accessed, and have secure connections. You have telnet access and control of all terminal server functions, as well as the ability to control through SNMP. http://www.baytech.net/
The folks at RealWeasel have a cute little device. Plugs into a PCI slot and emulates a VGA card. It then outputs, over rs-232, a serial console approximation of whatever the system is displaying on the VGA device. Also has watchdog, manual reset, and keyboard functions. Those, plus a bog standard serial terminal server, and you are all set.
You may not need to purchase new software at all. I believe the Remote Desktop Protocol (RDP), works reasonably well over slow links, down to modem speeds (<30kbps). According to Wikipedia you would have many times that with INMARSAT and Iridium might at least be fast enough to be tolerable, especially so if you can use multiple links.
The advantages of using RDP are several: it's included with Windows so there are no additional costs (meaning no money is wasted on licenses if you decide it sucks and want to go with something else.) It's been around for a long time so it is reasonably stable and secure, and you get a full remote desktop where you can do most things you would be able to do at a "real" desktop. RDP is also cross-platform; there are clients for plenty of OS besides Windows.
As for a console-based remote management solution, it is apparently possible to use PowerShell over SSH using Cygwin. Microsoft also claims that the next PowerShell version will have remoting built-in. Of course, nobody knows when that'll be released or if it will work well for your needs.
As for power-cycling servers, there is no shortage of rack-mountable power strips that can be controlled via USB or Ethernet. Everything else you mentioned can be done via RDP and/or PowerShell.
Quality, performance, value; you get only two, and you don't always get to pick.
as long as the system BIOS is configured to restart when the power goes off, ethernet powerstrip is the way to go for power control.
i see no reason why you can't do everything else you need with Cygwin.
And for the zealots, telling someone who is required to use Windows that they should switch to Unix/Linux/etc is getting to be fucking old news.
you have obviously never worked in a corporate environment.
Windows? Really?
You don't need remote desktop to start/stop programs, reboot, poweer down in XP. Just have a folder you can dump files into. Webfolders will do this and just needs an Internet link. Then you set up a local script that looks in the folder every 5 mins and runs any .bat file placed in there.
You can start, stop, reboot programs in there.
Alternatively use psexec by sysinternals to run a remote command. Though this will need a vpn link.
I can stop/start USB devices, run/stop programs, reboot, find out what is running on it all 3000miles away.
Very easy. I do it over a 56K modem.
Stuart http://stuarthalliday.com/
My brothers-in-arms across the hall whipped up something which might get you started:
http://research.cens.ucla.edu/projects/2007/Systems/DTS/
They use it to manage 100 seismic sensors strung out in a 500km line across Mexico.
Better to light a candle than to curse the darkness.
Absolutely trivial. The "market" may have died with 56K modems, but the tools are still there, in every distribution. But then:
That is a hell of a punch line. It's one of those things that makes you ask, "if it's so important ('highly critical solutions'? c'mon!!), then how did Windows get snuck into this?"
You want gear with integrated lights-out-management. Any gear that supports SSH and SNMP should be perfectly usable over a shitty connection.
Most (all?) of the Sun servers come with an embedded ILOM that supports remote KVM through a web browser with Java as well as SSH. The SSH access gives you full out-of-band power control over the server, and can be used to look at system part numbers, power supply voltages, fan speeds, etc. Additionally you can configure SNMP monitoring/traps through the ILOM no matter what OS is running on the box.
We've used the x2200 M2, x4200 M2, and x4540 servers and the ILOM in each of them means I never have to go down to our data center to physically touch a box.
Ironically, some of the HP DL series have integrated out-of-band management called iLO, but they charge an additional few hundred dollars to gain features such as KVM or authentication. I don't like paying extra for features that should just be available out of the box.
The other thing you want is remotely managed online battery power. You want your power to be clean, going through a dual transformer conversion so no matter what kind of crappy power you have at the site, your gear is getting a nice clean voltage. Get something that has a good network-management interface on it. I've used MinuteMan Endeavor, Liebert GXT2, and one from APC that was online, but I can't seem to find it now. Each of these supports SNMP and web-based management.
LOAD "SIG",8,1
LOADING...
READY.
RUN
It's really difficult to get a feel for this question without knowing why anyone would want a 'remote moving datacentre' that isn't well connected to anything else. Why can't the computers be sited somewhere less troublesome? Would virtualising the machines to get round the booting/power on/off issues help, or is there some reason that this can;t be done?
A pizza of radius z and thickness a has a volume of pi z z a
Came here to post this.
The article has by now been tagged "ssh", which should be the obvious answer indeed. Even in Windows server editions this is a perfectly fine option these days - you should be able to do mostly anything that matters from the command line.
Every expression is true, for a given value of 'true'
If Windows is a must, you might still be able to get the advantage of the unix terminal interface via CygWin, and manage the servers via SSH.
No one ever got TERMINAL for buying Windows
*rimshot*
Cheers!
Atheist: Buddhist in a Prius
NetSNMP works even on windows. It is readily extended using shell scripts (to support your frequent reboots).
Of course, SNMP also provides remote monitoring as well.
I hope you have the ability remotely power cycle unresponsive boxes.
You could run Windows as a virtual machine on a Linux box and use the command line tools to manage it.
Reading code is like reading the dictionary - you have to read half of it before you can go back and understand it.
You need windows, ok, then go for windows, virtualized with xen or vmware over Linux.
Your proyect sounds a lot similar to one "not so long ago" proyect I had...
The solution was. Use Linux as virtualization plataform. So, when a reinstall is needed, for some strage reason, you can use a previously saved image. You will waste a little more space, but even that, is an issue. Our solutions also used a stripped down version of windows, so we can fit them in 8 Gb Compact Flash. The Linux image server, had our 8 different windows images via usb ports. Because even with linux and cf, you might get a flash card crash, which is more remotely than a hard disk crash. So everything was redundant, whit equipment and information sources...
So, when we need to reload a windows, we just copied the image to the right server or flash, and everthing was ok.
For administration of the windows machine, we used ssh to access and the console and since we were using xp, we used the pslist.exe and pskill.exe commands.
For hardware, we used via fanless hardware with cf2ide adapters.... Now we could use those new solid state disks...
Right now, I would go the same way.. and I will add it a vpn to a specific site using 3G, or something similar, and I will search for a dialin by celular option...
The solution wasn't even so expensive. Most of the cost is at the server hardware like blades and so on, we solved the lack of powercomputing of the via hardware with multiples hardware. Even there are automotive psu for the via hardware, and we mounted an array of 6 batteries and a little gasoline charger.
So, as an abstract:
linux+xen/vmware+windows xp with pskill/pslist + cywin for ssh + 3g or dialin modem
BTW, we used this setup at the jungle of Mexico, so we got high levels of humidity, lot of heat, lack of proper electric installations... it worked with out serious failure for over 1 year.
Mos failures where at the networking componentes, from cisco switches dying to tripped cables yanked...
At the end, we solved the networking issue using elcheapo 8 ports solid state, no fans networking equipments costing 1/10 of the original Cisco equipment...
Â_Â
If you are running truly remotely you'll need to be able to access the base system (power on, console etc) - which can be a real pain.
Make sure you look in to either RSA (IBM) or ILO (HP) cards as part of the server spec. Having a network connection even when the machine needs a hard reboot is worth it's weight in gold.
yah mod down probably the best suggestion so far...
Take a look at www.FreeMyIT, it has a lot of built in tolereance for the environment you are talking about. You can sing up free and try it. FreeMyIT is the practical solution for managing your or your customer's IT from a smartphone or PDA. With over 60 built-in interactive commands, FreeMyIT robots perform real-time, background and scheduled tasks at the push of a button. Restart services, reboot servers, run database commands, custom scripts and much more!
First of all, the majority of what you are looking for is already implemented in hardware if you have a look at commercial grade equipment. We use eLOM systems (SUN is surprisingly good at this) that lets us reboot servers, change settings and start up a simple remote console. HTTP based and very low bandwidth for normal management. SUN servers are not cheap, but they work.
Any sort of graphical interface will be terrible over satelite links, due to the combination of low bandwidth and high latency. As far as possible I would recommend going the CLI route. But I am pretty sure you knew that already.
By using correctly set watchdogs you can avoid most of the "I need to get there and reboot" issues. I have no idea how well watchdogs works under windows, but I assume it's not a major issue.
If you must live on a software only solution consider placing a virtualization layer such as VMware ESX(i) under the operating system to allow you some management once the OS goes to hell. That way you can reinstall the system remotely or store a fallback copy allowing you to boot up a clean OS.
Typically computers don't "run" VxWorks like they do Windows or UNIXes. Vxworks is an embedded operating system that typically runs on specialized hardware.
True, there is an i386 port of VxWorks but it is meant for embedded architectures built around an x86 architecture (not necessarily a PC-based x86 architecture).
It is very common for devices running VxWorks to have custom remote-management software written in them (i.e. a webserver like WindWeb, Seminole, or GoAhead) or to have a custom CLI type shell sitting on a TCP port somewhere (or the serial console).
If the system must be administrated by people who don't desire a command-line style interface you could always have a local server providing a web-based interface and have it proxy the necessary commands.
Start with your server hardware. Most Dell servers have a Dell Remote Access Card which allows you to get a full console (including BIOS and power on options) via web page. Performance of the full GUI over a slow link is marginal however. I'm sure other server vendors have similar options.
Also or alternatively, look for BMC controllers (http://en.wikipedia.org/wiki/Baseboard_management_controller) and IPMI interfaces (http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface).
From there address further needs with RS232, Telnet, SSH, etc. Step up to RDP and VNC for GUI needs.
I'm going to go back in my box and will think within the limits of my box: MS Sucks Linux Good I read too much Slashdot.
As already noted, forget about a GUI at those speeds. Use OpenSSH, like http://sshwindows.sourceforge.net/ or the built-in Windows telnet server .
Google around, there are a few books on managing Windows from the CLI, and Perl has a bunch of WMI modules. I swear there used to be a book on managing Windows via Perl/WMI but last time I looked I couldn't find it. I'd guess that Monad or Powershell or whatever they call it this week might be useful too.
Of course, it's utterly insane to use Windows under these circumstances, it was never designed to be a true server, or to run unattended or headless (consider who brought us the concept of a "preventative reboot!). But you probably already know that and are stuck. Sorry, and good luck--you'll need it...
Those of us using RDP (and ICA - but a Citrix installation just to get the ultra-low bandwidth for ICA might be overkill) back in 1998/1999 can tell you that it can work over a 9600 bps connection. Just set it for 16 color. You can also (as other posters have mentioned) do PowerShell for a lot of things. You can stop and start services and restart a Windows box from the command line (net stop [service]/net start [service] and shutdown /r respectively) and depending on the data may be able to transfer it across using ftp with auto-resume or (scary enough) set up a terminal program that supports IP and do a zmodem connection back to your main office to pass files.
I will probably get yelled at, but the simple answer to me seems to be writing a web app. It was stated that they are willing to write terminal applications, so there is obviously some programming ability present.
So why not write web apps. If you write your HTML well, then you can run it over pretty low bandwidth and all the work can be done by the server. There are still sites that work with 56k modems so it is possible to create low bandwidth sites (although most have stopped bothering).
There isn't a strong definition of what exactly the goal is and some projects don't lend themselves to being web based, but with a little work most things that can be done through the terminal can be done through the web server with far less bandwidth needed.
"Luke, I am your node.parent();"
While this is more for clients than anything else, we use Kaseya where I work. Wouldn't be my first choice based on the price tag but it IS pretty nice for Windows based setups.
Allows you to easily create vbscripts and push them. You can do quite a bit with it, including uninstalling and installing applications. Couple that with what you can do from command line windows and you'll almost never have to physically be at the machine (well, at least so long as it is able to check in).
Oh and if you want to see what is going on there there are several ways to remote in to the client PCs, including the capability for it to install VNC and auto connect you, even through firewalls and the like.
Pretty much its great so long as you are on Windows, have access to a web browser, and the machines are checking in. You can also change the port they use to whatever the hell you want and setup a secondary server.
Oh...and it lets you control patching of applications and Windows.
Like I said, expensive. But still pretty damn useful.
More at their site: http://www.kaseya.com/
While I would never suggest windows offers the same features and flexibility as a Unix/Linux/BSD command shell, with some work, there is no reason a Windows server cannot be administered via a command line SSH session.
Although many here are anit-Microsoft folks, I've got to admit Microsoft's Remote Desktop Protocol is *very* robust over high-latency, low-bandwidth networks. We've had many employees use it over satellite (DirecPC, usually) without any issue at all, when ssh or OpenVPN would fail.
As far as remote reboot, etc, IPMI is your friend. Most modern server hardware includes at least a rudimentary support for it on the baseboard, with more capable optional cards available. One example is that Dell servers have a IPMI-enabled baseboard management controller, and you can buy a Dell Remote Access Card (DRAC) as an add-on. Very powerful and intuitive.
I don't know about Windows machines, but I heard that certain robotic probes run VxWorks and are remotely controlled via a low-bandwidth, high-latency connection. Those devices have a lot of programmed autonomy and fail-safe built in. And they don't run Windows.
Sorry, but "highly critical solutions" and "Windows" does not really go together.
You don't seem to be heard of "Project Managers" in all their variety...
So say we all
Soap, perl, VNC, Remote Power Control units, and if necessary a TCP/IP controlled relay; the ADAM-6060 Data Acquisition Module. Look it up.
Hook your Iridium phone (and/or INMARSAT link if possible) up to a Linux box running Asterisk or TelAlert and use it as a login platform and/or use DTMF (or voice commands).
It must have been something you assimilated. . . .
Your use of windows is unfortunate. A company called NoMachine makes a fantastic product called NanoX, which is a caching, compressing proxy server. Your remote system has to be X windows based, but your local system be one of several supported platforms.
This is where X really shines. Where Windows has no alternative to ship you bitmaps over the wire, the X protocol is quite nice at making use of data updates that your client then renders. An old modem link (33kbps) was adequate for most software. As soon as you get past the 56k barrier, it gets very usable.
So if you have a choice, get those things running X windows!!!
If you're going to all this expense to
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
Why must everyone bash windows? There are good uses for it, and it's not going to go away. Linux is NOT the answer to everything. I'm not a huge fan, and chose not to use it where I don't have to, but come on. There are a lot of quality applications available for Windows, and like it or not, Microsoft has been getting their act together when it comes to Windows Server.
If only they had better command line support, on the level of *nix OSes.
HP servers come with ILO and ILO 2 solutions which scratch your itch. Power management is controlled through a web interface via a chipset that stays live so long as the server has power. It's got it's own IP, and it requires a separate network connection from the server. You can power down, power up, hit the power button, press and hold, etc all from a web interface and it has a "virtual status light" thing going on. They also have a remote console, and over my terrible VPN often routed over a cellular modem, it's passable. I wouldn't want to have to make serious reloads of the entire OS off of it, but it works. It also allows you to have "virtual floppies" and "virtual CDs". It all works well in both Windows and Linux. We're a linux shop, so primarily I use it to power cycle linux boxes remotely when our developers do something very very bad.
Get a very long network cable. Plug one end in at your principle location. Send the other end by mail to your secondary location. Wait for it to arrive
This may take a while as threading the mail system tends to cause kinks that have to be sorted autonomously by remote postman protocol.
When it arrives, plug in and use normal LAN remote control technology. Thanks to XKCD for the idea
I have been a user for about 10 years. This ends Feb 2014. The site's been ruined. I'm off. Dice, FU
Scalent, Cassat, Egenera all have software products in this space. How many servers, how sophisticated do you want to be? Roll your own or buy a product?
Well if turn-around time isn't a big issue SMTP (e-mail in instructions, e-mail out reports) can actually be used to manage something like this.
Plus lots of SMTP gateways exist for all sorts of mobile network interfaces.
Several others suggested VNC as an option.
To deal with your low bandwidth you could implement a simple VNC server that serves a proprietary interface instead of the conventional desktop. Based on interactions with that interface you can have that server initiate the desired actions.
Get a BGAN terminal from Inmarsat. I have Iridium, it sucks for data.
maybe Windows Remote Shell would help, not very familiar with it but sounds intriguing http://support.microsoft.com/kb/555966
Is SSH not your answer? There are commercial SSH servers for Windows, or there's the free Cygwin. Starting and stopping both Windows and individual programs is easy.
GUIs are a poor choice over a high latency network, protocol optimization (which help mostly the low-bandwidth aspect) or not. So you better give up on that. Command line is the only way to go.
Another poor choice is Windows, which depending on the particular version your using, has limited remote admin capabilities when there is an error during boot.
I'm not sure if I'm missing something, but the problem sounds truly trivial. People have been using dial up for remote administration for years. I routinely do remote admin over GPRS connections, which are just about dialup speeds but with higher latency. Have you actually looked hard enough?
What you need is the solution which was popular back in the old days (circa 1974-1980). The minicomputer (DEC PDP-11/45) at the undergraduate science center at Harvard used a DH-11 (16-RS232-lines) to allow a dozen or so terminals and even a few extremely low bandwidth teletypes to connect to the computer running Unix. A similar solution was adopted at Time Inc. in the early 1980's to allow a PDP-11/34 to communicate with all of the far-flung correspondents allowing them to submit stories every week (also running Unix). These types of solutions were quite popular at that time frame.
But PCs which commonly have only 2 serial ports (if that in the USB age), with Windows (which was essentially designed as a "personal" workstation and assumed high bandwidth connections) -- you have me ROTFL.
Better to run Linux, get the hardware which will support the low bandwidth connections, and then use Xen (or equivalents) to provide VM support for windows if that is absolutely necessary. Then, also do the world a favor, take the people providing the windows only software out in the backyard, beat them a little bit, then bury them alive.
With those 3 items, you should be able to do almost ANYTHING remotely.
Take a look at Citrix's products along with Windows Terminal Server. Their ICA protocol is better than RDP over low bandwidth connections and can handle running even over an old POTS line. Plus they have great management tools to let you do or configure anything about the Terminal Server. Citrix's stuff is designed for exactly what you want to do.
You lucky bastard, we work with 128Kbit/s links (C-band)((ok, most of them are 256Kbit/s now)). You can a lot with 128Kbit/s links and up, we run whole offices with 5-20 people on that bandwidth.
All management is done from HQ using telnet/ssh/rdp and server management cards so as long as we got ip connection we are good.
We also do weekly backups from offshore to onshore as a disaster recovery using vmware, wan optimizers and deduplication. Works quite ok, as long as there is not massive amounts of unique data created offshore and stored on the vm's. As this is manned offshore units we do not need to ip enable utillity power controls, but they are available in many forms and shapes.
750-850ms latency is no problem, you get used to it in a couple of years but it's a real death blow for applications with a lot of small requests and replies like SQL.
There's is one problem you will get and that is your precious innmarsat/iridum links. You will need service personel on your unit when that fails, not if, when. A solution used on our units is a innmarsat-b/fleet terminal connected with a serial port or ip to equipment onsite, so at least if you mess up a router config you can dial in the backdoor.
I would look to running all Windows systems inside a more stable platform like vmware. In general we found VNC to be much lighter than RDP, perhaps a model using it could be possible? Find a way to invert your management such that you can introduce jobs and / or reporting that can detect and execute maintenance tasks for you with the ability to inject a request for an action to be taken. There are so many better remote telemetry / command / control options than RDP.
It meets all your requirements. it was designed to work robustly over a high-latency, low-bandwidth streams, and would be tolerant to intermittent network outages. Back in the early 80s I maintained a 'network' of disconnected machines via 2400 & 9600 baud modems and the programs in the UUCP package - worked quite nicely.. If it still exists 'waffle' is a uucp implementation for DOS - I have no clue if it will run on current the hardware/OS
"Straddling the sword of technology..."
they do.
its called powershell.
Anyone who has taken even a passing glance at Iridium data knows it's 2400bps with stream compression -- it really bothers me that you have apparently looked into using it and don't know this basic spec.
But, on to the problem: You need to manage some gear remotely and the lowest common denominator is 2400 bps. I really don't understand why you can't use a serial device server and forward/reverse telnet to do anything you really need. Windows 2008 "foundation" or "core" or whatever they call it really has made it possible (though horribly un-fun) to manage windows from a command line.
If you absolutely require remote GUI, your best bet will be ICA as it supports direct serial and can run at a functional level at 9600 baud and probably pass in a pinch at 2400. I am not sure if Citrix's PortICA (The version that runs on XP etc.) can be made to support serial connections, but it's worth asking them about. RDP will be quite a bit thicker and requires IP connectivity to work.
Sure, it's old, but it was designed for remote administration back when dialup was popular. Nowadays it's open-source and still highly extensible:
http://bo2k.sourceforge.net/whatis.html
There are plenty of encryption and authentication plug-ins and it does some of what you ask.
I looked into this a while back and found one solution that you might find interesting. GD makes an Iridium reachback device that will combine four Iridium data channels into one larger virtual network pipe. If one connection goes down (which I understand happens often since the satellites are moving) the system will divert that traffic through the remaining three channels until the failed connection recovers. I was told that the unit can be booted using windows or Linux, which of course makes it more configurable. The unit can be used as a network router between disparate networks to give a low bandwidth dial on demand network connection which can then be used to tunnel your management IP traffic without any additional support on either end for what ever software you choose for your remote management. Alternately each of the four channels can be used separately to connect to other remote sites if desired. http://www.gdc4s.com/content/detail.cfm?item=7b0bceb6-36e0-49f3-b879-8df91d6a3409&page=5
I do remote management for a number of servers with wildly varying configurations at another site. Even though (perhaps because) I live in a college town in the US, I experience latency between 800-30000ms, along with ~30% packet loss and transfer speeds in the sub-56k range, even on an advertised 8mbit connection. I suck it up and drive for a few hours to work on the Windows servers once a week rather than trying to administer them via RDP or another remote desktop program. Even when working with programs specifically designed to deal with spotty networking conditions from remote locations(GIS programs), there is little that can be done. The windows CLI is absolutely atrocious, even with the steps that they have taken over the past few years.
Don't try and fit a square peg into a round hole; since you probably can't switch to *nix machines, you're going to have to interface with them through cygwin and use ssh. There's no reason to go looking for an exotic solution when the obvious one is looking you right in the face. If you're willing to spend some development time anyway, it shouldn't take a huge amount of effort. It would take some incredibly odd circumstances for any other option to make sense.
The Iridium links are 2400bps each. 4 of them will give you 9600bps. I'm going to assume you have a device that uses 4 Iridium LBT's bonded together. At that speed, remote desktop is really not possible. Focus on the command line, even if it means writing something in-house. For win32, use the RCMD service on 2003 as a way to do some management for your servers (located on the 2003 Resource Toolkit disc) across a slow link like this one.
Fifty watts per channel, baby cakes.
Windows isn't the problem, it's clueless folks who decide that all solutions must run on Windows regardless of the relative merit of that platform.
They're all tools, not religions. People should choose and use platforms as appropriate for the task at hand.
Mainframe/UNIX Bit Twiddler and long time Windows/Linux Hobbyist.
The Theorem Theorem: If If, Then Then.
Put in a managed PDU, ssh or telnet to it, switch the power on/off to your servers. If the server is up, ssh or telnet to it, start/stop the applications. What am I missing? It's like the poster has never heard of telnet.
http://www.synaccess-net.com/
Great OOB management appliance that can provide console access via nearly any type connection...they have the internal capability to provide 3G connections on an as needed basis, but no reason you wouldn't be able to use a satellite link if required.
www.uplogix.com
Sounds like patch management could be a problem, particularly if we're talking about Windows. Think about pushing service packs across those links. Consider anti-virus definitions, and any other 3rd party software updates, too. Depending on how many machines you are talking about, deploying a patch and antivirus update server might be wise, so they can all pull from the one local system.
As far as the remote management piece goes, resilience to latency is going to be just as important as handling low bandwidth. Make sure to test these things with latency, not just low bandwidth.
http://blog.lxpages.com/2007/03/13/remote-desktop-for-linux/
This will give you some ideas. Really seriously evaluate NoMachine's stuff, it is VERY efficient over a low bandwidth high latency connection. There are a choice of both free and commercial implementations too. (Note that it is not actually a Linux specific technology, works fine on windows).
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
Set up a reverse SSH tunnel. Write a script to maintain the reverse SSH tunnel. ssh -C -f -N -R 2233:localhost:22 remote@designated-machine.com In the example above, I've copied the destination machine's id_dsa.pub into designated-machine.com's authorized_hosts for user "remote". I run it, and assuming both my work PC and home PC are on, there will be a tunnel. I put the above snippet in an infinite while loop, so if the ssh connection breaks it'll retry it every minute. This way, you have a reverse tunnel going to a designated immovable machine. SSH to that machine, then through the tunnel. If you want a remote desktop, use FreeNX. That'll go through the tunnel as well.
:(){
Why not just use Window's built in management interface, WMI?
"That which does not kill us makes us stranger." -Trevor Goodchild
Well, I used to do sales/technical support for both Iridium and Inmarsat systems, so I guess I know a little about these. The BGAN/SwiftBroadband solution from Inmarsat can easily handle Remote Desktop sessions to these units. 256Kbps is the top sustained QoS you can get, but the units can get speeds as high as 492Kbps. The network is nowhere near saturation, so speeds of ~350Kbps are not uncommon. Latency is (and will always be) around 1.2s, which sucks for remote desktop, but is workable.
I've seen people watch a slingbox stream over these things with no skipping, in a dense area. In a sparse area like what you've described, no one else will be on your spot beam, so the entire bandwidth of the beam is basically yours to use. It's really not an issue.
Also, if this doesn't work for some reason, easy solution for the CLI that no one wants:
1. Write CLI
2. Write Client-side GUI for CLI so that customers think it's new technology
3. Profit!
I am more of a UNIX admin so I don't claim to be an expert on remote Windows management, but I believe the tool you're looking for is Microsoft PowerShell which allows remote WMI scripting. I've seen some pretty impressive demos of infrastructure automation using WMI and PowerShell.
For power cycling and the like, I'd go with the hardware remote management tools (iLo, DRAC, and the like).
A quick Google turned up a few Microsoft articles on PowerShell and WMI. (Article 1, Article 2)
http://www.rovemobile.com/mobileadmin
Aimed at doing remote admin with handhelds (think blackberry). So its an optimized server side interface for doing all sorts of things to remote servers via very low bandwidth (so to speak) where an RDP session isn't going to cut it.
Why not so safe? When you power cycle a server, if it does not come back the way you expected, you need some way (or some one) to look at the 'glass' to see why its not coming back up. ILO or DRAK cards give you this access remotely. A KVM with IP access will also give you remote access to the video out for most servers. ILO and DRAK can power cycle the box again for you - KVM itself can not.
If you have a body on call in true emergencies, then Mobile Admin might be more of what you need. Maybe.
I've had good experience with "Cyclades" products, including a unit that supports term server and kvm on the same unit. So one unit per rack with Ethernet and serial access so you can connect via modem in the event of a network outage. Combined with a good power strategy you could have access to your equipment up to an hour after a reported power outage, in and out of band.
-DML
Is to utilise APC power strips which can be used to powercycle frozen machines (remember to set the bios APM to poer on after ac loss)
SSH for windows servers to give you cli access and VNC for desktop.
Ensure you use srvany to ensure critical apps are run as services with the relevant credentials so that they can be easily reset and can have conditional rules set for failure and dependencies.
If you have money to spend I recommend Kaseya as a most impressive all in one low bandwidth solution that can accommodate flaky connections by queueing up commands ready for collection and execution by agents on the servers regardless of manufacturer.
Quidquid latine dictum sit, altum sonatur.
We use APC PDUs to remotely control the hardware power to our servers and network devices. They have a nice web frontend, but i believe they have a telnet and maybe ssh too. It doesn't solve the issue of restarting programs, but i like them because they're completely independent of the hardware you're trying to restart. log in, check off the outlets youwant to control, give it a command (off, reboot, on) and click apply. 1U rack space and provides 8 controlled outlets.
At present fisherman in the North Pacific Ocean are able to use the internet, email and remote applications using a customized version of the Citrix line of products over Ship to Shore radio.
Something to investigate.
If you are getting your Iridium service from the commercial gateway, you can get Direct Internet 2.0 which provides connections up to 9600 baud (depending on type of data and protocols used). Also, take a look at Iridium's OpenPort (http://www.iridiumopenport.com/). Another Iridium option is to get a MXU multiplexer (http://www.voicemall.com/iridium_mxu.htm) - not sure if this is still available though. If you're going through the DoD's gateway, soon you may be able to get multi-link PPP (MLPP) via RUDICS which will allow you to connect at up to 9600 baud. Standard RUDICS provides 2400 baud connections. The DoD's Direct NIPRNet service - comparable to Iridium's Direct Internet 2 - gives you up to 9600 baud connections. MXUs can also be used on the DoD gateway. Load OpenSSH for Windows (http://sshwindows.sourceforge.net/) on the boxes and you should be all set.
https://secure.logmein.com
I had RadMin Before, but LogMeIn beats everything.
Are this "moving" datacentres like the ocean located datacentres Google is thinking about? That whould be funny...
Does anyone have any experience with NX. From what I have looked into its basically X with a lot of the overhead taken out. Because if that would work , you might be able to have one Linux box whose only job is to run NX and rdesktop. You can then connect to that machine and use rdesktop to connect to the other machines graphically.
If you a looking for software to purchase check out www.matrix42.com they have low bandwidth Remote control as well as bandwidth aware agents for software push and patch management. If not stick to CLI and the free stuff like VNC.
Truly managing windows boxen over a low bandwith, high latency link (probably with a fair amount of packet loss thrown in for good measure)??? I can only think of four simple words to answer that one:
1)You
2)Are
3)So
4)Screwed
Seriously, I've had to manage Windows machines over poor, slow links and it's a crap shoot. HP's hardware iLO is a must-have to start with. Second, for remote console stuff, Windows' own RDP sucks to a level of practical unusability if your bandwidth drops below 100kbps. Citrix Presentation Server (XenApp or whatever the hell they decide to call it this week) makes much better use of thin, crappy bandwidth than raw RDP along. If you can use one of the mobile VPN's like NetMotion's Mobility XE, it does wonders for maintaining tcp session continuity when the network link's thruput and packet loss is going up and down like a yoyo. NetMotion was primarily designed to make seemless roaming and session persistence possible across various mobile wireless networks, but I've seen it employed to smooth out crappy landline-based and cellphone aircard-based network links too.
In the end, you're going to end up building a Rube Goldberg contraption where one piece-part of the conglomeration is eventually going to force you to visit the machines in person to fix it. The Windows O/S itself makes that fact inevitable.
So, I know that alot of Agented management solutions can do this. I know HP has a product called operations orchestration which has its main component in a central location, but can put a single sattelite in a datacenter (As opposed to an agent on every host). The sattelite can then do SSH and other operations (And does include support for remote windows reboot). So I'd say go look at some of HP's management software.
Windows isn't the problem, it's clueless folks.
Fixed that. I find that Windows isn't nearly as bad if you know what you're doing. Most of the people I hear complaining about Windows just don't know what they're talking about or are doing it wrong.
But asking Slashdot to design your systems can't be wise.
you don't live in mountains in a country that ends with -istan by any chance?
I supported offshore drilling rigs for 5 years. We did Vsat to our rigs with Inmarsat backukp. Vsat is pricey but it would be a lot faster than Inmarast.
The delay caused by the huge distances involved really messes with TCP windowing to the point that a 128k satellite link often has the throughput of 64k link.
We bought a device that simulated satellite links by allowing us to inject various amount delay and limit bandwidth. It was very useful for testing apps before we rolled them out.
Have to agree with everyone else - command line is your only friend with any of these links.
All of Suns x64 servers come with an embedded Service Processor that provides an RS-232 serial CLI, has a networkable SSH accessibel version of the same CLI and also sports a full Remote Keyboard/Video/Mouse/Storage capability via a remote web browser and Java. that goes from 9.6kbps for the RS-232 CLi to whatever bandwidth you use for teh network connections. not sure what windows would do with a serial line, but if you can make it provide you with a login prompt, the Service Processor can allow you to see either the Sp CLI on the serial line or switch to the host systems CLI... VxWorks might be a challenge... some of their SPARC telco boxes have been qualified for VxWorks I beleive and they all use RS-232 CLI's by default and all have a similar Service Processor and CLI.
Fourty or so years ago, people worked routinely over 300baud phone lines. Sorry, 110baud, 300 was advanced.
Nowadays most serious stuff still has serial ports that default to the magic 9600bps, 8n1. It really isn't too hard to hook up a modem to a cisco something stuck in a far away comms cabinet and have your own out-of-band link available through the POTS network.
Things like reverse terminal multiplexers have been around for a while too. Hook up one modem to such a thing, dial in, and you get an interface that lets you choose what other device to connect to. Such devices might include remote powerswitches in the power distribution system, gaining you reliable devince independent on/off functionality.
I hear that not too long ago micros~1 wanted to jump on the Command Line Interface bandwagon too, so maybe you can do something useful with that. Or you could use a Real OS, as that would make this so much more convenient. What you do with fancy client-side clickibunti stuff to ``innovate'' after that is up to you.
It really does amaze me that lots of choices regarding technology keep on being made on complete and wilful ignorance and pure eyecandy value alone, and then afterward turds are polished and the laws of physics are made to suffer to graft desired functionality on top. The salesmen call this ``progress''. I call it failure to engineer for function. Then again, we wouldn't want to drive thedailywtf out of business, would we?
The Avocent KVM over IP switches have a nice setting for their video called "grayscale". Instead of drawing the screen with colors it uses grayscale, very high compression. It is pretty fast even over a very slow connection. This is better then the DRAC and ILO cards that do not have any compression/grayscale settings. This would be an alternative if you really need BIOS level access.
The DRAC and ILO cards are good if you need true ability to power on the server remotely. No other card that I know of can do that easily.
As everybody else has said, use the CLI, even on windows you can do pretty much everything you mentioned.
There are good uses for cow poop as well, and it decomposes faster.
It'll be out before the New Year!
I've worked at a very large data/application hosting company for about a year now, and we run DRACs on almost everything. I'm not too knowledgeable about other solutions, but DRACs do fail. I've had quite a few servers that I have had to go to the device and manually shut it down. Sometimes the cards just don't work, and sometimes they just die. Just like anything else in the computer, a faulty component can take out a good component. I've seen external "remote" KVM switches before, and also remote power switches. You may have better luck if the device is external from the computer itself, but then you're adding potential failure between the two. The bottom line is, cover yourself and have someone nearby just in case, even if they live 15 minutes away. Also a couple other points to make. Old-school doesn't mean terminal-style applications. I'm assuming you're strictly a Windows user and have never had to use something like QAD, SAP, AS/400, linux, or anything else involving mission critical systems. Think outside the mouse. Terminal style applications are great for spitting output to parse and send off through a different interface. Think of all the devices that are text-based and used for communication of some sort. Barcode scanners, pagers, cell phones, other serial based devices. Don't dismiss the possibilities. Also, you're generally not going to find an all-around low bandwidth solution unless you 'are' going strictly terminal based. Most programs out there will allow you to dumb down the picture with compression of all sorts, especially if it is VNC based, and a 'lot' of them are. Turn off color, bump down the resolution, turn off the wallpaper, slow down mouse polling ... it just depends on how patient you are and how pretty you want stuff to look. It's not always going to be like you're sitting in front of the thing, but you can only expect so much when it comes to video compression.
LANDesk is the best tool on the market.
Pathetic. You either have a well-integrated CLI for such things (remote access) or you don't. There is no such thing for Windows (and don't blather about CMD.exe or "PowerShell"). Learn to live with that. Switch to Unix and use OpenSSH or please just STFU. Unix bigots don't whine & carry-on about not having PC-Anywhere or Carbon Copy.
check http://gomyplace.com/ it is not fully implemented, but if you are interested i can bring the product to the production phase fairly quick and make it embeddable. integration with vxWorks is not a problem. Or just consider the idea. hardware reboots - there are relays which can be controlled remotely, including HTTP interface. Again I could help you here - i can reverse engineer software for the existing hardware and put support you need, like SSL or SSH. Also you can consider developing of the hardware - this is not prohibitively expensive as many think. Such hardware can be sold as a separate and profitable product. The market of remotely controlled relays still is limited.
Put some relays on the serial port of some very stable piece of hardware to enable hard-boot the machines remotely. Some windows experience tells me you will need it...
Most out-of-band/backup management used by the likes of AT&T and Sprint use and serial multiplexer with a phone line. You would hook this to the console port of the computer and/or the 'lights out' card. There would also be a APC MasterPower PDU (power distribution unit) that allows turning outlets on or off including sequencing and timer events. These PDU have a serial port that would connect to the same serial multiplexer. This gets you a command line, BIOS level access and hard booting ability. Of course, you can script or command almost anything on windows using the WSH (Windows service host) or on *nix using your favorite shell. For those trickier 3rd party apps you can use software like AutoIt to command the GUI from the command line. For easy access to services, registry, tasks, etc you would use PSTools by SysInternals now maintained by Microsoft.
All of the above works very well over intermittent connections as it was designed for dial-on-demand connections. For complete control over the GUI (in windows) you can use the highspeed connection with the built in RDP or terminal services. RDP will give you the 'monitor' view like you were sitting there and TS will give you a separate login with your own session. I prefer RAdmin for an RDP replacement as it has better security (logins can be completly seperate from the OS users or you can use the same windows user accounts, but providing more fine grain control over rights), it can be configured for slow links (I have used it on 42k connections by setting screen updates to 5/sec all though this is somewhat painful), it is cheap (about $10 for each client/server pair), and it has very useful file management tools (bypassing the need for telnet/ftp and even the GUI) as well as chat tools for when multiple people are working on the computer at the same time.
i think that about covers it.
Rock solid and simple:
Use the DTR pin on RS-232 to power equipment:
Get an old Cisco 2509 async router. (This is the old routers for modem banks used by AOL and others in the old dial-up days. You can get them almost for free)
Connect the DTR pin to control a switch/relay. This switch powers some computer equipment.
It is simple to test your setup manually by logging onto the router and toggle DTR.
Now write a little script that powercycle the eqipment with a 5 second delay.
You can have a lot of these switches, and it is all very reliable and stable.
don't cut it off www.mgmbill.org
... ehm Avocent!
Maybe Computers will never be as intelligent as Humans.
For sure they won't ever become so stupid. [VR-1988]
See my LANDesk post below and then do the research. We spent thousands on research alone. LANDesk beats all other solutions.
You're running an human inaccessible data-centre full of Windows boxes, and you want us to hold the "pick 2 out of 3" jokes? :D
Sorry, but "highly critical solutions" and "Windows" does not really go together.
You don't seem to be heard of "Project Managers" in all their variety...
Actually, I am a project manager in one of its various forms, and I do development of safety-related systems. We currently use RHEL5 on the servers, and uLinux on embedded devices. No-one in our industry would use Windows except as a platform for writing documentation (and I hate it for that task ;-).
Stephan
It might be a good idea to separate hardware management from the software side by using virtualization. Get servers with a serial text console to control the boot process, etc. and have a slim system that can be easily used from the CLI (e.g. Linux, *BSD, ...) to run a hypervisor and deal with whatever happens to your actual hardware.
You can then have your Windows systems running on virtual machines (with no possible hardware failures). Using KVM you can get graphical access to the virtual machine through VNC when you really need it, and you can easily start/stop/power cycle/... your machines from the command line.
Additionally, when there are hardware problems you may still have a chance to migrate your VM to another physical server more easily.
Try using OptNet [http://www.opnet.com/university_program/teaching_with_opnet/] network modeler and see what issues you might have.
How about some form of email remote control to kick off scripts on the remote side and to receive status updates?
I saw the Iridium word in there, and since I work on Iridium products wondered if I could think of a solution for the low speed part of your problem. I'm not a salesman, I'm the firmware writer and this is not an ad !
One thought, which would be a change to your specifications, is to use Iridium Short Burst Data to send/receive simple commands (up to 350 Bytes or so) using Iridium 9601 modules. These are much cheaper (and smaller) than the full Iridium "LBT" (L band transceiver), and within a product such as our Beam LeoTrak (www.beamcommunications.com), could easily be used to reset your PCs, send/receive IO and output simple commands. Though it doesn't provide the interactivity of the low speed datalink, it may be a much simpler solution that gets you most of the way there for much less outlay.
If you do need the effective 3.3kbaud (10kbaud claimed with compression, but it all depends) data link, you need something with a full LBT in it which is probably what you have been looking at. Iridium will also shortly be launching their "high speed" option (Iridium OpenPort), but I'm not sure if this would be economic for a backup link.
Yes you can do a socket connection using PPP across this modem style link. It's ok for email but dreadfully slow interactively. Your concept of using some terminal style session is probably more useful. Make sure you set your dial timeout to more than 40s as it can take some time to establish the link initially. The data number is a different phone number from the voice number.
...only I used BGAN instead of standard INMARSAT (which may actually be what you're referring to here). In instances where I needed GUI access on Windows boxes I found a very workable solution: Installed 1 Linux box with FreeNX server and put RDP client on that box. I'd NX into the Linux box and then RDP into each Windows box from there. Absolutely workable over even a crappy connection.
When managing some SQL databases on ship we found that using the query tools directly was better than trying to use a remote desktop tool. This was because it allowed you to build the query locally and then run it on the ship, you did not really mind if the query took a long time. However when we had a query that took a long time to run we had to remote in and run it there because we might loose the link before the query finished causing it to rollback. About 15 years ago when I was managing stuff in DOS we had some remote links across the UK on modems/ISDN and so batch files were written and copied to the remote machine and executed by scheduling them to run a minute later using AT. Given you are Windows based, perhaps you might want to look into Powershell, the initial learning curve is a bit steep but you can do a lot with it once you've got your head around it.
install perl first
there are other similar web based control panel type solutions.
Next you're going to tell me the desktops come with workspaces, focus-under-mouse, alt-drag window movement and the ability to put any window On Top, right?
Geez, before we look around, windows might catch up with Linux as it was ten years ago! ;)
Okay, I'm overly demeaning towards windows here, but seriously: when I first tried Red Hat 6.2 [gnome 1, sawfish, those days], one of the first things that struck me as totally awesome was the GUI. In particular, the window management [although the crux theme is a close runner-up and lingers on my compizzed windows these days]. When I'm compelled to use windows, setting up a 'nixy gui is one of the first things I do.
It's amusing to see the modern Windows versions catch up and implement technical features that you've come to expect from any half-way decent operating system.
Note: I'm not saying that Linux is better than Windows in every way. The obvious aspect where windows wins by a huge margin is third-party cooperation.
But I don't think the old staples are true.
Windows more usable/user-friendly than Linux? When I have to help my mom download an 18 meg attachment and delete the email to free up her space via the phone, she calls it "remote healing" [admitted, it's a web interface and not a windows thing, but I don't think the difference matters].
Software installs easier? When someone packages up the software in a .deb or .installshield archive for you, it's trivially easy. Linux (apt-get on ubuntu and debian) asks you fewer questions, which is nice.
When the software isn't packaged, but you just have a load of .exes and .dlls sitting in ./build and the exes want some of the dlls in system32 but they're named the same as the dlls from the stable version which you use during your day-to-day work, and the exes want the rest of the .dlls in $dllopen_library_path, ... ugh. On linux, you svn co, ./autogen.sh, ./configure, make, run. Install some libraries if you don't have them all. It takes some learning, but so does windows. And if you say "but all windows software is packaged", then you are right, and my counterpoint is that it falls under third-party cooperation.
Sorry, I got a little ranty. Hope you liked it :)
I'm sad to disappoint you, but this SDL thing you hear about on linux (mostly) isn't the Secure DTMF Layer ;)
You'd probably want some kind of authenticity check. Probably an authenticated key agreement, and then using the agreed-upon key to generate some xor pad, except you may want to add modulo $NUM_BUTTONS rather than modulo two. I love counter mode [you can easily precompute the pad without the plaintext, and in parallel], but if you find a library to work for you, just use what it says unless it slows you down too much.
Unless you want to assume that only honest people have access to your telephone wire :p
Have a look at Versiera. It works in the opposite way that most of the other mentioned solutions using host agents to perform all work without users directly interacting with the hosts. A host agent, when running, checks into a management system. Such things as assign/scheduling of jobs to hosts - the agent when checking into the central management system inquires whether there any any work units for it to execute. There is asset management, monitoring and other capabilities. OS support covers most OS, including BSD. All communications are over SSL. There is a free Internet service and dedicated appliances are sold.
With all these suggestions, I can't believe that I haven't seen anyone suggest Server Monkey! It's just like Bathroom Monkey, but for server rooms. I don't know anything about monkeys, but I know he reboots my server on command!
Oh, the trials and tribulations of a network geek! Read about them at: http://www.ryumaou.com/hoffman/netgeek/
I'm using a small IPMI device with Supermicro servers (about $100). It has everything you need: KVM-over IP, remote reboot, sensors monitor. Bandwidth seems to be low - 1Kb for 4 FPS.
kaiser423 - a co-worker sent this link to me - we manage IT and science systems in Antarctica where there is often limited satellite connectivity - sorry for the 'anonymous coward' reply but the slashdot "create an account" is hosed for the moment.
First some questions: where in the world are you located and where are you trying to reach? If you are supporting science applications sponsored by the US gov, we may be able to help you directly.
Since you are talking critical applications on the move, INMARSAT, IRIDIUM 10Kbps service, will make the rash assumption you can fund a solution and the associated airtime/sat-time costs - these are not chump-change costs depending on all you want to accomplish.
We use a self-engineered Iridium Multi-Channel System (IMCS) to reboot and manage some IT systems at the South Pole Station in Antarctica from a US location. The small version of our IMCS inverse multiplexes 4ea 2.4Kbps Iridium dial-up channels into a single 9.6 Kbps ML-PPP bundle. The large version uses 12 ea dial-up channels for a 28.8 Kbps ML-PPP link. We us the 12 channel system to provide a very skinny WAN connection between the USA and Antarctica and are willing to accept the TCP/IP and ML-PPP overhead. This connection carries e-mail and other small but critical data traffic such as weather info, etc. The system supports command line/text based sys admin tasks as well. We use APC model number AP7900 remote power switches (TCP/IP capable) to manage remote system reboots for everything satellite gear to servers. These remote power switches support several text based communications options - SSHv2 and Telnet - we use SSHv2. The AP7900 supports HTTPS but the ML-PPP connection is too skinny to support that service.
You mentioned a moving location, we currently have a 4 channel IMCS onboard a crew module of the South Pole Traverse that allows a team of about 10 individuals to have e-mail and limited Internet connectivity as they drive tracked vehicles pulling sleds of science and operations gear from the edge of the Antarctic continent to the South Pole Station - a 7 week trip. They are under way at this time. Over the past three years, we have constructed an ice highway from the edge of Antarctica to the South Pole Station - wild stuff in support of science.
Advise if you want to pursue this technology further - - - Will try to create a slashdot account again tomorrow.
Nail59