Ask Slashdot: Tools For Managing Multiple Serial Console Servers?

An anonymous reader writes "I've recently been charged with updating our existing serial console access tools. We have 12 racks of servers each with a console server in it (OpenGear, ACS, and a few others). Several of these systems host virtual machines which are also configured to have 'serial' management (KVM, virt serial). In total it comes to about 600 'systems.' All the systems also have remote power management (various vendors). Right now our team has a set of home grown scripts and a cobbled together database for keeping this all together. Today any admin can simply ssh into the master, run 'manage hostname console' and automatically get a serial console or run 'manage hostname power off' to cut the power to a system. I'd rather use some tools with more of a community than just the 4 of us. What tool(s) should I move my group onto for remote serial/power management?"

Comments

How about SourceForge?

[Havokmon]

If you published your tools, you might find you're not the only ones who need the flexibility you've written into your toolset.

Why?

[Anonymous Coward]

You haven't identified any missing features or existing anti-features in your current toolset.
The only hint of anything wrong with your setup is "I'd rather use some tools with more of a community than just the 4 of us."

Q: What tool(s) should I move my group onto for remote serial/power management?
A: Yours. Clean the tools up, open source them, and market them. Your community will grow.

Conman and Powerman

[decepetion]

HPC scale: https://computing.llnl.gov/linux/downloads.html

Conserver is your answer

[Anonymous Coward]

I would recommend a look at http://conserver.com/

Re:Conserver is your answer

[TheCarp]

I actually was about to post that....with one caveat: Look carefully at versions

Admittedly I haven't worked with conserver since 2005, but it was solid then and I can't imagine it has changed much since. The last time I grabbed it, I found that the source had forked a couple of times and the name "conserver" actually refered to 2 or three different versions of the same program, each with slightly different feature sets.

Few things are more frustrating than trying to figure out why a program isn't working because you are reading the docs for the wrong one.

That said, the conserver we ended up using was simple, lightweight, and did exactly what we wanted.... provide named console access from a single place.

Conserver

[Anonymous Coward]

At a previous shop i worked we ran conserver against about 1k machines, some virtual but the big bulk of them was physical.

I also implemented a standardized way to control power on/off/reset on those machines as a patch to conserver. Those patches can be found at https://github.com/glance-/conserver/

if it ain't broke...

[tudorxpopescu]

So... let me get this straight, you have a system which is easy to use and works just fine, and is written in house. Obviously, you want to change it... because? Jeez.

Re:if it ain't broke...

[rnturn]

My thought exactly. Unless they expect their console access needs to explode soon and the current system cannot scale, I can't see the need to change. The existing crew knows how to use the current setup and surely no more than a couple of pages of documentation would be all that's needed for any newcomer to come up to speed. Switching to a 3rd-party console access tool will just be one more thing that'll wind up appearing on the job adverts for new administrators and one more thing that'll slow down the hiring process when the HR filtering software doesn't see `ConsolePro2013++ Gold' on a candidate's resume.

Or did Management issue an edict of "Buy Not Build"?

Re:if it ain't broke...

[TheCarp]

I completely agree except for one thing.... and I am looking at YOU redhat..... if the system default that they come to expect is that ls output is unreadable due to dircolors being the distribution default which assumes a light colored background.... that should be fixed with extreme prejudice.

Are you willing to monoculture your vendors?

[tlambert]

Are you willing to monoculture your vendors?

If the answer is "no", then you are stuck with your home-grown stuff. Vendors intentionally introduce incompatibilities to lock you into using only them, so you aren't going to find some project that provides a HAL, or at least not one that will live through the next software update from one of your vendors.

You should also be aware (I'm sure you are, if you understand the dynamics of your scripts, but some reading this probably aren't) that some systems won't negotiate a KVM style console unless they are selected active in the KVM prior to boot, so there's an interaction between your power management sequencing and the virtual serial and real serial, and that varies from vendor to vendor and software update to software update.

If you are also using Real KVM(tm) style virtual video consoles, you're probably already aware that Linux and most other Open Source OS's fail to negotiate EDID information correctly, unless you use the closed source video drivers, unless they are selected as the active input on the virtual/real video display device, since those implementations are usually not multithreaded, and so if you have 4 HDMI inputs, and #2 is selected, and #4 is where the device is that comes up and does it's one-time negotiation (this is what's broken about the OS drivers: they should retry periodically until they get a response, then echo up the response to the video driver, which if it's in X/Wayland in user space, it's not going to happen, since it only happens at startup) you are SOL. So your scripts probably have that knowledge, too. Not that it'll do you any good if you have a Linux box on the second input on a physical Samsung monitor, mind you, as they automatically switch away from inactive inputs, and default tho the first input if there are none active.

So good luck with your ask, unless you are willing to start your own project, and are willing to push to get UEFI, u-boot, Linux/BSD video drivers, and other things fixed as part of the project overhead.

rtty, old school technology

[Above]

There's a little known, but very useful program called rtty. You can find it at ftp://ftp.isc.org/isc/rtty/rtty-4.0.shar.gz. Yes, it was last updated in 2003. Yes, there are package for major open source distributions.

Here's serial consoles on the cheap:

Buy multiport USB to Serial devices. They are a USB hub with a bunch of USB to Serial adapters hung off of them. Here's a 16-porter for an example: http://www.startech.com/Cards-Adapters/Serial-Cards-Adapters/~ICUSB23216F

Hang them off a low end box, I like half-depth Intel Atom servers with lots of USB ports.

Run rtty. It records each console to a log file 24x7, and allows multiple people to connect at the same time (including typing).

Re:Uplogix

[tthomas48]

The advanced CLI is for the sysadmin. The WebUI allows you to lock down users to say 1 port on a machine and give them a nice shiny button to click on.

Re:Please help me understand

[Bill_the_Engineer]

Serial consoles allows you to fix boot issues remotely.