Linux Admininstration Resources?

shadfc asks: "I'm starting a new job as the system administrator for a small company in Tampa. They currently have 10 Red Hat servers (they are open to distribution change) that have not been actively maintained for a few months. I'm a Junior in College with a decent amount of Linux experience, but this will be my first job in this kind of position and responsibility. I'm asking for resources that can help fill in the holes in my knowledge and help make me a better administrator. Quality books on the subject would be preferred, but any advice is welcome. Thanks!"

Comments

All Linux commands

[prostoalex]

O'Reilly publishing has listing of all Linux commands, at least those that are expected to behave in a conformist way from distro to distro.

You better patch them first

[SpaFF]

They currently have 10 Red Hat servers (they are open to distribution change) that have not been actively maintained for a few months.

Can you give us the IP addresses of these machines?

Seriously though, make sure those babies are patched and secure before you worry about learning anything.

Re:You better patch them first

[j-turkey]

I'd recommend installing gentoo.

Without getting into a flame war (I won't reply to flames) -- I'd suggest thinking long and hard about installing Gentoo on 10 machines running an existing environment. By all means, install it on your home computer(s). It's quite flexible and is perhaps one of the coolest Linux distros that I've ever worked with.

Presumably, these Linux boxes are actually doing something useful, and they're important to the company that you're working for. I also presume that you'll be leaving the company in 18 months when you finish up your undergrad degree. Now, if you move to a semi-obscure distro like Gentoo, you will leave them with Gentoo. How many admins really know the ins and outs of Gentoo as well as they know some of the more popular distributions like RH, SUSE, Debian, etc? They may have a difficult time finding a competent SA to run those Gentoo boxes...they'll curse your name.

For many (and I'd say that these are the more forward-thinking competent SA's) a large part of systems administration is building an environment that's scalable, easily reproducable, and have everything documented so that he next guy can pick right up where you left off. Building very complex systems from the ground up is very cool...but IMO, is probably best for larger companies with money to toss into a proper staff. I always saw one of Gentoo's strongest points as being an easy way out for large companies who would want to roll their own. Unfortunately, you are one guy -- with ten systems (and I don't know how many users). Best bet for someone like you? Keep it simple.

I would suggest looking into a mainstream Linux distro that will be very easy for you to troubleshoot and maintain (I know that Gentoo is easy to maintain from an update perspective -- don't go there). There are a lot of good distributions out there, and I won't recommend any particular one.

Next step (and here's where I actually get into answering your question): Learn (bourne) shell scripting. Even if you already know shell scripting, your best bet is to learn how to do it in practical situations -- figure out which tedious tasks you tend to perform regularly and start there. The next step is to apply this knowledge to bigger jobs, that you might only perform once. While you're at shell scripting, regular expressions are of very high importance. Learn them. Finally, a higher-level language (like Perl) is very useful; especially when dealing with strings. In my earlier days of SA work, not knowing Perl was a big hinderance -- no sense writing 50 lines of shellcode when 15 lines of Perl could handle it (and faster taboot).

Anyway, that should get you started. Good luck at this gig -- and don't blow it. Good SA jobs can be tricky to come by.

Re:You better patch them first

[Zapman]

Gentoo is a wonderful distribution... It's the only thing recent that I could get to install on my sparc64 box.

That said, I would never run it in a production environment. It's tendancy to encourage bleeding edge packages WILL come back to bite you at some point.

RedHat is an excelent choice for production systems, if for no other reason than easily available and proven support contracts. I know that it's 'leet' to be able to look up things in google, but if you get hit by a bus, it will let the company survive while they find your replacement.

Having a support contract is also wonderful for getting to REAL support. If you're dealing with something really esoteric, you will often be much better off with a support contract. Let them fight to find the answer out of some kernel developer in New Zealand. You have the rest of your job to do too.

For your desktop box, I would urge you to do at least a gentoo stage 1 build, if not a Linux From Scratch install. These will take you forever to finish, but your knowledge of the linux as an OS will skyrocket. And while you're learning, you won't be affecting the company's bottom line, which ultimately provides you with the paycheck.

As for books, the armadillo book from ORA is wonderful, as is the 'purple book' (the successor to the highly acclaimed 'red book'. King of unix system admin books). The purple book will run you about $60-70, but reading through it will help you learn a lot.

Let's see: General notes:

1) Run postfix rather than sendmail. More secure, and easier to deal with. Less hair loss is to be encouraged.

2) Ban telnet, and use ssh.

3) Learn firewalling. Become hyper anal.

3a) Learn DMZ's. Limit exposure. There are some people who have 1 firewall interface per application (my company is moving that way). It's great for fine grained access control.

4) You don't and can't know everything. Admit this often. It's part of the key to learning.

Not a techincal reference

[deque_alpha]

but "The Practice of System and Network Administration" is very, very handy. Full of best practices and day-to-day scenarios and how best to handle them. See it here at Amazon. I have found the advice contained in there to be indispensible as I am maturing as a sysadmin.

UNIX System Administration Handbook

[Aniquel]

First link on Amazon. Indispensible.

Linux Administration Handbook

[linuxwrangler]

Check out the "Linux Administration Handbook" by Evi Nemith, Garth Snyder, Trent R. Hein et. al. It's published by Prentice Hall and is a pretty good overview of the tasks you'll be expected to do.

Also, check out the books in Sybex's Craig Hunt Linux Library series - he doesn't actually write all of them but most are pretty good. (Don't know how O'Reilly let him escape after writing the excellent "TCP/IP Network Administration".)

Nemeth

[the eric conspiracy]

Linux Administration Handbook by Nemeth et al. Her Unix System Administration Handbook is a classic. This one is targetted at Linux. Very nice. Great artwork too.

Skip the RPMs

[Graelin]

Before I get modded to oblivion, hear me out.

Whenever you install software, or perform an update, don't just jump into the RPMs. Build it from scratch on a dev box or something. Get really really familiar with the package. RPMs gloss over a lot of detail that a good sys admin should know or at least have written down somewhere. Aside from the minuta of the package you're bound to learn a thing or two about how to set up a system. Some packages require a lot of security prep-work before they will work. Others will not. After you've seen enough of both worlds you'll understand why they should and how to implement it. Last but not least, all the README files you'll go through will likely teach you some neat tricks that can be applied everywhere.

Second, embrace your distro. If you're going to stick with RedHat see if you can get up2date working properly. Or with debian, apt-get hourly from a local "approved" package mirror. These things make your life a lot easier if done right.

Books are fine and good but they're usually out of date. Understanding the system will enable you to handle the changes between the print date of the book and the release date of the software.

Try to get topic-specific books if you can. It's impossible to cram all aspects of the admin life into a great tomb - even a dozen of them. You'll certainly be lacking detail. Check out Safari (no link, sorry.) They have an enourmous library and their parent company makes some of the best techincal books ever.

Lastly, KISS. Use a real load balancer, get an SSL accelerator, get a hardware firewall. Yes yes, Linux can do all these things - but you'll spend much more time maintaining it than you would the Cisco box. (If that won't start a flamewar on here, nothing will.)

And, lest I forget, good luck!

Get a "playground" intranet box for experiments.

[DocSnyder]

Especially in enterprise environments, a wrong command or insufficient planning of some critical tasks can have severe side-effects. When I started administration, I installed GNU/Linux onto an old desktop PC which wasn't any longer good enough as a workstation but sufficient as a "playground" box. System upgrades, new kernel releases, complex shell scripts and even MTA or WWW server settings can be tested without disturbing other people's work. Internet access is only necessary as far as a HTTP proxy is concerned, to get updates.

BOFH

[arcanumas]

All you need is BOFH
Read up on the true professionals

Re:I'll just ask what everyone's thinking...

[linzeal]

Companies should be able to hire basic competancy and willing to learn enthusiasium over stodgy experience and self assuredness. I hate working with cocky sysadmins, and imho you run into more and more of them that are older nowadays. Young blood that feigns wisdom usually looks like a fool, and old folks that flaunt wisdom are no better. People that know who they are but do not need to keep reminding everyone else are the best people to do business or work with and I would rather deal with them over percieved security advantages for the ease of interaction when shit does go wrong.

Ideal solution!

[exp(pi*sqrt(163))]

I'm asking for resources that can help fill in the holes in my knowledge

Every time a hole comes up you can post your question to askslashdot. Hordes of geeks, eager to demonstrate their manhood, will answer your question for you. Best of all: it's free, yet you're the one who gets paid for your job.