Learning More About Linux?

teh moges asks: "From an administrator point of view, I know a lot about Microsoft Windows: where files are stored, where settings are, which registry keys to edit, how to change drivers, and so on. I made the initial switch to Linux a year ago. I now feel capable enough with using Linux, from an end user's point of view, so that when things go wrong, I can fix them. I now want to become even more familiar with Linux. Are there any great resources, such as websites, wikis or books for someone that wants to find out exactly how Linux works and how to fix and modify it?"

Comments

Only thing to understand...

[fimbulvetr]

Linux is files. The entire OS is based on files. Things to run on startup? Files. Opening hard drives? Files. Drivers? Files. (kernel mods)

No magical black box registry, windows drivers, etc. Once you understand this, other things will come easier.

Re:Only thing to understand...

[Timesprout]

Double cut flat files? Nail files? Diamond files? Needle files? Machine files?

Can you be more specific about the files please.

Re:Only thing to understand...

[twenex27]

Linux is files. The entire OS is based on files. Things to run on startup? Files. Opening hard drives? Files. Drivers? Files. (kernel mods) Not quite true, of course. Networking breaks the "everything is a file model". Arguably, device drivers do too (witness the hundred and one interfaces to control different devices).

Re:Only thing to understand...

[schon]

I would say that you made a nice example of selective quoting, but you couldn't even get that part right. You probably should have said "no [...] registry".

No magical black box registry

the /proc filesystem? That's a "registry" if I ever saw one. Could you explain how it's a "magical black box"?

And you know what? Everything in it shows up as (wait for it..) files - which was kind of the point of the message you replied to.

Re:Only thing to understand...

[Anonymous Coward]

Not exactly, but it's a close enough first approximation.

For the Ask Slashdotter: /bin stores essential programs. These tend to be very low level programs. /dev stores file-like interfaces to device drivers. You can read and write to them, just as you would a file. (Using C. You probably won't need to touch anything in there unless you're programming drivers. /etc stores configuration files. They're supposed to be text files. The syntax varies across them, so Google is your friend here. /home stores user directories. Each user gets a /home/ directory. /lib stores essential, low level libraries.

There are a few others, but these guys are the important ones for what follows. There is also a /usr hierarchy, which has it's own bin, lib, (possibly etc). It's meant for "shareable, read only data". This is where globally installed programs usually go.

GCC is your compiler. You probably won't have to play around with it much, but I'll talk about some tricks to give you more insight into Linux. Suppose there's a program you want to try. You create a dummy account called "dummy" for it (so your data won't be in harm's way if it's buggy). Now, when you compile the program, you'll probably have to write ./configure. But that sets up the compile so that it gets installed in the main hierarchy. That's not what you want. So instead, you use ./configure --prefix=/home/dummy/program. Now, when the program is done compiling, you issue the install command (make install) and make automatically generates /home/dummy/program/bin, /home/dummy/program/lib, and so on. You've basically just made a private /usr hierarchy. This can be very handy.

Now, suppose you end up really liking the program, and you want to be able to use it from your account, but want to deny everyone else access. You can just mv /home/dummy/program to /home//program. Everything will (should, really) continue to work. Now you'll want to change the hierarchy's permissions so that only can run the program. You can do that with chmod -r 541 /home//program.

Of course, you don't want to have to keep typing /home//program/bin/ to run it, so you have to add /home//program/bin/ to your PATH environment variable. You do this by issuing an export command of the form export PATH="/home//program/bin:/home//program/lib:$PATH"

Now you can type the program's name from any location, and it will run. But only until you restart your shell. To make the change permanent, you'll have to edit your .bashrc or .bash_profile file. These are a lot like DOS batch files, but are written in "Bash", which is more flexible. Just stick the export command above in either of them, and the command will be run when you log in.

There are a lot of magic numbers and constructs in my post. Think of it as an invitation to learn about what you're interested in.

Re:Only thing to understand...

[fimbulvetr]

I can treat everything in proc like a file. Like cpuinfo, or net/arp or THM/temperature. I can echo things into them to change processor speeds, networking options, etc.

Slightly more managable? You show me one time when proc corrupted simply by "being there" in a stable kernel and I'll give that assertion to you.

Re:What distro?

[corvair2k1]

Sure thing.

Get yourself a Haynes manual for your model vehicle.

Roll your own distro?

[8-bitDesigner]

I was in a similar situation about a year ago, and I found that the best way to learn about the guts of the system, so to speak, was to pick up Gentoo, which is something of a "Roll your own" kind of Linux distribution.

Now, by no means do I recommend this for day to day use. I love Gentoo, but it breaks. Frequently. And unless you know a fair bit about how the system works, you'll end up breaking it quite often yourself. This is a good thing, and introduces you to the various configuration files, locations of critical items, how everything slots together, and how to compile your own kernel. The Gentoo documentation is excellent, and if you go about it with a certain goal in mind (web server, router, media center) you'll end up learning a fair bit about Linux in the process.

Re:Roll your own distro?

[nschubach]

LFS (Linux From Scratch) is another way.

Honestly though, like stated above, once you understand that Linux basically treats everything like a file... you can fix pretty much anything. As far as a good reference or tips site. Google. 99% of the time, a quick cut and paste of an error will direct you to the right place. (That is if you don't understand it right away.)

Re:Roll your own distro?

[bersl2]

While I agree that the Gentoo documentation is excellent, I can't say I liked the distribution. Maybe if I were to have installed it myself from scratch, instead of having inherited an installation, I might have understood why stuff broke as often as it does. But my one experience with Gentoo was on a production machine, and the first time I tried to update the system, the mailing system broke.

My own advice to the submitter is to go between Gentoo and LFS with respect to automation and below both with respect to complexity and use Slackware. I've used it exclusively for my own machines for the several years I've used Linux, and in the process of taking a stable core system and trying to add onto it, compiling from source as necessary, I've broken and fixed things enough. The build scripts are available too, so you can customize as in LFS, but with the advantage of simple packaging.

Re:Roll your own distro?

[mollymoo]

Gentoo breaks as often as it does because portage's handling of dependancies is a bad joke. Actually, I don't know if it's portage itself or poor quality ebuilds, but "emerge --update world" breaks for me about 40% of the time (no, I'm not running the unstable ~x86). I'm not shitting you when I say it took me three days to bring a box which hadn't been updated for four months up to date. That wasn't all compile time, the damn thing just kept stopping. Portage apparently didn't know it would break before it started, despite the problems all being related to dependancies.

When it breaks, portage doesn't tell you why, you basically have to search the Gentoo forums for an answer. In the inevitable thread(s) related to the problem I often find a response from a dev which says that you need to update X and Y together, so both X and Y block each other. Having the package management system manage packages for you is evidently too much to ask.

I do like Gentoo's arrangement of config files etc. though. It's nice to work with, just shitty to update.

Re:Roll your own distro?

[harryman100]

I'm not shitting you when I say it took me three days to bring a box which hadn't been updated for four months up to date. In my experience, updating a gentoo box infrequently is a bad idea. You should either update every week/month or not at all. The way I look at it, you either want a system with the latest everything, or you want a system which works. If you want the first, you should expect to spend a bit of time working on it. If you want the second - why on earth are you updating every four months.

If I wanted a system to be stable, but not that far behind current stuff. I'd probably be updating once a year - you have to do a bit of work to fix some of the updates, but at least you only have to do it once a year.

On the other hand, if I wanted a system that's always up to date, I'd be updating every other week. This is what I currently do on my two gentoo boxes, and I've very rarely had problems. (The last problem I had was when it upgraded mysql from 4 to 5, some of the defaults changed - I spent ages trying to work out why I couldn't connect from other machines.)

One thing to note, if you're having problems when updateing world add the --deep flag to emerge, it will update all libraries that need it as well. Then follow the emerge with a revdep-rebuild, to check for things that have been broken by updated libraries (and fix them)

Re:Roll your own distro?

[baldass_newbie]

Running Linux is one of the best books for learning the overall structure of systems. Matt even notes the basic exceptions between Slackware and most other core distros in terms of structure and processing.
Indispensible.

Three letters

[i_should_be_working]

LFS

Re:Three letters

[kestasjk]

Why not just buy a good book?

Linux from Scratch

[jomas1]

Check out http://www.linuxfromscratch.org/livecd/ and http://www.linuxfromscratch.org/lfs/

Try creating a distro of your own and you should get a handle on the mysteries of the OS. If you don't have spare hardware check out virtualbox.org and try creating your lfs distro on a virtual x86 computer

Re:Linux from Scratch

[cyphercell]

The first time I went through LFS I purchased the book. I think it probably saved me a week or two. Also, I use LFS primarily as a sandbox, if I want to test something I use LFS, when I need to get some work done I currently use Debian. I feel it is absolutely imperative for a desktop system to be somewhat dependable. Go ahead and break your LFS system, reiterate over your LFS system several times experiment with package management, configurations, bootscripts, break it, fix it, and customize it. Currently I'm on my fifth iteration of LFS, it's really been the best thing I've found.

Break It.

[Tragek]

I've learnt oodles about all OSs by breaking them. Delete something. see how it changes the behaviour of the system. If it was somethign really important, you'll learn about system recovery, otherwise, trouble shooting.

I'm serious.

Use it.

[PAPPP]

Really. Just use it. Like many other things, the best way to become familiar with Linux is to use it for your daily tasks for a while, and find out how to fix any problems you run into. I've used a dual-booted machine for some time, and about 6 months ago I made the decision to switch my main OS over from WinXP to Linux, and relitively painlessly went from dabbling to a being well versed pro user in a few months.

Re:Use it.

[DaveCar]

Err, he stated:

I made the initial switch to Linux a year ago. I now feel capable enough with using Linux, from an end user's point of view, so that when things go wrong, I can fix them.

Now I know it is de rigeur not to read the article, but there wasn't even an article to read here - did you even read the question?

No one answer.

[normuser]

I dont have one magic answer.
Forums are good for getting and sharing information on specific problems as long as the "google it yourself" crowd havent invaded. (how else would you have found the forum?). http://www.linuxquestions.org/ is a good start.
There are varias wiki's specific to certian subjects.
And I dont mean to sound rude with this, but please read the man page first. Weather or not you understand it at the time.

On a side note the best way I have found to learn about something is to break it first. but maybe thats just me.

Wiki

[Shinsei]

I have found that http://gentoo-wiki.com/ is a great resource for knowledge about the basesystem itself of Linux. I do love my Gentoo of course, but you'll find resources on said wiki for Linux in general - and quite a few of them - and I've seen people from both the Debian and Ubuntu community also look to this Wiki from time to time.

SOme input

[TheRealMindChild]

Here is what I got, take it for what it is worth.

See 'man Linux Filesystem Hierarchy'. In case for some reason that doesn't work on your system, here is a link -> http://tldp.org/LDP/Linux-Filesystem-Hierarchy/htm l/

heres a few

[drfrog]

http://tldp.org/

this is the linux doc proj the one place i found indispensable while learning slackware back in the day

lately http://www.debuntu.com/ is a god send as well

and of course , unlike windows software, most linux software readme files are actually filled with useful information

othjer than that either a quick google on a specific question or jump onto irc will usually get you some help

Re:heres a few

[drfrog]

oops sorry

it should be debuntu.org

Do what I did

[stinerman]

I fancied myself a Windows "power user" and then GNU/Linux was foisted upon me because of a hardware crash. My backup system wasn't good enough to run XP, so I tried Red Hat 9. It certainly was a bitch, but I leaned GNU/Linux by doing the same thing I did when I was learning Windows: poke and prod at the system and see what changed. Doing is the best way of learning.

Don't really worry too much about a specific distro. I went from RH9 to Fedora Core to Ubuntu to Debian over the span of about 4 years. Once you learn enough, you can pick up any distro without too much hassle.

My best advice is to pick a distro and dual boot with XP/Vista. Every day try to use your GNU/Linux distro a bit more each day. Be sure to try to fiddle with settings. Just make sure you have a backup ready for any important data. For awhile there, I was reinstalling the OS weekly. Don't be afraid to experiment.

Read a general introduction to Unix

[Ivan Raikov]

First of all, you should have in mind that Linux is just a kernel, and what you are probably more interested in are all the userland programs that comprise your typical Linux distribution. I think it is best to start with a general Unix introductory text, because the fundamental principles have not changed in 25 years, and it is much better to understand the core Unix system utilities and how they plug together to accomplish complex tasks, rather than waste time with all the modern Windows-like interfaces that are fashionable in Linux distributions today.

There is one "classic" Unix introduction book that I can strongly recommend, and that you can probably buy used for a dollar: Exploring the Unix System by Stephen Kochan and Patrick Wood. Make sure to get the paperback edition that is about 400 pages. Also, apparently the authors are going to release an updated version of that book -- check http://www.kochan-wood.com for updates.

Once you learn the fundamentals of Unix systems, then you would be ready to learn the modern tools available in Linux distributions. Remember that is much more important to learn the principles and philosophy that Unix was built upon, rather than attempting to memorize arcane details.

Re:Real men

[twistedcubic]

Bzzt! Real men use both. Sometimes pictures/illustrations/explanations are better in one than the other.

Start with Slackware. Seriously.

[SadGeekHermit]

Your best bet is to start with Slackware 11, it's a manual distribution which will force you to actually get involved with it and learn how things work under the hood.

For example, you have to write your own iptables firewall script. But by doing this, you'll be able to understand the output of "iptables --list" on any distro out there and see what it's doing behind the scenes (for instance, amusingly, what holes does it leave open if any?).

You can download the distro here, for free:
http://www.slackware.com/
(my favorite mirror is: http://slackware.cs.utah.edu/pub/slackware/slackwa re-11.0-iso/)

There's a good book on it available here: http://www.slackbook.org/

Think of it this way (using a car analogy like the other guy, but more seriously):

If you learn to drive in a car with a five speed stick and a clutch, you'll be able to drive almost any wheeled vehicle on Earth with very little futzing around. It's almost like having a superpower.

But if you start out driving an automatic, you'll ONLY be able to drive automatic until somebody teaches you manual. And you won't have any reason to learn it, so you'll miss out on a potentially important skill.

It's better to start out with something challenging and switch to the easy stuff later.

Go Slackware, be a nerd like us! You'll thank me later.

Old school, learn Unix

[dru]

When I was learning about Linux, back in the mid-90s, the most valuable resource I found was The Internals of the 4.3BSD Operating System by McKusick, Quarterman, Leffler and Karels. This book acquainted me with the design goals of unix-like operating systems, and the issues of implementing these patterns.

Also, I'd pick either Aileen Frisch's Essential System Administration or UNIX System Administration Handbook by Evi Nemeth.

Fast forward to the 21st century, I now spend the bulk of my time using FreeBSD.

Linux is great, but remember that the thing that makes it great is that it's a unix-like OS. Learning the skills to be comfortable on Linux, Solaris, *BSD, HP-UX, AIX, or whatever the flavor of the day, will take you further than limiting yourself to just one.

Good luck!

Just read this

[swillden]

There are probably some good books out there, and I hope you get some recommendations for them, but there is one key thing that you should learn about Linux that will get you 99% of everything you'll ever need to know about how your system works: How it boots.

If you understand how everything gets started you'll understand how it all fits together, and, even better, you'll have the starting point you need for tracking down anything else you need to figure out. And the great thing is that it's simple enough to described reasonably completely in one brief slashdot post.

The boot process consists of the following steps:

1. The boot loader loads the kernel
2. The kernel mounts the root file system and finds and runs /sbin/init, the first userspace process
3. /sbin/init finds and reads /etc/inittab, which tells /sbin/init what shell script to run to kick everything off (/etc/init.d/rc, usually).

That's it. All you have to do is go read that shell script and you'll find out how absolutely everything running on your system gets started, from the file systems that are mounted to the network devices that are configured to the graphical user interface. Of course, along the way, you'll run across dozens of commands and hundreds of configuration files that you'll have to look up, but with 'man' and a little persistence you will gain an understanding of each major component, where it lives, what it does, how it gets started, restarted, killed and modified.

Even better, you don't have to worry about understanding it all at once. Once you find the /etc/init.d/rc script, and see how it executes all of the other scripts in /etc/init.d, you'll be well equipped to track down the answer to any question you have about how your system works. Sometimes getting an answer will mean traipsing through a few levels of indirection, but all the information is there. No magic, nothing hidden, all there for your perusal and/or modification.

Use the source luke, use the source !

[pruneau]

Well, for starters, never forget two things:

• even if it's not documented, you _have_ access to the source
• even if you do not have time to access/read the source, the problem are usually in the open, generally discussed by someone else you could easily locate. If you are the first one, someone is usually going to answer you.

On linux, most of the configuration is done in _text_ files, and most of the system set-up is done by sh/bash shell script. Get familiar with shell scripting (it's a minimal requirement anyway), and you usually can read you way through. For example, it's a very good way to understand how the system starts, in details.
Of course, RTFM/RTFMan is always a good idea, but usually, a " --help" gets you through most of the time.
Now, of course, not everything is a shell script, and you do not have the time to study the manual pages or the documentation. Make sure that you have various tracing tools (system/network/etc):

• strace : for system calls
• lsof : for system calls and file/network access
• tcpdump/wireshark : for network tracing
• /procs : can do some of this all, but whithout real-time tracing.

Using those tools on misbehaving/unknown programs is going to give you some insight on how they work, and _what_ exactly they are doing. Their output can really be intimidating, but once you get some minial knowledge about protocols and other inner workings
And of course, having some basic idea in programming in the langague of you main applications is going to be helpful, but that really depend on how much time you want to devote to it and how programming-inclined you are.
Never forget that the unix philosophy is not about monolythic applications, but rather distributing tasks to smaller utilities. So learn about all those funny commands that are displayed when you press any keyboard letter and . You are going to be surprised: did you know that linux has "cut" and "paste" has commands ?

In my opinion the best way...

[XenoBrain]

Is to take a class at your local college or University. That's better than any book you could buy or forum you could visit. It really makes you process and apply what you learn, as well as making sure your education is well rounded. The UNIX classes at my college have no prereqs, and cater to career professionals in atmosphere, attitude, and availability and applicability. I'm sure you can do just as well if you take a look.

Another great website: Librenix.com

[KWTm]

From time to time I'll visit http://librenix.com/, a "linux tutorial" aggregator site where people collect various tips about Linux and its various applications. This is often how I will hear about various applications, methods to secure your computer, tricks for administering Linux, etc. For example, as of this writing, among the first page list of articles we have a tutorial on installing VirtualBox in Linux, emacs essentials, how to install dual monitors, etc. Most articles are good, although the styles can vary since Librenix just points to various web pages; they weren't created for Librenix itself.

Recommended.

Re:Real men

[Anonymous Coward]

I wish I could have modded this insightful AND off-topic! ;)

Books

[Knuckles]

BOOKS:
Essential System Administration By Æleen Frisch
http://www.oreilly.com/catalog/esa3/index.html

Unix Power Tools By Shelley Powers, Jerry Peek, Tim O'Reilly, Mike
Loukides
http://www.oreilly.com/catalog/upt3/index.html

Running Linux By Matthias Kalle Dalheimer, Matt Welsh
http://www.oreilly.com/catalog/runlinux5/index.htm l

The UNIX Systems Administration Handbook by Evi Nemeth, Garth Snyder,
Scott Seebass, Trent R. Hein, et al.
http://www.amazon.com/UNIX-System-Administration-H andbook-3rd/dp/0130206016/ref=pd_bxgy_b_text_b/104 -2587738-8696715?ie=UTF8&qid=1176522696&sr=1-1

The Practice of System and Network Administration by Thomas A.
Limoncelli, Christine Hogan
http://www.amazon.com/Practice-System-Network-Admi nistration/dp/0201702711/ref=pd_sim_b_4/104-258773 8-8696715?ie=UTF8&qid=1176522696&sr=1-1

Martin F. Krafft: The Debian System: Concepts and Techniques
http://debiansystem.info/

Benjamin Mako Hill, Jono Bacon, Corey Burger, Jonathan Jesse, Ivan
Krstic: The Official Ubuntu Book
http://www.amazon.com/Official-Ubuntu-Book-Benjami n-Mako/dp/0132435942

Why not do the most obvious thing?

[Eggplant62]

Find and join your local Linux Users Group. Start here on GNU.org's List of Linux User Groups and see what you can find. Most of everything I know from Linux is either:

1. What I learned from my local LUG
2. What I learned from my best friend, the Linux Guru
3. What I learned from reading a multitude of books and websites
4. Through classwork at the local business college with a Linux-friendly IT program

Interact with people who know about Linux. Ask questions. Read HOWTOs. Get reference books and read them.

Tools are the key

[simm1701]

The real key to unix (any unix) is knowing how to use the tools that you get installed with the OS to make your life easier.

First things first is the terminal (xterm, kterm whichever it doesn't matter) use it. Forget the GUIs, use the shell.

That brings us onto the shell itself. Pick one and stick with it for a while. On linux most people prefer bash, its a good choice as even though its not on all unixes by default its not difficult to obtain (just don't try to set the root users default shell to bash on solaris)

Learn the language of the shell, pipes, redirects, command line interpretation of special characters, handy tricks like tab complete, loops, variables, tests and use these all on one liners - progress to script files and also learn about functions - shell scripts are usually going to be fairly primitive tasks but they are the key to an easy life as an admin.

Man is your friend - and should always be the first place you turn for help, then google, then forums.

Learn the basic commands, ls, mv, cp, rm, learn their options and understand the justifications for using them (-i? -f?)

Pick an editor and learn to drive it - this is a long process but well worth it. Don't bother with a GUI one, consider that later. On a default linux install you will probably have vim and emacs - try them both, see which you like and use it. Check out guides on how to customise them until they behave just how you want them (I have a 10 line .vimrc file I can create from memory that makes vi behave just as I like) - ok yes vim != vi, but to shock vi purists I like to be able to use the cursor keys while in insert mode!

learn atleast the basics of the other important tools - at the very least find and grep. Awk and sed should certainly be on the list as you will encounter many scripts that use them, atleast some basic knowlege of perl would be handy (I prefer to use perl instead of awk and sed but thats my preference not everyone would agree)

Set up services and experiment, run a webserver, database server, mail server and learn as much as you can stand to about iptables to secure your box.

Keep backups, don't be afraid to break things, fixing what you broke (after finding out what you did to break it) is some of the best education yuo can have

All in all

Have fun!!

Re:Thanks

[Budenny]

They are very good suggestions, but I would start somewhere a bit simpler.

Scott Graneman Linux Phrasebook
-- this isa fairly small but amazingly comprehensive and very clear book on using the command line. The OReilly Pocket Linux is also good but much more limited.
Ward How Linux Works
-- fairly discursive, but once you've read it, you understand how it all works and the detail will slot into place
Schroder Linux Cookbook (or actually, anything she writes)
-- well, she's brilliant, doesn't cover everything, but what is covered is clear, detailed and after you work through it, you can do it, and you understand it.
Linux in a Nutshell
-- this is a sort of paper version of man, Graneman's book is a subset with more examples. But if you have this you can find every option in every command in an instant. Have it for reference.

I agree about slackware. Install slackware and work through their brilliant documentation. This is a good accompaniment to Ward's book.

"How Linux Works" No Starch Press

[flotationIsGroovy]

How Linux Works describes the inside of the Linux system for systems administrators,
whether you maintain an extensive network in the office or one Linux box at home.

Some books try to give you copy- and-paste instructions for how to deal with every
single system issue that may arise, but How Linux Works actually shows you how the
Linux system functions so that you can come up with your own solutions.

After a guided tour of filesystems, the boot sequence, system management basics,
and networking, author Brian Ward delves into open-ended topics such as development
tools, custom kernels, and buying hardware, all from an administrator's point of view.

With a mixture of background theory and real-world examples, this book shows both
"how" to administer Linux, and "why" each particular technique works, so that you will
know how to make Linux work for you.

Amen

[matt me]

If it ain't broke, break it. Fix it. Understand it.

Here's some things to try:
Compile mplayer. Build a firefox package for your distro. Download, patch and compile a kernel. Add a cronjob to delete some files at random. Forget about it. Try and restore them. (ext2 does not delete file pointers in inodes). Create an LVM group. Siphon disk space between disks and partition. Try a file system written by someone suspected of murder. Post on a linux-devel list and join Linus in slagging off Gnome. Try KDE and XFCE and decide no-one needs a desktop environment. Try some ultra minimal *box window manager. Live in the terminal for a week. Log in to your computer using SSH from afar and wonder why when you play music you can't hear it. Rsync something. Write your own superior syncing script. Write some perl scripts. Python. Add a RAID. Try an old 2.4 kernel..