Linux Tuning Tricks?

Milo_Mindbender writes: "Over the weekend I was attempting to improve my CD ripping performance and discovered RedHat 7.2 was running my Ultra/ATA 100 hard drive in a very slow non-DMA mode. After a fair amount of searching for how to fix this, a trivial change (look here) improved drive performance from 3MBs to 38MBs! FSCK on my 40gb partition went from over 5 minutes to under 1! This issue wasn't documented in RedHat's manuals but it effected a number of boxes in our office so I'm betting many other people in the world have the same problem. This made me wonder how many other common Linux tuning snafus there might be that a lot of people are probably missing. Do you know of any?"

Comments

disable system services on startup

[babychess]

On RedHat, you can use ksysv, the init editor, to turn off boot-time system services that are not needed. Candidates are daemons for GPM, USB, SCSI, LPD, APM, which are all enabled by default, and which may not be needed.

Re:disable system services on startup

[eufaula]

with redhat (maybe others) you can also run the command "setup" and have a curses-based way to do it. or, you can go into /etc/rc3.d (or whatever runlevel your distro/os normally runs at) and move the S##xxxx scripts to K##xxxx to prevent them from starting. this is the standard way of starting/stopping services on SYSV UNIX, so this holds true for Solaris and others.

-- aside -- you CAN move them to anything other than S##xxxx, but i normally stick with the standard and use K##xxxx

Re:disable system services on startup

[Diamon]

Renaming them to K##xxxx is not necessarily a good idea. Instead of just not telling them to start, you're telling them to Kill off the service if it's running. This will slow down your boot as you try to kill of things that never started.

I always preferred what my predecessor used of renaming S##xxxx to _S##xxxx. This keeps things clear as to what the system *used* to do.

Re:disable system services on startup

[eufaula]

Renaming them to K##xxxx is not necessarily a good idea. Instead of just not telling them to start, you're telling them to Kill off the service if it's running. This will slow down your boot as you try to kill of things that never started.

true, but if you do this you have to make sure that they dont get started in any other runlevel. (runlevel 2, etc...). this is why i stuck to using the recommended K##xxxx. and if the /etc/init.d scripts are written properly, killing them off isnt much of an issue as there should be pid files in /var/run (or whatever is applicable per your OS).

Re:disable system services on startup

[briany]

Actually, the K* scripts are run when you leave that runlevel. Your shutdown will be slower as the system is trying to shutdown daemons that aren't running.

I usually move the scripts from S* to s*. This way I can tell quite easily what I've disabled by hand.

Re:disable system services on startup

[Diamon]

Oops, yeah sorry, my bad. Of cours the K scripts do not run at boot as I said. Must be a result of too much time using Windows clouding my mind.

Re:disable system services on startup

[jsprat]

you CAN move them to anything other than S##xxxx, but i normally stick with the standard and use K##xxxx

Actually, in the rc.? directories they are links to the scripts. You could actually delete them to keep them from starting without hurting your system at all (don't try this in the init.d directory). Changing them from S to K (as suggested in a later post) won't prevent the services from being started at run-level 2, unless they are changed in rc.2

As an aside, if you're running a RedHat based distro, try 'man chkconfig'. It might be useful;)

Just a tip.

[Anonymous Coward]

s/effected/affected/

Not trying to be a prick, just trying to let you know for future use! I see this mistake a lot, so...

"Effect" is a noun, whereas "affect" is a verb.

When you change something, you affect it. When something you did caused a change, you say the thing you did "had an effect".

(Effect can also be a verb meaning "to cause", but it's rarely used that way. i.e. "I effected a change.")

Sorry for the grammar nazi post!

Re:Just a tip.

[suwain_2]

This is a common mistake, as you said. I think this is one of those words where the meaning keeps evolving... I wouldn't be surprised if, at some point, some dictionary gives in and just says something like "In modern usage, effect and affect are interchangable."

For now, though, this helfpul tip from Merriam-Webster is pretty informative (and interesting):

"The confusion of the verbs affect and effect is not only quite common but has a long history. Effect was used in place of 3affect as early as 1494 and in place of 2affect as early as 1652. If you think you want to use the verb effect but are not certain, check the definitions in this dictionary. The noun affect is sometimes mistakenly used for effect. Except when your topic is psychology, you will seldom need the noun affect." -- From the "effect" entry in Merriam-Webster's online Collegiate Dictionary.

Re:Just a tip.

[Anonymous Coward]

Uhhh..."effect" can also be used a verb, meaning "to make manifest" or "to enable."

Prostitute Penguin

[JonBob]

I had to read this story's subject three times before I realized it did not say "Linux Turning Tricks."

Re:Prostitute Penguin

[WINSTANLEY]

I every time I see bugtraq I am sure it says butrag

Re:Prostitute Penguin

[EnVisiCrypt]

I thought the same thing when I read the headline.

> "Linux Turning Tricks."

I could make a joke about open "sores", but I won't.

Re:Prostitute Penguin

[flimbag]

You think that's bad? I initially read it as Linus turning tricks.

I thought he must have given up programming and taken up hustling his ass as a new career.

A very nice, recent article ...

[hubie]

Linux Journal has a nice article on fine-tuning your system (doing things like recompiling the kernel with the best compiler options and optimizations (it also covers hdparam).

My only gripe with LJ articles is that, even if you put them in print mode, they still run off the end of my paper when I print them.

Re:A very nice, recent article ...

[Anonymous Coward]

When will web developers learn the the standard resolution of printers it 600 wide (Thats from the left most margin)

When developing web pages remember, if you create a printable version tablize it to a width of 600 or less.

You pep's will be much happier!

Re:A very nice, recent article ...

[NotoriousQ]

cool article, thx for posting

however, something about it worries me.....they are recommending using -O9 for compiling the kernel.....does that not sound scary?

Re:A very nice, recent article ...

[dubl-u]

My only gripe with LJ articles is that, even if you put them in print mode, they still run off the end of my paper when I print them.

So I just looked up this "paper" thing on Google, and it sounds really cool, kinda like a flat-panel display with a built-in battery. But how do you slice the trees so thin?

Re:A very nice, recent article ...

[raju1kabir]

But how do you slice the trees so thin?

You use a very special tool called a "tunglade".

UDMA, etc.

[linuxator]

The truth is, that even on "leading desktop OS" called with a name that reminds sheet of glass between frame, has not UDMA/66 and higher mode turned on by default.

Problem is, that it breaks compatibility with older hardware... eg. you will put an old harddisk on your comp. and it will "blow up" becouse your fancy OS will think it can take faster transfer speeds... etc.

Re:UDMA, etc.

[dfreed]

So if it is not turned on, how do we activate it?

Re:UDMA, etc.

[Tuzanor]

Reg hack'll do it on a win2k/xp box: Reg Hack

Same here

[Otter]

I always thought "At first I thought the headline said _____!" posts were lame, but I was just about to post the same thing. I swear I read it three times too before getting it right.

Considered linking to that Penny Arcade strip of the User Friendly characters turning tricks but don't feel like investing the time to find it...

Re:Lame Grammer Post

[Anonymous Coward]

And yes the title's deliberate. What isn't deliberate is me posting as me instead of anon coward. Damn. Something always goes wrong.

Check for interrupt clashes, and test

[Mark Wilkinson]

We recently built a machine around an Abit VP6 with 5 hard disks on it (three on the main IDE controller, 2 on the HighPoint 370 secondary controller). After a few days I noticed we were getting bad blocks on the drives on the HighPoint controller. Running badblocks on the disks gave random errors all over the place.

I then noticed that the Ethernet card was being given the same IRQ as the IDE controller and got suspicious. I swapped the ether card into different PCI slots until it got its own IRQ, then ran the badblocks check again. Everything ran clean.

It's also entertaining to use lmbench to test your hardware: it can plot pretty graphs showing how the IO speed changes across the disk surface so you can decide where to put your partitions, if that's important to you.

Main point though: if at all possible tune your hardware and then test using badblocks, lmbench and such before you put the machine into production (or when you've got a solid backup). As the article says, problems with your disk subsystem can loose lots of information quickly.

Re:Check for interrupt clashes, and test

[LordNimon]

A good BIOS can assign IRQs to slots.

Re:Check for interrupt clashes, and test

[Strog]

Sometimes they are hardwired so bios doesn't cut it. Any good manual will tell which slots are sharing interupts. I guess you could say RTFM in that case.

You know you're reading /. too much when...

[itwerx]

I was reading the comments here and when I got to the end I read the little cookie/quote which is always at the bottom of the page. This one said:
Dr. Livingston? Dr. Livingston I. Presume?

I had to read it several times before I realised that it was an italic "I." and not "/."
(I can't get the font to set properly in here, so it doesn't look as much like a slash now)

wow.. I never knew that..

[josepha48]

.. okay I did, in fact my rc.custom startup script does this. This is called from rc.local, which never seems to be called from the init scripts on any of my RH installatinos. So I add it rc.sysconf.

This way when I upgrade I don't end up with issues like this guy.

I guess I am supprised as I would have though that this would be something you'd see on Linux.com not slash.

It is a good idea to make sure that before you go mucking with hdparm that you do so knowing that if you make mistakes you can really screw up your harddrive.

Re:wow.. I never knew that..

[Sentry21]

Amen to that. I had an old fujitsu hard drive, and I started playing with hdparm one day. Basically I turned on every possible option, including the ones marked in the man page as 'this will probably cause extreme filesystem corruption' and 'what are you, on crack? get the hell away from here!', and it worked great. Performance was simply excellent.

Overzealous, I recommended these options to friends, who all locked up when they tried them (pinging out from IRC, etc), but I was undaunted. When my old HD started to give up the ghost, I got a new (well, new to me, this was 6 months ago) 2 gig WD for the box, and turned all the options on. Hours later, the entire filesystem was corrupted. After the whole system nosedived, I spent more than two hours hitting 'y' in fsck before I decided 'screw it' and reformatted. Fortunately, it was a clean install anyway.

Moral of the story: test out your psychotic options before you put any important data on the drive. Other moral: say what you like about fujitsu, but their drives support more hdparm options than any other I've seen, and they don't break when you use them.

--Dan

Re:wow.. I never knew that..

[Anders]

After the whole system nosedived, I spent more than two hours hitting 'y' in fsck before I decided 'screw it' and reformatted.

You have heard of the -y switch to e2fsck, right? Not that it saves your file system, but it relieves you of pressing 'y' while learning that you are fscked.

Re:wow.. I never knew that..

[Sentry21]

'Unexpected incosistancies were found in the filesystem. Run e2fsck again without the -y or -a flags'

Or something. I was running e2fsck manually.

Thanks for the tip either way though. :>

--Dan

Re:wow.. I never knew that..

[Peale]

Other moral: say what you like about fujitsu, but their drives support more hdparm options than any other I've seen, and they don't break when you use them.

I know I'd never say anything bad about the drives. I've got two: an old 1G drive, and a 3.4 that's still the boot drive of my machine. Still lightning fast for it's time, and ultra quiet. And it's been thru more abuse than I care to mention, and it's still going. I'd buy more Fujitsu stuff, but didn't they sell to Maxtor?

Re:wow.. I never knew that..

[Anonymous Coward]

Wouldn't

yes e2fsk

have worked?

*Windows user*

Careful with hdparm!

[dead_penguin]

If you are going to be playing with hdparm, take my advice and make a backup first! Some interfaces aren't fully supported by the kernel yet, and trying to run drives off of them in certain modes could break in a bad way. In my experience, this then means massive filesystem corruption and a complete reinstall.

Of course I'm not saying *don't* play with hdparm; just be sensible and only try it on a system you have backed up and can afford to lose for a little while as you're rebuilding it.

Re:Careful with hdparm!

[dimator]

People, don't pretend you're cool and not make backups, thinking that nothing will happen if you go screwing with hdparm. I learned the hard way, and now I'll be digging up all my $HOME directory from cryptically named files in lost+found. It should be a fun night.

I have only myself to blame.

Re:Careful with hdparm!

[Stinking Pig]

On that note, don't think that being cool and running soft RAID means you don't need backups; a corrupted file system redundantly copied to another disk is, well, still corrupt.

Great tips!

[mteichrob]

This is a really cool article! Just wanted to say thanks for posting it.

Matt

Distributing software but not knowledge = problem

[tshoppa]

While Redhat is pretty good at making a distribution that boots and installs on a very wide number of machines, it's not so good at making this distribution be high-performance. Many things are set to the safest possible value (like the OP's IDE DMA modes) when a much more reasonable value would work on 99% of the hardware out there.

Life is made more difficult because there is buggy and/or broken hardware out there. I don't blame Redhat for accomodating this hardware, but by doing so they are making their distribution more complicated and less useful for those "in the know".

Redhat also, of course, distributes the non-kernel binaries optimized for Intel 80386 CPU's when the vast vast majority of installs are going on Pentium-class or better machines. And it doesn't help any that Redhat is using and distributing a very nonstandard version of GCC; see what the GCC developers say about such branches and what application developers say about this branch.

To actually learn a lot about Linux and all the associated tools that make it work, I highly recommend the Linux From Scratch method: build everything from source! You can optimize the build to your machine and end up with not only better performance, but a vastly superior knowledge of everything that used to be "under the hood".

Windows does the same thing.

[decep]

The same exact thing happens on Windows boxes, but we're not supposed to feel sorry for them, right?

Re:Lame Grammer Post

[SomethingOrOther]

Errrrrm,
Correcting someones spelling/grammer is a bit like shouting 'Hitler' in an argument.

You have just automaticaly lost. :-)

Re:Lame Grammer Post

[The AtomicPunk]

Spelling grammar correctly would help your case. :)

Re:Distributing software but not knowledge = probl

[LordNimon]

The Red Hat / gcc thing is old news. Just look at the date of that article on gnu.org.

PCI Bus speed

[optikSmoke]

Another thing I recently noticed which jumped out at me from my kernel messages during boot: the kernel was assuming a 33Mhz system bus speed for PIO. This was fixed by passing "idebus=66" as a kernel boot parameter. See ide.txt in the kernel documentation sources for more info.

Re:PCI Bus speed

[Gothmolly]

Your PCI bus speed IS 33MHz, unless you are overclocking or running a new, rare, high-end box.

Re:PCI Bus speed

[optikSmoke]

Except, if I'm not mistaken, on AMD systems where system bus speed = 133MHz, and since PCI bus = system bus/2, PCI bus = 66MHz, right? This was my reasoning, though I could very easily be wrong.

Re:PCI Bus speed

[Anonymous Coward]

The PCI bus still runs at 33mhz.

At speeds over about 120mhz FSB, you want a 1/4th PCI clock divider.
The 100mhz bus processors use a 1/3 divider

Re:PCI Bus speed

[SuiteSisterMary]

Or pretty much any server with 66 megahertz PCI slots.

Re:PCI Bus speed

[Anonymous Coward]

66 MHz PCI does exist (as a separate, but compatible standard) on higher-end motherboards (the Dual Athlon boards support it, as do Macs since the B&W G3s). The original PCI spec designates a bus speed of 33Mhz. The card expects to get a clock at this rate, and if it gets a 66MHz clock, it's going to be completely unstable. Boards designed for 66MHz PCI detect 33 mhz boards, and half the clock rate so as to run them at their spec'd speed.

May want to learn about 2.96 before you bash it.

[rjh]

GCC 2.96-RH fails with C code in one and only one condition: when it's not ANSI-conformant C. GCC 2.96-RH is much more restrictive than GCC 2.95.

GCC 2.96-RH is worlds better than 2.95 when it comes to generating C++ code. With GCC 2.96-RH and STLport-4.5.3, I was happy as a clam. GCC 2.95's C++ support was anemic, nonstandards-compliant, and just generally sucked for any sort of very serious usage.

When I was recompiling tons of applications to work with GCC 2.96-RH, every "compiler error" that I found-- every single one --was the result of a programmer doing something that wasn't ANSI/ISO and made an assumption that "well, since GCC 2.95 didn't complain about this, it's okay." I don't blame GCC 2.96-RH for being more standards-conformant; I blame developers for playing fast and loose with the language spec.

Re:Lame Grammer Post

[Louis_Wu]

lol

Oh to have mod points.

Mod this up! Everybody gets this wrong!

[andaru]

I see this one all of the time in posts, in both directions.

You would expect the hit rate to be at least 50%, but it seems like people almost always use the wrong one.

Re:Turning Linux

[Mastoid]

Back, foul penguins of darkness!

<rolling dice>

Powertweak and /proc twiddling

[embobo]

You may try to use Powertweak to alter settings to improve performance.

Then there is tweaking settings via /proc. I used to have a link to some excellent documentation on it but, alas, I can't seem to find it. You could try reading the various bits of info in the Documentation tree of the Linux source but it is pretty spartan.

"Tabilize".... Or just use plain text.

[Bob_Robertson]

I agree that wide tables are annoying, but HTML viewers (and printers) nicely re-wrap plain text to fit.

Sooooo much easier than tables is just to use preformatted text.

Just a side comment from someone who has always written his web page in vi.

Bob-

mod parent down! who gives a sh*t

[Anonymous Coward]

Really, who cares?, you knew what the poster meant, he knew, almost everyone knew what he meant. Why don't you try and answer the question rather than wasting time trying to make your self look "big" because you can pick people up on their spelling! Do something worthwhile... like someone sensible said: "if you can't say something nice, don't say anything" And yes I appreciate the irony telling you to only say nice things while at the same time flaming you so.... As for hdparm, redhat linux 7.2+ does quite a good job of automatically detecting the need to dma or not so long as you are using 2.4 kernel and modern hardware.. while hdparm is useful it is redundant for new kernels / hardware. You do however have to have the right ide drivers in the kernel.

hdparm is not usually needed...

[Manic Miner]

IIRC hdparm is usually not needed for 2.4 serial kernels as dma detection etc. is built into the ide drivers.

I know that the last view redhat 7.2 installs I have done on relatively modern hardware, and an old PII as well, have picked up the correct dma and 32bit IO modes automagically

mine works

[cloudmaster]

Funny, the Ultra/100 drive in my Redhat 7.2 box is using DMA by default:

# /sbin/hdparm -tT /dev/hde
/dev/hde:
Timing buffer-cache reads: 128 MB in 0.72 seconds =177.78 MB/sec
Timing buffered disk reads: 64 MB in 1.82 seconds = 35.16 MB/sec

# /sbin/hdparm -d /dev/hde
/dev/hde:
using_dma = 1 (on)

I know that I didn't have to do anything special to get it working - I just did a regular install and let it go.

Re:mine works

[Black Parrot]

> Funny, the Ultra/100 drive in my Redhat 7.2 box is using DMA by default:

I built a box and it worked fine too... until I changed out the motherboard, and then I had to go discover the stuff the author is talking about. It was very clearly enabled in the BIOS of the new m.b.; it just wasn't automatically picked up by Linux like it had been on the original m.b.

Re:mine works

[Anonymous Coward]

mine doesn't.

I'm running RedHat 7.1 on a Compaq iPaq Desktop legacy free (Cel 500/512MB). Allegedly the IDE controllers support UDMA 66, as does the Maxtor 10GB hard disk I have installed as replacement for the original 4.6GB one.

I've gone through a load of hdparm config options this afternoon, and have not seen transfer rates greater than 3.6MB/sec.

Maybe its a peculiarity of this machine... but with these settings:

hdparm -m16 -X66 -d1 -u1 -c3 /dev/hda
I get the following

hdparm -Tt /dev/hda

/dev/hda:
Timing buffer-cache reads: 128 MB in 1.55 seconds = 82.58 MB/sec
Timing buffered disk reads: 64 MB in 19.01 seconds = 3.37 MB/sec

Any ideas???

Practically speaking...

[aquarian]

Of course there are hundreds, possibly thousands, of little tuning tweaks to make Linux perform better. But practically speaking, this is the main one. The other is shutting down services you don't need.

In an effort to get a working system on every possible computer, Redhat defaults to no hard drive DMA and everything but the kitchen sink running in the background. Fix these two little things, and you're 99% there.

Pretty good resource

[Yoda2]

Lots of nice little Linux tidbits can be found on The Linux Pimp. Can't think of a site more ontopic for this post. The intro is pretty funny too.

Tweaking your GUI

[Linuxathome]

At the risk of getting flamed by KDE enthusiasts here, if you are using X windows, I would have to say that you should look into using window managers other than KDE (or even Gnome for that matter). Although KDE and Gnome very pretty, I still prefer to use just plain Enlightenment. If you want to run KDE or Gnome apps (which I still can and do), just install the libraries. I've removed all my window managers from my install and am just running a clean E desktop. My screen is not cluttered with window bars and icons; my apps are accessible through a pop-up menu with a click of the mouse; and best of all, it's still very fast to login to my account/desktop (faster than windows 95)--and this is all on a dinky Cyrix II 333MHz system that has uptimes on the average of 30-40 days before I reboot for a kernel upgrade or some other system setting change.

Re:Tweaking your GUI

[dt23507]

I'd also like to suggest fvwm2. It's also a very lightweight and fast window manager, and you can still use KDE and Gnome apps if you like.

Same for DVDs

[Alex Belits]

The same mode change fixes "jerky" movement when playing DVDs (for obvious reason). Observed on ogle and BDV-212B, (non-region-locked, firmware) drive.

performance tuning?

[zonker]

hasn't someone written a book on performance tuning linux? just a suggestion for those of you who might be interested in such a thing...