Optimizing Linux Use On a USB Flash Drive?

Buckbeak writes "I like to carry my Linux systems around with me, on USB flash drives. Typically, SanDisk Cruzers or Kingston HyperX. I encrypt the root partition and boot off the USB stick. Sometimes, the performance leaves something to be desired. I want to be able to do an 'apt-get upgrade' or 'yum update' while surfing but the experience is sometimes painful. What can I do to maximize the performance of Linux while running off of a slow medium? I've turned on 'noatime' in the mount options and I don't use a swap partition. Is there any way to minimize drive I/O or batch it up more? Is there any easy way to run in memory and write everything out when I shut down? I've tried both EXT2 and EXT3 and it doesn't seem to make much difference. Any other suggestions?"

Comments

Did you read the faq?

[agristin]

make sure you are on USB 2.0- interface can kill you.

Also did you check the faq-

No seriously:

http://www.linux-usb.org/FAQ.html#i5

especially the section on:

Q: What is max_sectors and how should I use it?

A:For USB Mass Storage devices (that is, devices which use the usb-storage driver) max_sectors controls the maximum amount of data that will be transferred to or from the device in a single command. As the name implies this transfer length is measured in sectors, where a sector is 512 bytes (that's a logical sector size, not necessarily the same as the size of a physical sector on the device). Thus for example, max_sectors = 240 means that a single command will not transfer more than 120 KB of data.

Re:Hrm.

[Creepy Crawler]

True, but some laptop and desktop designs have went away from USB connectedness.

For example, on my Thinkpad the SD reader is its own bus. The Bluetooth is a USB 1.1 (grr) device, so I need to rmmod the bt modules and remove usb old modules to be power efficient.

It really is a crapshoot on what the computer maker put on the USB bus. I just lucked out.

Re:Get a swap partition

[Anonymous Coward]

No, put it on flash. It'll be faster

No, it will most certainly be much slower. The memory page size is 4K, whereas the typical flash block size is much bigger. Flash can only be erased in blocks, so to write one small 4K page to flash memory, the controller has to read a whole block, erase the flash block and write back the modified block. That's why most flash storage devices suck so badly at storing many small files, even though the continuous write speed isn't too bad.

The performance penalty could be hidden by the OS: Instead of swapping out when you need the memory and have to wait for the write operation to complete, it could swap out preemptively and in larger chunks, mark the pages as swapped out but valid, and keep using the memory (invalidating the swap on writes). Then, if something else needs the memory, it only needs to swap out the pages which have been invalidated in the meantime. The drawback is that USB is processor-intensive, so you'd have to find a balance in order to avoid unnecessarily bogging down the CPU (thereby wasting battery life) with useless swap writes and not waiting too long for small swap writes. Needless to say, this is not how swap works today.

Dave Jones suggestions for the eee900 might help

[Mr Z]

Dave Jones recently posted elsewhere his notes for improving things on the eee900. Several of the steps focus on getting proper performance out of the flash, and so would also apply to booting from a USB thumb drive or other flash media. Here's what he had to say:

Making the eee 900 series suck less.

Recently I've read about or spoken with a few people using Fedora on eeepc's who have been making some fairly big blunders without realising it. I've played with a few of these now, in their various incarnations.

The current one I've been carrying around is the 900 model with a whopping 20GB of flash. It's quite deceptive, because there are actually two SSDs in there (one 4GB, and one 16GB)

These ssds are also pretty damn awful performance-wise compared to the newer generation of SSDs, but short of opening it up and retrofitting something, there's not much that can be done. The tips below should at least make it more bearable.

• First off, don't use the default partitioning scheme.

  By default, anaconda will choose to use lvm, and make a contiguous volume out of the two SSDs. This idea is fail, because the two disks aren't the same, and run at different speeds.

  # hdparm -t /dev/sda

  /dev/sda:
  Timing buffered disk reads: 108 MB in 3.04 seconds = 35.57 MB/sec

  # hdparm -t /dev/sdb

  /dev/sdb:
  Timing buffered disk reads: 86 MB in 3.05 seconds = 28.20 MB/sec

  So, don't do that. Just create regular partitions, and make sure you put / on the faster of the two disks (the 4GB one), and leave the 16GB one for /home

• Next, the default filesystem will be ext3. You really don't want this.

  Given the journal is in a fixed location on disk, scribbling to it every time a file gets written is a great way to wear out the flash. Go with ext2. (Given that you've only got a few GB of flash anyway, a fsck doesn't take that long should you need to). Additionally, not having to write to the journal means that you're doing less IO, which is obviously a win when it's on such slow media.

• This should go without saying - no swap.

  Not only for the flash wear problem in the previous bullet, but also, because it's slow as all hell. If you find you run out of ram and get stuff oom-killed in this setup, well, you probably need to add more ram, or consider a real laptop.

• After installing, change the fstab so that everything gets mounted with noatime.

  Writes to the disk are just painful, so minimising them is the path to success here. It's doubtful that you'll be running anything on an eee that would actually care about atimes anyway.

With these points taken into consideration, the eee isn't a half-bad machine. I still wouldn't want to be building kernels and such on it, but it's perfectly usable for email and such whilst travelling.

Use faster flash-drives

[blanchae]

There's quite a discrepancy over the speeds on flash drives. Cheap flash drives run USB 1.1 and the transfer rates are around 1 MB/sec. USB 2.0 drives range from slow 10 MB/sec to close to 40 MB/sec. The fastest drives will cost easily over $100. The size of the drive will slow the transfer rate also. 4 GB drives are faster than 8 GB and so on. Corsair GT drives have close to 35 MB/sec read speeds, write speeds are always dramatically slower. I've installed a bootable PBX in a Flash on a 4 GB Corsair USB flash drive with very acceptable performance for teaching purposes. I can see other LAMP bootable installations popping up. Each student can have their own server to configure and boot-up then take shut it down and take it home.

Re:Get a swap partition

[Mr Z]

Apparently wear leveling is no panacea. Some devices wear-level only within a zone, and so a lot of activity in a hot zone can still burn that zone out. I guess such an algorithm only really works well when writes are distributed across zones evenly.

See page 5 in this document for a description of zone-based wear leveling.