Slashdot Mirror
Optimizing Linux Use On a USB Flash Drive?
Buckbeak writes "I like to carry my Linux systems around with me, on USB flash drives. Typically, SanDisk Cruzers or Kingston HyperX. I encrypt the root partition and boot off the USB stick. Sometimes, the performance leaves something to be desired. I want to be able to do an 'apt-get upgrade' or 'yum update' while surfing but the experience is sometimes painful. What can I do to maximize the performance of Linux while running off of a slow medium? I've turned on 'noatime' in the mount options and I don't use a swap partition. Is there any way to minimize drive I/O or batch it up more? Is there any easy way to run in memory and write everything out when I shut down? I've tried both EXT2 and EXT3 and it doesn't seem to make much difference. Any other suggestions?"
USB has high cpu use and encrypted does not help a firewire based flash drive will be a lot faster. USB 3.0 may help but it too may still the limit of usb 2.0 high cpu use as well.
make sure you are on USB 2.0- interface can kill you.
Also did you check the faq-
No seriously:
http://www.linux-usb.org/FAQ.html#i5
especially the section on:
Q: What is max_sectors and how should I use it?
A:For USB Mass Storage devices (that is, devices which use the usb-storage driver) max_sectors controls the maximum amount of data that will be transferred to or from the device in a single command. As the name implies this transfer length is measured in sectors, where a sector is 512 bytes (that's a logical sector size, not necessarily the same as the size of a physical sector on the device). Thus for example, max_sectors = 240 means that a single command will not transfer more than 120 KB of data.
Puppy Linux is tiny and is set up to boot off of USB. After it's booted, if the system has enough RAM, the entire system is loaded into RAM. Makes for a damn snappy system.
http://www.puppylinux.org/
True, but some laptop and desktop designs have went away from USB connectedness.
For example, on my Thinkpad the SD reader is its own bus. The Bluetooth is a USB 1.1 (grr) device, so I need to rmmod the bt modules and remove usb old modules to be power efficient.
It really is a crapshoot on what the computer maker put on the USB bus. I just lucked out.
No, put it on flash. It'll be faster
No, it will most certainly be much slower. The memory page size is 4K, whereas the typical flash block size is much bigger. Flash can only be erased in blocks, so to write one small 4K page to flash memory, the controller has to read a whole block, erase the flash block and write back the modified block. That's why most flash storage devices suck so badly at storing many small files, even though the continuous write speed isn't too bad.
The performance penalty could be hidden by the OS: Instead of swapping out when you need the memory and have to wait for the write operation to complete, it could swap out preemptively and in larger chunks, mark the pages as swapped out but valid, and keep using the memory (invalidating the swap on writes). Then, if something else needs the memory, it only needs to swap out the pages which have been invalidated in the meantime. The drawback is that USB is processor-intensive, so you'd have to find a balance in order to avoid unnecessarily bogging down the CPU (thereby wasting battery life) with useless swap writes and not waiting too long for small swap writes. Needless to say, this is not how swap works today.
Every access to FAT writes to the same area of the file system (the FAT), and this is what destroys old-style floppy disks.
That's why people use a flash file system http://en.wikipedia.org/wiki/File_system#Flash_file_systems. http://en.wikipedia.org/wiki/YAFFS is a good choice; it seems to be used in Android.
Nothing for 6-digit uids?
Flash file systems are for raw NAND, not NAND behind a microcontroller handling wear leveling. FAT would kill the chip on the XO board, but the same chip in a USB drive would be fine.
Support my political activism on Patreon.
Yes and it works quiet well in real world performance testing.
Even a 1gb or 2gb stick can improve overall responsiveness enough that makes a tight XP installation feel slow. This is especially true with gaming and applications that push the limits of your RAM shoving the HD cache down.
There are a few crap Flash drives, that just don't perform well, avoid them and it works like promised.
no matter, you STILL don't want to be swapping to flash! better to load up on RAM and forget swap.
I run exclusively off a flash drive. I'm nomadic and it's easier to haul around than a laptop.
Many apps like to call fsync() needlessly, causing many writes to occur to the flash drive. Buffering all these writes in RAM until you halt the system works around this.
One of the largest performance gains I've gotten is by using tmpfs to store all of firefox's data.
Strategy is as follows:
On boot, mount ~/.firefox as tmpfs. Extract backup tarball into this dir.
On halt, generate tarball of ~/.firefox
Some scripts are at http://coder.home.cosmic-cow.net/src/usbstick/
I use this strategy with firefox, pidgin, and a few other apps. The performance gain is amazing.
You could also skip flash entirely and buy a very small hard drive. I've got a 60-gig USB drive from Apricorn that I carry around in my pocket, with an AES-encrypted root filesystem. Performance isn't spectacular, but it's certainly usable.
Reads can update access time on Windows and always update on DOS, which gets written to the FAT.
Fixed that for you.
Windows XP: Start > Run > cmd
enter this command:
fsutil behavior query disablelastaccess
Just to let you know, tmpfs ignores the device path, you can put whatever you want in it (and you aren't actually using /dev/ram* with that).
-:sigma.SB
WARN
THERE IS ANOTHER SYSTEM
Dave Jones recently posted elsewhere his notes for improving things on the eee900. Several of the steps focus on getting proper performance out of the flash, and so would also apply to booting from a USB thumb drive or other flash media. Here's what he had to say:
Program Intellivision!
Nowhere does op claim to have used ReadyBoost on XP, he compared Vista w/ ReadyBoost to a 'tight XP installation'
Meaning Vista+ReadyBoost is apparently 'faster' as far as responsiveness goes than a clean and properly tweaked windows XP installation.
I still think pics or it didn't happen, but never was ReadyBoost on XP mentioned.
There's quite a discrepancy over the speeds on flash drives. Cheap flash drives run USB 1.1 and the transfer rates are around 1 MB/sec. USB 2.0 drives range from slow 10 MB/sec to close to 40 MB/sec. The fastest drives will cost easily over $100. The size of the drive will slow the transfer rate also. 4 GB drives are faster than 8 GB and so on. Corsair GT drives have close to 35 MB/sec read speeds, write speeds are always dramatically slower. I've installed a bootable PBX in a Flash on a 4 GB Corsair USB flash drive with very acceptable performance for teaching purposes. I can see other LAMP bootable installations popping up. Each student can have their own server to configure and boot-up then take shut it down and take it home.
puppy linux can load to ram and then no longer needs the loading medium
Apparently wear leveling is no panacea. Some devices wear-level only within a zone, and so a lot of activity in a hot zone can still burn that zone out. I guess such an algorithm only really works well when writes are distributed across zones evenly.
See page 5 in this document for a description of zone-based wear leveling.
Program Intellivision!
2) Use a Linux distribution with a smaller footprint such as DSL (50MB) or Puppy Linux (standard edition is 68MB): the smaller the footprint, the less your drive has to read, the faster your system will load.
Puppy does exactly what the Original Poster asks for. It loads everything into RAM and when you shut down/reboot it asks if you want to save the changes to disk - this includes newly installed packages, updates, your documents, settings etc. You have the option for this "save file" to be encrypted (and password protected) or not.
Puppy has also done a lot of optimizing for running on a USB stick, and it can handle encrypted partitions. Check it out: puppylinux.org