Slashdot Mirror
Which Filesystem Do You Use On Portable Media For Linux Systems?
An anonymous reader writes "Most people use MS filesystems on Disk-On-Keys, and portable hard drives, as these are readable from most machines. But this way you lose the files' permission information, which many times is very inconvenient (you must agree that having Ubuntu asking you whether to execute or display every text file or image you open from a DOK is annoying). Using 'regular' Linux filesystems like ext keeps the permissions, but may require using the superuser when switching machines (as the UIDs are different). So do any of you have a creative solution for this problem?"
I don't use OSes other than Linux, so the choice is simple. If I did have to interact with Windows or OS X I'd probably use FAT32.
So do any of you have a creative solution for this problem?
Isn't the whole point of this "problem" that there shouldn't be a solution to the problem?
The higher the technology, the sharper that two-edged sword.
All my systems at home are Linux-based, ext3. NONE of my neighbors, family, or work associates have that, so it's a no-brainer.
I just use FAT32 because the main point of my USB drive is to transfer data between computers and provide a backup of my most important documents. To be perfectly honest I don't know why anyone would need permissions on a USB drive. Most programs on Linux are easy enough that with your .whatever directory in your home folder simply just copy that to your drive and paste it on the new machine. With APT and such most software is easily accessible (making portable binaries like on Windows needless). So why would you even need it?
Taxation is legalized theft, no more, no less.
Use NTFS?
For the most part, I still use FAT32 since everything can read it. Simple as that.
However, Linux has no issue reading HFS+ and my main machine is a Mac so it does the trick too.
Ever think of just making the uid's on your various machines match?
James
If you don't want things to look executable, mount it with the noexec option (which you could put in fstab). That way nothing on the device, even with FAT, will appear executable.
Since you are moving the files between computers, is the permissions loss really a problem? Aren't you just going to copy things off anyway?
If you need to limit access to certain users, you could use encrypted loopback file systems. But really, why not just use separate USB keys for different sets of permissions.
FAT is a lowest common denominator for a reason. If you want to interact with Windows, your only other real choice is NTFS, which isn't a bad option.
Sure you could use Ext3, or Reiser, or BTFS, or something else, but then you can't use your flash drive on any machine, thus defeating it's purpose.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
One of the annoying things about User ID's is that most Distros user utilities start at some number and count up. Then when you use nfs or removable media you find that the files are now owned by another user.
It would be nice if the default was to pick a random arbitrary and large UID so the chance of UID clashes would be remote.
Then, if I need to preserve Linux file settings I'll zip, tar, or cpio and store them on the stick that way.
Loading...
What? Who on earth calls it a Disk on Key?
I use tarballs. I have Macs and Linux boxes, and I occasionally need to share with windows users, so I use Fat32 as my flash drive FS. But when switching files between two of my boxes, or another Unix-like box, I use tar jcvf foo.tbz <files>, then tar jxvf foo.tbz on the other side. It works great. I suppose now that I have a 32gb flash drive, I could drop the j and avoid the slight time delay of the compression, but it's an old habit.
Until very recently, I had a 32GB USB flash card formatted with FAT32. Not that I find FAT32 particularly nice, but it was practical, as it enabled me to easily swap my stuff between my home Windows game PC, my Linux PC, my work Linux laptop and my work Windows PC. The problem was never Linux - the problem was Windows and a lack of ext3 support (I develop under Linux and need the chmod permissions, which all turns to crap when I copy it over to FAT32, which doesn't retain them)
Focus on the WAS. It WAS practical, until I was faced with the rather interesting prospect of copying an 7.5GB dual-layer DVD master image onto the stick. As we know, FAT32 has a file size limit of 6GB which causes all kinds of interesting problems.
NTFS with LUKS and FreeOTFE does the trick for me.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
This is hardly a problem unique to Linux, although as you point out Linux does have its own special requirements that may make using FAT32 a bit problematic.
My home network is a combination of Mac OS X clients and Linux servers (Debian is so easily made so Mac friendly...). I have a USB key that I don't tend to use too often (online storage has removed much of that need), but I did decide at one point that easy interoperability between OS's was important, while at the same time needing OS-specific support from time-to-time, for specific applications and data.
My solution? I formatted my key for FAT32, and then created some disk images on the key formatted them to whatever OS-specific format was suitable (HFS+, ext3, etc.). By leaving sufficient room on the main FAT32 volume, I can readily store platform-neutral data, and inside the images I can store whatever OS-specific data (such as applications) that don't need to be accessible on every system I encounter.
This does require an extra mounting steps. In OS X, it entails plugging in the key, and then double-clicking on the DMG file to mount it. In Linux, I have to mount the ext3 image using the loop pseudo-device. Of course, this is only necessary if attempting to access data in one of the OS-specific formatted images: accessing shared data merely requires mounting the key itself (generally automatically handled by the OS).
It's hardly perfect, but it does mean you can have one key that can have both shared and OS-specific data on it for as many OS's as you'd like to have at your disposal.
Yaz.
Sure it does.
http://macntfs-3g.blogspot.com/
If you don't want much hassle, just use explore2fs (http://www.chrysocome.net/explore2fs). It's an userland application which does not install any drivers or the like on Windows, and as such will execute as any user. However, you'll still need to transport it to the machines you'll want to use, and thus you'll need a tiny space of FAT32 for doing the trick.
If you have rights to install drivers on the windows machines you use, you can try the EXT2 driver available on www.fs-driver.org. It will mount your EXT2/EXT3 volume as a drive letter so you can transfer files between partitions.
--------
Fighting the herd since 1985.
I prefer to just dd my data to the raw device. If there's more than one file, I might pipe it through tar first. This process makes it much more portable and universal.
I have 3 Seagate FreeAgent 1 GB USB disks. They come with NTFS by default on them. Per disk:
1. I make a LUKS dm_crypt volume on it (for which support is well integrated into GNOME and hal in Fedora and Ubuntu.. just plug in and it pops up a dialog asking for the password).
2. I mkfs an ext3 filesystem on the encrypted volume.
I use this encrypted setup out of experience, having dropped an older 750GB USB disk from a height. It works from time to time and I have to physically destroy it because contents on it are not encrypted and otherwise anyone who finds this disk in the trash can mount and browse it.
Banu
First of all, FAT is patent encumbered and Microsoft's willing to go to court to protect it; so that's out. That includes the old UMSDOS file system Linux had at one time.
Someone needs to make a good file system that matches FAT, but is more extensible. A good choice is ext2 now... if we dropped a few things that wouldn't work nicely. Like device nodes, pipes, and Unix sockets. Like ownership, since it's assumed that the person mounting the system would own the files on it, along with groups. Simply access restrictions; they wouldn't apply.
This will simplify the structure a bit, which is a nice bonus and could let it be put on floppies. In other words, it's a light, anonymous, extended file system. LAEFS.
I got an .h file. Anyone want to help develop it as a FUSE driver?
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
0% of the time, it works every time.
DRM: Terminator crops for your mind!
I had the exact same problem a while back. My solution was a little less straightforward than some, but is still simple enough. Basically, I leverage the freeware software Ext2 IFS, which installs software onto Windows that allows it to recognize the contents of Ext2/3 partitions.
Basically, I have my disk formatted with two partitions:
On the FAT32 partition, I place the latest version of Ext2 IFS. When I access the system on my main Linux box, I just mount / use the Ext3 partition.
When I visit friends or family and I plug it into their Windows box for the first time, Windows recognizes the FAT32 partition, so I can install the Ext2 IFS software that I put onto that partition. From then on (and every subsequent access), Windows automatically mounts it!
Windows doesn't reflect the Ext3 permissions, but if you have physical, portable access to an unencrypted hard drive, those mean nothing anyway. And, of course, make sure to ask friends and family before installing filesystem drivers :)
UDF doesn't have a 2 GB file size limit like FAT32 and seems to be well supported by most operating systems. I don't really have any experience with it but I just formatted my USB stick with UDF just to see how it goes. /dev/disk/by-id/usb-LEXAR_JUMPDRIVE_ELITE
mkudffs --media-type=hd --vid=MyDiskLabel
It works fine in Linux.
You need at least one ext2 formatted thumb drive. When I move source code around I need to preserve symlinks. Can't do that with fat32 without zipping. USB drives are so cheap now you can easily have as many file systems as you want. I also have a 160gb external drive formatted as ext3 for backups - no brainer as it is never going to be read by a Windows machine and permissions and symlinks are preserved. Having two partitions is a neat solution if you only have one drive.
It was called UMSDOS, but it hadn't been maintained for years and was eventually removed from the kernel around 2004.
When I use the rsync command, none of the dates or permissions is altered. I use rsync from the command line, about once a week to back up my files onto an external USB hard drive. There is also at least one point-and-click GUI type front end for rsync, which I have not yet tried.
I used GParted to reformat my external hard drive as an EXT3 partition. GParted is an easy to use, free point-and-click GUI type front end for free Parted partitioning program. I have also used GParted to reformat several small USB keys as EXT3. GParted will can also create other types of partitions, such as FAT16, FAT32, NTFS, EXT2, EXT3, JFS, XFS, Reiserfs, and Reiser4 partitions.
If I were using the tar command to bundle my directories and files up into a tar ball or a compressed tar ball, the permissions and dates and everything would have been preserved inside the tar ball. In that case I could have stored the tar ball on a FAT32 partition, without loosing permissions or the correct dates. But instead of doing that, I have been using the rsync command to create a backup copy of all the directories and files and everything onto the external USB hard drive.
It's like the OP asks for a solution to a problem when using Linux, and then there's a grand cacophony of people denying that there is a problem! (when there is, why else was it presented?)
There are two solutions that I use:
1) Use tar to zip up the files with attributes, Then copy the .tar or .tgz file to the USB drive. Both Winzip and Archiver read tar files, so they can still be read on Win/Mac. Disadvantage? You can't easily save changes...
2) create a loopback file on the USB drive, format Ext3, then mount when it's needed. You get easy writes on Linux, but you can't read on Win/Mac w/o special drivers.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Or you could partition the disk, format the first partition (the only one Windows sees on removable media) with FAT32 and the other one with ext3/whatever when you want to keep the meta-data. The best of both worlds.
sofar wrote:
>mrcaseyj wrote:
>>
>>> C3ntaur wrote:
>>> I invite anyone who claims CO2 is not a pollutant to sit in a room full of it for 10 minutes.
>>
>> I invite anyone who claims pure water is not a pollutant to sit in a room full of it for 10 minutes.
>
> I invite anyone who claims pure oxygen is not a pollutant to sit in a room full of it for 10 minutes
I invite anyone who claims pure vacuum is not a pollutant to sit in a room full of it for 10 minutes.
You are all wrong: in all these fatal scenarios, the common element is the room. Those do-gooders in Copenhagen should be negotiating an agreement on room reduction.
-- open source? sounds like the real book --
At present you run into an issue where you could mount an ext2 or ext3 drive as a certain user, write files to it, and be unable to do anything with those files if you have a different UID on a different system.
A kernel patch has been proposed to allow you to remap ext2/3 UIDs when mounting a disk so that a standard UID can be mapped to whoever mounts the drive. This way, you'll be able to use ext2 or ext3 as your flash filesystem, preserve capitalization (another vfat weak point) and permissions (modulo the remapping) and still have decent interchange between different Linux boxes where you have different UIDs.
So there is no straightforward way to keep track of file creation times under POSIX systems? (nevermind translating between the conventions, if some obscure one is possible for POSIX, under dualboot scenario) That's a very dissapointing to me, since I find absolute file creation dates very usefull when navigating the filesystem...
One that hath name thou can not otter
Has anyone notice TrueCrypt v6.2a won't let you format MS FS (for a container) in a non-admin account? It only works with full administrator account. That doesn't makes sense!
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
I use ext2 on my USB flash thumb and ext3 on my portable HD along with http://www.fs-driver.org/ for mu WinXP work machine.
I always make sure that my user name is assigned the same UID number on both of my Linux computers, so that I avoid the permissions problem of my files supposedly belonging to a different user. In recent versions of Debian or Ubuntu, the first user created is assigned the UID number of 1000 (by default). So if Rick is the first user created on each computer there is not problem.
/etc/passwd
However, I also have a third, even older computer, which uses some older version of Slackware. By default, it assigns a UID number of something like 500 to the first user. To make sure that I had the same UID number on that computer, I had to manually assign myself the UID number that I wanted, when using useradd to create my user name. Just like you, I used this command:
useradd -u 1000 Rick
Then when I looked inside my passwd file, I could see that I had been assigned the number 1000. To check that on most Linux distros, I would have typed this:
cat
I had overlooked, how that was actually his main concern. I guess I should read more carefully, instead of speed reading.
I actually do have an external USB hard drive and several USB keys formatted as EXT3. Having the same UID number on each of my Linux computers, means that each computer knows that those are my files, and that I have my normal permissions for accessing and using those files.
Far superior to FAT32 and Ext2 is the Unix File System (UFS). Unparalleled reliability, extended from the early days, and still going strong. Also known as the Berkley Fast Filesystem (FFS).
Damn near everything supports it. Solaris/SunOS, HP-UX, Linux, BSD, MacOS, etc. And thanks to a couple of projects on SourceForge.net dedicated to writing UFS/FFS drivers for Windows, it's available there, too.
Unfortunately, there are a lot of minor variations on the format... Differing types of disklabels (partition tables), big/little-endian byte-swapping, et al. I find sticking to BSD-created UFS/FFS file systems works best.
Still, it's an incredibly solid filesystem, widely compatible, available, and just generally has everything you could want.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Look up the network effect.
Any filesystem which doesn't have universal availability is useless for this purpose. FAT works everywhere.
Deleted
HFS+ Can be read by both Linux and Mac. Preserves permisisons and such, and in SnowLeopard, Apple gently provided bootcamp drivers so that XP/Vista/7 can read HFS+ too. No hacks requiered, no big risk involved. There, solved that for ya!
*** Suerte a todos y Feliz dia!
Not sure I'd use Reiser - I hear it's murder on your USB drive.
It's easy to make fun of Reiser, the murderer, but don't forget, your laughs are at the expense of an innocent woman who was brutally murdered as well as two orphaned children.
If one cannot laugh about the bad things in life, then the world becomes a very bleak place.
Bow-ties are cool.