Slashdot Mirror
Which Filesystem Do You Use On Portable Media For Linux Systems?
An anonymous reader writes "Most people use MS filesystems on Disk-On-Keys, and portable hard drives, as these are readable from most machines. But this way you lose the files' permission information, which many times is very inconvenient (you must agree that having Ubuntu asking you whether to execute or display every text file or image you open from a DOK is annoying). Using 'regular' Linux filesystems like ext keeps the permissions, but may require using the superuser when switching machines (as the UIDs are different). So do any of you have a creative solution for this problem?"
I don't use OSes other than Linux, so the choice is simple. If I did have to interact with Windows or OS X I'd probably use FAT32.
So do any of you have a creative solution for this problem?
Isn't the whole point of this "problem" that there shouldn't be a solution to the problem?
The higher the technology, the sharper that two-edged sword.
All my systems at home are Linux-based, ext3. NONE of my neighbors, family, or work associates have that, so it's a no-brainer.
I just use FAT32 because the main point of my USB drive is to transfer data between computers and provide a backup of my most important documents. To be perfectly honest I don't know why anyone would need permissions on a USB drive. Most programs on Linux are easy enough that with your .whatever directory in your home folder simply just copy that to your drive and paste it on the new machine. With APT and such most software is easily accessible (making portable binaries like on Windows needless). So why would you even need it?
Taxation is legalized theft, no more, no less.
Use NTFS?
For the most part, I still use FAT32 since everything can read it. Simple as that.
However, Linux has no issue reading HFS+ and my main machine is a Mac so it does the trick too.
I use FAT32 almost exclusively. The universal compatibility is far more important, to me, than permission on a flash drive. With FAT32 I can easily move from various Windows flavors to various Linux and OSX flavors without a second thought.
The only time that I deviate from FAT32 is when I need to move files larger than 4GB. In those cases, I typically use NTFS because it is also more portable than EXT2 or 3.
I've not have Ubuntu ask me what to do with files from flash drives. Usually, the file type association in Gnome handles that for me. Perhaps there is an abnormality in your setup?
I like to live dangerously.
Ever think of just making the uid's on your various machines match?
James
If you don't want things to look executable, mount it with the noexec option (which you could put in fstab). That way nothing on the device, even with FAT, will appear executable.
Since you are moving the files between computers, is the permissions loss really a problem? Aren't you just going to copy things off anyway?
If you need to limit access to certain users, you could use encrypted loopback file systems. But really, why not just use separate USB keys for different sets of permissions.
FAT is a lowest common denominator for a reason. If you want to interact with Windows, your only other real choice is NTFS, which isn't a bad option.
Sure you could use Ext3, or Reiser, or BTFS, or something else, but then you can't use your flash drive on any machine, thus defeating it's purpose.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
Maybe we should have something like Rock Ridge for permissions on FAT32.
use DVD's, you'll still lose all of the security on the files, but they can be read by everything.
So your solution to a problem is to give up? How does that tend to work out for you?
I am the lawn!
One of the annoying things about User ID's is that most Distros user utilities start at some number and count up. Then when you use nfs or removable media you find that the files are now owned by another user.
It would be nice if the default was to pick a random arbitrary and large UID so the chance of UID clashes would be remote.
Then, if I need to preserve Linux file settings I'll zip, tar, or cpio and store them on the stick that way.
Loading...
What? Who on earth calls it a Disk on Key?
Your biggest problem isn't filesystem compatibility, but id compatibility. For transfer between system I don't control I use tar files onto fat32, then extract at the other end. Or zip files if I think a non-standard based system such as MS Windows might be involved. For ones I do control, they are all Linux, so ext3. And I have the same uid/gid across them (so I use the same one at home (and teh same account name) as I have been allocated at work.
When you mount media on a different Linux machine that you don't have control of, the uid numbers are different. You won't be able to access your files. Permissions won't stop anyone with root access on a machine from accessing files on the media.
I use a FAT filesystem since it works anywhere. If I want to share files with someone who has a Windows machine, it works. If I want to transfer files that I don't want to give anyone else access to, I simply don't hand out and don't lose the media.
use tar on fat32
I use tarballs. I have Macs and Linux boxes, and I occasionally need to share with windows users, so I use Fat32 as my flash drive FS. But when switching files between two of my boxes, or another Unix-like box, I use tar jcvf foo.tbz <files>, then tar jxvf foo.tbz on the other side. It works great. I suppose now that I have a 32gb flash drive, I could drop the j and avoid the slight time delay of the compression, but it's an old habit.
Until very recently, I had a 32GB USB flash card formatted with FAT32. Not that I find FAT32 particularly nice, but it was practical, as it enabled me to easily swap my stuff between my home Windows game PC, my Linux PC, my work Linux laptop and my work Windows PC. The problem was never Linux - the problem was Windows and a lack of ext3 support (I develop under Linux and need the chmod permissions, which all turns to crap when I copy it over to FAT32, which doesn't retain them)
Focus on the WAS. It WAS practical, until I was faced with the rather interesting prospect of copying an 7.5GB dual-layer DVD master image onto the stick. As we know, FAT32 has a file size limit of 6GB which causes all kinds of interesting problems.
Rosry 'bout teh lysdexia.
NTFS with LUKS and FreeOTFE does the trick for me.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
I use git or tortoisegit for my file transfer needs. There is also a samba share for my xbox to access movies and music. If that doesn't cover it, I have my flash drive somewhere with a portable copy of winscp.
My UID is prime... is yours?
FAT here; when I care about permissions, I make tarballs - which incidentally also make resolving the UID problem much easier.
Look at the fmask and dmask mount options for the fat/ntfs family.
regards,
iustin
machine 1) /dev/sdb /music/mp3/*.mp3 /dev/sdb
tar zcf
machine 2)
cd newmusicdir
tar xf
VOILA!
For flash stuff at least. Journal is just wasted there. All my boxen are nix, so I don't need fat.
Really? FAT32. Because it works on everything.
But it's possible to use ext2 and not lose cross-machine function ; there are filesystem drivers for Windows that are able to mount ext2. Of course, you have to pre-plan which machines you're going to use, and have administrator rights on them to install the driver, and it's not nearly as mature as the FAT32 driver is for both Linux and Windows.
So, FAT32. Shame about the filesize limits, but there you go.
This is hardly a problem unique to Linux, although as you point out Linux does have its own special requirements that may make using FAT32 a bit problematic.
My home network is a combination of Mac OS X clients and Linux servers (Debian is so easily made so Mac friendly...). I have a USB key that I don't tend to use too often (online storage has removed much of that need), but I did decide at one point that easy interoperability between OS's was important, while at the same time needing OS-specific support from time-to-time, for specific applications and data.
My solution? I formatted my key for FAT32, and then created some disk images on the key formatted them to whatever OS-specific format was suitable (HFS+, ext3, etc.). By leaving sufficient room on the main FAT32 volume, I can readily store platform-neutral data, and inside the images I can store whatever OS-specific data (such as applications) that don't need to be accessible on every system I encounter.
This does require an extra mounting steps. In OS X, it entails plugging in the key, and then double-clicking on the DMG file to mount it. In Linux, I have to mount the ext3 image using the loop pseudo-device. Of course, this is only necessary if attempting to access data in one of the OS-specific formatted images: accessing shared data merely requires mounting the key itself (generally automatically handled by the OS).
It's hardly perfect, but it does mean you can have one key that can have both shared and OS-specific data on it for as many OS's as you'd like to have at your disposal.
Yaz.
Sure it does.
http://macntfs-3g.blogspot.com/
If you don't want much hassle, just use explore2fs (http://www.chrysocome.net/explore2fs). It's an userland application which does not install any drivers or the like on Windows, and as such will execute as any user. However, you'll still need to transport it to the machines you'll want to use, and thus you'll need a tiny space of FAT32 for doing the trick.
If you have rights to install drivers on the windows machines you use, you can try the EXT2 driver available on www.fs-driver.org. It will mount your EXT2/EXT3 volume as a drive letter so you can transfer files between partitions.
--------
Fighting the herd since 1985.
fat32 works well enough for storing tarballs should I need to go Linux->Linux with permissions and all that. Most of the time though the UID/GID clashing isn't worth going so far as a tarball unless I need permissions preserved. I rarely transfer mixed permissions files, it's usually some form of document or media so again... fat32.
I'd use a loopback ext3 if I really needed to not use tar for some weird reason.
I prefer to just dd my data to the raw device. If there's more than one file, I might pipe it through tar first. This process makes it much more portable and universal.
I have 3 Seagate FreeAgent 1 GB USB disks. They come with NTFS by default on them. Per disk:
1. I make a LUKS dm_crypt volume on it (for which support is well integrated into GNOME and hal in Fedora and Ubuntu.. just plug in and it pops up a dialog asking for the password).
2. I mkfs an ext3 filesystem on the encrypted volume.
I use this encrypted setup out of experience, having dropped an older 750GB USB disk from a height. It works from time to time and I have to physically destroy it because contents on it are not encrypted and otherwise anyone who finds this disk in the trash can mount and browse it.
Banu
Because if it gets put in a Mac or Windows machine... that means I don't have the disk.
And, it will most likely look "empty" and get formatted.
Which is exactly what I want to happen when I lose that puppy!
(And my UID is 1000 on all my boxes)
--Phillip
Can you say BIRTH TAX
I have various usb keys and use them among my various systems - win2k, Vista, XP, Ubuntu 9.x, openSUSE 11.x - and have not had an issue with "moutning." In fact, I'd say the last time I had an issue was with SUSE 9.1. I guess the filesystem of choice is FAT32, which is older but mostly works. I have yet to been given a "do you want to open this?" on Nautilus or Konqueror. Come to think of it, my cameras memory cards (xD and SD) are all FAT32 and so are my sons' R4DS cards.
The Kai's Semi-Updated Website Thingy
First of all, FAT is patent encumbered and Microsoft's willing to go to court to protect it; so that's out. That includes the old UMSDOS file system Linux had at one time.
Someone needs to make a good file system that matches FAT, but is more extensible. A good choice is ext2 now... if we dropped a few things that wouldn't work nicely. Like device nodes, pipes, and Unix sockets. Like ownership, since it's assumed that the person mounting the system would own the files on it, along with groups. Simply access restrictions; they wouldn't apply.
This will simplify the structure a bit, which is a nice bonus and could let it be put on floppies. In other words, it's a light, anonymous, extended file system. LAEFS.
I got an .h file. Anyone want to help develop it as a FUSE driver?
--
# Canmephians for a better Linux Kernel
$Stalag99{"URL"}="http://stalag99.net";
0% of the time, it works every time.
DRM: Terminator crops for your mind!
To have the possibility to share very large multimedia files NTFS on USB HD, otherwise on usb sticks fat32.
Unfortunately it doesn't for me, as FreeOFTE requires admin access on the workstation you're using, which isn't always available.
Choosing a filesystem is the easy part, compared to finding a decent cross-platform encrypted FS solution.
For thumb drives I use FAT32 because I often stick these into devices like car stereos, BluRay players and whatnot for firmware updates and to play media. These devices know FAT and that is about all they are likely to ever support.
For external hard drives it is ext3. 95% of the time my drives are attached to Linux, and the other 5% I use an ext2/3 driver.
As usb sticks are for transferring virus & shit I only use network. That way the autorun.inf is bypassed and a lot of grief not had.
Here is what you do: XFS on the stick |->ISO |--->openVZ container running Haiku w/optional ZFS support compiled in |----->Co-Linux from Portable Apps and wine so that it can run it on Linux |----->42
Has anyone considered the 4GB maximum for file sizes on FAT32? I have Linux, OSX and Windows running on my network and run into this problem frequently. I am currently using NTFS to do the work but do not like having to mount a "dirty" filesystem if Windows doesn't unmount the drive properly.
I had the exact same problem a while back. My solution was a little less straightforward than some, but is still simple enough. Basically, I leverage the freeware software Ext2 IFS, which installs software onto Windows that allows it to recognize the contents of Ext2/3 partitions.
Basically, I have my disk formatted with two partitions:
On the FAT32 partition, I place the latest version of Ext2 IFS. When I access the system on my main Linux box, I just mount / use the Ext3 partition.
When I visit friends or family and I plug it into their Windows box for the first time, Windows recognizes the FAT32 partition, so I can install the Ext2 IFS software that I put onto that partition. From then on (and every subsequent access), Windows automatically mounts it!
Windows doesn't reflect the Ext3 permissions, but if you have physical, portable access to an unencrypted hard drive, those mean nothing anyway. And, of course, make sure to ask friends and family before installing filesystem drivers :)
That sounds like a problem with Ubuntu and not the filesystem. You might want to file a bug report.
Cory Doctorow talking about cloud computing makes as much sense as George W Bush talking about electrical engineering.
I use the driver from www.fs-driver-org on my Win2000 box, and it does seem to have one limitation. An ext2 file system created with gparted's default settings will not mount in Windows. You have to use the flag "I -128" in mkfs.ext2 during the formatting of the ext2 file system to make it Windows usable... after that, the driver works flawlessly.
Do what thou wilt shall be the whole of the Law
I have two partitions, a small one formatted as FAT32, on which I keep a copy of of Ext2 IFS for windows (www.fs-driver.org), which I find to be the best of the free ext2 drivers for windows, then I have the rest of the USB disk as a partition with an ext3 filesystem. The only thing you have to watch out for is the ext2 IFS software only works with certain inode sizes, so when you make the partition, check the defaults.
Of course if you can't install on the target windows system then you really have no choice except FAT anyway. In which case, just copy the file you need onto the FAT partition temporarily.
Si hoc legere scis nimium eruditionis habes.
UDF doesn't have a 2 GB file size limit like FAT32 and seems to be well supported by most operating systems. I don't really have any experience with it but I just formatted my USB stick with UDF just to see how it goes. /dev/disk/by-id/usb-LEXAR_JUMPDRIVE_ELITE
mkudffs --media-type=hd --vid=MyDiskLabel
It works fine in Linux.
It got depreciated in kernel 2.6.11, before USB thumb drives became ubiquitous. There is a purpose for it again.
You need at least one ext2 formatted thumb drive. When I move source code around I need to preserve symlinks. Can't do that with fat32 without zipping. USB drives are so cheap now you can easily have as many file systems as you want. I also have a 160gb external drive formatted as ext3 for backups - no brainer as it is never going to be read by a Windows machine and permissions and symlinks are preserved. Having two partitions is a neat solution if you only have one drive.
I recently updated to a 32Gb thumb drive and decided to partition it with a fat32 and a Ext3 partition. The logic is that if I want to share data it will be on the fat32 partition and things I dont want to share (private stuff) I put on Ext3. All the people were I work use Windows so if anyone gets their hands the thumb drive they cant read the Ext3 partition and what is on the Fat32 side really doesn't matter.
the ones that think disk-on-removable-key is to dorky for them.
Encrypt the whole partition.
TrueCrypt is available on the net, available for the big 3 and if you lose your drive, no worries. The pictures of yer weenie are safe.
Just boot off it. Then you can use whatever filesystem you want.
1984 was not supposed to be an instruction manual.
It was called UMSDOS, but it hadn't been maintained for years and was eventually removed from the kernel around 2004.
SFTP.
The latest Slashdot meme.
Yes, and "tar" when you need permissions. FAT32 still has the issue with screwing up the FAT (losing everything) when the storage is disconnected without proper ejection, NTFS doesn't have this problem.
At least few distros I've tried does that (and a) I wasn't bothered enough yet to search more thoroughly than quick google search b) it wouldn't help much anyway, since that's apparently a default behaviour...a big problem when using flashdisks with alien machines)
Namely, when moving files across volumes, only the modification date remained intact, with creation date being "reset" to the time of moving (essentialy it worked the same as copying and then deleting the original; and while that's what the OS is doing under the hood, it is not the intention of the user if he chooses "cut" instead of "copy")
That's a destruction of very, very basic file attribute; almost falls under the category of destroying user data... (yes, file creation dates are not important to some of you, I get it...but they are to others)
One that hath name thou can not otter
I have a 1TB external WD USB HDD, with most of the space allocated to an ext3 partition, and a tiny fat32 partition ( ~ 1GB) where I stored some small free windows programs (and instructions) that enable any windows user to see and use the larger ext3 partition. I prefer ext3 to ntfs or fat32, because ext3 is way much faster, and because I like to be consistent with things that I use. I also have a SD card (8GB) preformatted to fat32 (I think) that came with my digital camera, and there's nothing I can change about it, without risking to make it unusable with the camera.
I am unsure what is meant by "disk on key".
If you mean a simple storage area for copying files to and off-of your USB key... then:
-> YES fat32 is best.
But if you mean that you have installed linux onto your key and boot from there... then I have a lot of experience with this.
Solution: create at least 2 partitions on your bootable USB key. One filesystem should be 'ext2' and contain your root linux partition. The other should be formatted to vfat. The fat partition is then accessible to both Win and Linux. So just copy files onto that partition from whatever O/S you are currently running.
Note: always use 'ext2' on USB or SD cards as there are less writes to the disk. Since portable media reports to have a limited number of writes. Also, do not create a swap partition if you run your primary linux O/S from the portable media.
I invite anyone who claims pure water is not a pollutant to sit in a room full of it for 10 minutes.
Not that deadly dihydrogen monoxide stuff again!
When I use the rsync command, none of the dates or permissions is altered. I use rsync from the command line, about once a week to back up my files onto an external USB hard drive. There is also at least one point-and-click GUI type front end for rsync, which I have not yet tried.
I used GParted to reformat my external hard drive as an EXT3 partition. GParted is an easy to use, free point-and-click GUI type front end for free Parted partitioning program. I have also used GParted to reformat several small USB keys as EXT3. GParted will can also create other types of partitions, such as FAT16, FAT32, NTFS, EXT2, EXT3, JFS, XFS, Reiserfs, and Reiser4 partitions.
If I were using the tar command to bundle my directories and files up into a tar ball or a compressed tar ball, the permissions and dates and everything would have been preserved inside the tar ball. In that case I could have stored the tar ball on a FAT32 partition, without loosing permissions or the correct dates. But instead of doing that, I have been using the rsync command to create a backup copy of all the directories and files and everything onto the external USB hard drive.
I favor exFAT, MS's new format for removable storage. If there were Linux/BSD support, I would jump for joy. Alas, patent-encumbered, by a known patent troll, MS. I will settle for an open source equivalent to exFAT, or at least real Windows drivers for ext2/3/4/etc. One of you smart people want to step up?
I have an external HDD I use mainly for backups, but also occasionally for data transfer between computers. I had the same issue - I wanted the backups (which were basically just cp's of my Linux data partition) to retain permissions and ownership so that if I had to copy my entire backup back to my computer, everything would be retained, but at the same time I wanted to use the drive elsewhere on non-Linux machines.
I used gparted to create two primary partitions - an EXT3 partition, and an NTFS partition. Linux backups would be sent to the EXT3 partition, and anything I wanted to share or transfer between machines would go on the NTFS partition. If I knew I wanted to share content already on the EXT3 partition, I'd just copy it to the NTFS as well. Seems to work.
It's like the OP asks for a solution to a problem when using Linux, and then there's a grand cacophony of people denying that there is a problem! (when there is, why else was it presented?)
There are two solutions that I use:
1) Use tar to zip up the files with attributes, Then copy the .tar or .tgz file to the USB drive. Both Winzip and Archiver read tar files, so they can still be read on Win/Mac. Disadvantage? You can't easily save changes...
2) create a loopback file on the USB drive, format Ext3, then mount when it's needed. You get easy writes on Linux, but you can't read on Win/Mac w/o special drivers.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Or you could partition the disk, format the first partition (the only one Windows sees on removable media) with FAT32 and the other one with ext3/whatever when you want to keep the meta-data. The best of both worlds.
Tarballs solve the problem nicely. Even on a linux-formatted drive, file attributes (especially timestamps) can be lost.
Using 'regular' Linux filesystems like ext keeps the permissions, but may require using the superuser when switching machines (as the UIDs are different).
It sounds to me that the Question is not what filesystem to (ab)use, but how to make the files on the USB technology appear to belong to the same (ab)user on both/all boxen. Simply give yourself the same UID on both/all boxen and you are done. /home/youruser
usermod -u 1000 youruser
chown -R youruser:users
man usermod and man chown for additional information.
9/11: Never forget it was a false-flag operation
There is no 'creation time' attribute in Linux. Are you perhaps looking at 'ctime' and thinking that stands for 'create'?
stat filename will show you that it actually stands for 'change'.
split is actually GNU, part of the GNU coreutils. It should also be mentioned that the manual document indicates Torbjorn Granlund and Richard M. Stallman involvement with making the split technology.
9/11: Never forget it was a false-flag operation
USB drive FS is a major problem working with big video files between Mac, Windows and Linux systems because fat32 can't handle large file sizes. I found that there are open source drivers that will let you mount ext2 in both mac and windows. They are buggy, not-ideal drivers, but at least it's something. I'd do anything to avoid using NTFS (dun dun duuuun).
fat32 to boot portable vbox then whatever filesystem you want on the vm disk
sofar wrote:
>mrcaseyj wrote:
>>
>>> C3ntaur wrote:
>>> I invite anyone who claims CO2 is not a pollutant to sit in a room full of it for 10 minutes.
>>
>> I invite anyone who claims pure water is not a pollutant to sit in a room full of it for 10 minutes.
>
> I invite anyone who claims pure oxygen is not a pollutant to sit in a room full of it for 10 minutes
I invite anyone who claims pure vacuum is not a pollutant to sit in a room full of it for 10 minutes.
You are all wrong: in all these fatal scenarios, the common element is the room. Those do-gooders in Copenhagen should be negotiating an agreement on room reduction.
-- open source? sounds like the real book --
Using Windows terminology to describe it: there are two file attributes that interest me, "Created" and "Modified". Under any version of Windows, when moving a file, they remain intact (when copying we have of course new "Created"). That is, I believe, the correct behaviour - if you want to move a file, you want to retain as much info about it as possible.
Not sure what is the convention under OSX, haven't used it enough, but when using Linux GUI enviroments there are two corresponding attributes, which seem to be the same thing (from the name...). AFAI remember what seems to be the "Created" attribute indeed acts like 'ctime' that you describe. Perhaps I remember incorrectly.
But here's the thing: when I'm back to Windows, "Created" attribute has definatelly changed (to the date of moving operation). Irritating, to say the least...
One that hath name thou can not otter
However, Linux has no issue reading HFS+ and my main machine is a Mac so it does the trick too.
"No issue reading" is Linuxese for "For write support, you must format specifically as Mac OS Extended, Case Sensitive, No Journal, and one of Mac|Linux formats in a way that the other can't read at all, have fun figuring that out."
No joke.
I just format my external disks as ext3, reserving a small partition (maybe 2-3GB out of 1TB) as a FAT32 partition.
Throw the installers for Ext2FSD, an "Open source ext2/ext3 file system driver for Windows (NT/2K/XP/VISTA, X86/AMD64)" on the FAT32 partition. This allows you to bootstrap any Windows machines you come across to access the EXT partition(s).
There are some instructions for ext3 access on OSX on various sites online, but I haven't tried any out myself.
It seems like the best solution would be to have external drive enclosures be able to act like disks or like little servers. If you want them to just be a disk, then they can have direct access, but if you're trying to access an ext3 disk on a Vista or OSX machine that doesn't have support for that file system, the hardware could just run a little ftp or sftp server and the host machine could just access it over ip.
coding is life
Since I don't want to propagate any Microsoft filesystem, I format all of my USB devices with ext2. If anyone on Windows wants anything I have on those devices, they either install one of the Windows ext2 drivers, or I create a temporary Samba share for them use to get at the data over the LAN.
At present you run into an issue where you could mount an ext2 or ext3 drive as a certain user, write files to it, and be unable to do anything with those files if you have a different UID on a different system.
A kernel patch has been proposed to allow you to remap ext2/3 UIDs when mounting a disk so that a standard UID can be mapped to whoever mounts the drive. This way, you'll be able to use ext2 or ext3 as your flash filesystem, preserve capitalization (another vfat weak point) and permissions (modulo the remapping) and still have decent interchange between different Linux boxes where you have different UIDs.
Ah, so there are a couple of scenarios here.
1) Unix files created on a Unix system and moved between Unix volumes. In this case, there's no issue. Unix doesn't track creation time, so that particular metadata isn't lost during the move. The ctime represents the inode change time, so it's correct to change it when you change the destination inode.
2) Windows files created on a Windows system and moved between Unix volumes. This was actually a point of contention in the past. POSIX doesn't have a notion of a creation time, so when writing drivers or compatibility layers (e.g. Cygwin), there's no clear way to deal with creation time attributes (that the syscalls don't have any idea how to deal with--or even that they exist.) It sounds like once you put the file onto a Unix filesystem, you lose that metadata. It's not unlike transferring a Mac OS file with resource forks to Windows or Unix. I assume something similar happens with NTFS alternate streams.
Real men don't need a file system to write data to a usb thumb drive.
/dev/sdb /stuff
tar cf
Ask Slashdot: Where bad ideas meet poor googling skills.
So there is no straightforward way to keep track of file creation times under POSIX systems? (nevermind translating between the conventions, if some obscure one is possible for POSIX, under dualboot scenario) That's a very dissapointing to me, since I find absolute file creation dates very usefull when navigating the filesystem...
One that hath name thou can not otter
Has anyone notice TrueCrypt v6.2a won't let you format MS FS (for a container) in a non-admin account? It only works with full administrator account. That doesn't makes sense!
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
But if I need high performance on my portable drive then I use NTFS-3g on my OS X machine (which I think also supports linux too). But normally I just use fat32 since it's support by almost every modern OS out of the box.
You're thinking too hard. Look at archive formats like tar, not filesystems. The format originated on unix so unix permissions are in. Most decompressors work with it to some degree, often in-line as you browse the storage device with a GUI.
that will make if fairly portable
then you need the space to recombine the split files. face it, fat is archaic. use ntfs if your priority is windows support, or use either ext3 or reiser and you can still share with windows. there are programs that will run under windows and allow you to access and copy files from these linux file systems without installing a driver. for longer term use install a filesystem driver for windows and get transparent access..
hey i wonder what running reiserfs as xp's primary partition would be like???
I use ext2 on my USB flash thumb and ext3 on my portable HD along with http://www.fs-driver.org/ for mu WinXP work machine.
I'd either use a centralized auth solution that would ensure my UIDs are consistent accross systems or use a windows-compatible filesystem and create a zero'ed file of a few gigabytes, create an ext3/reiserfs filesystem on it and mount /that/ via loopback when I want to keep permissions around. For OS X i'd use a dmg, same idea, with the added bonus of natively supporting AES encryption and allowing HFS+ resource forks to exist.
My 16gb USB flash drive is formatted as NTFS because I need to accomodate large files (>4gb), but I do have a TrueCrypt volume on it with an ext3 filesystem on it which amounts to the same. Lack of native NTFS write support on OS X is annoying though...
For preserving permissions, isn't tar ideal?
Reiser is a write slow read fast file system.
Ballanced trees organize the data. This is really what you want on flash media.
The tail packing will compact the data into the filesystem so it's efficient.
You gain more space available than you'd otherwise get from the other offerings.
There is a driver which fits within the Windows file-system driver model.
Not sure that it works.
I've used Reiser for at least eight years. No major issue were experienced during this time.
Dataloss and corruption is something I've only read about. I've never actually experienced it
with this filesystem.
Ext3 randomly lost it's superblock on my external drive a few days ago and I was unable to recover any of the backup blocks. There was an error message about being unable to set a superblock. I researched this extensively on google over vast numbers of forums ( 5 ). Only I found there was no answer other than a suggestion to backup the data if it's valuable.
I'd never recommend Ext3 for anything. Hans Reiser was correct in freezing developement on Reiserfs3.
I think the only modifications that were made to Reiser3 were relating to SELINUX sometime back. I am a little skeptical of this but I've had no real stablilty issues. I don't use SELINUX.
If you can get by with it use Reiser but if you need exchablility then go with Fat32.
I'm not sure how to deal with user-id's.
>"Which Filesystem Do You Use On Portable Media For Linux Systems? [...] you must agree that having Ubuntu asking you whether to execute or display every text file or image you open from a DOK is annoying"
Linux (does not equal) Ubuntu.
That said, I typically use a mixture of FAT32, Reiserfs, and EXT3 on removable media.
Far superior to FAT32 and Ext2 is the Unix File System (UFS). Unparalleled reliability, extended from the early days, and still going strong. Also known as the Berkley Fast Filesystem (FFS).
Damn near everything supports it. Solaris/SunOS, HP-UX, Linux, BSD, MacOS, etc. And thanks to a couple of projects on SourceForge.net dedicated to writing UFS/FFS drivers for Windows, it's available there, too.
Unfortunately, there are a lot of minor variations on the format... Differing types of disklabels (partition tables), big/little-endian byte-swapping, et al. I find sticking to BSD-created UFS/FFS file systems works best.
Still, it's an incredibly solid filesystem, widely compatible, available, and just generally has everything you could want.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Since I work on Mac, play on Windows and have some devices like a dvd/divx player and a ps3 for play/video, I am restricted mostly to Fat32 for R/W on OSX and Windows.
I ended up carrying a 160G drive 80/80 gigs for Mac OS Journaled and Fat32......
In fact I had a dilemma with Firewire and USB, so I got aMacally enclosure that has both.
Oh yes, like that I sometimes end up copying stuff from partition to partition.
At least the PS3 can stream videos from my Mac share, so I am covered with that one, and unless it is something huge I just shove it over my gigabit hub (Time Capsule)
There's a straightforward way - put the creation date in the filename. Not amazingly useful, but it will work.
The thing that most people forget is that using Gparted is not really as useful as using mkfs. Reason is that the resulting partitions are given a fixed percentage for root space. If you have a storage device only, you may want to lower that percentage, which you can specify with mkfs -m. I have not seen Gparted give me that option, and on large drives you will recover many gigs of space.
Sure, FAT32 is a bitch. And MS are trying to get money for pre-formatted media. However, it's good enough transferring files from one system to another.
If you want to preserve protections and ownership, just bundle everything into TAR files. With respect to ownership, ask yourself if you want to keep UIDs and GIDs synchronized through all the systems you use, whether you want to map these or whether you let the current user own all files. (Hint, use the latter.)
Also, using archives like TAR and ZIP reduces the amount of files on your FAT32 partition. FAT32 is not knows for excelling in supporting of many files.
I hadn't the slightest objection to his spending his time planning massacres for the bourgeoisie... (P.G. Wodehouse)
Roll your files up into a tar or cpio archive, then save that file onto whatever filesystem you like.
tar cjvpf [archive file] [paths...]
man tar
No, your USB drive is okay with Reiser, it's your wife you have to worry about.[please keep this one in strict context to avoid confusion and extreme FUD] /.
Although, if your USB drive is your wife, then:
a. you belong here on
b. you have a problem finding your way out of that basement/dungeon you entered in 198x
c. you are comfortable with your wife plugging into you...[if you have a USB hub, then multiply a,b,and c by at least 100 for each USB port on your hub]...skipping the "In Soviet Russia..." memes
d. all of the above
e. I'm a Morman, you insensitive clod!
Let's just skip the Beowolf Cluster meme here to avoid the extreme mess and pain that would ensue!
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
I still use my old bookshelves, where I order everything alphabetically...
Like a normal person.
I use USB keys to exchange data with other people, so using FAT32 can't be helped. I use a USB hard disk for my backups, so ext3 is best (it's the file system I use on my PC). I attached that HD once to another Linux pc where I had a different uid and used sudo to be sure to copy all files. If I had to do this routinely I guess I'd make sure to save files with read permissions for all users. Note that using FAT32 won't preserve permissions (most important: exec).
Look up the network effect.
Any filesystem which doesn't have universal availability is useless for this purpose. FAT works everywhere.
Deleted
What you want seems to be like the "all_squash" option on nfs exports (with the anonuid and anongid options). You can use these with a loopback nfs mount directly.
It's possible to do something similar with some fuse filesystems too. (Perhaps encfs?)
Personally, I don't think it's work the effort and use vfat with the "fmask" option set to disable the execute bits.
HFS+ Can be read by both Linux and Mac. Preserves permisisons and such, and in SnowLeopard, Apple gently provided bootcamp drivers so that XP/Vista/7 can read HFS+ too. No hacks requiered, no big risk involved. There, solved that for ya!
*** Suerte a todos y Feliz dia!
there is a comment above that advocates using UDF rewritably, and its a fantastic idea. one of the reasons for getting large flash drives is to transfer large files, except thats impossible with the default fat provided. strangely its not possible to format flash as ntfs under windows, ive tried every commandline option available and it still refused. but once formatted ntfs with linux, windows will happily mount and read/write to it..
i understand that there may have been some issue with the mft being constantly rewritten and this not being compatible with the wear leveling that flash drives implement but ive never had a problem using ntfs on flash.. it could be so that the manufacturers dont get complaints about it not working on a mac, but im not sure.
Cut n paste should not be implemented on files in any situation. It's against the defined behaivour of cut and paste.
This behaivure should rather be a move command and not used by cut & paste. It's redicilous.
Apple is still holding firm grip of how thing shold be implemented, so there is no cut and paste of files In Finder that is a GOOD Thing!
Bad thing there is no move command either :(
Just my 2 on this issue that makes me puke each time i move a file in Windows.
Well, I'm a perfectionist due to my lightweight asperger syndrome. Could have more of that and less ADHD.
I was wondering the same, and wish you had been answered by someone not trying to be a comedic assclown.
I would think this possible, but not have went past installing Firefox Portable on my U3 Sandisk 4GB USB drive, and using that at work.(WinXP/IE7 environment)
I would love to be able to go in to work, plug my USB drive into the PC, and have U3 load Kubuntu 9.04 with my current pref's and config's from home...even if I had to reboot Windows.
Instead of 'RTFM-n00bs' flames, give up some FOSS love and goodness for your future generations, Peace!
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
If you use ext3 and set global permissions (for all users, not only for your user or users in your group, like a+r, a+w, a+x), then you should be fine with using it on other machines, as UIDs don't matter anymore.
Don't use Linux, problem solved.
I would take a fat32 or ntfs system an tar the files i need with permission. So its mountable in every distro and i have the full permissions.. Its as simple as that ;-)
...they're too fucking lazy to do anything more than pop in a CD and have it "just work". It's as if typing "apt-get" and spending a few minutes learning a "new" UI is like walking barefoot across broken glass.
The application or driver that I want is not in any repository presented to me. Under Windows, I could stick in the CD and have it just work, but under Ubuntu, searching the repository with Synaptic turns up nothing.
Store it in the metadata like EXIF for photos.
- Michael T. Babcock (Yes, I blog)
Doh...
Though it wouldn't allow for quick sorting of files according to creation date (or, if I'd put the date at the beginning of filename, would dissalow sorting by "true" filename)
One that hath name thou can not otter
this is called exFAT
http://en.wikipedia.org/wiki/Exfat
What about UMSDOS? ZipSlack uses it to add Linux FS features to FAT.
I just click the stupid 'display it' button when opening text files. It's not really that big of a deal for me, and I like having my USB key be readable by anything I stick it in.
PERL:
All of the power of Voodoo with most of the understandibility!
I use FAT32 on my portable drive. It works on Windows systems, Linux systems, Mac systems, and about everything else out there I'd need to connect to because you never know what you'll need to interact with. I was actually thinking about this on the way to work this morning and how much I hate FAT32's file size limitations, but it seems to be the most portable option out there and you can always split large files up--it's a pain but it works.
what i'm doing is create 1 small FAT partition contain windows ext3 driver from http://www.ext2fsd.com/ . and format the rest using ext3 file system.
The N800 has 3 different memory/disk locations.
1- Internal (built-in) which is ext2/3, I guess. That is where applications are placed.
2- Internal, but removable 8GB SDHC. I use this for GPS Maps and email data since I don't really do much with it. FAT32 today, but I'll probably convert it to ext2/3 to expand the number of programs available.
3- External 8GB SDHC - I use this for transferring files between systems. Mainly it holds encrypted data and mp3 files like podcasts, audio books and language training. It is FAT32.
FAT32 for flash drives, NTFS for hard drives, TAR files as a container if permissions are an issue.
"When information is power, privacy is freedom" - Jah-Wren Ryel
exFAT sounds promising, but real question - is it protected by patents? Currently, AFAIK there is proof of concept Linux kernel implemenetation that can read exFAT: http://www.phoronix.com/scan.php?page=news_item&px=NzAzMg Patches: http://www.gossamer-threads.com/lists/linux/kernel/1026144
Vlatko Kosturjak - Kost
I guess it all depends on who you are and what you're doing. My experience is that I still haven't found a situation where it's inconvenient. There's something about removable media, that makes file ownership and permissions irrelevant. You pretty much have to assume the worst case anyway, and if you are using owners, groups, permissions etc to enforce anything, you have big problems anyway.
We all know physical access overrules permissions on disk, so we give up and say, "Yeah, if someone takes the hard drive out and puts it in another machine, there's little we can do (other than encrypt), so we're just not going to deal with that. On the bright side, no one is likely to actually do that, much less without being detected." Ah, but with removable media, that last sentence simply doesn't apply.
So what do you do about it, in the rare case (IMHO, obviously your situation is different or you wouldn't have asked) that you still want owners and permissions anyway? What fs should you use? Mu. Use a container. Tar is your filesystem. And best of all, if you have security considerations, check this out: you can trivially run your read/write pipeline through gpg or something, without having to worry about setting up encrypted filesystems or block layers.
"Believe me!" -- Donald Trump
It depends, On jump drives which gets used every now and then, at clients, at office, sometimes at friends places FAT32, Can never be sure what OS the other person has and jump drives attached on to the keychain are used at a variety of places. So the argument that I only use Linux does not make sense as I use Linux but not everybody else does. For External Drives that I run my VMs off and I store a lot of media on I use ext3, of course because these disks only get used by me and Yeah to go to the old quote I only use Linux to access tohese media I do use the ext3 formatted drives on my wife's macbook as well.
On a related note, what is a good file system for use in embedded environments where you want 1) Fast booting without file system checking 2) Fast shutdown simply by disconnecting power and 3) Removable media (SD card or USB dongle)?
and make a nice table: computer1, computer2, .... all the machines you usually use with your removable media
Write under each of them the file systems it knows how to read. You'll be happy to see that fat32 is the only option. No matter how much you discuss it, unless you want to take the road explained by someone before me and partition your drives in fat32/ext2 and use something like ext2ifs which is fine (i use it myself) excepting some (rare) bluescreens. Oh and to that commenter I would be very *happy* if everybody visiting me would install some crap so that he can use his usb drive :)
Add to that that most stand-alone media players (car radios with usb, dvd players, etc) usually only know how to read from fat32 disks the decision will be simple. Need bigger files ? Split them !
... but a warning for an issue I've encountered in the past.
:-)
USB sticks, as far as Linux is concerned, are just another block device. As such, you can partition them. I did this in the past, making both a FAT partition for my Windows stuff, and an ext3 partition for my Linux stuff - SSH is picky about keyfile permissions.
I happened to put the ext3 partition first, and the FAT one second. Worked a charm, until I booted windows. Windows correctly identified the secondary partition as being FAT and stuck a drive letter on it, but the files I had dumped on there in Linux weren't visible. I assumed I made a mistake somewhere, dumped some more files on it, and rebooted back into Linux.
I found the original files on the FAT partition, so I hadn't made a mistake, but the new files weren't there. When I tried to access the ext3 partition, however, I found out where the new files went: apparently Windows identified a FAT partition on the key, but apparently then assumes it is in partition 1. After all, who would partition portable storage, right ? My ext3 partition was suitably fucked, of course.
I tried this several times over, and behaviour was consistent. I also tried it the other way round, FAT as primary and ext3 as secondary, and that worked fine.
This was back on Windows 2000, I've never tried it on later versions as I now just use the ext2 IFS driver, but you've been warned
What a depressingly stupid machine.
Huh, it seem that they mention exFAT in patents: http://v3.espacenet.com/textdoc?DB=EPODOC&IDX=US2009164440
Vlatko Kosturjak - Kost
Not sure I'd use Reiser - I hear it's murder on your USB drive.
It's easy to make fun of Reiser, the murderer, but don't forget, your laughs are at the expense of an innocent woman who was brutally murdered as well as two orphaned children.
If one cannot laugh about the bad things in life, then the world becomes a very bleak place.
Bow-ties are cool.
What about reviving umsdos? At one time, Linux's "umsdos" filesystem was FAT plus extra "hidden" information that let you store file permissions, uid, etc. If I remember correctly, one annoyance was what to do about differing UID's. But this wouldn't be that hard to handle; a mount option could be added that made it appear that the user-mounter owned all the files (which is not bad model for a USB stick). That way, you could easily store and retrieve files with the correct execute permissions (no, I do NOT need to execute JPEGs), note which files are read-only, and so on.
:-).
Or, if reviving umsdos is too much, maybe there'd be a way to modify Linux so that its FATS32 implementation is "a little more useful". When exchanging files on a stick, I find that you often don't need exotic permission capabilities. The uid is often unnecessary, since it's a different computer anyway. Just noting which files are executable and which are writable by the owner is all that most people need on a stick. But NOT having those is incredibly annoying; it's stupid that JPEGs come out executable. FAT32 already has a "read only" bit for file attributes; basically, interpret that as the inverse of a write bit (make everything writeable unless it's read-only). For "executable", maybe you could use an "unused" file attribute bit or two (00=unknown/use some other method, 01=executable, 10=not executable, 11=???). An implementation that knew about the new bits would use them, otherwise, situation normal. Byte 0xc has 6 unused bytes, and we only need 2 for this scheme. You could steal a few more bits and at least implement "user" and "other" (and make "group=other"), or even implement all 9 traditional bits. I think most people could live without the sticky bit, or setuid, on FAT32
NTFS and "Universal Disk Format" may be the best you can do for now if you want to exchange files with permission info. But there are still many systems which can't read them well, so FAT32 ends up being the "easy" road. If you "zip" up the files first, almost everyone can read them (tar works, but most Windows users can't read tar files... they're more likely to be able to read zip files).
- David A. Wheeler (see my Secure Programming HOWTO)
OK, this is kinda backwards and probably something of a joke, but I'd boot the other computer off of a PendriveLinux (I like the LinuxMint variety). Then I mount the computer's disks and copy files to/from them.
http://www.pendrivelinux.com/
Another wonky way to do this without rebooting into Linux is to have a small FAT partition with qemu or some other portable x86 emulator and boot the pendriveLinux in a virtual machine. Then use samba to transfer files from the USB filesystem to the host machine over the virtual network.
Hey, you did ask for creativity :P
FWIW, I actually have done both of these things before using KNOPPIX LiveCDs... just recently got around to making USBdrive versions of them now that large USB drives are a commodity. I'd never really gotten into USB pendrives since I mostly tend to move files over the network or would just access my computer remotely over VNC or ssh to get stuff done.
The best solution was UMSDOS, which was removed in Linux 2.6.11.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
What if you want to work, without a hassle I might add, on Linux, Windows and Mac. I think it might be down to FAT32 and NTFS then, but what if I want to have bigger files and not bother to split them? Is the NTFS only option then and Mac needs some tinkering before it can even write on to it?
To all the children modding me down this thread:
WTF is wrong with you? I'm not offtopic (the topic of the joke was murder, I couldn't be more on topic), nor further down this thread, flamebait.
Seriously, someone died, two children lost their mother (and father). Write a joke if you want, I never said not to, but don't rag on me for pointing out a real-life tragedy is what you're all laughing at.
I realize it only takes a handful of morons to swing moderation around one way or the other, but ffs. Way to promote Linux, and fight the image that Linux is for arrested development, socially awkward little boys burrowed deep into their mother's basement. "Oh no! He pointed out something that a grown up and mature person might think on a joke about murder. Burn him!"
Foolish kids.
Use fat32 and then when transferring files between Unix boxes use tar. Thus put them as one archive. And for simple other files - doc, so on, no normal distribution will try to execute them.
... led police to her body after he was convicted. The guy was guilty as sin.
That'd be an awesome formatting option when installing Windows 7.
Never have felt I needed anything special. If I need to touch the metadata on a USB Flash drive, I use a script to 'make it so'. Every one I have is FAT32 because on occasion I do have to share files with a Windows user.
There is nothing wrong with yr Internet. Do not attempt to adjust the picture. We are controlling the transmission - NSA
The few times I use such media, I use ext3. I don't need windows compatibility. But why use removable media? scp directly from device to device is so much easier.
Pidgeon shit
What about ext3 formatted with a small fat32 partition that contains Ext2 IFS for windows. Now it's portable, can be read in linux natively, and if you plug it into a windows machine, you just install the ext2 IFS and then mount your ext3 filesystem.
I have an external HD with 3 partitions: 1 main Ext3 partition, 1 Ext3 encrypted with LUKS for private stuff like pr0n and photos, 1 small FAT32 for when I need to copy stuff from Windows.
I also put the Ext3 Windows driver (http://www.fs-driver.org) on the FAT32 partition. That way I can still access the files on the main partition on Windows machines.
Simple and works ok.
I use FAT32 on my USB dick. But if I needed to keep permissions, etc. I'd use NTFS, which is the best file system in the world.
You can change it afterwards with tune2fs (for extfs family filesystems).
I would very much hope you don't want executable permissions to apply automatically to a mounted usb drive. The security implementations of this are pretty obvious I would think. The UID differences is a pinnacle feature of ext and this is exactly why its common practice to format removable drives as FAT. Although it was originally developed by Gates and could be considered legacy it is currently the most viable solution for operating system compatibility. I would agree a newer universal removable drive file system would be great but good luck getting MS to support it. If you want security with FAT use encryption.
If it ain't broke, don't fix it.