Slashdot Mirror
Fedora Aims To Simplify Linux Filesystem
jfruhlinger writes "Even Linux's most passionate partisans will admit that its filesystem, which stashes vital files in a variety of arcane directories, can be baffling to users. The developers at the Fedora project want to cut the Gordian knot and consolidate all executables into /usr/bin and all libraries into /usr/lib or /usr/lib64. One downside: this system would conflict with the standards developed by the Linux Standard Base, or the (rarely used) Filesystem Hierarchy Standard."
Make the directories you want compressed into mountpoints on the compressed partition.
Just store everything in /
What could be simpler?
The developers at Fedora can do whatever the heck they like. Pat knows what he's doing, and that's good enough for me.
Stick Men
This isn't 1995. Nobody cares about filesystem level compression anymore. Go buy a 2T drive.
The way I understand it, there's really no good generic way to handle file compression at the FS level.
Even the way NTFS does it is to create a compressed file to hold the contents of the original file, like an archive. But if you'll notice, whenever you open the compressed file, NTFS will expand the whole compressed data into another special file until you close it. Watch the disk space usage change and you'll see. You can easily set up a situation where you do not have enough free disk space to open a compressed file, and that is not intuitive for a user.
What you want is an ability to compress small individual blocks of the file that can be accessed separately without having to decompress the entire file. But doing that creates all sorts of other problems such as how to efficiently allocate the space for the compressed contents which might change over time (and change size) without causing a great deal of additional fragmentation. This is rather harder to do.
They don't need to. Linux has the ability to read/write compressed files directly (zclib?) and doesn't need the filesystems to support this. Which is great because it means compressed files will work under ALL filesystems ALL of the time (if you have the library installed) and you don't have to wait for each filesystem maintainer to add it. You also have no risks of one FS maintainer deciding another's implementation sucks and not being compatible with it. Which is very likely under Windows.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
I think it's important to realize why the four directions /bin, /sbin, /usr/bin, /usr/sbin exist (and similarly why /lib is separate from /usr/lib).
The reason is that once upon a time discs were small, so that /usr would be mounted separately from the root partition. So /bin and /lib are small directories containing as much of the operating system as you need to get going before you mount /usr and get everything else. In particular, this means the utilities needed to mount those other filesystems and to fix errors in them (e.g. fsck).
The separation between /usr/bin and /usr/sbin means that ordinary users don't have system programs (those from /sbin) in their search path.
Today most installations have the whole system (/ and /usr) on the same partition and it seems that many users use a GUI rather than a terminal. This means that the separation is not needed.
Note that this change is not about multiple-architecture situations like /usr/lib and /usr/lib64. It's about the separation between /lib and /usr/lib (or /lib64 and /usr/lib64).
Uhh, I want my stuff separate from system stuff. I want to be able to back-up my stuff without including standard stuff.
System essentials should be in /bin. Non-essentials in /usr/bin, and my stuff in /usr/local/bin.
Fedora has jumped the shark.
I agree. I suspect the libraries are the more 'needed' change, but your point is valid even there.
What I'd like to know is what's wrong with the FreeBSD file system, and why don't they just 'do that'? IIRC, everything non-standard is in /usr/local. Some configs are in /etc, but most (all?) non-base configs live in /usr/local/etc. If you blow your system away and have backups of /usr/local and /etc under FreeBSD then you can just reinstall the base system and be 'fine' (aside from the local user files, but that's an obvious restore/backup situation.)
This smells a lot like "we want to do things our own way" (tm). I suppose that's fine, but don't act like you're doing humanity a service by wiping your butt with a different hand ;)
PS: I don't reply to ACs.
As a kindly reader, I'd even put in a note for the editors whilst it was in the firehose queue that this should be "Knot". Not really a major typo and the editors can't be blamed for not reading all the comments, but it'd be good if more people DID pre-screen the submissions and enter USEFUL corrections so that the quality can be improved.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Actually there are valid reasons to compress at least some of your files beyond the need for saving space, namely that the speed differential between I/O(esp. platter drives) and CPUs is continuing to grow incredibly fast. The performance gap is getting to the point that for files that tend to compress(executables and libraries are among them), the time it takes to read the compressed file off the disk and then decompress it in memory may be less than the time it takes to read the entire uncompressed file from disk.
Now there are tools that allow you to do this just for executables, but since they don't run on all platforms you can be in kind of a bind. By putting your executables and libraries in their own file compressed file system, you can gain a lot of the advantages of executable compression while still being able to use it on pretty much any platform.
Monstar L
Add to that, in network attached storage solutions, every file you read is squeezed through something as small as a 1GBbps pipe.
Someone had to do it.
Sounds like you have a problem with file formats, not filesystems. Filesystem level compression is a stupid idea since it doesn't have any way to apply appropriate compression methods to the files. Should I apply zlib to uncompressed audio? No, use FLAC. Should I apply zlib to logs files? No, I should probably use something like LZO or Snappy that have block seeking.
Although this proposal sounds reasonable at first, actually implementing it is troublesome. Linux systems have an expectation that the root directory / and the /usr directory may be on different filesystems; thus /bin is expected to come with / and be available at boot time, where /usr may not be. This means that making /bin -> /usr/bin via a softlink would break that.
Although the article summary claims that the Filesystem Hirearchy Standard (FHS) isn't used very often, some distributions such as Debian actually do try their best to follow it and even have it as one of the specs for how to build packages. Debian developers discussed the idea of trying to follow Fedora in this proposal, but it looks like it's too troublesome to be worth it. For one thing, all of the filesystem recovery tools or anything else that would be required in an emergency at the command line would need to be built into the kernel initrd images, which could be done but which doesn't seem terribly reasonable.
As such I think most Linux distributions are going to need to wait and see how well it works out for Fedora on this effort.
wrong wrong wrong wrong wrong wrong wrong wrong just tried it. NTFS FS, 2GB free, file on there 30GB full size, compressed to 15GB. opens juts fine even though there isn't 30GB free there. also opens instantly (does NOT decompress 30GB anywhere)
-------
1. Enjoy your job
2. Make lots of money
3. Work within the law
Choose any two.
http://www.gobolinux.org/ Combines executables and necessary libraries each in their own directory under '/programs' Uses links to show files in traditional places.
It is well-specified. There's a folder for executables, a folder for binaries, a folder for configuration data, a folder for temporary stuff. And its layout hasn't changed for 20 years.
Compare it to Windows, where the file system layout changes from one Windows version to the other, there are no documents specifying most of its organization, and it doesn't matter anyway, because since Windows NT the file system is meant to be only managed by automated installation tools, and even an expert user can not hope to fix it when things go wrong.
What's wrong with /bin and /lib ? They serve a specific purpose, and the files they contain shouldn't be directly handled by a user who gets confused because of the presence of more than a single directory in his $PATH, so who will gain from their "semplification"? Don't tell me the real reason is that Fedora's next-generation self-aware omniscient init system has grown so complex that they're no longer able to support a split /usr installation because of its dependency hell.
Please do not turn Linux into an unmanageable mess as the one Windows has become.
End of rant
The one thing that baffles my mind is that Linux filesystems still don't offer compression of specific folders or files.
If you need to do this why not try a compressed, gzip'd or bzip2 tar, rar or zip file. You can even use your a graphical explorer to actually create and manage your archives (yes you can have more than one). When I say manage you can easily (ie. point and click) display, extract and even insert specific files.
Windows has had this for over a decade.
Well you could do this in Unix for over 20 years.
Why doesn't Linux have such a simple but important filesystem feature? And no, I don't want to make an archive file, because I want to access those files and folders while they are compressed.
Linux does, I think I already explained this above.
There ain't no such thing as proprietary standards only proprietary formats. Standards are by definition open.
tl; dr
try compressing your comment for greater readability
this post contain no useful information, no need to mod it down
So are you Wikipedia, the Gutenberg project or SourceForge? You're certainly not going to read 1TB of text unless you plan to live the next billion years or so, you are some kind of speciality site. The point is that most use appropriate compressed formats for their purpose. PNG beats BMP-in-a-ZIP. FLAC beats WAV-in-a-ZIP. Lossy formats like JPG, MP3, H.264 video are already well compressed.
Sure, compression is the way to go. But is it that vital that it's in the file system instead of working with zip files? Maybe to you. But I think you know you are an extreme minority on this one. Most people are happy having zip folders and a search engine that reads inside zip files and I know Linux has both. Or actually most people have no problem storing their text uncompressed at all because it takes up <1% of their drive.
I guess none of the people you talked to felt this was a problem worth solving. To me it would be a bit like learning Linux has issues with >1 TB RAM or >1000 cores, even if that was so I wouldn't exactly feel it is or ever would become a problem for my desktop. So yeah, it probably would get returned with "Well, if it's a problem to you feel free to do something about it, but I don't think anyone here will work on it."
Or something less polite, depending on who you ran into and how you formulated your feature requst.
Live today, because you never know what tomorrow brings