Tru64 Unix Advanced File System (AdvFS) Now GPL
melios writes "In a move that could help boost the scalability of Linux for grids and other advanced 64-bit multiprocessor applications, HP has released its Tru64 Unix Advanced File System (AdvFS) source code to the open source community. Source code, design documentation, and test suites for AdvFS are available on SourceForge."
Allow me to be the first to say: It's about fucking time.
Kid-proof tablet..
Who?
Is there some reason to pick this file system over any of the other 100 file systems you can get for Linux?
This is my sig.
The last file system I messed around with was absolute murder.
Hans Reiser is the developer of an open source file system but is currently jailed for murder.
Dedicated Cthulhu Cultist since 4523 BC.
I appreciate what they're doing, and I sincerely hope that it becomes a viable option within the next 6 months or so.
Cause I'm not using it for anything mission critical before that, anyway.
Check out my sysadmin blog!
I didn't know any of the details of what AdvFS was, so here is what Wikipedia has: http://en.wikipedia.org/wiki/AdvFS
...I hear WinFS will be in Win7...it should be legendary.
Will linux need to make it "more enterprise ready"?
I think we see this claim to fame almost weekly yet it seems less and less reliance on OS filesystems and more reliance on SAN/Hardware/NAS/NFS storage.
OS filesystem improvements are welcome sight but the headline seems sensational as if all the other filesystems are actually holding adoption back. (which seems absurd)
I just had a quick glance through the wikipedia page on this filesystem http://en.wikipedia.org/wiki/AdvFS
and it seems to share a surprising number of features with ZFS
http://en.wikipedia.org/wiki/ZFS
For example, pools, snapshots etc.
Cool, license squabbling aside I look forward to the massively fragmented UNIX codebase slowly coalescing in this area.
I'd love to hear Mark Crispin's comments on this.
...all I can say is that this would have been amazing news about ten years ago. Even five years ago it would have been pretty great.
Now? Well, it sounds like HPaq is just kicking it to the curb so it will probably be another year or two before anyone can beat it into a working filesystem for anything but HPucks. There is already no shortage of file systems that can do what AdvFS could do, so by the time it is ready for prime time prime time will have moved on.
Oh well. 1998 me is still pleased to hear this.
Woosh
Ubiquitously - A Ubiquity Developer Community
I used ADVFS when I worked at DEC/Compaq. It is a really nice filesystem to use.
If the utilities are GPL's as well that is even better news.
Copying whole filesystems is a breeze as is copying filesystem trees and traversing over volume mount points ( ie not including mount points and all their files.)
It also gives you the ability to add/remove extra space to mounted volumes just like LVM does but IMHO without having to pre allocate it. /S
I would expect that some of the features may well be in EXT4 but I think that some of the Utilities could be made to use EXT4.
Your sarcasm detector needs adjustment.
Certainly the Linux community doesn't really need to burn energy supporting a half dozen filesystems.
Talk to six linux admins and you'll get at least that many "every filesystem but the one I'm using sucks!" responses.
I'd gladly stand up for a lack of choice on the filesystem front. Pick one, make sure it's absolutely tested, make sure it supports a nice range of features.
Integrating a filesystem into another OS is a decidedly non-trivial task unless you just want to read files.
Thanks, HP, but I don't really want your no-longer-commercially-viable undead zombieware.
. Penguins Surely Ca
But can it run Linu... oh... right.
Stop the Slashdot effect! Don't read the articles!
claim this? I think that I hear SCO lawyers already knocking on the door.
I prefer the "u" in honour as it seems to be missing these days.
Another ReiserFS killer!
Why use this second rate filesystem when you can use the revolutionary Reiser4 designed by World Famous Computer Scientist Hans Reiser?
Around here it's hard to tell who's serious and who's not anymore. It's amazing some of the things that get asked around here seriously. Just look up the Ask Slashdot section and you'll see tons of it.
Dedicated Cthulhu Cultist since 4523 BC.
I'm always happy to see a company GPL code, but I have to say that having used both, I think ext3 is considerably more solid than AdvFS...
"Not an actor, but he plays one on TV."
I am totally serious: why does the back of my left ear smell like cheese doodles? I don't store any kind of foodstuffs behind my ear, and I bathe regularly. Please help.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
When somebody asks a question that could be answered by a very simple Google, they're either being funny or they're so terminally lazy it's silly to respond too them. And when the question is about a guy whose murder trial has been in the news (especially the nerdcentric news) for months, I think it's safe to assume that the questioner is not being lazy.
Everyone has been looking at ZFS to provide a whole lot of this same feature set, but the CDDL license has been a significant stumbling block. Releasing AdvFS as GPL could actually put it in the running for real world adoption and use on a large scale. I think Sun already considered this a battle won and may now have to rethink their strategy. If they released Sun as GPL in the next month, I'd be willing to bet AdvFS would probably be largely ignored and become a historical footnote. If Sun waits and lets it gain traction (as they tend to do) it could be they will find themselves with another cool technology they sat on too long and which has been replaced y the OSS community.
I find filesystems and BIOSs intriguing. They are kinda like voodoo, in that you don't really see or configure them (to a certain extent). They do what they do and you don't think about them. However, they can (possibly) have more impact on performance that any tweaks you can do (kernel or application).
I know that some people (read: sysadmins) definitely do think about their filesystem, but even then its usually only when you are installing a system or in the event something goes wrong.
When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
GameBoy Advance: ./ ./ ./ ./
Advance Speaker System:
Advance Laxative:
Advance Filesystem:
Advance Car Insurance: O_o
...not Insurrection. It wasn't perfect or anything, but to say it sucks just goes too far.
Don't you mean:
It'll be legen-- wait for it... ...
I really hope everyone will join me in thanking HP for this and encourage them to release more of the Tru64 OS, HP has been on my $&!â list since they bought and buried this years ago. They are sitting on so much good IP that I really wish that they would only make printers and just the 4000+ series at that.
Lend a hand to the masses Lest It be done incorrectly or woefully worse By those not versed in the ways of the Dogcow
Ok, I understand the reasons for moving from Ext2 to ext3.. but then all this effort to support a bunch of Me-too's journaled file systems each with some marginal improvements over the rest of a crowded field. It seems like a bunch of work for a minor payoff.... The problem that I see that isnt addresses in native linux file systems is small files. try and copy a folder with 50k files each sized from 1k to 10 k each 100 meg total.. so copying a tar file with this much information is an operation that can occur in 4 seconds or less. BUT as small files it is not even close to as fast.. the copy time can run 40 minutes... the file system isnt right for this kind of work. Anyway I would like to see a much wider array of file systems out there.. especially a file system that is designed to handle small files in a manner which is near the operation bandwidth of a conventional hard drive. Now I'm aware that this isnt a concern for a bunch of people.. but it really should be.. small files are what cause really nasty bottlenecks. look at windows it loads information off the hard drive way too slow during bootup
This is another end-of-life open sourcing. Tru64 is a legacy OS from the 1980s, but it still has some users, so HP is dumping the code out there.
That's not funny! It's wrong! It's wrong to laugh at other people's misery! Stop laughing!
I mean, look at this:
"The last file system I messed around with was absolute murder."
That is clearly meant to poke fun at how EXT3 is gradually replacing EXT2. A lot of people worked very hard on EXT2, it's served the Linux community well for a long time, so I don't think it's right to make fun of it like this!!!
Bow-ties are cool.
The irony of this is that Tru64, at the time of the HP/Compaq debacle, had (my estimate) 99% of the SVR4 compatibility layer complete and could have been vetted as HP-UX on Alpha and Itanium by recompiling the HP-UX environment on top of the Mach kernel that runs Tru64. The key is that Tru64 is itself simply a UNIX compatibility layer on top of Mach 2.5. The Itanium port was essentially complete at the time. This would have given HP-UX TruCluster and AdvFS functionality as well as providing Tru64 users a viable path forward under the HP banner rather than the wholesale defection that occurred. I find it interesting that HP is continuing to extend the lifetime of a "dead" product - now to 2012.
Well, ReiserFs handled this by tail packing, while traditionally you're limited to whole FS blocks with ReiserFS you can store multiple small files or the tails of files into a single block.
It's a little bit of a problem though because the FS is now doing more to write files, but from a performance perspective is very good for read performance (stat & file content often stored in the same block). That's very good for creating tar archives of large directory structures quickly which solves half the problem, but it's take longer to extract (I don't have the numbers for exactly how much longer).
Just as an example, ReiserFS + RAID1 is amazing in the right setting with less than ~10% writes, such as large content caches esp. with a large write-through cache. (*wonders what'll happen to the FS he loves now that Mr Reiser is in prison*)
The only way to get optimal performance for lots of small files is to be sequentially stored one after another, preceded by the directory structure and stat information in one big block that's loaded into memory. This is really for read-only data like uh.. tape drives, rom filesystems (seen cramfs?).
I guess what I'm trying to say is that filesystems have adapted to compromise between read & write ratios, and any that favors one too much is no longer general purpose or very useful to many people. There will be breakthroughs every now & then, and increasingly fast storage (flash? in-ram? holographic? quantum?) is invented and becomes cheaper, but there are only so many different ways you can reinvent the wheel.
You're not a PHB are you? Let me explain it, DEC/HP/Compaq is a PHB company. When these decisions were taken, unix was legacy and Windows NT was the Way Forward. To hell with technical or business requirements. With enough spin and shiny marketing, all things are possible. That's why we're all running 32-bit Windows PCs and the entire world's servers are running an NT-derivative on itanic. Unix is dead. RISC is dead. x86 is 32-bit only.
Stick Men
Isn't he the antagonist in Hellreiser? The daemonic killer who keeps a journal of its victims.
Ezekiel 23:20
I currently use Tru64 in production at least for another month. One of the issues with this encapsulation type FS process is it sucks. If I had to try and figure out how to merge two File systems by some vote of talking heads, this would be the result. It has some strong and good things it does well, but the way Tru64 merged it's file systems together, makes the final product a huge pain to administer and fix. Learn what you can from the code, and make something better. Do not try and port this crap to something else as is, you wont be happy.
Why do you think HP bought again the newer Veritas File system and didn't use the already payed for version they picked up with Tru64?
It has some good things in it. Pick them out carefully and learn from them. Then think about what is needed to administer your File systems in real life, and implement it.
Epic fail
Make SELinux enforcing again!
You had to be rich to afford IRIX. It was possible to get any storage problem solved - if your clients had the money. The guys in post production houses were using IRIX to shift around a lot of full-frame multi-layer video a good fifteen years ago. At the time IRIX was what was needed, not the offerings of IBM, Sun or HP.
I guess that HP-UX has moved on since the days of the 'PA-RISC' chip but it is late to the Linux party where even SGI'd legendary XFS doesn't get much of a mention. It is yet another file system.
This is HP opening their own (or rather, their acquired) code, not Linux kernel developers spending their own time on it.
Also, AdvFS is hardly some me-too filesystem. It was one of the most advanced filesystems of its time, and significantly predates ext3 - and ZFS.
HP are not (directly) trying to solve problems with Linux filesystems, so the rest of your post is pointless.
Sure, it doesn't show up on servers or workstations anymore, but FAT is alive and kicking on every digital camera, most Flash drives, and many cell phones and PDA's. I'm waiting for the day when the FAT table gets screwed up on one of these devices. I know it'll happen some day... but when and where...
actually I think this should be rated, Offtopic because, though he did such a thing, he didn't develope this one. I think the "who?" was in refering to the fact that few use True64 and I am suprised HP still dealt with unix instead of moving upgrading their clients to linux.
There are advantages to doing stuff at the filesystem layer. One of the biggest gains is that the filesystem is aware of the difference between allocated vs unallocated space. At the block layer, you can waste a lot of resources on snapshots, mirrors, encryption, etc., of unallocated space. (Copy-on-write block devices sound like a solution -- as long as you never delete anything and only do changes-in-place.) With snapshot capabilities in the filesystem, you can get more useful semantics (like the ability to easily access past versions of a given file without mounting umpteen snapshots). Similarly with encryption: People frequently only want to cipher selected files, but don't want to organize stuff between different filesystem partitions, but still want the convenience of having the cipher be transparent to the application. (Whether AdvFS in particular takes advantage of any of this, I have no idea.) I'm not saying *you* should care about any of this, but some people do.
And resizing a block device without resizing the filesystem inside it is... an interesting idea. ;-)
There are advantages to the layered approach, too, as you note. Personally, I'm of the opinion that the two approaches can co-exist in the universe without causing instability in the space-time continuum. Let the user chose what to use, based on what best suits their use. One can even use them both on the same system, or even layer the many-feature filesystems on top of LVM -- for the worst of both worlds! ;-)
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
You'd have to ask the people writing EXT2/3 code for Windows. Filesystem drivers do not spring into existence though spontaneous generation. They actually require someone to write them. And by all accounts, writing a filesystem driver is rather difficult. If there's little interest in having working EXT2/3 code for Windows, that's not the fault of Windows. (Well, it might be an indication that few people want to go back to Windows, and that might be due to the design of Windows, but that's a human behavior issue, not a technical one.)
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
But it doesn't go nearly far enough.
HP needs to kill HP/UX, IBM needs to kill AIX, and anybody else with a proprietary UNIX needs to kill it, and donate the source code to Linux. Including Sun with Solaris.
Had they done this ten years ago, Linux would be running the show now, instead of Microsoft.
The big companies have utterly no need for a proprietary UNIX that does nothing but jack up their development costs. Donate the existing code to Linux, wait until what fits and makes Linux sufficiently enterprise-level is adopted, then adopt Linux as their unified platform. Then they can devote development expenses to differentiating themselves with system management software, which is the sort of software open source tends to lag in producing.
By sitting on their asses, all they've done is give Microsoft an opening into the server market. Eventually the server market will be either dominated by Windows or shared equally with Linux, anyway. Nobody's going to care if the proprietary UNIXes go away as long as the necessary features from them are available in Linux.
Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
Since HP built the super cluster that's gonna be used to wiretap all of Sweden's internet and phones (and everyone who has any dealings with anyone in Sweden, or has any traffic that happens to pass through Sweden), one wonders if this is the FS that drives the beast? If so, it should be pretty gosh darn impressive.
...with ReiserFS you can store multiple small files or the tails of files into a single block.
AdvFS was doing that about 15 years ago.
Freedom is not the license to do what we like, it is the power to do what we ought.
You know, the ultrix source code is available via harhan's ftp sit:
ftp://ifctfvax.harhan.org/pub/UNIX/thirdparty/Ultrix-32/sources/
tinyurl 4alun4
Also the digital unix source code is on the edonkey2000 p2p network
I worked in the Digital UNIX engineering group from 1995 until 1998. The name Digital UNIX was already being used at the time that I joined the group, though the UNIX product in the Version 3 era was called Digital OSF/1. Digital UNIX V4.0 definitely used the UNIX name because it passed all of the branding requirements. I cannot recall if we also named V3.2 Digital UNIX or not - that was around the turning point from OSF/1 to UNIX in naming.
You are wrong about Compaq making the name Tru64 UNIX. We decided that before Compaq grabbed us. Whether we had a release out the door, perhaps not. Internally we calleed V4.0 the Platinum release. We called V5.0 the Steel release. I left after Build level 16. There were at least 20 builds in the release. Sometime near that time Compaq acquired Digital, so that is why it appears that the Tru64 name is a Compaq name, but that is not true, the Digital engineering and marketing people came up with the name.
Reminds me of a fun story. We had some cool sweatshirts made. We wanted to call it butt kicking (and a bunch of other stuff) Linux, but marketing named it instead and called it Digital UNIX. The engineers had a good sense of humor and a great deal of pride in the OS. Too bad the rest of the corporate culture stifled it. I believe in the 1995-2000 era it was the best Linux available.
Brian Masinick, masinick at yahoo dot com Linux
You are right on with this. HP wanted to close Spit Brook Road entirely. Several of the original AdvFS and TruCluster developers were long gone - I knew quite a few of them, and I used to regularly eat lunch with one of the early AdvFS developers. This was about HP getting rid of their former Digital (called Compaq) engineers and closing facilities. They may not have saved money by dumping AdvFS and Trucluster projects by their cost, but by closing Spit Brook Road, I am sure they saved plenty.
Brian Masinick, masinick at yahoo dot com Linux
2. I want selection in file systems, not just a bunch of GP choices.. I want special purpose choices for those spots where it is really needed.
3. yes, I like reiser4 and hope it goes on.
4. I kind of disagree about the concept of reading the small files as a big block only being for tape drives... I think that having a system auto-cache the small files in a directory once it's opened, having all of them read in one fell swoop is a do-able proposition.. with some limits of course.. but in most cases it's a slam dunk.
I have to agree; I work at Spit Brook Road for Dell EqualLogic, and can see the resignation written on the faces of many of the people still there on the HP side of the complex. Those who are still there will be either working from home or commuting 50 miles or more to Marlboro, Mass.
I know many of them from my times at DEC/Compaq/HP, and it's sad to watch.
On the other hand, on our side of the complex, we're doing rather well, with Dell EqualLogic arrays selling like hotcakes. And much of the development team is ex-Digital, so it's fun to be a part of building really good products inside the old ZKO complex.