IBM Open Sourcing AFS

Erik from IBM wrote to us to confirm that IBM will be a "open sourcing" AFS. What's actually going is that they are forking the code, as not all of the code can be opened for "technical or licensing reason". So, there will be IBM AFS, which they will support, and Open AFS which will be the open code. The license is going to the be the IBM Public License which is an OSI approved license. Overall good news for a very scalable, secure file system.

Comments

Re:When is this going to end?

[cwebster]

jfs is a wonderful filesystem with excellent ties
to the underlying lvm system. It lets you
do things like

[casey@pisces:~]$ uname -a
AIX pisces 3 4 002046657000
[casey@pisces:~]$ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 16384 7152 57% 1110 28% /
/dev/hd2 1712128 27072 99% 23282 11% /usr
/dev/hd9var 8192 4848 41% 206 21% /var
/dev/hd3 24576 23080 7% 59 2% /tmp
/dev/hd1 221184 49944 78% 4309 16% /home
/dev/lv00 32768 29328 11% 2519 62% /usr/vice/cache
/dev/vardce 24576 17752 28% 68 2% /var/dce
/dev/dfscache 204800 38904 82% 5139 20% /var/dce/adm/dfs/cache
DFS 18000000 18000000 0% 0 0% /...
AFS 144000000 144000000 0% 0 0% /afs
[casey@pisces:~]$ su -c chfs -a size=+20000 /home
root's Password:
Filesystem size changed to 245760
[casey@pisces:~]$ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 16384 7152 57% 1110 28% /
/dev/hd2 1712128 27072 99% 23282 11% /usr
/dev/hd9var 8192 4848 41% 206 21% /var
/dev/hd3 24576 23080 7% 59 2% /tmp
/dev/hd1 245760 73720 71% 4310 15% /home
/dev/lv00 32768 29328 11% 2519 62% /usr/vice/cache
/dev/vardce 24576 17752 28% 68 2% /var/dce
/dev/dfscache 204800 38904 82% 5139 20% /var/dce/adm/dfs/cache
DFS 18000000 18000000 0% 0 0% /...
AFS 144000000 144000000 0% 0 0% /afs
[casey@pisces:~]$

that is only one of the things (and the most
commonly used) that i like about jfs (i know
rieserfs with lvm can do the smae thing, thats
what i use on my suse box) but the jfs is a lot
cleaner.

Re:Lotsa licenses

[randombit]

Until I followed this link I wasn't even aware that there was an IBM Open Source License.

They've released stuff under it before, including the jikes compiler. Though they have released stuff under the GPL when neccessary (like the Haifa optimizations to gcc).

Personally, I agree with you, I think that BSD, GPL, and LGPL cover at least 95% of all code released under these other random licenses. But, hey, as long as it's free, right? ;)

Re:Lotsa licenses

[barracg8]

• Opening up the code for anything (even if MS did it) is a good deal,

Amen halleluja to that, brother ;-)

Here is a link to the IBM public license.

I guess people keep having to come up with their own licenses all the time, as the GPL isn't GPL'd. :-) Anyways, I doubt that the IBM legal department would be all that happy if people just started releasing code under of the shelf licenses - aside from many concerns they may have about the license, it would make them rather redundant!

At a glance, the licence looks quite GPL-ish to me, ie if you redistribute you must make source available& the license, or an equivalent one, propagates.

• IBM may publish new versions (including revisions) of this Agreement from time to time.

I guess one genuine reason that everyone needs their own license, is that everyone needs to state who it is who has the right to change the license at a later date. Remember: FSF retains copyright over the GPL.

cheers,
G

Re:Why? CODA

[Ledge Kindred]

"Yeah, but can I use CODA now?"

Absolutely. It was extremely stable in my experience and I haven't really looked too closely at it for a year or so. (It has some other drawbacks that make it less than ideal for my personal use.)

The "worst" thing about Coda is not its maturity or stability, IMHO, it's all "ease-of-use" problems, in that it really isn't very. It took me a couple days just to understand it well enough to try to start getting a primary server up. Trying to set up a replicated secondary server was quite a chore.

As I said before, it's been a while since I've looked at it, so that might have gotten better as well.

-=-=-=-=-

Re:This will be good news, if they do it

[Anonymous Coward]

Arla is distributed under the BSD license.

you probably don't want AFS

[jetson123]

AFS semantics are very different from UNIX file system semantics: permissions are associated with directories only, access is determined only by the containing directory, if multiple clients modify the same file, updates are lost, you can't have any special files in an AFS file system, etc. AFS uses its own authentication, it doesn't work well for big files, it always requires extra work to get it to work with daemons, and it has severe problems for scientific compute clusters. IBM has long ago moved onto DFS (unrelated to Microsoft DFS), which fixes many of the problems of AFS (but is itself big, even more complex than AFS, and hard to administer). Many places are trying to get rid of AFS because it's just too much of a hassle to run it (and converting back to a UNIX file system isn't easy because AFS encourages permissions and ACLs to mushroom unnecessarily).

AFS may be acceptable for specific applications (in fact, what it was designed for originally): a large untrusted user population, dedicated system management staff, and smallish files and problems (text file editing, small programming jobs). But for many environments where Linux is used--big software development projects, web servers, scientific computing, home networking--it just doesn't seem like a good fit.

If it's the security you care about, NFSv4 might be for you, although it clearly also has some problems. If you want something AFS-like, Coda might be an option (but I don't know how mature it is yet). MFS and GFS are options for compute clusters. Maybe we can get 9P or Styx up on Linux.

Re:you probably don't want AFS

[nconway]

IBM has long ago moved onto DFS

Um, no, they haven't. I was talking to a guy from the IBM JIT team, and we discussed AFS - it is still used extensively by IBM for their internal network. He wasn't overly impressed with it, but we agreed it does have its uses.

Perhaps someone from IBM can confirm this?

Re:you probably don't want AFS

[IcePic]

> it doesn't work well for big files

Yes, as if the Linux crowd didn't already have
that problem for a long time (except Alpha folks)
so I can't imagine why the slashdot community
would see _this_ issue as a big one. I mean,
clean up your own act first...

Re:you probably don't want AFS

[Anonymous Coward]

> AFS semantics are very different from UNIX file system semantics: permissions are associated with
> directories only, access is determined only by the containing directory,

Think about hard links: that's why it works this way.

> if multiple clients modify the same file, updates are lost

That's not entirely true but I agree it's stupid. Anyway, it doesn't matter, if you don't use file locking you should expect corruption anyway.

> you can't have any special files in an AFS file system

I hope you don't expect your users to be able to create /dev/mem nodes in their home directories...

> AFS uses its own authentication

Yes, it's called Kerberos... ever heard of it?

> it doesn't work well for big files

It works reasonably well with big files, unlike Coda which unfortunately doesn't work at all with them. Anyway for huge amounts of data you shouldn't be creating massive files anyway, look into databases or steaming software.

> it always requires extra work to get it to work with daemons

You mean you want root on a given machine to have "root" in your whole enterprise?

> and it has severe problems for scientific compute clusters

What, rsh doesn't work? Just patch it and it works fine. Otherwise what's the problem?

> IBM has long ago moved onto DFS

No they haven't

> (unrelated to Microsoft DFS)

Thank god. But I'm glad Microsoft has finally invented the automounter.

> which fixes many of the problems of AFS (but is itself big, even more complex than AFS, and hard
> to administer).

And nobody uses it...

> Many places are trying to get rid of AFS because it's just too much of a hassle to run it

There really is no better alternative, though.

> (and converting back to a UNIX file system isn't easy because AFS encourages permissions and ACLs
> to mushroom unnecessarily).

You mean it encourages security? :)

> AFS may be acceptable for specific applications (in fact, what it was designed for originally): a
> large untrusted user population, dedicated system management staff, and smallish files and
> problems (text file editing, small programming jobs).

It lets you solve problems on a big scale. I hope the open source release will make it even better and more available for everyone to use.

> But for many environments where Linux is used--big software development projects, web
> servers, scientific computing, home networking--it just doesn't seem like a good fit.

Big software development is one of the first things AFS was used for. It's only recently, ironically, that local disks+Linux have outperformed network file systems so much.

AFS makes sense on web servers for replicating site data and allowing many people to "upload" without the insecurity of FTP.

And I don't see why anyone wouldn't want to use AFS at home. Again, I hope the open source release will allow as many people to have real security in network filesystems as possible.

> If it's the security you care about, NFSv4 might be for you

Whenever that will be available...

> If you want something AFS-like, Coda might be an option (but I don't know how mature it is yet)

Coda is nice but not packaged well enough for everyone to start using it. It also chokes on big files much worse than AFS, unfortunately.

> MFS and GFS are options for compute clusters.

They're nice for high bandwidth to big files. But they give you no security... do you really want a root exploit on one machine in a cluster to destroy all data in the entire site?

Re:you probably don't want AFS

[jlrobins_uncc]

AFS makes great sense for Web server farms and/or mirrors of the same site across a WAN such as the Internet (think an east coast site and a west coast site). Just edit the file and pow, a server -> client callback notifies any clients caching the file that they need to refetch.

Couple this with having the content in a read-only replicated volume, then go ahead and update many files, get your new site look-and-feel redone, then once your happy with it, release the read/write volume for replication, and pow -- one atomic transaction to all of the mirroring servers on the WAN!

Mabye this is why AFS is a major component of IBM's Websphere platform. All of this, currently working like a champ, and it'll be free and open source!

Re:you probably don't want AFS

[Tower]

Actually, both AFS and DFS are in use here at IBM (and at every other site I've vistited... no AFS on the windows boxen, but everyone using the RS/6ks seems to prefer AFS... Personally, I prefer the ACLs of AFS to traditional permission structures, and they are really rather flexible. You can still set rwx on the files, so it doesn't take a whole lot away...

I agree that AFS isn't meant for clustering, but it works well from a security standpoint, especially with Kerberos.

--

Re:you probably don't want AFS

[thimo]

look into databases or steaming software.

Hmmm... Don't know if I'll do that. I've got some steaming *hardware* which I iron my shirts with though!

Thimo
--

Re:you probably don't want AFS

[jetson123]

Of course, IBM uses AFS extensively internally, as they have been for many years, and there are environments where people like and use AFS.

My point was not that IBM has somehow discontinued AFS (clearly they haven't), my point was that IBM themselves has recognized the technical problems that AFS has and tried to address many of them with DFS.

Re:you probably don't want AFS

[jetson123]

If you want to keep a bunch of web sites synchronized over a WAN, rsync is much simpler to deploy (it's just a user program) and considerably more efficient than AFS. You also get much more control over what gets updated where and when.

For serious wide area, distributed authoring, no distributed file system, no matter how good, is going to be adequate by itself. Distributed authoring requires workflow support, version and revision control, support for disconnected operation, and other features. For that, something like WebDAV or CVS are more appropriate choices.

Don't get me wrong: AFS isn't all bad. Some of its core ideas are really great. But some of its practical aspects (e.g., differences in semantics from UNIX, simplistic caching strategy) make it a pain and rather inefficient in many real world settings. There are some areas where people can live with those limitations (e.g., university computer labs), but I think for most environments, NFS and SMB, despite their many warts, are still more practical systems.

Re:you probably don't want AFS

[joshua_doesnt_know]

My school uses AFS and because of it I can access the same home folder anywhere on the network as if it were on that machine, transparently. Isn't that what Unix is all about, transparency? :)

_joshua_

Re:you probably don't want AFS

[Gerdts]

The problems you mention with "not working well with daemons" is likely related to the fact that it uses Kerberos IV. If the daemon needs to have more access to AFS directories than you are willing to give to any other user on the system, there is a lot of work to do.

Specifically, you need to stash a password away such that the daemon can authenticate and periodically reauthenticate so that it does not lose the rights that it has.

AFS does allow you to have ACL's based on IP address. As such, if you are running a daemon on a machine than only system administrators have access to, it may not be a big deal to allow everyone on that machine to write to a directory. Other machines, though, may have read-only or no access to the directory.

NFS 4 will have the same problem, as a requirement for it is that Kerberos V is supported as an authentication mechanism. If you don't give world write to a file/directory, then you cannot write to it without a kerberos V ticket.

Re:you probably don't want AFS

[jetson123]

Your response is generally "well, AFS doesn't do this, but you shouldn't be doing that anyway".

The fact is that people do deal with gigabyte files over networked file systems (video editing, scientific datasets, server logs, etc.), they do run UNIX installations that don't use AFS Kerberos as their authentication method, they do need to create named pipes and UNIX domain sockets on networked partitions, and they do expect that UNIX access semantics are preserved when using remote files. AFS fails to deliver on all of those. The AFS designers simply thumbed at UNIX semantics and didn't give a damn.

NFS does deliver on all those points. For small and mid-size installations, NFS management is pretty simple, and NFS security is getting better and less of a problem with switched Ethernets anyway.

The suggestion that AFS is good for content distribution to web server farms also strikes me as silly. Installing AFS to achieve synchronization between web servers is like driving a truck to pick up a carton of milk. rsync and similar tools are much simpler to deploy and much more flexible.

People can get excited about whatever they want, and if AFS makes you happy, great! I have used AFS for many years, and my recommendation is that people should look at its incompatiblities and quirks very carefully. I think for most UNIX environments, it is not a very good choice.

But what this announcement may mean is that, after years of neglect, maybe people will roll up their sleeves and fix those rather serious problems that AFS has. Then, it could potentially become a good distributed file system. Until then, it is a solution that, in its own way, is as flawed as NFS, and quite a bit more work to manage.

Re:AFS Client availability

[syates21]

What OS do you want a client for?

And IBM is distro neutral.

[EdlinUser]

IBM's Linux efforts are amazing. The negative IBM comments about this minor happening seem to be the worst things said about IBM on /. for over a year. They seem to make all the right moves--even to spreading distro support. This week IBM announced a Thinkpad with OpenLinux and AS400 with SuSE(RSN). Over a year ago John Patrick of IBM told a group free software developers:
"We realize we are the 800 lb. gorilla here, so our manners must be impecable."
And they have been! Amazing, just amazing.

Re:And IBM is distro neutral.

[Zurk]

yup. wish they would look after their own products tho. as an ibm customer its damn annoying when ibm doesnt do simple stuff...for example, theres no type iv jdbc/odbc driver for DB/2 yet (well..there is for DB/2 400 but not the regular DB/2). whats up with that ? its just stupid. and this is from the company which wrote their own jdk ? and they cant write a type iv jdbc/odbc driver ? wha ? the mysql guys wrote it in a matter of weeks for mysql.

Re:When is this going to end?

[jlp2097]

What I wanted to point out, is that more and more firms are opensourcing their fs (sgi their xfs (oss.sgi.com/projects/xfs/), ibm jfs and afs (for jfs have a look at oss.software.ibm.com/developerworks/opensource/jfs /index.html)). For explanation: namesys is reiserfs, and for tux2 have a look at the current kerneltraffic at http://kt.linuxcare.com/kernel-traffic/latest.epl.

Bye

Too little, too late?

[qralston]

As someone who has worked with AFS for the past 8 years, I have to say that I greet this announcement with a somewhat more pessimistic view.

Namely: AFS is now officially dead.

I say "officially" because, IMO, AFS is already dead, and has been for years (ever since Transarc (now IBM Transarc Labs, but I'll refer to them as Transarc for brevity)) came out with DCE/DFS, really).

Oh, there were bouts of heavy maintenance and limited development. These periods were inevitably precipitated by Transarc's AFS customers becoming vocal and complaining. But when the complaints died down, so did Transarc's commitment.

Transarc has never treated AFS like a real product. Their "development" efforts have been limited to ports to new versions of the same operating systems, a few ports to new architectures, bugfixes, and very limited feature additions (mostly backports from DFS).

In fact, this year has seen Transarc's AFS support sink to a new low. From what I've been able to garner, all AFS development is being outsourced to India. Responses from Transarc's AFS hotline support (a support service which customers purchase!) have been inept. There was no Decorum (Transarc's yearly AFS conference) this year, nor even an announcement concerning it. It's been ages since anyone from Transarc has posted on the AFS mailing list.

So, why is Transarc (now IBM Transarc labs) open-sourcing AFS? For one simple reason: AFS is IBM's red-headed stepchild, and they don't know what else to do with it.

If you read the announcement at http://www.transarc.com/News/pre ss/opensource.html, you'll note this entry in the FAQ:

Is IBM still investing in AFS?

Yes. IBM recognizes that many of our customers will still want a commercially-supported version of AFS IBM AFS. IBM/Transarc will still sell, maintain, port (to new versions of currently-supported OS), support, and provide minor enhancements to "IBM AFS".

Good software grows or dies. AFS died a long time ago. I, personally, think this is tragic, because AFS had great potential. But Transarc never made a long-term commitment to anything other than keeping it on life support. Perhaps it can be resuscitated back to health, but I can't help but wonder if the Open Source community's effort would be better spent towards other distributed filesystems efforts, such as CODA (which I admittedly haven't investigated, but plan to).

Re:Silverwear

[IcePic]

Well, if you owned a source-license for AFS before
you'd still not get everything, like the efs/xfs
that's needed to run a AFS server on Irix. My
guess is that those parts are what keeps them
from releasing everything. What you _did_ get
if you had a source license was enough to make
yourself a server and clients.

Re:At this rate...

[kronoman]

yes, but installing it alongside another OS is a train wreck. It's very easy to inadvertently (a simple typo. You don't even have to be stupid) munge another OS's partition by defining the partition in the disklabel just one sector too big. Outside of this, it's great. I still use Linux, and FreeBSD, but OpenBSD definitely has my attention.

Re-Releasing as GPL.

[AftanGustur]

from IBM's licence page.:

2. GRANT OF RIGHTS

• a) Subject to the terms of this Agreement, each Contributor hereby grants Recipient a non-exclusive, worldwide, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, distribute and sublicense the Contribution of such Contributor, if any, and such derivative works, in source code and object code form.

Is there anything in the licence that prevents me to simply re-licence the code as GPL

--
Why pay for drugs when you can get Linux for free ?

IPL & Linux

[Amphigory]

My big concern is this: presumably, AFS includes some kind of kernel driver. If the IBM public license covers AFS, then will we be able to integrate AFS into the Linux kernel?

Incidentally: for those of you claiming that AFS is obsoleted by Coda, think again. There's no way I could get my employer (one of the biggest Internet providers out there) to buy into Coda at this stage of development. AFS on the other hand they would /definitely/ go for. The biggest problem has been that IBM doesn't really push it, so it's hard to get executive attention for it. If it's oss, I don't need executive attention -- I just do it.

--

Try killing a free software...

[deno]

It is a difficult task: once there is a source code of a package some good programer considers "interesting", the only way to kill that particular software would be to kill all the programers willing to work on it.

Then you would have to hount ALL the copies of the source-code on the earth and burry them in the dirth... And a few months later someone would dig it out again. I do not claim that AFS will re-emerge as a next big thing now, but if there is freely available code, someone will hack on it.

Since there seams to be a base of users ready to actually PAY for maintainance, it is very likely that some programmers may even end-up beeing payed to work on AFS in the future... It does not look so dead to me...

Link to announcement

[ColdN]

I found the announcement over at IBM Transarc Lab. I also includes a short FAQ.

Re:This will be good news, if they do it

[ejbst25]

FYI...The actual source is slated to not be released until next month (if all goes as planned). At that time it will be in developerWorks. After this CNET article mentioned this...it has been announced as public news. The link I sent in was where it will eventually be put. For the press release click here . For more specific information on AFS, you might try the Transarc website.

-Erik (from IBM) :)

Re:When is this going to end?

[import]

AFS is a network file system. xfs, ext(2|3) are not. (I don't know about the other ones.)

Re:Once again...oh, puh-leeze

[gizmoNaut]

I won't say all the things that others have already in reply to this one. I agree with them completely, and I'll just add one point: IBM has been, is, and likely will be for quite some time, the best friend Linux and open source have among the Big Companies.

Re:Lotsa licenses

[aeloff]

In my experience with mid- to large-sized companies, I've found that legal departments freak out about licenses that were not developed in-house, regardless of the actual content of the license. In fact, it's quite similar to the "NIH" complex that some engineers develop. ("We can't use that RTOS because , we've got to spin our own.")

Re:AFS isnt dead

[cwebster]

AFS isnt dead.

[casey@pisces:~]$ uname -a
AIX pisces 3 4 002046657000
[casey@pisces:~]$ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 16384 7152 57% 1110 28% /
/dev/hd2 1712128 27072 99% 23282 11% /usr
/dev/hd9var 8192 4848 41% 206 21% /var
/dev/hd3 24576 23080 7% 59 2% /tmp
/dev/hd1 245760 73720 71% 4310 15% /home
/dev/lv00 32768 29328 11% 2519 62% /usr/vice/cache
/dev/vardce 24576 17752 28% 68 2% /var/dce
/dev/dfscache 204800 38904 82% 5139 20% /var/dce/adm/dfs/cache
DFS 18000000 18000000 0% 0 0% /...
AFS 144000000 144000000 0% 0 0% /afs
[casey@pisces:~]$

my comments are mine alone and do not represent IBM's

Re:Glancing at the FAQ

[mayonaise]

Sorry! Didn't see the post with the link to the FAQ before i started writing up my submission!

This takes you straight to the answer to my question!

Re:Why? CODA

[Tower]

While at RPI, there was always a linux client for AFS, though you had to know who in the computing center to ask... The linux client in use here at IBM works well, but there doesn't seem to be a 95/8/NT/2k client worth anything... doesn't bother me though 8^)


--

Re: Lotsa licenses -- TMTOWTDI

[Icebox]

I'm not saying that everything has to be GPL. Right now all of these open releases, regardless of the license, have a community feel to them. Programmers helping programmers you could say. The problem with all of the different licenses shows up at the same time all of the lawyers do. The way Microsoft released their Kerberos extensions should have taught us that the big guys will are still looking for ways to open their code but still leave some strings attached.

AFS is rough to configure, on purpose I think.

Re:They don't fully get it.

[finkployd]

It's been outplaced by Coda and DFS.

I don't think DFS is registering to high on the "alive" scale. I can only think of two companies that support it, and sometimes it feels like my shop is the only place on earth using it :)

Finkployd

Re:At this rate...

[JebOfTheForest]

I don't know about Open/Trusted/SecureBSD, but I honestly think FreeBSD is a little easier to use in some areas than linux. Like the ports tree. I love the whole ports tree. I'm no unix guru at all, but I actually tend towards FreeBSD in some ways, ease-of-use-wise. I dunno...just an opinion, and honestly, not a very well informed one.

jeb.

IBM + OpenSource = $$$IBM$$$

[Hal_9000@!!!@]

It seems to me (not that this is a bad thing) that when companies like IBM decide to open their software, they wan't a good way to make more money.
Not only is this great PR for IBM, but this also means that IBM has just scored another brownie point with the Free Software community, and will still probably sell more licenced copies of AFS then they did before. In the end, IBM wins a lot every time opens software.

Honestly, I don't disapprove of this, its a good buisness model, and I hope more companies follow suit. Its a win-win situation for all involved parties.

Re:They don't fully get it.

[Tower]

I'll still take AFS over DFS any day - despite the 'improvements' that DFS brings. I haven't seen Coda running in a large enough enviornment to convince me yet, but it looks pretty sound.

--

Why this has so much potential for good.

[jlrobins_uncc]

AFS is a very stable, tested, enterprise filesystem. It offers the following features:

• Cross platform: Many UNIXen as well as NT as either client or server.
• Secure: Uses Kerberos IV for user authentication.
• Client-side caching: client machines use disk or virtual memory to cache MRU files, greatly reducing # trips down the wire on reads.
• Unified naming scheme: names of files don't indicate what file server they're on. Makes moving of volumes from one fileserver (or drive on the same fileserver) to another a cinch, since no client-side changes need to happen.
• Read-only replication: Make your application install directories replicated in each building on campus.

Now, it's not a perfect product, but it is way cooler than vanilla NFSv2 or NFSv3, especially on the server-side management side of things. It doesn't do disconnected operation (which CODA strives to do), byte-range locking, strict UNIX file semantics (data most recently written == data viewiable by all file handles to that file), or Kerberos 5, but it is a far simpler system to get running than DCE, which does address some of those issues.

One would hope to see the following things from this open sourcing:

• *BSD client / servers.
• MacOS X client (at least!)
• Millineum / Win2K clients (NT clients exist currently).

If the MacOS X client happens, then there will be a secure, scaleable enterprise filesystem for the three major computer platforms -- Wintel, UNIX, and Mac, and it'll even be freely available! I don't believe that there are any products available today that offer secure, robust support for all three platforms (and no, I don't consider protocol translators, such as Samba or CAP, which require you to set up the clients to use cleartext passwords over the wire to authenticate (not to downplay in any way the role of either technology -- it's not their fault that you've got to set up the clients in that fashon to interoperate with AFS as it is now), or using NFSv2 or v3 on the UNIX end to talk to something like Novell 5 (which, AFAIK, doesn't talk at all to Macs anymore)).

This will give us one protocol on the wire, multiple server-side implementations (interoperable in the same cell!), multiple client-side implementations, WAN scalability, and secure authentication. A good day for the world!

Re:Why? CODA

[icqqm]

This shouldn't be seen as a bad thing. Getting big companies to open-source their code because of open-source projects is a VERY GOOD THING. If every company reacted this way then everything would be open-sourced, and our lives would be just a bit easier.

Re:Just a note...

[DrFickle]

I verified this. I will alert the developerWorks team to this. Thanks for finding it.

Re:Lotsa licenses

[Ded Bob]

Unlike the release of the JFS from IBM, this can be incorporated along with the *BSD's. A BSD license would have been better, but at least I will be able to see it on FreeBSD.

Re:Why? CODA

[Tough Love]

Why open source it? Because coda is about to replace it.

At UKUUG this year, Owen LeBlanc, a Coda expert if there ever was one, said "if you have a small number of users and a relatively small amount of data, then Coda may be just what you need". I also seem to recall him saying he thought AFS is pretty darn nice. He'd be the one to know.
--

Re:Once again...

[Type-R]

When you're at a company where you have to use AFS and have to run an old Linux kernel to get AFS support, you'll realize that this is a good thing.

Hmm, you guys should have bought the source license then. (I realize the added cost, but it's worthwhile.) I've been building AFS for every version of the Linux kernel(offical and not) that we run.

IBM Public License vs. GPL

[ddstreet]

As far as I can tell, the only substantial differences between the IBM Public License and the GPL are

Any contributions become IBM's property (Copyright IBM, All Rights Reserved)

You can charge $ for the program (although you must provide source) unlike the GPL (cannot charge for the actual code, only related services)

I know the main reason IBM doesn't like the GPL is the 'Viral Effect' where code that uses GPL'd code must be GPL'd itself (unless it dynamically links?), but it kinda (?) looks like the IBM Public License has the same problem...?

Re:Once again...

[StarKruzr]

I'm still waiting for them to opensource OS/2. What a glorious day that will be....

Email me.
Don't trust anyone over 90000.

Re:AFS Client availability

[Quietust]

considering that NT normally doesn't let you have files with names like .signature (it'd have to be "my.signature" or preferably "JWEILL.SIG").

Really? I thought NTFS supported that. Then again, I could be wrong...

I know that Windows 9x's LFN support allows such filenames (I've got one such file on my system right now), though their 8.3 components look slightly ugly.

-- Sig (120 chars) --
Your friendly neighborhood mIRC scripter.

Re:AFS Client availability

[generic-man]

I'm pretty sure that NTFS can handle such filenames as well. However, because so many programs are written for both NT and Win9x, the system directories are riddled with ugly 8.3 files (DLGCTRLS.DLL instead of DialogControls.DLL, for example). That's why I said "preferably," at least in the parlance of Microsoft "cross-platform" apps.

Couple that with Explorer's insistence on determining a filename by the extension, and hiding the extension by default. That way, a file like ".signature" appears as "" (blank) unless you change your settings.

So yes, you're right in that NTFS will let you create such a file, but the UI will effectively discourage you from doing so.

Where is the story?

[The Big Bopper]

The linked web site makes no mention of AFS, and typing "AFS" into the search field for the site reveals nothing.

Where's the beef?

Lotsa licenses

[Icebox]

Curious link...

Opening up the code for anything (even if MS did it) is a good deal, I'm just wondering about everyone wanting to write their own license. Until I followed this link I wasn't even aware that there was an IBM Open Source License. Why is it that the BSD license or the GPL wouldn't work? Even under those they could have kept the some parts of the code closed....or am I wrong somewhere?

Re:Lotsa licenses

[Anonymous Coward]

Under the GPL, anything statically linked with the GPL'ed code has to be GPL'ed. This is called the viral effect. As for IPL, there is no viral effect. Part of the code can be covered under IPL, and other parts can be covered under other licenses which don't have any viral effect. I presume there must be a good reason for choosing IPL. Hey, it's open, isn't it?

Re:Lotsa licenses

[Icebox]

Of course I meant IBM Public License. My bad.

AFS

[ceez]

I was an AFS admin for 6 years, and did DFS too. The problem with both is that they both have one foot in the grave. I don't put too much faith in Coda or ARLA either. Why? Hardware (SAN, NAS, etc...) approaches are going to take over, and push software solutions out of the way. Interestingly enough all of the original developers that left CMU to go to Transarc (even the faction that went on to Fore Systems) are up to something! There is a startup which is quickly getting to critical mass. (And they have a huge brain trust in Distributed File System technology). Look up Spinnaker Networks. I found out through some contacts that this game was on. I'm really anxious to find out more myself. They're to storage techology what Transmeta is to processor design.

Excellent

[drift factor]

This is great news, even though we just bought AFS...oh well. I wonder what the guys at the arla project will do now?

Re:Excellent

[cxreg]

I wonder what the guys at the arla project will do now

I suppose since there are still parts that can't be opened in the "real" AFS, the arla guys might be able to contribute in those areas of the code. Who's to say that the arla developers didn't have some ideas that were better than the original implementation anyway, ala SAMBA?

Re:Conspiracy, anyone?

[peccary]

:-) In fact, AFS uses the MIT DES implementation.

Re:GPL allows you to charge for software

[ddstreet]

With the GPL you may charge for the program.

I thought this too, at first, until my company lawyer pointed out term/condition 2a in the GPL :

"You must cause any work that you distribute or publish...[that contains GPL'ed code]...to be licensed as a whole at no charge to all third parties..."

You can charge for associated services, such as a transferral charge, warranty, etc. But you can't charge for the code itself.

At least, that's what my company's lawyers have concluded, according to what they told me. I work for IBM.

This will be good news, if they do it

[Hairy_Potter]

The IBM link is non-existent, it must have been retracted. I found this faq that explains AFS.

But when Linux incorporates this, it will be a lot easier to cluster servers, and share files. And maybe we can kiss of NFS forever.

Re:This will be good news, if they do it

[Fizgig]

Well, I wouldn't expect Linux to incorporate it, if only because I doubt the IBM license is compatible with the GPL. What I'm wondering about is arla, the existing free AFS implementation (I can't find what the license is, though I think it's GPL). It's amazing how many different kernels they support with that thing (a reason it won't/doesn't need to be incorporated into Linux). The server support is only experimental, but they client has been a blessing for those of us who wanted to use AFS but didn't want to wait for Transarc to release a new module to upgrade our kernels.

Re:This will be good news, if they do it

[Mr. Bones.]

Did you read the release? From their FAQ section:

When will AFS code be available to open source?

The AFS source code will be made available next month (Sept 2000) in the IBM DeveloperWorks Open
Source Zone (stay tuned to: http://oss.software.ibm.com/developerworks/opensou rce/).

AFS vs. NFS

[mayonaise]

I've messed with AFS more than i have NFS (as a student manager at college), so I kinda know what it can do. And i've setup and used NFS.

Could somebody explain the differences (advantages/disadvantages) between AFS and NFS?
Is there anything better in AFS from which NFS could benefit - or vice-versa?

Justin

Re:AFS vs. NFS

[shandrew]

Access Control:
AFS: Allows user-controllable access control lists
NFS: /etc/groups

Scalability:
AFS: Thousands of volumes, users, and clients? Ok!
NFS: *choke*

AFS is great in that it works, and is really the only mature, well-tested large-scale file system out there. It is horrible in that IBM has killed development on it since it bought Transarc. It usually takes 6-12 months these days between when a new Solaris release is out and when Transarc ports the AFS client to it.

Maybe they figured that if they open source it, they can move those two contract programmers they have doing the ports elsewhere, and just have random AFS sysadmins do the work for them.

Re:AFS vs. NFS

[bgalehouse]

Been a while, but IIRC...

• Authentication
  • NFS: Not much
  • AFS: kerberos token assignment
• Privacy
  • NFS: Cleartext data transfers
  • AFS: I think DES - whatever comes in the kerberos libs.
• Client side cacheing
  • NFS: huh?
  • AFS: Local disk cache. Cache invalidation and systemwide file locking.
• Move files between servers, loadbalance.
  • NFS: Change the client configs, loadbalance?
  • AFS: invisible to user.

Re:AFS vs. NFS

[mayonaise]

Mod the above post by Anon. Coward down please! The link is false and leads to something that, well, just isn't what you'd think it'd lead to.

They don't fully get it.

[Mr T]

With JFS they opensourced it after enough had been done that it didn't matter. Reiserfs and XFS had been making great progress. Result: JFS mailing list is dead, the only people working on it are IBMers. AFS? It's been outplaced by Coda and DFS. It's a welcome addition, I love AFS but there isn't any reason to use it instead of Coda. And if I was to support something, I'd jump on to the coda bandwagon before I'd go to AFS. They are doing stuff though. That is good. IBM still hasn't risked anything though. I feel much more comfortable if they were willing to take some risk and put a non-dead product out.

Yay!

[icqqm]

Despite the technical specifications of AFS, it's good to see a company do the right thing and open-source their code, instead of just making a driver for linux. To me, this is more important than what some idiot wants Star Wars Episode 2 to be named.

Re:Once again...

[Enoch Root]

Dude, your talking about the company that's donating millions of dollars to the open source community, the company the continues to make damn impressive break-throughs with processors, and the company that's porting Linux to a good deal of their massive servers.

Sure. It's called successful PR. How much do you think a marketing campaign costs? Millions donated to reach geeks all over is as effective as any TV ad you could ever produce, and much less expensive.

Them opening up this file system can be useful to everyone, despite the fact that this may not affect IBM one way or another.

You mean to say, 'Them opening up this file system may be useful to some people, especially since it does not affect IBM other than by generating good PR.'

But to mention IBM and Micros~1 in the same sentence is almost criminal.

Should I then also fail to mention that they are both monolithic giants whose sole purpose is to make money? That IBM coined the term 'FUD' to describe their past activities? Or that IBM succeeded where MS didn't, in resisting the attack of the DoJ for monopolistic practices?

Re:Once again...

[Enoch Root]

If they wanna do this instead of a traditional marketing campain, I'm not gonna complain. I hope it does exactly what they expect it to - so that they do it again.

Finally, a modicum of intelligence; thank you. I never said IBM is not helping out the Open Source community; I'm saying they're doing it for reasons that are as selfish and as profit-oriented as MS. They just orient it towards techies by playing on their Open Source sensibilities, and obviously it's working well.

What they're *not* releasing

[peccary]

among other things -- DES. Go figure.

GPL allows you to charge for software

[Error27]

>You can charge $ for the program (although you must provide source) unlike the GPL (cannot charge for the actual code, only related services)

With the GPL you may charge for the program. After you give them the program though you have to offer to give them the source for no charge besides the cost of materials, shiping and handling.

From the GPL:

"You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee."

" b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange;"

The right to sell software is a very important one. This is what made the old version of mySQL non-free (now mySQL is GPL).

Re:GPL allows you to charge for software

[Error27]

You should have stuck with your initial beliefs. They were right and your lawyer friend was wrong.

In section 1 it says:

"You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee."

This means that I can charge an arbitrary amount of money for the GPL program I downloaded.

Section 2b (not 2a) that you are refering to says:

"b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License."

The key word is "licensed." 2b does not describe the act of selling someone a program. It describes the charge for lisencing the program. It is simply another way of saying that the program MUST be GPL. You can sell the program but you MUST license the program under the GPL at no cost.

A theoretical situation where this might occur is if I say, "This program is released under the NPL license if you want the program released under the GPL send me $5." If I had included GPL code in the program that would be illegal.

Re:AFS Client availability

[generic-man]

Preferably Linux (2.4.0-test6 is my current kernel of choice) or Win98. I know the latter one would be a bit of a stretch -- the AFS client on WinNT cluster machines is sometimes kind of buggy. But it manages, considering that NT normally doesn't let you have files with names like .signature (it'd have to be "my.signature" or preferably "JWEILL.SIG").

Re:Once again...

[fgodfrey]

Uh, *every* public company exists solely to gain value for its shareholders. IBM's board of directors wouldn't be doing their jobs if they weren't making money. The same goes for Slashdot, which is part of a public company. What's the profit to VA to run a discussion site where they are occasionally flamed? To get their name in front of a lot of potential customers - that's why. Companies open source software only because they think that in some way they will profit from it whether it is by having other people fix bugs or from PR.

As for the difference between IBM and M$, IBM (at least now) makes money leagally. With M$ it's questionable (and if the current ruling stands, then they *are* making money illegally).

So the bottom line is, just because they do it only for money doesn't mean that it's not good for the people receiving it.

Re:Once again...

[Doodhwala]

Actually transarc has released linux binaries for 2.2.16 (though I have to admit its been a considerable delay).

Re:AFS Client availability

[DaKrzyGuy]

You can get AFS Clients from CMU with out a problem. However you can't always get it for the latest kernel. Take a look at /afs/andrew/system/archive/transarc/afs3.6/binary/ You can get both the kernel modules and the client software from there.

As one of the architects designing DFS in IBM

[gelfling]

We've always had a hard time selling DFS internally. In fact we've stopped trying to do that because there weren't enough internal customers. The hurdle costs were too high the skills were hard to find and expensive and customers still wanted SMB shares via Samba which drove the cost even higher. The client side DCE licence costs drove Samba since the per client cost was $65/seat in bulk. AFS as open source can only be a good thing since we can always find someone to pick up the development and maintenance and foregoing DCE-Kerberos is really not that big a deal from an internal perspective. In our environment the challenge was to collapse hundreds of LanServer domains. DFS or AFS fit the bill and the cost dynamics work very well compared to staffing 1 headcount/25-35 servers in the LanServer world. The problem anyone will find though is backup and storage management. butc or buta just don't scale very well even with multiple replicas of the fldb core so whoever tries to manage this, as we did, will be forced to write extensions to their storage management code, as we did with ADSM. Also you will find that Samba doesn't scale nearly as well as you want with only a few hundred accounts on a Samba server even if it sits on a huge Unix machine. This leaves you will a few hundred or more SMB gateways if you try to scale up to the huge numbers we did.

Once again AFS open source can only be a good thing - it will propagate a great technology into large sites where they would shied away from it previously.

Re:Once again...

[drift factor]

When you're at a company where you have to use AFS and have to run an old Linux kernel to get AFS support, you'll realize that this is a good thing. How many times I've wished I had the source so I could compile a 2.2.16 module... Transarc isn't exactly on top things as far as Linux support goes either, so without it being opensourced you're basically just screwed until they decide to compile a module for your kernel version.

Why? CODA

[Anonymous Coward]

Why open source it? Because coda is about to replace it. CODA (http://www.coda.cs.cmu.edu/) is a Free (free software), scalible, distributed file system. It covers every feature of AFS, and goes quite a bit further.

Coda is reaching a point of stability and availablity that it's nearly ready for widespread production deployment.

It appears that IBM has sensed the death of AFS coming and is now trying to buy some time.

I suggest we all just ignore this AFS announcement and stay focused on coda.

Re:Why? CODA

[Mr. Frilly]

Yeah, but can I use CODA now?

I've been wanting to set up a centralized, non NFS based file system for my lab for years, and CODA has been "almost there" for a while.

Is anyone seriously using CODA in small production deployments yet? I'd love to know, 'cause I'd love to use it....

Re:Why? CODA

[generic-man]

CMU itself uses AFS as its file system. Maybe now that AFS is open sourced, there will be better clients to use on campus for it (read: any clients at all).

Re:Why? CODA

[paggietto]

Actually there is a client for NT. It is a little buggy but does the job (and it's a blessing having it, I can tell you!). It works with win98 too, but requires a NT machine to act as gateway. There is no client for win2k though, you're right here. It's expected in the third quarter, according to Transarc's support.

Silverwear

[oman_]

Hrmm....

'What's actually going is that they are forking the code, as not all of the code can be opened for "technical or licensing reason". '

I guess if you can't knife it or spoon it... then just FORK IT.

Just a note...

[jd]

JFS has been around for a while, but it's still very incomplete, and the CVS server has yet to deliver a copy of the sources without crashing with all sorts of read permission errors.

Porting AFS as well might slow JFS still further, until both teams are tripping over each other.

(If anyone from IBM is reading this, the problem is a read lock error on linux-2.2.12/fs/jfs/utils/extendfs)

I would -love- it if IBM saw fit to put JFS and AFS on Sourceforge or some similar development site. Open Source projects should not simply be Open, they should be =SEEN= to be Open. A problem I suspect many a project has faltered over. It would also allow developers to concentrate on the development and not on maintaining servers as well (unless they wanted to, which is different from feeling like they need to!)

Re: Lotsa licenses -- TMTOWTDI

[InitZero]

I'm just wondering about everyone wanting to write their own license.

{sigh}

When Larry Wall notes 'There's More Than One Way To Do It', we cheer and write folk songs. When IBM says that the GPL isn't right for what they want to do, we get a bad feeling in the pit of our collective our stomach. Why?

Not everything needs to be GPL. Not everything should be GPL. Let's not make the license the issue. Let's talk about what a great product AFS is and how much a pain in the buttocks it is to configure it correctly.

InitZero

Re:Ha you lame troll

[mayonaise]

It'd be nice if we didn't have to check the links before clicking on them (not that it's very harmful anyway, although i would rather not click on "bad" links). =)

May be the best they can do, and they do a lot...

[driehuis]

I just wonder, why would anybody want to partially open-source a product?

The wonderful ferroconcrete world we live in has more lawyers than rats. There are patents underlying the most obvious software designs (yes, a simple lawsuit showing prior art will defeat three quarters of them, but I for one won't spend my life savings on them, and companies with pockets that are deep enough prefer not to invalidate competitors patents for fear of getting blasted themselves).

Patent issues aside, there's the legal debate about licenses. If we (the Open Source developers) cannot put our legal squabbles aside (my license is more free than yours -- no, mine is), how would anyone expect to put big business to put theirs aside? Beside ego, they've got shareholders to take into account.

I've been mighty impressed with IBM's venture into the Open Source arena. I think they've taken the boldest steps of all. It's not just half-baked Java stuff (with tremendous investments behind them) or stuff without direct revenue potential (like jfs, which they couldn't sell as long as competitors think their mouse trap is better). If you search for "IBM Visual Data Explorer" on www.ibm.com, you'll get a price list with a rather hefty price tag (and if you dig deeper, you'll find an impressive array of Fortune 500 companies and research institutes that paid those prices and got their moneys worth). If you look at opendx.org, you'll see the same software, free. The stuff is awesome!

Whatever their motivation, I rate IBM highly for its commitment to Open Source. It's a rather stunning move, given their revenue streams and the fact that they spearheaded the move from free to paid-for software eons ago.

Conspiracy, anyone?

[driehuis]

DES? Oh wow -- there *must* be truth to the rumor that the original DES implementation by IBM still has comments from the NSA explaining the weakening of it then :-)

The _real_ reason why they forked it

[yerricde]

if you can't knife it or spoon it... then just FORK IT.

They couldn't "spoon it" because... (everybody say it with me) <matrix>there is no spoon.</matrix>

adopt a NORMAL bird
<O
( \
XGNOME vs. KDE: the game!

Mozilla

[yerricde]

Releasing only part of the codebase as free software is what Netscape had to do to get the first Mozilla releases out the door. If you remember, half of Netscape 4 was proprietary licensed crap, which is part of why it took Mozilla so long to get a working Internet client out the door.

GO MOJIRA!
<O
( \
XGNOME vs. KDE: the game!

At this rate...

[jd]

Linux will have more Filing Systems than Microsoft has installed platforms!

On the other hand, if somebody will get round to Kenobi-izing Linux (OB1 is SGI's B1 set of patches, which need porting), then AFS + OB1 could make Linux THE platform of choice for secure installations.

Finally, if the Linux kernel is audited, we could see Open Source =genuinely= dominate the world, as it'll be the only widely-usable OS that is secure and stable.

(For BSD purists, I accept that OpenBSD is one very secure system, and TrustedBSD & SecureBSD are extremely promising ideas. However, general users have enough trouble handling real life. Why else print directions on a packet of toothpicks??? They are certainly not going to contend with all of the challanges the BSD installers have to offer, never mind the subtle but significant change in mindset. BSD, for now, is mainly a server OS, which means that it isn't "widely-usable". Usable, sure. Secure, certainly. Stable, without a shadow of doubt, but not widely-used.)

Re:Once again...

[sirwired]

Why do you say that IBM is not giving away anything substantial? Have you even looked at what they are giving out? When commercial, low-level, products are "opened", it is quite common for some parts to be witheld because the company simply does not exclusively own the rights to everything in the code. It probably has nothing to do with corporate treachery or evil PR minions, just simple licensing rights over which the IBM Software Group has no control.

Re:Once again...

[chris88]

Dude, your talking about the company that's donating millions of dollars to the open source community, the company the continues to make damn impressive break-throughs with processors, and the company that's porting Linux to a good deal of their massive servers.

Them opening up this file system can be useful to everyone, despite the fact that this may not affect IBM one way or another.

But to mention IBM and Micros~1 in the same sentence is almost criminal.

Glancing at the FAQ

[Hairy_Potter]

I can see that AFS allows client side caching, for better performance.

AFS has server based drive mappings instead of client based drive mappings like NFS does.

AFS has kerberos for better security.

There's more at the AFS Faq.

Re:Once again...

[bgalehouse]

If they wanna do this instead of a traditional marketing campain, I'm not gonna complain. I hope it does exactly what they expect it to - so that they do it again.

Seriously though. AFS is everything NFS fails to be. Load balancing, client side caching, easy client config, and security. AFAIK It has been a marketplace failure. They should have at least made the client free years ago. Certainly, IBM won't likely notice the lost revenue.