Slashdot Mirror

Anyone considering making a "Time Machine" for Linux should look into the work of the InterMezzo File System layer. It is not a classic file system due to the fact that it doesn't store anything itself. Instead it sits as a layer between the applications and an actual storage file system (such as ext3) to track all changes as they occur. While rsync must do a fairly heavy weight scan for new/changed files, InterMezzo is able to skip the scanning step due to having already tracked these files at the time they where written.

There have been some efforts in the area of networked filesystems with disconnected operations. I remember checking out AFS, Coda, and InterMezzo years ago. At the time, I found something wrong with each of them, but they may have improved since then. Of the three, I think Coda is your best bet.

I think the idea here is that the home directory is mirrored on the internal hard disk AND an external device of some kind. Then again, I think InterMezzo has prior art on that. So this may seem like a novel idea for your average PC user, but it's not novel enough to warrant a patent.

Of course, it's not like the USPTO hasn't ever issued a patent on something that should never have been patentable...

Coda is long dead, never lived up to anything. Most Coda people have gone to Intermezzo, which is still not really usable yet.

Go use Openafs

Finkployd

Check it out...fits the bill nicely.

• Open Mosix Transparent process migration, intended for clustering.
• UML Self hosted virtual machines.
• Adeos Nanokernel.
• RTLinux Realtime microkernel/macrokernel work. Hell, it _is_ patented.
• ReiserFS Filesystem based on dancing trees, with a plugin archtecture.
• ZisoFS Transparant handling of compressed ISO9660 filesystems.
• Seperate LLC stack. Logical Link Control is handled by a single stack, rather than embeded into underlying protocols.
• InterMezzo Distributed filesystem, with network interrupt transparacy.

AFS is for distributed computing, GFS is for fault-tolerant cluster computing

ah. if you're right about that, then this is probably still not quite what i want.

i've been wanting a distributed, fault-tolerant filesystem to play with for a while now; NFS is getting old and clunky. i want something i can share between several machines, that would keep a local copy on each machine involved, and that could seamlessly tolerate disconnects/reconnects of machines in the cluster. ideally, i'd also like it to do security, authentication and encryption decently.

i haven't found anything. Intermezzo and Coda seem to come closest, but they're both more research project than solid product. OpenAFS seems stuck in the same niche, and all three of them are almost-but-not-quite POSIX compliant. (i'm not really sure if non-POSIX semantics would be a problem or not, but i'm a pessimist; i'd like to take on as few problems at a time as i can.)

GFS seems to hold a lot of promise, and its Sistina heritage is a good sign, but if it can't (easily) replicate files across a network for me, then it's not quite what i'm looking for. ah well, maybe that Unison thingy i heard about can be a poor man's substitute...

How does this compare to other SAN hacks like Inter Mezzo, coda or the Open Mosix File System (find text: mfs)?

Intermezzo and Coda both do this, but I don't think there's any windows versions available. There are some Microsoft things available too, but obviously those aren't for linux. NBD (which everyone else has mentioned) isn't distributed, so that's not really what you're looking for.

What you might be able to do is put together a microcosm of Freenet or something like it, running on just your home computers. There may be other Peer-to-Peer solutions available that are faster/more stable. Do some searching on peer-to-peer distributed storage networks. I know of two researchy ones: OceanStore and Chord. Good luck!

Intermezzo is designed for this and a bit more - if one of the machines is a laptop you can take it away and work on it, and it'll resync when you get back.

It isn't particularly high-performance, from what I know, and may be more complexity than you need.

Sounds like Coda or InterMezzo would fit the bill, but they won't address non-linux systems directly. You'd have to export the InterMezzo file systems with Samba and mount them on the MS Win boxes.

Unfortunately, you have two issues to deal with. First and foremost is simply getting access to your data, and the second (and harder) is making that data usable to whatever os/apps you may be using.

For example, I use Konqueror on my Linux box(en), but use Mozilla on Mac OS X. How do you get bookmarks from one browser to work on the other?

As for the first problem, NFS works enough for LANs, but I sure wouldn't want to use it over the public 'net.

Coda seems promising, but I've never found a distro that actually supports it, and there's a fair amount of manual stuff you have to do to use it. I've never managed to get it working properly, but from what I understand, it's somewhat similar to CVS in that you have to update/commit. This has the advantage of working when disconnected.

InterMezzo also has some promise, but I haven't played with it as of yet.

USB keys seem like a better idea, until you realize that if you lose the little sucker, you're SOL. So, keep backups. Also, I keep a ridiculous amount of stuff in my home directory - multiple GBs - so being able to move a few hundred MB at a time just doesn't work for me.

If I understand LVM correctly, it doesn't do what the poster wishes. It will allow him to choose which logical subdirs reside where, but not have all his files everywhere.

It seems like a distributed filesystem, like InterMezzo or Coda would be a better match. That way his files are everywhere and the fs automatically manages updates regardless of where changes are made - including after disconnected operation.

No need for new projects -- already good distributed filesystems that you can set up big servers with
afs? (or here)
coda?
intermezzo?

CMU, for example, uses AFS campus-wide. Your login scripts and dotfiles and whatnot all reside in your home directory (on AFS) so preferences migrate with you.

You can make things world-readable, and because AFS has a global namespace, anyone can see them. If I do research at MIT as well, I just need to grab a Kerberos ticket from their KDC and start using my files over there.

Just plonk a server in place, put an array of 100GB drives in place, make things readable by whomever you want, and you're good to go.

If you want a system designed with fancy automated caching that people can use without dicking around with Kerberos, freenet's a good choice. Of course, there's no guarantee that the data will stay around, but cest la vie.

As happens too often, this proposal concentrates entirely too much on distributed computation, and pretty much ignores the problem of distributed storage. They're quite different problems, each requiring its own solution, even though it's intuitively obvious that any true "Internet Scale Operating System" would have to deal with both.

If you're interested in this "other half of the problem" here are some links:

• Farsite (Microsoft; focus on many nodes, not long distances, but still relevant)
• OceanStore (UC Berkeley)
• CFS (MIT)
• Publius (ATT/NYU)
• Intermezzo

There are many more. The bibliographies for the above will mention many earlier systems, while a quick Google search for these project names will show more recent ones.

InterMezzo shold be able to do this.

If your home computer is online 24/7 (which is presumably is if you're on broadband) t's cooler to use SAMBA, AFS (or here), Coda, InterMezzo, NFS, or the unfinished Lustre. If you're not big on effort, set up an http or ftp (or gopher!) server. That way, you have an automatically up-to-date menu of your mp3s, where you can access all your music any time you can connect to the 'Net.
This box is just itching to be a Coda server.

The journaling capability isn't just about improving fsck times. Check out Intermezzo. The company that's working on Intermezzo (Mountainview Data) has some other cool sounding products that take advantage of some tricks only possible with journals (ie, taking snapshots of filesystems).... looks like mostly vaporware right now, though. I'm sure there are some other applications, but I'm not imaginative enough to see 'em. :-)

First, don't get me wrong, I'm not against the idea at all and I do think that Coda is really great (on paper).
Second, I'm also surveying Coda's enhancements for more than 2 years now and I really think it's a great software with lots of people behind but my question really is : "can Coda achieve enough scalability and stability in order to be installed for 1000+ clients ?" (the FAQ and the latest changelog doesn't help me think it could handle 2500 clients without any problem at all...)

Because frankly, when I hear all the feedback from different people using it 'in real life', it's not all so bright...

Besides, what about Intermezzo that is derived from Coda with most of it's features because of such "unforseen" scalability issue that were not planned back when the developement began... (appart from the fact that Intermezzo seems a little beta to me)

You can also find some good information concerning Coda/Intermezzo/NFSv4 here

As a conclusion, why not having one or more big editor (IBM, SGI, HP...) put some big bucks on the table in order to help any of these 2 projects being finalized through funds (SourceXchange, CoSource or any other way)

It is my hope that it will prove a lot better integrated with Linux(-based GNU systems :-) than Coda. If it fulfills it promise, I have at least a hundred machines which I am looking to install it on.

Matthew.

The Inter-Mezzo file system would definitely be a good things for you to check out. Sounds quite a lot like what you are looking for.

The InterMezzo site:
http://www.inter-mezzo.org

...it's essentially a layered filesystem -- sort of a filter driver. It adds its own "journaling" (but not for recovery purposes) to any existing filesystem. The file server is written using POE - the Perl Object Environment (originally written to make making MUDs easier). It's a single-threaded state machine-type server that uses non-blocking I/O to multiplex active requests.

A filesystem written in Perl, of all things...

I've never played with software raid or the network block device.. was the network block device around in the 2.0.x kernel versions? .. I doubt i'd see any performance increase- i'm sure the bottleneck is the network & the fact that things often go back and forth more than once before ending up at their destinations.

However, i've just grabbed myself a copy of intermezzo, and it looks like it might be able to do everything i wanted and more. I hope to somehow get my 240 disc cdrom changer into the mix so it appears as a single drive instead of 2 drives and a serially controlable robot. :p From my brief look at intermezzo, i am hopeful.

--sean