Slashdot Mirror

"HiveCache's revolutionary SwarmBackup and SwarmStorage technology give you high-reliability backup/restore and data storage services by storing data in the free disk space on desktop PCs in your enterprise. HiveCache technology uses peer-to-peer technology to build a reliable and fault-tolerant distributed storage mesh for your backup data, eliminating the network bottlenecks usually associated with network backup systems and without forcing you to purchase costly server storage that will quickly become obsolete."

A really well designed system for backups (not a RAID replacement, but that did not seem to be teh question).

Many responses, even highly-rated ones, seem to be talking about simple replication via NBD (worst-written code I've ever seen) or DRBD. That's not the same as what the original poster was asking about. Neither are fully-distributed but non-transparent file stores such as HiveCache. AFS/DFS/Coda/Intermezzo are probably the closest in the sense of being both transparent and resistant to failures. There have also been a couple of very closely related projects at Microsoft (Farsite and Pastiche) but I'm not sure if there's anything you can actually download and use.

Hivecache is an outgrowth of the Mojo Nation P2P project. Mojo was mainly a file sharing environment; Hivecache is pointed towards business data backup environments (partly because Mojo didn't reach the ...5 Profit!!! stage...)

Hivecache is a P2P distributed backup system that grew out of Mojo Nation. Files are encrypted and shredded into multiple RAID-like pieces, so no individual piece can be used to reconstruct the original data. You don't know what's on there, and you can't find out, because you don't have the information to do it, which provides you some protection as well as providing protection to the people whose data you're storing.

A correction, Mojo Nation's URL was always mojonation.net. Someone else is there now but it's a related company, another effort by MN founder Jim McCoy.

That's exactly what the Mojo Nation folks are doing now. Info at http://www.mojonation.net/.

several emntions of patents on the web page.

I only see mention of a patent (singular) pending on their "MojoNation" page. Where else do they mention a patent or multiple patents?

It looks to me like it is US patent application number 20010037311. I am definitely not a lawyer, but it looks like the patent is on a method for determining how much use each computer gets to make of the system based on what they provide to the system, and not on the concept of P2P backup in general. I certainly hope I'm reading this right because I have my own P2P backup software that I'm about to release and I don't want to run afoul of their. I know there's plenty of prior art for P2P backup in general out there, but most people don't want a drawn out legal battle even if they're right.

The information is not 'unencrypted'.. RIGHT THERE on the front page of the site. "Strong cryptography protects these files from prying eyes..." The data is encrypted, then chopped up into little chunks, and then the data (and share map) is spread out amoung multiple hosts (like handing out pieces from a broken bottle). The amount of redundancy you want for your little data-cubes is also selectable.

No, you missed it. Click on the 'MojoNation' Hive-Hex tab and you will find a link to the LGPL sites of both the EGTProtocol and the MNET verison of MojoNation.

• Higher security
• Less network usage for verifications
• Good network performance even in case of a few percent of lost shares

Is BitTorrent basically the same thing as the transport layer of MojoNation (apparently discontinued, but with parts still alive as MNet>?

Sounds like this is similar to what MojoNation is/was trying to do. Their site doesn't seem to be responding right now, but here's the Google Cached version of the technical docs.

Should I guess the missing 40% from the available 60%?

Yes! Error-correcting codes will make it possible to guess the whole file from fragments that add up to 50%. Mojo Nation already does this.

A while back someone told me about Mojo Nation and it seemed pretty nifty. The idea is that users get "mojo" for contributing to the network as a whole, which they cam then "spend" on services that other users provide. It's trying to solve the problem of "freeloaders" on networks such as Gnutella who use lots of bandwidth with searches but basically bog the network down. Capitalism meets P2P. It's kinda-sorta decentralized in that files that are shared are broken down into little (redundant) bits that are stored for retrieval on lots of hosts. Again, neat idea, but the problem I had with it was that it cost mojo to share files! It costs mojo to do anything that uses bandwidth, basically, so users are implicitly discouraged from sharing. Oy vey.

Aside from the more scalable architecture offered by napster and fasttrack in comparison to gnutella they also had a major advantage in user/freeloader ratios.

I'd guess that 90% of napster users went with the default installations that allowed the client programs to scan their hard drives and automatically share all mp3s. Furthermore, I'd guess that a similar ration never had any siginificant cognition about the FT or napster clients continuing to run as background processes when they 'exited' the program.

Gnutella has a real reputation as a freeloaders network and it's not surprising. Many of the clients do not stay running when you close them (and even if something like LimeWire did, I'm loathe to have a huge chunk of memory taken up by a bloated JRE). Furthermore, a lot of the clients don't do a good job of making it extra work to *not* share your files. In the original gnutella client for windows as well as current incarnations of gtk-gnutella, you have to explicitly enter the config screen and tell the program which directories you want to share. For a lot of people with weak ethics or concepts of fair exchange that extra step is just enough to give them an excuse to be a leetch on the network.

It is intriguing to see what happens as more and more clients are punishing freeloaders in even the most rudimentary fashion. For instance, Limewire now has an option that will allow you to set preferences against those sharing less than a specific number of files. This in theory should encorage people to share their directories especially as the controls become more fine-grained and reward those sharing large collections/bandwidth with preferred access in exchange for offering their services to the network.

It's a little less cumbersome, if also a little less elegant and perfect, than the mojo nation system of a credit based economy. However, as in the curren tstate of most p2p, it is potentially missing the bigger picture by concentrating only on the health of the community qua community and ignoring the potential problems of freeloading within the scope of society. Namely, rewarding artists for their work.

P2P gains some respect if you accept the arguments that it encourages more CDs or concert tickets to be purchased, and thus greater rewards to the artists. This is no doubt true for many, however there are also plenty of people who haven't bought an album since they got broadband,a nd these people are gaining unfairly on the goodwill of thsoe who do have a sense of ethics on fair exchange with artists.

What I'd like to see is a similar system to the idea of giving preferential bandwidth to those who share that is integrated with sites like fairtunes. It seems possible that a p2p protocol could be developed or extended to check a user who is requesting a download for tokens representing 'tips' that they have made at fairtunes in exchange for the pleasure they have received for downloaded music. It would definitely add some overhead to the protocol to authenticate the tokens against a fairtunes server and/or public key, however offering perferential performance on the network would serve as a gentle pressure to encourage a more ethical, and arguably a more sustainable, system which artists would have less trepidation of participating in and may very well be able to earn reasonable incomes from if their music is enjoyed by enough people.

For example, require that if people have nothing to share that they agree to devote a small part of their disk space to cache popular files which automatically get placed there by the system.

I believe this is the approach adopted by Mojo Nation. Resources cost mojo to access, but you gain mojo by providing resources (CPU cycles, storage space, network bandwidth) that others are willing to pay mojo for, whether it is your own content, or caches. It scales nicely for provider bandwidth, and allocates storage in a distributed fashion, you say how much space you want to give it, but not what you want to store there. But it's not strictly anonymous, and they respect intellectual property rights.

There are two systems that are sort of working towards this: OpenCola and of course, Mojo Nation

In the case of OpenCola, they ask that after you leech a file from the swarmcloud that you keep your client open so that other people can use your pipe. In other words they expect you to be community minded out of the goodness of your heart or belief in Karma.

A solution would be that everyone eventually gets broadband by default so that there is no reason to stop your client, ever.

This way, it would be possible to contribute your pipe even when you are asleep. In other words, to make it so that it doesnt matter if someone is using some of your bandwidth.

Europeans use the American Cloud when they are asleep, and vice versa.

This is the first P2P application that I've seen with encryption built into it.

Besides, Intel's library isn't an application.

Burris

Look at Mojonation for an implementation of exactly what you just described.

The technique is called Forward Error Correction. I don't know much about it, but I know that you can do things like break up a file of N*B bytes into 2N blocks of B bytes each and then be able to reconstruct the file from any N of the 2N blocks. The GPL'ed Mojonation system uses it, if I recall correctly, to distribute each 32kB chunk of data into eight 8kB blocks allowing reonstruction from any four of them.

Keepin things in sync when they are stored in a distributed fashion is a bitch. Still, if you merely want to save bandwidth, you may want to give Mojo Nation a try, which works quite well already.

Foraging techniques (similar to pheremones) are used to propagate meta-info describing how to contact and find other agents within the mesh and the self-organizing, emergent nature of the filesystem owes a debt to random algorithms and similar techniques from ant colonies. The swarm downloading feature we pioneered is also derived from how an ant colony gets food back to the colony, with lots of expendable agents taking individual paths during the delivery task.

While it is sometimes not obvious to casual observers, I think you will find many distributed systems which take their cues from the natural world if you peek under the hood.

Many hands make light work...

I'd like to know what legal problem the LGPL poses. I'm not aware of any, and as my project Mojo Nation is released under the LGPL, I would be concerned if there were any.

Oh -- well I suppose the LGPL does prohibit you from altering the code and releasing your variation without contributing the changes back to the open source project. I could see a lawyer considering that to be a big problem.

Regards,

Zooko

It isn't true that Mojo Nation is "not focussed on performance". I'm one of the Mojo Nation hackers, and we care about performance. It is true, however, that Mojo Nation is pretty complex, providing both data transfer (using a "swarm" like technique), and data storage, and a queriable search engine. The end result is something like a distributed, non-deletable World Wide Web. (Sort of like Freenet plus persistent data, or the earlier concept of Ross Anderson's "Eternity Service".)

Performance isn't that great on Mojo Nation right now, but it is good enough, in my experience, for daily use.

I'm pretty excited about the Swarmcast open source release, both because I think Swarmcast is a cool app in itself, and because I can now start taking ideas and code from Swarmcast to put into Mojo Nation, and vice versa.

In the long run, both Mojo Nation and Swarmcast will improve because of this sharing, as will other related open source projects like Freenet and Free Haven.

Regards,

Zooko

P.S. I've been talking with Justin Chapweske, the Swarmcast, lead, on irc.openprojects.net, and he's already pointed out a potential bug that we need to avoid in future versions of the Evil Geniuses Transport Protocol...

The previous poster makes a vary good point. The system seems to depend on a corum which may not exist. How is corum maintained? This is unclear.

This is the same problem currently being experienced with Mojo Nation, a P2P file sharing system which implements a similar method of delivering blocks of data from various locations then re-assempling the requested file.

Perhaps OpenCola's solution to this problem is explained someware on their website but I looked and didn't see it.

--CTH

--

Coool.

Nice one, Orasis! Hopefully someone will be quick enough to mirror the next release on Mojo Nation as well.

I'm looking forward to the imminent open-sourcing of Swarmcast. I think that the emergent ("p2p") networks are maturing enough that we will start trying to link them together soon. Perhaps the second O'Reilly p2p conference will spark some work in that direction...

Regards,

Zooko

You may want to check out Mojo Nation. It attempts to solve some of the problems with Gnutella. It's interesting, but not in widespread use. It's actually a lot more like Freenet than Gnutella.

And finally, as to who has committed a crime? You commited contributory copyright infringment. Or at least, that's what the RIAA will say (and if what's going on with Napster is an indication, the courts will probably agree).

Guido:

Please take a deep breath and go in for one last go-around with the FSF lawyers. Pretty please?

As far as I could tell, the remaining issues are just "legalese exhaustion" on your part rather than actual conflicting goals. Maybe you could deputize a legalese wrangler to finish negotiations for you, or you could take a month-long break in which you never think a single thought about licenses, and then you go back and finish the negotiations.

This is really important to me, although I am not a GPL fanatic, because if it remains the case that the licenses are (allegedly) incompatible, then there will be lots and lots of people who will refuse to combine GPL code with Python, and that would really suck.

For example, I want to package up my open source application Python, Mojo Nation to be included in Debian. This would be a way to reach hundreds of thousands (? maybe fewer. Anyone know how many Debian users there are out there?) of highly clueful users and hackers who would otherwise never install Mojo Nation. The Mojo Nation code source code itself is under the LGPL, and some of the open source libraries that it uses are other under free licenses. Would this cause a legal conflict that would force the debian people to keep it off of their servers? I don't know (since it is LGPL instead of GPL), but I would feel so much better if the Python license were officially GPL compatible.

Regards,

Zooko

Fascinating, but I want the next generation platform to evolve, not out of Sun, Microsoft, IBM, HP, Exodus, etc., but out of Mojo Nation, E, Chord, FreeNet, etc.

Open source projects, with ambitious goals for self-healing, self-organizing networks, tolerant of diversity, resistant to any conceivable attack, and free from the manipulations that mega corps inevitably introduce in their unceasing quest to gain monopoly power.

Excellent point about the bandwidth issue. If Libranet wants to (shock of shocks) actually get paid for the service it provides, perhaps a p2p solution such as Mojo Nation is the way to go. At the very least their bandwidth problem is diminished while setting themselves up with a Pay Lars-style tip jar.

"just connect this to..."
BZZT.

At the height of Slashdot's reporting on Napster (twice or thrice a week) I couldn't understand what relevance it had with regards to being "News for Nersa" or "Stuff that Matters", some service that is primarily used to pirate songs was getting sued, big deal.

Now that Napster has been rendered useless as a file sharing service by the RIAA and a court of law, why is Napster still news? Everyone I know has moved on from Napster and now uses a service that surpasses Napster's poorly designed service in one way or the other. For simply sharing and obtaining music there are iMesh, Audiogalaxy, Music City, Ohaha, Gnutella and a host of others. For uses of P2P beyond simply grabbing MP3s we have Mojo Nation, Freenet and Publius.

Why doesn't slashdot start reporting on these systems instead of beating the dead Napster horse?

Don't think so? You better tell MojoNation then. They're building a complete online p2p based micrpayment system. Looks very promising. There are a number of other p2p efforts, but most are non-commercial.

-----
"People who bite the hand that feeds them usually lick the boot that kicks them"

Burris

I woke up this morning to find about 500 messages in our trouble ticketing system about this. Heh.
(I'm one of the cofounders of HavenCo, and the CTO, if you didn't already know; I'm also an active slashdot reader (what else do you think we do for fun out in the middle of the north sea?))

First of all, www.fairtunes.com is hosted in Canada, is slashdotted, and isn't our fault!

Second, I can't comment on confidential discussions with customers w/o their permission, but yes, from looking at the fairtunes site, it looks like they're trying to raise money to pay for a year of service on one of our boxes with the goal being to host an offshore OpenNap server. I'm personally a user of napster (although I mainly use Mojonation now. We definitely would like to have them as a customer -- what they're doing doesn't violate our AUP, and we're happy to offer service to anyone who will pay. Of course, what they are doing is NOT being done by us; if they choose to host with us, it's still their responsibility.

We have network connectivity through multiple providers around the world, and can easily add more. We assign customers a /29, so if any government decides they must protect their citizens from human-rights information, music trading services, etc., they would need to block the customer's /29, affecting only that customer. I personally think the chances of IP blocking at the borders of a country are pretty slim in any marginally free country -- it's difficult from a technical perspective, would be widely opposed by users, and is generally not worthwhile.

As for HavenCo's service, we've been up since May 2000, and now that we have high-speed low-latency network, fully debugged power systems, etc. we're offering commercial service to anyone who is interested and obeys our AUP. Our pricing is standardized, and is USD 1500/month for a 2U box with redundant power, cooling, 24x7 network monitoring, armed security, etc., and 256kbps of Internet bandwidth (local 100baseTX is free, so people can offer services to other HavenCo customers without paying for bandwidth). We charge a USD 1500 one-time setup fee, and USD 3500 for hardware (we can use any high-quality 1U or 2U box, and pricing is US cost; we don't try to make a profit off hardware, but we can't accept non-rackmount, low quality, etc. stuff). We have about 3-5 days lead time, from receipt of payment, before we can have a server up and running, and as long as you're not doing spam/spam support, child pornography, or hacking from our machines, we'd love to have you as a customer; contact sales@havenco.com for more info.

We're in the middle of a web redesign, and have been trying to focus on getting services fully up, rather than getting more press, but we're about to begin a big sales and press push. This is a bit earlier than was planned, but now that people are getting slashdotted, might as well post. :)

What about MojoNation ?

Burris

All ISPs should be interested in running their own set of "P2P servers" themselves and they'll come out ahead without needing to raise prices.

A perfect example is mojo nation: if an ISP runs several boxes with mojo brokers covering a large or full part of the mojo data address space then both the ISP and its users will benefit. Users mojo brokers will naturally prefer the ISPs mojo brokers because they have a much lower latency and a higher reliability compared to brokers elsewhere on the net (outside of the ISPs internal networks). That means less overall outside/internet bandwidth usage directly by the users due to the caching the ISPs brokers effectively perform. It's a very large content neutral distributed caching system. In the end is saves the ISP money!

People misunderstand "P2P" in its buzzword hype. ISPs are Peers too!

You are quite right about trust. The term is "Distributed Trust Metrics" ... At the O'Reilley P2P conference, Zooko of Mojo Nation and Raph of Advogato gave a presentation on "Attack Resistant Metadata." Presumably a system of that sort will be integrated into Mojo Nation in the near future. For it to work your system needs hash based identification of data and signed metadata.

Burris

While it is probably not very important to the people reading this, there be dragons ahead for this project that I do not think the implementor is aware of. We implemented a system very much like this for Mojo Nation to achieve the swarm distribution (parallel downloads) which is one of the key features of our technology. Windows does not like to hold lots of open connections and you quickly eat up local resources and run out of file descriptors. It works like a charm under Linux and other "real" operating systems, but backporting this to make it available to the un-enlightened will be a very, very unpleasant task for whomever tries to actually implement this. jim

Is it possible?

Yes!

By using an internal microcredit/payment system (called mojo) and localized reputations Mojo Nation aims to do exactly that. Better connected brokers (peers) will naturally become more "server like" due to having a better uptime, lower latency and a lower mojo cost overall for other brokers (peers) to use.

The resources in the system are allocated dynamically. No strict heirarchy needs to be defined, it will establish itself appropriately for each individual peer as it is needed.

PS a new version (0.950) was released today.

What is Mojo Nation?

Mojo Nation is a revolutionary new peer-driven content distribution technology. While simple data distribution architectures like Napster or Gnutella may be sufficient to allow users to trade mp3 files they are unable to scale up to deliver rich-media content while still taking advantages of the cost savings of peer-to-peer systems. Mojo Nation combines the flexibility of the marketplace with a secure "swarm distribution" mechanism to go far beyond any current filesharing system -- providing high-speed downloads that run from multiple peers in parallel. The Mojo Nation technology is an efficient, massively scalable and secure toolkit for distributors and consumers of digital content.

I'll be there, giving a presentation on a subject dear to the hearts of slashdotters: how to find the good stuff amidst the bad stuff, even when some of the moderators are malicious.

Of course, my real goal will be to find out how other p2p systems work in order to use the knowledge to improve Mojo Nation. Therefore one of the talks I'm most looking forward to is Wes Felter's overview of technical decisions in the deployed p2p networks.

Regards

Zooko

Mojo Nation runs on Linux/x86, Windows 95, Windows 98, Windows ME, Windows NT, Windows 2000, FreeBSD, OpenBSD, NetBSD, Irix, Linux/Alpha, Linux/PPC, Linux/Sparc, and MacOS X. It is written in Python, with the performance-intensive parts and the standard open-source libraries written in C or C++. Last I checked, Java 2 still isn't available for FreeBSD, not to mention all the other platforms.

I hate to fan the flames of language holywars, and I don't think that Python, C, or C++ are the be-all and end-all of programming languages, but this "Java == portability" myth is starting to bug me. Currently, I would only choose Java for portability if I were aiming at the various embedded systems out there, and even then I would probably end up going with C or embedded C++.

I do appreciate the difference between recompiling versus shipping byte-code, but most of the actual program logic in Mojo Nation is in (Python) byte-code. And anyway, for a "download, install and run" application, the only cost of having platform-specific compilation is that users have to click on the correct hyperlink on our download page.

Regards,

Zooko

Mojo Nation runs on Linux/x86, Windows 95, Windows 98, Windows ME, Windows NT, Windows 2000, FreeBSD, OpenBSD, NetBSD, Irix, Linux/Alpha, Linux/PPC, Linux/Sparc, and MacOS X. It is written in Python, with the performance-intensive parts and the standard open-source libraries written in C or C++. Last I checked, Java 2 still isn't available for FreeBSD, not to mention all the other platforms.

I hate to fan the flames of language holywars, and I don't think that Python, C, or C++ are the be-all and end-all of programming languages, but this "Java == portability" myth is starting to bug me. Currently, I would only choose Java for portability if I were aiming at the various embedded systems out there, and even then I would probably end up going with C or embedded C++.

I do appreciate the difference between recompiling versus shipping byte-code, but most of the actual program logic in Mojo Nation is in (Python) byte-code. And anyway, for a "download, install and run" application, the only cost of having platform-specific compilation is that users have to click on the correct hyperlink on our download page.

Regards,

Zooko

"What about if by providing bandwidth, a user was credited some way towards their own napster subscription, i.e. if you serve over a certain threshold, you get a 50% discount on your napster subscription? Would that change the incentive level for being a host in a subscription napster world?"

Sounds like MojoNation. MojoNation is another P2P file sharing system, but it has costs ("Mojo"). You get Mojo by sharing your resources (disk space and bandwidth). You pay Mojo to have your content on the system (which is in turn stored, encrypted, on other user's harddrives). Then you pay Mojo to get content. Interesting concept, although I've never personally run the software.

You'll be glad to know that we stopped requiring the weird proxy configurations a couple of versions ago. To run the new version is simply: "tar -xzf ./mojonation-*.tar.gz ; ./mojonation/evil/Broker --no-tail ; netscape ./.mojonation/broker/intropage.html".

The windows version is apparently even easier to install -- it has some kind of gui point-and-drool installer that untars and executes these things for you, or something. ;-)

We also recently fixed several really nasty "distributed bugs", which didn't show up at all on any particular broker, but which caused the entire network to degrade. Nowadays Mojo Nation is much faster and more robust than it was last time you tried it.

Regards,

Zooko

I'm presenting a talk at the O'Reilly p2p conference entitled "Attack Resistant Sharing of Metadata".

It's based on an idea of Raph Levien's, somewhat similar to the Advogato trust metric. Basically, you only trust meta-data from your friends, or from people whose meta-data has been good in the past, and then to a lesser extent you trust their friends, but you dynamically adapt if someone starts distributing bad meta-data. We can't really prove that it will work, but it has some promising characteristics.

We are going to implement it on top of Mojo Nation.

Regards,

Zooko