We put the meetings on hiatus until we got the commercial implementation released to the public. We did this just last week: http://blog.meritora.com/launch/ . We plan to start having meetings again within a month or two.
Bitcoin isn't a technological competitor, it's a currency. Bitcoin isn't going to be the last currency of its kind, there will be many Bitcoins just like there are many currencies today. Each one is fit for the group of people that uses the currency. The PaySwarm standard is a financial protocol and is thus currency agnostic. We plan to support Bitcoin, and Ven, and a variety of other currencies.
You could argue that Bitcoin is also a protocol, but that is where Bitcoin is fairly weak. Instead of building Bitcoin on the Web's architecture, it was decided to invent a new protocol. While the new protocol works, it's not very Webby, and because of that there is a great deal of heavy lifting that needs to occur to participate on the Bitcoin network. The PaySwarm work builds on top of the Web (HTTP, HTML, RDFa, JSON, etc.), so the financial protocol lives as a core part of the Web. It also doesn't conflate currency with protocol, so it's capable of supporting more than one currency, including the one that will eventually replace Bitcoin.
Regarding your micropayments statement, you're conflating the value of a currency with the divisibility of the currency. Micropayments support is about divisibility, not value. The only thing to discuss about Bitcoin's micropayments feature has to do with the latter, how divisible it is, which is (8 decimal places) vs. how divisible PaySwarm is (10 decimal places). It's not a very interesting discussion, I admit.:P
The short answer is that there aren't a lot of people working on the problem. There are 7,000,000,000+ people in the world. There are 60 people in the Web Payments Working Group at W3C, of which only around 10 are actively working on the problem. It's a hard problem and there aren't that many programmers, systems engineers, standards makers, writers, bloggers, lawyers, etc. that are willing to put in the hard work to solve the problem. If you think this is an exception to the rule, you'd be wrong. There are only around 40 people really working on HTML5... and that work reaches over 1.5 billion people.
We've been working on the Web Payments stuff for 7+ years and I've always been kind of floored at how quickly most tech folks backpedal away from creating a truly revolutionary financial system on top of the Web.
If you're interested in lurking or especially helping, please join the Web Payments group... we need every helping hand that's available: http://www.w3.org/community/webpayments/
Things that are legal in most states: gambling, adult media, cash transfers.
Things that are illegal in most states: sexual services, selling hookers (human trafficking), and blow (drug trafficking).
Payment services tend to avoid gambling and adult media because there is a huge fraud problem with them, and in the grand scheme of things, they're not as profitable as the vast majority of other "safer" transactions. Cash transfers require a huge amount of money to get a license to operate in all 50 states in the US.
If you want to do something illegal, use Bitcoin or cash.
If you want to do something cash-based, we're working on a Bitcoin-like alternative, but it's not a high priority. More on the commercial service here: http://blog.meritora.com/launch/
PaySwarm will eventually support Bitcoin. However, that is a separate issue from the one of doing micropayments. They're two orthogonal concerns. They do have a slight bit of overlap, but not enough to tie the design either of the solutions to one another.
+1 to Devcoin. You might also want to check out Gittip: https://www.gittip.com/
We can support both with PaySwarm (since PaySwarm is currency agnostic). It's also fairly trivial to setup something like Gittip using PaySwarm (recurring payments). More here: http://blog.meritora.com/launch/
The PaySwarm specifications allow anybody to implement the specification and interoperate on the network. So, if your country doesn't have a PaySwarm Authority, there is a huge incentive for somebody to launch one in your country.
In our system, anybody (in any country) can become a vendor. At the moment, we only deal in USD, so if you want to withdraw your money, you need a bank that can talk to the US banking system (many international banks can already do this).
The only thing preventing us from branching into your country is a slew of regulations that we have to follow to make sure that we're operating the service legally in your country, using your currency.
So, I think you partially mis-read what we're doing. We only support USD now because we just launched. Eventually, we hope to support all major currencies in the world. If we don't choose to support your currency, somebody else will. Their PaySwarm Authority (the thing that acts like the bank on the network) will allow you to use whatever currency you want to in your country of origin, and it will be up to them if they want to interface with other PaySwarm Authorities around the world.
The bottom line is: The system is designed to make it such that every currency in the world will eventually be supported if there is a profit to be made in doing so.
In the beginning currencies will be exchanged at whatever the market rate is, automatically. So, if you are sending USD to someone that only has EUR accounts, the amount will be converted automatically based on current market rates and deposited into their account as EUR. The future plans hope to bypass the currency exchange markets for a more direct model, like Ripple, that doesn't have currency exchange fees that are as high as most international banks utilize today.
... and none of those are open, patent and royalty-free Web standards.
You could argue that Bitcoin is such a beast, but it is more of a financial protocol and currency wrapped into one. PaySwarm will eventually support Bitcoin as a currency (along with hundreds of other currencies), so there is no real conflict there.
Sorry, but this is Slashdot. If you're going to link to XKCD, you should at least make sure that what you're linking to is a good analogy.:P
PaySwarm is currency agnostic, so it can support all national currencies, as well as alternative currencies like Bitcoin and Ven. We don't have Bitcoin support in there yet, but it's on the roadmap and we hope to sooner than later. There are regulatory issues that we have to work through. More here: http://blog.meritora.com/
We are building the technology out in the open, transparently. Anyone can join the group. There are no fees, there are no prerequisites for joining. You can read the minutes from every one of the design meetings, and even listen to the audio here (we record everything): http://payswarm.com/minutes/
Why design the financial system in this way? We need to show people that, unlike the way our current financial system is developed and run (behind closed doors), that we're taking a radically new approach to building the basis of the financial network that we hope all of humanity will use. This financial network is open and decentralized, like the Web.
PaySwarm, which is part of the Web Payments work at the W3C, supports micro-transactions. All transactions in the system are accurate up to 0.0000000001 of a fraction of the currency specified. See this for more details: http://blog.meritora.com/
It could count as an in-app payment and I have no idea if the in-app purchase patent you're talking about applies, nor am I going to go take a look at it:
Our experience in this area, after looking at lots of patents, is that they tend to be badly written and/or easily easily worked around. We did file provisional patents for the technology in 2004 to establish prior art for the express purpose of ensuring that nobody else could patent the technology and that we could offer it patent and royalty-free in a Web standard.
The folks working on the PaySwarm stuff believe in data portability, so you own your private data and can take it with you when you leave the system. That's a design goal.
As far as access to your hard drive, that's a bit vague - does writing a cookie to your hard drive count? Same with access to the network, vague. Care to elaborate?
There are a number of open source implementations of clients now:
https://github.com/digitalbazaar/payswarm.js/https://github.com/digitalbazaar/payswarm-wordpress/
Since it's an open, patent and royalty-free spec, there will be open source implementations of a PaySwarm Authority (the things that process payments and move money around in the network) in time.
The mozPay() API is built so that Mozilla has a whitelist of organizations that are allowed to be vendors. You have to get permission from Mozilla to get on that list, and that's not very Webby. That said, Mozilla will be the first to admit that this isn't ideal and that they want to move toward a more decentralized solution. They designed it this way because decentralized payments is a really hard problem and they didn't have time to solve it and launch FirefoxOS at the same time. Luckily, we (Digital Bazaar and other folks at the W3C) have been working on decentralized payments for years and have a working solution that we're coordinating with Mozilla on trying to find a way to get it integrated with the mozPay() API.
Hi, I'm the chair of the Web Payments group at the World Wide Web Consortium (W3C). Just pointing out that the Mozilla mozPay() API is part of a greater push in the standards community to make payments a core part of the Webs architecture. This includes buying/selling digital goods, donations, crowd-funding, all the way to equity and loan-based crowd-financing for start-ups. Note that the mozPay() API is centralized, which even folks at Mozilla will tell you is not ideal. The eventual goal is to create a decentralized payment architecture that is designed for the Web from day one. We plan to put these advanced financial tools into the hands of all Web developers so that anyone with a website or blog has access to this open financial network.
You can read more about the PaySwarm standardization work here, which is mentioned at the end of the Mozilla mozPay() blog post: https://payswarm.com/
Hi, I'm Manu Sporny - the person that authored that article. You ask good questions, but your answers to your own questions do not come from knowledge or understanding of how W3C and IETF work. You seem jaded by the world, let me try and convince you that things are not as awful as you believe them to be. Let me start by answering your most important question:
Since when has anyone been able to make a change to the status quo?
Many people do this at W3C, IETF, WHATWG and many other online communities every day. They help make the world a better place. They lead the Web to its full potential. At W3C, they're called Invited Experts and they are just like you and me. They don't come from large companies or people with deep pockets, they do amazing work and are then asked to participate in standards work at no cost to them.
You are often asked to join W3C because you have a particular expertise in an area. I am an Invited Expert - I've never paid a dime to W3C, but hope to some day because the work they do is so important. In my case, I was working on Microformats, music markup, online payments and structured data in HTML. I approached the RDFa Working Group chair at the time and asked to be a part of the discussion at W3C about RDFa. The chair of the RDFa Working Group at that time was Ben Adida, and he said that he would love to have the Microformats community's input. I was invited to join. I was required to pay absolutely nothing. I've never had to pay anything to W3C and this is because they value expert input on their standards.
It currently costs $0 to join the HTML Working Group at W3C. You could easily have an effect on HTML5 if you would put in the time to read the spec and comment on it. W3C is legally obligated to respond to you - anyone can make a difference, you just have to try. Here's how you can do it: become an expert in something Web related - invent something new or do lots of implementations and gain more knowledge than your peers. Work hard. When I say "Work Hard", I mean really, really hard. You have to write specs, you have to do implementations, you have to be familiar with at least 20-50 IETF RFCs and you have to be passionate about the future of the Web as a tool to help make humanity better.
You don't have to care about what schema.org or I say, but don't belittle the great technical work that all Invited Experts (and paid participants) do at the world standards bodies. The Internet and the Web wouldn't be what they are today without these organizations researching, creating and publishing open, patent and royalty free specifications.
If you contribute to the Web, you are a part of the entire Web community. It's important that you understand that - if you speak, your voice will be heard. As for questioning if the status quo can be changed - it happens every day. I was able to go from relative obscurity in the standards world all the way to chairing a Working Group at an International Standards body based purely on the hard work that I did to get here. It's fine if you want to be jaded about the world, but there is no $8000 fee required to make a difference (just join the mailing list - it's free). Those of us that are not jaded and believe that the Internet and the Web can make all societies better are working as hard as we can to do just that.
[FULL DISCLOSURE: I am the President/CEO of Digital Bazaar, we created Bitmunk - a legal P2P music trading network that uses watermarking to protect both artists and customers].
This isn't a new concept, Bitmunk has been doing it for three years. We have over 1 million songs on our network - it's something that works for both the artists and the music fans. We're one of the only companies on the net that have empirical evidence that artists are willing to sign up to this setup. We believe it's a fair balance between the two approaches... there is a very long debate that I and Bill Rosenblatt had on this issue over 2 years ago:
You can't really compare GFS and Starfish. GFS is a shared-disk clustered filesystem. For example, GFS is useful when you have 10 machines sharing a cabinet of hard drives via FibreChannel or shared SCSI. The disks must be situated very close to the machines.
Starfish is a distributed clustered file storage network. Starfish is capable of using any commodity grade hardware and uses software to ensure high availability and node-level fault tolerance. Hard drives do not have to be shared between machines (a requirement for GFS) and it is meant to scale to 100s if not 1000s of nodes (GFS doesn't scale to that size easily). Starfish is capable of using GFS as its block-level file storage mechanism. Starfish is a file-level network file system - GFS is a block-level local file system. I hope that clears things up... in short, you don't have to pick between GFS and Starfish - they are complementary.
Lustre
The Lustre team is a great bunch of people - we ran their clustered file system software for two years before we needed to create our own. Lustre is run on most of the highest performing supercomputers in the world. The biggest reason we had to create Starfish was because Lustre does not distribute its metadata. There is a comparison between Lustre and Starfish on our website. You should check them out if you're trying to decide on a good distributed clustered file system - Lustre focuses on ultra high performance, Starfish focuses on data redundancy and high availability.
IBRIX
No idea - we haven't had a chance to get some real-world benchmarks from their clusters. As far as bottlenecks in their system, we don't have access to their source code, so we can't do a thorough analysis. It seems that their system design is close to ours, I would expect that they currently perform better than Starfish due to the maturity of their project. You tend to have to export IBRIX filesystems via NFS, which limits the fault-tolerant aspects. Starfish is fully POSIX compliant and can be mounted just like any Linux filesystem. Wish I could tell you more - in short, they probably perform better, have more features and cost far more than Starfish.
GFS and Lustre have free downloads available via their respective websites - it's worth taking them for a spin. You can also download and play around with Starfish, we even provide a quick start tutorial. You can even use it for up to 1TB of storage or 10 machines at no cost.
Do you have anyone mounting multiple Starfish presentation nodes via SAMBA using DFS. This would seem to be the ideal way to access the redundant data from a Windows host without introducing a single point of failure with the SAMBA presentation server.
Not really, but it's a good idea. We actually don't see a great deal of Windows usage in heavy-duty computing clusters. I wonder why that is... =)
Absolutely -- a system like that should be ready for one node going down for a short time. Yes. But handling multiple-disk failures will not give you that, if all of those disks happen to be inside that single node:-) for example. I'm sure, you (your company) has thought of that...
That is why there is forthcoming mirroring support for Starfish - which would allow you to mirror files across nodes to guarantee node-level redundancy. So, if a power supply, cooling fan, or OS kernel fails in a storage node and takes it down - you are still guaranteed to be able to retrieve the file from a different storage node.
Now, are you gaining much from working at a file-system level, rather than offering a device (SCSI, FC, or SATA)? It seems like a lot more OS-specific drivers need to be written using a file-system approach? Is it worth it?
It is actually the other way around - you reduce the amount of code that you have to write by several factors when authoring file systems as POSIX-compliant user level programs. Starfish is built on top of FUSE, which means that it works without modification on Linux and Mac OS X - a Windows port is forthcoming. Even without a windows port, Starfish file systems can be mounted via Samba or NFS.
Due to the way Starfish is designed, we didn't have to implement any OS-specific drivers... in fact, there is hardly anything that is OS specific in Starfish.
Really? Cause, your download page warns that it's a beta product and should not be used in production.
We do that as a preventative measure for people that don't get support through us. We don't want anybody to assume that the file system is ready for a highly-available cluster without talking to us first. As with all file systems, there are trade-offs to using Starfish. We are being honest - the software is stable as far as we can tell, but it doesn't have a great deal of field use (it was released to the public in March 2006).
We must be especially careful with file systems - data is very important to people. If somebody uses our system and loses data, we can't fix that - not that it has ever happened. We put the beta message as a warning that people should talk to us before thinking about putting our system into production. After all - it doesn't have nearly the amount of testing behind it that EXT3 does. It is common for a file system to remain as a beta product for the first year or two.
On the other hand, there stable ZFS support in OpenSolaris, (non-open) Solaris, and in a few months in FreeBSD, and since it's available fully free and open source, there are no necessary licensing costs. Are there any advantages of Starfish over ZFS?
Hmm... ZFS and Starfish aren't really meant to address the same storage problem. Take a bit of time and read through what ZFS does and what Starfish does. ZFS is a block-level file system. Starfish is a file-level distributed clustered storage system. Those are two very different things - at the end of the day they store files, but in very different ways and for very different purposes. Starfish can use ZFS as it's block-level file system... it can also use Reiser and EXT3.
Here are a couple of reasons to use Starfish (even though we think that ZFS is a fantastic solution for block-level file system problems):
Starfish is a decentralized, multi-node fault tolerant file storage solution that provides N-way redundancy.
Starfish is useful when you have a large number of nodes that need to access data in parallel. The ability to perform asynchronous parallel throughput is one of Starfish's biggest advantages.
Starfish runs in userspace - which means that it is capable of using much smarter algorithms and databases to manage metadata and file placement. For example, a SQL database is used for file system metadata, which means that a variety of optimizations (such as creating metadata indices) to the file system can be performed at runtime.
However, this really isn't a "what is better, Starfish or ZFS?" discussion. You can have the best of both worlds: Starfish as the file-level network storage cloud using ZFS as the block-level file system.
Here are my attempts (the text is the Gnuplot script, which produces the graphics), what do your company's experts say?
The first problem with your gnuplot script is that you're assuming a Poisson distribution for HDD failures (which is incorrect). Statistical failure distribution follows a Weibull distribution with k roughly equivalent to 7.5. Unfortunately, because you build your argument off of a Poisson distribution approximation, the rest of the analysis doesn't make much sense.
If you are interested in HDD failure rates and failure prediction, there is a fantastic paper done by Bianca Schroeder and Garth Gibson of CMU. I think this is the link to their main research website.
Even if my calculations are wrong, I suspect, the a failure of another disk, while the RAID is recovering from an earlier disk-failure is so improbable (even if the RAID spans dozens of drives), no efforts to reduce that already minuscule risk can possibly be justified.
I think you miss the point of systems such as Starfish and other distributed clustered file systems. You have many other points of failure in a system: memory, CPU, power supply, power outage, motherboard, network switch, OS kernel, router, network cable, and the all important "oops, I tripped over the power cord". There are also times that you want to take down nodes in a highly-available cluster for maintenance without affecting your applications - to do this, you need a file system that assumes and can work around node-level failure.
There is much more to highly-available clustering than just making sure your disk sub-systems are bulletproof.
I read on the web site that mirroring should only be available in August 2007, is that true?
Yes, at the present speed of development and testing, mirroring should be available by the end of August 2007.
Also, is it possible to have StarPeers of different size? I have a server with 80 GB in RAID1 and one with 500 GB in RAID1. If I set both of them as StarPeers will both have access to 580 GB of storage?
Yes, absolutely. Keep in mind that preference will be given to the 500GB storage node until it fills up to around 420GBs. Starfish tries to load-balance storage across all nodes, thus it always picks the nodes with the most amount of free space remaining.
There will be different storage node selection strategies in the future. Currently there is round-robin (which load-balances based on number of files per storage node) and largest-free-storage (which load-balances files to the storage node with the largest amount of available storage).
Slashdot Mirror
We put the meetings on hiatus until we got the commercial implementation released to the public. We did this just last week: http://blog.meritora.com/launch/ . We plan to start having meetings again within a month or two.
Bitcoin isn't a technological competitor, it's a currency. Bitcoin isn't going to be the last currency of its kind, there will be many Bitcoins just like there are many currencies today. Each one is fit for the group of people that uses the currency. The PaySwarm standard is a financial protocol and is thus currency agnostic. We plan to support Bitcoin, and Ven, and a variety of other currencies.
You could argue that Bitcoin is also a protocol, but that is where Bitcoin is fairly weak. Instead of building Bitcoin on the Web's architecture, it was decided to invent a new protocol. While the new protocol works, it's not very Webby, and because of that there is a great deal of heavy lifting that needs to occur to participate on the Bitcoin network. The PaySwarm work builds on top of the Web (HTTP, HTML, RDFa, JSON, etc.), so the financial protocol lives as a core part of the Web. It also doesn't conflate currency with protocol, so it's capable of supporting more than one currency, including the one that will eventually replace Bitcoin.
Regarding your micropayments statement, you're conflating the value of a currency with the divisibility of the currency. Micropayments support is about divisibility, not value. The only thing to discuss about Bitcoin's micropayments feature has to do with the latter, how divisible it is, which is (8 decimal places) vs. how divisible PaySwarm is (10 decimal places). It's not a very interesting discussion, I admit. :P
The short answer is that there aren't a lot of people working on the problem. There are 7,000,000,000+ people in the world. There are 60 people in the Web Payments Working Group at W3C, of which only around 10 are actively working on the problem. It's a hard problem and there aren't that many programmers, systems engineers, standards makers, writers, bloggers, lawyers, etc. that are willing to put in the hard work to solve the problem. If you think this is an exception to the rule, you'd be wrong. There are only around 40 people really working on HTML5... and that work reaches over 1.5 billion people.
We've been working on the Web Payments stuff for 7+ years and I've always been kind of floored at how quickly most tech folks backpedal away from creating a truly revolutionary financial system on top of the Web.
If you're interested in lurking or especially helping, please join the Web Payments group... we need every helping hand that's available: http://www.w3.org/community/webpayments/
Things that are legal in most states: gambling, adult media, cash transfers. Things that are illegal in most states: sexual services, selling hookers (human trafficking), and blow (drug trafficking). Payment services tend to avoid gambling and adult media because there is a huge fraud problem with them, and in the grand scheme of things, they're not as profitable as the vast majority of other "safer" transactions. Cash transfers require a huge amount of money to get a license to operate in all 50 states in the US. If you want to do something illegal, use Bitcoin or cash. If you want to do something cash-based, we're working on a Bitcoin-like alternative, but it's not a high priority. More on the commercial service here: http://blog.meritora.com/launch/
PaySwarm will eventually support Bitcoin. However, that is a separate issue from the one of doing micropayments. They're two orthogonal concerns. They do have a slight bit of overlap, but not enough to tie the design either of the solutions to one another. +1 to Devcoin. You might also want to check out Gittip: https://www.gittip.com/ We can support both with PaySwarm (since PaySwarm is currency agnostic). It's also fairly trivial to setup something like Gittip using PaySwarm (recurring payments). More here: http://blog.meritora.com/launch/
The PaySwarm specifications allow anybody to implement the specification and interoperate on the network. So, if your country doesn't have a PaySwarm Authority, there is a huge incentive for somebody to launch one in your country.
In our system, anybody (in any country) can become a vendor. At the moment, we only deal in USD, so if you want to withdraw your money, you need a bank that can talk to the US banking system (many international banks can already do this).
The only thing preventing us from branching into your country is a slew of regulations that we have to follow to make sure that we're operating the service legally in your country, using your currency.
So, I think you partially mis-read what we're doing. We only support USD now because we just launched. Eventually, we hope to support all major currencies in the world. If we don't choose to support your currency, somebody else will. Their PaySwarm Authority (the thing that acts like the bank on the network) will allow you to use whatever currency you want to in your country of origin, and it will be up to them if they want to interface with other PaySwarm Authorities around the world.
The bottom line is: The system is designed to make it such that every currency in the world will eventually be supported if there is a profit to be made in doing so.
In the beginning currencies will be exchanged at whatever the market rate is, automatically. So, if you are sending USD to someone that only has EUR accounts, the amount will be converted automatically based on current market rates and deposited into their account as EUR. The future plans hope to bypass the currency exchange markets for a more direct model, like Ripple, that doesn't have currency exchange fees that are as high as most international banks utilize today.
... and none of those are open, patent and royalty-free Web standards. You could argue that Bitcoin is such a beast, but it is more of a financial protocol and currency wrapped into one. PaySwarm will eventually support Bitcoin as a currency (along with hundreds of other currencies), so there is no real conflict there. Sorry, but this is Slashdot. If you're going to link to XKCD, you should at least make sure that what you're linking to is a good analogy. :P
Not true: http://payswarm.com/ Also: http://blog.meritora.com/
PaySwarm is currency agnostic, so it can support all national currencies, as well as alternative currencies like Bitcoin and Ven. We don't have Bitcoin support in there yet, but it's on the roadmap and we hope to sooner than later. There are regulatory issues that we have to work through. More here: http://blog.meritora.com/
We are building the technology out in the open, transparently. Anyone can join the group. There are no fees, there are no prerequisites for joining. You can read the minutes from every one of the design meetings, and even listen to the audio here (we record everything): http://payswarm.com/minutes/
Here's an example of one such meeting: https://payswarm.com/minutes/2012-07-10/
Why design the financial system in this way? We need to show people that, unlike the way our current financial system is developed and run (behind closed doors), that we're taking a radically new approach to building the basis of the financial network that we hope all of humanity will use. This financial network is open and decentralized, like the Web.
If this interests you, I urge you to join and lurk (or preferably, participate): http://www.w3.org/community/webpayments/
PaySwarm, which is part of the Web Payments work at the W3C, supports micro-transactions. All transactions in the system are accurate up to 0.0000000001 of a fraction of the currency specified. See this for more details: http://blog.meritora.com/
It could count as an in-app payment and I have no idea if the in-app purchase patent you're talking about applies, nor am I going to go take a look at it:
http://itlaw.wikia.com/wiki/Treble_damages
Our experience in this area, after looking at lots of patents, is that they tend to be badly written and/or easily easily worked around. We did file provisional patents for the technology in 2004 to establish prior art for the express purpose of ensuring that nobody else could patent the technology and that we could offer it patent and royalty-free in a Web standard.
The folks working on the PaySwarm stuff believe in data portability, so you own your private data and can take it with you when you leave the system. That's a design goal. As far as access to your hard drive, that's a bit vague - does writing a cookie to your hard drive count? Same with access to the network, vague. Care to elaborate? There are a number of open source implementations of clients now: https://github.com/digitalbazaar/payswarm.js/ https://github.com/digitalbazaar/payswarm-wordpress/ Since it's an open, patent and royalty-free spec, there will be open source implementations of a PaySwarm Authority (the things that process payments and move money around in the network) in time.
PaySwarm is currency agnostic and is designed to support both national currencies and alternative currencies like Bitcoin and Ven.
The mozPay() API is built so that Mozilla has a whitelist of organizations that are allowed to be vendors. You have to get permission from Mozilla to get on that list, and that's not very Webby. That said, Mozilla will be the first to admit that this isn't ideal and that they want to move toward a more decentralized solution. They designed it this way because decentralized payments is a really hard problem and they didn't have time to solve it and launch FirefoxOS at the same time. Luckily, we (Digital Bazaar and other folks at the W3C) have been working on decentralized payments for years and have a working solution that we're coordinating with Mozilla on trying to find a way to get it integrated with the mozPay() API.
Hi, I'm the chair of the Web Payments group at the World Wide Web Consortium (W3C). Just pointing out that the Mozilla mozPay() API is part of a greater push in the standards community to make payments a core part of the Webs architecture. This includes buying/selling digital goods, donations, crowd-funding, all the way to equity and loan-based crowd-financing for start-ups. Note that the mozPay() API is centralized, which even folks at Mozilla will tell you is not ideal. The eventual goal is to create a decentralized payment architecture that is designed for the Web from day one. We plan to put these advanced financial tools into the hands of all Web developers so that anyone with a website or blog has access to this open financial network.
You can read more about the PaySwarm standardization work here, which is mentioned at the end of the Mozilla mozPay() blog post: https://payswarm.com/
The first commercial implementation of these specifications launched three days ago: http://blog.meritora.com/launch/
If you're interested in following what's going on, join the Web Payments group at W3C: http://www.w3.org/community/webpayments/
Hi, I'm Manu Sporny - the person that authored that article. You ask good questions, but your answers to your own questions do not come from knowledge or understanding of how W3C and IETF work. You seem jaded by the world, let me try and convince you that things are not as awful as you believe them to be. Let me start by answering your most important question:
Many people do this at W3C, IETF, WHATWG and many other online communities every day. They help make the world a better place. They lead the Web to its full potential. At W3C, they're called Invited Experts and they are just like you and me. They don't come from large companies or people with deep pockets, they do amazing work and are then asked to participate in standards work at no cost to them.
You are often asked to join W3C because you have a particular expertise in an area. I am an Invited Expert - I've never paid a dime to W3C, but hope to some day because the work they do is so important. In my case, I was working on Microformats, music markup, online payments and structured data in HTML. I approached the RDFa Working Group chair at the time and asked to be a part of the discussion at W3C about RDFa. The chair of the RDFa Working Group at that time was Ben Adida, and he said that he would love to have the Microformats community's input. I was invited to join. I was required to pay absolutely nothing. I've never had to pay anything to W3C and this is because they value expert input on their standards.
It currently costs $0 to join the HTML Working Group at W3C. You could easily have an effect on HTML5 if you would put in the time to read the spec and comment on it. W3C is legally obligated to respond to you - anyone can make a difference, you just have to try. Here's how you can do it: become an expert in something Web related - invent something new or do lots of implementations and gain more knowledge than your peers. Work hard. When I say "Work Hard", I mean really, really hard. You have to write specs, you have to do implementations, you have to be familiar with at least 20-50 IETF RFCs and you have to be passionate about the future of the Web as a tool to help make humanity better.
You don't have to care about what schema.org or I say, but don't belittle the great technical work that all Invited Experts (and paid participants) do at the world standards bodies. The Internet and the Web wouldn't be what they are today without these organizations researching, creating and publishing open, patent and royalty free specifications.
If you contribute to the Web, you are a part of the entire Web community. It's important that you understand that - if you speak, your voice will be heard. As for questioning if the status quo can be changed - it happens every day. I was able to go from relative obscurity in the standards world all the way to chairing a Working Group at an International Standards body based purely on the hard work that I did to get here. It's fine if you want to be jaded about the world, but there is no $8000 fee required to make a difference (just join the mailing list - it's free). Those of us that are not jaded and believe that the Internet and the Web can make all societies better are working as hard as we can to do just that.
[FULL DISCLOSURE: I am the President/CEO of Digital Bazaar, we created Bitmunk - a legal P2P music trading network that uses watermarking to protect both artists and customers].
This isn't a new concept, Bitmunk has been doing it for three years. We have over 1 million songs on our network - it's something that works for both the artists and the music fans. We're one of the only companies on the net that have empirical evidence that artists are willing to sign up to this setup. We believe it's a fair balance between the two approaches... there is a very long debate that I and Bill Rosenblatt had on this issue over 2 years ago:
http://wiki.digitalbazaar.com/en/Manu_Sporny_vs._Bill_Rosenblatt_DRM_Debate
It is a shame that we have been pushing this concept ever since our company was founded and it is just now reaching the mainstream...
You can't really compare GFS and Starfish. GFS is a shared-disk clustered filesystem. For example, GFS is useful when you have 10 machines sharing a cabinet of hard drives via FibreChannel or shared SCSI. The disks must be situated very close to the machines.
Starfish is a distributed clustered file storage network. Starfish is capable of using any commodity grade hardware and uses software to ensure high availability and node-level fault tolerance. Hard drives do not have to be shared between machines (a requirement for GFS) and it is meant to scale to 100s if not 1000s of nodes (GFS doesn't scale to that size easily). Starfish is capable of using GFS as its block-level file storage mechanism. Starfish is a file-level network file system - GFS is a block-level local file system. I hope that clears things up... in short, you don't have to pick between GFS and Starfish - they are complementary.
The Lustre team is a great bunch of people - we ran their clustered file system software for two years before we needed to create our own. Lustre is run on most of the highest performing supercomputers in the world. The biggest reason we had to create Starfish was because Lustre does not distribute its metadata. There is a comparison between Lustre and Starfish on our website. You should check them out if you're trying to decide on a good distributed clustered file system - Lustre focuses on ultra high performance, Starfish focuses on data redundancy and high availability.
No idea - we haven't had a chance to get some real-world benchmarks from their clusters. As far as bottlenecks in their system, we don't have access to their source code, so we can't do a thorough analysis. It seems that their system design is close to ours, I would expect that they currently perform better than Starfish due to the maturity of their project. You tend to have to export IBRIX filesystems via NFS, which limits the fault-tolerant aspects. Starfish is fully POSIX compliant and can be mounted just like any Linux filesystem. Wish I could tell you more - in short, they probably perform better, have more features and cost far more than Starfish.
GFS and Lustre have free downloads available via their respective websites - it's worth taking them for a spin. You can also download and play around with Starfish, we even provide a quick start tutorial. You can even use it for up to 1TB of storage or 10 machines at no cost.
-- manuNot really, but it's a good idea. We actually don't see a great deal of Windows usage in heavy-duty computing clusters. I wonder why that is... =)
-- manuThat is why there is forthcoming mirroring support for Starfish - which would allow you to mirror files across nodes to guarantee node-level redundancy. So, if a power supply, cooling fan, or OS kernel fails in a storage node and takes it down - you are still guaranteed to be able to retrieve the file from a different storage node.
It is actually the other way around - you reduce the amount of code that you have to write by several factors when authoring file systems as POSIX-compliant user level programs. Starfish is built on top of FUSE, which means that it works without modification on Linux and Mac OS X - a Windows port is forthcoming. Even without a windows port, Starfish file systems can be mounted via Samba or NFS.
Due to the way Starfish is designed, we didn't have to implement any OS-specific drivers... in fact, there is hardly anything that is OS specific in Starfish.
-- manuWe do that as a preventative measure for people that don't get support through us. We don't want anybody to assume that the file system is ready for a highly-available cluster without talking to us first. As with all file systems, there are trade-offs to using Starfish. We are being honest - the software is stable as far as we can tell, but it doesn't have a great deal of field use (it was released to the public in March 2006).
We must be especially careful with file systems - data is very important to people. If somebody uses our system and loses data, we can't fix that - not that it has ever happened. We put the beta message as a warning that people should talk to us before thinking about putting our system into production. After all - it doesn't have nearly the amount of testing behind it that EXT3 does. It is common for a file system to remain as a beta product for the first year or two.
Hmm... ZFS and Starfish aren't really meant to address the same storage problem. Take a bit of time and read through what ZFS does and what Starfish does. ZFS is a block-level file system. Starfish is a file-level distributed clustered storage system. Those are two very different things - at the end of the day they store files, but in very different ways and for very different purposes. Starfish can use ZFS as it's block-level file system... it can also use Reiser and EXT3.
Here are a couple of reasons to use Starfish (even though we think that ZFS is a fantastic solution for block-level file system problems):
However, this really isn't a "what is better, Starfish or ZFS?" discussion. You can have the best of both worlds: Starfish as the file-level network storage cloud using ZFS as the block-level file system.
-- manuThe first problem with your gnuplot script is that you're assuming a Poisson distribution for HDD failures (which is incorrect). Statistical failure distribution follows a Weibull distribution with k roughly equivalent to 7.5. Unfortunately, because you build your argument off of a Poisson distribution approximation, the rest of the analysis doesn't make much sense.
If you are interested in HDD failure rates and failure prediction, there is a fantastic paper done by Bianca Schroeder and Garth Gibson of CMU. I think this is the link to their main research website.
I think you miss the point of systems such as Starfish and other distributed clustered file systems. You have many other points of failure in a system: memory, CPU, power supply, power outage, motherboard, network switch, OS kernel, router, network cable, and the all important "oops, I tripped over the power cord". There are also times that you want to take down nodes in a highly-available cluster for maintenance without affecting your applications - to do this, you need a file system that assumes and can work around node-level failure.
There is much more to highly-available clustering than just making sure your disk sub-systems are bulletproof.
-- manuYes, at the present speed of development and testing, mirroring should be available by the end of August 2007.
Yes, absolutely. Keep in mind that preference will be given to the 500GB storage node until it fills up to around 420GBs. Starfish tries to load-balance storage across all nodes, thus it always picks the nodes with the most amount of free space remaining.
There will be different storage node selection strategies in the future. Currently there is round-robin (which load-balances based on number of files per storage node) and largest-free-storage (which load-balances files to the storage node with the largest amount of available storage).