What is the question I'm answering? "Where are the Napster users going to go if they leave?" Well duh, observations yield that the GnutellaNet isn't that large, and apparently the few critics of my premises so far "brush over" the actual question I'm answering, and even more importantly all the attention I draw to the potential invalidity of my results.
And for all the ignorant folks out there, 2 million users is not an absolutely huge network for Napster. AAMOF, last I checked Napster peaked at 1.8M aggregate concurrent last week. The problem is that ignorance of Napster's true usage is so pervasive that the comparisons I make and the conclusions I draw from them are sometimes lost.
I don't think my numbers are realistic. Please re-read the previous sentence. I tried my best to convey that impression throughout the paper, and I don't understand how some folks missed it. I do think that the numbers are still useful in formulating a perspective on what will happen if even a few percentage points of the Napster population decide to try and use Gnutella.
I still don't see why it's FUD. FUD is about deception, about spreading information that aren't true. My math, at least AFAICT, is not blatantly off, and while my assumptions are numerous, at every turn I question them and raise the reader's awareness that they could be off, and why. I think it's ridiculous to characterize my research as FUD. I'm not attempting to deceive anyone.
As for why Gnutella can't scale, the point of my paper was not to duplicate other work or research. I don't mention a lot of the reasons because I think they are either irrelevant or different methodologies arriving at the same point. The premise of my paper wasn't to cover the practical limitations of Gnutella, since those have pretty much been beaten like a dead horse. The premise was to take an alternative angle at addressing the question "Can Gnutella Scale?", simply by calculating network impact with some math, and provoking some thought.
In other words, you look at 6GBps and say "FUD!!! That number is wrong!!" I look at it and say, "Hmm, well, 6GBps or 4GBps, makes no difference why or how, it ain't gonna work."
No, I don't think you missed anything at all; as a matter of fact I think you're dead on.
I wholeheartedly agree, the concept of an R value and figuring out an accurate way to represent realistic demographics is extremely complex and very prone to error. I never really thought that it was completely accurate (and I felt as though I did a good job giving the "grain of salt" talk), because as you so aptly point out, there's a lot of variables in the mix that can affect the demographic. All I wanted to do was present some math and methodologies, and test some example cases to provoke thought.
I hope people don't think that every search for "grateful dead live" will yield such uncharacteristic results, but I do hope that people see the numbers and think to themselves, "wow, an 18 byte search query could potentially generate this much traffic." And it's true, it possibly could happen. Who knows; you point out the relevancy of human psychology in deriving a more realistic number, and I don't think anyone can take an accurate stab at how it factors in to a P2P network of 1M people.
Did you check the math? Seriously, the numbers are not "big and scary like" because I was spinning them a certain way, they are big and scary because they are.
I even share the equations and methodologies I used, and try to poke holes in my own conclusions.
Further, I'm not a competitor. I haven't worked for Napster in 3 months. Before Napster my background was in poking holes in things anyway. All I did was finish a personal project I started a long time ago.
You actually sound more like FUD than anything.:-)
You're absolutely right! NNTP, SMTP, HTTP, etc. are all forms of distributed systems.
However, comparing news to file-sharing is absolutely not an apples-to-apples comparison: NNTP servers cache data of interest locally. When you want to search that data, it searches its local cache. Gnutella clients, OTOH, only know what the local system has, hence the need to broadcast out search queries.
It's not the data transfer that can't scale, it's the methodology for locating that data. Having the right infrastructure is key, as you say, and Gnutella and other fully-distributed file-sharing applications use a different methodology.
Well, first of all, I didn't destroy any hard work; in fact, all I did was prove mathematically what the smart folks at Xerox PARC and Clip2 DSS have been saying all along.
But you're right, what's the next step? Well, I think there's a lot of great ideas out there already, but the technology in general is all still quite juvenile. I don't believe we'll see wide-scale adoption without a better Internet infrastructure to carry the traffic. Coming up with smart ways to ferry around data will always help, but in the end 14.4k is 14.4k, and 56k is 56k. Following the logic of Power-Law Phenomenon, fully distributed networks will probably never scale without the lowest-common-denominator amount of bandwidth being raised significantly.
That's actually not true. March of last year is when Gnutella was released, and the only thing anyone said was how it would surpass Napster's capabilities. And that was the company line for a few months afterwards, as well.
As I said in the paper, architectural flaws are fundamental in nature and cannot be effectively mitigated without redesign.
Aside from the obvious reality that Non-Disclosure Agreements were designed to mitigate these specific types of occurrences, everyone should keep high in their minds that in the end, whether or not an NDA has been signed holds no bearing on a company's available legal recourse against an ex-employee.
While an active NDA certainly reinforces the legal position, a company can bring suit against anyone for just about any reason and petition a court seeking injunctive relief against that individual or group of individuals. In the final assessment, even if a court petition is tenuous at best, the individual is likely to face a significant cost for defending themselves. In this fashion, many companies can just discourage individuals from disclosing what they know by threatening costly lawsuits.
Unless the previous relationship with the company is more substantial than just being an "employee", everyone should think twice, no, three times, before publically disclosing ANYTHING about a company.
God forbid they would thank the hard-working folks who _ACTUALLY_ run the service. Thanks Eddie, the about box in Beta8 was everything you said it would be!:-)
1. Bill Bales is not a founder of Napster, Inc. He joined the company in late August, 1999. 2. Bill Bales was forced to leave Napster, Inc. for gross misconduct December 17th, 1999. 3. Napster, Inc. != AppleSoup. The relationship is disingenuous in nature and constitutes a fabrication to gain press attention.
Realistically, Decker's humanity is only relevant if you read the book (Do Androids Dream of Electric Sheep). Few if any can legitimately question his humanity just from watching the movie, relative to how that question is continually raised throughout the book itself.
Not sure how anything is official though, since Philip K. Dick, the only real author of the story and the only one who could make such a claim, is dead.
I guess this sort of cruft explains why Philip wouldn't let them make the movie. What a damn shame.
Small price of bandwidth? Are you kidding? Gnutella is mathematically impossible to scale because of bandwidth.
Good luck with your small price. Hope you have some left for checking email or web surfing.
--jordan
The right tool for the right job.
on
Why Not MySQL?
·
· Score: 1
For all the (very valid) faults MySQL has, it is still a great database. The real answer to the question of whether MySQL is a good database, though, is another question -- what are you trying to do with your data?
Early on at Napster, Inc., we discovered that MySQL was quite the capable database; it was easy to install, easy to move around, and easy to administer. Permissions were simple to understand and manipulate, and for basic data storage, you just can't beat its ease of use. However, MySQL's shortcomings really constrain it to a narrow audience -- low- to middle-end systems.
IMHO, the biggest shortcoming of MySQL is table-level locking. Foreign keys, stored procedures, etc, those are arguably good things to have, but at the basic, essential level, they're extras. Table-locking will now and forever keep MySQL from ever being a high-end database solution.
You can throw hardware at MySQL, but only to a point. Eventually you will find that millions of rows, and either inadequate keying or over-keying will hurt you considerably. If you key inadequately, you limit your searching capabilities (or incur large search times over non-keyed data). If you over-key, you'll incur a performance hit with every update, insert, or delete. And this is where table-locking hurts you too; the table is locked while the indexes update.
And let's not forget the lack of supported replication and redundancy. If the daemon dies, it's gone -- go reload it. Replication? Not supported, so your best bet is to copy the table files somewhere else. Of course, copying the table files locks them too, and all your clients' writes and updates will block. (Yes, I know about the MySQL clustering stuff. Kludgy, and not terribly reliable. Definitely not proven.)
Sure, it's free software, I'm not complaining. Just pointing out the facts.
For tables with millions and millions of rows, there comes a point (very quickly in fact) when throwing gigahertz around just won't solve MySQL's problems.
In MySQL's defense, of course, you have INSERT and REPLACE DELAYED, so from the client side you can continue on without blocking. Eventually, though, you'll have to pay the piper, and for a table whose transactions are largely SELECTs peppered by a (relatively) smallish number of UPDATEs, INSERTs and DELETEs, your client's SELECTs will block when that delayed thread wakes up to dump down the data.
Even better, in the latest alpha vers there is the concurrent_insert business. Unfortunately, in our testing the only alpha version that DIDN'T crash under our kind of load was the version right before concurrent_inserts were implemented (3.23.7 has it, but 3.23.6 is the most recent alpha that won't core dump). I suspect this functionality might be to blame.
All that said and done, MySQL might still be the right database for you! It's really just a question of using the right tool for the right job.
MySQL has interfaces into almost every language, and as a developer of MySQL-aware software, I have to say I really, really like mysql++. As long as you're not doing anything transaction-intensive or operating over millions of rows, MySQL is a rock-solid choice.
You've missed the point entirely -- it's a conflict of interest because of the topic of the lawsuit: Contributory and vicarious copyright infringement.
Time-Warner/AOL/Winamp/NullSoft/GNullSoft cannot sue Company X for doing Y, and do Y themselves at the same time. Nullsoft is owned indirectly by Time-Warner, and is, therefore, indirectly involved in the suit against Napster.
Slashdot Mirror
While McAfee.com was first to be awarded the patent, there are certainly competing (pending) patents trying to nab the same thing.
Note currently pending patent #20010010053, Service framework for a distributed object network system, comes frightfully close as well.
Of course they filed it this year, and it will probably be rejected as a result of McAfee's award.
How depressing.
--jordan
These things have been around for years. Why the spotlight all of a sudden?
--jordan
They failed to mention that no one has been able to get the guy, since he left the country a while back when all of this started.
Gotta catch him first before anyone gets paid..
--jordan
And for all the ignorant folks out there, 2 million users is not an absolutely huge network for Napster. AAMOF, last I checked Napster peaked at 1.8M aggregate concurrent last week. The problem is that ignorance of Napster's true usage is so pervasive that the comparisons I make and the conclusions I draw from them are sometimes lost.
I don't think my numbers are realistic. Please re-read the previous sentence. I tried my best to convey that impression throughout the paper, and I don't understand how some folks missed it. I do think that the numbers are still useful in formulating a perspective on what will happen if even a few percentage points of the Napster population decide to try and use Gnutella.
--jordan
As for why Gnutella can't scale, the point of my paper was not to duplicate other work or research. I don't mention a lot of the reasons because I think they are either irrelevant or different methodologies arriving at the same point. The premise of my paper wasn't to cover the practical limitations of Gnutella, since those have pretty much been beaten like a dead horse. The premise was to take an alternative angle at addressing the question "Can Gnutella Scale?", simply by calculating network impact with some math, and provoking some thought.
In other words, you look at 6GBps and say "FUD!!! That number is wrong!!" I look at it and say, "Hmm, well, 6GBps or 4GBps, makes no difference why or how, it ain't gonna work."
--jordan
I wholeheartedly agree, the concept of an R value and figuring out an accurate way to represent realistic demographics is extremely complex and very prone to error. I never really thought that it was completely accurate (and I felt as though I did a good job giving the "grain of salt" talk), because as you so aptly point out, there's a lot of variables in the mix that can affect the demographic. All I wanted to do was present some math and methodologies, and test some example cases to provoke thought.
I hope people don't think that every search for "grateful dead live" will yield such uncharacteristic results, but I do hope that people see the numbers and think to themselves, "wow, an 18 byte search query could potentially generate this much traffic." And it's true, it possibly could happen. Who knows; you point out the relevancy of human psychology in deriving a more realistic number, and I don't think anyone can take an accurate stab at how it factors in to a P2P network of 1M people.
--jordan
I even share the equations and methodologies I used, and try to poke holes in my own conclusions.
Further, I'm not a competitor. I haven't worked for Napster in 3 months. Before Napster my background was in poking holes in things anyway. All I did was finish a personal project I started a long time ago.
You actually sound more like FUD than anything. :-)
--jordan
However, comparing news to file-sharing is absolutely not an apples-to-apples comparison: NNTP servers cache data of interest locally. When you want to search that data, it searches its local cache. Gnutella clients, OTOH, only know what the local system has, hence the need to broadcast out search queries.
It's not the data transfer that can't scale, it's the methodology for locating that data. Having the right infrastructure is key, as you say, and Gnutella and other fully-distributed file-sharing applications use a different methodology.
But you're right, what's the next step? Well, I think there's a lot of great ideas out there already, but the technology in general is all still quite juvenile. I don't believe we'll see wide-scale adoption without a better Internet infrastructure to carry the traffic. Coming up with smart ways to ferry around data will always help, but in the end 14.4k is 14.4k, and 56k is 56k. Following the logic of Power-Law Phenomenon, fully distributed networks will probably never scale without the lowest-common-denominator amount of bandwidth being raised significantly.
--jordan
As I said in the paper, architectural flaws are fundamental in nature and cannot be effectively mitigated without redesign.
Don't design around the problem, fix it.
--jordan
--jordan
FUD? Just read the math, man. Make your own decision, sure, but read the paper first. There's nothing FUD-like about the mathematics in the paper.
--jordan
Sure you can, at least in the United States. You can sue anyone for any amount for any reason. It absolutely works both ways.
If you decide to go head-to-head with a company, though, you should well understand the significant cost of such an undertaking.
--jordan
Aside from the obvious reality that Non-Disclosure Agreements were designed to mitigate these specific types of occurrences, everyone should keep high in their minds that in the end, whether or not an NDA has been signed holds no bearing on a company's available legal recourse against an ex-employee.
While an active NDA certainly reinforces the legal position, a company can bring suit against anyone for just about any reason and petition a court seeking injunctive relief against that individual or group of individuals. In the final assessment, even if a court petition is tenuous at best, the individual is likely to face a significant cost for defending themselves. In this fashion, many companies can just discourage individuals from disclosing what they know by threatening costly lawsuits.
Unless the previous relationship with the company is more substantial than just being an "employee", everyone should think twice, no, three times, before publically disclosing ANYTHING about a company.
It's just not worth it.
--jordan
Whoever/whatever says that DWP wrote Blade Runner is absolutely wrong.
--jordan
God forbid they would thank the hard-working folks who _ACTUALLY_ run the service. Thanks Eddie, the about box in Beta8 was everything you said it would be! :-)
--jordan
What you meant to say was: "Napster will become subscription based."
Saying Napster will now be subscription-based is misleading, as it infers that the current Napster will go away. This is not the case.
They didn't mean "there's a shortage of IT work labour".
What they meant to say was "there's a shortage of COMPETENT IT work labour."
--jordan
1. Bill Bales is not a founder of Napster, Inc. He joined the company in late August, 1999.
2. Bill Bales was forced to leave Napster, Inc. for gross misconduct December 17th, 1999.
3. Napster, Inc. != AppleSoup. The relationship is disingenuous in nature and constitutes a fabrication to gain press attention.
jordan@napster.com
Not sure how anything is official though, since Philip K. Dick, the only real author of the story and the only one who could make such a claim, is dead.
I guess this sort of cruft explains why Philip wouldn't let them make the movie. What a damn shame.
Small price of bandwidth? Are you kidding? Gnutella is mathematically impossible to scale because of bandwidth.
Good luck with your small price. Hope you have some left for checking email or web surfing.
--jordan
For all the (very valid) faults MySQL has, it is still a great database. The real answer to the question of whether MySQL is a good database, though, is another question -- what are you trying to do with your data?
Early on at Napster, Inc., we discovered that MySQL was quite the capable database; it was easy to install, easy to move around, and easy to administer. Permissions were simple to understand and manipulate, and for basic data storage, you just can't beat its ease of use. However, MySQL's shortcomings really constrain it to a narrow audience -- low- to middle-end systems.
IMHO, the biggest shortcoming of MySQL is table-level locking. Foreign keys, stored procedures, etc, those are arguably good things to have, but at the basic, essential level, they're extras. Table-locking will now and forever keep MySQL from ever being a high-end database solution.
You can throw hardware at MySQL, but only to a point. Eventually you will find that millions of rows, and either inadequate keying or over-keying will hurt you considerably. If you key inadequately, you limit your searching capabilities (or incur large search times over non-keyed data). If you over-key, you'll incur a performance hit with every update, insert, or delete. And this is where table-locking hurts you too; the table is locked while the indexes update.
And let's not forget the lack of supported replication and redundancy. If the daemon dies, it's gone -- go reload it. Replication? Not supported, so your best bet is to copy the table files somewhere else. Of course, copying the table files locks them too, and all your clients' writes and updates will block. (Yes, I know about the MySQL clustering stuff. Kludgy, and not terribly reliable. Definitely not proven.)
Sure, it's free software, I'm not complaining. Just pointing out the facts.
For tables with millions and millions of rows, there comes a point (very quickly in fact) when throwing gigahertz around just won't solve MySQL's problems.
In MySQL's defense, of course, you have INSERT and REPLACE DELAYED, so from the client side you can continue on without blocking. Eventually, though, you'll have to pay the piper, and for a table whose transactions are largely SELECTs peppered by a (relatively) smallish number of UPDATEs, INSERTs and DELETEs, your client's SELECTs will block when that delayed thread wakes up to dump down the data.
Even better, in the latest alpha vers there is the concurrent_insert business. Unfortunately, in our testing the only alpha version that DIDN'T crash under our kind of load was the version right before concurrent_inserts were implemented (3.23.7 has it, but 3.23.6 is the most recent alpha that won't core dump). I suspect this functionality might be to blame.
All that said and done, MySQL might still be the right database for you! It's really just a question of using the right tool for the right job.
MySQL has interfaces into almost every language, and as a developer of MySQL-aware software, I have to say I really, really like mysql++. As long as you're not doing anything transaction-intensive or operating over millions of rows, MySQL is a rock-solid choice.
Unfortunately, for Napster, Inc., it just wasn't.
You've missed the point entirely -- it's a conflict of interest because of the topic of the lawsuit: Contributory and vicarious copyright infringement.
Time-Warner/AOL/Winamp/NullSoft/GNullSoft cannot sue Company X for doing Y, and do Y themselves at the same time. Nullsoft is owned indirectly by Time-Warner, and is, therefore, indirectly involved in the suit against Napster.
And just to dispel any retort about the separation of Nullsoft and GNullsoft -- read the bottom of the gnutella.org webpage: "This stuff ©2000 Nullsoft, Inc, a subsidiary of America Online, Inc.".
Yeah, we all remember the /search mechanism in IRC, don't we? Oh yeah. Glorified IRC client, indeed. :-)
--jordan
> Is Napster going to eat into CD sales?
> Absolutely--and I believe that the end result
> is going to be a lot of prudning in the
> industry.
That's interesting; the RIAA posted reports that CD sales are at record numbers this past year. Yeah, Napster sure is eating into their pockets.
--jordan