when you think that you're solving a technical problem in a couple lines in a Slashdot comment, that declares the problem to be "simple" or "obvious", that the CTO of InfoWorld and his team can't solve right away.
"I only worked there for five years, including two on infoworld.com."
So did I, and I helped Chad and other folks redesign their front-facing stuff, including the migration from the Solaris/NetscapeServer-based sucky CMS-published content to Apache on Linux.
Unless you worked for Chad, I would guess that you would not at all recognize the entire IT organization, forget about the infrastructure, if you were to visit.
Are you implying that you're armchair critiquing InfoWorld's RSS issues in 5 lines of a Slashdot comment ?
understood, but the calculation and re-lookup needs to happen *each time* in order to be current.
also...messages and testimonials (i.e. that are displayed on every profile page) I doubt would fit into that footprint.
the profile page on friendster does your friend count, lists you friends, and any searching is within your network, which is affected by the use of other users on the network.
-every time someone might change their profile...your search criteria would presumably change, and therefore the data -every time someone adds a friend who is up to 3 degrees from you...your data changes again
image url is probably the only thing that doesn't change depending on the use of other people.
probably pretty likely, given the amount of traffic the site gets.
But even if it didn't, it's much more likely that my 3rd degree network will change, given that it's about 77,000 people. All user/interest searches are within your network, depending on the search criteria. (2nd, 3rd, or everyone), and the default is 2nd degree.
Also, with mysql, each read is going to be a write anyway. I'm not arguing that there is zero caching going on, just not much to be significant at all.
If my friend adds another friend, then my 2nd degree network just changed, and any query that will include my 2nd degree network (i.e. just about every search I could possibly do) and then that query is no longer cacheable. that happens thousands of times, within minutes.
because they have 10 million users, and each page will be different as users add and delete friends, over time.
the likelihood of having a database query happen again anytime soon after is basically zero, because of the friend count changing.
Re:Author seems to live in a vacuum
on
On PHP and Scaling
·
· Score: 2, Insightful
"since they simply converted their JSPs to PHPs and called it good."
naw.
friendster's load characteristics have to be totally uncacheable, because of how many users they have, and the amount of disparate data sources needed for the pages. no other social networking site has even close to their load.
update a friend ? needs to be instantaneous. what happens then ? just about everyone on the entire system's friend count must change, real-time, with 1st, 2nd, and 3rd degrees. that means every addition/delete of a friend will effect hundreds of thousands of users to be correct.
JSP or PHP, it doesn't matter...like I said before, social networking sites have many different things happening than other high-volume database sites.
seeing how they had pretty smart people writing the J2EE app they started with, I doubt that they didn't get it right with Java.
When you have the authors of "Tomcat: The Definitive Guide", and "Mastering Tomcat Development" writing your app, then I would assume at the very least that they are as good as any other Java team.
I think people should consider the possibility that Friendster has/had scaling issues that other sites plainly don't have. All of their pages have to be dynamic, and I doubt that any of barely 1% would be cachable, besides images.
I also think that people should consider the possibility that they don't have any idea why Java didn't work for Friendster, because frankly, they don't. And your app is only going to be as good as the stuff behind it.
seeing how they had pretty smart people writing the J2EE app they started with, I doubt #3.
When you have the authors of "Tomcat: The Definitive Guide", and "Mastering Tomcat Development" writing your app, then I would assume at the very least that they are as good as any other Java team.
I think people should consider the possibility that Friendster has/had scaling issues that other sites plainly don't have. All of their pages have to be dynamic, and I doubt that any of barely 1% would be cachable, besides images.
I also think that people should consider the possibility that they don't have any idea why Java didn't work for Friendster, because frankly, they don't. And your app is only going to be as good as the stuff behind it.
Did you read all of the authors opinions ? I see many good reasons for right tool for the right job listed here, and Friendster is obviously one of them.
the point of the analogy is that google has more machines that suck individually much worse than each node in the LLNL cluster.
google is hardly powerful when compared to computation. it *is* powerful when you're talking about redundancy, because even tho they have thousands of shit boxes, if they lose some (and they do, many per day) the site still can be covered.
agreed, but I'm not sure that a social networking site needs to be that finely-grained, or would be used, even if it was. and again, the quality of friendships can change so easily that I don't think people would want (or care) to keep those quality metrics updated on their profile.
for one, I'm not sure the average user (i.e. someone really not so technical) will care, and two, these judgements can take place in real life. I think that to make social networking software replace what can be quickly done in real life is counter-intuitive.
I use social networking sites to get connected to people who I am 'connected' with, no matter what the quality of that connection. I will do better than the software in figuring out the quality of that connection.:)
I think that could be done very efficiently and incrementally.
but could it be done in 'real-time' ? social networks under considerable load are having problems just keeping up with managing the updates to a connections graph database as it is. I don't think it would be trivial to handle this amount of calculation.
Right, but that's not helping the real-world life of your connections. I want to use social networking sites to ENHANCE my real world life, not live in yet another chat room.
While I understand your point, again, how I feel about my friends can change daily, and keeping track of that is just not feasible.
Plus...if I am indeed friends with someone, I'll know what sort of activities I'll do with that person. I can make that judgement in real life, and leave the "finding connections" problem to the software.
Having to "rate" my friends could possibly be the worst concept to hit social networking.
I'm either friends, or not, with someone, and my own classification of my friendships change so often that updating (forget about revealing it to other folks) a website is just impossible.
Slashdot Mirror
how the hell is that comment "informative".
If you're not afraid, do it, and if you are, don't. ??
what does that tell the guy asking the question ?
Here's where an alarm should go off in your head:
when you think that you're solving a technical problem in a couple lines in a Slashdot comment, that declares the problem to be "simple" or "obvious", that the CTO of InfoWorld and his team can't solve right away.
"I only worked there for five years, including two on infoworld.com."
So did I, and I helped Chad and other folks redesign their front-facing stuff, including the migration from the Solaris/NetscapeServer-based sucky CMS-published content to Apache on Linux.
Unless you worked for Chad, I would guess that you would not at all recognize the entire IT organization, forget about the infrastructure, if you were to visit.
Are you implying that you're armchair critiquing InfoWorld's RSS issues in 5 lines of a Slashdot comment ?
understood, but the calculation and re-lookup needs to happen *each time* in order to be current.
also...messages and testimonials (i.e. that are displayed on every profile page) I doubt would fit into that footprint.
the profile page on friendster does your friend count, lists you friends, and any searching is within your network, which is affected by the use of other users on the network.
-every time someone might change their profile...your search criteria would presumably change, and therefore the data
-every time someone adds a friend who is up to 3 degrees from you...your data changes again
image url is probably the only thing that doesn't change depending on the use of other people.
probably pretty likely, given the amount of traffic the site gets.
But even if it didn't, it's much more likely that my 3rd degree network will change, given that it's about 77,000 people. All user/interest searches are within your network, depending on the search criteria. (2nd, 3rd, or everyone), and the default is 2nd degree.
Also, with mysql, each read is going to be a write anyway. I'm not arguing that there is zero caching going on, just not much to be significant at all.
you're not understanding.
If my friend adds another friend, then my 2nd degree network just changed, and any query that will include my 2nd degree network (i.e. just about every search I could possibly do) and then that query is no longer cacheable. that happens thousands of times, within minutes.
believe me, it's uncacheable.
because they have 10 million users, and each page will be different as users add and delete friends, over time.
the likelihood of having a database query happen again anytime soon after is basically zero, because of the friend count changing.
"since they simply converted their JSPs to PHPs and called it good."
naw.
friendster's load characteristics have to be totally uncacheable, because of how many users they have, and the amount of disparate data sources needed for the pages. no other social networking site has even close to their load.
update a friend ? needs to be instantaneous. what happens then ? just about everyone on the entire system's friend count must change, real-time, with 1st, 2nd, and 3rd degrees. that means every addition/delete of a friend will effect hundreds of thousands of users to be correct.
JSP or PHP, it doesn't matter...like I said before, social networking sites have many different things happening than other high-volume database sites.
I don't think you have any idea how much bandwidth/memory/db size Friendster uses.
In-memory databases are not a possibility, and caching is impossible.
seeing how they had pretty smart people writing the J2EE app they started with, I doubt that they didn't get it right with Java.
When you have the authors of "Tomcat: The Definitive Guide", and "Mastering Tomcat Development" writing your app, then I would assume at the very least that they are as good as any other Java team.
I think people should consider the possibility that Friendster has/had scaling issues that other sites plainly don't have. All of their pages have to be dynamic, and I doubt that any of barely 1% would be cachable, besides images.
I also think that people should consider the possibility that they don't have any idea why Java didn't work for Friendster, because frankly, they don't. And your app is only going to be as good as the stuff behind it.
in Friendster's case, there's just no such thing as in-memory caching. it can't be done.
seeing how they had pretty smart people writing the J2EE app they started with, I doubt #3. When you have the authors of "Tomcat: The Definitive Guide", and "Mastering Tomcat Development" writing your app, then I would assume at the very least that they are as good as any other Java team. I think people should consider the possibility that Friendster has/had scaling issues that other sites plainly don't have. All of their pages have to be dynamic, and I doubt that any of barely 1% would be cachable, besides images. I also think that people should consider the possibility that they don't have any idea why Java didn't work for Friendster, because frankly, they don't. And your app is only going to be as good as the stuff behind it.
Did you read all of the authors opinions ? I see many good reasons for right tool for the right job listed here, and Friendster is obviously one of them.
secureFTP, AFAIK, is ftp-over-SSL, and not SFTP, or SCP. WinSCP is better.
Indeed, that is true. I stand corrected, but also like and have seen x86_64 run insanely fast. :)
the point of the analogy is that google has more machines that suck individually much worse than each node in the LLNL cluster. google is hardly powerful when compared to computation. it *is* powerful when you're talking about redundancy, because even tho they have thousands of shit boxes, if they lose some (and they do, many per day) the site still can be covered.
Powerful = fastest computation, not biggest. A roomfull of Chevettes do not make a Corvette.
x86_64 will be the arch of choice, and this proves it.
agreed, but I'm not sure that a social networking site needs to be that finely-grained, or would be used, even if it was. and again, the quality of friendships can change so easily that I don't think people would want (or care) to keep those quality metrics updated on their profile.
:)
for one, I'm not sure the average user (i.e. someone really not so technical) will care, and two, these judgements can take place in real life. I think that to make social networking software replace what can be quickly done in real life is counter-intuitive.
I use social networking sites to get connected to people who I am 'connected' with, no matter what the quality of that connection. I will do better than the software in figuring out the quality of that connection.
I think that could be done very efficiently and incrementally.
but could it be done in 'real-time' ?
social networks under considerable load are having problems just keeping up with managing the updates to a connections graph database as it is. I don't think it would be trivial to handle this amount of calculation.
users want up-to-date stats.
but do they have proof ? see my post about Alexa's traffic graphs.
How the hell did my comment about traffic get modded as a troll ? I'm only posting facts:
http://tinyurl.com/3gc7o
most social networking sites, friendster included, count the degrees of separation already.
:)
and I would guess that there is indeed a large computing requirment.
Right, but that's not helping the real-world life of your connections. I want to use social networking sites to ENHANCE my real world life, not live in yet another chat room.
While I understand your point, again, how I feel about my friends can change daily, and keeping track of that is just not feasible.
Plus...if I am indeed friends with someone, I'll know what sort of activities I'll do with that person. I can make that judgement in real life, and leave the "finding connections" problem to the software.
Having to "rate" my friends could possibly be the worst concept to hit social networking.
I'm either friends, or not, with someone, and my own classification of my friendships change so often that updating (forget about revealing it to other folks) a website is just impossible.