10. Do #9 until replication becomes too much. Then, federate your databases and stop load balancing them. Build some smarts into your frontends so they can direct traffic to the right db, which are all masters at this point, slaves are only for backup.
All excellent points, and I love Netscalers as well. I will add to..."As a result, you're going to run into cases where you run out of epheral ports"...but remember, this is only a port limit per *IP*. Adding more subnet IPs to the other side of the connection (like on a Netscaler) can help HUGELY to break down that limit.
and to "6. HTTP request does not equal TCP connection. Don't assume that."
I will add: HTTP multiplexing (Netscaler or not) will just plainly not work without Keep-Alive connections built into all of the tiers.
I will add a number 9 and 10.
9. Hardware load balance all of your lower tier MySQL slave read traffic, and make writes go to the master. This will bring you a very long way.
better than ab is siege, which can deal with HTTP/1.1 Keep-Alives, and give more regression-style stats. it's at joedog.org.
better than siege would be something like httperf, and autobench, which will give you some indication whether or not your client generating the requests is still healthy. autobench also allows you to run multiple instances of httperf on different machines, and then aggregate the numbers after the test.
remember folks, there are only 65535 (minus 1024) ports that any machine can be using with one IP...that has to be considered as well, including at the load balancer layer.
Yahoo does well over a BILLION pageviews a day. I'm not so sure that the database that they are using sucks, just because a hundred slashdot trolls says that it does.
what I meant was, the USDOT has been doing ITS research for at least 10 years, officially, and a lot longer than that.
A hint for you: consider the possibility that you're not going to somehow come to some groundbreaking transportation research idea in a slashdot post. USDOT engineers know more about the topic than all of the people posting on this topic.
At least, about how the DOT does ITS research in some sort of vaccuum.
The research that has been going into ITS has been happening for years, and it's been going on in the same building as the rest of the DOT agencies research projects.
I know, because I worked there.
There are a LOT of things that the US government does with respect to transportation safety and efficiency, and no one pays attention to it. The fact is, the USDOT has been doing excellent research on a lot of topics that takes the (at least US) auto manufacturers *YEARS* to adopt or evaluate. Because it's like this:
NHSTA and Federal Highway come up with very smart ideas and research. State budgets and car manufacturers fight these good ideas, tooth and nail, because they cost money.
Lee Iacocca and Chrysler didn't come up with airbags, the USDOT did, years before.
while you're obviously feeling strong about those things....they are subjective. Alarm works fine for me, and I'm liking the fixes....1.4 had a lot to be fixed.
Admin salary differences between *nix and M$ admins can be peanuts when compared to the amount of money wasted on downtimes, virus damage, patching downtimes, and security outages due to an exploitation.
Because SuSE is basically in bed with AMD, and the support for AMD from SuSE makes RedHat look like a garage effort.
My prediction: RedHat will slowly go away if they don't adapt to what SuSE is doing....namely, bringing better hardware support. Case in point: 64bit. RedHat's 2.6 kernel 64bit support sucks it on many levels.
Chad's columns and weblog prove that CTOs do NOT have to be PHBs, or void of coding skills. He basically brought OpenSource to a good part of the online publishing world.
http://weblog.infoworld.com/dickerson/
There's still some of his code running on Salon's mod_perl-based publishing system.
"How to Lie With Statistics" is the title of an excellent book about it can be done, I should have made that clear.
I'm saying that the author of Kismet is lying. In fact, he makes the effort of saying where he is getting his 80% from, at least.
My point is that the Slashdot post is overly sensational (as usual) quoting the 80% stated in the article, without giving the sample size, which is what the book "How to Lie With Statistics" is about.
My statistics are that 100% of access points have very strong WEP, given that the sample size of that 100% is the 3 access points in my building, which I set up myself. Here's another example:
"90% of all houses are white"*
(*note: all houses within my line of sight, from my apartment, right now)
Now, I'm not trying to be a wiseass. I'm trying to point it out because people see those Slashdot headlines, don't bother to read the article, and think that the world is coming to an end of wireless security. I live in San Francisco, and as of 2 months ago, I only stumbled within about 4 blocks, 2 WAPs that didn't have WEP turned on, out of about 30 or so that my Zaurus (kismet) sniffed out, which is not 80%.
for example, let's compare something like finding a roomate to share an apartment with.
In real life, or without social networking, it's like this:
-have to bother all of your friends, over email, on the phone, at a party, etc. to spread the word. that means *requiring* your friends/family to keep an ear out for you, which they might not remember.
-go thru craigslist and spend days interviewing freaky random people who might not turn out to be good roomate material after all
WITH social networking, you can peruse people who you know through your friends, post a bulletin board for only your 2nd degree friends to see, and possibly find someone that has more real-life vouching (i.e. you can ask your friends about them before you commit living with them) than the random people you see on something like craigslist.
Slashdot Mirror
"MySQL can't keep up.. does it even support row-locking yet"
It does, and has, for quite some time. Innodb.
This setup certainly can go some distance, but at some point, replication can become too much for each read slave.
oh wait:
10. Do #9 until replication becomes too much. Then, federate your databases and stop load balancing them. Build some smarts into your frontends so they can direct traffic to the right db, which are all masters at this point, slaves are only for backup.
All excellent points, and I love Netscalers as well. I will add to..."As a result, you're going to run into cases where you run out of epheral ports" ...but remember, this is only a port limit per *IP*. Adding more subnet IPs to the other side of the connection (like on a Netscaler) can help HUGELY to break down that limit.
and to "6. HTTP request does not equal TCP connection. Don't assume that."
I will add: HTTP multiplexing (Netscaler or not) will just plainly not work without Keep-Alive connections built into all of the tiers.
I will add a number 9 and 10.
9. Hardware load balance all of your lower tier MySQL slave read traffic, and make writes go to the master. This will bring you a very long way.
better than ab is siege, which can deal with HTTP/1.1 Keep-Alives, and give more regression-style stats. it's at joedog.org.
better than siege would be something like httperf, and autobench, which will give you some indication whether or not your client generating the requests is still healthy. autobench also allows you to run multiple instances of httperf on different machines, and then aggregate the numbers after the test.
remember folks, there are only 65535 (minus 1024) ports that any machine can be using with one IP...that has to be considered as well, including at the load balancer layer.
ah, I see. I agree. It's why we use it at work, too. (a certain un-named high-volume social networking site)
so are you implying that having the most features make the best database ?
Yahoo does well over a BILLION pageviews a day. I'm not so sure that the database that they are using sucks, just because a hundred slashdot trolls says that it does.
Bicycles, trains and pedestrians are the topic of many research areas at the DOT. I'm sure they're sorry they beat you to the punch.
Good luck with that sarcastic technique, it's obviously working for you.
what I meant was, the USDOT has been doing ITS research for at least 10 years, officially, and a lot longer than that.
A hint for you: consider the possibility that you're not going to somehow come to some groundbreaking transportation research idea in a slashdot post. USDOT engineers know more about the topic than all of the people posting on this topic.
Why do I know this ? Because I was one of them.
At least, about how the DOT does ITS research in some sort of vaccuum.
The research that has been going into ITS has been happening for years, and it's been going on in the same building as the rest of the DOT agencies research projects.
I know, because I worked there.
There are a LOT of things that the US government does with respect to transportation safety and efficiency, and no one pays attention to it. The fact is, the USDOT has been doing excellent research on a lot of topics that takes the (at least US) auto manufacturers *YEARS* to adopt or evaluate. Because it's like this:
NHSTA and Federal Highway come up with very smart ideas and research. State budgets and car manufacturers fight these good ideas, tooth and nail, because they cost money.
Lee Iacocca and Chrysler didn't come up with airbags, the USDOT did, years before.
They don't need the hint, they've already been looking into trains for about 10 years.
The same research done on ITS is done in the SAME building as light rail, maglev, and crashworthiness research.
while you're obviously feeling strong about those things....they are subjective. Alarm works fine for me, and I'm liking the fixes....1.4 had a lot to be fixed.
Admin salary differences between *nix and M$ admins can be peanuts when compared to the amount of money wasted on downtimes, virus damage, patching downtimes, and security outages due to an exploitation.
Because SuSE is basically in bed with AMD, and the support for AMD from SuSE makes RedHat look like a garage effort.
My prediction: RedHat will slowly go away if they don't adapt to what SuSE is doing....namely, bringing better hardware support. Case in point: 64bit. RedHat's 2.6 kernel 64bit support sucks it on many levels.
Chad's columns and weblog prove that CTOs do NOT have to be PHBs, or void of coding skills. He basically brought OpenSource to a good part of the online publishing world.
http://weblog.infoworld.com/dickerson/
There's still some of his code running on Salon's mod_perl-based publishing system.
you're so charasmatic. It's such a surprise only one of your neighbors wants to talk to you.
Please yourself, I'm not refuting or arguing the 80%, why is it that people think I am ?
Re-read my post, I'm arguing that posting statistics without context is ignorant, which is what Slashdot did. I'm arguing nothing more than that.
Slashdot readers are not politicians, you're right. But I'm assuming that most of them want to read articles that are accurate. Maybe I'm wrong.
"How to Lie With Statistics" is the title of an excellent book about it can be done, I should have made that clear.
I'm saying that the author of Kismet is lying. In fact, he makes the effort of saying where he is getting his 80% from, at least.
My point is that the Slashdot post is overly sensational (as usual) quoting the 80% stated in the article, without giving the sample size, which is what the book "How to Lie With Statistics" is about.
My statistics are that 100% of access points have very strong WEP, given that the sample size of that 100% is the 3 access points in my building, which I set up myself. Here's another example:
"90% of all houses are white"*
(*note: all houses within my line of sight, from my apartment, right now)
Now, I'm not trying to be a wiseass. I'm trying to point it out because people see those Slashdot headlines, don't bother to read the article, and think that the world is coming to an end of wireless security. I live in San Francisco, and as of 2 months ago, I only stumbled within about 4 blocks, 2 WAPs that didn't have WEP turned on, out of about 30 or so that my Zaurus (kismet) sniffed out, which is not 80%.
"(*) Percentage gathered from the pc running in my car that monitors all the time"
right. and how far does that car travel ? in which city ? and for how long ?
don't say 80% if it's not.
"When comparing Friendster to the real world"
that was your first mistake. your second was not having enough out-of-box brains to see what social networking is about.
here's some help:
it's not MEANT to replace the real world. it's meant to make it BETTER.
here's another hint to help you:
like a TELEPHONE does, like a yellow pages does.
think of friendster as being an insanely detailed and annotated grouping of all your friend's Rolodex's, and you're halfway there.
Come on, don't be afraid....connect the dots, and you'll see that social networking isn't for people to MAKE FRIENDS OUTSIDE OF THE REAL WORLD.
bonehead.
I think you're missing the point of Friendster.
for example, let's compare something like finding a roomate to share an apartment with.
In real life, or without social networking, it's like this:
-have to bother all of your friends, over email, on the phone, at a party, etc. to spread the word. that means *requiring* your friends/family to keep an ear out for you, which they might not remember.
-go thru craigslist and spend days interviewing freaky random people who might not turn out to be good roomate material after all
WITH social networking, you can peruse people who you know through your friends, post a bulletin board for only your 2nd degree friends to see, and possibly find someone that has more real-life vouching (i.e. you can ask your friends about them before you commit living with them) than the random people you see on something like craigslist.
sounds like it hit a sore spot for you.
just goes to show, if I click on links, I might learn something. I stand corrected.
it's not InfoWorld's problem. It's not a bandwidth issue. It's the timing, and the format.