It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times as slow at dealing with individual connections as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2. LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, to put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times as slow at dealing with individual connections as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2. LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, too put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times at dealing with individual connections as slow as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2. LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, too put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl may implement persistant connections, it isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times at dealing with individual connections as slow as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database. You give it a set of data to search for and it returns the relevant rows from the database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2.LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, too put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
Slashdot Mirror
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times as slow at dealing with individual connections as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2. LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, to put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times as slow at dealing with individual connections as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2. LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, too put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times at dealing with individual connections as slow as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2. LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, too put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX
It's been a couple of weeks since my last analysis of slashcode's deficiencies. We've all seen the evidence of Taco's totalitarian methods. Now let's look a little deeper at his incompetence and short-sightedness as a coder. To paraphrase Taco's words to Anne Tomlinson, was he dropped on his head at birth?
Part 1: Why is Slashdot the only site on the internet that DOS's itself?
I'm sure you've all noticed the frequent protracted outages over the last couple of weeks. Many have demanded an explanation, or at least an acknowledgement that something has gone wrong. Slashdot has been down for hours at a time, but the editors act as if nothing had happened. Since Rob "Cmdr. Whitewash" Malda and friends aren't willing to open up to their readers, it's up to me to give you the dope on why Slashdot is such a piece of crap.
The problem is of course, MySQL, as we have always suspected. mod_perl is another slice of the problem, with the last factor in the equation being unscaleable hardware.
MySQL + mod_perl + PC hardware = crash
MySQL has long been a favourite among people who want to set up a database backed web page, but are too cheap to pay for oracle. It has a reputation for serving up pages quickly, which it does admirably on sites that have low loads. As the number of connections to the DB increase past 50, MySQL seems to lose a lot of stability. In fact, it is not uncommon for it to crash and lose data in some circumstances.
In most circumstances, this isn't a problem for slashdot. It's rare for the site to reach more than 50 connections, however it does happen. It is at these times that slashdot has an increased tendency to experience the slashdot effect firsthand. The reason is that while mod_perl may implement persistant connections, it isn't quite a replacement for a proper middleware layer. mod_perl includes persistent database connections, but these are irrelevant. MySQL is known for it's high connection speed, and persistent connections don't really lift it's game much. They also don't solve the problem of handling excessive connections. When mod_perl runs out of open connections, it just opens another one. In other words, mod_perl does nothing to protect the MySQL database from overload.
The other problem is the hardware. A site like slashdot, receiving approximately 600 connections per minute should be running on high end enterprise hardware, not PC hardware. Taco has tried to overcome this with clustering, but this has limitations in an IO intensive area like running a website. PCs are not known for having good network performance, and this is one area that cannot help but cause a bottleneck, particularly when the site is running under load. One advantage of running on Sun or IBM hardware is that you get good IO performance combined with the ability to utilise multiprocessing. Running your site spread over 12+ low-end machines in a network just isn't anywhere near as good.
Compare slashdot to any other high traffic site. Amazon.com for instance. Have you ever seen amazon go down for four solid hours without being DOSed by canadadian hackers? Amazon copes with their load because they run a sensible database, a well-designed front-end and hardware that can cope with the load. Thanks to the open-source ideology of this site, only one of these options are open to the administrators. PostGreSQL would reduce the number of crashes, however it is about 3 times at dealing with individual connections as slow as MySQL.
Essentially, we have a situation where the site is periodically hit with a large number of simultaneous connections, and they cause the database to keel over and die. This does not reflect well on Open Source software, and puts this site in the ironic position of bringing disrepute upon Open Source though their success at evangelism. It's no wonder the slashdot editors are unwilling to acknowledge their site's incredibly fragile nature.
I'd like to make it clear at this point that I don't actually know for sure that this is what's happening. For all I know, Taco is working on the code that is running on the actual slashdot server, and keeps breaking it. I'm just making educated guesses here.
As a bonus final note to the first half of this bumper double issue, has it occurred to anyone else that this site's codebase shares it's name with a slang term for homoerotic fan fiction? Linux gay conspiracy indeed!
Part 2: What in holy fuck?! search.pl under the microscope.
I have conclusive proof that Cmdr. Taco. is a gibbon. search.pl and Search.pm. Here's something cool for you to try at home: go to the search page, and search for all comments containing the word "competent". Now wait a few seconds. Wait a few minutes. Go grab a bite to eat. When you come back, it should be done.
Yep, this is the slowest search page in the universe. If you didn't believe me last time I told you that Malda has all the coding ability of a starving five year old from Ghana, you will believe me soon.
OK, you don't need to look at search.pl. All the meat is in Search.pm, and this time it's nicely placed at the top. Find the _keysearch function. Notice the for loop in there. What that loop is doing is constructing a "LIKE" clause to insert in a query string. It surrounds every word you entered into the search field with wildcards and uses them to search against comment text and title. This is then used in the findComments function as a clause in a select query which searches the entire comment database. You give it a set of data to search for and it returns the relevant rows from the database.
I have two major problems with what Taco has done here. They are as follows:
1. Even retards are laughing at him for writing code this fucken stupid.
2.LIKE queries are slow. They are text substring searches. They can't be optimized very well by databases like MySQL. These systems simply aren't designed with this sort of thing in mind, because for applications that aren't designed by twits, this functionality is seldom needed, and best implemented in an application specific way. Let me reiterate what's going on here. Slashcode is performing a substring search on every comment and comment title in the entire database. That is, too put it mildly, a shitload of text to search through. It cannot be done quickly, as our little demonstration should have proven to you. It is an idiotic thing to implement in a web page like slashdot, and Taco should be hanged by his testicles for having even had the idea.
You're probably thinking, "Hey, wait! Doesn't google do this sort of thing all the time, with incredible speed?" Yes, google does, but google's developers actually took the time to implement data storage methods geared towards fast search and retrieval of data based on substrings. Examples of this, for those who care, would be digital search tries and ternary trees. These are the best methods I know of off-hand for implementing the type of search slashdot is providing. Evidently Taco doesn't know of them, but that's hardly surprising, since gibbons can't read or study computer science. I also doubt that they can be implemented efficiently in perl.
I didn't think I'd be able to find a clear, simple example to top the postercomment compression filter's ability to demonstrate what an imbecile taco is, but I guess I understimated him. This is without a doubt, the dumbest thing I have ever seen in code anywhere.
XXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXX
XXX XXXX XX X
X X XX XXXX XXXXXXX
X XXX XXXXXXX XXXXXX
X XXX XX XX X
X XXX XX XXXXXXXXXXXX
X X XXX X X
XXX XXX XX XX
XXXXXXXXX XXXXX XXXXX