Slashdot Mirror

Funny this story should come up today. My community website has been getting attacked for the last couple of days by a botnet (I think) of zombie computers. I wrote the Spambot Trap article that was published here in 2002, and I've been using the trap successfully to block spambots ever since. Usually, the block list is a couple of dozen repeat offenders. But day before yesterday, it suddenly spiked up - there were dozens of spambots coming in from all kinds of different IP addresses. I'm pretty sure it's a botnet of zombies, because a) they all report exactly the same User-Agent, and b) they all come in directly to the guestbooks and forums (probably using a search engine) and c) all the IP addresses resolve to dialup, cable or DSL accounts (some businesses too). It's getting a bit much, because the block list has suddenly ballooned to over 160, constantly changing. The trap is coping ok, because the blocks will fall off after a while (the block time goes up as the power of 2 for each repeated offence). I have added some logfile snapshots to the article. (Look down the page to see how the number of blocks has suddenly increased in the last couple of days, and also notice how all the browsers are identical). I think this is some kind of virus that may still be spreading, because the number is only increasing.

Anybody else seeing this kind of stuff happening?

I would recommend a Pocketmail device. You can compose email and upload it to a toll-free service from practically any phone with the built-in acoustical modem. (Just hold the device up to the handset.) Also, bring a good phone card; payphones can be a bitch, and collect calls are risky.

I've been using MySQL for about six years now, and it's been working very well for me. I utilize it on my crazyguyonabike.com, a bicycle tour journals website. It has about 750 journals on there, with over 60,000 pictures. I use replication to back up the database remotely, and all in all it works very well. I honestly can't understand the level of hatred towards the tool that emanates from many of the posts here.

I have to say that I cringe every time I see a MySQL story on slashdot these days, because it just seems like there is a legion of PostgreSQL zealots just waiting for any chance to denigrate MySQL. It's the same littany every time - PostgreSQL is so much better, have they fixed the "Gotchas" yet, etc etc. Even when MySQL AB adds a feature or does fix some perceived failing, then the detractors simply ignore this and move on to some other apparent showstopper. For example, it's not enough that MySQL has transactional capabilities - no, now they simply moan that it's not the default (MyISAM still is).

We seem to have people who have what can only be described as a religious mindset when it comes to these issues. "Religious" in the sense that their minds are closed, and no matter what new facts come to light, they will simple twist everything around to match with their existing worldview. So, in these people's minds, MySQL AB adding features is not a positive thing, it's rather a sign of how wrong Monty was in the past to suggest that most people really don't need transactions for everything. Well, at what point exactly do we have "proof" that I don't really need transactions for my website? Is six years of 24/7 use enough? If not, then how long exactly?

Yes, I've had problems, of course I have. You will with any tool, PostgreSQL included. No matter the fact that PG has had transactions from day 1, people still got corrupted tables occasionally. But at the end of the day, the results are the same - do you still have your data? Is it intact and internally consistent? I can answer yes to that. I don't mind having some logic in my application to delete some records when some other records get deleted. It works really well, and while in theory it could cause data inconsistency, in practice this has never happened. Even if it did, a quick perl script would be sufficient to clean things up - I'm doing that kind of thing all the time anyway, as the database evolves and I need to shift stuff around or change table structures. It's no big deal, really! Some will say No, this is a Horrible Solution and you should put business logic into stored procedures... I say, get a life. That's *your* solution, it's not everybody's. You're simply moving your complexity around, you'll never really get rid of it. Some people are more comfortable with their complexity in stored procedures, I'm perfectly comfortable with it in my Perl application. So what, does it work for you? If so, then who cares.

There *are* some things in MySQL that disturb me, but I don't know if they are common to other DBMS solutions out there. One of the big ones for me currently is that the query optimizer only uses one index in queries. I know you can have multi-column indexes, but I still see this being a problem for some of my more complex queries. Does PostgreSQL do this better? Informed opinions please, rather than fanboy noise.

Also, speed. I hear lots of anecdotal tales about how much faster PostgreSQL is these days, especially under load from multiple connections. I'd like to hear from anybody who has actually made a transition from MySQL to PostgreSQL for a high-load Web application. Can PostgreSQL really replace MySQL now? Or is this another case of wishful thinking?

Thanks,

-Neil

I agree, the issue of what to set for the expiration times is critical. I use a combination of approaches: Short expiration times (1 minute) for "what's new" type pages, which ensures that people will never see very old content (one minute is fast enough for anything but real real-time data such as stock quotes, but that's probably a case where you'd be setting 'no-cache' anyway)... this also means that the backend is being called at most once a minute for these pages, which isn't going to be a problem for any db unless the queries are just insane.

Then I also use URL arguments to make links look "new" to the front-end proxy when things change. For example, there is a 'v' field for documents that gets incremented whenever the document is modified. So then if a page is added, then all the links on subsequent generated pages have '&v=123', which looks like a new page and so the user doesn't get stale data. I also use the same technique for user options, which (as well as storing in a cookie) I compress and pass around as 'o=xxx'. This is useful for browsers, some of which do not distinguish pages which are otherwise identical but with different cookies.

Finally, user editing pages get 'no-cache', since these really are dynamic and are only being seen by one person anyway. But if I got slashdotted by someone posting a link to one of the journal pages on a popular site, the server would hardly break a sweat because it would generate that page just once and then serve it to all the other requests as static from the front-end.

I use these techniques successfully on my bike journals website: http://www.crazyguyonabike.com/

I'm one of the crazy guys on that site:

-jeff!

Better yet, ride a bike around the country. Bio-powered. Some emissions, but nothing the environment can't handle. :)

Checking some of his other websites I found a nice picture of him from one of his long bike trips. He looks like a real sharp one whose viewpoints are to be seriously considered. Truly a Renaissance Man of our time to be reckoned with.

I get so tired of reading all these "MySQL sucks" comments from the PostgreSQL crowd every time there is any mention of the two databases. Well, for whatever reason, this time I feel compelled to comment in return.

First of all, I remember back when I was trying to decide which of the two databases to use for my bicycle touring community website, crazyguyonabike.com, and I heard all the same arguments that MySQL is a "toy" that's simply an SQL interface to a flat file system, and how PostgreSQL was a "real" database with actual transactions, subqueries and so on... but then, digging further into PostgreSQL, I found out about the 8k row size limit, and the requirement to halt everything to do a VACUUM occasionally. That really set me back on my heels - 8k? That's really pathetic. Ok, so they've fixed that in the later versions, and the VACUUM also seems to be bit less blocking now, but still - it left me wondering just what else these zealots were conveniently choosing to forget about when recommending PostgreSQL. How about speed? How about all the anecdotal accounts of corrupted databases (again, probably improved somewhat more recently). What other major limitations are they conveniently neglecting to mention these days? I ask, because these people were making exactly the same noises back in the 8k rowsize limit, blocking VACUUM days as they are now.

Thing is, MySQL works really well right out of the gate for most things that people want to do in real life. Here are a few factoids for you:

MySQL *does* have real transactions. It has for quite some time now - just use the InnoDB table type. And for anyone who whines about this not being the default table - get a life! Choice is good, and I would say that for 99% of applications out there, you just don't need transactions. Go ahead and crucify me for saying this - I don't care. It works for a lot of people, many companies all over the world use it for heavy duty database work. I've used it for four years now, and it has *never* crashed on me, *never* lost any data. I use RAID and backup regularly too, but MySQL is ROCK SOLID.

MySQL is getting all these things like subselects and procedures, just with a different priority to PostgreSQL. So what? Again, I have never missed subqueries, and I have some pretty complex queries in my website. Again, go ahead and flame me, I don't care - it works for me, and it works for a LOT of other people too. Which brings me to another thing:

There is a definite sense of intellectual snobbery and elitism in many of the comments from PostgreSQL fans slamming MySQL. This reminds me of some of the people I used to know back at University - these are people who are so wrapped up in their own little cozy theoretical worlds that anything remotely practical or pragmatic in its approach seems to be extremely threatening. I think this is one of the big reasons for the bile - these people are simply threatened by the fact that MySQL is so popular and so much more used across the world. Could there also be a little tribalism creeping in here? You know, my team vs your team. They just can't seem to understand that BOTH databases work very well and BOTH databases have weaknesses, which BOTH development teams are working very hard to fix and make better. Getting religious about this sort of thing helps nobody. But, I guess, given the fact that Religion is in fact so widespread in the world, this would then also point to there being a somewhat large number of people who are so hung up on their own little certainties and fixed viewpoints that anything else just can't be left to stand. Like the Christian missionaries, they have to have everyone else adhering to THEIR point of view, regardless of the fact that there are, in fact, many "ways" to spiritual enlightenment (or data management). Again: Not everyone needs transactions. Many people just need simplicity and speed, and they get it with MySQL. And you know what, if they want transactions then they can have that too! Th