He actually specifically asked for a program to manage a "flat-file database." One example of such a database is a tab-delimited file. "Database" does not imply locking, indexing, SQL, etc. -- that's a Database Management System.
He doesn't mean that they hand-code every page -- he says very clearly that they use a CMS with templates. All he said is that they don't use a GUI tool to create the templates. This is true of just about any significant site. What is the imagined news here?
the problem is like in any other country the fact that the people don't know much about how their taxes are spent. i'm not an american but if i would have been i would be pissed to find out that the military gets some 20% while nasa gets less than 1%
There are many examples like that. Most Americans think we spend a lot on foreign aid, but it's actually about 1% of the budget, i.e. 20 times less than we spend on the military. And that vastly underestimates how much we spend on the military because they now shift the money around to hide many of the expenses in areas outside the Pentagon, like the State Department.
The worst part is, you know that war in Iraq? The one that we're spending billions of dollars on? That's not part of the budget at all. That's all paid for by borrowing. Yes, the Iraq war is going on a credit card. We are so screwed.
Yeah, because the defaults on every other piece of software are PERFECT. MySQL tries hard to maintain compatibility with older versions. It's not that outrageous to ask people to specify that they don't need backwards compatibility by turning on the strict mode.
Transactions for your full-text search? That's not something most people have or want. Typical usage of full-text search engines is to refresh the index periodically, often once-a-day or less.
With MySQL, you'd probably keep all of your normalized data in InnoDB tables and keep separate full-text search data up-to-date with triggers or a cron job. I wouldn't call it ideal, but it seems a lot less important than other things I'd like to see changed (e.g. better performance for subqueries).
With MySQL, I have seen tables get badly corrupted, sometimes even beyond repair(!) if a disk runs full. Perfect, an anecdote witout details or any way to reproduce the claimed problem.
The syntax is also pretty lax. Adding an integer and a string? No problem. String and a float? Sure. Turn on the strict mode.
You want a contraint? Sure, it'll accept that query. Will it honour the constraint? Not so much. Turn on the strict mode.
Createing an InnoDB table, for (some) referential integrity? Sure, it'll give no errors, but if innodb support is disabled for any reason, it will create MyISAM tables instead, without any hint or warning. That would be a fundamental configuration mistake. You would get a warning, and any time you looked at the table definition it would tell you it was a MyISAM table, not an InnoDB one.
Inserting a row with a primary key value outside the legal range? It'll give no errors, but it also wont insert the row. Instant data loss. Turn on the strict mode. Seriously, this stuff has all been there for YEARS and you have only yourself to blame if you haven't figured it out yet.
If you need something lean and fast, try SQLite. Give me a break. SQLite is a neat project and great for times when you don't want to bother installing a database daemon (e.g. the music database in Amarok), but its performance is terrible compared to MySQL, especially for concurrent access.
PostgreSQL's Generalized Search Tree (GiST) indexing is still better than anything MySQL has to offer, in terms of performance and capability. Since you offer no benchmarks, this is nothing more than FUD.
The PostgreSQL OpenFTS full text search engine is another marvel of engineering. It routinely outperforms similar extensions for MySQL in terms of performance, memory usage, and concurrency. Same here.
For textual representations of certain geographic system data, it's not unusual these days to have individual fields that need to store 500 to 600 MB of data. No, that is VERY unusual, and probably a sign of poorly normalized data.
In large companies, "innovation" is defined as a VP deciding to switch from one multi-million dollar commercial software package to another, in order to look like something is happening. Large companies tend not to trust their in-house staff to come up with anything. This is why I now work for small companies that can't afford to ignore their employees.
Think of all the huge companies that use Linux or FreeBSD as their primary server platform: Amazon.com, Yahoo.com, Google.com, AOL.com, etc. If it's good enough to run the biggest on-line companies in the world, isn't it good enough for your company?
How quickly we forget the details. There were locks. I recall hearing that on at least one plane the pilots locked the door, but they eventually gave in and opened up after the terrorists stabbed one of the flight crew and threatened to hurt the rest.
Web UIs do not compete well with an actual local application using a real UI toolkit. That's all I'm saying. Regarding the JavaScript problems, I have a very fast computer with plenty of RAM and the latest Firefox, but it still feels like running in molasses when I use the Google AJAX apps. After a couple of days I have to restart the browser because it has sucked up most of the RAM on the machine. The current crop of JavaScript implementation is just not up to what is being asked of them.
Dude, it's a 2.4Ghz P4 with 1GB of RAM and the latest Firefox and it's still glacial when running Google's apps compared to an actual local application.
You don't say why you're against Apache with FastCGI for PHP. Maybe you think the FastCGI implementation is not good? A lot of people use it. Anyone running mod_php on a popular site should run a reverse proxy in front of their mod_php server for static content. Then you have separate control for static and PHP content (effectively the same as FastCGI).
If you use separate processes for your static content and your PHP (either Apache with FastCGI or a reverse proxy), you won't have problems with memory. The size of an Apache process that's just serving static stuff is tiny. It's the PHP processes that take the RAM, and they will do the same when run from Lighttpd.
All that apache does when serving PHP requests with mod_php is run the early phases (auth, URL mapping) and pass the request to PHP. Do you really think that Lighttpd + FastCGI is going to be significantly faster than that? The bottleneck is your PHP code, not the web server.
Lighttpd is probably faster at serving static content. Most sites don't have enough bandwidth to find out.
Working for a giant financial company can make anything lose its fun. Why not try doing IT for a different company, working in an area that you care about? Maybe you love playing guitar, so you go do IT for a guitar manufacturer. Maybe you want to feel like you're helping people, so you join the IT staff at a non-profit you support.
I cut my teeth on Oracle. I know what it can do, and I know it will be faster than MySQL at many things involving large databases. A good example of MySQL's shortcomings would be subqueries, where very little work has been done on optimization of query plans. However, a simplistic statement like "never use MySQL for a project involving more than 10000 rows per table" is obviously false.
Incidentally, MySQL has partitioning now and can parallelize queries across partitions.
I'm currently looking at a table on a production MySQL db with 176 million rows in it. It has no problems at all, other than being a huge amount of data. Your ideas about MySQL's limitations are not based on fact.
Many browsers allow JavaScript to set the document.referrer property.
Re:Okay, I'll be the first to ask.
on
Web 2.0 Under Siege
·
· Score: 2, Interesting
Checking the referer header is ultimately going to fail because it would mean trusting the client to not lie about the referer. There are some better techniques described in the Wikipedia XSRF entry.
I can vouch for Bill's work on open source. We worked together at a web startup before he went to IBM. We mostly worked on a LAMP platform based around mod_perl, and he put a lot of effort into making sure that our patches to the open source code we used (and there were quite a few) were contributed back. We presented a paper together at an OSS conference about the work we did there. He's the real deal.
Slashdot Mirror
http://www.schneier.com/blog/archives/2008/01/my_open_wireles.html
He actually specifically asked for a program to manage a "flat-file database." One example of such a database is a tab-delimited file. "Database" does not imply locking, indexing, SQL, etc. -- that's a Database Management System.
He doesn't mean that they hand-code every page -- he says very clearly that they use a CMS with templates. All he said is that they don't use a GUI tool to create the templates. This is true of just about any significant site. What is the imagined news here?
It's a lot of money compared to spending nothing, but as a percentage of the budget it's less than almost any other developed nation spends.
There are many examples like that. Most Americans think we spend a lot on foreign aid, but it's actually about 1% of the budget, i.e. 20 times less than we spend on the military. And that vastly underestimates how much we spend on the military because they now shift the money around to hide many of the expenses in areas outside the Pentagon, like the State Department.
The worst part is, you know that war in Iraq? The one that we're spending billions of dollars on? That's not part of the budget at all. That's all paid for by borrowing. Yes, the Iraq war is going on a credit card. We are so screwed.
Yeah, because the defaults on every other piece of software are PERFECT. MySQL tries hard to maintain compatibility with older versions. It's not that outrageous to ask people to specify that they don't need backwards compatibility by turning on the strict mode.
With MySQL, you'd probably keep all of your normalized data in InnoDB tables and keep separate full-text search data up-to-date with triggers or a cron job. I wouldn't call it ideal, but it seems a lot less important than other things I'd like to see changed (e.g. better performance for subqueries).
In large companies, "innovation" is defined as a VP deciding to switch from one multi-million dollar commercial software package to another, in order to look like something is happening. Large companies tend not to trust their in-house staff to come up with anything. This is why I now work for small companies that can't afford to ignore their employees.
Think of all the huge companies that use Linux or FreeBSD as their primary server platform: Amazon.com, Yahoo.com, Google.com, AOL.com, etc. If it's good enough to run the biggest on-line companies in the world, isn't it good enough for your company?
"Kiss your dreams of WIFI reliance goodbye"
What was that supposed to mean? Reliance is not it. Reliability? Independence?
How quickly we forget the details. There were locks. I recall hearing that on at least one plane the pilots locked the door, but they eventually gave in and opened up after the terrorists stabbed one of the flight crew and threatened to hurt the rest.
Web UIs do not compete well with an actual local application using a real UI toolkit. That's all I'm saying. Regarding the JavaScript problems, I have a very fast computer with plenty of RAM and the latest Firefox, but it still feels like running in molasses when I use the Google AJAX apps. After a couple of days I have to restart the browser because it has sucked up most of the RAM on the machine. The current crop of JavaScript implementation is just not up to what is being asked of them.
Dude, it's a 2.4Ghz P4 with 1GB of RAM and the latest Firefox and it's still glacial when running Google's apps compared to an actual local application.
And here I thought it was the hideous UI and sluggish, memory-sucking JavaScript. Now I know better.
You don't say why you're against Apache with FastCGI for PHP. Maybe you think the FastCGI implementation is not good? A lot of people use it. Anyone running mod_php on a popular site should run a reverse proxy in front of their mod_php server for static content. Then you have separate control for static and PHP content (effectively the same as FastCGI).
If you use separate processes for your static content and your PHP (either Apache with FastCGI or a reverse proxy), you won't have problems with memory. The size of an Apache process that's just serving static stuff is tiny. It's the PHP processes that take the RAM, and they will do the same when run from Lighttpd.
All that apache does when serving PHP requests with mod_php is run the early phases (auth, URL mapping) and pass the request to PHP. Do you really think that Lighttpd + FastCGI is going to be significantly faster than that? The bottleneck is your PHP code, not the web server.
Lighttpd is probably faster at serving static content. Most sites don't have enough bandwidth to find out.
Working for a giant financial company can make anything lose its fun. Why not try doing IT for a different company, working in an area that you care about? Maybe you love playing guitar, so you go do IT for a guitar manufacturer. Maybe you want to feel like you're helping people, so you join the IT staff at a non-profit you support.
I cut my teeth on Oracle. I know what it can do, and I know it will be faster than MySQL at many things involving large databases. A good example of MySQL's shortcomings would be subqueries, where very little work has been done on optimization of query plans. However, a simplistic statement like "never use MySQL for a project involving more than 10000 rows per table" is obviously false.
Incidentally, MySQL has partitioning now and can parallelize queries across partitions.
I'm currently looking at a table on a production MySQL db with 176 million rows in it. It has no problems at all, other than being a huge amount of data. Your ideas about MySQL's limitations are not based on fact.
Many browsers allow JavaScript to set the document.referrer property.
Checking the referer header is ultimately going to fail because it would mean trusting the client to not lie about the referer. There are some better techniques described in the Wikipedia XSRF entry.
I can vouch for Bill's work on open source. We worked together at a web startup before he went to IBM. We mostly worked on a LAMP platform based around mod_perl, and he put a lot of effort into making sure that our patches to the open source code we used (and there were quite a few) were contributed back. We presented a paper together at an OSS conference about the work we did there. He's the real deal.