Really, this goes against what I've seen. But when I say large projects I generally mean many many users with a fair percentage of them writing to the database, while for other folks "large project" means LOTS of nearly static data for things like a repository.
For mostly read depositories, MySQL is pretty good. When you start mixing in more and more writes, it tends to not do so well with MyISAM tables, and innodb don't quite keep up with PostgreSQL. But they're pretty good.
uhhhm. I've got a workstation here at work. it's running apache 2.0 (process mode) with php5 and acccesses our Oracle and postgresql databases just fine. Or are you talking about the new PDB classes? Or the third party apps?
Most third party apps will gladly accept patches that make pgsql work. i've submitted a few can still recal in the dark recesses of my memory a few times.
Jeeze. Reading comprehension here is droppig quickly. My point was that Windows can get hosed esaily by a new driver install, while in linux a new driver that causes problems is almost always an easy fix (rpm -e packagename).
Of course, I do check to see if the hardware I want works with Linux, but wouldn't you check to make sure the battery you were putting in your car was gonna fit before you bought it?
Bravo, wonderful post. Completely missed my point, you and all the other responders get the haircut 2005 award.
My point, and I'll type this very slowly so as not to confuse you.
In windows, it's quite easy for a new piece of software, like a driver, to wedge the system in such a way that it either won't boot, or takes so long to fix it's easier to just reimage the machine and start over.
In Linux, the worst I've ever had was a machine lock up while trying a new kernel module and have to fsck the drives on reboot.
Why is that such a serious limitation. Use pgpool for the front end two master servers (in synchronous replication,) then slony-I to make as many children of these as you want (in asynchronous replication.)
What problem are you trying to solve, a performance one, a reliability one, or what?
The fact is there is no one size fits all replication method, and expecting to find something that just plugs in and does everything is likely to lead to disappointment.
Mysql and postgres are great little databases for non-critical projects
You mean like the.org registry which runs on PostgreSQL? Oracle screamed and hollared when Afilias / Liberty RMS suggested setting up the.org domain on postgresql, saying such things as that it didn't support transactions (something that's been built in from the beginning) and carrying on a like a child deprived of its favorite toy.
Then, 3 months after going online, when the whois registry started pointing to the right place, and some dolt at the Register hadn't updated his client, he posted a screed about how PostgreSQL had failed at hosting the.org domain, and how we should have all seen this coming.
The sheepish apology was the last time I heard anyone voice a complaint about PostgreSQL. While there are certain high load environments that still favor Oracle over PostgreSQL, those particular applications are becomer fewer and more rare all the time.
Postgres, good solid database. Badly misoptimizes some queries (Don't dare say VACCUUM, I did all that, I read the explain output, I changed every setting, it just screws up some queries, end of story).
1: It's analyze, not vacuum. If you didn't analyze, there's little to no chance the query optmizer will get complex queries right on most data sets.
2: I've often seen folks post poor performing queries to the -perform list, get a couple of hints, try everything, post their test case, watch one of the core developers reproduce it, and make a patch in a week or two. It usually gets into the next minor version, no problem.
The query planner in PostgreSQL gets better with each release, so if you haven't tried postgresql in a few major versions, you are likely to find all those poorly planned queries of old running quite quickly now. If not, see point 2 above.
What about airline reservations? Most companies in that field ARE only employing 100 people, running a medium sized database handling a million transactions a day.
Really, I'd expect Microsoft to have designed the two very different operating systems to NOT take each other's patches. It couldn't have been that hard to do, just toss some identifier in there somewhere in the file and if XP sees a win2k id or vice versa, refude to install the update.
It's still Microsoft's fault, because they designed a system that accepts updates for the wrong system, and after that update is installed, it's damned near impossible to back it out. EDS has fault here too, but let's face it, they couldn't have screwed the pooch nearly as well with a non-MS based system.
FWI, RAID 1+0 and RAID 5 get a lot of testing on the postgresql-performance mailing list, and what we've found there is that with a high quality RAID controller with battery backed cache, the difference from one config to another is minimized until one or the other starts to saturate its SCSI busses. Since most database accesses are random, and therefore only use a small percentage of the maximum throughput available from an individual drive, it takes a lot of drives to saturate the SCSI busses, so for most users, with less than a few dozen hard drives, RAID 5 and RAID 1+0 are pretty much the same. But only when under a fast, battery backed caching RAID controller.
With software RAID, the configuration VERY much matters, and RAID 1+0 is usually the clear winner.
Here's some logic: RAID-5 needs to write across all disks to update parity on writes, which slows them no matter how much fancy hardware you've got to improve them.
This one statement shows that you do not know much about RAID 5.
RAID 5 only requires that the parity stripe and the changed data stripe be updated. It's really quite simple, you read the old data and parity stripe, XOR the new data stripe with the old one, XOR that output to the parity stripe, and write out the new parity and data stripe.
Furthermore, if you have a battery backed write cache on your RAID controller, the write happens immediately to the OS / application point of view, so unless you're streaming write after write faster than the write cache and parity generation and subsequent writes to drives can handle, it's just as fast as any other RAID array. With enough drives, it usually outruns RAID 1+0, since the higher number of drives means that it can handle even more parallel writes with less chance of two writes needing to hit the same drive. Hot spares (and RAID 5+0) handle your argument for running with more than one failed drive.
While software RAID 5 is simply too slow to compete with software RAID 1+0, hardware RAID 5 with battery backed cache is every bit a competitor for hardware or software RAID 1+0.
I was under the impression that the debates recieve public funding, and were therfore subject to the US laws about equal access. So the "property rights" here would be yours and mine.
Oh, OK, so I take it that for threads, the memory used that looks like it would be private for a process is actually probably shared as well, so the total usage was just whatever the biggest thread looked like?
Hmmm. That's nice to know (if I got it right) but again, our big issue was that it just died during stress testing. One of those things where the process is alive and running, but no longer answers on the LDAP port.
Since almost all of our input was scripted, having a nice GUI didn't mean a lot. Other than a web based group editor, and a few admin utils, everything else came from HR so all we wrote were simple update / filter apps for those.
As a PostgreSQL user, I'm quite familiar with the phrase "Did you performance tune that?" and I likely would have tried performance tuning it if it didn't just die on me so much. I always figured if it died under load in one configuration, it was just a matter of increased load in another configuration to kill it. I know that's not always the case.
Yeah, I understand. It's just nice to be able to look somebody up and see what else they've posted lately / ever on the subject. And to recognize them later when you see them again.
If you're both so confident, how about a post with your uid, not anonymous. I'm not saying either of you are right or wrong. Just that if you're sure of what you're saying why AC?
The Netscape DS does not require or use multiple processes - it is a multi-threaded server.
You are aware of the fact that older Linux kernels (remember it was three years ago) showed threads and processes the same, right?
The DS I tested was the iPlanet one, btw, so I don't know if that's an issue or not. Around V 4 or so if I recall correctly. I'd be more than willing to try the newer version but a: I don't work there no more, and b: Lyle Lanley has sold them on Microsoft technology...
About three years ago (admittedly, my knowledge is pretty old now) I tested and compared the two. The Netscape LDAP server used up a huge chunk of memory, even sitting idle, and could handle only a few authentication's / searches per second on our dual P-III 750 machine with 1 gig ram. The memory usage, if I recall correctly, was about 50 megs per process (not shared mem, individual memory usage by the way) with a default of something like 5 of them running.
OpenLDAP used about 20 megs of memory total, ramping up to 50 to 100 megs under heavy load. It could handle about 30 to 40 auths / searches a second.
Worse for the Netscape server was that it would just plain stop working after an hour or so of heavy load testing.
We went with OpenLDAP, and wrote our own edit screens for it since at the time it came with nothing very useful to a user (only ldapadd, etc... command line stuff).
After about a year of only handling the web server it was on we pointed our Peoplesoft implementation at it, which proceeded to increase our load from one auth every couple of seconds to about 10 auths a second. Other than the slightly larger number of openldap processes running, we never really noticed the load.
Hope that helps anyone looking at the two. I certainly would hope the Netscape server has gotten better, but everything I've read about it since then seems to say it hasn't.
Microsoft goes on a bit about how much better their commercial software is because they have commercial code reviewers to catch this kind of thing, i.e. people who have a job to do and are getting paid to do it must be doing a better job than the great unwashed masses.
Microsoft tells us they do these kinds of things better, but the reality of the situation is that fixing security issues require a group of people who know what they're doing, and honestly, I don't think Microsoft has a whole lot of those people.
Slashdot Mirror
Really, this goes against what I've seen. But when I say large projects I generally mean many many users with a fair percentage of them writing to the database, while for other folks "large project" means LOTS of nearly static data for things like a repository.
For mostly read depositories, MySQL is pretty good. When you start mixing in more and more writes, it tends to not do so well with MyISAM tables, and innodb don't quite keep up with PostgreSQL. But they're pretty good.
uhhhm. I've got a workstation here at work. it's running apache 2.0 (process mode) with php5 and acccesses our Oracle and postgresql databases just fine. Or are you talking about the new PDB classes? Or the third party apps?
Most third party apps will gladly accept patches that make pgsql work. i've submitted a few can still recal in the dark recesses of my memory a few times.
Jeeze. Reading comprehension here is droppig quickly. My point was that Windows can get hosed esaily by a new driver install, while in linux a new driver that causes problems is almost always an easy fix (rpm -e packagename).
Of course, I do check to see if the hardware I want works with Linux, but wouldn't you check to make sure the battery you were putting in your car was gonna fit before you bought it?
Bravo, wonderful post. Completely missed my point, you and all the other responders get the haircut 2005 award.
My point, and I'll type this very slowly so as not to confuse you.
In windows, it's quite easy for a new piece of software, like a driver, to wedge the system in such a way that it either won't boot, or takes so long to fix it's easier to just reimage the machine and start over.
In Linux, the worst I've ever had was a machine lock up while trying a new kernel module and have to fsck the drives on reboot.
Is that more clear?
lemmesee.
Number of times I've been forced to reinstall my entire windows partition due to a buggy driver scramming my whole box? dozens and dozens.
Number of times I've been forced to reinstall linux for the same problem: 0
All my peripherals (mp3 player, digital camera, printer, USB mouse / graphics pad, etc...) work on both windows and linux.
So, again, how are people spoiled by windows driver support?
Unfortunately, slony is only compatible with versions as far back as 7.3 (because slony relies on schemas, a feature first added in tahat version.)
But the autovacuum daemon isn't enabled or even compiled by default. You have to go into contrib, compile and install it, and then turn it loose.
Soon, 8.1 or so, it will be integrated into the backend. It almost made it into 8.0, but missed it by "" - that much.
I've got an old empathy chip if you need one!
Why is that such a serious limitation. Use pgpool for the front end two master servers (in synchronous replication,) then slony-I to make as many children of these as you want (in asynchronous replication.)
What problem are you trying to solve, a performance one, a reliability one, or what?
The fact is there is no one size fits all replication method, and expecting to find something that just plugs in and does everything is likely to lead to disappointment.
Mysql and postgres are great little databases for non-critical projects
.org registry which runs on PostgreSQL? Oracle screamed and hollared when Afilias / Liberty RMS suggested setting up the .org domain on postgresql, saying such things as that it didn't support transactions (something that's been built in from the beginning) and carrying on a like a child deprived of its favorite toy.
.org domain, and how we should have all seen this coming.
You mean like the
Then, 3 months after going online, when the whois registry started pointing to the right place, and some dolt at the Register hadn't updated his client, he posted a screed about how PostgreSQL had failed at hosting the
The sheepish apology was the last time I heard anyone voice a complaint about PostgreSQL. While there are certain high load environments that still favor Oracle over PostgreSQL, those particular applications are becomer fewer and more rare all the time.
Have you looked at pgpool in strict mode? I think it might do what you need.
In my experience, a MSSQL expert and a PostgreSQL expert go for about the same salary.
Postgres, good solid database. Badly misoptimizes some queries (Don't dare say VACCUUM, I did all that, I read the explain output, I changed every setting, it just screws up some queries, end of story).
1: It's analyze, not vacuum. If you didn't analyze, there's little to no chance the query optmizer will get complex queries right on most data sets.
2: I've often seen folks post poor performing queries to the -perform list, get a couple of hints, try everything, post their test case, watch one of the core developers reproduce it, and make a patch in a week or two. It usually gets into the next minor version, no problem.
The query planner in PostgreSQL gets better with each release, so if you haven't tried postgresql in a few major versions, you are likely to find all those poorly planned queries of old running quite quickly now. If not, see point 2 above.
We have one Oracle Guru, and one PostgreSQL guru, who cross over and cover for each other. and no outside gurus.
What about airline reservations? Most companies in that field ARE only employing 100 people, running a medium sized database handling a million transactions a day.
:)
and they typically have 2 or 3 dbas, tops.
I know, I work at one.
bs walks, money talks.
Really, I'd expect Microsoft to have designed the two very different operating systems to NOT take each other's patches. It couldn't have been that hard to do, just toss some identifier in there somewhere in the file and if XP sees a win2k id or vice versa, refude to install the update.
It's still Microsoft's fault, because they designed a system that accepts updates for the wrong system, and after that update is installed, it's damned near impossible to back it out. EDS has fault here too, but let's face it, they couldn't have screwed the pooch nearly as well with a non-MS based system.
FWI, RAID 1+0 and RAID 5 get a lot of testing on the postgresql-performance mailing list, and what we've found there is that with a high quality RAID controller with battery backed cache, the difference from one config to another is minimized until one or the other starts to saturate its SCSI busses. Since most database accesses are random, and therefore only use a small percentage of the maximum throughput available from an individual drive, it takes a lot of drives to saturate the SCSI busses, so for most users, with less than a few dozen hard drives, RAID 5 and RAID 1+0 are pretty much the same. But only when under a fast, battery backed caching RAID controller.
With software RAID, the configuration VERY much matters, and RAID 1+0 is usually the clear winner.
Here's some logic: RAID-5 needs to write across all disks to update parity on writes, which slows them no matter how much fancy hardware you've got to improve them.
This one statement shows that you do not know much about RAID 5.
RAID 5 only requires that the parity stripe and the changed data stripe be updated. It's really quite simple, you read the old data and parity stripe, XOR the new data stripe with the old one, XOR that output to the parity stripe, and write out the new parity and data stripe.
Furthermore, if you have a battery backed write cache on your RAID controller, the write happens immediately to the OS / application point of view, so unless you're streaming write after write faster than the write cache and parity generation and subsequent writes to drives can handle, it's just as fast as any other RAID array. With enough drives, it usually outruns RAID 1+0, since the higher number of drives means that it can handle even more parallel writes with less chance of two writes needing to hit the same drive. Hot spares (and RAID 5+0) handle your argument for running with more than one failed drive.
While software RAID 5 is simply too slow to compete with software RAID 1+0, hardware RAID 5 with battery backed cache is every bit a competitor for hardware or software RAID 1+0.
I was under the impression that the debates recieve public funding, and were therfore subject to the US laws about equal access. So the "property rights" here would be yours and mine.
Oh, OK, so I take it that for threads, the memory used that looks like it would be private for a process is actually probably shared as well, so the total usage was just whatever the biggest thread looked like?
Hmmm. That's nice to know (if I got it right) but again, our big issue was that it just died during stress testing. One of those things where the process is alive and running, but no longer answers on the LDAP port.
Since almost all of our input was scripted, having a nice GUI didn't mean a lot. Other than a web based group editor, and a few admin utils, everything else came from HR so all we wrote were simple update / filter apps for those.
As a PostgreSQL user, I'm quite familiar with the phrase "Did you performance tune that?" and I likely would have tried performance tuning it if it didn't just die on me so much. I always figured if it died under load in one configuration, it was just a matter of increased load in another configuration to kill it. I know that's not always the case.
Anyway, I've rambled enough.
Yeah, I understand. It's just nice to be able to look somebody up and see what else they've posted lately / ever on the subject. And to recognize them later when you see them again.
If you're both so confident, how about a post with your uid, not anonymous. I'm not saying either of you are right or wrong. Just that if you're sure of what you're saying why AC?
You are aware of the fact that older Linux kernels (remember it was three years ago) showed threads and processes the same, right?
The DS I tested was the iPlanet one, btw, so I don't know if that's an issue or not. Around V 4 or so if I recall correctly. I'd be more than willing to try the newer version but a: I don't work there no more, and b: Lyle Lanley has sold them on Microsoft technology...
About three years ago (admittedly, my knowledge is pretty old now) I tested and compared the two. The Netscape LDAP server used up a huge chunk of memory, even sitting idle, and could handle only a few authentication's / searches per second on our dual P-III 750 machine with 1 gig ram. The memory usage, if I recall correctly, was about 50 megs per process (not shared mem, individual memory usage by the way) with a default of something like 5 of them running.
OpenLDAP used about 20 megs of memory total, ramping up to 50 to 100 megs under heavy load. It could handle about 30 to 40 auths / searches a second.
Worse for the Netscape server was that it would just plain stop working after an hour or so of heavy load testing.
We went with OpenLDAP, and wrote our own edit screens for it since at the time it came with nothing very useful to a user (only ldapadd, etc... command line stuff).
After about a year of only handling the web server it was on we pointed our Peoplesoft implementation at it, which proceeded to increase our load from one auth every couple of seconds to about 10 auths a second. Other than the slightly larger number of openldap processes running, we never really noticed the load.
Hope that helps anyone looking at the two. I certainly would hope the Netscape server has gotten better, but everything I've read about it since then seems to say it hasn't.
Microsoft goes on a bit about how much better their commercial software is because they have commercial code reviewers to catch this kind of thing, i.e. people who have a job to do and are getting paid to do it must be doing a better job than the great unwashed masses.
Microsoft tells us they do these kinds of things better, but the reality of the situation is that fixing security issues require a group of people who know what they're doing, and honestly, I don't think Microsoft has a whole lot of those people.