Yep, thanks for correcting me there. I was half asleep and without coffee:-) It is write-back cache that enables a massive speed boost when using InnoDB and the fully ACID settings. (parent probably knows this, but im putting the rest of this message here for anyone else reading the thread) Write back-cache is possible without a battery, however having a working battery makes the write-back cache safe; when power is lost in the middle of a write, the cache keeps this data until the computer regains power, then it is written to disk. It is possible but ill-advised to have a write-back cache and no battery:-) If you do that you shouldn't bother with InnoDB's ACID safe settings. According to the manual, some drives and controllers have write-back cache (and no battery) which is not configurable, so be careful putting your fancy ACID applications on cheap SATA controllers/drives.
My test setup is a Dell 2950 2G ram RAID 1 SAS drives. The controller has a battery and 256m of cache. I had to turn write-back on, as it is configured to write-through by default. I did several unplug tests while I had 50 threads updating different records in the table on one machine while unplugging the DB server. After each unplug test the number of successfully committed transactions matched the data when the machine came back up. Switching between write-back (fast) and write-through (slow) had a very significant impact on the number of write transactions per second.
Disabling the syncs altogether boosted the performance several fold, MySQL was maxing all 4 CPUs and was not bottlenecked on I/O at all. If you can afford to lose 1 second of transactions with these fly-by-the-seat-of-your-pants settings then go for it. Stuff like web forums, content sites, etc could probably survive. Places where the worst case is that someone has to repost is not going to kill you. Online stores, etc should not go that route, as losing a second of transactions could leave you with a lot of billed and not shipped customers! Some people will tell you that with these settings you will have inconsistent state, but they are full of it. You will have state that is transactionally consistent, you just might have a slightly older version of it as the last second of transactions may have been lost. Uncommited transactions are rolled back. InnoDB crash recovery still works fine.
I have not found any significant read performance difference between innodb and myisam on identical tables.
Did they fix that thing where it always sacrifices data integrity for speed?
You can use InnoDB tables with a write-through battery-backed cache and the performance is very similar, yet 100% ACID safe. Dell 2950s come with this standard (not a plug, just something i noticed)
Prior to giving tests, we asked for code samples. Bad mistake.
Only awful samples tell you anything. Good samples are deceptive because the applicant could have spend days cleaning up the code and properly documenting it. Most of the time you don't get a functional sample and even then you have to judge based off of what you think it is supposed to do, without any context of the specifications or requirements.
From an employer standpoint, what is a good test to give? What is the best way to do the coding interview: leave the interviewee alone, stand over his/her shoulder, just sit nearby, or don't bother with real code and just whiteboard something?
Is it better to have them code during the interview (and how long should it last?), or write something for you on their own time? To me, the code someone produces in an hour or two session for a small problem tells me more than something they can take a few days on before getting back to me. Ex do they understand, design, code, test in that order, or backwards? Is it even reasonable to force someone to develop software when they are outside of their normal dev environment?
I've made the mistake of giving overly challenging problems, and I'm leaning toward quality/elegance of solving simple problems over ability to solve complex problems in a short period of time. Anyone care to comment what they've found most useful as an interviewing tool?
In the last round of hiring, we gave out a test that involved parsing some input file, massaging the data slightly based on predefined criteria, and dumping the data to a single DB table. Out of a dozen applicants, nobody got it working, not even the 2 people we hired (1 worked out very well afterwards, 1 didn't). Lessen learned: don't expect applicants to come close to what your dev team does every day in the same time frame.
Can the/. crowd help me out with some suggestions?:-)
It sounds like you have a buggy version. I use MyISAM tables whenever I can get away with it, which happens to be most of the time and I can count on one hand the number of times I've had a problem with the MySQL server itself, and each was an easy fix.
How long is long enough?
<shameless self promotion> I've been building and supporting enterprise class web applications since early 2001 (and other web stuff going back to 95). Most of the stuff I build is hosted at a colo or sold and redistributed to enterprise customers. Some of my MySQL based products are certified and resold by Cisco (hopefully that doesn't scare you;-)
</shameless self promotion>
I've used MS SQL server, PostgreSQL and MySQL but I prefer MySQL for speed, ease of use, and development time. I do thoroughly test each release that I use, but I do the same for any other software I use (open source or otherwise).
And how many users were actively using this system at any given time?
I don't want to slam your project, it sounds very nice, but given that your telemarketer client is too small to need some kind of predialer (they actually look each number up manually?), I'm going to assume its something like 20-50 hits a minute. Thats not exactly proof of performance.
ACID is wothless when your data is worthless. Following Sturgeon's Revelation, mySQL is perfect for most web use.
The inverse of that isn't necessarily true however.
But if you're using a dbms for an actual application -- web-based or not -- you (more than likely) need the data integrity that ACID stives to provide.
Couldn't agree more. Any application, web or not, requires data integrity. ACID does strive to provide this, I totally agree. However, data integrity and ACID are not the same thing, even though they are often treated like it. There is more than one way to skin a cat, and many more ways of ensuring data integrity.
For me, its about saving me time while accomplishing the same task. If I can ensure data integrity and have a DB that is an order of magnitude faster than PostgreSQL at the same time, then I just saved the time and resources needed to scale out PostgreSQL.
Your data is pretty worthless if you can't be bothered with ACID complience to make sure it is consistant.
You don't need a fully ACID compliant DB to guarantee consistent data. It depends on what your data looks like, and what you can get away with. Throwing ACID at everything isn't going to solve your problems if you don't take failure into account in your design. If you do take it into account, then you know what parts of ACID you need.
For instance, in MySQL's MyISAM, any single update or insert is attomic. If thats all you need to do, then you already have the A in ACID. If you can fit your updates into a single query, your data will be consistent even if your app crashes. The update is attomic, therefore it is isolated. MyISAM tables are already durable.
MyISAMs are perfect for read mostly tables, especially ones where you need to perform ad-hoc queries on that can't be efficiently cached.
If you give up A, C, I, and D, of course you get better performance- just like you can get better performance from a wheel-less Yugo if you slide it down a luge track.
I think that a 4 wheeled Yugo that isn't sliding down a luge track to be the best analogy to PostgreSQL's performance I've ever heard!
I've never seen PostgreSQL perform, ever. Thats not to say that it can't or doesn't (perhaps when thrown down a luge track, memcached anyone?). ACID is worthless when you have to restrict how much data you process so that the DB doesn't choke.
It would be more like breaking into your own empty house after you just bought it. It is legal to break and enter your own house, but then again the DMCA does not cover real estate. Virtual estate/property has much more protection.
The contents are irrelevant because the DMCA doesn't deal with the validity of the copy/access protection, or an entities right to install such protection, or any valid reasons for breaking the protection.
I am waiting for some upstart software company to install spyware/adware/junkware/spam/virus/trojan to encrypt my hard drive for ransom, then jail me when I download and run removal programs.
If someone wants to make a statement, then make such a software, make the user click through a EULA, then "protect" all of their data and make it illegal to break the "protection". Then charge them $50 a kbyte to access the data.
Slashdot Mirror
Yep, thanks for correcting me there. I was half asleep and without coffee :-) It is write-back cache that enables a massive speed boost when using InnoDB and the fully ACID settings. (parent probably knows this, but im putting the rest of this message here for anyone else reading the thread) :-) If you do that you shouldn't bother with InnoDB's ACID safe settings. According to the manual, some drives and controllers have write-back cache (and no battery) which is not configurable, so be careful putting your fancy ACID applications on cheap SATA controllers/drives.
Write back-cache is possible without a battery, however having a working battery makes the write-back cache safe; when power is lost in the middle of a write, the cache keeps this data until the computer regains power, then it is written to disk. It is possible but ill-advised to have a write-back cache and no battery
My test setup is a Dell 2950 2G ram RAID 1 SAS drives. The controller has a battery and 256m of cache. I had to turn write-back on, as it is configured to write-through by default. I did several unplug tests while I had 50 threads updating different records in the table on one machine while unplugging the DB server. After each unplug test the number of successfully committed transactions matched the data when the machine came back up. Switching between write-back (fast) and write-through (slow) had a very significant impact on the number of write transactions per second.
Disabling the syncs altogether boosted the performance several fold, MySQL was maxing all 4 CPUs and was not bottlenecked on I/O at all. If you can afford to lose 1 second of transactions with these fly-by-the-seat-of-your-pants settings then go for it. Stuff like web forums, content sites, etc could probably survive. Places where the worst case is that someone has to repost is not going to kill you. Online stores, etc should not go that route, as losing a second of transactions could leave you with a lot of billed and not shipped customers! Some people will tell you that with these settings you will have inconsistent state, but they are full of it. You will have state that is transactionally consistent, you just might have a slightly older version of it as the last second of transactions may have been lost. Uncommited transactions are rolled back. InnoDB crash recovery still works fine.
I have not found any significant read performance difference between innodb and myisam on identical tables.
You can use InnoDB tables with a write-through battery-backed cache and the performance is very similar, yet 100% ACID safe. Dell 2950s come with this standard (not a plug, just something i noticed)
I should have mentioned:
Prior to giving tests, we asked for code samples. Bad mistake.
Only awful samples tell you anything. Good samples are deceptive because the applicant could have spend days cleaning up the code and properly documenting it. Most of the time you don't get a functional sample and even then you have to judge based off of what you think it is supposed to do, without any context of the specifications or requirements.
From an employer standpoint, what is a good test to give?
/. crowd help me out with some suggestions? :-)
What is the best way to do the coding interview: leave the interviewee alone, stand over his/her shoulder, just sit nearby, or don't bother with real code and just whiteboard something?
Is it better to have them code during the interview (and how long should it last?), or write something for you on their own time? To me, the code someone produces in an hour or two session for a small problem tells me more than something they can take a few days on before getting back to me. Ex do they understand, design, code, test in that order, or backwards? Is it even reasonable to force someone to develop software when they are outside of their normal dev environment?
I've made the mistake of giving overly challenging problems, and I'm leaning toward quality/elegance of solving simple problems over ability to solve complex problems in a short period of time. Anyone care to comment what they've found most useful as an interviewing tool?
In the last round of hiring, we gave out a test that involved parsing some input file, massaging the data slightly based on predefined criteria, and dumping the data to a single DB table. Out of a dozen applicants, nobody got it working, not even the 2 people we hired (1 worked out very well afterwards, 1 didn't). Lessen learned: don't expect applicants to come close to what your dev team does every day in the same time frame.
Can the
It sounds like you have a buggy version. I use MyISAM tables whenever I can get away with it, which happens to be most of the time and I can count on one hand the number of times I've had a problem with the MySQL server itself, and each was an easy fix. ;-)
</shameless self promotion>
How long is long enough?
<shameless self promotion> I've been building and supporting enterprise class web applications since early 2001 (and other web stuff going back to 95). Most of the stuff I build is hosted at a colo or sold and redistributed to enterprise customers. Some of my MySQL based products are certified and resold by Cisco (hopefully that doesn't scare you
I've used MS SQL server, PostgreSQL and MySQL but I prefer MySQL for speed, ease of use, and development time. I do thoroughly test each release that I use, but I do the same for any other software I use (open source or otherwise).
And how many users were actively using this system at any given time?
I don't want to slam your project, it sounds very nice, but given that your telemarketer client is too small to need some kind of predialer (they actually look each number up manually?), I'm going to assume its something like 20-50 hits a minute. Thats not exactly proof of performance.
For me, its about saving me time while accomplishing the same task. If I can ensure data integrity and have a DB that is an order of magnitude faster than PostgreSQL at the same time, then I just saved the time and resources needed to scale out PostgreSQL.
You don't need a fully ACID compliant DB to guarantee consistent data. It depends on what your data looks like, and what you can get away with. Throwing ACID at everything isn't going to solve your problems if you don't take failure into account in your design. If you do take it into account, then you know what parts of ACID you need.
For instance, in MySQL's MyISAM, any single update or insert is attomic. If thats all you need to do, then you already have the A in ACID. If you can fit your updates into a single query, your data will be consistent even if your app crashes. The update is attomic, therefore it is isolated. MyISAM tables are already durable.
MyISAMs are perfect for read mostly tables, especially ones where you need to perform ad-hoc queries on that can't be efficiently cached.
I think that a 4 wheeled Yugo that isn't sliding down a luge track to be the best analogy to PostgreSQL's performance I've ever heard!
I've never seen PostgreSQL perform, ever. Thats not to say that it can't or doesn't (perhaps when thrown down a luge track, memcached anyone?). ACID is worthless when you have to restrict how much data you process so that the DB doesn't choke.
It would be more like breaking into your own empty house after you just bought it. It is legal to break and enter your own house, but then again the DMCA does not cover real estate. Virtual estate/property has much more protection.
The contents are irrelevant because the DMCA doesn't deal with the validity of the copy/access protection, or an entities right to install such protection, or any valid reasons for breaking the protection.
I am waiting for some upstart software company to install spyware/adware/junkware/spam/virus/trojan to encrypt my hard drive for ransom, then jail me when I download and run removal programs.
If someone wants to make a statement, then make such a software, make the user click through a EULA, then "protect" all of their data and make it illegal to break the "protection". Then charge them $50 a kbyte to access the data.