The user just wants the fastest response. So, you have to look at the system as a whole; you can't look at the pieces in isolation and ignore the performance features they offer. Also, you have to look at the many different types of users of a system, not just one particular user.
Those questions can be answered quite easily using either a materialized path representation (using something like ltree, or WITH... RECURSIVE.
Materialized path would be an efficient way to answer those questions. However, if you try to use some graph or hierarchical database system, then that will be the only way you can efficiently access your data; with a SQL system, it's easier to allow the user to ask a variety of very different questions without bias toward any one particular type of question.
Consider storing something like an iPod. You can store it at:/hardware/apple/ipod; or/apple/hardware/ipod
The former makes it very hard to find all the products that are made by apple (because you have to search other higher-level categories, like software, etc). The latter makes it very hard to find all hardware products for the same reason (you have to search/ibm/hardware,/microsoft/hardware, etc).
In a relational system, you'd just say something like "where vendor=apple" or "where category=hardware". The relational organization more closely resembles reality because the hierarchy is 100% artificial.
I'm sure you can always find a particular set of queries that run faster on your favorite hierarchical or graph database system, but that's not the point. The point is that it biases the queries you ask (both in terms of ease and also efficiency) so heavily that the mixed queries found in most real businesses will cause problems for a hierarchical/graph databases system.
SQL can't handle most datastructures and complex relations, only very simple one dimensional ones
That shows ignorance of the relational model. Relations are n-dimensional, where n is the number of attributes.
While it does use SQL syntax, in reality you're simply rummaging through a flat file with superspeed.
1. SQLite is so far from the standard you can hardly call it SQL. I think even MySQL is closer to the standard, at least if you turn on the right set of options.
2. What makes you think that SQLite is fast? It has a small footprint, and can be fast in some situations, I'm sure, but I would just assume that it's fast.
Lots of people in business (like accounting, management, and executives) are actually very savvy and able to ask questions very precisely. Particularly if they are just reading data, what's the problem? Not everyone works at a company with a team of DBAs; sometimes the "business people" have to fend for themselves. It's either SQL, or they are downloading the entire thing into spreadsheets. I think SQL is an improvement (of course spreadsheets can still be useful in conjunction with a database system).
This is, in know way, an endorsement of SQL's attempt at a "natural language". I think that was a bad idea. However, to be so dismissive of business people is quite arrogant.
I tend to agree, but there are no serious implementations of alternative relational languages. NULL in particular is an abomination, and I don't think most developers really understand its behavior.
It failed at being a natural language search tool for business people.
But it succeeded at bringing good implementations of an approximation of relational calculus to a lot of people.
I would rather that SQL followed some better language design principles, but it's far better than a non-relational language.
It is possible to go back to a previous state, but then often applications depending on the DB schema break.
I have no idea what you're talking about here. It's not "I feel like resetting the database state on my production machine". It's point-in-time recovery. That means, if you delete or drop something important, you can restore to the time right before you did that, hopefully on a non-production machine and then you merge the old data in where it's supposed to be.
The theme of the manifestos is how to avoid the 'object-relational impedance mismatch'...
Having read a lot of The Third Manifesto, I don't think I'd put it that way at all. For one thing, I don't think the authors use the words "impedance mismatch" anywhere. But correct me if I'm wrong.
But what I'm tired of is people not understanding their data.
I'd agree with that, but it's pretty vague.
A lot of developers put too much emphasis on the writing of data somewhere, without much emphasis on how they're going to interpret it later amongst some other few million pieces of data. If you store everything as a text blob, that severely limits your searching options.
One of the reasons is because RDBMSs offer a lot of tools, like atomicity, durability, backup/restore, centralization, point-in-time-recovery, etc. Many application developers need these things without actually needing the abstraction of a relational system.
I always thought postgresql was a little more "developer friendly", at least to me.
For instance, it has a great set of built-in types with useful operators. You can do date math, array stuff, calculation using precise numbers (numeric), string manipulation, and all that stuff that developers just expect to be able to do in any language. If you can do it in the application, great, but sometimes you are joining based on some condition that requires a little manipulation.
They also do stuff like allowing you to add fields as an O(1) operation without even stopping the application (it requires a table lock, but that should only cause a very short delay). Or writing functions in whatever language you want.
At my old DBA job, one of the things I did early on was write a simple function in perl that fetched some LDAP data and returned the results as a table in postgresql. Now I could join application data against network user data and whip out an ad-hoc report for someone in 10 minutes (which I did on numerous occasions, usually when they weren't expecting the results for at least a day). The developers were impressed at how simple it was, and I could tell they were imagining all the interesting possibilities.
The thing with postgresql is that you, as a developer, never have to leave your developer mindset. You don't have to think of the database as some magical box with a tiny hole where data can go in with "INSERT" and out with "SELECT". With postgresql, if you run into a problem, you can use all of your creativity to solve it, more like a UNIX admin with perl. Postgres is a solid database system, but it gives you a lot of tools to do pretty much whatever you want.
I can't fathom why anyone chooses it when starting out from scratch anymore.
The excuses are quickly running out, that's for sure. Lack of "easy" master/slave replication with readable slave is probably the only serious complaint.
Even in that case, it's not like you can't do it, but it will be a challenge. Slony is my favorite of the current systems (assuming you have to read from the slave), but "easy" is not one of its strong points. Robust and flexible, but not easy.
I might say that SQL standard MERGE would be another thing to make coming from MySQL a little easier (not that MySQL supports MERGE, but it does have some funky ON DUPLICATE key syntax that's better solved by MERGE), but that's a pretty minor complaint.
On the whole, it seems harder and harder to imagine starting a new project with MySQL. If it's a small project and I'm looking for "easy", replication is unnecessary. If not, then PostgreSQL just has so much to offer that makes your life easier through the entire lifecycle of a series of applications that use that data. In that case, learning slony well enough to make it do what you want might be a good investment of all the time you save.
This is the feature that didn't quite make it in 8.4. Look at how much effort and research went into the design. This isn't a situation where someone threw some code together; this is real design and real code by a highly reputable hacker, that answered feedback from other highly reputable hackers.
The only reason this didn't make it is that PostgreSQL has a reputation for getting things right (really right), and that's more important than one feature.
But try it out. If the code hasn't rotted too much, it should work fine.
Check it in a couple days. As I understand it, it was mostly waiting for 8.4.0 to be released.
Online replication. Now you mostly have offline filesystem based replication
The most common replication schemes in postgresql are probably warm standby (which doesn't currently allow reads on the slave), Slony (not builtin) and londiste (not builtin). All of those are online replication (unless you consider warm standby to be offline). Maybe you already know this, but I am just clarifying.
The next release will prioritize hot standby (same as warm standby but you can read the slave), and sync rep as built-in replication systems.
VACUUM now makes use of a "visibility map", which means that it doesn't need to process old data each time VACUUM is run. If you run VACUUM on a large table, and then immediately run it again, the second run will be instant.
The recursive queries are the SQL standard common table expressions, that is, WITH and WITH RECURSIVE.
The window functions is a great addition, but with PostgreSQL it's even better because you can define your own custom window functions with CREATE FUNCTION.
There are also a huge number of little improvements, like "auto explain" which is a module that can automatically log the "EXPLAIN ANALYZE" output when a query takes a long time. This is a great convenience for DBAs, because you don't have to look at long-running queries in the log and attempt to EXPLAIN ANALYZE them manually.
Full disclosure is a common practice elsewhere and doesn't result in the negative consequences TFA claims people are worried about.
It's not a legally requirement though, is it? Just because something is ethical does not mean it should be made into law.
The problem is that you're increasing the stakes for everyone that writes opinion blogs. Before, they just had to avoid libel, inciting violence, and other blatantly illegal speech. Now, rival bloggers can stir up suspicion about your blog, complain to the FTC (and maybe get their readers to join in), and then the FTC might sue.
Keep in mind that the FTC files civil suits, which mean that they don't have to have probable cause. They don't need search warrants, because they can just force you to turn things over during "discovery". By the time they realize that you're doing everything legally, you might be out a lot of money in legal fees.
Who wants to be exposed to that kind of risk if they are making peanuts and just doing it out of interest? They will be afraid to make enemies with rival blogs, and just stick to bland observations that don't challenge the opinions of anyone else.
so I suppose requiring full disclosure or potential conflicts of interest is necessary
It is? It has not been a legal requirement before, as far as I know.
What is worst thing that can possibly happen if we don't pass new laws? People might take bad advice from someone they never should have trusted in the first place, and buy overpriced consumer crap that they don't need, and maybe be disappointed with it.
What's the worst thing that can possibly happen if we do pass new laws? People's legitimate opinions may be silenced on the mere accusation that they aren't disclosing everything that they should. A lot of these bloggers have very little keeping them going aside from personal interest, so even if they are doing everything 100% ethically, an offhand accusation and a letter from a government agency will shut them up quickly. How long before these new laws are applied to public policy opinions, and they can silence underfunded opposition?
for any false claims or failure to disclose conflicts of interest
Since when is disclosing a conflict of interest a legal requirement? Ethical, of course. But a legal requirement? Aren't people free to express their opinions regardless of what their motivations might be?
Some teachers are more talented than others, but I've yet to meet a lazy one.
I have. Perhaps your statement applies to teachers at a university, but not in general.
A lazy teacher will quickly move to an easier (and better paying) job.
That is a myth. Below the university level it's almost impossible to remove a teacher for poor performance, laziness, etc. That means there's no incentive for a teacher to leave, even if they don't keep up with the work required to be an effective teacher. All of their raises are tied to seniority, meaning that if they just continue to be lazy, they will eventually be well-paid and lazy. Staying the course is the MO for a lazy person, almost by definition, so it's unlikely that a lazy person will leave a cushy teaching job with guaranteed raises in the absence of strong incentives.
What a bizarre comment. Lessons are not independent of other lessons; that is just a fact. For many topics, if you don't understand an earlier lesson, you are guaranteed to have a tougher time on the later lessons.
This is a lazy instructor working to maintain his laziness.
When teaching entry-level courses, and even fairly advanced courses, having original assignments each semester has very little benefit aside from preventing cheating. The only laziness he's showing is that he doesn't want to do a lot of extra work to prevent students from cheating, because he's a teacher, not the academic honesty police.
certainly isn't staying with the times
Because a class full of undergrads is on the frontiers of the computer science field? Or because you think the teacher should update their material to match the latest fad?
Imagine if you were to tell a mathematician: "Hey, I just made your numbers better. Now every number can be attached to something, like a blog or twitter post. That way you know where the numbers came from. Closure? What's that?"
Not only is it a bad idea in principle, as it would surely be abused, it also would harm a lot of innocent people like employees and stockholders.
And it means that we'd be treating someone differently under the law based on what they say. That is not free speech, and it is not following the Rule of Law.
Nancy "less co2 ! everyone save ! where's my jet ?" Pelosi
The CIA waterboarded her to make her do that after they lied to her and told her they didn't waterboard anyone.
[end sarcasm]
Pelosi is probably just too accustomed to San Francisco, where saying things like "the Bush administration and the CIA conspired to lie to Congress to protect their torture policies" is taken as an axiom on which to build other arguments.
Slashdot Mirror
I just want the fastest one.
The user just wants the fastest response. So, you have to look at the system as a whole; you can't look at the pieces in isolation and ignore the performance features they offer. Also, you have to look at the many different types of users of a system, not just one particular user.
Then design queries to answer questions such as
Those questions can be answered quite easily using either a materialized path representation (using something like ltree, or WITH ... RECURSIVE.
Materialized path would be an efficient way to answer those questions. However, if you try to use some graph or hierarchical database system, then that will be the only way you can efficiently access your data; with a SQL system, it's easier to allow the user to ask a variety of very different questions without bias toward any one particular type of question.
Consider storing something like an iPod. You can store it at: /hardware/apple/ipod; or /apple/hardware/ipod
The former makes it very hard to find all the products that are made by apple (because you have to search other higher-level categories, like software, etc). The latter makes it very hard to find all hardware products for the same reason (you have to search /ibm/hardware, /microsoft/hardware, etc).
In a relational system, you'd just say something like "where vendor=apple" or "where category=hardware". The relational organization more closely resembles reality because the hierarchy is 100% artificial.
I'm sure you can always find a particular set of queries that run faster on your favorite hierarchical or graph database system, but that's not the point. The point is that it biases the queries you ask (both in terms of ease and also efficiency) so heavily that the mixed queries found in most real businesses will cause problems for a hierarchical/graph databases system.
SQL can't handle most datastructures and complex relations, only very simple one dimensional ones
That shows ignorance of the relational model. Relations are n-dimensional, where n is the number of attributes.
While it does use SQL syntax, in reality you're simply rummaging through a flat file with superspeed.
1. SQLite is so far from the standard you can hardly call it SQL. I think even MySQL is closer to the standard, at least if you turn on the right set of options.
2. What makes you think that SQLite is fast? It has a small footprint, and can be fast in some situations, I'm sure, but I would just assume that it's fast.
Business people shouldn't be programming.
Lots of people in business (like accounting, management, and executives) are actually very savvy and able to ask questions very precisely. Particularly if they are just reading data, what's the problem? Not everyone works at a company with a team of DBAs; sometimes the "business people" have to fend for themselves. It's either SQL, or they are downloading the entire thing into spreadsheets. I think SQL is an improvement (of course spreadsheets can still be useful in conjunction with a database system).
This is, in know way, an endorsement of SQL's attempt at a "natural language". I think that was a bad idea. However, to be so dismissive of business people is quite arrogant.
I really hate SQL.
I tend to agree, but there are no serious implementations of alternative relational languages. NULL in particular is an abomination, and I don't think most developers really understand its behavior.
It failed at being a natural language search tool for business people.
But it succeeded at bringing good implementations of an approximation of relational calculus to a lot of people.
I would rather that SQL followed some better language design principles, but it's far better than a non-relational language.
It is possible to go back to a previous state, but then often applications depending on the DB schema break.
I have no idea what you're talking about here. It's not "I feel like resetting the database state on my production machine". It's point-in-time recovery. That means, if you delete or drop something important, you can restore to the time right before you did that, hopefully on a non-production machine and then you merge the old data in where it's supposed to be.
The theme of the manifestos is how to avoid the 'object-relational impedance mismatch'...
Having read a lot of The Third Manifesto, I don't think I'd put it that way at all. For one thing, I don't think the authors use the words "impedance mismatch" anywhere. But correct me if I'm wrong.
But what I'm tired of is people not understanding their data.
I'd agree with that, but it's pretty vague.
A lot of developers put too much emphasis on the writing of data somewhere, without much emphasis on how they're going to interpret it later amongst some other few million pieces of data. If you store everything as a text blob, that severely limits your searching options.
One of the reasons is because RDBMSs offer a lot of tools, like atomicity, durability, backup/restore, centralization, point-in-time-recovery, etc. Many application developers need these things without actually needing the abstraction of a relational system.
I always thought postgresql was a little more "developer friendly", at least to me.
For instance, it has a great set of built-in types with useful operators. You can do date math, array stuff, calculation using precise numbers (numeric), string manipulation, and all that stuff that developers just expect to be able to do in any language. If you can do it in the application, great, but sometimes you are joining based on some condition that requires a little manipulation.
They also do stuff like allowing you to add fields as an O(1) operation without even stopping the application (it requires a table lock, but that should only cause a very short delay). Or writing functions in whatever language you want.
At my old DBA job, one of the things I did early on was write a simple function in perl that fetched some LDAP data and returned the results as a table in postgresql. Now I could join application data against network user data and whip out an ad-hoc report for someone in 10 minutes (which I did on numerous occasions, usually when they weren't expecting the results for at least a day). The developers were impressed at how simple it was, and I could tell they were imagining all the interesting possibilities.
The thing with postgresql is that you, as a developer, never have to leave your developer mindset. You don't have to think of the database as some magical box with a tiny hole where data can go in with "INSERT" and out with "SELECT". With postgresql, if you run into a problem, you can use all of your creativity to solve it, more like a UNIX admin with perl. Postgres is a solid database system, but it gives you a lot of tools to do pretty much whatever you want.
I can't fathom why anyone chooses it when starting out from scratch anymore.
The excuses are quickly running out, that's for sure. Lack of "easy" master/slave replication with readable slave is probably the only serious complaint.
Even in that case, it's not like you can't do it, but it will be a challenge. Slony is my favorite of the current systems (assuming you have to read from the slave), but "easy" is not one of its strong points. Robust and flexible, but not easy.
I might say that SQL standard MERGE would be another thing to make coming from MySQL a little easier (not that MySQL supports MERGE, but it does have some funky ON DUPLICATE key syntax that's better solved by MERGE), but that's a pretty minor complaint.
On the whole, it seems harder and harder to imagine starting a new project with MySQL. If it's a small project and I'm looking for "easy", replication is unnecessary. If not, then PostgreSQL just has so much to offer that makes your life easier through the entire lifecycle of a series of applications that use that data. In that case, learning slony well enough to make it do what you want might be a good investment of all the time you save.
git repo
Design spec
This is the feature that didn't quite make it in 8.4. Look at how much effort and research went into the design. This isn't a situation where someone threw some code together; this is real design and real code by a highly reputable hacker, that answered feedback from other highly reputable hackers.
The only reason this didn't make it is that PostgreSQL has a reputation for getting things right (really right), and that's more important than one feature.
But try it out. If the code hasn't rotted too much, it should work fine.
That was a feature that caught my eye, too, which is why I mentioned it. It seems like a small thing, but it makes life easier.
Not only that, but it allows you to explain nested plans, like those from within functions executed in the outer plan.
In place migration tool is still in beta
Check it in a couple days. As I understand it, it was mostly waiting for 8.4.0 to be released.
Online replication. Now you mostly have offline filesystem based replication
The most common replication schemes in postgresql are probably warm standby (which doesn't currently allow reads on the slave), Slony (not builtin) and londiste (not builtin). All of those are online replication (unless you consider warm standby to be offline). Maybe you already know this, but I am just clarifying.
The next release will prioritize hot standby (same as warm standby but you can read the slave), and sync rep as built-in replication systems.
One phrase: repeatable SQL.
How about: "insert ... select ... where not in (...)"?
Upgrade in place is done via pg_migrator.
VACUUM now makes use of a "visibility map", which means that it doesn't need to process old data each time VACUUM is run. If you run VACUUM on a large table, and then immediately run it again, the second run will be instant.
The recursive queries are the SQL standard common table expressions, that is, WITH and WITH RECURSIVE.
The window functions is a great addition, but with PostgreSQL it's even better because you can define your own custom window functions with CREATE FUNCTION.
There are also a huge number of little improvements, like "auto explain" which is a module that can automatically log the "EXPLAIN ANALYZE" output when a query takes a long time. This is a great convenience for DBAs, because you don't have to look at long-running queries in the log and attempt to EXPLAIN ANALYZE them manually.
Full disclosure is a common practice elsewhere and doesn't result in the negative consequences TFA claims people are worried about.
It's not a legally requirement though, is it? Just because something is ethical does not mean it should be made into law.
The problem is that you're increasing the stakes for everyone that writes opinion blogs. Before, they just had to avoid libel, inciting violence, and other blatantly illegal speech. Now, rival bloggers can stir up suspicion about your blog, complain to the FTC (and maybe get their readers to join in), and then the FTC might sue.
Keep in mind that the FTC files civil suits, which mean that they don't have to have probable cause. They don't need search warrants, because they can just force you to turn things over during "discovery". By the time they realize that you're doing everything legally, you might be out a lot of money in legal fees.
Who wants to be exposed to that kind of risk if they are making peanuts and just doing it out of interest? They will be afraid to make enemies with rival blogs, and just stick to bland observations that don't challenge the opinions of anyone else.
so I suppose requiring full disclosure or potential conflicts of interest is necessary
It is? It has not been a legal requirement before, as far as I know.
What is worst thing that can possibly happen if we don't pass new laws? People might take bad advice from someone they never should have trusted in the first place, and buy overpriced consumer crap that they don't need, and maybe be disappointed with it.
What's the worst thing that can possibly happen if we do pass new laws? People's legitimate opinions may be silenced on the mere accusation that they aren't disclosing everything that they should. A lot of these bloggers have very little keeping them going aside from personal interest, so even if they are doing everything 100% ethically, an offhand accusation and a letter from a government agency will shut them up quickly. How long before these new laws are applied to public policy opinions, and they can silence underfunded opposition?
for any false claims or failure to disclose conflicts of interest
Since when is disclosing a conflict of interest a legal requirement? Ethical, of course. But a legal requirement? Aren't people free to express their opinions regardless of what their motivations might be?
Some teachers are more talented than others, but I've yet to meet a lazy one.
I have. Perhaps your statement applies to teachers at a university, but not in general.
A lazy teacher will quickly move to an easier (and better paying) job.
That is a myth. Below the university level it's almost impossible to remove a teacher for poor performance, laziness, etc. That means there's no incentive for a teacher to leave, even if they don't keep up with the work required to be an effective teacher. All of their raises are tied to seniority, meaning that if they just continue to be lazy, they will eventually be well-paid and lazy. Staying the course is the MO for a lazy person, almost by definition, so it's unlikely that a lazy person will leave a cushy teaching job with guaranteed raises in the absence of strong incentives.
That'll teach them not to be perfect!
What a bizarre comment. Lessons are not independent of other lessons; that is just a fact. For many topics, if you don't understand an earlier lesson, you are guaranteed to have a tougher time on the later lessons.
What policy are you suggesting here?
This is a lazy instructor working to maintain his laziness.
When teaching entry-level courses, and even fairly advanced courses, having original assignments each semester has very little benefit aside from preventing cheating. The only laziness he's showing is that he doesn't want to do a lot of extra work to prevent students from cheating, because he's a teacher, not the academic honesty police.
certainly isn't staying with the times
Because a class full of undergrads is on the frontiers of the computer science field? Or because you think the teacher should update their material to match the latest fad?
Mod parent up.
Imagine if you were to tell a mathematician: "Hey, I just made your numbers better. Now every number can be attached to something, like a blog or twitter post. That way you know where the numbers came from. Closure? What's that?"
Not only is it a bad idea in principle, as it would surely be abused, it also would harm a lot of innocent people like employees and stockholders.
And it means that we'd be treating someone differently under the law based on what they say. That is not free speech, and it is not following the Rule of Law.
So, would we be allowed to opt out of Social Security, then?
Nancy "less co2 ! everyone save ! where's my jet ?" Pelosi
The CIA waterboarded her to make her do that after they lied to her and told her they didn't waterboard anyone.
[end sarcasm]
Pelosi is probably just too accustomed to San Francisco, where saying things like "the Bush administration and the CIA conspired to lie to Congress to protect their torture policies" is taken as an axiom on which to build other arguments.