More conflation. You don't bother to ask my opinion on anything; you just project a model what you think an archetypal 'denier' is onto me. I accept HIV causes AIDS, evolution, all the things you list; because of these things when I have looked into them; I have become satisfied with the veracity of the evidence. Now climate science is something I have been drawn into because it has been positioned as a community issue. I originally accepted it on good faith, but more I looked the more doubts I had. I've even gone to the point of getting some CC papers published by my national weather bureau and replicating the results. I was appalled by poor statistical and analytical workmanship demonstrated. I want them to redo it and do a better job of it; because I want to be able to cultivate an opinion on this subject I can have confidence in; yet people such as yourself uncritically grovel at the the feet of those you presume to have expertise. It matters me little whether or not CAGW is real, what matters is that I cultivate a proper understanding of the issue, so I can respond optimally.
But reading your post, you sir are a bore and a fool. The way you went ahead and wrote sentence after sentence dissecting my point of view on a topic without even bothering to ask me what my point of view is; assuming I read a book years ago or something without even bothering to ask how I came about my point of view. You are clearly an idiot who has an overinflated view of your handle on this specific topic and an overinflated view of your own intelligence overall, an uncritical person and thus someone who is easily manipulated.
Funny, most 'deniers' I've met are educated and intellectually competent people, typically engineers of various disciplines, educators, academics. So the hypothesis that 'deniers' are conspiracy nutters is falsified; as such it is a straw-man. But feel free to cling onto your examples of people you have met who are nutters (yes there are plenty of them out there; and I have met a few), as a basis for rejecting any and all criticism of your point of view and allow you to cultivate absolute conviction that beyond all reasonable doubt your are correct.
Good for you for moving onto the problem of saving the human race from this modern reincarnation of doomsday hysteria. I'm sure in generations to come they'll build statues of you and celebrate how awesome you were.
It is remarkable how you insist on conflating opinions you don't like to the point where there are ridiculous and thus easily dismissed; inspite of the fact I not-so-subtly pointed out that this is what you are doing. I guess I could continue to try and articulate my point of view in the hope that we could eventually reach a point where a real conversation would commence, but I suspect that it would be just as rewarding talk climate science with you as it would be with my pet dog.
You've completely missed the possibility of group-think within a tight community of people for whom professional diligence, competency and quality, critical workmanship have been substantially weakened by a tribal quasi-religious zeal to save the human race. Go read Mackay's 'Popular delusions and the madness of crowds' to appreciate, how generation after generation after generation, our civilisation uncritically rushes into some new ridiculous mass-belief. Although the book is quite old now; we are no different.
But by all means, cling to the strawman that sceptics are conspiracy theory nutters if you wish. It will save you the hassle and bother of having to properly consider and analyse the views of those whom you disagree with.
Flawed analogy. It is not about seeking independent diagnosis and then selecting the one which was more popular (how long is the emperors nose problem). It is about analysing how a diagnosis/recommendation is arrived at and determining if the practitioner performed their professional duties with diligence and competency; if you lack the reasoning functions and capacity to analyse work and you uncritically assume that someone else is policing them; you can look forward to going through life been duped and manipulated by others; be it carpenters, doctors and climate scientists. If a practitioner is confident in their diagnosis they have no reason to fear an adversarial critique of their work. And like the parent said, an adversarial review is far more useful for everyone than a friendly one.
My sister-in-law, a rural citizen in a developing country went to a local GP to inspect a breast lump. The doctor proscribed her a very expensive drug. I researched the drug name on the net and discovered it is nothing more than a basic anti-inflammatory. I have no medical background, but I smelled a rat. Put her on a plane to the capital city to get some proper medical diagnostic treatment straight away.
If Wikileaks positioned themselves as a whistleblower safe harbour; then there would be no contradiction.
But if people in Wikileaks and within the institution's orbit get preachy about the necessity of an open and free society then yeah; they are hypocrites.
I guess they could say their ultimate goal is open and free society and whistleblower support is a first tentative step along the way. (whether this is actually achievable is beyond my capacity to visualise, I suspect that it is impossible to construct a broad community that is completely open and transparent. Human institutions are inherently imperfect and there are some conditions, that seem to me at least, that they will never be able satisfy with any stability or longevity.)
Not least it's ability to make seemingly educated adults believe absurd things, like this article or that the end of civilisation as we know will occur in their life time. (Who else believed that the end of days was going to occur within the span of their natural life?).
If only there was a way we could harness this awesome power... and use it to fuel our civilisation. Go beyond fossil fuel with global warming power. There is no problem it can't create.
Spot on. Actual verification of skill in interview/recruitment process is essential. If you don't get them to show you code or write code then you are asking for trouble.
I even got QA candidates to validate they can actually do the job. I coded up piece of software which was deliberately buggy, wrote a spec for it. Candidate test has to write and execute test plan as part of interview process. The more comprehensive & professional the plan, the more bugs they found, the more likely they were to get the job. Best tester I ever hired even found a real bug, not a deliberate one; but he was shy and did not interview well. Also best programmer I ever hired was shy and did not interview well either. But he absolutely killed the programming test dead. Made him an offer on the same day he did the test.
No need to reinvent the wheel. Plenty of stuff out there, based on functional programming model which by design can be setup to parallelise well. I know some folk messing around with this: not my particular area of interest, but demonstrates that this is a well understood problem space with alot of clever people already having committed alot of hours of brainwork over long periods of time to progress solutions in this problem space.
Mercury Programming Language
You're trying to portray humans as insignificant specs
I made no such statement, in fact I implicitly say the opposite when I said Will not be content until... and not exerting any influence over the biosystem. If you parse what I said I assert that human's are indeed influencing the environment; I just did not quantify by how much. For what it's worth, I believe we are exerting a hell of alot of influence; and part of that influence is indeed destructive; but I consider the notion that we are approaching end-of-days as lacking merit. Paul Ehrlich in 1970 claimed we were all boned and that civilization as we know it would not make it to the new millennium and yet here we, safer, more secure and more capable than 40 years ago, yet we are bickering about our capacity to see out the next 20 years or so. No shortage of fools through history who tried to convince other fools the end of the world is nigh. My primary force of polemic is to assert that eco warriers are closet Luddites. So that's two Wooshes so far.
Parent is merely pointing out that some notion of 'natural' is an ethereal thing open to wild range of interpretation and that the biosystem on Earth is dynamic and changing all the time.
The position of eco warriors is simply thus: human/progress bad, nature good. Even nature that includes feral wildfires that destroy millions of hectares of forest and every living thing in it . Will not be content until we are back in the stone age worshipping some mythical GAIA eden and not exerting any influence over the biosystem what-so-ever. Bunch of nutters that lack the intelligence/imagination to properly comprehend how their extraordinary quality of life and prosperity is intrinsically tied to our success as a innovative, intelligent, technology using, problem solving and environment shaping species.
And NATing everything is not going to be disruptive and cost a lot of money?
NAT is already here and in widespread use in every small office and multi device household; whereas ipv6 is not. To insist some sort of cost equivalence between the two projects; where option one involves hacking an existing framework to extend the network's reach, vs swapping in a brand new network on a global scale; is just staggeringly irrational. By all means, have your ipv6 if it is so precious to you, but when you break, in probability, the internet during the transition stage, which our civilization has now become heavily coupled to; be prepared to answer your critics.
NAT or no NAT, IPv4 is no longer viable for widespread use.
And so say the religious zealots. In all probability your ipv6 utopia will arrive; and in all probability, the disruption this transition I suspect will cause, will impact the viability and usefulness of the Internet for a number of years and it will become regular topic of discussion amongst general population and media.
Why not? This is how the overwhelming majority of people interface with the internet anyway: content consumption. ipv6; by virtue of the reality of the fact we are not running it yet, appears to be a project failure in terms of it's stated goal to supercede ipv4. We could press ahead with it, or consider alternatives such as NAT.
Most folk I know who need an IP address fall into one of two categories:
People who p2p fileshare. (Services like Skype and VOIP solve issue of NAT by having peer clients send comms via a intermediate node server)
People who run webservers. I know hardly anyone who does this from their bedroom these days. Thanks to VPS, Amazon/Rackspace etc. Cost is next to nothing
The precise problem many folk here have with going to NAT I fail to fully grok. It will not limit how you can use the Internet; but it will modify the way certain types of problems are solved. Big deal; this is network protocol stuff, and working around problems (such as fact that TCP/IP is unicast, or HTTP is client/server send/receive) is par for the course. Maybe some people like their Internet 'pure' or something; me I try to take a more pragmatic approach. ipv6 transition I fear is going to be massively disruptive over a period of at least 2 years and it is going to cost us all alot of money. Maybe Utopia indeed awaits us on the the otherside, but having been promised Utopia many times on many different disruptive technology transition projects; I can't help but feel a little cynical
Thanks for the primer. Actually, I've been working with DB's for over 15 years (mostly RDBMS), and considered by those who have worked with my directly as being quite skillful at modelling large volumes of data and being able to extract it efficiently.
My original question remains, how an object graph database solves this problem faster than a SQL database. Now I know, for a fact, that a non-SQL relational database can solve the specific use case quite efficiently, but the SQL abstraction layer over the top of the relational database removes the capacity to access the specific query plan that is useful for this use case. The OP's point is slightly different to my own, he claims that the SQL statement is too cumbersome, and for people new to SQL, he is probably correct.
Maybe object graph system provides a means to make the query expression simple. I know from experience that OR layers such as JPQL can certainly help in this specific case. But the underlying data fetch, when dealing with large volumes of data (billions of transaction records and millions of customer records) is tediously slow; and performance is my primary interest. SQL is great because it centralises the concern of cost planning, but it's plan vocabulary is limited by the language, and also under some circumstances you do not want a statistics based cost planner at all (but that's a different issue). I think that in order to understand whether an object graph system can actually deliver a better result, one would need to benchmark and one would need to understand how the plan is converted into b-tree scans/walks and tuple fetches to understand it's proper cost.
Thx. I have trialed this solution previously with mixed results depending on volume/cardinality of data.
The OP's point was that the query, and how it is expressed, is complicated. For someone relatively new to SQL, this sort of query is beyond them; which compared to alternative/older methods of working with relational query data I tend to agree. My point was slightly tangental to this. My point was that not only is it complicated, but the plan when it hits the database is inefficient too. The transaction tuples are fetched twice. Once to determine the correct max tuple, and a second time to actually get the tuple. This is a limitation of SQL itself.
Compared to my list of other things I dislike about SQL, this one is fairly trivial. There once was I time were I just got on with doing SQL and never gave it much thought if it was well designed or not. But a few years ago I heard a colleague complain bitterly about it (not about relational databases, but about SQL itself). And it planted a seed in my mind, and I have been finding and cataloguing imperfections ever since. My single biggest problem with SQL is that it is confused about who it's intended audience is. It is not clear if it is targeting programmers, DBAs or end users and it fails to serve any of these audiences particularly well. My next biggest problem with SQL is that relationships are not strongly typed/enforced. I can join anything to anything, I can join a customer_id to account_balance and SQL will not complain. I enjoy working with relational databases and particularly with big data, trying to extract interesting facts from massive data sets and doing it efficiently. But, after 10+ years of doing SQL, I've come to dislike it; but put up with it.
Thanks for the links. Look like interesting products.
I did not mean to come off as rude, but I did. My apologies. In prior post you said "But that is not a problem of SQL!!! It is a problem of relational data bases.". Can you please clarify this/point me towards a clarified. It is not evident to me how a object graph database somehow provides a efficient and meaningful solution to this specific generalised problem and how precisely it does this presumably by avoiding some unstated limitation of RDBMS.
I do not think you are fully groked the example, and why it fails to optimize well via SQL and why SQL itself is the problem.
Query one is select cust_id,max(timestamp) from customers. This in theory can optimize well. But then the output of this needs to be pushed into another query in order to extract actual txn tuples. Which results in a a b-tree root to leaf walk into the transaction table for every customer. Even though the database has already discovered those tuples. It is both redundant and inefficient; and strictly a limitation of SQL
An alternative I suggest and am currently using is to use window functioning; select rank() and filter rank output. Problem with this is you do a full b-tree date-to-date walk when under many circumstances a more efficient traversal of the b-tree, skipping parts of the walk, is theoretically possible.
I am loving this thread. People who appear to know little about the underlying mechanics of how databases perform coming forward and saying 'use magic elixir X!'. OO databases to solve this problem? Are you serious and do you even know what you are talking about? An 'OO' database is just a relational database that includes a notion of inheritance. i.e. PostgreSQL. Though it may be possible that you may be talking about something radically different, in which case how about you point me towards some links on your 'magic elixir X' and detail on how it elegantly solves this specific generalised problem.
The problem again, is SQL itself. SQL is an abstraction of relational indexed queries. the abstraction fails to to provide a meaningful ways to exploit certain possibilities under certain generalised use cases, and these limitations have nothing to do with the underlying technology but the abstraction layer. As I indicated in my original post, the problem is solvable if you have access to the b-trees; but difficult/cumbersome if you are working through SQL.
Of course this is not necessarily a basis for throwing out SQL because it fails to deliver on on generalised case; but it is just one of many issues I have with SQL as a language.
Actually, the example makes perfect sense and it is a problem I've encountered regularly and one of my many bug bears with SQL.
The generalized problem is 'find last txn' for every customer before date X.
Now if you can access the btree directly, you can efficiently walk it and get this data. But with SQL you cannot efficently get it. PostgreSQL windowing functions make getting such data simple in terms of constructing the SQL (once you get your head around windowing functions), but these queries walk large sections of the dataset and are not optimized fully.
The parents actual example is 'find latest txn'. This can be simplified by materializing details of the last transaction to the customer record on update. But this only works for this specific case. The generalised problem is not readily solved in SQL or at least I have not found a nice solution. The best I can come up with is to materialize data for a given date granularity; (i.e. first of every month). It works to a degree, but the solution is ugly and I know better solutions are possible (depending on cardinality of the data) if you are closer to the metal and are able to bypass the SQL layer.
Hear Hear! Nudity in art that is sexually playful or provocative rocks (M and/or F); celebrate this part of our natures. Whiny art depicting weakness or insecurity or fear or whatever is just tediously dull and turgid emo stuff. This notion of naturist de-sexualization seems to me to only seek a duller and less interesting world.
Either you are a troll or you are so embedded in this dispute that you are completely unable to approach this issue with a rational critical mind.
This thread of conversation has nothing to do whatsoever with making apologies for the behaviour of Palistinians or any other group that has inflicted harm upon the Jewish community. What part of "it's really hard to grant either side any sort of moral authority." are you failing to comprehend?
Cult: A group or movement exhibiting a great or excessive devotion or dedication to some person, idea, or thing and employing unethically manipulative techniques of persuasion and control . . . designed to advance the goals of the group's leaders to the actual or possible detriment of members, their families, or the community.
I accept your broad point about the word "cult" being a heavily emotionally loaded word; and as such it is difficult to have a rational discussion of this nature with the "c" word being dropped. But I disagree that the word to be sufficiently powerful/tainted enough to implicate any notion of "hate speech". Personally, I have no emotional reaction to the word; it is a neutral, useful adjective to me and when I use it my intent is to convey a factual, rational description (at least factual from my understanding of whatever issue is under scrutiny). Probably because I have been hanging around the issue of Scientology for years now; on account of my peer group 10 years ago falling into Scientology and the ensuing excitement as I with an open yet skeptical mindset, grasped for a basis of opinion on this particular organization; I am desensitised to the emotional impact of this particular word. In my estimation, the word, "cult" as I define it above is appropriate description of Scientology.
I am in the camp that says no hints is the lesser of two evils; yet I am of the view that the SQL language standard is deeply flawed and this whole argument persists because of the flaws in SQL itself. Recognizing and correcting those flaws will close the argument.
SQL implicitly requires a cost based planner, for everything. CBP is great for reporting and analytics, but CBP on primary key OLTP operations keep me awake at night. Some things you just want the software to always use a precise b-tree index in order to get consistent and predictable results. Now sure, the CBP under some rare circumstances will find an even better plan then the intended b-tree, but more often then not, it wets the bed, because of bad histograms of whatever and destroys the application by b-tree walking a utterly inappropriate key or seq scanning. CBP require careful feeding and handling.
Worse, I've have yet to work with a DBA who even remotely understands this. If I had a dollar for every argument I had with a DBA on such issues and every hour I spent benchmarking to demonstrate that the preconceptions of these "professionals" are false, I'd be a few thou up easily.
SQL databases are just too complicated for the average IT professional, let alone the average person. And their proliferation into even desktop software, such as various accounting packages, is a development that will keep our industry on it's toes for some time to come.
Wikileaks reached a huge audience with titillating trivia and entertaining news sound bites. I would assert that the core audience that consumed the 'Batman and Robin' stuff and talked about it on twitter, facebook and around the water cooler paid little heed to the harder stuff, and actually studiously ignored it; the content and it's implications. Wikileaks have certainly increased awareness and engagement in 'hard issues'; yet I am doubtful that their success in doing this radically surpasses any of their notable predecessors.
I personally think it is impossible to do so. Because majority of the population, beyond minority of young passionate activists, just don't engage with such information unless it becomes a real and immediate threat to their personal pursuits for continued prosperity. For me wikileaks disseminating information is nothing new or interesting; their methods are not novel nor is the information they disseminating anything substantially new. The promise of wikileaks that interests me is the possibility of providing support and safe harbour to whistle-blowers. Life of a whistleblower is a hard, risky and lonely life: any institution that can ease this and lower the barriers preventing people from blowing whistles deserves our full support and respect.
Slashdot Mirror
More conflation. You don't bother to ask my opinion on anything; you just project a model what you think an archetypal 'denier' is onto me. I accept HIV causes AIDS, evolution, all the things you list; because of these things when I have looked into them; I have become satisfied with the veracity of the evidence. Now climate science is something I have been drawn into because it has been positioned as a community issue. I originally accepted it on good faith, but more I looked the more doubts I had. I've even gone to the point of getting some CC papers published by my national weather bureau and replicating the results. I was appalled by poor statistical and analytical workmanship demonstrated. I want them to redo it and do a better job of it; because I want to be able to cultivate an opinion on this subject I can have confidence in; yet people such as yourself uncritically grovel at the the feet of those you presume to have expertise. It matters me little whether or not CAGW is real, what matters is that I cultivate a proper understanding of the issue, so I can respond optimally.
But reading your post, you sir are a bore and a fool. The way you went ahead and wrote sentence after sentence dissecting my point of view on a topic without even bothering to ask me what my point of view is; assuming I read a book years ago or something without even bothering to ask how I came about my point of view. You are clearly an idiot who has an overinflated view of your handle on this specific topic and an overinflated view of your own intelligence overall, an uncritical person and thus someone who is easily manipulated.
Funny, most 'deniers' I've met are educated and intellectually competent people, typically engineers of various disciplines, educators, academics. So the hypothesis that 'deniers' are conspiracy nutters is falsified; as such it is a straw-man. But feel free to cling onto your examples of people you have met who are nutters (yes there are plenty of them out there; and I have met a few), as a basis for rejecting any and all criticism of your point of view and allow you to cultivate absolute conviction that beyond all reasonable doubt your are correct.
Good for you for moving onto the problem of saving the human race from this modern reincarnation of doomsday hysteria. I'm sure in generations to come they'll build statues of you and celebrate how awesome you were.
It is remarkable how you insist on conflating opinions you don't like to the point where there are ridiculous and thus easily dismissed; inspite of the fact I not-so-subtly pointed out that this is what you are doing. I guess I could continue to try and articulate my point of view in the hope that we could eventually reach a point where a real conversation would commence, but I suspect that it would be just as rewarding talk climate science with you as it would be with my pet dog.
You've completely missed the possibility of group-think within a tight community of people for whom professional diligence, competency and quality, critical workmanship have been substantially weakened by a tribal quasi-religious zeal to save the human race. Go read Mackay's 'Popular delusions and the madness of crowds' to appreciate, how generation after generation after generation, our civilisation uncritically rushes into some new ridiculous mass-belief. Although the book is quite old now; we are no different.
But by all means, cling to the strawman that sceptics are conspiracy theory nutters if you wish. It will save you the hassle and bother of having to properly consider and analyse the views of those whom you disagree with.
Flawed analogy. It is not about seeking independent diagnosis and then selecting the one which was more popular (how long is the emperors nose problem). It is about analysing how a diagnosis/recommendation is arrived at and determining if the practitioner performed their professional duties with diligence and competency; if you lack the reasoning functions and capacity to analyse work and you uncritically assume that someone else is policing them; you can look forward to going through life been duped and manipulated by others; be it carpenters, doctors and climate scientists. If a practitioner is confident in their diagnosis they have no reason to fear an adversarial critique of their work. And like the parent said, an adversarial review is far more useful for everyone than a friendly one.
My sister-in-law, a rural citizen in a developing country went to a local GP to inspect a breast lump. The doctor proscribed her a very expensive drug. I researched the drug name on the net and discovered it is nothing more than a basic anti-inflammatory. I have no medical background, but I smelled a rat. Put her on a plane to the capital city to get some proper medical diagnostic treatment straight away.
If Wikileaks positioned themselves as a whistleblower safe harbour; then there would be no contradiction.
But if people in Wikileaks and within the institution's orbit get preachy about the necessity of an open and free society then yeah; they are hypocrites.
I guess they could say their ultimate goal is open and free society and whistleblower support is a first tentative step along the way. (whether this is actually achievable is beyond my capacity to visualise, I suspect that it is impossible to construct a broad community that is completely open and transparent. Human institutions are inherently imperfect and there are some conditions, that seem to me at least, that they will never be able satisfy with any stability or longevity.)
Not least it's ability to make seemingly educated adults believe absurd things, like this article or that the end of civilisation as we know will occur in their life time. (Who else believed that the end of days was going to occur within the span of their natural life?).
If only there was a way we could harness this awesome power... and use it to fuel our civilisation. Go beyond fossil fuel with global warming power. There is no problem it can't create.
Spot on. Actual verification of skill in interview/recruitment process is essential. If you don't get them to show you code or write code then you are asking for trouble.
I even got QA candidates to validate they can actually do the job. I coded up piece of software which was deliberately buggy, wrote a spec for it. Candidate test has to write and execute test plan as part of interview process. The more comprehensive & professional the plan, the more bugs they found, the more likely they were to get the job. Best tester I ever hired even found a real bug, not a deliberate one; but he was shy and did not interview well. Also best programmer I ever hired was shy and did not interview well either. But he absolutely killed the programming test dead. Made him an offer on the same day he did the test.
No need to reinvent the wheel. Plenty of stuff out there, based on functional programming model which by design can be setup to parallelise well. I know some folk messing around with this: not my particular area of interest, but demonstrates that this is a well understood problem space with alot of clever people already having committed alot of hours of brainwork over long periods of time to progress solutions in this problem space. Mercury Programming Language
You're trying to portray humans as insignificant specs
I made no such statement, in fact I implicitly say the opposite when I said Will not be content until ... and not exerting any influence over the biosystem. If you parse what I said I assert that human's are indeed influencing the environment; I just did not quantify by how much. For what it's worth, I believe we are exerting a hell of alot of influence; and part of that influence is indeed destructive; but I consider the notion that we are approaching end-of-days as lacking merit. Paul Ehrlich in 1970 claimed we were all boned and that civilization as we know it would not make it to the new millennium and yet here we, safer, more secure and more capable than 40 years ago, yet we are bickering about our capacity to see out the next 20 years or so. No shortage of fools through history who tried to convince other fools the end of the world is nigh. My primary force of polemic is to assert that eco warriers are closet Luddites. So that's two Wooshes so far.
Woosh
Parent is merely pointing out that some notion of 'natural' is an ethereal thing open to wild range of interpretation and that the biosystem on Earth is dynamic and changing all the time.
The position of eco warriors is simply thus: human/progress bad, nature good. Even nature that includes feral wildfires that destroy millions of hectares of forest and every living thing in it . Will not be content until we are back in the stone age worshipping some mythical GAIA eden and not exerting any influence over the biosystem what-so-ever. Bunch of nutters that lack the intelligence/imagination to properly comprehend how their extraordinary quality of life and prosperity is intrinsically tied to our success as a innovative, intelligent, technology using, problem solving and environment shaping species.
And NATing everything is not going to be disruptive and cost a lot of money?
NAT is already here and in widespread use in every small office and multi device household; whereas ipv6 is not. To insist some sort of cost equivalence between the two projects; where option one involves hacking an existing framework to extend the network's reach, vs swapping in a brand new network on a global scale; is just staggeringly irrational. By all means, have your ipv6 if it is so precious to you, but when you break, in probability, the internet during the transition stage, which our civilization has now become heavily coupled to; be prepared to answer your critics.
NAT or no NAT, IPv4 is no longer viable for widespread use.
And so say the religious zealots. In all probability your ipv6 utopia will arrive; and in all probability, the disruption this transition I suspect will cause, will impact the viability and usefulness of the Internet for a number of years and it will become regular topic of discussion amongst general population and media.
Why not? This is how the overwhelming majority of people interface with the internet anyway: content consumption. ipv6; by virtue of the reality of the fact we are not running it yet, appears to be a project failure in terms of it's stated goal to supercede ipv4. We could press ahead with it, or consider alternatives such as NAT.
Most folk I know who need an IP address fall into one of two categories:
The precise problem many folk here have with going to NAT I fail to fully grok. It will not limit how you can use the Internet; but it will modify the way certain types of problems are solved. Big deal; this is network protocol stuff, and working around problems (such as fact that TCP/IP is unicast, or HTTP is client/server send/receive) is par for the course. Maybe some people like their Internet 'pure' or something; me I try to take a more pragmatic approach. ipv6 transition I fear is going to be massively disruptive over a period of at least 2 years and it is going to cost us all alot of money. Maybe Utopia indeed awaits us on the the otherside, but having been promised Utopia many times on many different disruptive technology transition projects; I can't help but feel a little cynical
Thanks for the primer. Actually, I've been working with DB's for over 15 years (mostly RDBMS), and considered by those who have worked with my directly as being quite skillful at modelling large volumes of data and being able to extract it efficiently.
My original question remains, how an object graph database solves this problem faster than a SQL database. Now I know, for a fact, that a non-SQL relational database can solve the specific use case quite efficiently, but the SQL abstraction layer over the top of the relational database removes the capacity to access the specific query plan that is useful for this use case. The OP's point is slightly different to my own, he claims that the SQL statement is too cumbersome, and for people new to SQL, he is probably correct.
Maybe object graph system provides a means to make the query expression simple. I know from experience that OR layers such as JPQL can certainly help in this specific case. But the underlying data fetch, when dealing with large volumes of data (billions of transaction records and millions of customer records) is tediously slow; and performance is my primary interest. SQL is great because it centralises the concern of cost planning, but it's plan vocabulary is limited by the language, and also under some circumstances you do not want a statistics based cost planner at all (but that's a different issue). I think that in order to understand whether an object graph system can actually deliver a better result, one would need to benchmark and one would need to understand how the plan is converted into b-tree scans/walks and tuple fetches to understand it's proper cost.
Thx. I have trialed this solution previously with mixed results depending on volume/cardinality of data.
The OP's point was that the query, and how it is expressed, is complicated. For someone relatively new to SQL, this sort of query is beyond them; which compared to alternative/older methods of working with relational query data I tend to agree. My point was slightly tangental to this. My point was that not only is it complicated, but the plan when it hits the database is inefficient too. The transaction tuples are fetched twice. Once to determine the correct max tuple, and a second time to actually get the tuple. This is a limitation of SQL itself.
Compared to my list of other things I dislike about SQL, this one is fairly trivial. There once was I time were I just got on with doing SQL and never gave it much thought if it was well designed or not. But a few years ago I heard a colleague complain bitterly about it (not about relational databases, but about SQL itself). And it planted a seed in my mind, and I have been finding and cataloguing imperfections ever since. My single biggest problem with SQL is that it is confused about who it's intended audience is. It is not clear if it is targeting programmers, DBAs or end users and it fails to serve any of these audiences particularly well. My next biggest problem with SQL is that relationships are not strongly typed/enforced. I can join anything to anything, I can join a customer_id to account_balance and SQL will not complain. I enjoy working with relational databases and particularly with big data, trying to extract interesting facts from massive data sets and doing it efficiently. But, after 10+ years of doing SQL, I've come to dislike it; but put up with it.
Thanks for the links. Look like interesting products.
I did not mean to come off as rude, but I did. My apologies. In prior post you said "But that is not a problem of SQL!!! It is a problem of relational data bases.". Can you please clarify this/point me towards a clarified. It is not evident to me how a object graph database somehow provides a efficient and meaningful solution to this specific generalised problem and how precisely it does this presumably by avoiding some unstated limitation of RDBMS.
I do not think you are fully groked the example, and why it fails to optimize well via SQL and why SQL itself is the problem.
Query one is select cust_id,max(timestamp) from customers. This in theory can optimize well. But then the output of this needs to be pushed into another query in order to extract actual txn tuples. Which results in a a b-tree root to leaf walk into the transaction table for every customer. Even though the database has already discovered those tuples. It is both redundant and inefficient; and strictly a limitation of SQL
An alternative I suggest and am currently using is to use window functioning; select rank() and filter rank output. Problem with this is you do a full b-tree date-to-date walk when under many circumstances a more efficient traversal of the b-tree, skipping parts of the walk, is theoretically possible.
I am loving this thread. People who appear to know little about the underlying mechanics of how databases perform coming forward and saying 'use magic elixir X!'. OO databases to solve this problem? Are you serious and do you even know what you are talking about? An 'OO' database is just a relational database that includes a notion of inheritance. i.e. PostgreSQL. Though it may be possible that you may be talking about something radically different, in which case how about you point me towards some links on your 'magic elixir X' and detail on how it elegantly solves this specific generalised problem.
The problem again, is SQL itself. SQL is an abstraction of relational indexed queries. the abstraction fails to to provide a meaningful ways to exploit certain possibilities under certain generalised use cases, and these limitations have nothing to do with the underlying technology but the abstraction layer. As I indicated in my original post, the problem is solvable if you have access to the b-trees; but difficult/cumbersome if you are working through SQL.
Of course this is not necessarily a basis for throwing out SQL because it fails to deliver on on generalised case; but it is just one of many issues I have with SQL as a language.
Actually, the example makes perfect sense and it is a problem I've encountered regularly and one of my many bug bears with SQL.
The generalized problem is 'find last txn' for every customer before date X.
Now if you can access the btree directly, you can efficiently walk it and get this data. But with SQL you cannot efficently get it. PostgreSQL windowing functions make getting such data simple in terms of constructing the SQL (once you get your head around windowing functions), but these queries walk large sections of the dataset and are not optimized fully.
The parents actual example is 'find latest txn'. This can be simplified by materializing details of the last transaction to the customer record on update. But this only works for this specific case. The generalised problem is not readily solved in SQL or at least I have not found a nice solution. The best I can come up with is to materialize data for a given date granularity; (i.e. first of every month). It works to a degree, but the solution is ugly and I know better solutions are possible (depending on cardinality of the data) if you are closer to the metal and are able to bypass the SQL layer.
Hear Hear! Nudity in art that is sexually playful or provocative rocks (M and/or F); celebrate this part of our natures. Whiny art depicting weakness or insecurity or fear or whatever is just tediously dull and turgid emo stuff. This notion of naturist de-sexualization seems to me to only seek a duller and less interesting world.
Either you are a troll or you are so embedded in this dispute that you are completely unable to approach this issue with a rational critical mind.
This thread of conversation has nothing to do whatsoever with making apologies for the behaviour of Palistinians or any other group that has inflicted harm upon the Jewish community. What part of "it's really hard to grant either side any sort of moral authority." are you failing to comprehend?
Cult: A group or movement exhibiting a great or excessive devotion or dedication to some person, idea, or thing and employing unethically manipulative techniques of persuasion and control . . . designed to advance the goals of the group's leaders to the actual or possible detriment of members, their families, or the community.
I accept your broad point about the word "cult" being a heavily emotionally loaded word; and as such it is difficult to have a rational discussion of this nature with the "c" word being dropped. But I disagree that the word to be sufficiently powerful/tainted enough to implicate any notion of "hate speech". Personally, I have no emotional reaction to the word; it is a neutral, useful adjective to me and when I use it my intent is to convey a factual, rational description (at least factual from my understanding of whatever issue is under scrutiny). Probably because I have been hanging around the issue of Scientology for years now; on account of my peer group 10 years ago falling into Scientology and the ensuing excitement as I with an open yet skeptical mindset, grasped for a basis of opinion on this particular organization; I am desensitised to the emotional impact of this particular word. In my estimation, the word, "cult" as I define it above is appropriate description of Scientology.
Thx for the links.
I am in the camp that says no hints is the lesser of two evils; yet I am of the view that the SQL language standard is deeply flawed and this whole argument persists because of the flaws in SQL itself. Recognizing and correcting those flaws will close the argument.
SQL implicitly requires a cost based planner, for everything . CBP is great for reporting and analytics, but CBP on primary key OLTP operations keep me awake at night. Some things you just want the software to always use a precise b-tree index in order to get consistent and predictable results. Now sure, the CBP under some rare circumstances will find an even better plan then the intended b-tree, but more often then not, it wets the bed, because of bad histograms of whatever and destroys the application by b-tree walking a utterly inappropriate key or seq scanning. CBP require careful feeding and handling.
Worse, I've have yet to work with a DBA who even remotely understands this. If I had a dollar for every argument I had with a DBA on such issues and every hour I spent benchmarking to demonstrate that the preconceptions of these "professionals" are false, I'd be a few thou up easily.
SQL databases are just too complicated for the average IT professional, let alone the average person. And their proliferation into even desktop software, such as various accounting packages, is a development that will keep our industry on it's toes for some time to come.
Nothing helps drive a wedge between people and their money than a fear incessantly pounded into their brain like a rusty nail.
IPv6 caper should help pay off the mortgage. Then 2038 should set me up quite comfortably for retirement.
I disagree.
Wikileaks reached a huge audience with titillating trivia and entertaining news sound bites. I would assert that the core audience that consumed the 'Batman and Robin' stuff and talked about it on twitter, facebook and around the water cooler paid little heed to the harder stuff, and actually studiously ignored it; the content and it's implications. Wikileaks have certainly increased awareness and engagement in 'hard issues'; yet I am doubtful that their success in doing this radically surpasses any of their notable predecessors.
I personally think it is impossible to do so. Because majority of the population, beyond minority of young passionate activists, just don't engage with such information unless it becomes a real and immediate threat to their personal pursuits for continued prosperity. For me wikileaks disseminating information is nothing new or interesting; their methods are not novel nor is the information they disseminating anything substantially new. The promise of wikileaks that interests me is the possibility of providing support and safe harbour to whistle-blowers. Life of a whistleblower is a hard, risky and lonely life: any institution that can ease this and lower the barriers preventing people from blowing whistles deserves our full support and respect.