Researchers Create Database-Hadoop Hybrid

ericatcw writes "'NoSQL' alternatives such as Hadoop and MapReduce may be uber-cheap and scalable, but they remain slower and clumsier to use than relational databases, say some. Now, researchers at Yale University have created a database-Hadoop hybrid that they say offers the best of both worlds: fast performance and the ability to scale out near-indefinitely. HadoopDB was built using PostGreSQL, though MySQL has also successfully been swapped in, according to Yale computer science professor Daniel Abadi, whose students built this prototype."

Comments

Re:If it works as described it will be VERY import

[Anonymous Coward]

It won't deliver. In the mean time for those of us living and working in the real world, hard-drives will be bigger and faster, file systems will get better, and SSDs will start to shit all over spinning platters.

The SQL language is also an issue

[chrysalis]

Scalability is one thing, but what we appreciate in SQL-free databases is also that they don't require SQL.

When what we want is just to retrieve a record, calling get(id) is way easier and more secure than building an SQL statement, and way cheaper than using an ORM.

The Tokyo Cabinet API is absolutely excellent in this regard. And there's no need to learn yet another domain-specific language like SQL, just use the language you use for the rest of the app.

Now, SQL-zealots would troll "but how would you do with ?".
And yes, for complex requests as in data mining, SQL and XPath make sense. For people who aren't developpers, SQL makes sense as well. For interoperability with 3rd-party apps, SQL is also useful, just as FAT is still useful today in order to share filesystems between operating systems.

But for the rest of us, SQL is cumbersome. Databases like MongoDB make you achieve similar results in a more natural way instead of forcing you to learn SQL and to rethink everything in a tabular way.

Re:The SQL language is also an issue

[BitZtream]

When what we want is just to retrieve a record, calling get(id) is way easier and more secure than building an SQL statement, and way cheaper than using an ORM.

Yes, I'm an SQL troll, but ... if using SQL to get a row by a unique ID is too hard for you and too insecure, there is no amount of code which is going to fix your problem, which is that you are a shitty developer who is far too lazy to make a function or macro to wrap around the simple sql request.

There are PLENTY of reasons to not like SQL, but your example is so simplistic that it just makes me think the problem you have is you, not what the database backend is or how its used.

As far as being forced to think of your data in a tabulare way due to using SQL, you think this is a bad thing. It isn't. Its retarded that you're going to use an SQL backend anyway, and then try to think of the data in some other form. Adding some silly layer on top to make it easier for you to understand just indicates you're too lazy to understand your own data and how to use it efficiently.

I hate when people make stupid posts like yours. I end up sounding like an SQL fanboy and I don't even fucking like it.

Re:The SQL language is also an issue

[ChaosDiscord]

"a record, calling get(id)"

So you're relating "id" to "a record." I assume that the record in question is a blob of potentially binary data that your program parses however it wants. So you want to relate unique identifiers to blobs. You can do that quite easily with SQL. Looking up a given unique identifier quickly is something your average relational database is very good at. And writing the wrapper function to implement your hypothetical get() function is trivial in most languages. I'm completely at a loss for what your SQL-free database is offering me in this case. It's saving you from the horror of writing 10 lines of code, once, to implement get(in)? 60 minutes with a good SQL tutorial will teach you everything you need to know. Sure, there is a lot more you can learn, but for the simple case you're describing you can understand SQL at only the most simple level.

Or are you handwaving the "a record" is actually automatically squeezed into one or more variables or objects in your code? You say get("ChaosDiscord") and out pops the UserObject populated with the relevant information. Of course, at this point you need to start teaching you database, or at least your database wrapper, how your objects are structured, and how to serialize them. This is admittedly a bit of a nuisance, but an SQL-free database doesn't magically make the problem go away. Sure, an SQL-free database can provide a layer to simplify or automate it, but so can a layer on an SQL database (Ruby on Rails is perhaps the best known). Sure, you'll need to tell it that username is a string, userid is an integer, and so on, but you only have to say it once in SQL instead of in your program. The total work hasn't gone up.

Ultimately, you appear to be complaining that SQL is too powerful (and thus complex) for your needs. But you can easily learn and use a subset of SQL that corresponds to what you claim you're looking for in an SQL-free database! You might as well complain that Java is too powerful it has thousands of classes you don't need. The time to learn the relatively minor amount of SQL you need is insignificant compared to the time to develop any non-trivial application. If even that hour is too much, you can outsource the work to a geeky college student for some pizza and soda.

There are some compelling reasons to look at SQL-free databases, but "SQL is too powerful" isn't one.

Re:The SQL language is also an issue

[MikeBabcock]

I understand putting another API above SQL to make it simpler to use, but avoiding using SQL because its powerful makes no sense.

Re:The SQL language is also an issue

[mabinogi]

The answers to those questions will say a whole lot about why PHP sucks, but very little about SQL.

in particular:

Why does a stock PHP have 5 different APIs just to issue basic MySQL queries?

Because the PHP developers have re-invented the wheel five times and still haven't figured out it's not supposed to have sharp corners. Nothing to do with SQL. Perl's DBI is a good example of a database abstraction layer done right.

Re:Please stop

[CorporateSuit]

Considering that "Ubermensch" was translatable to "Superman" then "Ubercheap" would be "Supercheap"

It's called a prefix. We use them in the English language. This one has recently been adopted into our language. Pick up the pace or shut up about things you don't know.

Re:Please stop

[Anonymous Coward]

Uber-cheap is not a word, and it doesn't even make sense because you're saying it's "above cheap".

You remind me of my English teachers. Every year they kept saying that "ain't" isn't a word because it's not in the dictionary. Then one day I looked in the dictionary and it was there. The lesson I learned was that humans create words and "rules" of language aren't really rules at all. They are merely traditions. I suppose you think the French are just speaking bad Latin? No, languages change. From Old English to Middle English to Modern English it changed. I bet all along the way there was some know-it-all jackass pedant saying "thither isn't a word!"

Re:Please stop

[Anonymous Psychopath]

We've been co-opting other language's words into English for a long, long time now. To a growing number US citizens prefixing anything with "uber" is the same as saying "ultra" or "super". You know the saying "it's all over except for the shouting"? Yeah, that's pretty much where this is.

Feel free to mod this entire thread, including the parent, uber off-topic.

Re:Please stop

[moonbender]

Compounds parse easier with correct parantheses: (herz)(kreislauf)(wiederbelebung) or (heart)(circle-run)(re-activation), where each of the bracketed words is itself a common compound. FWIW, Cardiopulmonary resuscitation has more characters than the German term. German and English aren't very different, in fact, in terms of compounds; English also has a huge number of compound words, even though they are often not spelled as a single word: circuit breaker, for instance. As English compounds get increasingly entrenched, the compounds tend to get hyphenated, and eventually they are written as a single word.