Big Data's Invisible Open Source Community

itwbennett writes "Hadoop, Hive, Lucene, and Solr are all open source projects, but if you were expecting the floors of the Strata Conference to be packed with intense, boostrapping hackers you'd be sorely disappointed. Instead, says Brian Proffitt, 'community' where Big Data is concerned is 'acknowledged as a corporate resource', something companies need to contribute back to. 'There is no sense of the grass-roots, hacker-dominated communities that were so much a part of the Linux community's DNA,' says Proffitt."

Comments

So... I read the article...

[bmo]

And I have to ask...

What was the point of the article? That the trade show is like every trade show ever?

Really, I'll write a report the next time I go to EASTEC and whine about the lack of "Makers" (in the geek culture sense of the word) among the vendors of Big Machinery.

--
BMO

Re:Sorry

[Anonymous Coward]

Parent poster nailed it.
Try to get support from "the community" when you discover a bug in a code path that nobody except you encounters. Suddenly the community becomes very small indeed.
There just aren't that many geeks out there who handle petabyte datasets. Prove me wrong, dear reader.

A very simple explanation

[Anonymous Coward]

"There is no sense of the grass-roots, hacker-dominated communities that were so much a part of the Linux community's DNA"

This is for one simple reason: most hackers don't need "BigData".

Perhaps if the typical hacker had a cluster of servers to play with, this would change. But as long as most hackers are bound to using a single personal computer, they're just not going to be very concerned with clusterware.

They're also not concerned with plenty of other things that are essential to big corporations, like payroll software and CRM (customer relationship managment) software.

Re:A very simple explanation

[evilviper]

This is for one simple reason: most hackers don't need "BigData".

Perhaps if the typical hacker had a cluster of servers to play with, this would change.

"Most hackers" don't need a lot of things that are, never-the-less developed as successful open source projects. Anybody think there's a huge audience for DReaM?

Storage is getting big... Even a tiny shop can afford obscene amounts of storage. Each 2U server can have 6 x 2TB SATA (3.5") drives pretty inexpensively. As soon as you've got a dataset that needs more space than you can store on one of those, you'd benefit from thesee "big data" solutions, rather than the standby (more expensive) solution of "throw in a monster SAN".

And you don't even need that much infrastructure. The virtual servers (cloud) service providers aren't very expensive, particularly when you don't care about SLA, and will give you as big of a cluster "to play with" as you could want.