That might be how others have implemented an object store, but not how I am doing it. There are not two separate systems to manage in my case. I don't use a file system to store the unstructured data and store all the metadata in a database and thus have two different systems to try and keep in sync. Instead, I built a new system from the ground up (actually from the block pool or disk partition up). It stores structured data and unstructured data natively in a very unique fashion.
Working on it. And unlike the "storagedude", I do have the expertise to implement it. I have it about half implemented so far. I have created an object store where every object can have lots of attributes or tags attached to them. Unlike extended attributes, you can actually find things based on them quickly. For example, I can create a container and put 100 million of my data objects in it (photos, mp3s, software, documents, etc.) and find anything and/or everything in just a few seconds. If I had 10 million photos and they all had tags attached, I could find the 50,000 photos of my family vacation in Hawaii in 2010 in just a couple of seconds. Check out the video of my demo at http://youtu.be/2uUvGMUyFhY
On the contrary....I have had several people tell me it is something they need. They wish it was finished right now. But for whatever reason, they won't put any resources toward getting it finished. I have put a lot of personal time and money into the project (a couple hundred thousand $) but it is still only about half finished. It is a very ambitious project. I finally had to go back to work to feed my family, but I still work on it in my spare time. I am determined to finish it, but it will be years later than it could have been with some funding.
Good luck with that. Nobody seems to want to pay for system level software anymore. They might shell out a few bucks for a game that they will grow tired of after a few weeks, but they expect their OS, tools, and other platform software to be free (as in beer). You might build a system, library, or algorithm that collectively saves the world economy a $ billion dollars per year in saved time, electricity costs, and/or hardware upgrades; but don't expect to get paid anything for doing it. Sad as that might be, it seems to be the reality we currently live in. For the past few years I have been building a new data management system that blows the doors off a bunch of file systems and databases in terms of performance, but I can't get anyone to fund any of it. They all have the same reaction to the demo of my partially working system..."That is amazing. Let us know when the development is complete and tested in an enterprise environment and we will use if we can have it for free" (as if I had a bunch of magic slave elves coding for me all night in my basement in order to accomplish those things without any resources).
Some people seem to be under the impression that free software is always a better choice than proprietary software. Some of the stuff released as open source software is garbage and there is often little or no incentive for those who wrote it to fix it. There is also a lot of good stuff out there with wide community support as well. I have used a lot of open source AND proprietary software and there is a lot of good and bad stuff in both camps.
It is amazing to me how many people will spend many hours and extra training costs in order to get something working just so they don't have to spend $20 for a license to something else that works a lot better. If I find some really good software and the guys who built it want $50 from me for their efforts, I am happy to pay it. My time is worth something.
I am a highly skilled engineer with over 30 years experience. I once had a headhunter call me and want me to interview for this "wonderful job" at a major company. At first I wasn't interested (I already had a great job) but I finally agreed to an interview out of curiosity.
One day, while running errands in my car I got a call from the HR hiring "screener" at the company who wanted me to do an on-the-spot interview over the phone to see if I was "qualified" enough to merit a real in-person interview. I reluctantly agreed to answer a few questions. He proceeded to ask a bunch of questions about algorithms I haven't thought much about since college (i.e. bubble sorts, tree balancing, etc.). He wanted me to basically quote him coding syntax over the phone as if I were typing on a computer. I'm sure my answers didn't sound the greatest on the other end of the phone.
What a joke! He ended the short conversation probably convinced that I don't know how to code even though I could probably write better C++ code in my sleep than he ever could. I never heard from them again and certainly have a bad taste in my mouth about that company (even though I'm sure there are lots of really smart engineers working there).
Wikipedia is a business. Donations to it should be viewed by the donor as paying (although voluntarily) for service. It's not like giving money to the Red Cross and then finding out some administrator make a million $ per year. The staff can use the donations for anything it chooses. If you don't like how they are spending the money you give, don't give and don't use their product.
This whole open source movement has created the idea that all software should always be free for anybody. Unless you always have an army of software engineers who want to use their talent for no pay (thus decreasing the value of that talent), then you have to pay something to get quality stuff. While you might get something for nothing for awhile, I doesn't work over the long haul.
Take all the drives that have signs of failure, put them in a testing environment where you can read and write them all day but don't care about any of the data on them and see how long it takes for them to really fail. That will give you an indication of how reliable the SMART stats are at predicting real disk failure.
Somehow, I don't think we are talking about old Hollywood movies sold in VHS format here. Probably home movies of weddings, vacations, etc..
The cloud can help you get your stuff archived and accessible from anywhere, but it can be expensive and takes forever to push a ton of data to someone's cloud storage. Even then, it becomes a data management headache..."now where did I put that video of little Billy's 10th birthday party...is it on Dropbox, Apple's iCloud, company XYZ's cloud, or ???"
I think I read somewhere that traffic lights are designed so that it is impossible for both sides to get a simultaneous green light. They have some kind of physical switch that enforces this. In other words, even if the system is hacked, you can't make cars crash by changing all the lights to green. That doesn't mean that a hacker can't cause some problems by making the lights stay red for 10+ minutes or other such mischief.
Free software is....well...free. The people who wrote it don't get paid. That means they generally only want to do the "fun stuff". In my experience as a programmer, all the grunt work (error checking, documentation, well formed error messages, etc.) is generally avoided by coders until the company says "you have to do X, Y, and Z before we can ship this product and customers will pay for it. You do it because you are paid to do it.
FOSS software has no such motivations so all the "not fun" stuff goes largely undone. Some free software is great. A bunch of it is garbage. Without a profit motive, why should anyone be surprised that most of it is half-baked at best.
...And why don't companies spend as much time and money making sure the software is working? It is because no one wants to pay for software anymore. We will shell out big bucks for fast processors, flashy screens, tons of memory and disk space...but we want complicated software for ZERO dollars. Don't get me wrong, I like a lot of open-source software and it can be a good business model, but when there is no money to be made in good software development anymore, why are we surprised when its quality is low on the priority list for companies that make it.
...And if every gas company just happens to coincidentally charge $20 for each gallon of gasoline, then we are all in trouble because we would have to pay it, right? And if every worker just happens to coincidentally demand a base salary of at least $100,000 then small businesses would have to pay it because they would need the workers, right?
No. The real "free market" does not work that way. Someone will decide to sell you gas cheaper, and then everyone will compete on price until the price reflects the true market value. Likewise, enough workers will settle for less until the right price for labor is reached. Businesses who adopt "antiworker policies" will lose them to companies with better policies. (Until the government starts demanding all kinds of regulations that drive out competitive behavior...)
The "small world" nature of modern travel is a double edged sword. Yes, infectious diseases can spread rapidly and can quickly affect people over long distances, but because societies are constantly interacting with other societies, a large segment of the population is able to develop immunities to a large number of pathogens. When Europeans first came to the Americas, large numbers of the native populations were decimated by smallpox and other diseases. Because they had never been exposed to these diseases before and had no immunities built up to defend against it, a whole villiage would be wiped out within a short time. I have heard that far more Native Americans died from diseases this way than were ever killed during wars.
So by your reasoning, if someone steals your car, phone, computer, money...it isn't a loss...because it was someone else's gain??? Even by your calculations, there are real global losses when individuals and companies figure out that their property rights are not protected and thus fail to produce something of value in the first place because the chance of it being stolen are so high.
These things will really take off when you can easily reproduce that $500 part that is broken in your oven, dishwasher, furnace, or AC unit. You know the parts that only cost about $10 to manufacture, but because only the appliance maker can make them, they charge an arm and a leg to replace.
This is exactly the problem that lead me to develop a whole new data management system. It turns files into objects called 'Didgets' (short for Data Widgets) and lets you tag them any way you want. Unlike extended attributes on files, these tags let you find your data fast and easy without something like Spotlight or Windows Search indexing all your metadata into its own database (taking a few hours to do each time). I can import my whole boot volume (about 500,000 files) and can then find anything in a second or less. "Find all JPEG photos with tags Vacation=Hawaii and Year=2011" will give me all my photos with those two tags in less than a second. It can do that if there are 5 photos that match or 50,000. Check out DidgetMaster.blogspot.com for info and video demonstrations.
The mechanism is certainly in place. The problem is that too many inexperienced entrepreneurs don't know how to use it correctly. The guy starting the project has to know about most of the "large quantity" issues up front and plan for them. Too many projects get delayed because they didn't plan for them and now they are stuck dealing with many of the issues enumerated by tlhingan.
At $.90 per GB, SSDs are still about 15 times more expensive than the same amount of hard disk space. Forget about trying to put your 2 TB of data on SSDs. I like the trend of reduced prices for SSDs. They are finally affordable enough to put my most active data on (e.g. boot files, applications), but if you think they will be a viable complete substitute for hard drives anytime soon, think again.
Imagine all the scientists surprise when this thing crashes into the "solar system wall" with lots of stars and galaxies painted on it just like Truman did with his boat in the "Truman Show".
Backwards compatibility and support for legacy software, hardware, and archaic architectures can be a great stumbling block for innovative endeavers. Adoption of new technology requires a huge improvement in order to make the pain of migration bearable. Since the bar is often set so high, we tend to get more incremental improvements rather than revolutionary changes. How can this change?
So, how do we assume that the light we are seeing in the picture followed a straight line? In other words, could one (or more) of the galaxies we see in this picture be ours (the Milky Way). If the light our sun emits can eventually loop back on itself, we could look in any direction and see ourselves if we looked far enough.
Slashdot Mirror
That might be how others have implemented an object store, but not how I am doing it. There are not two separate systems to manage in my case. I don't use a file system to store the unstructured data and store all the metadata in a database and thus have two different systems to try and keep in sync. Instead, I built a new system from the ground up (actually from the block pool or disk partition up). It stores structured data and unstructured data natively in a very unique fashion.
Working on it. And unlike the "storagedude", I do have the expertise to implement it. I have it about half implemented so far. I have created an object store where every object can have lots of attributes or tags attached to them. Unlike extended attributes, you can actually find things based on them quickly. For example, I can create a container and put 100 million of my data objects in it (photos, mp3s, software, documents, etc.) and find anything and/or everything in just a few seconds. If I had 10 million photos and they all had tags attached, I could find the 50,000 photos of my family vacation in Hawaii in 2010 in just a couple of seconds. Check out the video of my demo at http://youtu.be/2uUvGMUyFhY
On the contrary....I have had several people tell me it is something they need. They wish it was finished right now. But for whatever reason, they won't put any resources toward getting it finished. I have put a lot of personal time and money into the project (a couple hundred thousand $) but it is still only about half finished. It is a very ambitious project. I finally had to go back to work to feed my family, but I still work on it in my spare time. I am determined to finish it, but it will be years later than it could have been with some funding.
Good luck with that. Nobody seems to want to pay for system level software anymore. They might shell out a few bucks for a game that they will grow tired of after a few weeks, but they expect their OS, tools, and other platform software to be free (as in beer). You might build a system, library, or algorithm that collectively saves the world economy a $ billion dollars per year in saved time, electricity costs, and/or hardware upgrades; but don't expect to get paid anything for doing it. Sad as that might be, it seems to be the reality we currently live in. For the past few years I have been building a new data management system that blows the doors off a bunch of file systems and databases in terms of performance, but I can't get anyone to fund any of it. They all have the same reaction to the demo of my partially working system..."That is amazing. Let us know when the development is complete and tested in an enterprise environment and we will use if we can have it for free" (as if I had a bunch of magic slave elves coding for me all night in my basement in order to accomplish those things without any resources).
Some people seem to be under the impression that free software is always a better choice than proprietary software. Some of the stuff released as open source software is garbage and there is often little or no incentive for those who wrote it to fix it. There is also a lot of good stuff out there with wide community support as well. I have used a lot of open source AND proprietary software and there is a lot of good and bad stuff in both camps. It is amazing to me how many people will spend many hours and extra training costs in order to get something working just so they don't have to spend $20 for a license to something else that works a lot better. If I find some really good software and the guys who built it want $50 from me for their efforts, I am happy to pay it. My time is worth something.
I am a highly skilled engineer with over 30 years experience. I once had a headhunter call me and want me to interview for this "wonderful job" at a major company. At first I wasn't interested (I already had a great job) but I finally agreed to an interview out of curiosity. One day, while running errands in my car I got a call from the HR hiring "screener" at the company who wanted me to do an on-the-spot interview over the phone to see if I was "qualified" enough to merit a real in-person interview. I reluctantly agreed to answer a few questions. He proceeded to ask a bunch of questions about algorithms I haven't thought much about since college (i.e. bubble sorts, tree balancing, etc.). He wanted me to basically quote him coding syntax over the phone as if I were typing on a computer. I'm sure my answers didn't sound the greatest on the other end of the phone. What a joke! He ended the short conversation probably convinced that I don't know how to code even though I could probably write better C++ code in my sleep than he ever could. I never heard from them again and certainly have a bad taste in my mouth about that company (even though I'm sure there are lots of really smart engineers working there).
Wikipedia is a business. Donations to it should be viewed by the donor as paying (although voluntarily) for service. It's not like giving money to the Red Cross and then finding out some administrator make a million $ per year. The staff can use the donations for anything it chooses. If you don't like how they are spending the money you give, don't give and don't use their product. This whole open source movement has created the idea that all software should always be free for anybody. Unless you always have an army of software engineers who want to use their talent for no pay (thus decreasing the value of that talent), then you have to pay something to get quality stuff. While you might get something for nothing for awhile, I doesn't work over the long haul.
Take all the drives that have signs of failure, put them in a testing environment where you can read and write them all day but don't care about any of the data on them and see how long it takes for them to really fail. That will give you an indication of how reliable the SMART stats are at predicting real disk failure.
Somehow, I don't think we are talking about old Hollywood movies sold in VHS format here. Probably home movies of weddings, vacations, etc.. The cloud can help you get your stuff archived and accessible from anywhere, but it can be expensive and takes forever to push a ton of data to someone's cloud storage. Even then, it becomes a data management headache..."now where did I put that video of little Billy's 10th birthday party...is it on Dropbox, Apple's iCloud, company XYZ's cloud, or ???"
I think I read somewhere that traffic lights are designed so that it is impossible for both sides to get a simultaneous green light. They have some kind of physical switch that enforces this. In other words, even if the system is hacked, you can't make cars crash by changing all the lights to green. That doesn't mean that a hacker can't cause some problems by making the lights stay red for 10+ minutes or other such mischief.
Free software is....well...free. The people who wrote it don't get paid. That means they generally only want to do the "fun stuff". In my experience as a programmer, all the grunt work (error checking, documentation, well formed error messages, etc.) is generally avoided by coders until the company says "you have to do X, Y, and Z before we can ship this product and customers will pay for it. You do it because you are paid to do it. FOSS software has no such motivations so all the "not fun" stuff goes largely undone. Some free software is great. A bunch of it is garbage. Without a profit motive, why should anyone be surprised that most of it is half-baked at best.
...And why don't companies spend as much time and money making sure the software is working? It is because no one wants to pay for software anymore. We will shell out big bucks for fast processors, flashy screens, tons of memory and disk space...but we want complicated software for ZERO dollars. Don't get me wrong, I like a lot of open-source software and it can be a good business model, but when there is no money to be made in good software development anymore, why are we surprised when its quality is low on the priority list for companies that make it.
...And if every gas company just happens to coincidentally charge $20 for each gallon of gasoline, then we are all in trouble because we would have to pay it, right? And if every worker just happens to coincidentally demand a base salary of at least $100,000 then small businesses would have to pay it because they would need the workers, right? No. The real "free market" does not work that way. Someone will decide to sell you gas cheaper, and then everyone will compete on price until the price reflects the true market value. Likewise, enough workers will settle for less until the right price for labor is reached. Businesses who adopt "antiworker policies" will lose them to companies with better policies. (Until the government starts demanding all kinds of regulations that drive out competitive behavior...)
The "small world" nature of modern travel is a double edged sword. Yes, infectious diseases can spread rapidly and can quickly affect people over long distances, but because societies are constantly interacting with other societies, a large segment of the population is able to develop immunities to a large number of pathogens. When Europeans first came to the Americas, large numbers of the native populations were decimated by smallpox and other diseases. Because they had never been exposed to these diseases before and had no immunities built up to defend against it, a whole villiage would be wiped out within a short time. I have heard that far more Native Americans died from diseases this way than were ever killed during wars.
So by your reasoning, if someone steals your car, phone, computer, money...it isn't a loss...because it was someone else's gain??? Even by your calculations, there are real global losses when individuals and companies figure out that their property rights are not protected and thus fail to produce something of value in the first place because the chance of it being stolen are so high.
These things will really take off when you can easily reproduce that $500 part that is broken in your oven, dishwasher, furnace, or AC unit. You know the parts that only cost about $10 to manufacture, but because only the appliance maker can make them, they charge an arm and a leg to replace.
This is exactly the problem that lead me to develop a whole new data management system. It turns files into objects called 'Didgets' (short for Data Widgets) and lets you tag them any way you want. Unlike extended attributes on files, these tags let you find your data fast and easy without something like Spotlight or Windows Search indexing all your metadata into its own database (taking a few hours to do each time). I can import my whole boot volume (about 500,000 files) and can then find anything in a second or less. "Find all JPEG photos with tags Vacation=Hawaii and Year=2011" will give me all my photos with those two tags in less than a second. It can do that if there are 5 photos that match or 50,000. Check out DidgetMaster.blogspot.com for info and video demonstrations.
The mechanism is certainly in place. The problem is that too many inexperienced entrepreneurs don't know how to use it correctly. The guy starting the project has to know about most of the "large quantity" issues up front and plan for them. Too many projects get delayed because they didn't plan for them and now they are stuck dealing with many of the issues enumerated by tlhingan.
At $.90 per GB, SSDs are still about 15 times more expensive than the same amount of hard disk space. Forget about trying to put your 2 TB of data on SSDs. I like the trend of reduced prices for SSDs. They are finally affordable enough to put my most active data on (e.g. boot files, applications), but if you think they will be a viable complete substitute for hard drives anytime soon, think again.
Imagine all the scientists surprise when this thing crashes into the "solar system wall" with lots of stars and galaxies painted on it just like Truman did with his boat in the "Truman Show".
Backwards compatibility and support for legacy software, hardware, and archaic architectures can be a great stumbling block for innovative endeavers. Adoption of new technology requires a huge improvement in order to make the pain of migration bearable. Since the bar is often set so high, we tend to get more incremental improvements rather than revolutionary changes. How can this change?
So, how do we assume that the light we are seeing in the picture followed a straight line? In other words, could one (or more) of the galaxies we see in this picture be ours (the Milky Way). If the light our sun emits can eventually loop back on itself, we could look in any direction and see ourselves if we looked far enough.