I have already designed and partially built a system that will help solve the problem. I just need some cash to finish it. $1 Billion will be more than enough.
Price is a real pesky thing. I would love to drive a top-of-the-line BMW instead of my Camry, but when the price difference is 4x the choice is simple. SSDs are still 10x more per bit than HDD. When you have 10+ TB of data, it gets real expensive to store it all on SSD. But I guess if you are super rich and price is no object....
If you only have a few hundred GB of data or less, then SSD is your best choice now. If you have many TB, it will still be several years before it is cost effective to store it all on SSD instead of HDD.
Or...maybe those talented people decided that they actually wanted to earn some money from their hard work and not just build out their CV or earn recognition. Paying jobs are not all 'soul-crushing monotony'. I like to work for things like food, clothes, vacations, house, car, etc..
I'm kind of surprised that the hard drive industry has not created bigger (i.e. size, not just capacity) drives. It seems that a large portion of hard drives these days are going into huge arrays in data centers. All the data that needs super-quick access times is moving to SSD. The multi-TB near line data is staying with HDD storage. It seems to me that the industry could put out a drive with something like 5 inch platters; 20 platters per drive; a really good motor; redundant heads per platter; and an extra, backup circuit board. They could fill it with Helium and get the cost down to below 1 cent/GB. No one would put one in their desktop, but data centers might really like it.
Kind of like how we 'closed the mail system' or 'closed the telephone and telegraph networks' we we found out that evil people could use those forms of communication to coordinate bad activities?
I wish I had hung on to the original machine too (AT&T 6300). I think it had a 4.77 MHz x86 processor in it. When you did a directory listing using DOS on a directory with a few hundred files in it (about all that would fit on the drive), you could actually read the file names as they scrolled up the screen. I wish I still had it to show my kids what life was like back then for computer programmers.
I just bought a 250 GB SSD for $75 because my 64 GB SSD filled up. I paid something like $100 for the old one about 2 years ago. That is a great improvement and I hope that I will be able to buy a 1TB SSD within a few years for less than $100. But I can buy a 4 TB HDD for less than $100 right now if you catch the right sale. By the time a 1 TB SSD becomes less than $100, you will probably be able to buy 8 TB of HDD space for the same price. On the bright side, decent sized SSDs are now affordable for consumers, but they have a long ways to go before they achieve 'parity' with HDD storage. I look forward to the day when some kind of solid state memory can completely replace the HDD (3D XPoint??), but the death of the hard drive has so far been greatly exaggerated.
I am building an object store where some of my data objects can each be a key-value store that is used as a column in a relational table. Some queries against a table require a full scan (e.g. SELECT * from my_table where address like '%Main Street%';). If I had a table with a billion rows, it can take awhile to scan the whole address column looking for matches (I dedup the values in each column, but there can still be 100 million unique address values in such a table.) The solution is to break each column into multiple segments and let separate threads scan each segment looking for matches. The scan can occur in parallel on multi-core machines and complete in a much quicker manner than forcing a single thread to scan the whole thing. It sounds very similar to what they are trying to do with PostgreSQL (except that database is row based where the whole row is stored together, instead of a columnar database like mine).
Here are two short demo videos of the system in action.
https://www.youtube.com/watch?...https://www.youtube.com/watch?...
I also like software that has at least 50 tabs on the menu bar and the choices are nested 5 levels deep. Life is so much easier when it takes you 50 hours to customize every preference. It's even more fun when they give you at least 3 different ways to set the same option.
...or a woman...or a kid...or a grandma...or black...or gay...or whatever. Personally, I have never looked at code or used a library and thought...this looks like a woman wrote it. Code works or it doesn't. Who cares who wrote it? Some women may face discrimination in the workplace. Some may be hired and/or promoted simply because of their gender. I once had a woman on my team who wanted the company to lay her off for the severance package. She did just about everything she could do...come in late...leave early...play games...etc.. The company instead let other men go at layoff time. They wanted her in their stats.
Still have The C Programming Language book...1982
on
30 Years a Sysadmin
·
· Score: 1
I think it was just a few years after its first printing. I think the Unix Programming Environment book I bought in 1985 was still the first edition.
Did the project fail because of incompetence on HP's part...or did the customer (the government in this case) keep changing the scope and requirements so often that it was impossible to actually do what they wanted? I know nothing about the details of this particular case, but either condition (or both) would not surprise me as the cause for the failure.
I was one of 4 programmers that wrote the first 3 or 4 versions of PartitionMagic. I was the only programmer on the Drive Image team for its first 2 versions. After it started bringing in a few million $ in sales, the company (PowerQuest) decided to add some other programmers to the team to help me:)
Well, not quite...I do have one other guy helping with a GUI admin tool that calls down into my system. But all the guts of my new, general-purpose data management system were written exclusively by me. I have a huge list of features yet to implement, so I could use a lot of help, but until someone steps up and wants to dive in with me, I am on my own. It is an incredibly ambitious product (think file system, relational database, key-value store, graph database, and distributed data management system all rolled into one big data object store that uses multiple data models) so there is nothing trivial about it. I have been in the data management business for 30 years (wrote file system drivers, custom file systems, PartitionMagic and Drive Image, cloud backup, etc.) and I have never seen anything that comes close to my system in terms of speed and flexibility. At this rate, it might still be more than a year before it is ready for use in a production environment but I still code on it every night and weekend. A video of my demo can be found at https://www.youtube.com/watch?...
Although the UI in the demo is running on Windows, it is built with Qt so it is easily ported to other platforms. I plan on getting it running on Linux and OSX as well. I know that need to get some better video of the demo as well. I have just been concentrating on getting features working a lot more than on trying to promote it.
I am working on it now but without funding it is going much slower than I had hoped. About half finished, but works great so far. See the video at
http://youtu.be/2uUvGMUyFhY
Slashdot Mirror
Well, if its at least case sensitive that will slow the hackers down...a few microseconds
Username = Admin, Password = Admin
I bought a 250 GB 850 EVO last year for $75 at Amazon. Unless you will be able to find these $75 MSRP drives on sale for $50....
I have already designed and partially built a system that will help solve the problem. I just need some cash to finish it. $1 Billion will be more than enough.
Price is a real pesky thing. I would love to drive a top-of-the-line BMW instead of my Camry, but when the price difference is 4x the choice is simple. SSDs are still 10x more per bit than HDD. When you have 10+ TB of data, it gets real expensive to store it all on SSD. But I guess if you are super rich and price is no object....
If you only have a few hundred GB of data or less, then SSD is your best choice now. If you have many TB, it will still be several years before it is cost effective to store it all on SSD instead of HDD.
Or...maybe those talented people decided that they actually wanted to earn some money from their hard work and not just build out their CV or earn recognition. Paying jobs are not all 'soul-crushing monotony'. I like to work for things like food, clothes, vacations, house, car, etc..
We will finally get an answer to the age old question: "Does the fridge light go out when you close the door?"
I'm kind of surprised that the hard drive industry has not created bigger (i.e. size, not just capacity) drives. It seems that a large portion of hard drives these days are going into huge arrays in data centers. All the data that needs super-quick access times is moving to SSD. The multi-TB near line data is staying with HDD storage. It seems to me that the industry could put out a drive with something like 5 inch platters; 20 platters per drive; a really good motor; redundant heads per platter; and an extra, backup circuit board. They could fill it with Helium and get the cost down to below 1 cent/GB. No one would put one in their desktop, but data centers might really like it.
Kind of like how we 'closed the mail system' or 'closed the telephone and telegraph networks' we we found out that evil people could use those forms of communication to coordinate bad activities?
I wish I had hung on to the original machine too (AT&T 6300). I think it had a 4.77 MHz x86 processor in it. When you did a directory listing using DOS on a directory with a few hundred files in it (about all that would fit on the drive), you could actually read the file names as they scrolled up the screen. I wish I still had it to show my kids what life was like back then for computer programmers.
I just bought a 250 GB SSD for $75 because my 64 GB SSD filled up. I paid something like $100 for the old one about 2 years ago. That is a great improvement and I hope that I will be able to buy a 1TB SSD within a few years for less than $100. But I can buy a 4 TB HDD for less than $100 right now if you catch the right sale. By the time a 1 TB SSD becomes less than $100, you will probably be able to buy 8 TB of HDD space for the same price. On the bright side, decent sized SSDs are now affordable for consumers, but they have a long ways to go before they achieve 'parity' with HDD storage. I look forward to the day when some kind of solid state memory can completely replace the HDD (3D XPoint??), but the death of the hard drive has so far been greatly exaggerated.
I think you remember it wrong. I also bought my first 10 MB HDD in 1986 and it cost over $500 if I remember right. That works out to over $50 per MB.
I am building an object store where some of my data objects can each be a key-value store that is used as a column in a relational table. Some queries against a table require a full scan (e.g. SELECT * from my_table where address like '%Main Street%';). If I had a table with a billion rows, it can take awhile to scan the whole address column looking for matches (I dedup the values in each column, but there can still be 100 million unique address values in such a table.) The solution is to break each column into multiple segments and let separate threads scan each segment looking for matches. The scan can occur in parallel on multi-core machines and complete in a much quicker manner than forcing a single thread to scan the whole thing. It sounds very similar to what they are trying to do with PostgreSQL (except that database is row based where the whole row is stored together, instead of a columnar database like mine). Here are two short demo videos of the system in action. https://www.youtube.com/watch?... https://www.youtube.com/watch?...
I also like software that has at least 50 tabs on the menu bar and the choices are nested 5 levels deep. Life is so much easier when it takes you 50 hours to customize every preference. It's even more fun when they give you at least 3 different ways to set the same option.
...or a woman...or a kid...or a grandma...or black...or gay...or whatever. Personally, I have never looked at code or used a library and thought...this looks like a woman wrote it. Code works or it doesn't. Who cares who wrote it? Some women may face discrimination in the workplace. Some may be hired and/or promoted simply because of their gender. I once had a woman on my team who wanted the company to lay her off for the severance package. She did just about everything she could do...come in late...leave early...play games...etc.. The company instead let other men go at layoff time. They wanted her in their stats.
I think it was just a few years after its first printing. I think the Unix Programming Environment book I bought in 1985 was still the first edition.
All you need to do is hire 494,304 programmers (41,192 x 12) and you could replace it all in a single month. Right??? See how simple math works.
Did the project fail because of incompetence on HP's part...or did the customer (the government in this case) keep changing the scope and requirements so often that it was impossible to actually do what they wanted? I know nothing about the details of this particular case, but either condition (or both) would not surprise me as the cause for the failure.
Glad you liked them (PM and DI). Some of the best words a programmer can hear is..."I used your software and it really helped me solve a problem."
I was one of 4 programmers that wrote the first 3 or 4 versions of PartitionMagic. I was the only programmer on the Drive Image team for its first 2 versions. After it started bringing in a few million $ in sales, the company (PowerQuest) decided to add some other programmers to the team to help me :)
Well, not quite...I do have one other guy helping with a GUI admin tool that calls down into my system. But all the guts of my new, general-purpose data management system were written exclusively by me. I have a huge list of features yet to implement, so I could use a lot of help, but until someone steps up and wants to dive in with me, I am on my own. It is an incredibly ambitious product (think file system, relational database, key-value store, graph database, and distributed data management system all rolled into one big data object store that uses multiple data models) so there is nothing trivial about it. I have been in the data management business for 30 years (wrote file system drivers, custom file systems, PartitionMagic and Drive Image, cloud backup, etc.) and I have never seen anything that comes close to my system in terms of speed and flexibility. At this rate, it might still be more than a year before it is ready for use in a production environment but I still code on it every night and weekend. A video of my demo can be found at https://www.youtube.com/watch?...
Although the UI in the demo is running on Windows, it is built with Qt so it is easily ported to other platforms. I plan on getting it running on Linux and OSX as well. I know that need to get some better video of the demo as well. I have just been concentrating on getting features working a lot more than on trying to promote it.
I am working on it now but without funding it is going much slower than I had hoped. About half finished, but works great so far. See the video at http://youtu.be/2uUvGMUyFhY
Couldn't agree more. That is why I am building a new kind of object store from the ground up. See it in action at http://youtu.be/2uUvGMUyFhY