The internet is full of bad stuff. Fake news, lies, ads, identity thieves, scam artists. But I worry when someone else gets to decide what is 'BAD' and what is 'GOOD' and try and eliminate or handicap everything they don't personally agree with. There is plenty of 'friction', but it is in my brain where I prefer it to be. I don't fall for everything that gets posted on social media, or even in mainstream news outlets. I am skeptical of almost everything I see today. You are a fool if you believe a significant portion of the BS being pushed on the internet.
...Even in compound highly abstracted layers of code you might see a 5 to 10 ms difference...
I hate to break it to you but 5 to 10 ms is a huge amount of time when you are dealing with big data. Try sorting a 1 billion row table when each row takes a whole millisecond to get in order. That's a million seconds! For the math challenged out there it is nearly 300 hours!
Why will people shell out an extra $300 for a processor that is 10% faster, but they won't pay $10 for a new software program that runs twice as fast as the one they are using?
It can be a huge pain to migrate an existing application from one vendor to another. A company I worked for moved from SQL Server to Postgres (because of licensing issues) and it took 3 years even though they are both 'SQL'. But even if you have a long laundry list of items that make it economically attractive to move from system A to system B; you still must fight the politics of 'not invented here' or 'we have always done it this way'.
I have invented a whole new kind of data management system. It handles unstructured data way better than traditional file systems. It does a bunch of NoSQL functions better than other systems. It is about twice as fast as SQL Server, Postgres, or MySQL at relational databases. But I have a terrible time convincing early adopters to give it a serious look. I had a potential customer who was having a big problem with Cassandra. They had a table with a couple hundred million rows and periodically they had to delete about 10 million rows out of it. (Cassandra is apparently built to ingest data but not to update or delete it very well.) The operation was taking them 2 weeks to complete. We put the same data in my system and it completed in just 17 minutes. Yet their management would not even consider adopting this technology and could not even give one valid reason why they wouldn't.
I am fine with programmers who are willing to contribute to open source projects without any compensation. I am fine with companies like Red Hat that find a way to make money off the generous contributions of others. I am fine with companies paying their employees to work on open source projects that will benefit the company.
What irks me are all the 'open source zealots' out there who insist that anything closed source is some kind of evil thing. If you build something and take great personal risk to get it ready for market, you are often portrayed as some kind of 'greedy capitalist' if you want others who get value from your product to actually pay you something directly for your efforts. You built it. You own it. If you want to charge something for it, then you better make sure it adds more value than the price you are charging for it. Just don't let anyone tell you that you are less than human for not wanting to just give away the fruits of your labors.
There should be a requirement that the lawyers cannot collect any more money from the settlement than the collective payouts to the members of the class. If the courts awards $10M in the suit, then the most the lawyers can collect is $5M. And they can only collect that $5M if they are able to distribute the other $5M out to class members. Right now they have an incentive to have as few class members claim their 'winnings' as possible, since they get whatever is left.
Startups usually fail for one of two reasons. Either they were trying to grow organically and were never able to generate enough revenue to sustain the company; or they got a wad of cash from venture capitalists and they burned through it too fast. The country is full of startups that got $10M+ in cash but burned through it quickly because they were spending like drunken sailors. Executives with fat salaries and expense accounts. Plush offices and tons of perks. Expensive marketing events that try to create 'buzz' before the product is even ready to sell. These kinds of things can sink a company really quickly.
I can totally understand why we don't get nice printed manuals that ship in the box. They are expensive to print and quickly become outdated. It is often easier to read and online version. The real question is why there is rarely an up-to-date online manual to go with the product you buy. I have often bought something that has a few links on a postcard sized manual, but the online manual is the wrong model, has decades old information in it, or is so general that you can't find the answer to nearly any question.
Nearly no one provides a current, useful manual online anymore either!!!
While I welcome the thought of RAM and Flash dropping in price significantly, they have a long way to go to catch hard drives in terms of $/TB. Bits stored on SSDs are still about 10x more expensive. Even if they fall to where they are only twice as expensive, you still would not see a mass migration of all those petabytes of data stored around the world to them. If the price drops, I will likely buy more RAM and SSD space. I would love to put 64 GB of RAM and a 2 TB SSD in my next build without breaking the bank. I still would not completely abandon the good old HDD though.
Yes, some tasks only require a quick solution without a lot of engineering; but complex systems that run frequently need skilled programmers to design and build them.
Back in the day, software had to be as small and efficient as possible because CPUs were slow, memory was limited, and storage took forever. Programmers would sometimes take days to squeeze out a few percentage points of efficiency in a single function. Today, all kinds of horrendous software can run fairly quickly if you put it on a fast enough machine. The cloud can make things even worse. An atrocious algorithm against a large data set can still get done in a less than a minute when it runs on a 1000 server cluster. The fact that it could have run even quicker on a 10 server cluster if the algorithms were changed, is rarely explored...after all it works, right?
I am building a new, general-purpose data management system. One of the best things for its development, was my insistence on testing it on a 10 year old Core 2 duo processor (Q8300) with only 4 GB of RAM and no SSD. It forced me to really optimize a bunch of things so that it was running fast on the old box and is thus lightning fast on the latest hardware. It is an object store that can do file system, database, and NoSQL operations within a single system. The database queries it runs have so far been about twice as fast as PostgreSQL in my benchmarks. It can store 100 million files and find them thousands of times faster than file systems like NTFS or Ext3. I am currently working on ingesting data exported from MongoDB (Json) to see how much faster I can make that run too.
Companies are risk averse. That can be a good thing, but taken to extreme it holds them back. There are lots of little startups out there with innovative solutions to real problems. I am building a new kind of data management system and finding early adopters feels like trying to find life on Mars. Everyone wants someone else to take all the risks. They want someone else to test it; to give feedback on features; to devote resources to make the product better. Yet these same companies complain that the only solutions out there are expensive walled gardens when they have only themselves to blame for fewer options.
If I read this right, if the school fails to teach the student what they need to get an actual job, then they won't get paid. If the school signs up some flunky student who never comes to class because they partied all night long every night, then they won't get paid. If the school admits students who will never make it in the real world because they have no study or work ethics, or are just too dumb to learn the material, then they won't get paid either. Sounds, like they have an actual financial incentive to admit bright, hard-working students and teach them valuable skills.
...student electrocutes self by holding one end of a wet string and dipping the other end into a live electrical socket. Who would have thought? Copper isn't the only conductor of electricity.
Something tells me that if even a modest amount of money had come his way for his efforts, he would have put up with the hassle of keeping the software current. Money has a way of motivating lots of people to do things that they don't necessarily like (because it tends to let us buy things that we do like). It amazes me how many people want all their software for free and then complain when someone like this guy doesn't want to work for free anymore.
Company A builds product X. They design it with security in mind. They code it with security in mind. They test it a dozen ways from Sunday. They are constantly trying to break it and fix it every time they can. They want to charge $20 for using their software to recoup their investment.
Company B builds product Y that directly competes with product X. They give security a passing interest. They throw together the code and ship it as soon as it works in 90% of the cases. They don't bother testing. If a bug is reported, they might fix it in 6 months if they get around to it. They want to charge $1 or give it out for free.
Guess which product almost everyone chooses? Can't say I blame them all that much because it is nearly impossible to tell which product really is more secure when you buy it. Software companies will begin taking security really seriously when customers won't buy it because they didn't. Maybe we need something like Consumer Reports for software.
This is how cloud works. You turn over all your data to some centralized entity so that you can access it conveniently from all your mobile devices. What many do not realize, is that by doing that you have turned over all control over your data as well. The cloud decides who, when, where, and how the data can be accessed. The current IoT architecture (which is completely wrong in my opinion) does the exact same thing. It shovels all the details of your private life to the real owners of the data (the company who sold you the device) and holds it hostage. Hackers now have one target for a treasure trove of information. Subscriptions and other fees can be tacked on at a whim. But most importantly, all your data is now available to the highest bidder. It is time for the pendulum to start swinging the other way back to a decentralized web where not only computing but storage happens at the edge.
...the government was caught leaving copies of books it found in the public library in places where the public could see them! Seriously, this seems like a complete non-story if all the information in the S3 bucket was already public information. They just went out and gathered a bunch of stuff that you or I could already get by simply googling it and stored it in one place. Now if some of the information was not public already, then that is a different story...and would have been highlighted in this one if it really was the case.
Open source is not necessarily more secure than proprietary software. Because it is visible, good programmers can look for bugs and plug security leaks if they want to, but bad guys can also look for vulnerabilities to exploit. Nobody has to look at the code and/or fix anything. In fact, most people have ZERO interest in doing so. Plenty of security flaws have gone either unnoticed or unfixed for an awful long time in open source projects.
...you break them up into 20 pieces and display them in a 'slide show' format where every page has a dozen images and a bunch of advertising. If you have your brain turned off, you only realize after slide number 10 that the whole thing is fake and not leading anywhere at all and quit.
Actually that sounds about right. Open source software does not have to be free (as in beer). If you create something that has value and you want to get paid for it, you can choose to charge people money for using it, whether it is open source or not. Some people have no problem with giving away the fruits of their labor, so open source can be a great way to do that, but that does not mean everyone has to do that.
Because some people give away their software, the Pandora's box has been opened. A very vocal group of people seem to think that everything must be free. Hey, who doesn't like free stuff? But to expect everything to be free and to call people evil who want to be rewarded for their hard work is just plain wrong. If someone builds something and tells you that you must pay something for it, then if you turn around and steal it, then you are a thief. You can try and justify it by saying that it should have been free in the first place; that because it was a digital copy that the owner didn't really lose anything; or that they were just asking too much. But it is still stealing.
...that tries to hide nearly everything it does from the public? FOIA requests are regularly ignored or tied up in the bureaucracy. Backroom deals are done all the time without any oversight. Money pours into campaigns while reporting laws are ignored. Top Secret information and State Department emails are stored on private servers and then wiped clean (and not with a cloth) so no one can see what was in them. Subpoenas are regularly ignored. Yet if they can't see everything that we do, that is somehow 'unreasonable'????
Want to use your dishwasher because it still runs fine after 10 years? Well, that 'Smart Device' might not work any more because the manufacturer wants you to buy their newest model instead or wants you to sign up for some paid subscription service that you don't want or need. They just won't send some 'certificate' that allows the dishwasher to keep working past its 'due date'. Same for your fridge, your TV, your security camera, your stove, your sprinkling system, your....
Just like I am for Internet connectivity, I am willing to pay about $40 - $50 per month for TV. But it has to be the kind of TV that I want. Not 100 channels that I would not watch in a million years. Not two channels I like with the rest a bunch of infomercials or reality tv. Since my satellite service wanted over $120/month (didn't include any of the 'premium' channels either), I just cut the cable and got Netflix instead. Now they get ZERO from me for TV.
You don't see us in these cubicle farms working 80+ hours per week for peanuts. We have enough assets (and debts paid off) that we can work in a much less stressful environment. We can say goodbye to the 8 to 5 jobs doing grunt work too. If the project isn't interesting, we do consulting or some other side gig. If we want to take 6 months off, we can because we are not living paycheck to paycheck. Or we start our own software business (me) and don't stress out if it hasn't gone ballistic in the first year.
Slashdot Mirror
The internet is full of bad stuff. Fake news, lies, ads, identity thieves, scam artists. But I worry when someone else gets to decide what is 'BAD' and what is 'GOOD' and try and eliminate or handicap everything they don't personally agree with. There is plenty of 'friction', but it is in my brain where I prefer it to be. I don't fall for everything that gets posted on social media, or even in mainstream news outlets. I am skeptical of almost everything I see today. You are a fool if you believe a significant portion of the BS being pushed on the internet.
...Even in compound highly abstracted layers of code you might see a 5 to 10 ms difference ...
I hate to break it to you but 5 to 10 ms is a huge amount of time when you are dealing with big data. Try sorting a 1 billion row table when each row takes a whole millisecond to get in order. That's a million seconds! For the math challenged out there it is nearly 300 hours!
As a matter of fact, I did. But what does that have to do with my question?
Why will people shell out an extra $300 for a processor that is 10% faster, but they won't pay $10 for a new software program that runs twice as fast as the one they are using?
It can be a huge pain to migrate an existing application from one vendor to another. A company I worked for moved from SQL Server to Postgres (because of licensing issues) and it took 3 years even though they are both 'SQL'. But even if you have a long laundry list of items that make it economically attractive to move from system A to system B; you still must fight the politics of 'not invented here' or 'we have always done it this way'.
I have invented a whole new kind of data management system. It handles unstructured data way better than traditional file systems. It does a bunch of NoSQL functions better than other systems. It is about twice as fast as SQL Server, Postgres, or MySQL at relational databases. But I have a terrible time convincing early adopters to give it a serious look. I had a potential customer who was having a big problem with Cassandra. They had a table with a couple hundred million rows and periodically they had to delete about 10 million rows out of it. (Cassandra is apparently built to ingest data but not to update or delete it very well.) The operation was taking them 2 weeks to complete. We put the same data in my system and it completed in just 17 minutes. Yet their management would not even consider adopting this technology and could not even give one valid reason why they wouldn't.
I am fine with programmers who are willing to contribute to open source projects without any compensation. I am fine with companies like Red Hat that find a way to make money off the generous contributions of others. I am fine with companies paying their employees to work on open source projects that will benefit the company.
What irks me are all the 'open source zealots' out there who insist that anything closed source is some kind of evil thing. If you build something and take great personal risk to get it ready for market, you are often portrayed as some kind of 'greedy capitalist' if you want others who get value from your product to actually pay you something directly for your efforts. You built it. You own it. If you want to charge something for it, then you better make sure it adds more value than the price you are charging for it. Just don't let anyone tell you that you are less than human for not wanting to just give away the fruits of your labors.
There should be a requirement that the lawyers cannot collect any more money from the settlement than the collective payouts to the members of the class. If the courts awards $10M in the suit, then the most the lawyers can collect is $5M. And they can only collect that $5M if they are able to distribute the other $5M out to class members. Right now they have an incentive to have as few class members claim their 'winnings' as possible, since they get whatever is left.
Startups usually fail for one of two reasons. Either they were trying to grow organically and were never able to generate enough revenue to sustain the company; or they got a wad of cash from venture capitalists and they burned through it too fast. The country is full of startups that got $10M+ in cash but burned through it quickly because they were spending like drunken sailors. Executives with fat salaries and expense accounts. Plush offices and tons of perks. Expensive marketing events that try to create 'buzz' before the product is even ready to sell. These kinds of things can sink a company really quickly.
I can totally understand why we don't get nice printed manuals that ship in the box. They are expensive to print and quickly become outdated. It is often easier to read and online version. The real question is why there is rarely an up-to-date online manual to go with the product you buy. I have often bought something that has a few links on a postcard sized manual, but the online manual is the wrong model, has decades old information in it, or is so general that you can't find the answer to nearly any question. Nearly no one provides a current, useful manual online anymore either!!!
While I welcome the thought of RAM and Flash dropping in price significantly, they have a long way to go to catch hard drives in terms of $/TB. Bits stored on SSDs are still about 10x more expensive. Even if they fall to where they are only twice as expensive, you still would not see a mass migration of all those petabytes of data stored around the world to them. If the price drops, I will likely buy more RAM and SSD space. I would love to put 64 GB of RAM and a 2 TB SSD in my next build without breaking the bank. I still would not completely abandon the good old HDD though.
Yes, some tasks only require a quick solution without a lot of engineering; but complex systems that run frequently need skilled programmers to design and build them.
Back in the day, software had to be as small and efficient as possible because CPUs were slow, memory was limited, and storage took forever. Programmers would sometimes take days to squeeze out a few percentage points of efficiency in a single function. Today, all kinds of horrendous software can run fairly quickly if you put it on a fast enough machine. The cloud can make things even worse. An atrocious algorithm against a large data set can still get done in a less than a minute when it runs on a 1000 server cluster. The fact that it could have run even quicker on a 10 server cluster if the algorithms were changed, is rarely explored...after all it works, right?
I am building a new, general-purpose data management system. One of the best things for its development, was my insistence on testing it on a 10 year old Core 2 duo processor (Q8300) with only 4 GB of RAM and no SSD. It forced me to really optimize a bunch of things so that it was running fast on the old box and is thus lightning fast on the latest hardware. It is an object store that can do file system, database, and NoSQL operations within a single system. The database queries it runs have so far been about twice as fast as PostgreSQL in my benchmarks. It can store 100 million files and find them thousands of times faster than file systems like NTFS or Ext3. I am currently working on ingesting data exported from MongoDB (Json) to see how much faster I can make that run too.
Companies are risk averse. That can be a good thing, but taken to extreme it holds them back. There are lots of little startups out there with innovative solutions to real problems. I am building a new kind of data management system and finding early adopters feels like trying to find life on Mars. Everyone wants someone else to take all the risks. They want someone else to test it; to give feedback on features; to devote resources to make the product better. Yet these same companies complain that the only solutions out there are expensive walled gardens when they have only themselves to blame for fewer options.
If I read this right, if the school fails to teach the student what they need to get an actual job, then they won't get paid. If the school signs up some flunky student who never comes to class because they partied all night long every night, then they won't get paid. If the school admits students who will never make it in the real world because they have no study or work ethics, or are just too dumb to learn the material, then they won't get paid either. Sounds, like they have an actual financial incentive to admit bright, hard-working students and teach them valuable skills.
...student electrocutes self by holding one end of a wet string and dipping the other end into a live electrical socket. Who would have thought? Copper isn't the only conductor of electricity.
Something tells me that if even a modest amount of money had come his way for his efforts, he would have put up with the hassle of keeping the software current. Money has a way of motivating lots of people to do things that they don't necessarily like (because it tends to let us buy things that we do like). It amazes me how many people want all their software for free and then complain when someone like this guy doesn't want to work for free anymore.
Company A builds product X. They design it with security in mind. They code it with security in mind. They test it a dozen ways from Sunday. They are constantly trying to break it and fix it every time they can. They want to charge $20 for using their software to recoup their investment.
Company B builds product Y that directly competes with product X. They give security a passing interest. They throw together the code and ship it as soon as it works in 90% of the cases. They don't bother testing. If a bug is reported, they might fix it in 6 months if they get around to it. They want to charge $1 or give it out for free.
Guess which product almost everyone chooses? Can't say I blame them all that much because it is nearly impossible to tell which product really is more secure when you buy it. Software companies will begin taking security really seriously when customers won't buy it because they didn't. Maybe we need something like Consumer Reports for software.
This is how cloud works. You turn over all your data to some centralized entity so that you can access it conveniently from all your mobile devices. What many do not realize, is that by doing that you have turned over all control over your data as well. The cloud decides who, when, where, and how the data can be accessed. The current IoT architecture (which is completely wrong in my opinion) does the exact same thing. It shovels all the details of your private life to the real owners of the data (the company who sold you the device) and holds it hostage. Hackers now have one target for a treasure trove of information. Subscriptions and other fees can be tacked on at a whim. But most importantly, all your data is now available to the highest bidder. It is time for the pendulum to start swinging the other way back to a decentralized web where not only computing but storage happens at the edge.
...the government was caught leaving copies of books it found in the public library in places where the public could see them! Seriously, this seems like a complete non-story if all the information in the S3 bucket was already public information. They just went out and gathered a bunch of stuff that you or I could already get by simply googling it and stored it in one place. Now if some of the information was not public already, then that is a different story...and would have been highlighted in this one if it really was the case.
Open source is not necessarily more secure than proprietary software. Because it is visible, good programmers can look for bugs and plug security leaks if they want to, but bad guys can also look for vulnerabilities to exploit. Nobody has to look at the code and/or fix anything. In fact, most people have ZERO interest in doing so. Plenty of security flaws have gone either unnoticed or unfixed for an awful long time in open source projects.
...you break them up into 20 pieces and display them in a 'slide show' format where every page has a dozen images and a bunch of advertising. If you have your brain turned off, you only realize after slide number 10 that the whole thing is fake and not leading anywhere at all and quit.
Actually that sounds about right. Open source software does not have to be free (as in beer). If you create something that has value and you want to get paid for it, you can choose to charge people money for using it, whether it is open source or not. Some people have no problem with giving away the fruits of their labor, so open source can be a great way to do that, but that does not mean everyone has to do that.
Because some people give away their software, the Pandora's box has been opened. A very vocal group of people seem to think that everything must be free. Hey, who doesn't like free stuff? But to expect everything to be free and to call people evil who want to be rewarded for their hard work is just plain wrong. If someone builds something and tells you that you must pay something for it, then if you turn around and steal it, then you are a thief. You can try and justify it by saying that it should have been free in the first place; that because it was a digital copy that the owner didn't really lose anything; or that they were just asking too much. But it is still stealing.
...that tries to hide nearly everything it does from the public? FOIA requests are regularly ignored or tied up in the bureaucracy. Backroom deals are done all the time without any oversight. Money pours into campaigns while reporting laws are ignored. Top Secret information and State Department emails are stored on private servers and then wiped clean (and not with a cloth) so no one can see what was in them. Subpoenas are regularly ignored. Yet if they can't see everything that we do, that is somehow 'unreasonable'????
Want to use your dishwasher because it still runs fine after 10 years? Well, that 'Smart Device' might not work any more because the manufacturer wants you to buy their newest model instead or wants you to sign up for some paid subscription service that you don't want or need. They just won't send some 'certificate' that allows the dishwasher to keep working past its 'due date'. Same for your fridge, your TV, your security camera, your stove, your sprinkling system, your ....
Just like I am for Internet connectivity, I am willing to pay about $40 - $50 per month for TV. But it has to be the kind of TV that I want. Not 100 channels that I would not watch in a million years. Not two channels I like with the rest a bunch of infomercials or reality tv. Since my satellite service wanted over $120/month (didn't include any of the 'premium' channels either), I just cut the cable and got Netflix instead. Now they get ZERO from me for TV.
You don't see us in these cubicle farms working 80+ hours per week for peanuts. We have enough assets (and debts paid off) that we can work in a much less stressful environment. We can say goodbye to the 8 to 5 jobs doing grunt work too. If the project isn't interesting, we do consulting or some other side gig. If we want to take 6 months off, we can because we are not living paycheck to paycheck. Or we start our own software business (me) and don't stress out if it hasn't gone ballistic in the first year.