I agree. I was surfing the BBSs and Usenet back in the 80's and I remember well the flamewars to which you refer.
What needs to be drawn from this ruling is that you have to exercise discretion when you are dealing with online postings. You have to acknowledge that there is a tremendous gray area. And you can't just say anything that even remotely looks like a threat is criminal. You can't treat the world as black-or-white. You have to use your brain and evaluate each case rationally. It's more work and it's difficult but that's what it takes to ensure liberty.
It's often times difficult to differentiate between rantings of someone blowing off steam and real threats. But taking a zero tolerance approach is not the right way to do it. One needs to engage their brain when evaluating what really is a threat and what isn't. I know it's difficult for people to do but it's the only way to keep from stomping on the liberty of the people.
Not really. There's no good reason to have a log file filling up space in a file system on flight software. The data should be stored in a dedicated space and dumped when it is transmitted, ready for the next round. There's no need to store it locally after that at all. Flight software should be developed so that you don't need any systems administration at all. Embedded software 101 for systems that can't be easily serviced.
A lot of this type of process is SOP for pretty much any safety-critical real-time embedded software. The companies that design flight software for airplanes around the world follow a process that is not entirely unlike NASA's process. Because lives are at stake there too. Similar standards are applied to medical equipment as well, for much the same reason.
Working in an environment like this is tedious as hell but the software that you produce rarely has any problems. When it does, it's usually some obscure corner case that's virtually impossible to test for.
Except HPUX, AIX, and a handful of other Unixes and Linux flavors, so you'd better know how to recognize and translate betwixt them and modern storage arrays that don't, else your 40TB Oracle DB will run into read/write errors at around 36TB, and you'd damned well better have a hot failover plan, or you'll be updating your resume.
If you run an enterprise database and you don't have a hot failover plan regardless of the actual size you think it is able to handle, you should consider a voluntary career path adjustment before it becomes involuntary.
The problem is that people get away with it all the time because their server has terabytes of room for a log file that gets automatically backed up and zipped for them. So they never think to check before they blindly spew out data.
People do this all the time. But I won't call them dumb for doing it. Because there's no reason to even consider doing it another way when you consider the environment they develop their applications in. Being very pedantic about checking every single thing you do in your code is tedious and time consuming. And for a regular application, it's a bit of a waste of time.
But that's why even very good software developers can make mistakes like this when they jump from a "regular" environment to one where resources are far more limited and the consequences of failure are much more significant. Things they had no need to worry about suddenly become very important but how are they supposed to know that until it trips them up?
But that answers the question "why does it cost so much to develop software for satellites?". Because you have to spend a lot more time developing error handling and correction. It also answers the question "can we do it cheaper and faster?", to which the answer is "not if you want to do it right".
So sorry; just because you are an "engineer", doesn't automagically make you a better Developer. Stupid is as Stupid Does.
I agree 100%. I've worked with some pretty dumb engineers.
But that's not really the point I was making. There's a difference between writing an app for a phone or a desktop and writing software for a safety critical embedded systems. The whole approach you take towards developing the software is different. You can't just throw an exception and have the user restart the program. There is no user to restart the system. Beyond that, you're usually operating in a constrained resource environment. You can't just allocate resources on a whim. You have to plan your resource use. You also have to consider determinism and latency. How long does that vector push_back() call take to complete? It depends on whether it's currently full and needs to expand it's space, including all the time to copy over all the existing data. And that assumes there's space to expand into. It wouldn't be a good idea for the flight controls to freeze while you wait for your push_back() to complete now would it? These are the kinds of things you don't have to worry about in a desktop application.
It's not hard to train a smart person to be a good embedded developer. Many competent developers learn how every year. All I'm saying is that just being a smart person and a good developer isn't enough. You need to know what to look out for before you will be good at developing satellite software, regardless of how smart you are and how good you are at "coding". If you don't know what to look out for, you're going to run into problems like this the hard way, as this project did. Had they been a little more rigorous in their development with more experienced people who knew better what to look out for, it's likely they would have caught this problem before it bit them in the ass.
It's the old adage. "Good judgment comes from experience. Experience comes from bad judgment." Having more people with good judgment would have helped them enormously.
Too bad Canonical can't let go of it. But there will be purists who try to hold onto it at all costs, firmly entrenched in the idea that they are right and everyone else is wrong, even if no one else really cares.
The CSV file does not get culled at a specific point before failure. The application keeps blindly appending to it regardless of size. It would continue to grow until the fail point. It doesn't make one god damn bit of difference if the fail point is at 32MB or 32MiB.
Is that such a hard idea for you to understand?
Let me put it another way. Try driving your car at full speed towards a wall without stopping. Does it matter if the wall is 10 yards away or 10 meters away? I guarantee that when you're on the news, it won't make any difference.
If experienced engineers make these mistakes, it's unreasonable to expect that grabbing an average coder off the street will be able to foresee the problems. I've been doing this kind of work for decades and I have yet to find someone who doesn't have to be trained thoroughly to do it the right way. And even after that, we all review each other's designs before we start coding specifically because the trained engineers still make mistakes.
You're crazy if you think "coders" would not make these kinds of mistakes. It's a problem set they have never been exposed to and they have no idea that they even need to be looking for those problems in the first place. Hell, the problems are hard to find even when you know that you need to be looking for them in the first place.
This is rocket science we're talking about. It's hard.
The article is about overflowing a file system. It doesn't matter if it's 32MB or 32MiB. The file system would still have overflowed in the same way. The nit picky detail about exactly how big it was is immaterial to the entire discussion. The failure point is that the system was designed in such a way that the file was able to fill up the file system (regardless of how big it was) which resulted in bricking the whole system. Now can we stop wasting time debating a retardedly pedantic point that only autistic people care about but has nothing to do with the actual failure?
The point is that the original article was written for the general public. And while most good engineers understand the difference between 32 MB and 32 MiB, the distinction is lost on the general public. Hell, for the most part, most computer savvy people don't care about the distinction in general. Only the most anal-retentive, pedantic types get their knickers in a twist over it.
Sure, if I am designing something I want accurate specifications so that it removes ambiguity when I have write the code. But outside of that, I will use MB and MiB interchangeably, knowing full well that they are not exactly the same. But when you're having general discussions, you're rarely doing anything more than discussing an order of magnitude so you don't have to be precise all the time.
The few purists I have run into who piss and moan about the difference between MB and MiB in general use are some of the most annoying types of people I've ever run into. They're worse than any SJW trying to push their agenda down your throat. They're not just overly pedantic. They're passive aggressive about it. And that's what really makes people dislike them.
Bottom line, MB and MiB are interchangeable in the modern vernacular. Unless you're talking about the details of an engineering specification, it just doesn't matter. Please develop those interpersonal skills and suppress the urges to "correct" people. It will go a long way to improving your ability to win friends and influence people.
Speaking as an engineer working on software that is on the Orion spacecraft, I can say that rigorous testing is budgeted into the project from the beginning because it helps to avoid most of the problems like this. The testing that goes on with flight software is orders of magnitude more than you find for a traditional commercial product. You have to. The consequences of failure are, obviously, a lot more significant.
That being said, it's impossible to catch every single possible bug, especially as systems get more and more complex. But there are strategies that help reduce your risk. For example, you don't just run off to kernel.org and throw the latest stable release on a board. You pick operating systems that are maybe a bit harder to use (i.e. limited in what they can do) but are far better suited to real-time embedded work. And you certainly don't blindly append to a file without verifying that you're not going to overflow your space. And you always have an automated recovery plan for any dynamically allocated space in the event of an overflow.
This kind of failure is caused by amateurs making amateur mistakes. It was caused by application programmers who don't understand the consequence of failure in a constrained environment where you can't just click a mouse to restart the program. It was caused by poor planning and a lack of understanding of the environment in which they were designing. This was caused by hiring coders instead of experienced engineers. It was caused by trying to do it cheap rather than spending the money to do it right. They got what they paid for.
It's not either/or because wiring two complete sets of circuits in your house costs a lot of money and doesn't really give you any real net benefit. It's far cheaper and easier to wire with one standards and use plug-in adapters where you need them. It's even cheaper and easier to setup your whole house battery backup to power only a subset of your outlets so you don't have the big devices drain your batteries.
I wish we could get three phase power in a residential area. I could get a welder that was easier to fine tune and better machine tools for metal working.
It is the native voltage when you're dumb enough to write your article based on your use of 12V car batteries in your Airstream trailer with 12V solar cells to charge them. Further proof that the guy who wrote the original article has no idea what he's talking about and the article is complete bunk.
But this pedantic bickering about the specific voltage is all a side distraction over semantics and doesn't really have anything to do with the point I'm trying to make.
It doesn't matter what the voltage really is in the end. It could be any array of batteries providing any level of "low voltage" power. Using battery packs in your home along with solar in an effort to reduce your consumption from the grid is not going to drive anyone to do large scale replacement of 120VAC (or 240VAC where it is used outside the US) in the home. We as a society are not going to rewire our homes and replace all of our electrical devices to make the switch. There's too much inertia in the system to make a change that nets us no real gain. Using batteries directly in the house just isn't going to happen. They're going to have to go through an inverter to supply the needs of the average house because that's cheaper and easier than reconfiguring your house to use DC at any voltage.
Did you actually RTFA? The author specifically talked about 12V DC power being the "low voltage" standard that we need to convert to in the home because it's the native battery voltage.
The article was not about adopting industrial data center voltage standards. It's about using voltages that match the batteries that you are using so you don't have to convert to anything else.
I'm not sure that home batteries will drive a switch to low voltage DC. There's a ton of inertia to overcome. The cost of retrofitting the wiring to handle the higher amperage of using lower voltage alone will be thousands of dollars for every single house, apartment, and office. A simple 20A 120V circuit changed over to 12V will draw 200A. You're going to need to upgrade to 4 or even 2 gauge wire at a minimum to handle that kind of current. And that's a lot of money.
The switch from AC to DC inside the home might be feasible but there's no way you can convert the entire grid. You'd have to rebuild the whole grid from scratch to convert from AC to DC. The transformers to step the voltages up or down simply don't work unless they're pushing AC so how do you handle industrial level supply being stepped down to household voltages at the neighborhood transformer? And who's going to pay for the switch? And what about the industrial users who don't need to run low voltage DC? How do you satisfy their demand?
Then you have to deal with how a substantial number of appliances are built. Many are designed for AC current and won't work with DC, regardless of the voltages. Sure, you can swap out the power supply in your desktop PC to take a DC feed without a lot of trouble. And if electronics retailers had a standard DC wall voltage to work with, you'd see most consumer electronics move to those standards. But how do you deal with a cable modem that needs 12V and a home router that takes 9V? Who wants to go out and replace all of their equipment that is running just fine right now? Who has the money to do that?
And here's the kicker. What real benefit do we gain from a switch over to low voltage DC in the house? Sure, some of the consumer electronics we use won't need that big wall vampire to supply them with power. And sure, we don't really need to run our lights from 120V when 12V can still drive enough light from LEDs without any trouble. But what about the appliances in the house that really draw the bulk of the power in the house? The 240V electric stove or the heat and AC systems? What about your refrigerator and your washer/dryer? Hell, can you imagine the amperage draw trying to recharge your electric car with 12V? And are you going to just skip using those appliances when you're running on battery power?
So if you're going to have to keep your 120V AC based house wiring for your major appliances, do you really want to spend all the money installing a low voltage subsystem for a few consumer electronic devices to supplement the wiring you already have? I know I wouldn't want to.
Like everything else that is poised to "fundamentally change the way we do things", the dreamers never consider the practical reality of actually making the change. In reality, I think we're going to have to deal with the inefficiency of converting from DC battery power to 120V AC for the home. There's just too many things to overcome for little to no benefit.
Yep. Tons of details to keep track of. Lots of opportunity to miss something and have a mishap. A computer that is programmed right is a much better tool for keeping track of everything all at once. And the way it's being designed, it just sits there silently until the operator starts doing something that can cause a problem. It gives the operator a reminder while (if) there's still time before taking over and stopping the train. But it also provides useful information on the screen about what's coming up on the route, much like a GPS in your car, so it unobtrusively helps remind the operator of things they might be forgetting.
Once they get it installed and work out a good deal of the bugs, it will be a pretty good system.
Slashdot Mirror
Why do you automatically assume it's a threat? Spewing vitriol on the internet is not always a threat.
Why bother? They won't listen to "civilians".
I agree. I was surfing the BBSs and Usenet back in the 80's and I remember well the flamewars to which you refer.
What needs to be drawn from this ruling is that you have to exercise discretion when you are dealing with online postings. You have to acknowledge that there is a tremendous gray area. And you can't just say anything that even remotely looks like a threat is criminal. You can't treat the world as black-or-white. You have to use your brain and evaluate each case rationally. It's more work and it's difficult but that's what it takes to ensure liberty.
I quote a lot of song lyrics when I'm in a "less than pleasant" mood. Doesn't mean I'm actually going to carry out what I post on the internet.
https://www.youtube.com/watch?...
It's often times difficult to differentiate between rantings of someone blowing off steam and real threats. But taking a zero tolerance approach is not the right way to do it. One needs to engage their brain when evaluating what really is a threat and what isn't. I know it's difficult for people to do but it's the only way to keep from stomping on the liberty of the people.
Not really. There's no good reason to have a log file filling up space in a file system on flight software. The data should be stored in a dedicated space and dumped when it is transmitted, ready for the next round. There's no need to store it locally after that at all. Flight software should be developed so that you don't need any systems administration at all. Embedded software 101 for systems that can't be easily serviced.
A lot of this type of process is SOP for pretty much any safety-critical real-time embedded software. The companies that design flight software for airplanes around the world follow a process that is not entirely unlike NASA's process. Because lives are at stake there too. Similar standards are applied to medical equipment as well, for much the same reason.
Working in an environment like this is tedious as hell but the software that you produce rarely has any problems. When it does, it's usually some obscure corner case that's virtually impossible to test for.
Except HPUX, AIX, and a handful of other Unixes and Linux flavors, so you'd better know how to recognize and translate betwixt them and modern storage arrays that don't, else your 40TB Oracle DB will run into read/write errors at around 36TB, and you'd damned well better have a hot failover plan, or you'll be updating your resume.
If you run an enterprise database and you don't have a hot failover plan regardless of the actual size you think it is able to handle, you should consider a voluntary career path adjustment before it becomes involuntary.
The problem is that people get away with it all the time because their server has terabytes of room for a log file that gets automatically backed up and zipped for them. So they never think to check before they blindly spew out data.
People do this all the time. But I won't call them dumb for doing it. Because there's no reason to even consider doing it another way when you consider the environment they develop their applications in. Being very pedantic about checking every single thing you do in your code is tedious and time consuming. And for a regular application, it's a bit of a waste of time.
But that's why even very good software developers can make mistakes like this when they jump from a "regular" environment to one where resources are far more limited and the consequences of failure are much more significant. Things they had no need to worry about suddenly become very important but how are they supposed to know that until it trips them up?
But that answers the question "why does it cost so much to develop software for satellites?". Because you have to spend a lot more time developing error handling and correction. It also answers the question "can we do it cheaper and faster?", to which the answer is "not if you want to do it right".
So sorry; just because you are an "engineer", doesn't automagically make you a better Developer. Stupid is as Stupid Does.
I agree 100%. I've worked with some pretty dumb engineers.
But that's not really the point I was making. There's a difference between writing an app for a phone or a desktop and writing software for a safety critical embedded systems. The whole approach you take towards developing the software is different. You can't just throw an exception and have the user restart the program. There is no user to restart the system. Beyond that, you're usually operating in a constrained resource environment. You can't just allocate resources on a whim. You have to plan your resource use. You also have to consider determinism and latency. How long does that vector push_back() call take to complete? It depends on whether it's currently full and needs to expand it's space, including all the time to copy over all the existing data. And that assumes there's space to expand into. It wouldn't be a good idea for the flight controls to freeze while you wait for your push_back() to complete now would it? These are the kinds of things you don't have to worry about in a desktop application.
It's not hard to train a smart person to be a good embedded developer. Many competent developers learn how every year. All I'm saying is that just being a smart person and a good developer isn't enough. You need to know what to look out for before you will be good at developing satellite software, regardless of how smart you are and how good you are at "coding". If you don't know what to look out for, you're going to run into problems like this the hard way, as this project did. Had they been a little more rigorous in their development with more experienced people who knew better what to look out for, it's likely they would have caught this problem before it bit them in the ass.
It's the old adage. "Good judgment comes from experience. Experience comes from bad judgment." Having more people with good judgment would have helped them enormously.
Too bad Canonical can't let go of it. But there will be purists who try to hold onto it at all costs, firmly entrenched in the idea that they are right and everyone else is wrong, even if no one else really cares.
The CSV file does not get culled at a specific point before failure. The application keeps blindly appending to it regardless of size. It would continue to grow until the fail point. It doesn't make one god damn bit of difference if the fail point is at 32MB or 32MiB.
Is that such a hard idea for you to understand?
Let me put it another way. Try driving your car at full speed towards a wall without stopping. Does it matter if the wall is 10 yards away or 10 meters away? I guarantee that when you're on the news, it won't make any difference.
If experienced engineers make these mistakes, it's unreasonable to expect that grabbing an average coder off the street will be able to foresee the problems. I've been doing this kind of work for decades and I have yet to find someone who doesn't have to be trained thoroughly to do it the right way. And even after that, we all review each other's designs before we start coding specifically because the trained engineers still make mistakes.
You're crazy if you think "coders" would not make these kinds of mistakes. It's a problem set they have never been exposed to and they have no idea that they even need to be looking for those problems in the first place. Hell, the problems are hard to find even when you know that you need to be looking for them in the first place.
This is rocket science we're talking about. It's hard.
The article is about overflowing a file system. It doesn't matter if it's 32MB or 32MiB. The file system would still have overflowed in the same way. The nit picky detail about exactly how big it was is immaterial to the entire discussion. The failure point is that the system was designed in such a way that the file was able to fill up the file system (regardless of how big it was) which resulted in bricking the whole system. Now can we stop wasting time debating a retardedly pedantic point that only autistic people care about but has nothing to do with the actual failure?
The point is that the original article was written for the general public. And while most good engineers understand the difference between 32 MB and 32 MiB, the distinction is lost on the general public. Hell, for the most part, most computer savvy people don't care about the distinction in general. Only the most anal-retentive, pedantic types get their knickers in a twist over it.
Sure, if I am designing something I want accurate specifications so that it removes ambiguity when I have write the code. But outside of that, I will use MB and MiB interchangeably, knowing full well that they are not exactly the same. But when you're having general discussions, you're rarely doing anything more than discussing an order of magnitude so you don't have to be precise all the time.
The few purists I have run into who piss and moan about the difference between MB and MiB in general use are some of the most annoying types of people I've ever run into. They're worse than any SJW trying to push their agenda down your throat. They're not just overly pedantic. They're passive aggressive about it. And that's what really makes people dislike them.
Bottom line, MB and MiB are interchangeable in the modern vernacular. Unless you're talking about the details of an engineering specification, it just doesn't matter. Please develop those interpersonal skills and suppress the urges to "correct" people. It will go a long way to improving your ability to win friends and influence people.
Speaking as an engineer working on software that is on the Orion spacecraft, I can say that rigorous testing is budgeted into the project from the beginning because it helps to avoid most of the problems like this. The testing that goes on with flight software is orders of magnitude more than you find for a traditional commercial product. You have to. The consequences of failure are, obviously, a lot more significant.
That being said, it's impossible to catch every single possible bug, especially as systems get more and more complex. But there are strategies that help reduce your risk. For example, you don't just run off to kernel.org and throw the latest stable release on a board. You pick operating systems that are maybe a bit harder to use (i.e. limited in what they can do) but are far better suited to real-time embedded work. And you certainly don't blindly append to a file without verifying that you're not going to overflow your space. And you always have an automated recovery plan for any dynamically allocated space in the event of an overflow.
This kind of failure is caused by amateurs making amateur mistakes. It was caused by application programmers who don't understand the consequence of failure in a constrained environment where you can't just click a mouse to restart the program. It was caused by poor planning and a lack of understanding of the environment in which they were designing. This was caused by hiring coders instead of experienced engineers. It was caused by trying to do it cheap rather than spending the money to do it right. They got what they paid for.
It's not either/or because wiring two complete sets of circuits in your house costs a lot of money and doesn't really give you any real net benefit. It's far cheaper and easier to wire with one standards and use plug-in adapters where you need them. It's even cheaper and easier to setup your whole house battery backup to power only a subset of your outlets so you don't have the big devices drain your batteries.
I wish we could get three phase power in a residential area. I could get a welder that was easier to fine tune and better machine tools for metal working.
It is the native voltage when you're dumb enough to write your article based on your use of 12V car batteries in your Airstream trailer with 12V solar cells to charge them. Further proof that the guy who wrote the original article has no idea what he's talking about and the article is complete bunk.
But this pedantic bickering about the specific voltage is all a side distraction over semantics and doesn't really have anything to do with the point I'm trying to make.
It doesn't matter what the voltage really is in the end. It could be any array of batteries providing any level of "low voltage" power. Using battery packs in your home along with solar in an effort to reduce your consumption from the grid is not going to drive anyone to do large scale replacement of 120VAC (or 240VAC where it is used outside the US) in the home. We as a society are not going to rewire our homes and replace all of our electrical devices to make the switch. There's too much inertia in the system to make a change that nets us no real gain. Using batteries directly in the house just isn't going to happen. They're going to have to go through an inverter to supply the needs of the average house because that's cheaper and easier than reconfiguring your house to use DC at any voltage.
Did you actually RTFA? The author specifically talked about 12V DC power being the "low voltage" standard that we need to convert to in the home because it's the native battery voltage.
The article was not about adopting industrial data center voltage standards. It's about using voltages that match the batteries that you are using so you don't have to convert to anything else.
I was quoting a movie.
I'm not sure that home batteries will drive a switch to low voltage DC. There's a ton of inertia to overcome. The cost of retrofitting the wiring to handle the higher amperage of using lower voltage alone will be thousands of dollars for every single house, apartment, and office. A simple 20A 120V circuit changed over to 12V will draw 200A. You're going to need to upgrade to 4 or even 2 gauge wire at a minimum to handle that kind of current. And that's a lot of money.
The switch from AC to DC inside the home might be feasible but there's no way you can convert the entire grid. You'd have to rebuild the whole grid from scratch to convert from AC to DC. The transformers to step the voltages up or down simply don't work unless they're pushing AC so how do you handle industrial level supply being stepped down to household voltages at the neighborhood transformer? And who's going to pay for the switch? And what about the industrial users who don't need to run low voltage DC? How do you satisfy their demand?
Then you have to deal with how a substantial number of appliances are built. Many are designed for AC current and won't work with DC, regardless of the voltages. Sure, you can swap out the power supply in your desktop PC to take a DC feed without a lot of trouble. And if electronics retailers had a standard DC wall voltage to work with, you'd see most consumer electronics move to those standards. But how do you deal with a cable modem that needs 12V and a home router that takes 9V? Who wants to go out and replace all of their equipment that is running just fine right now? Who has the money to do that?
And here's the kicker. What real benefit do we gain from a switch over to low voltage DC in the house? Sure, some of the consumer electronics we use won't need that big wall vampire to supply them with power. And sure, we don't really need to run our lights from 120V when 12V can still drive enough light from LEDs without any trouble. But what about the appliances in the house that really draw the bulk of the power in the house? The 240V electric stove or the heat and AC systems? What about your refrigerator and your washer/dryer? Hell, can you imagine the amperage draw trying to recharge your electric car with 12V? And are you going to just skip using those appliances when you're running on battery power?
So if you're going to have to keep your 120V AC based house wiring for your major appliances, do you really want to spend all the money installing a low voltage subsystem for a few consumer electronic devices to supplement the wiring you already have? I know I wouldn't want to.
Like everything else that is poised to "fundamentally change the way we do things", the dreamers never consider the practical reality of actually making the change. In reality, I think we're going to have to deal with the inefficiency of converting from DC battery power to 120V AC for the home. There's just too many things to overcome for little to no benefit.
"I'll tell you another thing. Their beer sucks."
Yep. Tons of details to keep track of. Lots of opportunity to miss something and have a mishap. A computer that is programmed right is a much better tool for keeping track of everything all at once. And the way it's being designed, it just sits there silently until the operator starts doing something that can cause a problem. It gives the operator a reminder while (if) there's still time before taking over and stopping the train. But it also provides useful information on the screen about what's coming up on the route, much like a GPS in your car, so it unobtrusively helps remind the operator of things they might be forgetting.
Once they get it installed and work out a good deal of the bugs, it will be a pretty good system.