Exactly. Trying to treat software development as a true engineering discipline today would be crazy, for the simple reason that we don't know how to reliably do it right yet. There are too many competing theories. There is too little evidence of which are better or worse. A lot of the loudest voices in our industry are not the people producing the best results, because the people who produce the best results are often too busy getting on with it.
Trying to license software developers as a profession too soon could result in the snake oil salesman consultblogspeakauthors writing the specs. Speaking as someone who does have to make highly reliable software, I find that idea horrifying. I would most certainly resent the implication that their unproven, ever-changing methods should constrain my ability to build solid, proven products for my clients.
Sadly, this may be the biggest reason that we don't use better programming languages and tools routinely today. People have been trained to accept bug-ridden, insecure, short-lived software as if it's somehow inevitable. It's probably the greatest con in the history of technology, and we geeks are way too willing to accept it ourselves, as many comments in this discussion demonstrate.
It is fundamental to C++ that its language tools have extremely low run-time overhead, often zero, compared to implementing the equivalent functionality manually.
In some cases, the tools in C++ can even let you generate better code than an idiomatic C implementation. One common example is metaprogramming using templates, which effectively gives you a compile-time ability to inline and optimise specialisations. Another is error handling using exceptions, where implementing stack unwinding using an automatically generated jump table can eliminate the need to manually check for error conditions at each level in the success path.
No-one has the knowledge and skills to write 100% correct programs in general without help. You don't. I don't. No-one does. It is simply beyond human capabilities, because we make mistakes. Arguing for a world that is based on unrealistic assumptions is futile. Better, IMHO, to argue for how to improve the world we actually have.
To that end, my position is still that C is a needlessly weak and dangerous programming language by modern standards, and as evidence, I cite once again the fact that so many system programs written in C have had serious bugs that could have been prevented in a safer language.
Well, I do quite a bit of work in high-performance, high-reliability environments and I've shipped systems that have gone years in production without a single reported implementation bug, so I'll take my chances when it comes to software quality and what good tools can do.
Those results are thanks in no small part to automated methods of catching design and implementation mistakes. I can encode invariants in types. I can write formal specifications and automatically generate large numbers of tests to validate code against them. I can use language features that let me write essential code but then automatically implement additional code to take care of necessary consequences. I can even implement a full DSL with a verified implementation, and have the structure of the DSL guarantee that inputs are valid in some sense.
These don't have to be big deals. They can be as simple as catching a careless direct equality check on floating point values, or ensuring that a temporary buffer is always deallocated even in error paths. They can also be as complicated as automatically generating and running thousands of test cases to check that a DSL compiler that has been implemented and updated over many years still provides output that satisfies the required conditions.
One way or another, most of these tools come down to cross-checking some specification that is relatively simple and easy for humans to verify against a code base that may be relatively large and complex. Computers are much better at that job than I am. It's not a substitute for careful design work or code reviews or any of the other good practices we might agree on. It's just using the computer's automation abilities so I can focus my human, creative skills where they are most valuable.
The problem is that almost no-one is working on potentially good alternatives. C is "good enough", as long as we're willing to tolerate the horrific bug rates and lack of productivity it comes with, and apparently most paying customers still are.
This is another false dichotomy argument. Nowhere have I said anything about using some high-level language with some heavyweight runtime framework to implement things like OS kernels and device drivers. Obviously those aren't the right tools for that job. However, that doesn't mean we should stick with a language as weak and dangerous as C for systems programming work. There's a vast range of possibilities in programming language design, and there's a lot we could do to improve safety and expressive power over C while still compiling to efficient, self-hosted, native code and providing low-level access to hardware where it's needed, which are the two fundamental requirements for a language to be useful for systems programming.
Sure, when you're talking about DMA or interrupts or other things driven by the hardware itself, the software alone can't protect you from everything. However, there's still no reason we should be passing around void* parameters and using text-based macros as a crude tool to implement missing language features in 2016. How much low-level hardware access a language supports and how the language presents that functionality to the programmer are two different issues.
Sorry, I'm not seeing anything there that negates my point. There is no technical reason we couldn't use a language with a modern type system and features, and with sane handling of low-level elements like pointers, ports and buffers, yet still have the kind of low-level control that Torvalds is talking about in your video. The problem is that hardly anybody has both the desire and the resources to write that language and build everything else that goes with it, because C is considered "good enough" by too much of the industry.
I think Bjarne Stroustrup, among others, could reasonably challenge your claim.
But as I said, the reasons for C's continued popularity aren't technical. There's a huge ecosystem around it, including using it as programming's lingua franca. For that to change, either we need an industry heavyweight with enough resources to create not just a better language but the tools and libraries and developer ecosystem around it, or we need some sort of external pressure to drive the change, so that enough professional developers start caring enough about improving quality to switch to new languages and tools despite C's established presence.
I chose my examples very carefully. All of the things I mentioned can be prevented to some extent (much more than C does) at compile time, with zero run-time overhead.
There are also plenty of techniques for managing memory that are significantly safer than C, yet still deterministic and with negligible run-time overhead rather than relying on out of band garbage collection.
As for your expert programmer, I don't think that person exists, and I say that as someone who writes a lot of high-performance, high-reliability code for a living.
A programmer that can automate its work will be replaced by his computer in a couple of years tops.
No problem. Once I've finished automating the routine parts of what I currently build, I'll be able to build more capable systems faster next year using the extra time.
Perhaps, but when close to 100% of a population have the same trait, arguments that the trait should be changed rather than designing tools and processes that accommodate that trait are unrealistic and therefore not very useful.
While we have yet to invent a totally foolproof language, there is still a matter of degree. We may never be able to eliminate all programming errors with tools alone, but we can certainly eliminate entire categories of them, and some languages are very much better than others at doing that.
You know well that the same can be said for whatever language you prefer.
Sorry, but I don't agree with that. I see no technical reason why, in 2016, we should still be writing high reliability systems in a programming language with cryptic, error-prone syntax where the default is to accept code that is almost certainly erroneous. Buffer/stack overflows, type mismatches, null pointer errors and numerous other classes of programming bug that are ridiculously common in C code should all have died out years ago, and the reasons for C's continued popularity have very little to do with its technical merit.
What happened with "We rely on the developer to do a good job?"
We tried that experiment, and it failed when roughly 0% of professional programmers turned out to be more reliable than an automated tool designed specifically to prevent certain types of programming error.
Can we just stop finding excuses to deliver crap quality code?
You're implying a false dichotomy. There are plenty of programmers who produce generally decent code but still make mistakes that better tools will catch before they go into production.
A good craftsman doesn't insist that his tools necessarily do the job for him either.
As programmers, automation is the essence of what we do. Any programmer who isn't insisting on their tools doing work so they don't have to do it themselves isn't making very good use of those tools. That is as true for safety, security and defensive programming as for any other aspect.
There ain't no such thing as free memory checking. It takes extra code and therefore takes extra time.
It wasn't clear to me whether the author meant run-time speed or development speed. Certainly better languages and tools can make a big difference to the latter.
It's also quite possible for better languages to generate run-time code that is more efficient. The more semantic information about programmer intent, restrictions and guarantees can be encoded using the language, the more scope there is for optimisers to produce better output.
Plus the memory-unsafe premise is BS. There is nothing preventing a programmer from adding their own memory checking in such languages.
But that only matters if programmers do add their own checks. Evidently in reality most do not.
It's not the language, it's the programmers and the rush to produce easy code.
Well, I think it's a lot the language as well. To a first approximation, every major piece of system and networking software written in C has had serious security issues at one time or another, even the ones written by the best programmers of their generation and hailed as being exemplary in their code quality. I think after the first few decades of evidence we're allowed to call this one now, and say that writing critical software in unnecessarily dangerous languages produces less than optimal results.
Is that some sort of misplaced arrogance, or do you really not understand how easy blocking WhatsApp/Facebook would be if the German authorities wanted to do it?
People write as if the Internet is some huge network that everyone has unlimited access to, but guess what? It's not. You have an ISP, and somewhere up the line they are hooked in to a relatively small number of pipes in and out of any given country, and those pipes are controlled by a major infrastructure provider that isn't going to argue with the national government.
The political fall-out could be a different question, but somewhere like Germany the people are very cautious about excessive surveillance and profiling for obvious historical reasons, so I wouldn't bet on WhatsApp/Facebook winning the PR battle either.
Well, that access could be abused and apparently with no worse penalty than losing a job they clearly shouldn't have had in the first place, for one thing.
Well, YouTube and the record labels did figure this one out a while ago. They have various forms of advertising along with the content when it's served from YouTube, they're all getting some cut from it, and listeners are free to enjoy the music.
It's reasonable to claim that the ripping tools are undermining that, reducing YouTube usage by promoting illegal copyright infringement as an alternative, and that they are doing so on a commercial scale for profit. So the businesses who have the legal rights are suing, and I can't imagine any likely ending for this that doesn't involve an injunction and significant damages being awarded.
Slashdot Mirror
All true, but security is not black and white, either in airports or in programming.
Exactly. Trying to treat software development as a true engineering discipline today would be crazy, for the simple reason that we don't know how to reliably do it right yet. There are too many competing theories. There is too little evidence of which are better or worse. A lot of the loudest voices in our industry are not the people producing the best results, because the people who produce the best results are often too busy getting on with it.
Trying to license software developers as a profession too soon could result in the snake oil salesman consultblogspeakauthors writing the specs. Speaking as someone who does have to make highly reliable software, I find that idea horrifying. I would most certainly resent the implication that their unproven, ever-changing methods should constrain my ability to build solid, proven products for my clients.
Sadly, this may be the biggest reason that we don't use better programming languages and tools routinely today. People have been trained to accept bug-ridden, insecure, short-lived software as if it's somehow inevitable. It's probably the greatest con in the history of technology, and we geeks are way too willing to accept it ourselves, as many comments in this discussion demonstrate.
What baggage, specifically?
It is fundamental to C++ that its language tools have extremely low run-time overhead, often zero, compared to implementing the equivalent functionality manually.
In some cases, the tools in C++ can even let you generate better code than an idiomatic C implementation. One common example is metaprogramming using templates, which effectively gives you a compile-time ability to inline and optimise specialisations. Another is error handling using exceptions, where implementing stack unwinding using an automatically generated jump table can eliminate the need to manually check for error conditions at each level in the success path.
No-one has the knowledge and skills to write 100% correct programs in general without help. You don't. I don't. No-one does. It is simply beyond human capabilities, because we make mistakes. Arguing for a world that is based on unrealistic assumptions is futile. Better, IMHO, to argue for how to improve the world we actually have.
To that end, my position is still that C is a needlessly weak and dangerous programming language by modern standards, and as evidence, I cite once again the fact that so many system programs written in C have had serious bugs that could have been prevented in a safer language.
Well, I do quite a bit of work in high-performance, high-reliability environments and I've shipped systems that have gone years in production without a single reported implementation bug, so I'll take my chances when it comes to software quality and what good tools can do.
Those results are thanks in no small part to automated methods of catching design and implementation mistakes. I can encode invariants in types. I can write formal specifications and automatically generate large numbers of tests to validate code against them. I can use language features that let me write essential code but then automatically implement additional code to take care of necessary consequences. I can even implement a full DSL with a verified implementation, and have the structure of the DSL guarantee that inputs are valid in some sense.
These don't have to be big deals. They can be as simple as catching a careless direct equality check on floating point values, or ensuring that a temporary buffer is always deallocated even in error paths. They can also be as complicated as automatically generating and running thousands of test cases to check that a DSL compiler that has been implemented and updated over many years still provides output that satisfies the required conditions.
One way or another, most of these tools come down to cross-checking some specification that is relatively simple and easy for humans to verify against a code base that may be relatively large and complex. Computers are much better at that job than I am. It's not a substitute for careful design work or code reviews or any of the other good practices we might agree on. It's just using the computer's automation abilities so I can focus my human, creative skills where they are most valuable.
The problem is that almost no-one is working on potentially good alternatives. C is "good enough", as long as we're willing to tolerate the horrific bug rates and lack of productivity it comes with, and apparently most paying customers still are.
This is another false dichotomy argument. Nowhere have I said anything about using some high-level language with some heavyweight runtime framework to implement things like OS kernels and device drivers. Obviously those aren't the right tools for that job. However, that doesn't mean we should stick with a language as weak and dangerous as C for systems programming work. There's a vast range of possibilities in programming language design, and there's a lot we could do to improve safety and expressive power over C while still compiling to efficient, self-hosted, native code and providing low-level access to hardware where it's needed, which are the two fundamental requirements for a language to be useful for systems programming.
Sure, when you're talking about DMA or interrupts or other things driven by the hardware itself, the software alone can't protect you from everything. However, there's still no reason we should be passing around void* parameters and using text-based macros as a crude tool to implement missing language features in 2016. How much low-level hardware access a language supports and how the language presents that functionality to the programmer are two different issues.
Sorry, I'm not seeing anything there that negates my point. There is no technical reason we couldn't use a language with a modern type system and features, and with sane handling of low-level elements like pointers, ports and buffers, yet still have the kind of low-level control that Torvalds is talking about in your video. The problem is that hardly anybody has both the desire and the resources to write that language and build everything else that goes with it, because C is considered "good enough" by too much of the industry.
I think Bjarne Stroustrup, among others, could reasonably challenge your claim.
But as I said, the reasons for C's continued popularity aren't technical. There's a huge ecosystem around it, including using it as programming's lingua franca. For that to change, either we need an industry heavyweight with enough resources to create not just a better language but the tools and libraries and developer ecosystem around it, or we need some sort of external pressure to drive the change, so that enough professional developers start caring enough about improving quality to switch to new languages and tools despite C's established presence.
I chose my examples very carefully. All of the things I mentioned can be prevented to some extent (much more than C does) at compile time, with zero run-time overhead.
There are also plenty of techniques for managing memory that are significantly safer than C, yet still deterministic and with negligible run-time overhead rather than relying on out of band garbage collection.
As for your expert programmer, I don't think that person exists, and I say that as someone who writes a lot of high-performance, high-reliability code for a living.
Thank you. Yes, that's pretty much what I meant.
A programmer that can automate its work will be replaced by his computer in a couple of years tops.
No problem. Once I've finished automating the routine parts of what I currently build, I'll be able to build more capable systems faster next year using the extra time.
Perhaps, but when close to 100% of a population have the same trait, arguments that the trait should be changed rather than designing tools and processes that accommodate that trait are unrealistic and therefore not very useful.
While we have yet to invent a totally foolproof language, there is still a matter of degree. We may never be able to eliminate all programming errors with tools alone, but we can certainly eliminate entire categories of them, and some languages are very much better than others at doing that.
You know well that the same can be said for whatever language you prefer.
Sorry, but I don't agree with that. I see no technical reason why, in 2016, we should still be writing high reliability systems in a programming language with cryptic, error-prone syntax where the default is to accept code that is almost certainly erroneous. Buffer/stack overflows, type mismatches, null pointer errors and numerous other classes of programming bug that are ridiculously common in C code should all have died out years ago, and the reasons for C's continued popularity have very little to do with its technical merit.
What happened with "We rely on the developer to do a good job?"
We tried that experiment, and it failed when roughly 0% of professional programmers turned out to be more reliable than an automated tool designed specifically to prevent certain types of programming error.
Can we just stop finding excuses to deliver crap quality code?
You're implying a false dichotomy. There are plenty of programmers who produce generally decent code but still make mistakes that better tools will catch before they go into production.
A good craftsman doesn't insist that his tools necessarily do the job for him either.
As programmers, automation is the essence of what we do. Any programmer who isn't insisting on their tools doing work so they don't have to do it themselves isn't making very good use of those tools. That is as true for safety, security and defensive programming as for any other aspect.
There ain't no such thing as free memory checking. It takes extra code and therefore takes extra time.
It wasn't clear to me whether the author meant run-time speed or development speed. Certainly better languages and tools can make a big difference to the latter.
It's also quite possible for better languages to generate run-time code that is more efficient. The more semantic information about programmer intent, restrictions and guarantees can be encoded using the language, the more scope there is for optimisers to produce better output.
Plus the memory-unsafe premise is BS. There is nothing preventing a programmer from adding their own memory checking in such languages.
But that only matters if programmers do add their own checks. Evidently in reality most do not.
It's not the language, it's the programmers and the rush to produce easy code.
Well, I think it's a lot the language as well. To a first approximation, every major piece of system and networking software written in C has had serious security issues at one time or another, even the ones written by the best programmers of their generation and hailed as being exemplary in their code quality. I think after the first few decades of evidence we're allowed to call this one now, and say that writing critical software in unnecessarily dangerous languages produces less than optimal results.
Is that some sort of misplaced arrogance, or do you really not understand how easy blocking WhatsApp/Facebook would be if the German authorities wanted to do it?
People write as if the Internet is some huge network that everyone has unlimited access to, but guess what? It's not. You have an ISP, and somewhere up the line they are hooked in to a relatively small number of pipes in and out of any given country, and those pipes are controlled by a major infrastructure provider that isn't going to argue with the national government.
The political fall-out could be a different question, but somewhere like Germany the people are very cautious about excessive surveillance and profiling for obvious historical reasons, so I wouldn't bet on WhatsApp/Facebook winning the PR battle either.
Germany already took action over the Facebook-WhatsApp data sharing, and probably won't be the last EU member state to do so.
Well, that access could be abused and apparently with no worse penalty than losing a job they clearly shouldn't have had in the first place, for one thing.
Well, YouTube and the record labels did figure this one out a while ago. They have various forms of advertising along with the content when it's served from YouTube, they're all getting some cut from it, and listeners are free to enjoy the music.
It's reasonable to claim that the ripping tools are undermining that, reducing YouTube usage by promoting illegal copyright infringement as an alternative, and that they are doing so on a commercial scale for profit. So the businesses who have the legal rights are suing, and I can't imagine any likely ending for this that doesn't involve an injunction and significant damages being awarded.