Slashdot Mirror
New Book Argues Automation Is Making Software Developers Less Capable
dcblogs writes: Nicholas Carr, who stirred up the tech world with his 2003 essay, IT Doesn't Matter in the Harvard Business Review, has published a new book, The Glass Cage, Automation and Us, that looks at the impact of automation of higher-level jobs. It examines the possibility that businesses are moving too quickly to automate white collar jobs. It also argues that the software profession's push to "to ease the strain of thinking is taking a toll on their own [developer] skills." In an interview, Carr was asked if software developers are becoming less capable. He said, "I think in many cases they are. Not in all cases. We see concerns — this is the kind of tricky balancing act that we always have to engage in when we automate — and the question is: Is the automation pushing people up to higher level of skills or is it turning them into machine operators or computer operators — people who end up de-skilled by the process and have less interesting work?
I certainly think we see it in software programming itself. If you can look to integrated development environments, other automated tools, to automate tasks that you have already mastered, and that have thus become routine to you that can free up your time, [that] frees up your mental energy to think about harder problems. On the other hand, if we use automation to simply replace hard work, and therefore prevent you from fully mastering various levels of skills, it can actually have the opposite effect. Instead of lifting you up, it can establish a ceiling above which your mastery can't go because you're simply not practicing the fundamental skills that are required as kind of a baseline to jump to the next level."
I certainly think we see it in software programming itself. If you can look to integrated development environments, other automated tools, to automate tasks that you have already mastered, and that have thus become routine to you that can free up your time, [that] frees up your mental energy to think about harder problems. On the other hand, if we use automation to simply replace hard work, and therefore prevent you from fully mastering various levels of skills, it can actually have the opposite effect. Instead of lifting you up, it can establish a ceiling above which your mastery can't go because you're simply not practicing the fundamental skills that are required as kind of a baseline to jump to the next level."
Wearing shoes prevents you from developing a thick layer of skin on the bottom of your feet.
Farming equipment gave us farmers with weaker muscles and fatter bellies. However, it did give us more food.
'nuff said.
Why are programmers singled out? Arguably computer automation is helping everyone. Skills are only important to have if there is value in the skill. I don't think theoretical computer scientists are any less intelligent because they never program in C or C++.
I hear this argument from some old programmers who've been around forever and seem allergic to learning new technologies. They're still attached to C, producing code with buffer overruns and memory leaks, where the rest of us have moved onto managed code and garbage collection. It isn't about less or more capable, it's that humans are imperfect and if you force them to get the tiniest details right ALL the time, by hand, well that just isn't something human brains are good at doing.
Moving up to higher level tools is a GOOD thing. Otherwise we'd all still be writing in assembly language. When people get stuck at one level and don't want to keep learning, they tend to argue that better approaches are not manly enough. "It was hard in my day dammit and it should be hard today too!" But the goal is to make quality software, not to cowboy the shit out of everything.
I train all the new people that I hire and so I made it a very conscious decision from the very beginning to make sure that they understand the underlying technologies that they are trained in. If it is a build procedure, use ant, not maven, make sure they write build scripts by hand first of all, no generated scripts. Ensure they understand every configuration file that we use, require that they use as much simple code as possible as opposed to configuration heavy frameworks, etc. The exception are code generators that we create, we automate away large tasks by creating code generators that use meta code to generate a bunch of source code for a specific type of use case. But in this case you are not looking at running a configuration, you are running the generated source code and when source is generated it has to be fixed by hand in a number of ways that requires that the developer reads the code.
This produces developers that can debug, and if you can't debug you are not actually a developer.
You can't handle the truth.
Kids These Days!
It is not just about the skills, it is also about the innovation and creativity. Confronted to less challenging problems and solutions, someone will be less prone to innovate. At the end, businesses may put themselves in a position they are no longer capable to maintain their leading position in their own market because creativity has been killed for better control of quality and processes.
Achille Talon
Hop!
Back in my day you had to write your own compiler. Fucking automation codifying standards. What a joke! I don't have time to build a rocket, I'm debugging ASM, so I'm obviously smarter than those Rocket monkeys. Why stand on the shoulders of those who came before when you can just wait till you grow tall enough?
Firstly, programmers are lazy by design. That's a good thing because that means energy efficiency, which also equates to better algorithms in the pattern of thought (that's why we have coffee, to move a lazy programmer into coding). Does automation make a programmer lazy? Not if they practice both. At home as a hobby, open up a Hex Editor and code with directly inputting 0s and 1s like a real man! At work, use your IDE's bean generators and the like, the key point being, as long as you are not helpless when the automation doesn't function as it should.
After all, like he said, "IT doesn't matter".
It's true, though. IDEs full of wizards and other magic make it look like less-capable (cheaper) people can do software. So that's what management hires.
Problem is, the IDEs can make a capable developer's job easier, but an incapable developer becomes utterly lost when the IDE falls short or breaks down.
Why are programmers singled out? Arguably computer automation is helping everyone. Skills are only important to have if there is value in the skill. I don't think theoretical computer scientists are any less intelligent because they never program in C or C++.
I believe this is true.
Fundamental skills such as algorithmic thinking and attention to detail are still important, and they'll remain so even with increasing automation.
Fredrick Brooks spoke to this idea way back in the 1970's and rightly concluded that programmers are going to be with us forever. The author of TFA needs to read and understand "The Mythical Man-Month" (2nd Edition) by Fredrick P. Brooks, mainly Chapter 16, but likely the whole book. This book should be REQUIRED READING for ANY computer related undergraduate degree program.
There is NO silver bullet. You will always need and have programmers. It was true 30 years ago and it's true now. We have not automated our way out of needing programers to ply their craft. Yes, we have "automated" a lot of logistics around computer operations, but this has ALWAYS been a low skill job. Back in the day they ran punch card readers and hung data tapes, none of which took too much knowledge of computers or programming. You needed a system programmer to make the system do anything different than what it did before. System programming was a highly skilled task.
The names and languages have changed, but you have "operators" and "system programers" still today. We call them Administrators and Programmers today. One operates the machines, loads software, manages backups and keeps paper in the printers, the other comes up with programs and systems that do new and unique things. The latter still requires the unique programming skills.
This author is wrong, and would have known had they done their reading.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Please re-read:
Brooks, F. P. , J. (1987). "No Silver Bullet—Essence and Accidents of Software Engineering". Computer 20 (4): 10. doi:10.1109/MC.1987.1663532
http://faculty.salisbury.edu/~xswang/Research/Papers/SERelated/no-silver-bullet.pdf
Automation removes accidental complexity. /discussion
It does not make people's lives harder when the massive proprietary build system goes wrong, the original vendor sucks at support for any number of reasons, and no one in house knows where to begin to fix the build issues.
IF the high-level build system was written using free low-level tools, and there are a few guys around who understand how it fits together, this is a MUCH better position to be in.
Being a software engineer by trade, I can say this is somewhat true. I definitely have coworkers who would fail or barely eke by if it weren't for boilerplate code generation and autocomplete. They certainly don't understand the code they're "writing" or why it works.
I like this attitude. It means I'll be making a bajillion dollars when called to fix your build system, middleware and low-level interface code that all of your current employees are terrified (and unqualified) to touch.
time to forum a union before your automated out of a job or are one of few people left likely pulling 60-80 hour weeks.
It's not automation that's making us "less capable". It's the incessant expectation that we - (software) engineers - be senior, mono-maniacal, obsessive-compulsive, experts in whatever the bloody fad du jour is. That's because darn management and HR dolts have no clue how to assess an engineer, yet expect to make the decision.
Therefore yes: we're "less capable" because we can't keep up with their fads... go fuck yourself. Seriously.
Mi domando chi à il mandante di tutte le cazzate che faccio - Altan
Automating stuff means I am more effective and I have more time to write /. comments and this is good because sometimes solving hard problems means I should just chill for a while, while the automation does the work. Automation doesn't make me less capable, but it does mean that I have 3 times the output of anyone else around me, making me more relaxed and generally easier going while people wonder how I do it.
Automation makes me smart lazy and achieving that is hard work.
My ism, it's full of beliefs.
Cars make people worse at riding horses.
Computers make life easier for everyone, but programmers need to be able to diagnose and correct when it fails to make life easier for everyone else. If a programmer constructed things out of all sorts of middleware/frameworks and automation they did not understand, they are ill-equipped to handle unexpected consequences.
In the same way, car engines make everyone's life easier. You push a pedal and it magically pushes your car harder. If it does not do that, the mechanic better understand how that engine was built in order to competently identity the problem and fix it. That mechanic won't be actually building engines, but knowledge of how to build engines is a necessary part of his job to fix engines built by others.
I don't want to *have* to deal with the meticulous memory management of writing in C, but at least knowing how it's done is a valuable aid in understanding how the language I do use might be affected by choices I make. I understand various concepts around automatic reference tracking and how languages use it. 98% of the time, I don't need to think about it. However, ever so often I have to debug a high-level language using gdb as if it were a C program. This is truly a last resort, but if all the higher level facilities fail to identify the source of an issue, gdb or strace can be a godsend.
So yes, keep learning, but don't skip the fundamentals. Skipping the fundamentals can impair your debug skills, cause you to embrace a middleware package that really doesn't add anything to your solution, or fail to provide actionable bug information upstream to the framework you are using that isn't behaving as expected.
Oh please... I'm an old guy who cut his teeth on "C" programs, but your attitude is just wrong.
Tools change, languages change, computers change, methods change, but programming remains the same under all the trappings. Who's a better programmer? One that does Java or one that hacks out Assembly? Neither. Yes the programing model changes and the tokens you use to manipulate this model changes, but I don't care what you code in or how familiar you might be with some past tool I've used, if you program, you use the same skills I used hacking out "C" even if you are using some language I've never heard of.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
You mean the automation tool, make?
Yea, it was great when every single browser implemented "getelementbyid" slightly differently, so you had a cluster fuck of "if browser = 'x', then y"
That was one thing I don't miss.
So, a developer who is a slave to a black-box set of tools which he doesn't understand is equal to one who can look into a whole spectrum of problems? No, sorry, not the case. I'd much rather hire a C programmer to write Java than the other way around. Only one will deliver something usable.
Every time something gets easier when it comes to software development, people just push things further (as they should).
Thats why no matter how many versions of Internet Explorer we stop supporting, web development is still a pain in the ass. The moment people stop doing something hard, they just take all the time they saved, and tackle something else that all of a sudden become worth it (ie: supporting mobile operating systems, optimizing stuff with webgl if its available, whatever, you name it)
Things that went from "No way, we don't have time for this!" become standard, and the cycle continues. Every minute we save because a task is automated or redundant, is a minute someone spends doing something that was once impossible. And if that person works for a competitor, you now have to catch up.
The summary seems a bit misleading. The main thrust of page I saw what that the push to replace work with automation can have consequences at a certain level. Does decision making really work well in automation, or does it lead to problems? There's evidence in both camps. An example, some traders on Wall Street have complained about removing people from the process, they that really do add value at times. And sure, it's hard to imagine a human would issue a massive amount of bad orders, but a computer model with a bit of glitch might. But, is that enough to slow things down. Just one example of many.
In my mind, critical thinking does have value, and no, there is nothing in data science, machine learning, etc. that really comes even close to what humans can do in that area. There's a big debate in Medicine about following best practices and if just following algorithms would work better. Some note it would reduce unneeded tests and procedures. Others have noted that actually, doctors are much better at noting when something is going really wrong and that following a script could lead to unnecessary deaths that would be avoided by relying on clinical judgment. Is there is a need for better data? Sure, but can you really automate judgment? And what real value is there of taking the craft out of everything for humanity as a whole?
The problem is that some people don't think software engineering, programming, coding, whatever requires critical thinking, or that there is a craft or art to programming. And you can increasingly do it that way. Cut and paste, copy from the web, and when things don't work out, post on the web and hope somebody answers.
What is lost is somebody has to have the skills to figure out what is going wrong or that it can be done better. Where do those answers come from on the web after all? At some point, somebody has to know how to actually approach the problem from the fundamentals and solve it, and that's when all those things that we (okay, at least me and my schoolmates) studied in CS come into play.
I'm on a project and they are just throwing idea after idea to figure out a performance problem. Sure, it's tricky, but I realize, they have a huge blind spot. They don't know how to attach a low-level debugger to a process, to monitor OS resources, or even realize that you can debug something without sources. Sure, it's a Java enterprise application, so that's another layer of hard, but it can be done. Cripes, we had to debug core dumps. I'm glad (thrilled) that I don't have to do it anymore, but the skills that I learned doing it were invaluable.
A related aside. The problem is not better tools, it is not knowing there are better (or any) tools or that you can make better tools.
... but so what?
My calc tools elevate me way beyond that inaccurate analog piece of crap.
It little behooves the best of us to comment on the rest of us.
Not quite a fair comparison because cars don't rely on horses under the hood. All high-level software and tools is built using low-level software and tools. One does not simply replace the other.
By that argument, we should have a serious shortage of mathematicians since the invention of the scientific calculator.
We should also have a lot of bookkeeppers and no accountants.
Automation reduces the "grunt work"; it does not remove the need to think. If your job can be accomplished purely through the "grunt work" done by something like http://msscodefactory.sourceforge.net/ without you having to do any customization work or handle any special cases that aren't automated, you were never a real programmer to begin with, and your project was a joke in the first place.
I do not fail; I succeed at finding out what does not work.
While I'm sitting here reading about other people bitching about abstraction libraries such as jQuery, my first thought was actually about testing processes.
Pretty much everywhere I look online, projects are FLOODED with automated testing tools to ensure their code works. And sure enough, every bug that I submit has an "automated test" that didn't test that particular condition. Developers are relying more upon these testing tools and less upon actually USING their own services they're developing to ensure they work properly.
A great example: a recent bug I had to file was brain dead simple. File opening ignored current working directory, so if you changed directories, the file open command would still assume you're in the base directory. Why wasn't this caught? All of their file handling automated testing routines ONLY checked absolute paths, none checked relative paths. On top of that, when they finally did add relative path testing after my bug report, they only added it as relative to the base path, and not testing against current working directory.
Now, let's think about this for a second. How long has the concept of changing directories been around? While most of us will go "oh, DUH!" to the bug mentioned above, newer developers may simply not be in that same mindset, because they're not actively traversing their filesystems themselves. The automated toolkits are doing all that work for them, leaving the developers less experienced in this area, and thus less forethought when building the next generation of tools to test these exact sorts of issues. It is a downward spiral.
The flourish of automated tools appeared as a result of the 90s and early 2000s generations assimilating advanced programming skills. These generations knew how to directly handle hardware interrupts (and write peripheral drivers), write a tcp socket from scratch, code proper random access file management, program a perspective calculator for 3D rendering, etc. Now all these are deemed unnecessary because it's considered reinventing the wheel.
The question is who will continue the tradition of scientific innovation in computing if the younger generations aren't taught these basic skills? We should all ensure that our children go through the ritual of learning the fundamentals as we did, even if they eventually end up primarily using xcode (or whatever other heavily sandboxed coding environment exists then).
Your calc tool was not built using slide rules, and no part of it's production relies on slide rules. High-level software tools on the other hand, rely on low level software tools in order to work.
As a C programmer I wouldn't hire a C programmer to program Java, if I also had the option of hiring a good Java programmer.
First of all, as a C programmer that person will probably dislike java, and write C like code in java.
Second of all, the Java programmer will already be familiar with the tools, API, libraries etc. and will therefore be more efficient.
Good programmers write good code regardless of language, where good is determined by some criteria you have for the job.
So, when the tractor breaks, the farmer fixes the tractor. However, when the compiler breaks, many of our modern "programmers" are incapable of identifying the problem. Further, the farmer is aware of what can break the tractor and actively avoids that. The modern "programmer" is not aware of what will break the compiler, and so poorly prepared.
I'm not anti-agile, but come on, agile generally changes the user interface, not the business logic.
Yea, it was great when every single browser implemented "getelementbyid" slightly differently, so you had a cluster fuck of "if browser = 'x', then y"
This has actually never been the case with getElementById, specifically. There was *one* slight and completely avoidable corner-case in some older versions of IE... deprecated years before jQuery existed.
At one point we needed horses to bring the raw materials to the car factories; now we have cars and trucks bringing the raw materials to build newer/better/more advanced cars. Not as different as you think.
In the dev world, we are required to do all things from R&D , BA, coding, testing, tech writing.
In the medical world, the ops table has 12 people each with minor specialization.
Imagine if there was one doctor doing the prep, drugs, cleaning, cutting, sewing up, the works.
Liberty freedom are no1, not dicks in suits.
that as time goes on, you end up using more and more complicated and advanced tools (oftentimes built out of a multitude of 3rd-party components that get sewn together like a sort of Frankenstein monster). As long as everything works properly everything is fine. But when they stop working for some reason, you have a problem. As a developer, the inner workings of the tools are by design pretty opaque, and yes in *theory* you can download the source and try and debug the thing yourself, but that's an incredibly poor use of ones time.
It's sad how many "programmers" I meet who have never even typed "make" let alone more interesting things like readelf and strace.
That's just elitism.
I agree that jQuery has it's flaws, but it is also a godsend in many other ways. The company I'm currently working for writes web applications for K-12. Some of our customers are running XP with IE7, others are using iPads, Chromebooks, and everything in-between. While the majority of the client-side code is just in plain JavaScript, we've started incorporating some HTML5 functionality into our products recently, unfortunately some of the older browsers are a bit lacking in support. Often times jQuery will have a wrapper that will implement that same functionality if the browser doesn't natively support it, but if it does then it will just act as a wrapper to call the native implementation with a minimal performance impact.
Hopefully a few years from now when XP has finally disappeared entirely and everyone is running a standards-compliant browser (as if there is such a thing) there will no longer be a need for a jQuery-like project, but until then I'm very glad that it exists.
Hand crafted code and makefiles will probably yield a superior result.
Studies showing the errors per thousands of lines of code in open-source vs proprietary software, time and time again, seems to support such an assertion.
Uh, Linux geek since 1999.
We automate in order to reduce the costs of development, so we can maximize profits.
That is the only reason. Employer's have no incentive to care whether or not their employees are bored. They don't pay their employees to like their work. They pay their employees to do their work. And if automation makes that cheaper, then full speed ahead! If automation means I can hire a stupider, and hence cheaper, class of laborer, then I win again! Concerns about the long-term cultural implications be damned.
Individual software developers might occasionally write a script that automates some tedious thing they frequently do. That is different, since it is employee-initiated and is an effort at avoiding boredom. The employer tolerates this behavior so long as the net effect is either neutral or positive in terms of productivity. The moment you, as an employee, start arguing that you should invest the employer's time in something that is less profitable but more interesting, you will be replaced.
> what possible reason can there still be to do it by hand any more?
My baby daughter will probably always have a smartphone / calculator with her when she grows up. She'll have a tool that can so arithmetic for her. Yet, I plan to teach her arithmetic with jelly beans, hands on, doing it manually, so that she UNDERSTANDS what multiplication is all about. Once she really understands it, she'll know when and how to use it. She can then use the calculator as a shortcut, but use it effectively.
I do the same with my employees. First, they learn the process manually so they understand it. Then, they use the automated tools. Whwn the automated tool doesn't work or needs an extra argument because of a special case, the employee can handle it because they understand what the tool is trying to do. One of our most recent hires improved the main automation tool considerably, something he couldn't have done without first truly understanding what it does by having gone through the process manually.
I'm a giant nerd who can't throw a ball, can't dance, can't sing, and that's given me time to develop some skills that give me a reputation as a bit of a programming and sysadmin savant. Someone calls up with a problem and they start by saying "it's really strange, a lot of people ", I interrupt "hold on one second. Yep, the time must be set wrong on one of your web servers. You're using a cluster, right?" Aghast, they say yep, two weeks ago they switched from a single server to a true cluster. A large part of the "magic" is simply understanding the stack from top to bottom. Not USING assembly or C, but being CAPABLE of doing so, so I can imagine what the PHP calls are doing, what the code would look in C. Then I can picture what the system calls must be, and what that means to the drive heads. I'm darn sure not expert in C, in .Net, or most anything else, but I've done just enough low level stuff to picture what curl must be doing in memory as it fetches a web page.
It's what JavaScript should have been.
My first PC had a 43 MB hard drive. And it had a keyboard. I recently saw a keyboard box stating requirements of 150 MB of hard drive space. And it was a pretty bare bones MS keyboard. WTF!?!
Yeah, I miss the days when we coders worked closer to the metal, calling getelementid directly instead of using some high level abstraction like jquery.
It's sad how many "programmers" I meet who have never even typed "make" let alone more interesting things like readelf and strace.
That's just elitism.
Wow, who knew Ben Affleck reads Slashdot?!
Airline pilots. They're hardly teaching them how to fly the plane anymore.
“He’s not deformed, he’s just drunk!”
Remember Crystal Reports? This was a tool that came out in 1991, that was supposed to make it possible for ANYONE to create their own business reports. No programming know-how needed. Remember?
For the past three years, I ran a team that created customized Crystal Reports for customers...because they couldn't figure out how to create them for themselves. It wasn't so much that the tool was hard to use, but more that the customers had only a foggy idea of what they wanted their reports to show.
For example, a doctor's office would call saying "I want a report that shows a list of patients who haven't had a mammogram in the last two years." Sounds easy enough. Then we would start asking questions: Do you want only active patients, or all? Do you want only those within a given date or age range? What patient information do you want in the report? Do you need totals or counts? Then, after we delivered the first draft, there would always be changes: "Yes, I like this, but can you change this filter or that column?" They needed to have experts guide them to a report that made sense for them.
Yes, software is getting better, but it will never replace the need for understanding.
Our politicians used to talk like this. Here are few examples:- 1. Computers will take the jobs of millions of people 2. Teaching English will destroy our civilisation 3. Creating Dams for hydel power projects will make the water useless for irrigation
I've done punch cards (Fortran) and assembler. Now I use .NET and C# to create line of business applications I couldn't have dreamed of building 20 years ago.
Anyway, coding is challenging, but the really hard part of application development is knowing what to code. If you read the reports, when projects fail it is usually because the parts don't fit together properly, or the damn thing does the wrong thing.
I create applications that reduce errors and save money for organizations. Sometimes they allow types of interactions that weren't possible before. I am not a computer scientist, in fact my degree is in psychology. Still, I like to code, and my applications do what they are supposed to do. That works for me and for my clients. Perhaps I should add that these apps are not small or simple. One a few years ago went well over a million lines of code. Others use mapping, public server APIs, etc.
I haven't done C or assembler in decades, and there is no reason that I would.
I'm fully supportive of the idea that programmers shouldn't write assembly. I'm a lot more hesitant about a programmer who never reads assembly. Compilers do odd things. Sometimes they do clever odd things, sometimes they do stupid odd things. And sometimes they just contain bugs.
I am TheRaven on Soylent News
Yet, I plan to teach her arithmetic with jelly beans, hands on, doing it manually, so that she UNDERSTANDS what multiplication is all about. Once she really understands it, she'll know when and how to use it. She can then use the calculator as a shortcut, but use it effectively.
There are a few questions to ask about this sort of teaching:
In the case of simple arithmetic, if you include the time of getting the phone out of your pocket, then the answer should be yes to all of them - it's just useful. The education system (in the UK, at least) unfortunately tries to take the same approach for things like calculus. You can understand how to solve differential equations (although more by applying rote-learned rules than really understanding what's going on), and that's useful. After a year of practicing, you may take 10 minutes instead of 30 to solve some complex equations, but the computer will take under a second, so was it really worth spending that year practicing?
The same is true for most automated tools. There's no benefit in writing your own, but there's a big benefit in being able to write your own. Or in writing your own and then understanding why it's not as good as the one that loads of other people have been working on for years...
I am TheRaven on Soylent News
As Brooks would say, there is no silver bullet!
There is no way to reduce the essential problem, automation can only reduce the accidental problem.
Consequently, automation, will only free thinking time to solve the esential better.
I was a "capable" hardcore C programmer who knew pointers and language quirks inside out. But work dried up. Unless you program for embedded processors, a field I was never exposed to in any way, there's just no jobs for C programmers these days. I'm doing less capable work with Java, but making a living. So what are you going to do? I can't even remember C these days. No one will pay me to write anything in C.
The real problem is slightly different.
High level tools are usually very good at solving some specific kinds of problems/patterns.
Many programmers just learned how to use these tools and do not understand the basic concepts.
This way, when a slightly different variant of the same problem appears, requiring a slightly different solution,
they just cannot solve it, or else create a very inefficient solution.
All silly examples aside, this rings true across any area of work where becoming more technically proficient is a requirement for development and promotion. The time freed up with envitabely be spent doing more of the same work (but in an automated way). Although your bosses are happy that they are getting more output from you without paying you more, without needing to fully understand the technical aspects to do the task any more you simply won't be given the opportunity to do so.
If you think programming is far more engineering than art, then you must be in a highly structured environment, program for the DoD, etc. Out here in the real world, people around me code without a shred of design work or planning of any sort. They just sit down and start typing...whatever organically evolves is what happens. Code is created with no foresight at all. I wish it were more engineering than art, though. Sad to see all the effort wasted so often.
As actually practiced, it should be called a craft, rather than art or engineering. And most developers I watch are quite Amish.
Back in the day, we coded against the metal or darned close to it. I started out writing Fortran on paper, then typing it on punch card machines. I knew my code like I knew the back of my hand. I used to say to coworkers that if I could get my software to compile, it would run just fine. I knew it that well. As automation and IDEs came onto the scene, I could crank out code like never before - and I lost track of the details. Instead of knowing the exact resource requirements of my code and my algorithms, I had to turn a blind eye to it because I had to worry about other things. How the classes relate to each other, the workflow of the user interface, standards adherence, internationalization, networking, etc, etc, etc. The applications of old were low functioning and a developer could know the entire thing. Now, applications are so complex that we can only hit the top layers of the complexity stack. Everything below that is handled by the tools. And the tools simply cannot be as intuitive as an engineer's own thought process.
I very much dislike being disconnected from the bottom layers. I have to trust that they're going to work well enough not to get in the way of my product's functionality. I have to trust that packages won't change functionality willy-nilly, that operating systems won't bloat up a feature that I'm using, and so on.
In short, the industry started screwing up from the start when it focused on enabling engineers to crank code. We don't need more code, we need less. When I'm going to build a product, I should know what foundation functionality is available to me. It should be reliable and stable, having been hammered on relentlessly by the authors before being released. Everyone should be using it so that we can have a collective understanding of how to use it. In my opinion, the fragmentation of understanding in the industry is appalling.
So the fact that we've departed from having "metal truth" and now have a vague notion of "logic truth" means that we just can't have that iron-clad notion of understanding that we once did. I think we're dumber as a result. We're not practicing the rigor of days gone by. We can't do that and still produce high-end functionality with the team sizes and in the time frames available.
Compilers are just programs.... Bugs happen.
In my nearly 30 years at this, I've only ever had to crack open the compiler/assembler once to track down a bug. Usually these "bugs" are avoided by turning off (or controlling) the automatic optimizations. I would suggest you leave the compiler debugging to the compiler experts because what *you* describe as stupid may actually have a larger purpose behind it. (Full disclosure, yes, I've written assembly code and even had to drop into assembly in the middle of a C Function from time to time.)
I've known programmers who have NEVER seen assembly and wouldn't know a "Jump To Subroutine" instruction from a "Load Program Counter" who I greatly respect. They may not be thinking bits, bytes and how there data is arranged in memory, but they can produce some pretty amazing things anyway. Don't sell somebody short, just because they don't understand something you cut your teeth on. Everybody has their own strengths.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
I would suggest you leave the compiler debugging to the compiler experts because what *you* describe as stupid may actually have a larger purpose behind it.
I spend most of my time fixing compiler bugs (when I'm lucky, I get to add new compiler features), so perhaps I'm slightly biased about compiler code quality. Most recently, I was working on an error in LLVM where functions with more than 8 integer / pointer arguments were mangled on big-endian MIPS (if they were smaller than register size), because they were spilled to the stack in the little-endian position. If you encountered this as a programmer and weren't able to look at the function call and prologue, then you'd have absolutely no idea why your code was doing seemingly random things.
I am TheRaven on Soylent News
There might be more software developers now simply because automation has lowered the bar to getting anything out the door. So you might see the average developer be "less skilled' now because less skill is required and so more people say "hey I'd like to make 100k a year too".
Someone has to make the automation tools. After that why should everyone go deep into the kernel/windowing system/whatever and understand at a really low level? Occasionally useful to know, and sometimes interesting but IMO we are paid to deliver business value not spend time learning all the details that the guy that made the tool had to figure out. Being capable of learning when it becomes necessary is far more important than what you already know.
Assembly? That's for you mollycoddled youngsters. You don't know how to really program until you've entered raw machine code via toggle switches on the front panel of a CPU you built yourself out of nothing but vacuum tubes and a spool of wire. And don't get me started on macro assemblers! You may as well be using COBOL if you need training wheels that big.
Chelloveck
I give up on debugging. From now on, SIGSEGV is a feature.
My experience with the software development industry outside of the googles and facebooks, is that only very few knows how to program. So usually, there will be a dedicated team to create frameworks and tools to be used by 90% of the team. Sure, it gives the 90% less exposure on the real engineering side of software development, but it's their choice really. For most of them, their goal is just to climb the ladder and finally become managers. Also, these guys are the ones running around talking shit and taking credit for all that little things they accomplish. Bottom line, it's the industry's fault, for only paying attention to the management, which again is composed of that 90% I mentioned.
LOL... Well, I DID do the toggle switch thing a few times, but I didn't build the system myself, nor was it vacuum tube based... It had a heck of a lot of 7400 series chips and a huge 5V supply that all came packaged in a box about the size of your fridge....
Shouldn't you be retired by now?
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
And why should I need to know how the compiler's breaking? In gcc or llvm, I can get the source code, but that doesn't mean I know how to debug the compiler. I would normally document the problem, come up with the smallest breaking code I could easily do, and send it off. (I have received the answer, "yes, it's a bug" before, and that was commercial support.)
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
First, to know that it actually is a compiler bug. You can spend days debugging your code and failing to find the error if you regard your compiler as infallible. Second, because you want it to actually be fixed. There are a few thousand bugs open in LLVM and GCC. If you want yours to be fixed, then you need the people doing bug triage to be able to work out who to assign it to. A simple reduced test case and an explanation of exactly what the compiler is doing wrong means that it is likely to be fixed the same day that it's filed. Without that... come back in a year and we might have fixed it if someone better at writing bug reports than you has filed a duplicate.
I am TheRaven on Soylent News
In my experience, compiler bugs are rare. If I run into one, I do spend days debugging my code and simplifying things until I've got a clear compiler bug. I do regard my compilers as perfect unless proven otherwise, and that's because I've found that I'm far more likely to write a bug than trigger a compiler bug. It's not that compiler bugs don't exist, it's that they almost never cause problems (or they'd already be fixed). Optimizer bugs are a special case, since different behavior between optimized and non-optimized code (unless there's undefined or unspecified behavior involved) means there's a bug, so they're usually easier to identify as compiler bugs.
Once I've got a reduced test case, I can send in a bug report. I can sometimes say what the compiler is doing wrong. I can point out variations in my test case where the bug doesn't happen. I do know something about good bug reporting.
And none of this requires source code.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes