Slashdot Mirror
Why You Should Choose Boring Technology
An anonymous reader writes Dan McKinley, a long-time Etsy engineer who now works at online payment processor Stripe, argues that the boring technology option is usually your best choice for a new project. He says, "Let's say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet. Clearly this model is approximate, but I think it helps. If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that's existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you're in trouble. ... The nice thing about boringness (so constrained) is that the capabilities of these things are well understood. But more importantly, their failure modes are well understood."
A personal corollary for code related to this theme is to always try to make the code I write as "boring" as possible. I've found that programmers often get themselves in trouble by trying to be "clever", which often makes for horribly unintuitive or unnecessarily complex systems. I've heard people asking about how to perform crazy language tricks and I nearly always think to myself "My God, why in the hell would you even *think* about doing something like that?" Such things nearly always point to very fundamental flaws elsewhere.
Irony: Agile development has too much intertia to be abandoned now.
It's the same reason that your own small company should be trying to implement its own CRM (assuming that's not its core business), or drastically changing the way that it considers sales compensation. Being revolutionary in one area is hard enough - don't make anything even harder than it has to be.
You're special forces then? That's great! I just love your olympics!
Mature technologies are proven. They've gone through their growing-pains. They may have limitations, but those limitations and workarounds are usually well known by seasoned professionals. There's a reason why COBOL, Fortran, and RPG are still in use in business applications almost sixty years after their initial development, because they reliably work.
I've tried to work with NodeJS projects for production. It's a nightmare. NodeJS itself is revised too often, the actual project is revised too often, and the dependencies became a nightmare. It's not mature enough and not worth it.
Do not look into laser with remaining eye.
In about 1996, Oracle introduced the "Oracle Webserver", allowing you to serve dynamic webpages generated from stored procedures in the database. The beauty of this is that all of your website code is in the database, making it centrally managed and all application security logic is enforced by the database. The webserver is just a dumb client with no code, and has no permissions on any database tables.
In 2001, it was now a mod for Apache and as since been opensourced (mod_owa). I convinced our client try it for a central website that we were developing, as the middle tier crap they were using didn't work. That system went live 2 weeks later with a few very simple webpages. It has been in production ever since and the website has over 50k users and 20+m hits a day.
"Be grateful for what you have. You may never know when you may lose it."
I've recently been getting more heavily into javascript (proper apps, not just UI animation snippets). The whole scene is a mess. You try to learn about good code structure and you've got crockford saying use constructs to simulate classical OO and then a whole lot of other coders giving you really good reasons to use prototypical inheritance and stop trying to simulate a classical language. Then a whole other bunch are telling you to use coffee script but others point out the difficulties debugging it so then you should use typescript because it will eventually be ecmascript6. Once you get your head around the code structuring options, you then have to decide whether to just use jquery or an mvc like ember or angular. And when you choose a shiny new bbc because it is easy you realise that two way data binding makes your code super slow and start having to hack away at it to get it working on a mobile targets. Oh and don't forget the TDD framework unless you go for BDD because that is the new thing. And then do you use the closure compiler, requirejs for amd or should you even use amd at all.
I will happily go with tried and tested, if only web developers would stop reinventing things every six months.
I think it is important enough to have atleast one 'skunk-works' type project that every developer needs to work on, just to keep up with what will be boring a year or two down the road.
I avoided "not boring" for a couple of years and for the last month, while I look at hybrid mobile apps, I am stunned by my lack of knowledge and the abundance of terms, concepts and technologies that mean nothing to me ... angular, ionic, grunt, promises, JSX, reactjs, compass, gulp, firebase... the list could go on and on and on, these are just things I've started researching over the last few weeks, to make sure I make the right choice.
Every organisation needs a "not boring" slot of time for their developers. Not for product that needs to ship NOW.. but for stuff that may need to ship next year.
It's like Rincewind said: "I like boring. It lasts."
-- Cheers!
This basically sums up things I've said over the years and thought about. You and your team have a problem to solve. Anything that gets in the way of solving that problem is the wrong solution. Learning a bunch of new tech looking for a silver bullet or because it is 'kewl' is the wrong answer. There is the learning curve to consider as well as the instability inherent in a new tool set which needs to be understood when making these decisions.
What is even more distressing is when you see bad ideas return like bad pennies, network databases are now graph databases for example. ACID compliance sacrificed for 'speed'. Having to write you own threading management code. Those immediately pop into my mind. Sure, there may be some short term gains but what about 5 years from now? After a huge investment throwing something away and replacing it becomes difficult and expensive. I am looking for the citation but I once heard that the maintenance costs of software far outstrip development costs by about 10 to one.
My advice is to first learn the problem domain which includes keeping a long term view in mind, then pick the tool set with a bias towards the tired and true workhorses. Spend less time thinking about your tools and more time solving the actual problem.
putting the 'B' in LGBTQ+
As someone who has a lot of Perl on his resume, I heartily endorse companies hiring people who work with boring old technologies!
Oddly enough, I've seen that cause more problems than it's solved.
It's often misinterpreted, in the same way you so conveniently put it, which is then used to justify some pretty awful decisions regarding third-party libraries. I'm convinced that this is the leading cause of bloated software.
Even when used correctly, I've seen some pretty impressive code-contortions to avoid even a very small amount of duplicate code. Sometimes, it's okay to just do a check twice. As long as the code is easy to read and modify, you're fine. Really.
But that's the problem with programming, isn't it? It's little more than wishful thinking and folk-wisdom. That would be okay (it's an art, after all) but too many developers have deluded themselves in to thinking it's more in-line with mathematics or engineering. They've convinced themselves the cute little acronyms they repeat to one another have some objective, rational, basis and must be obeyed at all cost.
It's silly, really.
Required reading for internet skeptics
Yes. I know this for a fact. I'm going through this right now.
Oh, this new NoSQL database is SO fast and wonderful. Except you can't get it to replicate without an Enterprise version, the price of which no one actually calculated into the cost per user budget. Because it's free and open source, except when it's not.
And you need to have at least three servers to run it even without replication. And no, I am not talking about sharding. I mean running a standard one instance of this garbage requires three instances.
Mind you, if you want NoSQL, there are versions out there that have replica sets for free and somehow manage to complete all standard operations with only one server. But we can't use those. No. They're too boring and predictable and STABLE. Not exciting at all.
Not blaming the current developers, who are left with the pile of shit that the other developers made while having all sorts of fun trying shit out and making everyone else figure out how to actually make it *work*.
The last job, they were circling the toilet bowl because we couldn't finish features. Yet their brilliant idea was to switch their technology, and even change from svn to git, and also chef to ansible. This for no discernible reason, other than the fact that it just seemed like a good idea to them at the time.
Bear in mind, I understand why you might want git or ansible or whatever new libraries, but I'd think that we could wait to redesign our whole build process and switch to a new version control system until after we had enough finished features to attract a customer. Or something.
A lot of good people lost their jobs because some architects and a certain CTO wanted to wank off about how bleeding edge they were. Of course, that dumbass still has a job. I hear they're considering a mobile app now. Or something. Good luck with making an app when you laid off all the testers.
With all that talk, you'd think I had gotten laid off. Luckily I managed to get the hell out, but all the people I worked with there got canned a few weeks later. And none of them were fuck-ups. That is the price of not understanding how to maintain a solid foundation to build a project on.
So yeah. Boring. Functional. Boring tech behind your app or site doesn't make it a bad. Bad design does that. Lack of features does that. Failure to understand operating an application on something more complex than your MacBook does that.
The fact that you using the latest thing does NOT make your app good without you knowing what the fuck you are doing.
Personally I think that one of the problems with software development is that we don't treat it like engineering enough. Not engineering in the sense of building a bridge, but engineering in the sense of design (designing a circuit for example). Engineering is inherently a pragmatic discipline where creativity is constrained by various physical, budgetary, and time constraints. Because software has less of the physical constraints, I think that the "art" side can get a little carried away at times. But the main issue is lack of discipline; and that is more of an artefact of the culture that can be associated with software development, rather than anything inherent in the work itself. I'm biased (with an engineering background), but I think that many software companies could benefit from the attitude that comes with engineering.
The same thing happens with the hope for the "next generation" product solving all the ills of the current generation. Or the assumption that the code you have inherited was created by fools a number of years back.
The reality is that software has a set of maturity related bugs and a set of structural, intractable issues that are related to the design and architecture of the system. Each piece of software has it's unique set of intractable issues.
Software that has been in production has typically reduced it's maturity related bugs. The software built on top or that integrates with it is built around those intractable issues. When you move to a new piece of software - either a new architecture or the "groundbreaking ng version of XYZ", you end up with swapping a set of *known* intractable issues, for a set of *unknown* intractable issues plus a set of maturity related bugs.
Similar to TFA, the risks of old+known vs new+immaturity+unknown needs to have another factor similar to "value-add". If the value-add *really* adds a lot of stare the risks in the eye and march forward. If the value add is marginal, make sure the meta-benefits (performance, maintainability, etc) are clear and understood, otherwise you may be facing a train wreck of an upgrade.
Seen it many times, always wary of the ngXYZ project...
> with the sole exception that all other things being equal clearly the more exciting one will pay bigger dividends due to increased developer interest.
I'm afraid that's the fine print. "All other things being equal". All other things are not equal. "Boring" technologies tend to have a competent, with documentation, load testing, and syntax checking tools. New technologies to "get the project done" often not only fail, but even when they work fail to scale, and introduce _another_ complex environment to manage.
I'd once written what I thought at the time was a fairly simple C program to interface with a medical lab system, calculate the checksum (a very, very basic CRC), and pass the results to a file. For the record, I'm a self-taught programmer who started with the Data General Nova assembler, but I digress....Several years later, I happened to meet a programmer who'd inherited my code and, basically, said that it was the best commented, self-explanatory code he'd ever seen and later changes were really easy to make. At the time, I'd just written a device driver was so feeling all punky, but later I realized that what that dude had told me was the highest compliment I could ever receive. $.02 brothers and sisters....
Case in point: Look at the religious hatred for "goto" that has no real technical basis anymore in modern languages.
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
Yes, but his brilliance was in minimising the number of components required to perform a certain function. His "art" was in solving two pragmatic problems. a) Correct function of device and b) minimising the costs (components). The constraints (physical and budgetary) are what channeled his creativity.
I'd never trade readability for "elegance" or, worse, "performance".
Code golf may be fun, but it needs to be kept very far away from everything outside those games.
Required reading for internet skeptics
You have a good point, but only most of the time.
Thomas Kuhn, in the context of science, spoke of 'normal' and 'extraordinary' science. Normal science was as you described; you stay in the paradigm and follow the conventional methods for resolving issues. However, these methods did not appear out of nowwhere; somebody was being a clever cowboy and decided its time to do things different. This is where revolutionary science came in. Of course most of these innovations - like any innovation - fails miserably. But if it wasn't for all the failures we would never have the successes that change paradigms and got us to the methods we use today.
And I think this dichotomy does apply to almost every human endeavor.
That said, for normal day to day operations, being a cowboy ALL the time is foolhardy and dangerous. But for people to NEVER have a little bit of an experimental and innovative mindset is also risky in its own way. Sometimes this is balance (known as a bimodal or barbell strategy) is maintained within a single individual by balance of exploiting the traditional and exploring the novel; sometimes its divided between individuals, with a good balance keeping more people stable than unstable.
However the wheel can still be improved. I sure wouldn't want solid wooden wheels on my bike.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
>developers have deluded themselves in to thinking it's more in-line with mathematics or engineering
Except of course that it IS mathematics. Indeed if engineering is the application of scientific knowledge to the solving of practical problems -then programming is the engineering of the science of mathematics.
From the point of view of a computer every program is just one big number. You can reproduce any and all programs that have or ever could be written by simply counting in binary for long enough.
Yes, count long enough and check the results at every count and eventually you will have a number that, if executed, is microsoft outlook.
That is, however, a rather inefficient way to find useful numbers - to get to outlook THAT way would take centuries even on the fastest computers we have.
So what is programming ? It's a sophisticated way to take shortcuts, to find useful numbers without counting through ever possible number (most of which are not useful at all - i.e. if you dump it as a binary to a file it wouldn't run).
That sophistication of figuring out what the program should do first (i.e. defining what the number should be useful for) and then counting in large chunks (i.e. writing components that help satisfy that over-all design goal) is a form of engineering.
It's a highly creative form of engineering and it is very much an artform too. Art and engineering are generally much more closely related than we usually think: just consider the Eiffel Tower, or ask any sculpture about the constraints the laws of physics place on his designs and choice of materials.
Programming, at least at it's current stage of knowledge, is still at a point where the line is extremely blurred and techniques from both art and engineering can be very valuable.
Over time we may find that it becomes more the one or the other, depending on what produces the best results the most efficiently - but I wouldn't trust any wager on which way it would ultimately go. I will say it would never be just one or the other, by it's very nature it will always have at least some elements of both.
Unicode killed the ASCII-art *
The original justifications for hating goto referred to a non-local goto (or, exceptions, as the kids call them these days) which made it very difficult to reason about control flow in a program. The new reasons for hating goto in language like C/C++ relate to variable lifetimes and making it difficult to reason about when variables go out of scope.
I am TheRaven on Soylent News
"...to get to outlook THAT way would take centuries even on the fastest computers we have".
Which would still be FAR FAR TOO SOON.
I am sure that there are many other solipsists out there.
Modern languages have much less need for "goto" than the languages back when Dijkstra wrote that letter, meaning that any use is more likely to be part of something that simply shouldn't be done. Modern languages also concern themselves more with blocks and variable lifetimes and initializations, which unstructured "goto"s work badly with. Also, modern compilers rely much more than before on being able to suss out the control flow, which non-structured use of "goto" messes up. I'd say that there's more reason not to use it than there was back then.
Do you have examples or a real technical basis for favoring "goto" nowadays?
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
In a project I am currently working on we have started using the C# goto case "state" construct.
In this case it offers us a huge performance improvement as the state machine only executes one step when it receives a heartbeat from a piece of networked equipment. The heartbeat only comes in once a second and there are instances where we can perform more than one step of the state machine without needing a response from the remote equipment.
An example of this is uploading files to the remote equipment. We use a Web Socket to establish the transfer task, but once we start sending files (HTTP POST) we don't need a response from the remote equipment between each file. Prior to the use of the goto we could only send one file every second, now we can loop through all the files by looping with the goto.
This is a specific example, but does show that there are limited instances where the use of a goto can be well justified. I also always comment every use of non-standard behavior (use of goto)
In programming there is a best possible solution to the given problem. You can maximize memory usage and speed and any of a number of other variables to come to best answer. I use a scientific approach to achieve this. I set my metrics, I run my baseline tests, make my improvements and test again. I can then quantify my results.
There is no right answer in art. There is no possible quantification on quality. Beauty is truly in the eye of the beholder.
I'd argue the opposite. Programmers who call themselves artists are just glorifying their own work. These tend to be the people who don't understand the logical nature of their work. Not that it matters, but logic was one of my strongest subjects.
Additionally, your tag line isn't a real quote.
Brilliance without wisdom, power without conscience. Ours is a world of nuclear giants and ethical infants.
The "religious hatred" for goto came from a very real problem: so-called "spaghetti code".
The older programming languages favored having all (or most) of the logic in one big source module. Use of goto to navigate around in a large module often resulted in a veritable maze of logic that was virtually impossible to make sense of or apply automated analysis to. It could get especially bad in languages that supported an assignable goto target.
Structured Programming was conceived as a solution to that. By limiting logic paths, the number of possible logic paths was reduced to more manageable proportions.
The problem with pure structured programming was that for exceptional cases, the nests of loops and conditions could themselves become unwieldy. This lead to such constructs as throw/catch where a sequence could be aborted and control transferred to a well-defined outer location. A thrown object is effectively a goto without the free-for-all that the old-fashioned vanilla gotos caused.
A related trouble source was in methods that had multiple entry and exit points. Again, it made possible logic flows difficult to predict. Most modern languages do not permit multiple entry points, and as long as there are ONLY multiple entry points or ONLY multiple exit points, the code becomes much easier to follow.
Personally, I favor a style where most of the early-exit logic is at the beginning of the function, thus filtering whether the main processing will be done. Anything that requires exit after normal processing has commenced usually rates a thrown exception.
The best possible solution would have 100% code coverage of unit tests, be 100% documented, 100% rules compliance on top of being the fastest, lowest memory consuming solution possible.
How disappointing. At least you tried. How do you know that your solution is the fastest, lowest memory solution possible? How do you know that your "best practices" are really the "best" and that there aren't better options?
Since you mention it, how do you know that the language or platform selected is capable of producing the optimal solution? You mention using Java a good bit, so you know that you're already adding a lot of overhead and thus very unlikely to produce a mythical "best possible solution" given your performance and memory requirements.
See, there is no such thing as the "best possible solution". What you're describing is an "acceptable solution" given a set of constraints, many of which (like your silly "best practices" which change with incredible frequency) are just folk-wisdom. They're not scientific, mathematical, or objective in any way. Your other constraints (memory and speed) are impossible to meet as you can't tell if there is a faster solution that uses less memory.
Instead of trying to win an argument here, reflect a bit. Particularly on how readability factors in to your above criteria. Are those criteria subjective or objective? Can they be measured? Are you really measuring what you think you're measuring or do your methods merely quantify something you believe to be related? On things like memory and cycles, how do you know that they are as low as possible and that there aren't different approaches that you could use to minimize those?
Further, when you trade memory for speed or speed for readability how do you know that your decision will lead you closer to this alleged "best possible solution"?
Required reading for internet skeptics