Slashdot Mirror
Invented-Here Syndrome
edA-qa writes: Are you afraid to write code? Does the thought linger in your brain that somewhere out there somebody has already done this? Do you find yourself trapped in an analysis cycle where nothing is getting done? Is your product mutating to accommodate third party components? If yes, then perhaps you are suffering from invented-here syndrome.
Most of use are aware of not-invented-here syndrome, but the opposite problem is perhaps equally troublesome. We can get stuck in the mindset that there must be a product, library, or code sample, that already does what we want. Instead of just writing the code we need a lot of effort is spent testing out modules and trying to accommodate our own code. At some point we need to just say, 'stop!', and write the code ourselves.
Most of use are aware of not-invented-here syndrome, but the opposite problem is perhaps equally troublesome. We can get stuck in the mindset that there must be a product, library, or code sample, that already does what we want. Instead of just writing the code we need a lot of effort is spent testing out modules and trying to accommodate our own code. At some point we need to just say, 'stop!', and write the code ourselves.
One example that really drove me crazy was a developer who was tasked with putting in a 'retry' mechanic. Rather than simply working with the socket he pulled in some spring library that altered the socket behavior according to an external XML file and was praised by the lead for doing it this way. The code (java + XML) was several times longer than it would have been otherwise, plus a new library had to be included in the build/distribution, but it was still 'proper'. I think that was the breaking point for me working on that project.
What do you do when some complex, external code has a bug ?
Do you want to keep up with the world around you?
I was having a discussion with a programmer who is 50 (I'm 38) the other day about the time when you used to be able to write an OS entirely by yourself, and how we both miss that time. It wasn't trivial, but the OS was that small, it could be done. Remember, Linux is just a kernel, not 'an OS', and Linux hasn't been the work of one make for 20 years at this point (Not trying to discount what Linus has done, but he has help :).
Today, if you want to be able to produce useful software with sufficient features for most purposes, you can't write it all yourself. Well you can, but you'll be the last person to bring your 'whatever' to market/public view and they'll be 20 others that have more features and more shiny than you because they relied on some other libraries. A single person isn't making Windows 3.1 or newer in any realisticly useful time frame. Windows 95 is well beyond the scope of one person.
Even a good modern text editor isn't going to be written from scratch, you've just not got the time to write all the basic editor features and things like a regexp engine. Yes, you CAN, but not while staying relevant.
Sure, there are places where you can get by without dependancies or MUST have no dependencies. Embedded work and Cryptography are two things I have experience with where re-implementing the wheel isn't uncommon due to various constraints placed on you. Note: Not reinventing, reimplement
With that said ... I'm writing an OS by myself. It will never do anything impressive and no one will ever use it, but it will exist and has already taught me why I hate the x86 processor line in about 18,000 different ways :)
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
You will always have to write some code of your own. Even if you use a CodeIgniter, AngularJS, and every prewritten function on StackExchange, still, you will have to write some code to configure the frameworks and to pull it all together.
You will always use some of someone else's code, too. Aren't you using Linux or something? You didn't write your own OS, did you? You're using a database, like MySQL or PostgreSQL or something, right? You didn't write your own database system, did you? And are you using a web server like Apache or Nginx?
So the question is not, should I write my own code or use someone else's. The question is where to draw the line.
I'm a web programmer, your typical LAMP developer (well, LAPP --- I use PostgreSQL). Like many PHP programmers, I first concentrated too much on the PHP. PHP is not the best language, as many have said, but I don't think it's quite as bad as people make it out to be. Anyway, I never took up any of the PHP frameworks. They seemed like too much trouble to adapt. (I should point out that I started with an intranet with a dozen or so applications already built.) I would research PHP frameworks from time to time, but always rejected them all, and felt a little self-doubt in doing so: "Do I suffer from Not-Invented-Here Syndrome?"
But PHP, and scripting languages in general, provide the right level of abstraction, I think. It takes care of memory management. It provides a bunch of functions for HTTP. It has its own templating syntax. It's great if you don't overuse it. In other words, in the MVC pattern, PHP does great for the View and, together with Apache, the Controller. But if you write a lot of your Model in PHP, with all this data processing, checking, calculations, etc. --- well, that's what the database is for, I think.
So, instead of eventually adopting a PHP framework, I learned more and more about Apache and PostgreSQL, and I learned that a lot of the things that I was doing in PHP could be done in SQL or in the Apache configuration, with a lot less typing (a lot more reading but a lot less typing). While most people are busy trying figure out how to write the practically all of the MVC in PHP, I realized that Apache was part of the Controller, PostgreSQL was part of the Model, and the browser was part of the View. I use PHP just to help them out, only when needed.
What if the thing you write that substitutes for the missing external dependency has ten thousand times as many bugs?
People always underestimate the investment in software development. I'm sure most people think someone pulls up a chair, types some code, puts it on a floppy disk and KAPOW it's on the market in every device. Nope. We have testing teams, build specialists, cross-platform planning, packaging requirements, release processes, A/B testing, refactoring, branching, madness.
Any time you write a piece of code there are leagues of people involved who will have to deal with your code, whether you know who they are or not (and most of them you won't know). The problem with code is that we believe we can handle it and there'll be no problem. The code will just work first time for everyone, everywhere, on every platform. We think we're rockstars and our guitars are out of tune just a little.
Having access to the code can be as dangerous as *not* having access to the code for a naive or time-pressured programmer. We might know not to put business logic in the GUI, but the 19yr old new hire doesn't necessarily know that yet. Tomorrow you're hiring a 14yr old and the day after that it's going to be some 8yr old girl-wonder from India who aced the Microsoft certification. The software industry skews young and the first thing that gets cut is rigorous testing process and quality control. As an older programmer I now spend the majority of my time cleaning up the broken crap left behind by the last pack of fly-by-night rockstars.
Get off my lawn.
I've told this story elsewhere, but it applies directly to this issue, so I'll recap in short:
Vender is contracted to create an integrated support application for large sums of money ($millions) over a 6 month period; contractor chooses an obscure commercial java framework to build the system on. The application is delivered and appears to work fine for several months, then starts getting sluggish, then a month later the application locks up - and has to be restarted. This progressively gets worse, and is asymptotic with the growth of the underlying customer base - and soon becomes completely useless - shutting down within minutes of being started with a memory exhaustion error.
The main problem we found was the equivalent of a memory leak in Java. The code would instantiate objects based upon the framework in the main loop, and they would never go out of scope. Furthermore the code imported hundreds of libraries that were never used - further impacting clarity and understanding of what the thing was doing.
To make a long story short, since this was already in production and now there was even more pressure to get a solution in place fast (and all the lawyers threats in the world can't replace a knowledgeable developer) - we rebuilt the whole system using perl in a little over 1 week. That solution is still running today - even as we've scaled orders of magnitude since then.
So - to your point - this stuff really does happen, and wastes godawful amounts of time and money, when a more simpler home grown solution would do just as well, if not better.
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain