Slashdot Mirror
How One Dev Broke Node and Thousands of Projects In 11 Lines of JavaScript (theregister.co.uk)
An anonymous reader quotes an article written by Chris Williams for The Register: Programmers were left staring at broken builds and failed installations on Tuesday after someone toppled the Jenga tower of JavaScript. A couple of hours ago, Azer Koculu unpublished more than 250 of his modules from NPM, which is a popular package manager used by JavaScript projects to install dependencies. Koculu yanked his source code because, we're told, one of the modules was called Kik and that apparently attracted the attention of lawyers representing the instant-messaging app of the same name. According to Koculu, Kik's briefs told him to take down the module, he refused, so the lawyers went to NPM's admins claiming brand infringement. When NPM took Kik away from the developer, he was furious and unpublished all of his NPM-managed modules. 'This situation made me realize that NPM is someone's private land where corporate is more powerful than the people, and I do open source because Power To The People,' Koculu blogged. Unfortunately, one of those dependencies was left-pad. It pads out the lefthand-side of strings with zeroes or spaces. And thousands of projects including Node and Babel relied on it. With left-pad removed from NPM, these applications and widely used bits of open-source infrastructure were unable to obtain the dependency, and thus fell over.
See this is the exact scenario I've talked about before, typically when learning a new coding language. Dependancies and libraries are BAD. 1: because they remove the need to understand what's actually going on (premade functions for example) 2: because some asshat could yank them away and leave your project broken, then YOU have to figure out how to replace the dependencies. I get not reinventing the wheel. But you really need to pay attention to what your code is dependent on and remove that dependency if possible
So, what have we learned?
External dependencies are unsustainable;
JavaScript is unmaintainable;
Dozens of mainstream projects relying on a trivial bit of string padding code from an external JavaScript dependency is unconscionable.
This is just hilarious. What a shit-show, from the bullshit legal threat to the developer's hissy fit to the dependence on an apparently obscure package to implement (lol) left-padding.
Reminds me of someone I knew who was wringing his hands for a few days over which license to use for his super-awesome R function library. He asked me for advice, and I told him that it's ~30 lines of syntactic boiler-plate code so get over yourself and just put it in public domain so that the two people who ever use it can do so easily. But of course, he had to deeply consider the political implications of which flavor of "freedom" he would support.
"They were pure niggers." – Noam Chomsky
Don't know who they are or what they do, but fuck them and boycott whatever it is they sell.
Those who do not learn from commit history are doomed to regress it.
Just kidding, I have no problem with Javascript. By the way, that summary was confusing as hell.
One of the beauties of JS is that it's easy to provide your own functions, so as long as it's only left-pad missing, you could provide your own, right?
function left_pad(str, min_length, pad_char){
if(str.length min_length){
str = Array(min_length - str.length).join(pad_char) + str;
}
return str;
}
(note, I did not do any sanity/error checking in the function, so do not simply copy/paste, please fill it out if you intend to use it)
Everything is one letter away of meaning something in some language.
Roll your own libraries. No outside dependencies, and you'll probably leave out a lot of the cruft that is there "because."
"Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
Also, when was it made, originally?
Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
Why would you ever build an app that assumed the perpetual existence AND availability of remote, opensource, Internet-hosted code?
the Kik application referenced has iOS, Android, and Windows Phone implementations all rely on node.js for both overt client side as well as server side processing.
Good people go to bed earlier.
Dependencies are unavoidable, specially on big projects - you are not expected to reinvent the wheel every time you code.
Now, having a dedicated library dependency for padding strings is a bit of a stretch though...
Installed Babel. Strange Error messages and babel borked and unusable. Same problem popping up all over the interweb.
Sad. Wanted to start with classes in JS. :-(
Does anyone know when this gets fixed and what the plan is?
We suffer more in our imagination than in reality. - Seneca
But it illustrates a key lesson for open source. So much of the project is not just code, it's governance and culture and how to make smart decisions under pressure in a way that respects the people involved. Node failed to do that, the guy pulled his code, they learned a harsh lesson. Let's see what the post-mortem letter is like to see if they really learned what they needed to.
Kik is one letter short of being kike (a racist name for a Jewish person). How brilliant.
It's also one letter short of being "kick" (an racist action to be performed on a Jewish person). How brilliant.
I've always thought this interconnected pile of stuff, linking across a bunch of domains was lazy, dangerous, and likely to be very brittle.
Sorry, but the interwebs have shown me I can't afford to trust arbitrary code from all over the place, which can change at a moments notice, and which I know nothing about.
If you've created an infrastructure where tons of stuff breaks because some asshole corporation forces some guy to say "fuck you, you can't have my code", you have a terrible mess. What happens if someone adds some malicious code?
What I find really odd is they've over-ruled him and said "no, you can't un-publish your own stuff, we own it". So, what, they've decided his stuff was too important to still be his own? So he got fucked because of corporate assholes only to have his copyright infringed?
Jenga tower indeed, it sounds like the state of the art is a bunch of brittle dependencies controlled by a few places, and subject to causing a shit top of things to happen when someone makes a change.
This reminds me of a company I worked at which had a universal build system ... everything build from scratch every day and wouldn't build if any of its dependencies didn't build. So when some guy broke a components 3 components upstream, nobody could get anything compiled because the system was too stupid to go with the last known good ... and hundreds of developers sat around all day going "but, what do you mean we can't do anything because some guy checked in shit code".
Wow, just wow.
Steaming Heaps of Innovative Technology.
Lost at C:>. Found at C.
Bullshit.
Dependencies and libraries are fine. If you want to use SSL encryption in your software you should try and re-implement it all, because while SSL libraries have been having security issues in the last few years there's almost zero chance you make something that doesn't have bigger ones. Multiply that by all the domain specific pieces of code you need.
Not having to understand what's actually going on is a feature not a problem.
What is retarded is using a dependency that exists out in the internet somewhere only, so that your code breaks if some random internet service disappears. As long as you have your own copy of the dependency it's fine. Some asshat can't yank them away since you have your own copy that doesn't magically stop working. Obviously you want the source code, or at the very least the source code escrowed somewhere that you obtain if the provider vanishes in the future.
I don't think this proves libraries per-se are bad, but blindly depending on out-of-house software repositories for every build or (worse) every startup/deploy is crazy.
We're a Java house (go ahead, get your ki[c]ks in...), and we use around 100 open source libraries. They're all mirrored locally in an Ivy repository we maintain. No new versions get checked in unless they've passed a reasonable level of scrutiny, and nothing gets deleted unless we delete it.
This is what can happen when you use Other People's Code.
The more a project says "requires" something other than the language it's written in, you're making the risks worse.
The better the programmer, the less OPC they will use.
The best programmers are known by the announcement in their projects that their code was black box and has no external project dependencies. If you must use OPC, you should be looking hard for such a statement.
Of course, today, most "programmers" aren't deserving of the name in the first place. Glorified scriptkiddies at best.
I guess? Every language with a package manager (like Python's pip) is "vulnerable" to someone pulling the rug on a project.
Damn bossy underwear!!!
It is 2016. You need to use libraries. You just copy them to your local repo so they are in a known state if you use them.
Does anyone know when this gets fixed and what the plan is?
You could try to read the article.
To fix the internet, Laurie Voss, CTO and cofounder of NPM, took the "unprecedented" step of restoring the unpublished left-pad 0.0.3 that apps required. Normally, when a particular version is unpublished, it's gone and cannot be restored. Now NPM has forcibly resurrected that particular version to keep everyone's stuff building and running as expected.
And your nick is one character away from including the word "reamer", and two letters away from being "screamer".
It is simply not possible to exclude every word which is one or two letters away from offending some random idiot who thinks being one letter away is the same thing.
People can give something a name which is totally innocuous to them and which someone else is going to get into a hissy fit about.
So, what's more likely, he had no idea some random guy on the interwebs would make the comparison to a racial slur, or he used a word which sounded cool to him?
Lost at C:>. Found at C.
Dependencies are unavoidable, true, but it's a good practice to vet libraries before you include them and make sure they're worth the risk.
The risk with Open Source libraries is usually long term maintenance - will the project be maintained for the lifetime of my project? If the project is not maintained, is it something you're willing to take on and will the license and community let you fork it? If not, there's risk.
You also have the risk we see here - ideology trumping pragmatism (that's a nice way of saying the developer had a fit and took his toys away for reasons even he doesn't fully understand). In those cases, you also have the chance to fork, assuming the license/community allows it.
For trivial operations, such as left padding, it's almost never worth the risk to use a library unless it's part of the language's standard library. Sure, don't write your own FFT or ORM, but if you can code a function in a few minutes for a well defined problem, there's no reason to add an external dependency.
-Chris
From what I've noticed in Python, the most important packages tend to be available in a distro's package manager anyway. But packages (in both Python and node) can manually un-break such dependency issues by directly specifying the repo location of the package rather than relying on pip/npm.
If you write *anything* that assumes the perpetual existence of a linked library from somewhere on the internet, you deserve what you get.
Most kids these days don't remember a time where internet access required a dial-up modem -- or it wasn't a 100% certainty it would be available.
If telephones are outlawed, then only outlaws will have telephones.
Nonsense. Laughable, even. Quality programmers can build anything. If they're wise, they will.
I guess you never worked with code then, eh? It is not about whether they can or cannot - is about the resources involved. Writing and testing a production-level software library is not a trivial task. It takes time.
Again, this does not apply to string padding, which is usually resolved on a couple lines of code on the language of your choice...
It's fucking unbelievable how much trouble JavaScript has caused for so many people.
Let's ignore how fundamentally broken it is, as a programming language, in almost every respect. That includes its fucking awful type system, its total lack of real OO (sorry, prototypes are complete shit), its ultra shitty standard library (which is why NPM and this problem exist in the first place), and similar problems.
JavaScript has allowed too many unskilled cranks to shit out way too much broken code. It was one thing when they did it client-side, where it was isolated. Now it's being done server-side, and it's a motherfucking disaster!
Worse, JavaScript has enabled the web advertising industry. JavaScript makes it trivial for them to track your every move online. If you don't want to fall victim to it, then you have to waste your time disabling it everywhere by default, and selectively enabling it where you need it.
JavaScript needs to go.
If you really need to use a scripting language server-side, use Lua, or Python, or even goddamn Tcl. All three of them are better than JavaScript in every way.
It also prevents versionitis: where the package didn't go away, but was changed in such a way that it no longer works the way it used to.
Your stance may not be "popular", but it is 100% correct — and very smart.
We can still be hosed by irresponsible changes in the underlying language, and/or irresponsible changes in the underlying OS (if there is one... not always the case.)
Python and Perl have both outright broken older code that was designed to the language spec. Windows and OS X have both broken APIs that were used properly to spec. I'm sure the lists are much, much longer than that -- those are just the cases I'm personally aware of.
And we should take very seriously the idiot "X has been deprecated" warnings in a language or an OS API, because that means some lame-ass bonehead is thinking about doing that very thing to us. Javascript, c libraries, OS APIs...
I've fallen off your lawn, and I can't get up.
Building on the work of others is what open source is about. Heck, it's what proprietary code is about too.
I think what we're looking at here are community rules of which are too permissive of unsharing without cause. It's one thing to unshare for legal reasons but the extent of unsharing should be strictly limited to the affected modules. The deal should be that if you encourage people to depend on your car work you don't get to take it away without good reason.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
You can't just take hundreds of man-years of Ph.D level work and dump it into the public domain.
Is like no one ever told him he should peddle the work to journals to keep the grants flowing. That's how it goes, see, who is gonna pay up when the payers no long get paid?
Did you read even the synopsis? He had a module named the same as a Instant Messaging program. The IM program lawyers insisted that that module be pulled (it didn't do IM). When it was pulled he got frustrated and pulled everything he did from the repository. He didn't "dump it into the public domain". He chose a name that someone else had used.
You have enemies? Good. That means you've stood up for something, sometime in your life.
And your nick is one character away from including the word "reamer", and two letters away from being "screamer".
I got the nickname "Reamer the Screamer" in the engineering class at junior high school. My model car required a larger opening to fit the CO2 cartridge ("reamer") and it whistled down the string ("screamer"). I never liked that nickname as it fit the reputations of several girls quite well.
Wise people and quality programmers don't waste their time reimplementing functionality that has been written, improved, reviewed, and tested by a large number of people.
Ironically, it may be Kik's attorneys that acted improperly here. Trademark law allows similar names to be reused for different fields of use, so long as there is not a possibility of confusion/loss of market. Here I seriously doubt that anyone would confuse a Javascript module with a chat application. So quite possibly this was a bogus assertion in the first place, which ended up causing serious damage to a lot of folks.
It's a bit like asking the bum in the alleyway to hold the keys to your business for you over nights, instead of carrying them in your own pocket.
I follow the development of Signal (https://github.com/WhisperSystems/Signal-Android) and its fork SMSSecure (https://github.com/SMSSecure/SMSSecure). They had a similar problem too, where the developer of material-dialogs decided to remove all old versions of his library after an interface change, resulting in breaking builds (https://github.com/WhisperSystems/Signal-Android/issues/4138). Both projects solved it initially by hosting their own version, and then remove the library completely.
That's why I host all my dependencies myself, per project and on all my projects.
Special font? Self-hosted.
jQuery? Self-hosted.
CSS Toolkit? Self-hosted.
Massive monster webapp lib (like Googles Polymer)? Download, adjust URLs, move to project subdir, host yourself.
Some other lib? Downloaded, stashed and hosted in the project too.
Dependencies are fine, but should always have them under your control.
I'd do the same with binary code.
This is, btw., one of the big problems with many Linux programms.
We suffer more in our imagination than in reality. - Seneca
Thou shalt always mirror your dependencies. Never assume that everything will always be available. That's continuous integration 101.
Second paradigm: mirror even your dependencies source code, if you can.
Stupidity is the root of all evil.
and Azer's unpublished code, along with desiring it not be hosted @NPM on github.
uh oh.
He could have updated the module to delete and format the contents of every machine it was run on. I'm kind of surprised this hasn't happened before considering how many modern environments have such slapdash dependency systems. At the very least a packaging system should by default generate and use a lock file which contains a version and a hash of the dependent package. Npm supports a "shrinkwrap" flag but it should be the default.
No everyone. The smart people create their own branches. That is what git is for.
I work with lots of code. I include 3rd party components. However, I am not dependent upon any external sources for that third party code. IOW, you can have dependencies, just make sure you own the servicing of said dependencies internally, and not some unknown 3rd party. It's even better if you have the source for all third party dependencies and build the artifacts yourself. Yes, this takes a little more time, but it significantly lowers the potential problems you may encounter later, plus your codebase will be guaranteed repeatable builds, which cannot be stated for the lazy approach.
The cesspool just got a check and balance.
Yes, this takes a little more time, but it significantly lowers the potential problems you may encounter later, plus your codebase will be guaranteed repeatable builds, which cannot be stated for the lazy approach.
Much agreed. And sadly, this is not the node.js way...
What a crock of shit. Do this and you increase your development time 1000-fold. It's stupidity to think that just because someone uses a dependency that they're incapable of understanding it and it blows my mind that this absolute garbage has been voted up. No sane developer would tell you this.
BeauHD. Worst editor since kdawson.
WTF!
The coder did what was totally normal for a coder. Just enforce is moral rights. The stuff every authors should defend because that is why our income are that high compared to manual laborers!
Everyone out of JS told them that there was a problem with DEPENDENCY hell.
They said no. The problem is unsound technical practices where basically the assumption that all will go well is made to build everything.
The removal of a module was expectable like a lot of other things still bound to happen. But JS community did not cared to protect for such a small potential problem.
The problem is never someone doing what he is entitled to. It is people using code without understanding licenses and taking stupid risks.
"Dependencies are unavoidable, true, but it's a good practice to vet libraries before you include them and make sure they're worth the risk."
But that's not what is being said. What is being said is "dependencies are bad, you shouldn't have them" which is just ridiculous ideological bullshit. If nobody used dependencies half the software today wouldn't exist in any stable form because people would still be testing and debugging their newly-invented wheels.
BeauHD. Worst editor since kdawson.
It's fucking unbelievable how much trouble plumbing has caused for so many people.
Let's ignore how fundamentally broken it is, as a technology, in almost every respect. That includes its fucking awful historical association with toxic lead, its total lack of real modularity (sorry, reservoirs are complete shit), its ultra shitty set of mutually incompatible pipe sizes, materials, and connections (which is this problem exist in the first place), and similar problems.
Plumbing has allowed too many unskilled cranks to shit out way too many leaky pipes. It was one thing when they did it in Ancient Rome, where it was isolated to a fountain in the town square. Now it's being done in people's houses, and it's a motherfucking disaster!
Worse, plumbing has enabled the for-profit water supply industry. Plumbing makes it trivial for them to track every drop of water you use. If you don't want to fall victim to it, then you have to waste your time turning off a bunch of valves, and digging wells everywhere.
Plumbing needs to go.
If you really need to use a water delivery technology, use a river, or a pond, or even a goddamn barrel . All three of them are better than plumbing in every way.
No, it's really nothing like that.
BeauHD. Worst editor since kdawson.
This. Exactly this. Someone needs to allow your score to go to 6 for combating the stupidity around here.
BeauHD. Worst editor since kdawson.
I agree - if you use libraries then provide them with your stuff since otherwise you may suffer from the library getting updated and your package crashing for someone else that uses a library that's too new.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
Serious question, guys. Why do people use NPM or other dependency managers in the first place? Each and every language seems to have their own different dependency manager with its own quirks and problems, such as the one described in TFA. In my company, we just use git with submodules for dependencies. This allows us to easily pull in dependencies regardless of programming language used, or which online git repository their in,our own or open source. Since we're already using git to manage our own source code, this just made perfect sense from day-one, using a single tool to manage all of the source code. So, seriously, what's so great about fragmenting to multiple tools that all do the same job, only for different programming languages, when there is already a centralized tool that we're already using (git) along with these other tools (NPM or otherwise)? Why not just drop these other tools entirely, and avoid the issues mentioned?
Wait, what? A package manager has a CTO? Why is there a SPF in the Javascript world? In the Java world, you would just add an additional repository to your Maven pom.xml and move on. (Or even better, you would already have had your own Artifactory listed, with all your required libraries mirrored there.)
Just one more reason to hate dumbshit "hip" project names instead of actual descriptive names.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
Number one rule of programming is you never rely on external libraries being hosted somewhere else. You can't rely on those libraries being available for the lifetime of the project. It is the same using for example Maven to rely on external libraries. This is something you should never do. Always have the libraries local.
What an absurd statement. You only need to understand the implementation details if you have the responsibility of changing or fixing something - the whole basis of high(er) level languages is that you don't need to understand how absolutely everything works in order to use it; and the same applies to third party libraries.
Depends on your definition of "mastery". A good programmer with 6 months of experience is on average just as good as someone with 10+ years of experience. The reason for this is the average programmer will never reach mastery no matter how much time they spend. Most people don't get 10 years of experience, they repeat the first year of experience 10 times.
Stuff you don't want to reinvent is security, datetime, or fundamental libraries. And never use any code that you don't understand how it works. Not at the operation level, but characteristics and edge cases. I can't tell you how many times I've seen people unable to debug their issues because of some complex interaction between some "free library" that is popular. Rule of thumb. If someone contrives a hypothetical case, you should be able to tell them how your code will work. If you can't answer the question, then you didn't program, you threw code at a wall and it passed some crappy unit tests.
Yes, it's the Hipster False Switcheroo Fallacy. Also known as satire.
"Mr. Swift, cease this frippery at once! I fail to see what infant cannibalism has to do with wealth inequality and class relations in Ireland!"
And how dare you, sir or madam or other. I'll have you know I've put countless man-or-woman-or-other-hours into my artisanal gluten-free bread.
Also, how dare you, Slashdot! Putting a squiggly red underline under "artisanal" in 2016. This is disappointingly regressive!
Try writing an application that handles all the common image file formats, (at least JPEG, GIF, PNG and BMP) and get back to us.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Kik is one letter short of being kike (a racist name for a Jewish person). How brilliant.
It's pronounced "kick" (which is also one letter away). This is how companies create trademarks these days. They take a normal, everyday word, misspell it, then trademark it. Let's see.. "SyFy", "Cuore", "Stihl" (an original!), "Lite", "Lync", "Google" even (the spelling of the big number is "googol").
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
Actually, that's a pretty good analogy.
You shouldn't trust Joe Developer to never make some change that breaks one of your dependencies.
Mirror your dependencies, and pull changes in from outside only when it's appropriate to do so.
Nah, if it's random, there's a chance you might get two vowels in a single word.
We do the same thing, except with maven and artifactory. I thought it was dependency 101. Must have missed some sort of memo.
How about this? Dependencies are bad; you should avoided them.
You can apply that to third-party code as well as code you've written yourself. GIS for "dependency graph" if you'd like to see why you see this "rule" posted so frequently.
Required reading for internet skeptics
And there is also 3: Many libraries are of bad quality, often not readily obvious so. Frameworks are a worst offender here. The only thing you can safely accept from libraries is encapsulated functionality with clean interfaces that you can re-implement if necessary (which also addresses your first point: Never ever use libraries where you do not really understand what they are doing and were you would be unable to code a reasonable replacement yourself). You know, like you would ordinarily divide a larger project anyways.
Of course, a basically semi- to incompetent community makes things worse. Using a function like "leftpad" in so many places indicate people that take shorter to find it than to code it themselves. That requires a serious lack of skill.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Fatally wrong. If you are incompetent, then you "need to", otherwise it is a convenience and a trade-off that you need to understand. I do agree that currently most "developers" fall under "incompetent" though, so most of them actually need to use library, but they also would have needed to never get hired in the first place. (Yes, one of the things I do is clean up behind people that cannot code even basic things right and that write "enterprise" software. "Incompetent" is the industry standard for coders these days.)
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
That this even has to be mentioned (how stupid do you have to be to _not_ do this?) exemplifies the larger problem nicely: The people that got hit have no clue how to write code professionally.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
It's one thing to say if I need to use SSL encryption, or some other sure-to-have-been-developed-already function, I should use a library.
But it seems more and more that library developers suck at the fundamentals of API development, and will indeed import some whiz-bango 3rd, 4th, and 5th-party libraries, each for one tiny function. And the extra libraries will always called something like sheboyganMarmoset.
The same holds for applications and package managers. How many RPMs in RHEL carry a dependency on ModemManager, despite the fact the virtually no one still uses modems? Installing one RPM often carries 50 cascaded dependencies on far-flung libraries or applications.
It is true that dependencies are unavoidable, but I have seen things reach the point where to build something, the build tools will have to hoover the Internet and download a bunch of gunk. What is it used for? You may know why your direct dependencies are present, but those dependencies may have other dependencies and it is all opaque as to why they are there.
What is the code quality for each module? Honestly you have no way of knowing with certainty. With something you purchased you might have higher expectations than for something that unknown individuals on the Internet might have written, but even that's no guarantee. I would bet that most people don't look at the source nor do they try and understand what the module does. They just accept it as another black box that their application needs and move on after letting the tooling download the binaries. And you might pay attention to reported bugs and/or security issues reported in modules that are your direct dependencies, but the dependencies of dependencies are another matter. There could be horrible bugs buried in there that you know nothing about.
And then you have version-hell issues. You might need version 1.2.5 of some package, but something you depend on uses version 1.2.6. And something else you depend on uses version 1.2.7. Some things like Felix try and keep things straight for you, but it just seems like laziness to me that people accept this sort of thing as normal.
I make the analogy between the software dependency tree and the public park. Hundreds of people use it, and walk their dogs, and clean up after them, but it only takes one dog owner who doesn't to stop you and your kids from rolling around in the grass. Unless your dependency tree is locked down completely, you're just waiting for the one piece of s**t to ruin it. And laughing at node while using maven or APT or any other public repo system is hypocrisy.
It looks like the comment is moderated funny, and I agree.
Don't look now, but the hipster might be you.
"Our two-party system is like a bowl of shit looking at itself in a mirror." - Lewis Black
For trivial operations, such as left padding, it's almost never worth the risk to use a library unless it's part of the language's standard library. Sure, don't write your own FFT or ORM, but if you can code a function in a few minutes for a well defined problem, there's no reason to add an external dependency.
Generally I've found that anything taking less than two days (one day for writing, one day for testing) is worth rewriting yourself instead of adding a dependency (arguably, anything taking less than a week is worth rewriting, depending on the quality and stability of the dependency).
"First they came for the slanderers and i said nothing."
FYI, to give an idea of how long that would take, I did that a while ago for GIF, and between understanding the documentation and writing the code and debugging, it took 40 hours (my original estimate was ~8 hours ha!). So extrapolating based on that, the time required for the total collection would be 160 hours, pad it up to 200 hours to account for complications. Given the relative stability of the image libraries, it's unlikely to be worth re-implementing them.
"First they came for the slanderers and i said nothing."
If you are competent and not independently-wealthy, you need to. I suspect that you are neither wealthy, nor a professional developer. Probably not a software developer at all, really.
I don't think this proves libraries per-se are bad, but blindly depending on out-of-house software repositories for every build or (worse) every startup/deploy is crazy.
We're a Java house (go ahead, get your ki[c]ks in...), and we use around 100 open source libraries. They're all mirrored locally in an Ivy repository we maintain. No new versions get checked in unless they've passed a reasonable level of scrutiny, and nothing gets deleted unless we delete it.
You beat me to it! I work in a Java/Maven house and we use Nexus to maintain an internal mirror of the open source libraries we use from Maven Central. The developers pull their build dependencies from the internal mirror not the Internet. In addition, we also mirror the source code for the libraries we use as we submit patches and enhancements for them.
The initial programmer didn't respond professionally; neither did NPM.
This was a cease-and-desist letter over a trademark. The programmer's public statement about the guy being a patent lawyer, even if it's true, it's irrelevant.
All they had to do was either (1) have a lawyer send back a letter saying there was no likelihood of confusion and nobody in their right mind was going to think a node module was an instant messaging app and the like, or (2) change the name--did they even have a lawyer call back *explain* the problem with a name change and ask the Trademark holder to let them mark it as deprecated for a year? Or (if they cannot afford an hour or three from a lawyer) do it themselves?
And when withdrawing his packages, the programmer should have been responsible to the open source community and, again, marked packages as deprecated for a period of time before withdrawing them. This was just irresponsible.
I just did the deep dive on NodeJS, Javascript libraries, and build tools in general.... my thoughts are "run.. run away". It reminds me of the early days of Smalltalk, except less organized and completely without discipline. And that's stating it in nice terms. Unfortunately, for some components I need, I'll be using some of these tools because I'm not rewriting them from scratch, although I am attempting to influence some library maintainers to incorporate some bug fixes that would help me. I'd rather have that than me maintaining those fixes.
The cesspool just got a check and balance.
it's unlikely to be worth re-implementing them
Very true.
I am TheRaven on Soylent News
It's fucking unbelievable how much trouble people interacting with other people on the internet has caused for so many people.
Let's ignore how fundamentally broken it is, as a technology, in almost every respect. That includes its fucking awful historical association with trolls, its total lack of real insight (sorry, Anonymous' are opinions complete shit), its ultra shitty set of mutually incompatible ideas, memes, and non sequitur invective (WHICH IS THIS SHOUT SHOUTY SHO), and similar problems.
Typing stuff on the Internet has allowed too many unskilled cranks to shit out way too many words. It was one thing when they did it in Ancient Rome, Cicero or Julius Caesar #vinividivici. Awesome. Now it's being done in people's basements, and it's a motherfucking disaster!
Worse, communication has enabled the for-profit media industry. Google makes it trivial for them to track every word you type. If you don't want to fall victim to it, then you have to use a VPN or anonymous mode or TOR or something, I don't know who cares? Just give me my groupon, OK?
Forums need to go.
If you really need to use an idea delivery technology, use a letter, or a parchment, or even a goddamn cave painting . All three of them are better than Internet in every way.
What is being said is "dependencies are bad, you shouldn't have them"
It's true. It's also true to say 'reinventing the wheel is bad, you shouldn't do it'. Eventually, you have to pick the lesser of two evils, but neither solution is particularly enjoyable.
I am TheRaven on Soylent News
Yeah, it was satire. No, satire doesn't need to meet your checklist.
Oh, and most existing image libraries have been reasonably well fuzz-tested recently and had hundreds of security holes fixed, because parsing binary formats in C without introducing exploits turns out to be hard.
If you want to avoid security holes you need to have the security mindset from the beginning, thinking about how to avoid security holes. You can't 'reasonably well fuzz-test' a project like that......if you've fixed 100 security holes, then most likely there were more than 100 and you've missed some. Having worked with the source code for libjpeg and libpng, I am certain I can write more secure code if that is the goal. I would probably double the time estimate, though.
Incidentally, glibc is another library that worries me for security because of its ubiquitous nature.
"First they came for the slanderers and i said nothing."
If you are competent and not independently-wealthy, you need to.
I do not think you understand what need to means.
The GP is correct. If you are competent, you do not need to use libraries. Libraries are a convenience.
Ya totally just cemented owning that laughable nickname by your own compete lack of understanding and usage of it.
I understand perfectly what it meant. But people who give me a negative nickname become uncomfortable when I take ownership of the nickname, turn it around and wear it as a badge of honor. I used to be called "Tortuga" (Spanish for turtle) when I worked in a restaurant. I got removed from working the line as captain after a month because I worked the Latinos too hard and too fast.
Yeah, except Stihl was the founder's name; it's also a German company so no relation to the English word.
Yeah, except Stihl was the founder's name; it's also a German company so no relation to the English word.
Can't be German. It's too short. :)
I remember the controversy about the WingDings TrueType font that Microsoft made. It had a bunch of random little pictograms, and as this was long before Unicode support was common, it had the pictures mapped to random characters.
Someone noticed that if you typed "NYC" and then changed the font to WingDings the result was a skull-and-crossbones, a Star of David, and a hand making a thumbs-up gesture. So obviously, this was a shorthand way of saying: death to Jews in New York City is a good thing. And obviously, this was done on purpose by some black-hearted person at Microsoft.
http://www.snopes.com/rumors/wingdings.asp
The moral of the story: no matter what you do, someone will find a way to get upset by it.
lf(1): it's like ls(1) but sorts filenames by extension, tersely
...use external libraries so you're not re-inventing the wheel but keep your own copy of those libraries. So, you end up with your own unique island of code, basically cut-and-paste writ large?
I'd say the person who needs to learn a lesson is the author, not Node.
The best thing about this?
1. It's a shitty algorithm because it does repeated string concatenation. It runs in fucking exponential time.
2. In any reasonable fucking language, this is printf("%Ns", str)
It's a shitty ecosystem.
Would be if the messaging app that had it yanked down used his code and that was rendered unusable now
no matter how good it is, it is human nature always wants to make things better
Funny. Spoken like one of the incompetent masses. As I do know better about your unsophisticated Ad Hominem, I see no reason to correct your misconceptions.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Exactly. If you _need_ them, then you have no business coding the project you _need_ them for in the first place because you do not understand what you are doing.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Let me be upfront about my biases first: Node is trying to a solve a problem that really doesn't need to exist: To write everything in one language. It's amazing how much demand there is for it. It's clear that the core libraries and language just can't keep up with developer demands and the number of libraries to fill those demands has exploded out of control. Npm is packed to the gills with vanity projects that are made as a resume item for developers. Sure, there's plenty of these in other ecosystems, but it's amazing what has come to depend on them.
The Node ecosystem is amazingly fragile and it's going to get worse and worse. I fully expect there will be lots of work in the future unwinding the messes people made with it and replacing it with a more appropriate platform.
In one fell swoop, this person did exactly what free software is trying to prevent: a single overpowered entity who decides to leave and take his ball home with him, thus ruining it for everyone else. Power to the People? Only if our benevolent dictator also gets his way.
Define "need". Explain to me why I need to understand how a library works in detail. We use a geometry library that uses NURBS, which I haven't bothered learning enough to debug the library. We use a JPEG library, and I really haven't bothered to learn how JPEG works. Add up all the third-party libraries we use, and the company would never have gotten off the ground if we'd had to write all those ourselves.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
It is true that dependencies are unavoidable, but I have seen things reach the point where to build something, the build tools will have to hoover the Internet and download a bunch of gunk. What is it used for? You may know why your direct dependencies are present, but those dependencies may have other dependencies and it is all opaque as to why they are there.
Yeah, this is an exciting thing to deal with. At the last place I worked, the "architect" decided that the existing Java/.NET framework was crap (even though the code had worked just fine for years at production sites) and declared that we were moving it all to Node.js. Where it got fun was when the engineer in charge of builds saw his distributable package jump to almost 40,000 files, and the master build was unable to run every couple of weeks because of broken dependencies, which had to be isolated and dealt with each time. Meanwhile, problems that had continued to plague the system because of the 20-year old "roll-your-own" DBMS that didn't even support concurrency went unaddressed because they just weren't as much fun for him to work on.
Please stand clear of the doors, por favor mantenganse alejado de las puertas
Worse then that, because you probably didn't check for security issues and all the corner cases that other libraries developed over 2+ decades have dealt with.
No, I would take an approach similar to formal verification.
"First they came for the slanderers and i said nothing."
When should I avoid them? In what situations? There are times where the problems to be solved are so complex that not using a library could set you back years. In some cases it may kill a project entirely because the domain specific knowledge required to build the library might not be something you already have. I mean, where do I draw the line? Should I not use established database formats? Maybe I should build my own TCP stack?
The WHOLE IDEA of FOSS relies on interdependency of things written by a multitude of people. You may as well just throw Linux and all the rest out while you're at it as well.
BeauHD. Worst editor since kdawson.
When should I avoid them? In what situations?
As often as is reasonable. See, you have this amazing ability to use your own judgement and discretion. You need not be beholden to simpleminded absolutes.
There are times where the problems to be solved are so complex that not using a library could set you back years.
When we talk about dependencies, we're talking about far more than simply the use of third-party libraries. Take a moment to think about that, I've little doubt that you'll come to understand what the rest of us are talking about.
Required reading for internet skeptics
Define "need".
Need means required. Need means that you have to use a library in order to complete your project. Libraries are never required; you can always write your own, given enough time and effort.
That is why libraries are a convenience: they save you time and effort. But they are not required; they are not needed.
If developers are working under a license where they can withdraw their source, there is risk to anyone using their code downstream.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
People reinvent the wheel all the time. You'd have to or else we'd still be using stone wheels. Even a bigmath library gets reinvented, I've seen many versions. That's because one size does not fit all and some of those libraries or their RAM usage will literally not fit on some machines.
Modern programming religion says "thought shalt not write original code anymore". The JavaScript people take this seriously and their coding skills have atrophied to the point that they don't even know how write the code to pad strings.
Most of them won't break overnight though. JavaScript doesn't store its routines locally on your computer so every single time one of these library routines are used it is redirected the site that implements that routine. Whereas with Python using the package manager copies the code to your machine. If the library code later vanishes you can still reimplement it before hordes of web users start complaining that the world is ending. It would cause problems if someone goes and downloads your utility and follows the instrudtions to "install these packages first" but that's a minor problem compared to every user of the code suddenly finding that it has stopped working.
Wait, I implemented a new libc when I started a project, because the existing one was too large. Not entirely from scratch though, there was a lot of copying of our earlier code followed by a lot of slash and burn. Seriously, newlib was too big and had incompatible design choices. Not that we use a lot of libc, but you do need a small handful of basic functions.
Unless the functionality doesn't work or does not meet your requirements. It's called programming. If the only thing you've ever done is call libraries but are unable to write code similar to what a library routine does, then that's not really programming.
Remember, actual human beings with less than divine powers wrote those libraries! I may be a good rule of thumb or guideline for novice programmers to not reinvent the wheel but it should not be an absolute taboo for all programmers of all skill levels and all projects. If there is no wheel that you can find or afford for your project then by necessity you must create one; it's not even reinventing the wheel but instead reimplementing to match the requirements.
Bignum math libraries may be a good example as mentioned earlier. You may not be able to find one that is affordable and not burdened with some open source license that your legal team feels is safe to use (you can not just copy GPL code willy nilly and stick into your proprietary product). They often come with cryptographic libraries but may be too large for your actual system or have runtime requirements that are unacceptible (need 1KB stack or more). It is reasonably common to find optimized variants of such libraries making use of inline assembler for key routines. You will also find a handful of such libraries and need to be able to decide which one is appropriate (ie, read the source code, run tests, do measurements). Libraries are very often overly general purpose and bloated with unnecessary features.
I've implemented some of all those categories. Security because there's rarely a security library that does just what you want, or because the library you do have is too slow or too large (seriously there are some very poorly written SHA algorithms in professional libraries). Datetime because the system had a bad library that we had to replace and I would not just steal code from GPL and break their license, BSD was a guideline to start but had it's own issues because of historical system issues. And fundamental libraries because we needed much much smaller versions of things like memmove, strtoul, and the like that were optimized for severe space requirements. Sometimes you have a requirement of "make it boot in less than half a second" and then you find you can achieve that by optimizing the libraries. Sometimes a lot of commercial stuff comes with really bad libraries; there are network stacks or operating systems that decide to stick in their own basic C libraries for some stuff that are of very dubious quality.
People don't have to do this, but they should at least know how if they ever need to and should be able to recognize when it is needed. Like when your JavaScript library to pad out strings vanishes overnight.
(for security I should mention that we vetted all of this and the changes and fixes to the original commercial library, we weren't just cocky people thinking we knew better than the experts)
....but that's exactly what people are doing here when they tell you to avoid dependencies as though it's something you need to be doing all the time - they're making a stupid binary statement about something that is not binary at all. As programmers we need to stop trying to pass the things we say off as set-in-stone rules.
I already understand very well what you all are talking about. The point of my statements is to say that you can't make such a stark statement as though it is true in every case. It makes you look like you're parroting something someone else said and took it as a simple inviolate rule that you're either 100% following or entirely wrong about.
BeauHD. Worst editor since kdawson.
build tools for projects shouldn't do this. ever. there is no excuse which justifies it.
the project should just document the requirements/dependencies and point the user at them (and their install instructions). Or rely on the fact that distros package most important libraries (and many unimportant ones) for most languages.
trying to do a distro's job is a pointless waste of time and effort on the part of project developers. also, they tend to be very bad at it, often perpetrating unforgivable coding-atrocities such as 'bundling'.
It gets worse than just requiring different release versions of libraries. the 'bundling' atrocity mentioned above is often compounded by having the bundled lib be a private custom-hacked version that only works with/for the project concerned, with no attempt made to upstream any changes. at some time in the future, the upstream lib gets bug-fixes and security fixes that are unavailable to the forked/bundled version.
Yeah, the article says "gone and can not be restored" when it's clearly "will not be restored" since they clearly CAN do it. Logic fail.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Of course that's what abstraction layers are all about.
I do not have to understand the mathematical details and theorems that make SSL work in order to use an SSL library.
I do not have to understand how the letter T is displayed on a computer screen in order to use I/O libraries.
I do not have to understand Z-buffering in order to use a high level graphics library.
I do not have to understand how a modem encodes data for transmission in order to use a library that uses one.
In a real world context, "enough time and effort" is not always available, whether we're talking about a grad student trying to get a Ph.D. thesis done, a startup trying to get a revenue stream before running out of investment, or accounting software that legally has to be in place by a specific date. Moreover, it requires knowledge of what the library is doing. To replace the libraries we use at work, I'd have to be good with data compression, encryption, XML processing, PDF generation, and an expert on 3D computational geometry, as a start. I know a lot of what goes into this, but not nearly everything I'd need.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
So obviously, this was a shorthand way of saying: death to Jews in New York City is a good thing.
It depends on who you ask. Some said that it meant "Jews did it".
For stone wheels, fine. For software, wasteful. One of the benefits of open source should be the pooling of effort and intellect. Instead of many people writing their own library, and then having to refine and optimize it, many hands can get the same work done faster - and hopefully with cross-checking and review.
People reinvent software for the same reason that they reinvent the wheel. A truck tire doesn't fit on a kid's bike. The mountain bike tire is too heavy for competitive cycling. New materials are available to make stronger tires while being lightweight. Technology evolves, we shouldn't be stuck using code written in 1970 in places it wasnt' designed to go. I work on embedded systems, some of them very small. Glibc will not work on those systems very easily as it is a bloated and overly generic mess. So new libraries are written that are more appropriate. Lots of new libraries. Lots of new RTOSs. Lots of new everything. Linux is essentially a reinvention of the wheel. Competition is good.
It would save me a lot of time if there were libraries I could trust that fit the requirements I need.
Yeah, the article says "gone and can not be restored" when it's clearly "will not be restored" since they clearly CAN do it. Logic fail.
It's a common policy statement to avoid being inundated with requests to recover deleted files. The website did have backups and was able to recover that deleted file.
It's a common policy statement to avoid being inundated with requests to recover deleted files. The website did have backups and was able to recover that deleted file.
Yep. They negotiated once, now they will have to do it again next time or be called liars. What dumbasses. Almost as big dumbasses as the people linking external scripts
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"