Slashdot Mirror
Node.js's npm Is Now The Largest Package Registry in the World (linux.com)
Linux.com highlights some interesting statistics about npm, the package manager for Node.js.
- "At over 350,000 packages, the npm registry contains more than double the next most populated package registry (which is the Apache Maven repository). In fact, it is currently the largest package registry in the world."
- In the preceding four weeks, users installed 18 billion packages.
- This translates into 6 billion downloads, "because approximately 66 percent of the installs are now being served from the cache."
- ping.npmjs.com "shows that the registry's services offer a 99.999 uptime."
- Every week roughly 160 people publish their first package in the registry
But what about the incident last year where a developer suddenly pulled all their modules and broke thousands of dependent projects? npm's Ashley Williams "admitted that the left-pad debacle happened because of naive policies at npm. Since, the npm team have devised new policies, the main one being that you are only allowed to unpublish a package within 24 hours of publishing it." And their new dissociate and deprecate policy allows developers to mark packages as "unmaintained" without erasing them from the registry.
Packages on npm still aren't signed - something that Java repository servers have had since inception.
Might as well just open up your firewalls and let the hackers inject whatever code they want.
Broken by design!
Captcha: "amateurs"
When you get such trivialities as left pad in the registry, why should anyone care that the raw number of packages is large?
Quick everybody: how do you write "hello world" in javascript?
npm install hello-world
I think the debacle really just opened up a lot of eyes as to when it's appropriate to start npm installing a bunch of crap instead of writing your own code.
There's a fetish for modules in the JavaScript world that defies reason.
"What? Use the built-in keyword "function" for defining functions? Heavens no you fool, we install Sindre Sore-Ass's woopee-unicorn-function-creator package!
It's cancer all the way down on NPM.
I'll stick with Microsoft's Nuget Package Manager for all my development needs, thanks.
There's no choice on the client, but why do people put up with all of Javascript's many rough edges and missing features when there is a universe of more appropriate server-side languages?
Ash ? Guess jacksonville got boring after a while eh ..
A huge problem with JavaScript, compared to other languages, is that its standard library is totally lacking, even after 20 years of existence.
A lot of common library functionality that Java, C#, Perl, Ruby, Python, Tcl, Go and even C++ include by default just aren't present when using JavaScript. Or worse, if JavaScript does include some functionality it's often really shitty, sometimes to the point of being unusable.
So if you're using JavaScript you pretty much have no choice but to start using external packages almost right away. That's why npm has become so widely used: it's because JavaScript itself is so goddamn lacking in the most basic of ways.
Npm is basically a bandage that you have to apply to JavaScript to make it even barely usable. And you have to apply it for pretty much each and every project written in JavaScript.
Yeah...
I've worked with a lot of programmers (or pseudo-programmers) over the years, and the ones who like JavaScript tend to be the most ignorant of them all. I'm using the term "ignorant" in the sense of "they don't know that alternatives exist" or "they don't know how bad they have it".
Not knowing that other languages exist, or having never used them if they are aware, these people pushing JavaScript really have no idea how bad they have it.
I mean, if you've never used a real standard library like other major languages provide, you wouldn't know how awful JavaScript's is.
If you've never used a language with a proper type system, you wouldn't know how awful JavaScript's is.
If you've never used a language with good support for OO (sorry, prototypes are a shitty hack, which is why more recent versions of JavaScript have actually started hacking in the idea of classes), you wouldn't know how awful JavaScript's is.
This is why good programmers, who have typically used a real programming language, can't stand JavaScript. They see how bad it so often is, and they want nothing to do with it.
But those who have a limited awareness are blind to the flaws. They'll gladly use inferior technology because they just don't know any better. They don't know that they should expect better.
There is such a thing as "too big" for package repositories: at some point, the benefit of being able to find packages for obscure uses is outweighed by the cost of having to sift through endless lists of redundant packages, the incompatibilities arising from many people using incompatible frameworks, and the inability to tell easily whether a given package works well. In JavaScript, that's compounded by the extremely loose type system and error checking.
And tomorow, it get replaced with the next 'greatest thing'... But back to the topic, how many useful items, and not dead end projects or simple crapola ?
The amount of code needed to write a web application using Node.js is tiny compared to even PHP, which itself requires a lot less code than java or others. Performance is excellent, especially if you combine it with a web server for static content (like you would do with most web technologies).
Even without using frameworks (like Express), Nodejs is a technology that is well-suited for web applications. There is a learning curve because of the asynchronous paradigm (which can be mitigated if one uses promises) but overall the language is decent and favors good practices, such as MVC or code reuse. In the age of the API this is a fantastic platform to quickly put together a REST architecture.
Also, don't bash Nodejs for server-side code. Because of the self-contained nature of npm it can prove quite convenient for all kinds of applications and utilities, not just web applications. Whenever I need a quick script that involves database access or interacting with web services, I no longer use bash and tools like curl or wget, I get what I need a lot faster with Nodejs. There are so many excellent packages on npm it's just a no-brainer.
lucm, indeed.
I'm really confused about node.js. Can anyone give me an example of something that they used it for that would have been a lot harder or wouldn't work as well as a traditional web framework? I guess what I'm trying to ask is, if someone is already comfortable with a framework is there reason to experiment with node.js?
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
... were even engaged.
It little behooves the best of us to comment on the rest of us.
made by amateur 17 years olds....
Scary so many people are using what is arguably one of the worst languages ever created.
Wow, the NSA has been busy writing code!
What would be telling (especially in light on left-pad) about npm, JS developers, and JS itself is how many of those packages are larger than a size that would be considered ridiculously small in another repository: 25 lines of code (which is being quite generous), measured the same way that left-pad becomes 11 lines.
I cannot understand my fellow slashdoters that make fun of leftpad, node is useful for so many more things.
For example just recently huge innovations were made within the node community and we are now proud to announce 1325 different variants of rightpad.
Can C++ do this?
Didn't think so!
The no .de movement must end. Germans have as much right to a domain on the internet as any other country has.
The node ecosystem has made a conscious choice to split code into a huge amount of very small (tiny) packages that in many cases only provide a single function. Irrespective of whether you consider that to be a good idea or not: if you compare that to most other packaging ecosystems, where packages are typically larger because they bundle a lot more functionality (and packages that small are frowned upon), then of course there are going to be more packages in npm than anywhere else.
It's like buying two identical cakes, giving one to Alice unsliced, slicing the other one into 12 pieces, accidentally dropping one on the floor, giving the other 11 to Bob, with Bob now claiming to have 10 pieces of cake more than Alice.
Can you really call 10-line code snippets... packages?
Not only is it larger than the next largest package registry, but it's also the largest package registry in the world! It would be amazing for a package registry to accomplish just one of these feats, but it managed to accomplish both!
The best thing about npm is that it can re-create the Ruby experience where the first step of running some trivial app is to install 230 packages! It's a real language!
And god help you if you actually decide to use the app for the long term, because in twelve months half its dependencies will no longer be maintained, and the other half will require updates after you do an OS upgrade, so you'll be in there debugging errors yourself. This will help train you for a 21st century job!
You like JavaScript because it helped one of your bad developers? Are you a moron?
He's not a bad developer.
A bad developer is not someone with an imperfect knowledge of some aspects of software development. A bad developer is someone who can't see the big picture and can't deliver code that is fit for purpose in the specific context where that code is needed. From corporate drones who write code for legacy Oracle Forms applications to Chinese engineers who design on-screen menus for low-end plasma TV, from Wall Street mathletes writing micro-trading platforms to people who crank out heaps of custom Wordpress theme for small businesses, it's a big world with amazingly different requirements for time-to-market, quality and solution resilience.
I remember years ago we had this massive migration project with a brutal deadline. We had a "genius" on the team who went home to work on a perfect solution, using all the fancy languages and libraries. But he never finished; after a week the deadline was looming so while the genius was writing unit tests for his migration framework, as a Plan B an intern patched together a bunch of VBScripts crammed with "on error resume next" and hard-coded file paths that got the job done in a day. Who was the bad developer in that story?
lucm, indeed.
The largest package registry, which does not host a thing, it only uses github. It is just an awful hack. How about creating an efficient and properly namespaced registry? Even maven central is not so badly constructed.
you too can have all 350,000 packages installed in your node_modules/ folder!
i love node, but even the smallest projects can end up with 30,000+ files (files, not modules) in that folder.
scares me a little bit.
"This just in! NPM's record on packages has been broken by an 'EmeraldBot' from Slashdot, who now hosts the largest number of packages in the world, at 550,050 and growing. Each one represents a single byte of the compiled program leftpad++, and is soon expected to double in number with the introduction of rightpad++."
On a more serious note, NPM's claim is dubious at best because they split programs up into so many packages, some only providing a single function. A WHOLE PACKAGE FOR A FUNCTION. That view is largely dependant on what exactly you consider a package, and me personally, I need something more substantial than a literal copy and paste.
"Set a man a fire, he'll be warm for the rest of the night. Set a man afire, he'll be warm for the rest of his life."
> When you get such trivialities as left pad in the registry, why should anyone care that the raw number of packages is large?
So you're saying node.js developers have small packages?
> I remember years ago we had this massive migration project ...
> after a week the deadline was looming
> Who was the bad developer in that story?
The one who said "yes, we can do this massive migration in a week or so."
There are many different uploads of jquery and every other popular library.
Ever installed some nodejs stuff?
You do "npm install" and watch an endless packagelist being downloaded. No, not to the central installation, but into the project. And they are like modules with 5 lines. See for example the "left-pad" thing. Yes, people include other programmers code for 5 lines of a function which you can create without even thinking about it. And they include such 5 line functions from hundereds of different people in their project. Not only one missing package can break millions of builds (see the left-pad example), but one malicious programmer can infect millions of production systems by issuing an update, which includes one malicious line, which loads some external script he will be able to change on demand. Because who re-reads the code of the modules, if he even read it the first place, when adding it because the name and short description seemed to match the requirements.
The node.js ecosystem is fucked up. Working, but still a working mess.
Seriously, javascript was initially made in 10 days, by someone with not much expertise in programming language design, and it remains around. I wonder how much agony could have been saved if a decent programming language had been used instead.
"In the preceding four weeks, users installed 18 billion packages."
because of broken system of dependencies. npm is total shit when it comes to dependencies.
One challenge I have with npm are the dead projects and the apparent inability to take over the dead project, even if your project has become the accepted source of truth in GitHub. The workaround is to create a new package, but that just adds to the confusion.
It would be nice if there was a way for a project to either be flagged as possibly dead or require some other mechanism to red flag a project, either automated or via reporting.
Maybe I am alone in this feeling?
Jumpstart the tartan drive.
You have to search through a ton of crap to find whats good.
Non sequitur: Your facts are uncoordinated.
From the same guy who wrote the funny mongodb webscale argument is his take on Rockstar node.js arguments.
So you see nothing wrong with blobs of callback after callback with making your own threading and scheduling from scratch?
Wouldn't it make sense to use nginx for performance and let that and your OS do the threading instead?
http://saveie6.com/
So you see nothing wrong with blobs of callback after callback with making your own threading and scheduling from scratch?
Different problems are solved with different tools. If the problem to solve is answering as fast as possible a tsunami of lightweight requests (such as a typical web app), Nodejs is awesome. However if there are fewer, long-running requests that tap into heavier resources (such as crypto or image processing) then maybe it's best to consider alternatives.
But even if a web app is based on Nodejs, does that mean I would use it alone? Not at all; while Nodejs can do SSL and static content, I prefer to offload that part to nginx or Apache. Which doesn't reflect badly on Nodejs because that's what I would also do for a java webapp running in Tomcat. At that point though we're no longer talking about handling multi-threading inside or outside the app server, we're talking about solution architecture which is a totally different matter.
lucm, indeed.
The one who said "yes, we can do this massive migration in a week or so."
That would be someone in management.
That's the usual team work; the manager makes crazy promises and the underpaid techies deliver. And usually everyone is rewarded; the manager gets a fat bonus check, and the techies get free pizza and Red Bull for working all weekend. Win-win!
lucm, indeed.
JavaScript has been accused of many things, but "threading" isn't one of them.
I don't mind blobs of callback in principle. Continuation-passing style doesn't scare me at all (especially in a language like Scheme or Haskell). But I agree with you. The more I used Node.JS, the more I realised I really wanted Erlang.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
I'm not the person you replied to, but ...
> And in what way is JavaScript reinventing threading - it's single-threaded!
As you know, a generic single-core cpu is also single-threaded. (Max 16 threads with 8 core cpu and hyperthreading). The operating system simulates running many threads at once. It's actually only running one thread at a time, switching between the two, below that level it's actually single-threaded (or has just a few cpu threads).
JavaScript / Emacscript does something similar - simulates running many tasks at once. In some implementations, it does so very poorly. In one particular implementation I had to use for several years, there were two underlying OS threads - one for network connections and UI interaction, one for everything else. That's possibly the stupidest possible division of labor - the part that should be fast, UI calls, had to wait for the slowest part (by far), the network connections. There is a reasonable argument that it would work a heck of a lot better if JavaScript exposed real threads, which have been implemented well by very smart people improving the implementations over many years. If Javascript exposed pthread_create() or some other interface to OS threads, "div.visibility = hidden" wouldn't have to wait for some unrelated, slow network connection to retry its DNS lookups.
There are also advantages to Javascript model of callbacks with a single real thread, just as there are disadvantages. There are reasons most languages expose real threads rather than offering *only* their own built-in simulation of multiple concurrent tasks - most languages expose threads because there are benefits of doing so.
The ideal may be a model where youb don't *need* to use real threads, a JavaScript-like model where the language wraps concurrent tasks in a pretty model, *and* clone() and fork() are available for the few times that you need them. Programming the Apache web server, writing Apache modules, is like that. It's callback-based and I don't recall offhand ever needing to create threads in an Apache module, I forked an external shared process for one module once. But that one time I needed to fork a process, I was able to.
JavaScript has been accused of many things, but "threading" isn't one of them.
I don't mind blobs of callback in principle. Continuation-passing style doesn't scare me at all (especially in a language like Scheme or Haskell). But I agree with you. The more I used Node.JS, the more I realised I really wanted Erlang.
Yeah man. Erlang is the real rockstar language
http://saveie6.com/
JScript.NET is basically Javascript, and it also works with strong typing, and threads. It's been around for more than a decade. Microsoft doesn't really support it, but JSC is a javascript compiler that can create .exe's and .dll's out of javascript, and it is still included with .NET and is on every windows computer with .NET installed.