Slashdot Mirror
Botched npm Update Crashes Linux Systems, Forces Users to Reinstall (bleepingcomputer.com)
Catalin Cimpanu, reporting for BleepingComputer: A bug in npm (Node Package Manager), the most widely used JavaScript package manager, will change ownership of crucial Linux system folders, such as /etc, /usr, /boot. Changing ownership of these files either crashes the system, various local apps, or prevents the system from booting, according to reports from users who installed npm v5.7.0. -- the buggy npm update. Users who installed this update -- mostly developers and software engineers -- will likely have to reinstall their system from scratch or restore from a previous system image.
A shitscript package manager that does a chmod of /etc and /boot? This thing had to have been written by that Poettering asshole.
Only the State obtains its revenue by coercion. - Murray Rothbard
If it is a file permission issue... boot from install disk into rescue mode... chmod and reboot. I don't get it.
I remain of the opinion that none of those "language specifically package managers" have no place on Linux systems. They should use the operating systems package managers and tools.
Ahhh...the great dumpster continuum. Many a free computer will be found there. -- sowth (748135)
I'll bet some new Linux users would freak out. Hey it happens, and yet this probably affects way fewer users then it happening on Windows. So hardly anyone will care.
Time to introduce containers like Singularity or Docker!
Another good example of why millenial hipsters have a lot to learn. While javascript outside a browser isn't the core problem here, it's a symptom of the problem. Another symptom of that same problems is a package manager that does idiotic stuff like this, and apparently needs more than userspace perms to run?
If one knows the correct permissions, you could just change them back (via grml or so).
That said, this is bad. It underlines related issues of the apparently bad package management situation we're in. Why can I not install all my software through a good package manager? GNU Guix?
Why the fuck would you have to reinstall the system are these people run 1994-era sls systems or lfs. Most linuxsystems in production has a basic rights database in at least one place that will allow you to fix those permissions and if you value your systems integrity in more than one.
Enough said
windows has better group permissions and multi user ones.
Why, for the love of God, would you do that?
yum reinstall * and reedit any changes by hand?
That will be $75/HR or I can just reload from backup.
so one script for each os base line? and not any added software that has it's own per app rights.
Not really. It's just that a lot of Unix people don't use/know about ACLs in Linux. try man setfacl.
In general, you can achieve similar results using conventional unix permissions, but it takes a bit more work. You just create a group to represent a directory tree that needs to be useable by a subset of the users on the system, and then assign the relevant users to that group.
The quick response and patch actually shows the advantages of agile, iterative code-first development using pair programming and Scrum, continuous integration and release early and often.
but how much software does it by ACL's? some from repo installs may just do it the old non ACL way.
Fix available now.
Better than what? Windows uses the same sort of versatile ACLs that other OSes have, but lacks the convenient oldschool UNIX permissions. Windows' group permissions are slightly worse in one specific way but don't have any advantages over the competition to make up for it.
So even Linux isn't immune to shit updates.
This is silly.
Who is pulling down and running unknown, untested code onto production servers?
Why is anyone even using npm as root or with sudo? After years of node use I have never had to do that.
You do run Btrfs, don't you? And you do make timely snapshots of your system, don't you?
If you do, then rollback to your last good snapshot and reboot. Done.
"Not really. It's just that a lot of Unix people don't use/know about ACLs in Linux. try man setfacl.
In general, you can achieve similar results using conventional unix permissions, but it takes a bit more work.
"
In practice it really only comes up in a small subset of use cases.
People who use node typically aren't the beardy types who know this sort of thing. So by their own ignorance they end up having to retry, reboot, reinstall. Guess why poettering software is so popular in those circles these days.
1. There is no reason to run a language-specific packager as root, whether npm, pip, composer, maven, etc. Either the package manager makes packages available to the user in $HOME, or there exists some kind of virtual environment tool. Use them.
2. Why is NPM chowning anything?
3. Read the thread, the attitudes there are unfortunate to say the least. A new version of NPM is provided when using NPM to upgrade itself without any arguments, and it grabs a "pre-release" version without warning? The version number is 5.7.0, not 5.7.0-beta or 5.7.0-rc1 or whatever. The NPM people violated semver. So there was no obvious way to know this is not an official release.
I'm god, but it's a bit of a drag really...
Non programmers, thatâ(TM)s who.
What a fucking joke. JavaScript sucks.
What is not mentioned in the summary is that the bug only shows up when using sudo.
Sudo is a nightmare, both technically and psychologically (strangely, it's seems easier to run 'sudo npm' or 'sudo fuck_me' than running the same commands when logged in as root).
It makes me laugh any time when I try to build some shitty program (inside a vm, of course), and more often than not, it tries to run 'sudo' from the install rule and trash over my system by writing and overwriting files inside /usr and /etc, and ignoring any PREFIX option, despite that convention being almost 40 years old.
I really don't understand the appeal of 'sudo' -- what's the problem with ssh root@localhost with public key authentication?
[coward@omm ~]$ sudo pacman -S npm
resolving dependencies...
looking for conflicting packages...
Packages (4) http-parser-2.7.1-1 nodejs-9.3.0-1 semver-5.4.1-1 npm-5.6.0-1
Total Download Size: 8.19 MiB :: Proceed with installation? [Y/n] n
Total Installed Size: 39.46 MiB
sudo is used to bypass normal sane security mechanisms ... don't do that
if you think you need sudo, you're doing it wrong
Chroot into your damaged install, mount a devtmpfs and a /proc, maybe /sys,
then, "for p in $(rpm -qa); do rpm --setperms $p; done"
Works on rpm-based distros at least.
If it's changing all the modes to 000 or something, do a "chmod -R 755 /etc" or something on whatever files are causing trouble, then run the above to restore them to the correct state.
Maybe those "developers" and "engineers" need to learn how to fix their systems with the super-advanced CHOWN command, instead of reinstalling??
I work for the Department of Redundancy Department.
;-]
Does it?
Since NPM now bricks systems, it clearly falls into the category of malware/ransomware. I hope distributions treat it as such, and prevent installation thereof
Granted, but that's not necessarily a good argument. Standard Unix permissions provide a fairly limited set of abstractions. You have groups, users, and anyone; read, write, and execute; and a few other flags. It's concise, capable of expressing anything you like, and very efficient. One can say the same thing about assembly code. In both cases, having better abstractions is necessary, and the most important factor is the amount of time required by the various developers. And on that note, I must say that I have no idea whether either modern Linux or Windows has an advantage over the other, simply that the argument of "it doesn't come up often" isn't necessarily a good one.
Those who advocate genocide deserve every protection afforded by law, and none afforded by common human decency.
I had a clever response to this, but then my computer had to reboot for updates, and I was waiting so long that I forgot what it was :(
who the hell approved the pull request after seeing code that changes permissions in the file system? "Oh hey, looks like you're changing permissions in system folders, forget testing, we're going straight to production with this"
NAME
mtree -- map a directory hierarchy
SYNOPSIS
mtree [-LPUcdeinqruxw] [-f spec] [-f spec] [-K keywords] [-k keywords]
[-p path] [-s seed] [-X exclude-list]
DESCRIPTION
The mtree utility compares the file hierarchy rooted in the current
directory against a specification read from the standard input. Mes-
sages are written to the standard output for any files whose character-
istics do not match the specifications, or which are missing from
either the file hierarchy or the specification.
bash$
Phil
Someone already thought a few steps ahead ...
https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5
This NPM insanity has to stop
I vaguely recalled a Linux distro where e, "each program gets its own directory tree". With a little searching, I found it:
https://linux.slashdot.org/story/16/12/17/0655249/gobolinux-016-released-with-its-own-filesystem-virtualization-tool
I don't have any experience with GoboLinux, but I thought I'd throw it out there in case anyone does and can chime in.
This is why I always reject anything that has requirements that I install the latest version of everything and use a language specific package manager to manage dependencies. Javascript packages seem the worst for the "bleeding edge" requirement, but Java, PHP, Python, Ruby and even Perl have long had issues with requiring the language specific package manager to be used.
If my distro maintainers have not packaged it and tested to the level that the rest of the OS gets tested, then it has no place on my server.
"profession"
setfacl -m u:joe_dragon:rwx filename
setfacl -m u:joe_nobody:rx filename
- Michael T. Babcock (Yes, I blog)
Imagine if that had happened with Apple.
Slashdot would have completely filled-up their Servers by now with the Hand-Wringing and Apple-Hating Diatribe.
Think about it.
Do you mean the acutely difficult: sudo diskutil repairPermissions /
It's the kids whose grasp of linux is all about Ubuntu
This explains a lot. A couple weeks ago I was complaining about having to build a Javascript app written using Babel and NodeJS. The build was designed to automatically check for and download the latest version of all packages with NPM before building. After the inevitable swath of updates, things were so broken beyond repair that even re-installing all the packages didn't fix the build system. The only way I was able to get anything to build again was by resetting my VM image and reinstalling from scratch. Do the project developers seriously expect me to reinstall my OS every time they push out a project update? Given that this is a web app, updates happen pretty damn often!
When it comes to the point where web developers are trying to do things as root, you know this shit has gotten totally out of hand.
This is more than horrible design, even etc is not for all programs or applications, they should use: "/home/$USER/.npm/etc".
The update script for a custom application our company uses had been issuing the " rm -rf ./* " command, and a bug caused it once to issue it from the root directory... ouch.
I demand value for money from all the free software I use.
It's not the first time something like this happens, remember Bumblebee uninstall script running "rm -rf /usr"?
https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/issues/123
Or when EVE Online deleted C:\boot.ini on Windows systems?
https://www.eveonline.com/article/about-the-boot.ini-issue/
Other language-specific package managers have their situation sorted-out. Take Perl on Debian, for example, where most of the Debian packaging derives well from the Perl package management.
It's those "special snowflake" (aka antisocial) language environments which are always causing problems. I avoid those as much as I can.
This is why I always reject anything that has requirements that I install the latest version of everything
Please send me through your IP address. I look forward to having another internet connected machine at my disposal.
The first tweet in TFA really takes the cake:
"“if I run sudo npm --help my filesystem [changes] ownership of directories such as /etc, /usr, /boot”"
You don't even need to install or make a change to the system. Simply sudoing npm is enough to trigger the bug. Ironically those people who are ballsy enough to run everything as root are unaffected since this seems to be some iteration with sudo only.
Epic fail. Wait are we still doing Epic fail or are we just saying "Sad!" ?
You know something is wrong when your language package management tool is allowed to modify core files in your system.
But, wait a moment. Using a completely crappy unsafe language such as javascript to make server side programming is also worse than letting your development tools to mess up your fs.
For everyday use, BTRFS is actually pretty stable, and is used by default by openSUSE.
(and used to be used by default by Jolla's Sailfish).
Some specific *features* (mainly : RAID5/6) aren't considered stable yet.
(But that's the case of even XFS: they working to add CoW and snapshotting too)
But if you stick to default settings, it works perfectly.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
At least his server works, unlike those with this NPM update.
How stable is btrfs?
I use it as my everyday FS on laptop (default for openSUSE), on smartphones (default for Sailfish OS), and on a few servers I take care of.
The default options work perfectly well nowadays. (CoW has been able to get me out of very tricky situation like hardware crashes)
The thing that are considered still unstable are features that aren't enabled by default (namely: RAID5/6, there has been progress lately, but it's still not considered stable. Unlike ZFS's RAID-Z/Z2)
The "paint yourself in a corner CoW madness" that typically plague young CoW/Log FS ("enosp" while trying to delete files) has disappeared long ago.
Warning : another peculiarity of CoW /Log FS :
- DO NOT RUN "fsck" EVER. It doesn't make sense to fsck, better use the features of CoW / Log to roll back to an older version
(at least only use it as a desperate last resort, when all the CoW's / Log's feature have failed)
(In BTRFS' case, that means first try to mount with "usebackuproot" (formely "recovery") option to use an older pre-corruption version, or use the "btrfs recovery" to read the files out of a corrupted filesystem/image. So basically by then you've already fixed your system before even reaching the point where you'd consider fsck. So don't run it.).
I would still recommend :
- big files that get tons of random write (e.g.: databases, virtual volumes, torrents), and that basically have their own integrity checking built-in (db's log journal, the checking/journaling of the filesystem inside the VM, etc.) can benefit if you mark then "not CoW" (chattr +C, usually done only on new files) to avoid excessive CoW fragmentation and redundance between CoW and their respective integrity checking.
(I've read somewhere the ZFS can do this automatically detect such files for you. In BTRFS you need to manually tag these files with chattr)
- running "btrfs scrub" periodically to check data integrity and use btrfs' built-in checksum to detect bit rot (big feature of ZFS/BTRFS compared to oldschools FS such as EXT4). (on openSUSE look for package "btrfs-maintenance" that can setup cronjobs for you)
- running "btrfs balance" with filters like "-musage=60 -dusage=60" to tidy up allocated chunks that are more or less empty. It helps to avoid running out of allocatable chunks. (on openSUSE look for package "btrfs-maintenance" that can setup cronjobs for you, on sailfish "btrfs-balancer" can help automate this work and is used by the system updater to free allocatable chunks) (eventually it will be automated in the background by BTRFS).
- running "fstrim" after above-mentioned balance so the SSD's firmware knows the freed chunks can now be claimed back as free erase blocks for wear-leveling purpose. (on openSUSE look for package "btrfs-maintenance" that can setup cronjobs for you)
That's just my personal recommendations, it's not mandatory.
And now the interesting parts :
- openSUSE provides a tool called "snapper" that will automate taking snapshots with BTRFS
- YaST interract with the above whenever you make an upgrade.
- the GRUB script they provide can conveniently setup an "advanced" boot menu entry that propose to directly boot into an older snapshot, circumventing any damage done by a botched upgrade.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
v5 was not an LTS, and it's also quite old. The latest LTS is v8 and before that v6. Not many people will be using v5 and no one should be using it in production.
Except for the 16 groups per user limit, of course.
Yes I can see that. Its a very good machine to have at my disposal.
Drop npm. Use yarn. It has a lock file, it's quick, doesn't DoS the DNS until it decides to timeout, it doesn't take half an hour to install the packages in Gitlab CI, and it feels professional... Yarn is just lovely.
"Everybody's naked underneath" -- The Doctor
You are conflating the latest version, which is generally unstable and full of potential zero days, with security patches, which distributions do a good job of backporting to the stable versions they are shipping when the ADHD developers of the upstream are too lazy to maintain anything other than git HEAD.
Something is wrong with the summary. It doesn't make any sense. /usr, /etc/, and /boot are all owned by root. Only root is allowed to add files or modify these directories. Changing the ownership would not change this, root is still allowed to do everything. It would be a security issue, but would not have any effect on booting or system stability.
this is why you always have a Stage or Test on ANY change, or at least a daily/weekly image so nobody should be complaining
I only tangentially use node for development work, but once I saw people on message boards tossing around sudo commands to install stuff I immediately looked up how to avoid that. The solution is here: https://github.com/creationix/....
Basically, it installs node and all node executables into an .nvm directory in your home dir and then modifies your path to point to those distinct version. It also allows you to utilize multiple different node versions across different projects through the use of project specific .nvmrc files.
Using nvm would have avoided this bug, assuming you run as a non-privileged user...you do, right?
for i in etc usr boot
do
cd $i
sudo git revert HEAD
done
You are conflating the latest version, which is generally unstable and full of potential zero days, with security patches,
And with 99% of software on the market they are one and the same and therefore should be conflated.
Except for the 16 groups per user limit, of course.
If you've been asleep in a cave for a couple of decades.
Current Linux limit is 65535. Previous limit was 32 for kernel version earlier than 2.6.3
...you probably should not use linux.
A simple example is "btrfs balance". It is fundamentally impossible to do this safely without having to re-write all of the data, which means it's literally impossible to do with a drive more than 50% full.
Huh, no.
A "balance" is not literally rewriting a new copy of the whole drive in one single atomic step.
Even the large scale "rewrite everything" rebalancing usually works one allocation chunk at a time. So even that one is possible to do, as long as you have at least one free chunk.
The "usage" filters let you usually try to free allocation chunks by taking the least full chunk and appending it's content on allocation chunks that still have free room (that's what "-dusage=0" does).
that enables you to balance when you con't even have a single allocation chunk free, and only have free-space equivalent to an allocation chunk spread all-over the remaining non-full chunks.
That's how you can get yourself out of "stuck with enosp errors" situation. (That's also how opensuse's btrfs-maintenance and sailfish os' btrfs-balancer begin their work, and only then start increasing "usage" in steps).
Also all the data that is represented inside those allocation chunks is handled in a CoW fashion :
- new copy of the data is written over some new free space inisde the new allocation chunk
- pointers are updated
- the data from the old allocation chunk is marked as "free"
- repeat enough of the above and the whole old chunk is emptied and available again for chunk allocations (and can be signaled to be free through "fstrim" so the firmware on the SSD knows that it can claim back the erase-blocks that it allocated to that logical block address)
Basically, there isn't much difference between a normal (partial over-) write operation and a balance, except that :
- in case of balance, the newly written extents of data have the same content as the original extents of data.
- as you're sequentially running through all (or the part that is considered by the filters) data, the new data extents will be all stored nice and tidy in their new allocation chunks instead of being sparsely spread all-over the place, due to extents in between having been freed and/or moved around.
- an indirect result of the above : the free space gets "de-fragmented" and new space is available to be allocated for upcoming chunks.
There's no point in time in this procedure where you can't retrieve coherent data.
A BTRFS balance interrupted by a reboot/hardware crash/whatever) can simply resume where it left of at the next boot.
(and usually does this automatically).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
I want better than "pretty stable" for my file system, I want stable. Pretty stable suggests not completely stable.
If you want "stable" - you only stick to the recommended features that are marked as stable.
Sorry if I wasn't clear.
The rest works well enough to be used in production by multiple companies (Facebook, SUSE, etc.)
As a non omniscient-being, I can't vouch that there will never be some weird bug hidden in the darkness that will suddenly jump at you when you hit that weird corner xase that nobody has though about ever. But that's valid for any other part of the Linux kernel and any other piece of software in general.
But enough people are pouring resources in and using it in the real world to keep the probability of such an event occurring extremely low, to the point that it seems acceptable to me.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]