subscribe

Npm package author revokes his packages, breaking tons of builds

I just came across an interesting post via Hacker News, from an author of several hundred NPM packages (some of which quite popular) that just removed all of his packages from NPM.

Tons of other projects around the world depending on his packages broke as a result of this. The NPM project responded by un-un-publishing the packages:

While you can say that the original author was not very nice to do this as a protest, and without warning, I think it highlights a larger underlying problems, in not just NPM but also other packaging systems:

  • We’re currently relying on the trustworthiness and ethics of many package authors.
  • Package repositories are a critical piece of our infrastructure.

Both are single points of failure for a lot of projects, except the few that actually commit their node_modules, vendor, etc directories to their github repository.

Another interesting thing is that package authors can not just un-publish their packages, they can even modify already-released packages.

I think this is a very weak link in our infrastructure. What we need is a packaging system that is:

  • Immutable / Append-only
  • Decentralized
  • Distributed, anyone should be able to run a mirror.

Append-only means that once you publish a package, it can never be changed or unpublished. It can’t be censored or taken down. This puts the control back in the hands of the user, and we’re no longer at the mercy of package developers or centralized repositories.

Web mentions

Comments

  • KJ Singh •

    Immutable makes sense! Many maven repos don't remove the older versions of the packages (aka jars)

    • Magino •

      Yes immutable is the best option. Nice article, it's good to launch this debate, so many projects are based blindly to npm.

  • bSzala •

    I don't think packages should be append-only. This could end up with hundreds of sub-versions & garbage package(s), which normally could be abandon/removed. I would instead recommend to allow package owners to make a remove request, without actually removing a package. Then let say Packagist could clean up all of those flagged packages at a specific time.

    • Julien •

      I don't like the idea of being able to remove packages that others depend upon on a whim. When a package is published, it's public and should stay that way. Maybe a mechanism for marking a package or version as obsolete or insecure would be desirable, but not removing it outright.

      I do like the idea of a decentralized, distributed system, but until that shows up I'm more interested in fixing what we already depend upon every day.

    • Evert

      Evert •

      Just like a blockchain, it would not be needed for everyone to have the whole history. You'd need some other system to help people figure out which packages are relevant and useful. The model I'm describing is a data-model, not a UI.

  • Pm •

    Isn't un-unpublishing even worse than unpublishing? Who's the owner of the package and who can make decisions for it, the author or NPM? So yes: decentralized all the way but also remember that if the author decides to make a package unavailable there might be licenses that would prevent us to keep using it no matter if we have thousands of local copies.

  • Spudley •

    In the PHP world, there is Toran Proxy (https://toranproxy.com/), which is basically an extra layer on top of Composer designed to allow you to have a local build process that doesn't necessarily rely on Packagist and Github. Something like that would definitely have solved this problem on NPM; the package could have been completely removed, but as long as you already had a working local build, your build would carry on working just fine.

    • Robclancy •

      We also get a vendor prefix, composer.lock that actually locks (unlike shrinkwrap) and even though it is slow as fuck it isn't anywhere near as slow as NPM.

      Funny how PHP has a far far better package manager than node when node is apparently the best language ever.

  • wscott •

    We should be using something like Nixpkg (https://nixos.org/nix/) but using packages distributed on IPFS (https://ipfs.io/). This way anyone can publish a package, but the 'hosting' of this package is distributed and as long as someone is using it, then it still exists.

  • Robclancy •

    NPM is easily the worst package manager unless you classify things like pip and bower as package managers (really they are just download managers).

    I have to deal with a new NPM issue every week. Be it breaking randomly for no reason or someone doing something on some random repository that NPM decides you need because it is too stupid to be able to lock dependencies reliably.

    Hopefully this starts the end of NPM.

  • domm_plix •

    You might want to check out how CPAN handles all of this, especially BACKPAN: http://backpan.cpantesters....

    Perl++

  • Jim Lehmer •

    I think in general immutability is the correct approach, but (there's always a but), how would you handle a package version that either (a) is discovered to have a severe security vuln (can be quite likely, depending on the package), or (b) is discovered to actually be malicious (less likely), or (c) as in this case is actively under attack for some IP issue that may not just drag the author into the fray but also anyone using it? If there isn't an "unpublish" (forget the "un-un-publish" fiasco), how do you mark such a package as "Do not use" in a way that isn't in effect an "unpublish"? Not arguing your point (again, I think it's correct), just trying to think through the real-world scenarios.

    • Evert

      Evert •

      I think we'd have to accept that there will be broken packages in the history. But in the same way you can create a revokation certificate for gpg, it should be possible for an author to also mark someone as 'do not use' after the fact, without making real changes.

    • Adam O'Grady •

      On top of these reasons, what happens if someone accidentally publishes personal info they were using in development but forgot to delete? People publish API keys, usernames/passwords, and the like to GitHub all the time by accident; "unpublishing" (versions, preferably not whole repos) allows them to fix this.

  • Erik Rose •

    Mozilla’s npm-lockdown gets you immutability, which is a start, verifying packages against local hashes before they get installed: https://github.com/mozilla/.... We use it on our production servers, along with similar strategies for Python code.

    I'm the author of peep and of the new hash-verification functionality in the latest version of the Python installer pip. We went through these same growing pains in the Python world. Step 1 was to make uploaded packages immutable. Combined with hash verification (where you check the hashes into your source tree), that obviated the need for many personal mirrors. Delete-proofing is a tantalizing next step.

  • joelatone •

    Related...

    https://github.com/mhhf/spore is a decentral package manager for dApp development based on ethereum and ipfs.

    https://github.com/whyrusle... is a packaging tool built around the distributed, content addressed filesystem IPFS.