The ability to fork and gain control if necessary is one of the main reasons one wants open source in the first place. So fork it.

Other than that: I find the microdependency / mass dependency trend in NPM very irritating. As a rule of thumb I think you should be able to know at least roughly all your dependencies + transitive dependencies and take some care that they're healthy projects. If you can't name them all - you probably have too many of them.

I think the problem with forking is you can't take the 'lodash' noun on NPM. So you make a fork of lodash, fix the outstanding issues - but getting every single dependency to actually accept it is a whole 'nother mission. And as long as the original project 'owns' that noun, this is true for everyone.

Does npm have a facility to say a package provides another? Would npm resolve that? Or at least accept that if I have package lojack-mine and it "provides:lojack" then this dependency is resolved?

Yup, you got it; the migration/reclaiming of package names is pretty key here.

One tricky aspect of the 'metapackage' approach (where, for example, hypothetical 'lodash-legacy' or 'lodash-ng' packages could both fulfil a 'lodash' dependency slot) is that someone needs to be responsible for determining which packages can fill which slots.

In cases where the package alternatives are close forks and API-compatible, that's easier to agree on - but in a large ecosystem, the solution needs to scale in terms of human and machine review time, while also providing a good level of trust to developers when they are selecting or migrating dependencies.

There are likely to be some tricks to reduce the assessment workload required in each case -- running test suites from the 'original' package and applications which rely on it against each candidate, for example -- but ultimately either the package maintainers, the community, the NPM organization themselves -- or some mix of all of the above -- have to determine the packages that could fulfil each slot.

Good packaging is hard, and it's maintenance work rather than green-field development. We (collectively) really should incentivize and fund it more, and ideally learn from and credit existing solutions.

I'm sure there are people who enjoy reviewing (and automating review of) packages, knowing that they're improving code quality and performance for huge numbers of developers. And similarly, there are plenty of developers who provide useful feedback on package quality by reporting bugs, running benchmarks and writing qualitative comparisons.

To guess at where we may be heading in the near-term: a pure Microsoft-subsidiary ecosystem is developing in this area -- GitHub and NPM provide all the data and infrastructure to host, scan and gather user feedback for the processes mentioned above.

I think they'll collectively do a good job, although any approach does take time to build and deploy. I imagine it'll also raise longer-term questions around the governance of open source software development and what role Microsoft would like to take in that.

On a more abstract level the problem space relates to network effects and influence goals. Microsoft believes that the open source developer community is either here to stay, or is here at least for a sufficient duration of time for them to be important -- and it would like to provide influence and guide rails to that community.

> If you can't name them all - you probably have too many of them.

Well, I want a unicorn, too. That is simply not feasible for anyone using any common javascript/node frameworks - many people's builds started failing with literally thousands of `npm audit` failures, even when their direct dependencies were relatively small.

Github/Microsoft just bought NPM, I really hope they are able to add some features that make it easier to specify different resolution algorithms when pulling dependencies so it's easy for any individual to apply a patch up-and-down the dependency chain, even if the main project hasn't been updated.

It should be enough to know your direct dependencies.

The problems that can be caught by knowing transitive dependencies but can't be caught by knowing direct dependencies, are rare.

Perhaps open projects could make a pledge of some sort that guarantees important properties as long as all dependencies also commit to the same pledge.

I saw a great suggestion the other day -

> The npm ecosystem needs something like the distinction between Ubuntu's "main" and "universe" repositories, so that you have a smaller subset of known-good packages with harmonized transitive dependencies, stronger centralized curation, a lot more direct scrutiny, tighter control over versioning, and some party that is responsible for addressing vulnerabilities and ensuring appropriate maintainership.

https://news.ycombinator.com/item?id=23680734

Establish an initiate like ‘Ruby Together’

> Ruby Together is a grassroots initiative committed to supporting the critical Ruby infrastructure you rely on.

> Your financial backing allows Ruby Together to pay expert, professional developers to work on improving critical infrastructure projects.

> The Ruby Together board of directors, elected by members, selects the most impactful projects to fund.

> Ruby Together publishes monthly updates showing progress and our proven results.

It brings together companies and individuals concerned about stability and health of the ecosystem, provides a aggregated place to do high impact sponsorship of the ecosystem and the elected board then sets about making a detailed workplan and working out the best distribution of funds.

One dev on lodash? Discuss its importance; offer the dev funds to keep things up-to-date and if not them, have the resources to work out another way.

Check out their early reports to get a better idea of the initial refactoring work they did within the ecosystem.

https://rubytogether.org/

The approach embodied by package managers like npm - but also Python, Go, and others besides - is fundamentally broken. The software maintainers should not be the sole point of failure in publishing their package.

The approach used by distributions is much, much better. Having a separate team of people responsible for packaging up software is a much better approach. This would allow the security patches to get in, help enforce a sane baseline of packaging standards, prevent the introduction of malware, and have someone accountable whose interests align more with downstream than upstream.

Using a package manager like npm does not prevent having a separate team of people responsible for validating and re-publishing packages under their own namespace. A new organization could step in and make their own whitelist at any time.

It is a cultural problem.

Good question! One Python example is docopt. It is a very well-liked command line parsing library but haven't seen any maintainer activity in two years. https://github.com/docopt/docopt But people don't realize that so they keep filing bug reports and submitting pull requests.

There is a docopt-ng package here: https://github.com/bazaar-projects/docopt-ng But users haven't found it yet. And sadly, there is no guarantee that the new maintainer won't also lose interest and abandon the project.

Definitely not an ideal situation. Yes, you can fork but how do you get visibility? How can the developer announce to the community that he or she wants to maintain the software?

Really there just need to be more people involved in open source. The web is being built on the unpaid contributions of a few and when they don’t have time or mess up due to being rushed, piles of things collapse.

I suspect if a human resource audit of open source were done, there would be surprisingly few people involved in the maintenance of these things.

I think companies with deep pockets need to step up a bit. I mean if anyone at FAANG etc are using packages like lodash, they should ideally team up and create some type of a funded group/co-op that funds these libraries and their maintenance and removes dependency from one person. You cannot just "charity" the way out of these situations.

Completely agree. I hate that companies do not give back to this -- but it is changing and that's a positive.

Stripe & Segment have their open source grants, WhatsApp Co-Founder (Jan Koum) donated $1MM USD to FreeBSD, DuckDuckGo donates to companies since 2011 (https://duckduckgo.com/donations) etc.

At the company I have founded, I am putting my money where my mouth is -- we have a budget to allocate up to $25,000 per year to open source products that we find value in. As we hit certain revenue milestones, I have every intention of donating an even larger % to funding open source initiatives as part of our "giving back" corporate social responsibility.

It is a shame most companies do not do the same. I cannot change them, but I can control change within what I do. So, thank you to all the makers and builders out there, I will make it my personal mission to figure out a way to give back to you!

I want to do it at my company as well (albeit at a much smaller scale), but how? I asked previously without much luck https://news.ycombinator.com/item?id=23439670

(Not to mention try to get it as a taxable donation... which complicates things further)

I'm not sure about the taxable donation (did you perhaps mean a tax-deductible donation instead?) but I have a few ways. It looks like that thread links to a company that's specializing in that, were they able to solve your use case? If not, feel free to ping me and I can connect you w/ the folks on our team who have this as a key objective for them and have figured out most of the logistics.

I really appreciate you attempting to do this though, thank you!

Tax-deductible yep :) sorry about that

I was in contact with these guys but nothing came of it so far ... I think they’re still working on it but I don’t know. Would be great to find out options. Thank you for your help.

Side note: lack of funding for open source is a common complaint it seems, but I was personally surprised it’s actually not trivial at all to give money open source ...

I mostly try to reduce dependencies, both direct and indirect, regardless of the programming language in use.

If you're using yarn, apply the patch in your GitHub fork, bump the version, use a resolutions block to point to your repo (I think ref based doesn't work well, might be old info that it doesn't, though).

IMHO coordination based methods will work poorly here.