stdlib icon indicating copy to clipboard operation
stdlib copied to clipboard
verified publisher icon stdlib-js

Installing single package very slow and causing installing huge amount of packets with different version

Open Busyrev opened this issue 2 years ago • 3 comments

Description

Encountered an error when: In empty project

> npm install @stdlib/random-base-normal
added 8175 packages in 5m
428 packages are looking for funding

it results size of node_modules: 355 MB (372 784 894 bytes) 95 305 Files 44 344 Folders

Its not only about random-base-normal, tried some other random generators, same problem, more than 8k packages installed

It looks that many versions of same packets were installed as dependencies, for example array-int8 installed 42 times

node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-arcsine\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-bernoulli\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-beta\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-betaprime\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-binomial\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-box-muller\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-cauchy\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-chi\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-chisquare\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-cosine\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-discrete-uniform\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-erlang\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-exponential\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-f\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-frechet\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-gamma\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-geometric\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-gumbel\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-hypergeometric\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-invgamma\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-kumaraswamy\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-laplace\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-levy\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-logistic\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-lognormal\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-minstd\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-minstd-shuffle\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-negative-binomial\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-pareto-type1\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-poisson\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-randi\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-randn\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-randu\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-rayleigh\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-reviver\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-t\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-triangular\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-uniform\node_modules\@stdlib\array-int8
node_modules\@stdlib\random-base-weibull\node_modules\@stdlib\array-int8
node_modules\@stdlib\utils-copy\node_modules\@stdlib\array-int8

in empty projet @stdlib/stdlib installs 191 packages, intead of 8k

> npm install @stdlib/stdlib
npm WARN deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
added 191 packages in 1m
54 packages are looking for funding

Reproduction

npm install @stdlib/random-base-normal

Expected Results

less than 8000 packages installed

Actual Results

added 8175 packages in 5m
428 packages are looking for funding

Version

No response

Environments

Node.js

Browser Version

No response

Node.js / npm Version

v20.3.1 / 9.6.7

Platform

Windows 10

Checklist

  • [X] Read and understood the Code of Conduct.
  • [X] Searched for existing issues and pull requests.

Busyrev avatar Dec 20 '23 15:12 Busyrev

:wave: Hi there! :wave:

And thank you for opening your first issue! We will get back to you shortly. :runner: :dash:

stdlib-bot avatar Dec 20 '23 15:12 stdlib-bot

@Busyrev Thanks for reporting this issue. We are able to reproduce on our side and have identified one of the key issues leading to larger than expected node_modules trees. We are working on a patch and will circle back once we have an update. Thanks again!

kgryte avatar Dec 20 '23 18:12 kgryte

@Busyrev We released a patch for one of the transitive dependencies of @stdlib/random-base-normal, which caused the number of dependencies to blow up like that. Assuming no package lock file being present and the node_modules folder being removed, re-installing @stdlib/random-base-normal should now cause 263 packages to be added instead of >8k. It is still a lot because currently, @stdlib/random-base-normal installs all of @stdlib/random-base. We have located the reason for this and are exploring solutions, but cannot make any promises with respect to a timeline yet.

Planeshifter avatar Dec 20 '23 22:12 Planeshifter

@Planeshifter Did we push a release which has addressed this issue and can this now be closed?

kgryte avatar Mar 21 '24 00:03 kgryte

@kgryte yes, the published random packages now (direct or transitively) depend on @stdlib/random-base-shared and don't pull in all of @stdlib/random anymore.

This issue should thus be considered resolved.

Planeshifter avatar Mar 21 '24 00:03 Planeshifter