Npm clean-install in mounted-directories.sh super slow on tutor nightly dev with mounted edx-platform

Development
1

I noticed running tutor dev launch on nightly with my local checkout of edx-platform mounted felt like it was taking forever.

I updated npm clean-install tutor/tutor/templates/jobs/init/mounted-directories.sh at d8993f629374b7d25214c1f3aba1a059da254176 · overhangio/tutor · GitHub to be npm clean-install -d to try to diagnose the issue and found:

  • Every package is a cache miss
  • A couple packages required reattempts which took absurdly long amounts of time
npm http fetch GET 200 https://registry.npmjs.org/kind-of/-/kind-of-3.2.2.tgz 968094ms attempt #2 (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/fill-range/-/fill-range-4.0.0.tgz 968094ms attempt #2 (cache miss)

968094ms - Over 16 minutes.

Compared to running

rm -rf node_modules
nvm use
npm clean-install -d

on my host machine where the entire command completes in seconds, and every package reports (cache revalidated)

2

Can you bind mount the .npm directory? That should let the npm ci call use the local cache in your containers.

3

Feanil, for confirmation, did you mean this folders | npm Docs when you mention .npm? But isn’t it confusing that in-container cache is not working as expected?

4

I tried mounting cache using tutor mounts add (docs), but ran into the issue of

# each service is added to a coma-separated list
tutor mounts add lms,cms,lms-worker,cms worker:/path/to/edx-platform:/openedx/edx-platform

This command line is a bit cumbersome. In addition, with this explicit form, the edx-platform repository will not be added to the build context at build time.

I instead worked around this issue by:

  • creating a cache directory inside my edx-platform directory
  • adding that directory to .gitignore
  • mounting edx-platform using the implicit form (tutor mounts add /path/to/edx-platform )
  • adding npm config set cache right before npm clean-install in mounted-directories.sh