Local NPM registry for MFE builds

Working Groups
,
1

Hello, everyone!

I’m excited to share some recent POC that I’ve been working on to enhance our OpenEdX deployment tool, Tutor. We’ve recognized the need to optimize the build time and stability of Micro-Frontend (MFE) applications, which has led me to develop a simple Tutor plugin to bring a local npm registry to the MFE’s build step.

Key Benefits:

  • Faster Build Times: By hosting our npm packages locally, w reduce the dependency resolution time during MFE builds.
  • Increased Stability: A local npm registry mitigates issues related to external npm outages or network problems, ensuring a more reliable build process.

I’ve chosen Verdaccio as the local NPM registry for our proof of concept. The plugin implements the following features:

  • A local Docker Compose service with a volume for local npm caching: reference
  • A default NPM_REGISTRY URL: reference
  • Verdaccio Docker image declaration: reference

The outcomes of my local testing are as follows:

tutor images build mfe --no-cache
<verdaccio is filled by a cache>
<max 2 parallel processes>
took 12m 15s

tutor images build mfe --no-cache
<verdaccio is NOT filled by a cache>
<max 2 parallel processes>
took 13m 39s

tutor images build mfe --no-cache
<verdaccio is filled by a cache>
<no max parallel processes - use default docker context>
took 27m 17s

tutor images build mfe --no-cache
<verdaccio is disabled>
<max 2 parallel processes>
took 13m 32s

tutor images build mfe --no-cache
<verdaccio is disabled>
<no max parallel processes - use default docker context>
Build failed: getaddrinfo ENOTFOUND registry.npmjs.org
took 2m 44s

It seems that NOT having restrictions significantly degrades the performance of parallel builds. I’ll investigate it.

There are a couple of possible ways to use the proposed idea with Tutor:

  • Move the plugin logic directly to the Tutor MFE plugin.
  • Incorporate the idea into the Tutor core to be used by all services, not only by the MFE.
  • Implement the idea without the need to add the local registry to the Docker Compose declaration (I’m not yet sure how this can be implemented).

P.S. The plugin is ready for use in building the openedx image, as the NPM_REGISTRY variable is utilized in the openedx Dockerfile.

I encourage everyone to check out the work, provide feedback, and discuss how this can be further refined and utilized in your projects.

For a deeper dive into the implementation and to get involved with testing or enhancements, please visit the following link: Verdaccio Tutor plugin

@kmccormick @regis @MoisesGonzalezS FYI

Looking forward to your thoughts and feedback!

1 Like
2

Hi @Maksim_Sokolskiy!
I’m glad to see this effort to address MFE image building, which is a pain point of Open edX deployment.
I don’t understand your testings. I see little difference with Verdaccio disabled, filled or not filled, all around 12 to 13 minutes. Are these measures correct?

Another question: can it work in K8s?

3

@Andres.Aulasneo Thanks for reaction!

I’m also confused with the testing but posted as is.
The main issue with MFE build is that it mostly fails to build at all in case we forgot to set parallelism limitation. I’m trying to resolve this issue by providing an easy to use plugin.

Regarding the testing in general, I just posted the latest results, and it’s possible that some caching was involved. I’m eager to see testing by the community to gather more data.

k8s: the plugin doesn’t work with kubernetes yet since we want to test it. But in RG we use Verdaccio in kubernetes cluster as a shared registry for MFE builds.

P.S. sorry for the late response.

1 Like
4

Hi, the stats of testing by some of the Tutor maintainers are present on Test Verdaccio plugin for build improvements · Issue #220 · overhangio/tutor-mfe · GitHub. We will be adding more to it. Thanks

1 Like