Problems applying Customised MFE libraries (eg "frontend-platform") to a production open edx platform (using "tutor local")

Hello everyone.

I have been trying to make some modifications to the frontend-platform part of the MFEs so it will also support the Greek Language “el” (for the time being only a handful of languages are supported, I think just 14 of them), as, our platform runs in Greece in Greek language and without it we cannot keep up with the new Open Edx releases after tutor became the default installation method and MFEs are being used by default instead of the old django/mako templates in certain parts of the platform.

The changes needed to be made are minimal, just some minor additions to 3-4 files in the frontend-platform library.

After I bindmounted the forked MFEs’ code (eg the Learning MFE to my tutor dev installation) and edited the frontend-platform files directly inside the node_modules/@edx/frontend-platform path where they were installed, the Greek language worked without problems, but it seems impossible to transfer the changes to our “tutor local” open edx installation which we’ll use for our production platform.

First, I tried to use the following tutorial from here:
https://github.com/openedx/frontend-build#local-module-configuration-for-webpack

So I forked the frontend-platform to be able to make my tutor dev instalation use the forked one (so to edit its original code instead of the installed library inside node-modules/@edx/frontend-platform of each MFE), but it didn’t work due to some errors ending up in ignoring my “module.config.js” and using the locally installed frontend-platform inside each MFE’s node_modules.

I did the test with frontend-app-learning bindmounted to my tutor dev…

My “module.config.js” content is this:

module.exports = {
  localModules: [
    { moduleName: '@edx/frontend-platform', dir: '../src/frontend-platform', dist: 'dist' },
  ],
};

When I run “npm install && npm run build” it appears to be completed with just some warnings, although one of them seems to contain an error:


WARNING in ./node_modules/react-responsive/dist/react-responsive.js
Module Warning (from ./node_modules/source-map-loader/index.js):
(Emitted value instead of an instance of Error) Cannot find source file 'webpack:///dist/react-responsive.js': Error: Can't resolve './webpack:///dist/react-responsive.js' in '/home/user/WORK/PEK/OPENEDXREPOS/TMP/frontend-app-learning/node_modules/react-responsive/dist'
 @ ./node_modules/@edx/paragon/dist/index.js 106:0-102 106:0-102 106:0-102 106:0-102
 @ ./src/course-home/progress-tab/ProgressTab.jsx
 @ ./src/index.jsx 21:0-65 91:46-57

At the end I get this message: “webpack 5.50.0 compiled with 3 warnings in 29774 ms

Since it seems to not produce any errors I then try to mount the frontend-app-learning with “tutor dev start learning --mount=.” but I get the error trace shown in the following file:

https://gist.github.com/dagg/46fe73a107ae4c2c3773a9cd5926e858

That wasn’t a big problem though since the changes I wanted to make were minimal so I thought of just making the changes to my forked frontend-platform, pushing it to my github repo and then creating and enabling a tutor plugin with a “mfe-dockerfile-post-npm-install” patch to make my “tutor local” open edx platform use frontend-platform from my github repo instead of the original, following these steps from here:

https://github.com/overhangio/tutor-mfe#customising-mfes

My tutor plugin’s code is very simple as it’s shown here:
(it actually shows the path to the original frontent-platorm by open edx but, before trying that, I used my own repo)

from tutor import hooks

hooks.Filters.ENV_PATCHES.add_item(
    (
        "mfe-dockerfile-post-npm-install",
"""
RUN npm install '@edx/frontend-platform@git+https://github.com/openedx/frontend-platform.git'
"""
    )
)

This produced errors (when I used my own repo) and failed to even continue with the “tutor local quickstart” installation.

The error trace is shown here:

https://gist.github.com/dagg/bfbfade3f53cee0f45f9650c5a266073

To eliminate the possibility that I had errors in my changes of the frontend-platform library (since I could not test the changes before applying them to production for the reasons I explained firstly above), I tried to use the original repo of frontend-platform from github (as shown on the plugin code I have quoted right above), but I kept getting the same errors, so it wasn’t my changes that produced the errors but the method of applying them to the platform.

I have even tried using the exact same examples as shown in the help, using the following in my plugin:

First I tried to aply the original frontend-component-header from the npm repo:

RUN npm install '@edx/frontend-component-header@npm:@edx/frontend-component-header-edx@latest'

And then the frontend-component-footer from its github repo:

RUN npm install '@edx/frontend-component-footer@git+https://github.com/edx/frontend-component-header-edx.git'

(Actually, there is an error I think in this step as it is shown in the help, the second part directs to the “frontend-component-header” instead of the “frontend-component-footer”, I believe this must be corrected. Although I tried both and none worked with similar errors)…

Again, the errors are shown in the following corresponding files:

https://gist.github.com/dagg/7a779d8ea86343870231448a5278d91d

https://gist.github.com/dagg/19d211fa51671a15f72cbfb5de8db262

Because it’s very important for us to be able to use the new releases in our native language, I would really appreciate some help on how to overcome those problems and also know if it’s something I am doing wrong or there is something wrong with the method of editing those components and applying them to the platform.

It would be very important and helpful for anyone who needs to make certain changes to the MFEs and their libraries, to be able to apply those changes to their production platform with ease and without such errors or too complicated steps.

For the record: I have no problem editing one or more of the actual MFEs (eg the Learning MFE) and applying them to the production platform (tutor local open edx installation) by changing the value which contains their source repo (eg: MFE_LEARNING_MFE_APP).
To my knowledge so far, the problem appears only with such libraries as the “frontend-platform” that are used by those MFEs and are installed inside each MFE’s node_modules.

Moreover, I can make a PR with my changes on the frontend-platform for including the Greek language (el) but even if my PR will be accepted and included to following versions, I believe this would take time and I’d prefer for the time being a solution where I can apply these changes myself instead of waiting for the next releases, as it will be much harder to migrate our data from Lilac to the latest future open edx release.

The machine I am using for my tests has the following versions of tutor, docker, docker-compose, docker compose, in case they are of importance:

tutor, version 14.1.0
Docker version 20.10.21, build baeda1f
docker-compose version 1.29.2
Docker Compose version v2.6.0

Thank you in advance for any help on this matter! I hope it’s something trivial!

@regis @arbrandes @mrtmm @djoy

Hey @Dimitris_Angelakis,

You’re hitting some of the difficulties in using Tutor to develop MFEs locally. For the moment, most frontend developers still use the devstack, and that is what I suggest you do. The main advantage is being able to run an MFE outside a docker container (via regular npm start) and still have it connect to the rest of the Open edX instance. This allows you to configure webpack module aliases as instructed in the frontend-build README.

(@kmccormick has been hard at work at improving Tutor for development: you might want to try his quickdev plugin instead of the devstack.)

This doesn’t solve the issue of bringing the change into Tutor, of course. More on this below.

You’re on the right track regarding using your own repositories. That is the easiest way to bring changes into Tutor.

However, I don’t recommend a manual install of the latest versions of any npm packages unless you’re an experienced Open edX frontend developer; you’re very likely to run into the dependency errors you’re seeing. You should instead base your modifications on the exact version of a package that the version of the MFE you’re running installs, commit them, and then install that. This applies not only to frontend-platform, but also frontend-component-* and others.

You should also consider forking the MFE itself, and making the dependency modifications in its package.json. This will make it much easier to check that builds will run as intended. You can then point Tutor to your fork via:

tutor config save --set MFE_LEARNING_MFE_APP="{'name': 'learning', 'repository': 'https://github.com/youruser/frontend-app-learning', 'port': 2000}"

Nice catch: there’s clearly a copy-paste error in there. PR opened to fix it. :+1:

1 Like

I’ve been following Dimitris’ issues pretty closely as we’ve been working together on this matter. To the best of my understanding, there is an issue with installing frontend-platform from source. This is not a problem of installing the right version.

To understand what is going on, run the following commands in, for instance, the frontend-app-profile repo (open-release/nutmeg.master):

$ npm install --no-save @edx/frontend-platform@1.15.6 # this the required version in Nutmeg
$ ls node_modules/@edx/frontend-platform/
analytics  config.js      constants.js      i18n      index.js.map   initialize.js.map  logging       package.json  pubSub.js.map  README.md  utils.js
auth       config.js.map  constants.js.map  index.js  initialize.js  LICENSE            node_modules  pubSub.js     react          testing    utils.js.map

The installed package contains all the necessary files.

Now, install from the GitHub source repo:

$ npm install --no-save @edx/frontend-platform@git+https://github.com/openedx/frontend-platform#v1.15.6
$ ls node_modules/@edx/frontend-platform/
docs  jsdoc.json  LICENSE  node_modules  openedx.yaml  package.json  README.md  service-interface.png  src

Notice how in the latter example the package source files are all contained in the src folder. This causes npm run build to fail further down the road.

This leads me to conclude that there is a problem in how frontend-platform is packaged.

The fact that we cannot install frontend-platform from GitHub means that it’s extremely difficult to install a forked version of frontend-platform – which we need in some cases, such as installing an extra language/locale, which @Dimitris_Angelakis is trying to do.

How are frontend-platform developers installing their local forks in the MFE environments?

This is in the README:

If you look at the build process it becomes a little clearer why these acrobatics are necessary, and why npm-installing directly from github might fail.

So… Yes, this makes customizing frontend-platform harder. It’s why I offered forking both the MFE and frontend-platform as a potential way out. At least it would help tracking the problem down.

I have a lot on my plate this week, but as soon as I get some time I can help look into a fool-proof, Tutor-flavored way of doing this so that we can put it in the README. (If anybody else wants to pitch in earlier, please do. :slight_smile:

Thanks for this detailed answer Adolfo, I appreciate it. With your help I’m sure we’ll be able to move forward.

Indeed the build process explains why we can’t install from GitHub, but I still don’t understand why the build process needs to be so complex. Having to manually cp and rm stuff around does not feel “right”. Of course this is a question that is less of an emergency.

Agreed. I’m going to try and see if there’s a better way to do it.

Adolfo and Regis thank you both so much for the help on the matter!

I should probably mention that I’m pretty new to node (and react) and I’m struggling a bit to understand how the whole procedure works (especially in combination with the tutor way of implementing things) and most of the time I’m not sure I know what I’m doing, I just try to follow the existing README help if available.

I tried lots of things the last few days after your suggestion but they don’t seem to work or I do something wrong.
I actually used “nutmeg.2” tag code of "frontend-app-learning" and tag v1.15.6 of "frontend-platform" (I saw in frontend-app-learning'spackage.json’ file that’s the version of "frontend-platform" used, like Regis mentioned already) without changing their code (just to test if it will work), and I tried to incorporate them to my production (tutor local) platform but I am still receiving some errors. I even disabled the other three MFEs so they won’t “complain” about the changes, using something like:

tutor config save --set MFE_ACCOUNT_MFE_APP=null

but still no luck (I’m not sure though if disabling them it also blocks them from being build in there)…

I also noticed that the other MFEs, instead of v1.15.6 of "frontend-platform" they use v1.15.1 which complicates things a bit more since it seems I cannot use a different version for each MFE using this method…

This seems to be the solution to the different version of "frontend-platform" in each of the MFEs, but when I tried it with frontend-app-learning again it failed…

If i understand well this is for the development procedure (as it is titled in the README), right? Still I also hit the wall when I tried it but I didn’t insist on this, as my main problem is with pushing my changes in "frontend-platform" in production, and not so much the development part.

Still I really appreciate all those suggestions and help and I will keep trying in case I missed something due to my limited experience on the matter!

This would be a huge help for us, especially if you manage to describe, not only the development part but also the “Tutor-flavored way” of applying any changes on frontend-platform (or other components) to a "tutor local" production implementation of the platform.

@arbrandes & @regis Thank you for all the help!

1 Like