Deploying MFEs in the community

Hey all,

since Juniper is out there and we have started working on our migration, we started researching the different ways we could use to deploy MFEs. We made a summary document that you might find interesting.

If you have any comments or thoughts, I’ll be happy to hear them.

4 Likes

Perhaps @fredsmith or @djoy have some thoughts?

What I would like to see is an example of how to hook this up in a native Open edX installation for example. Just one example would give us ideas on how to install the other MFE.

@Felipe’s document is an excellent starting block, but I would like to see more detailed instructions for at least one MFE. What to add? Where? When? How? What to modify? Where? When? How?

As will be the case for the installation instructions in the BTR working group in the future, we may need a reference installation that can be used by most operators. Not everyone has the staff and the ressources of edX to deploy MFE as per https://openedx.atlassian.net/wiki/spaces/FEDX/pages/1364886433/Understanding+frontend+app+deployments

@Felipe and team: thanks for putting the doc together.

Regarding having multiple sub-domains, note that this is a temporary situation for what edx.org has implemented for itself. The long-term plan is to have a single domain for all MFEs and use URL paths to route to different MFEs.

So as the community designs a solution for MFE deployment for Open edX, you’ll be ahead of the curve (and edX.org can follow) if you jump to a single domain design.

2 Likes

@fredsmith started an OEP about routing, I’m not sure how thinking has shifted since then: https://github.com/edx/open-edx-proposals/pull/111/files

So, we have a couple in progress but not adopted at edx.org solutions for routing MFEs. The most polished one is conductor, which is implemented in ansible: https://github.com/edx/configuration/blob/master/playbooks/conductor.yml

This playbook deploys an nginx server that routes paths to MFEs hosted in s3 buckets. A sample configuration is:

################################################################
#
#  Conductor Settings
#
CONDUCTOR_STATIC_SITES:
  - proxied_path: http://account.stage.edx.org.s3.amazonaws.com/
    router_path: account
  - proxied_path: http://gradebook.stage.edx.org.s3.amazonaws.com/
    router_path: gradebook
  - proxied_path: http://orders.stage.edx.org.s3.amazonaws.com/
    router_path: orders
  - proxied_path: http://payment.stage.edx.org.s3.amazonaws.com/
    router_path: payment
  - proxied_path: http://portal.stage.edx.org.s3.amazonaws.com/
    router_path: portal
  - proxied_path: http://profile.stage.edx.org.s3.amazonaws.com/
    router_path: profile
  - proxied_path: http://program-manager.stage.edx.org.s3.amazonaws.com/
    router_path: program-manager
  - proxied_path: http://publisher.stage.edx.org.s3.amazonaws.com/
    router_path: publisher
CONDUCTOR_REDIRECT_ROOT: true
CONDUCTOR_ROOT_REDIRECT_PATH: "/portal/"

Then running the conductor playbook (linked above) will route all your MFEs from whatever box you run it on.

The key here is that our MFEs use path based routing within their routes, so this nginx role handles some path based magic to serve the single page app:

 {% for static_site in CONDUCTOR_STATIC_SITES %}

    # Matches: <router>/<organization>/
    location = /{{ static_site.router_path }}/ {
        proxy_pass {{ static_site.proxied_path }}/index.html;
    }

    # Matches: <router>/<organization>/[.../]<file.ext>
    location ~ ^/{{ static_site.router_path }}/((?:\w+\/+)*)([\w\-\.]+\.[\w\-\.]+) {
        proxy_pass {{ static_site.proxied_path }}/$1$2;
    }

    # Matches: <router>/<organization>/<subpage>/[.../]
    location ~ ^/{{ static_site.router_path }}/([a-z0-9-]+)[/]? {
        proxy_pass {{ static_site.proxied_path }}/$1/index.html;
    } 

This isn’t a perfect solution, and our long-term goal is to use k8s and ingress to handle all of this, so we’ve kinda not made a decision on it. I would love to work with the community to find out what people need and how we can make this work across the board.

Doesn’t this defeat the purpose of S3, in terms of performance and CDN capabilities?

we use S3 because of resiliency and ease of deployment. Performance isn’t a motivation. We put a CDN (cloudflare) in front of our nginx servers to bring MFEs closer to the edge.

Thanks to the help of @morenol I have managed to create a working plugin for Tutor that deploys the Gradebook microfrontend application: https://github.com/overhangio/tutor-gradebook As all Tutor plugins, it’s really easy to install., so I encourage you to try it out (not in production, though :))

A demo is available here: https://grades.demo.openedx.overhang.io (login: admin@overhang.io, password: admin)

Creating this plugin has allowed me to get a first experience in MFE deployment. Generally speaking, it was fairly easy to create, apart from the usual head-scratching issues like CORS. Yet, I’d like to mention a few points where MFEs could be improved:

  1. Documentation: currently, the list of environment variables that need to be set for building the gradebook is spread in multiple files: src/config/index.js and src/index.jsx. I would like to see the list of all required environment variables in a single place, where they would be properly documented.

  2. Development: in the open-release/juniper.3 version, the webpack-dev server is run with host checking, such that it can only be accessed as 0.0.0.0, localhost or 127.0.0.1: https://github.com/edx/frontend-app-gradebook/blob/open-release/juniper.3/config/webpack.dev.config.js#L140 This is quite inconvenient, as in Tutor we access the gradebook at http://grades.local.overhang.io:1994. It would be great if the host that the dev server binds to were defined as a (documented) environment variable. (It is unclear to me if issue still happens in the master branch, as I am unfamiliar with the fedx-scripts executable: https://github.com/edx/frontend-app-gradebook/blob/aa39fcc7e045f231d93c7c8ae180e73cd0d16a69/package.json#L17)

  3. Production: as discussed here, the web server needs to redirect the /<course-id> paths to /index.html. This seems like an anti-pattern, as it prevents having well-defined 404 error. Also, it forces operators to add specific intelligence to the web server, when one of the promises of microfrontends was that we only needed to setup a dumb fileserver. Instead of adding the course ID in the path (/<course-id>), we should be using querystrings (/?course-id=...).

These are fairly minor technical hurdles that I’m sure will be easily resolved. Again, deploying the MFE was a rather pleasant experience. I don’t expect that integrating other MFEs will present much more challenging difficulties.

1 Like

Hi everyone!

We have made some progress at eduNEXT, thanks to @morenol!

His PR creates three roles to deploy the profile, grade book and account MFEs. It is possible to test this on a staging environment he set for this purpose, feel free to check it out, test it, and comment directly on the PR or on this thread:

User: staff@example.com
Pass: edx

User: honor@example.com
Pass: edx

As Luis mentioned on the PR, we are keen to receive the community feedback on this, and hopefully, clarify and stay aligned on the objective we want to achieve by the end of October in regard to the MFE deployments. Considering this, it could be useful to have a call to discuss this further and hopefully, as a result of the meeting, we will get a clear strategy and goal on this matter. I created a Doodle to reach a consensus about the day. The options I proposed are either Monday, Tuesday or Wednesday next week, 10-11 A.M, or 11-12 Eastern Time (ET), so please let me know which time works best for you all.

4 Likes

Hi!
Based on the Doodle results, I’ve scheduled the meeting Monday 10-11 A.M. Eastern Time (ET). Hope that fits everyone’s agenda. Just in case someone could miss the poll, here is the link of the meeting. Come join us!

Next time, include the time in the Doodle poll, please…

3 Likes

Thank you for joining the meeting, it was great discussion and collaboration. Here is the recording from this morning:

1 Like

Same here. :slight_smile:

Sorry! I will :sweat_smile:

Hi, updates on this:

  1. frontend-build and frontend-platform where updated to support the deployment in subdirectories, you can check these PRs for reference:


  1. There is a PR that fixes a problem with branding override in the role that we already created for MFE deployment:
    https://github.com/edx/configuration/pull/6049

  2. I created PRs against gradebook, profile and account MFEs with the latest version of frontend-build and frontend-platform (we should do that with all the MFEs that we plan to use in Koa).



  1. There is a PR in configuration that allows the deployment of MFEs in subdirectories (I tested that using the versions of the MFEs in the PRs above), and it can be checked in:

with the credentials:
honor@example.com
edx

5 Likes

Short and sweet! I love these PRs :slight_smile:

Just a reminder that we really want everything merged by Nov 9th if we can. Do we know of work that won’t make it by then?