Hey all,
since Juniper is out there and we have started working on our migration, we started researching the different ways we could use to deploy MFEs. We made a summary document that you might find interesting.
If you have any comments or thoughts, I’ll be happy to hear them.
Perhaps @fredsmith or @djoy have some thoughts?
What I would like to see is an example of how to hook this up in a native Open edX installation for example. Just one example would give us ideas on how to install the other MFE.
@Felipe’s document is an excellent starting block, but I would like to see more detailed instructions for at least one MFE. What to add? Where? When? How? What to modify? Where? When? How?
As will be the case for the installation instructions in the BTR working group in the future, we may need a reference installation that can be used by most operators. Not everyone has the staff and the ressources of edX to deploy MFE as per https://openedx.atlassian.net/wiki/spaces/FEDX/pages/1364886433/Understanding+frontend+app+deployments
@Felipe and team: thanks for putting the doc together.
Regarding having multiple sub-domains, note that this is a temporary situation for what edx.org has implemented for itself. The long-term plan is to have a single domain for all MFEs and use URL paths to route to different MFEs.
So as the community designs a solution for MFE deployment for Open edX, you’ll be ahead of the curve (and edX.org can follow) if you jump to a single domain design.
@fredsmith started an OEP about routing, I’m not sure how thinking has shifted since then: https://github.com/edx/open-edx-proposals/pull/111/files
So, we have a couple in progress but not adopted at edx.org solutions for routing MFEs. The most polished one is conductor, which is implemented in ansible: https://github.com/edx/configuration/blob/master/playbooks/conductor.yml
This playbook deploys an nginx server that routes paths to MFEs hosted in s3 buckets. A sample configuration is:
################################################################
#
# Conductor Settings
#
CONDUCTOR_STATIC_SITES:
- proxied_path: http://account.stage.edx.org.s3.amazonaws.com/
router_path: account
- proxied_path: http://gradebook.stage.edx.org.s3.amazonaws.com/
router_path: gradebook
- proxied_path: http://orders.stage.edx.org.s3.amazonaws.com/
router_path: orders
- proxied_path: http://payment.stage.edx.org.s3.amazonaws.com/
router_path: payment
- proxied_path: http://portal.stage.edx.org.s3.amazonaws.com/
router_path: portal
- proxied_path: http://profile.stage.edx.org.s3.amazonaws.com/
router_path: profile
- proxied_path: http://program-manager.stage.edx.org.s3.amazonaws.com/
router_path: program-manager
- proxied_path: http://publisher.stage.edx.org.s3.amazonaws.com/
router_path: publisher
CONDUCTOR_REDIRECT_ROOT: true
CONDUCTOR_ROOT_REDIRECT_PATH: "/portal/"
Then running the conductor playbook (linked above) will route all your MFEs from whatever box you run it on.
The key here is that our MFEs use path based routing within their routes, so this nginx role handles some path based magic to serve the single page app:
{% for static_site in CONDUCTOR_STATIC_SITES %}
# Matches: <router>/<organization>/
location = /{{ static_site.router_path }}/ {
proxy_pass {{ static_site.proxied_path }}/index.html;
}
# Matches: <router>/<organization>/[.../]<file.ext>
location ~ ^/{{ static_site.router_path }}/((?:\w+\/+)*)([\w\-\.]+\.[\w\-\.]+) {
proxy_pass {{ static_site.proxied_path }}/$1$2;
}
# Matches: <router>/<organization>/<subpage>/[.../]
location ~ ^/{{ static_site.router_path }}/([a-z0-9-]+)[/]? {
proxy_pass {{ static_site.proxied_path }}/$1/index.html;
}
This isn’t a perfect solution, and our long-term goal is to use k8s and ingress to handle all of this, so we’ve kinda not made a decision on it. I would love to work with the community to find out what people need and how we can make this work across the board.
Doesn’t this defeat the purpose of S3, in terms of performance and CDN capabilities?
we use S3 because of resiliency and ease of deployment. Performance isn’t a motivation. We put a CDN (cloudflare) in front of our nginx servers to bring MFEs closer to the edge.
Thanks to the help of @morenol I have managed to create a working plugin for Tutor that deploys the Gradebook microfrontend application: https://github.com/overhangio/tutor-gradebook As all Tutor plugins, it’s really easy to install., so I encourage you to try it out (not in production, though :))
A demo is available here: https://grades.demo.openedx.overhang.io (login: admin@overhang.io, password: admin)
Creating this plugin has allowed me to get a first experience in MFE deployment. Generally speaking, it was fairly easy to create, apart from the usual head-scratching issues like CORS. Yet, I’d like to mention a few points where MFEs could be improved:
Documentation: currently, the list of environment variables that need to be set for building the gradebook is spread in multiple files: src/config/index.js and src/index.jsx. I would like to see the list of all required environment variables in a single place, where they would be properly documented.
Development: in the open-release/juniper.3 version, the webpack-dev server is run with host checking, such that it can only be accessed as 0.0.0.0, localhost or 127.0.0.1: https://github.com/edx/frontend-app-gradebook/blob/open-release/juniper.3/config/webpack.dev.config.js#L140 This is quite inconvenient, as in Tutor we access the gradebook at http://grades.local.overhang.io:1994. It would be great if the host that the dev server binds to were defined as a (documented) environment variable. (It is unclear to me if issue still happens in the master branch, as I am unfamiliar with the fedx-scripts executable: https://github.com/edx/frontend-app-gradebook/blob/aa39fcc7e045f231d93c7c8ae180e73cd0d16a69/package.json#L17)
Production: as discussed here, the web server needs to redirect the /<course-id> paths to /index.html. This seems like an anti-pattern, as it prevents having well-defined 404 error. Also, it forces operators to add specific intelligence to the web server, when one of the promises of microfrontends was that we only needed to setup a dumb fileserver. Instead of adding the course ID in the path (/<course-id>), we should be using querystrings (/?course-id=...).
These are fairly minor technical hurdles that I’m sure will be easily resolved. Again, deploying the MFE was a rather pleasant experience. I don’t expect that integrating other MFEs will present much more challenging difficulties.
Hi everyone!
We have made some progress at eduNEXT, thanks to @morenol!
His PR creates three roles to deploy the profile, grade book and account MFEs. It is possible to test this on a staging environment he set for this purpose, feel free to check it out, test it, and comment directly on the PR or on this thread:
User: staff@example.com
Pass: edx
User: honor@example.com
Pass: edx
As Luis mentioned on the PR, we are keen to receive the community feedback on this, and hopefully, clarify and stay aligned on the objective we want to achieve by the end of October in regard to the MFE deployments. Considering this, it could be useful to have a call to discuss this further and hopefully, as a result of the meeting, we will get a clear strategy and goal on this matter. I created a Doodle to reach a consensus about the day. The options I proposed are either Monday, Tuesday or Wednesday next week, 10-11 A.M, or 11-12 Eastern Time (ET), so please let me know which time works best for you all.
Hi!
Based on the Doodle results, I’ve scheduled the meeting Monday 10-11 A.M. Eastern Time (ET). Hope that fits everyone’s agenda. Just in case someone could miss the poll, here is the link of the meeting. Come join us!
Next time, include the time in the Doodle poll, please…
Thank you for joining the meeting, it was great discussion and collaboration. Here is the recording from this morning:
Same here. 
Sorry! I will 
Hi, updates on this:
There is a PR that fixes a problem with branding override in the role that we already created for MFE deployment:
https://github.com/edx/configuration/pull/6049
I created PRs against gradebook, profile and account MFEs with the latest version of frontend-build and frontend-platform (we should do that with all the MFEs that we plan to use in Koa).
with the credentials:
honor@example.com
edx
Short and sweet! I love these PRs 
Just a reminder that we really want everything merged by Nov 9th if we can. Do we know of work that won’t make it by then?
This is a happy little set of instruction to get a native install of open edX working with the new MFE deployments.
It will download the master branches of most repos and a few selected branches that are being considered to be merged before the Koa release.
It should take about 2~3 hours to do all this, but most time is unattended ansible work.
Shopping list:
I started this on a brand new t2.large on AWS Using the baseline ubuntu 16.
Storage disk should be at least 25GB. Go for 30 to be safe.
This is a public facing web server with ports 80 and 443 open to the world.
Public IP: 52.12.48.91
I created a DNS A record for this IP to the domain wildcard:
.mfe-example.publictest.edunext.co
For the SSL cert I used certbot:
sudo apt udpate
sudo apt upgrade
sudo apt install letsencrypt
sudo certbot certonly --standalone
Note: certbot failed due to deprecated python2 libraries. To fix it I did:
pip install requests urllib3 --force --upgrade
I used the cli wizard to create a certificate for domains (a wildcard would work just as well):
lms.mfe-example.publictest.edunext.co
studio.mfe-example.publictest.edunext.co
react-apps.mfe-example.publictest.edunext.co
Following the instructions on the draft PR (https://github.com/edx/configuration/pull/6128).
mkdir mfe_example
cd mfe_example
export OPENEDX_RELEASE=nedbat/tk.1106c
wget https://raw.githubusercontent.com/edx/configuration/$OPENEDX_RELEASE/util/install/ansible-bootstrap.sh -O - | sudo -E bash
wget https://raw.githubusercontent.com/edx/configuration/$OPENEDX_RELEASE/util/install/generate-passwords.sh -O - | bash
Note: edited on 09.11.2020 to use the nedbat/tk.1106c which fixes the elasticsearch problems and it is similar to what koa will be.
nano config.yml
# config.yml
################################
# general settings for ansible #
################################
module_lang: en_US.UTF-8
module_set_locale: True
########################
# edxapp role settings #
########################
EDXAPP_LMS_BASE_SCHEME: https
EDXAPP_LMS_BASE: "lms.mfe-example.publictest.edunext.co"
EDXAPP_CMS_BASE: "studio.mfe-example.publictest.edunext.co"
EDXAPP_SESSION_COOKIE_DOMAIN: ".mfe-example.publictest.edunext.co"
EDXAPP_CSRF_COOKIE_SECURE: true
EDXAPP_SESSION_COOKIE_SECURE: true
EDXAPP_CROSS_DOMAIN_CSRF_COOKIE_DOMAIN: ".mfe-example.publictest.edunext.co"
EDXAPP_CROSS_DOMAIN_CSRF_COOKIE_NAME: "cross-domain-cookie-mfe"
EDXAPP_CORS_ORIGIN_WHITELIST:
- studio.mfe-example.publictest.edunext.co
- react-apps.mfe-example.publictest.edunext.co
EDXAPP_CSRF_TRUSTED_ORIGINS:
- react-apps.mfe-example.publictest.edunext.co
EDXAPP_LOGIN_REDIRECT_WHITELIST:
- studio.mfe-example.publictest.edunext.co
- react-apps.mfe-example.publictest.edunext.co
# Needed to link the LMS instructor dashboard to the writable gradebook micro-frontend
EDXAPP_LMS_WRITABLE_GRADEBOOK_URL: https://react-apps.mfe-example.publictest.edunext.co/gradebook
# Needed to link to the new profile micro-frontend.
EDXAPP_PROFILE_MICROFRONTEND_URL: https://react-apps.mfe-example.publictest.edunext.co/profile/u/
# Needed to link to the new account micro-frontend.
EDXAPP_ACCOUNT_MICROFRONTEND_URL: https://react-apps.mfe-example.publictest.edunext.co/account
EDXAPP_LMS_SSL_NGINX_PORT: 443
EDXAPP_CMS_SSL_NGINX_PORT: 443
EDXAPP_ENABLE_CORS_HEADERS: true
EDXAPP_ENABLE_CROSS_DOMAIN_CSRF_COOKIE: true
#######################
# nginx role settings #
#######################
NGINX_ENABLE_SSL: yes
NGINX_SSL_CERTIFICATE: '/etc/letsencrypt/live/lms.mfe-example.publictest.edunext.co/fullchain.pem'
NGINX_SSL_KEY: '/etc/letsencrypt/live/lms.mfe-example.publictest.edunext.co/privkey.pem'
#####################
# mfe role settings #
#####################
MFE_BASE_SCHEMA: https
MFE_DEPLOY_COMMON_HOSTNAME: "react-apps.mfe-example.publictest.edunext.co"
# This will override the logo in the header with a dummy logo
MFE_NPM_OVERRIDES: ["@edx/frontend-component-header@git+https://github.com/edunext/frontend-component-header-edunext.git"]
# This 2 variables make the whole process select branches of the MFEs repos where the changes are applied:
# Will not be necessary once the following PRs have been merged:
# https://github.com/edx/frontend-app-gradebook/pull/145
# https://github.com/edx/frontend-app-account/pull/340
# https://github.com/edx/frontend-app-profile/pull/386
# A similar PR will be necessary for all mfes
MFE_DEPLOY_GIT_PATH: 'edunext'
MFE_DEPLOY_VERSION: 'lmm/build'
And now we launch the full deployment:
wget https://raw.githubusercontent.com/edunext/configuration/mfe/community_deployment/util/install/native.sh -O - | bash
Note 1:
This native.sh comes from the mfe/community_deployment branch. What this does differently is that it clones the mfe/community_deployment branch before doing the ansible install. This would not be necessary once the PRs 6088 and 6049 are merged in master.
Note 2:
Running this failed once due to some deprecated SSL library. Perhaps I broke it when I did the urllib3 upgrade.
Fixed it with:
sudo apt remove python-openssl
Note 3:
Installing the forum role did not work because of a known error in Using branches named nedbat/tk.1106c fixes the forumforum: initialize elasticsearch
What we did was to comment out the forum role in the mfe/community_deployment. This should not be a problem for anyone following this instructions after we already commented it. Just note that the forum will not work in this installation.
The site is live at: https://lms.mfe-example.publictest.edunext.co/
We now need a super user:
sudo su edxapp -s /bin/bash
cd edx-platform/
source ../venvs/edxapp/bin/activate
source ../edxapp_env
python manage.py lms createsuperuser
With this, we go to https://lms.mfe-example.publictest.edunext.co/admin, log in as a super user and turn on the necessary settings.
3.1: Site configurations:
-https://lms.mfe-example.publictest.edunext.co/admin/site_configuration/siteconfiguration/add/
I edited the example.com site to match lms.mfe-example.publictest.edunext.co
Site values:
{
"ENABLE_PROFILE_MICROFRONTEND":true,
"ENABLE_ACCOUNT_MICROFRONTEND":true
}
3.2: Waffle flags:
-https://lms.mfe-example.publictest.edunext.co/admin/waffle/flag/
Here you need to add the 2 following flags for “everyone”
This is it. Now you can visit:
-https://lms.mfe-example.publictest.edunext.co/
And log in with any of the usual demo users. Honor, Verified, Staff …
All the MFEs will be deployed in the react-apps.mfe-example.publictest.edunext.co domain.
For example:
Tagging the people in the call today for more visibility: @nedbat, @sambapete, @nimisha, @antoviaque, @pdpinch, @regis. And @djoy for good meassure.
A million thanks to @morenol for all the hardwork.
@Felipe and @morenol, thanks for putting all this together. Is it possible to make this similar in complexity to the existing installation instructions (https://openedx.atlassian.net/wiki/spaces/OpenOPS/pages/1969455764/Koa+Native+Open+edX+platform+Ubuntu+20.04+64+bit+Installation)?
We want to make it as simple as possible for people to install this software. How can we smooth these steps?