Deploying the gradebook MFE

Working Groups Build-Test-Release
1

Following up on the efforts made by @Felipe and his team, I’m trying to deploy the frontend-app-gradebook micro frontend (MFE). In particular, I’m trying to deploy this particular MFE with a home-made Tutor plugin.

I’m currently facing a head-scratching issue, and I was wondering whether I could get some help from the edX team. (poke @iloveagent57 @fredsmith?)

The LMS points to the Gradebook as follows:

if is_writable_gradebook_enabled(course_key) and settings.WRITABLE_GRADEBOOK_URL:
        section_data['writable_gradebook_url'] = urljoin(settings.WRITABLE_GRADEBOOK_URL, '/' + text_type(course_key))

(this particular piece of code was authored by @iloveagent57)

This means that instructors go to the gradebook at a URL similar to: http://gradebook.edx.org/course-v1:orgid+courseid+courserun

If an MFE is supposed to be a simple collection of static files served by a web server, how can that web server serve the “course-v1:orgid+courseid+courserun” file, as indicated by the above url? Are we supposed to provide some sort of specific configuration to the web server?

2

Hi @regis,

The web server should be configured to try to get the file from the path and if it does not exists return the index.html file.

3

Thanks for your answer @morenol!

Does that mean that the Gradebook never returns 404 errors?

4

Yes, there are some MFEs that process the path and in some cases render a 404 like page, for instance the profile MFE.

But I think that the gradebook MFE does not do that.

But in terms of HTTP status code, I am not sure but I would say that it never return that.

5

This definitely clarifies my issue, thanks a bunch @morenol .

Now for a more open-ended question: why do things this way? I see at least two major issues with this approach:

  1. It forces platform administrators to provide a “smart” configuration to a web server in front of the MFE.
  2. It prevents the MFE from returning 404 errors when they need to.

AFAIU both issues could be addressed by relying on querystring parameters.

1 Like
6

Yes, I am also worried about the first issue. I think that we can live with the second issue.

In order to solve the first problem, we could use something like ExpressJS

We can create a file in the root of the MFE repo, with this content (Maybe it can live in frontend-platform)

// run.js
const express = require('express')
const path = require('path');

const app = express()
const port = process.env.PORT || 3000 

app.use(express.static(path.join(__dirname, 'dist')));
app.listen(port, () => console.log(`MFE is live on port ${port}!`));

And then we can run a service with node run.js. Then we can use that with nginx or run that in a docker container.

I also think that it would help for the deployments in services like Heroku and Vercel.

What do you think @regis?

1 Like
7

The MFE 404 handling should be considered nascent/rudimentary as it stands today. I think there are a few things at play:

Client-side routing is assumed to exist in most MFEs, usually via a mechanism like react-router. This means that the same MFE index.html should be served at a variety of URLs. With a traditional web server, you’d use a wildcard or a partial path in your routes/URLs to serve the same HTML from all those paths, effectively delegating further routing to the client for all the paths that are potentially ‘owned’ by the frontend application. On the client side, it manages the URL without page reloads via standard browser APIs under the covers, and only hits the server’s routing again on a hard page refresh.

In the edX implementation of MFEs, we have no server to provide that wildcard route today, as we’re deploying the MFE as static files to something like S3. In edx.org’s case, we use CloudFlare’s routing to serve index.html at any path under the sub-domain for that MFE. Again - I want to just reiterate that this is fairly rudimentary and gets the job done. In my view, no one feels strongly that it
needs to be this way.

One place it does get interesting, though, are 404s and other error codes. In the case of something like the profile MFE, we don’t know whether a “/u/whoever” path is valid until after the MFE loads and has hit up an LMS API URL to try to get user data for that username. With the current setup, it’s impossible to properly return a 404 on the initial page request because we haven’t actually talked to a server yet, and the S3/CDN routing is certainly not smart enough to know whether a user exists in our system. In this case, once we’ve hit our LMS API and find out it’s a bad user name, the MFE renders a soft 404 page - with a 200 status code, unfortunately! - to indicate to the user that that user doesn’t exist. Given this “serverless” (API-only) approach, that’s the best we can do in this case.

That said, we can certainly show proper 404 pages (with proper status codes) for URLs we know are totally invalid for a given MFE… “/not/a/valid/url” in profile, for instance. Today that’d have to be configured via the storage service (i.e., S3, Cloudfront, CloudFlare, etc.) because they can intercept the request and return a status code with a custom error page - that’s the best we can do without a server in between… we’re using these storage service platforms as our ‘server’ w/r/t routes/status codes, so to speak.

Another option that we don’t do today would be to have routes in Open edX IDAs/micro-services that are responsible for serving MFEs’ index.html files. This would give us the most flexibility - server-side processing for bad routes (and possibly running some DB queries up front!), delegation to the client for everything else.

Personally, I think it’d be reasonable for us to get to the above; it’d involve a slightly stronger coupling of the MFEs to their respective backend micro-services, but I think it’d make up for that in flexibility and ease of configuration. The routing would all just be right there in the backend, rather than needing to be configured manually as part of your deployment/ops setup. Note that this also implies that MFEs would be served from paths on a given micro-service’s subdomain, instead of a separate MFe subdomain. In general, after talking with folks internally at edX and in the community, this may actually be more desirable than the current sub-domain situation. (Sub-domains could still be used by mapping a sub-domain at certain paths to an existing micro-service.) I’m curious whether there would be any use cases for folks needing to change/customize the paths for an MFE, assuming the default was something reasonable.

That’s the lay of the land, anyway. I think we should be open to making improvements here.

2 Likes
8

Thanks for your detailed answer @djoy!

I would very much like to avoid tighter coupling between the Open edX backend and MFEs. This might be relatively easy to implement in terms of development, but it introduces a thick layer of complexity in the deployment process. Cross-app dependencies are not fun to deal with in terms of deployment.

Instead, I’d like to suggest the following:

  1. Replace /<course-id> urls by /#<course-id>, and thus automatically serve index.html at these new urls.
  2. On invalid course or user ID, do not return a 404, but instead display an error message in the frontend. Such errors should not be 404 but 400 anyway.
  3. All urls that do not point to a file would be 404.

I don’t have a strong opinion on subdomains vs path routing, as I usually recommend my users to CNAME all *.lms.com domain names to lms.com.

9

I think #2 and #3 make sense/is basically what’s happening today in our deployments, but just isn’t documented. :+1:

As for #1, I feel pretty strongly that don’t think we can/should make that limitation… for a few different reasons.

Semantically, a hash is supposed to be an optional fragment that identifies a portion of a document, allowing the browser to “jump” down to the pertinent content. It identifies something subordinate to the primary resources. We use it in account settings for navigating to various sub-sections, for instance. Our URLs are semantically much more akin to that ‘primary resource’, and it feels important to be able to structure our URLs in a semantically meaningful way/use the tools available to us in the URL.

Like, it feels a bit wrong to me to modify/limit our URL schemas because of potential limitations/additional config in the routing layer outside the app… Especially when it’d impose a lot of limitations on the client-side which it otherwise handles gracefully these days; the URL management APIs are pretty good, and using hash routes in SPAs isn’t as necessary as I think it used to be.

It’d have a lot of downstream consequences: no usage of query strings in MFEs, much harder to use anchor tags, forcing us to overload hashes with multiple pieces of information, etc. Anecdotally I’ve also seen some libraries (both server and client-side) strip out or ignore hashes, forcing some weird workarounds, especially around redirects.

10

@djoy your arguments against using anchors in urls make a lot of sense. How would you feel about passing arguments in the querystring instead, such as in /?course-id=<course-id>? It’s certainly not as elegant as /<course-id>, but I believe it would preserve the semantical meaning.