Optimize LMS (perceived) webpage loading time

Development Architecture
1

Authors note: I’m posting this proposal in the name of OpenCraft GmbH.

Proposal

I’m proposing a method to optimize the LMS (perceived) webpage loading time, by enabling asynchronous Javascript and deferring the render-blocking CSS.

Deferring the render-blocking CSS

Deferring the render-blocking CSS with <link rel="preload"> is known to be quite an effective technique for minimizing the perceived time it takes for a webpage to start rendering its content.

Enabling asynchronous Javascript

Asynchronously loading (and executing) JavaScript (with async or defer) is known to be an effective technique for eliminating parser-blocking JavaScript. Normally the browser would have to stop/block parsing the webpage’s HTML, and instead download and evaluate JavaScript scripts before continuing.

Based on these selected references on async vs defer vs “normal” loading of JavaScript files, defer seems to be a better option than async for this proposal:

Proof Of Concept (POC)

To keep this proposal short(ish) I’ll allow myself a few provisions about this POC and its performance analysis:

  1. edx:ironwood.master github branch will be used as the baseline, and open-craft:petarmaric/bb-2221-lms-lazy-loading for its POC

  2. focus will be given to the LMS dashboard page

  3. a heavily customized LMS (non-public) theme will be used when benchmarking, as this proposal is a direct result of work previously done for an OpenCraft’s client

  4. all client-identifiable information have been redacted from the Lighthouse audit reports, as the OpenCraft’s client wishes to remain unnamed

The crux of this POC is relatively simple and straightforward, as seen in this JavaScript-focused commit, as well as its CSS counterpart.

The results of LMS dashboard mini-benchmarks are quite promising - please see the attached Lighthouse audit reports (baseline, poc) for details. I’d like to point out a few keypoints WRT the POC’s (perceived) performance:

  1. overall Lighthouse performance score has jumped from 13 to 62

  2. First Contentful Paint time has gone down from 8 seconds to 0.8

  3. all of the render-blocking resources (both CSS and JavaScript) have been eliminated

I don’t expect (nor want) the POC to be merged in its current state, I’m merely using it as a reference point when discussing the potential performance gains obtainable by implementing this proposal in full.

Potential issues

Deferring the render-blocking CSS

Although the render-blocking CSS technique has shown to be quite effective, it’s also as important to consider possible issues we may encounter:

  1. it’s known to cause temporary visual degradation while the page is being loaded, the so-called flash of unstyled content. This can be resolved by employing the so-called critical CSS technique to identify the minimal subset of CSS responsible for styling the initial above-the-fold content, and inline it directly into the HTML document. I do have to warn that this needs to be done for every LMS rendered webpage - ultimately it’s an extensive, manual and error-prone process suffering from similar issues as the JavaScript solution discussed in this proposal.

  2. <link rel="preload"> may not be as well supported in all modern browsers, and at the moment of this writing about 85% of global browser users can use this feature directly. There exists a JavaScript polyfill that can be used to enable this feature in the browsers that don’t yet support it natively.

Enabling asynchronous Javascript

When the POC was first implemented there were several Uncaught ReferenceError JavaScript errors on the LMS dashboard alone, but I’ve managed to resolve them manually.

Actually using defer blindly when downloading JavaScript files will be challenging at best, as there are quite a few of “eager” (IOW not waiting for DOM to load) inline JavaScript codes that make use of the (yet-to-be) downloaded JavaScript files, which is causing said Uncaught ReferenceError JavaScript errors.

Resolving this would require a large enough effort of manually going through each of the LMS rendered webpages, reviewing their inline JavaScript codes to check if they’re raising Uncaught ReferenceError JavaScript errors, and then fixing them on a case-by-case basis.

I don’t really see an automatic option to safely defer JavaScript inclusions in a simple nor partial manner - everything affects everything and there appears to be no single point of responsibility for generating/rendering JavaScript code in HTML within the edX codebase.

Therefor, if we do decide to proceed with this we’ll need to carefully review all of the LMS’s HTML and Python files for any potential emission of JavaScript code and then find a way of “lazyfying” it.

This “lazyfying” process is simple, albeit manual. The “discovery” process of what needs to be “lazyfyed” is what’s extensive, manual and error-prone IMHO.

Verifying (automatically) that the entire process was done properly is an issue in and of itself.

Conclusion

Implementing this proposal properly and fully will definitely be challenging, however the reward may very well be worth it.

Questions

  • should this proposal be implemented into Open edX?

  • do you see any issues with this proposal?

  • are there better alternatives you have in mind?

  • does this proposal have enough impact to require an OEP?

Please let me know if you’d like me to expand with more details.

4 Likes
2

Hi @petar, very nice post and very nice proposal.

I would like this to be implemented site wide, but It could also be done in a few pages that are the slowest and at the same time are being rendered more often. The Dashboard seems like a good starting point.

Now, most changes in your PR can already be done in a theme. If I wanted to test this side by side I could actually prepare 2 sites in a multi-tenant environment and only change the commits you added in the theme. All except:

for package in PIPELINE_JS.values():
package['extra_context'] = {
    'defer': True,
}

Could I do it without it? or conversely, could I add this without impacting the theme that has not changed the tags?

3

Thank you for the kind words @Felipe. To answer some of your questions (in no particular order):

The PIPELINE_JS-related snippet you’ve posted is actually the crux of the asynchronously loading (and executing) JavaScript effort. Without it applied, you’re really focusing only on the “deferring the render-blocking CSS” aspect of this proposal, which by itself still yields significant improvements to the LMS (perceived) webpage loading time. And like I’ve said in the proposal:

The LMS dashboard page would indeed be an ideal starting point for your performance analysis, judging from my (somewhat limited) experience with the edX platform.

And yes, most (if not all) of the “deferring the render-blocking CSS” aspect of this proposal should be doable within the context of a customized theme.