ModuleStore and the XBlock runtime are the foundation that basically all course team authored content runs on in Studio and the LMS. It’s also been notorious for its complexity, odd edge cases, and mysterious memory leaks. This was the part of the system that teams avoided touching if at all possible, and doubled their estimates when they did have to dive in. If you’ve been an Open edX developer for a while, chances are that you already know about all of this.
What you might not realize if you haven’t been paying close attention for these past couple of releases, is that it’s come a long way. This past week, we achieved yet another milestone in this effort, specifically with respect to how the XBlock runtime is initialized for a user and when XBlock runtime services are created.
In the Olive release, XBlock runtime service initialization happened in get_module_system_for_user, a 339 line function with seven inner functions. Each separate XBlock got its own runtime object with its own set of XBlock runtime services, with initialization happening over and over for every block. The vast majority of this work was wasted, recomputing the same things over and over again, while being susceptible to memory leaks.
When runtime initialization actually differed between blocks, it was needlessly clever. Yes, we could have the XBlock runtime magically give three older XBlock types a special User service that returns the older style of anonymous user IDs that they need for backwards compatibility purposes. But it’s much simpler to understand if the User service simply generates both styles of anonymous user IDs and the three older XBlocks (all bundled in edx-platform) explicitly ask for the style they want.
The current version of get_module_system_for_user is prepare_runtime_for_user, a 160 line function that much more clearly states its purpose. This function is only called once per user in order to initialize the runtime, and the services are not recreated and reinitialized potentially thousands of times when working on large courses. It’s more efficient and it’s much easier to understand what’s happening.
What’s also exciting is that larger refactoring and simplification work has led to the removal of some head-scratching edge case code along the way. For instance, the snippet to show the debug staff info under XBlocks went from this:
To this:
To be clear, it’s not all rainbows and butterflies. There is still some complicated stuff here, and interesting edge cases remain. But it is vastly more approachable now than it has been at any time in the last ten years, and I would honestly encourage folks to go take a look at that module now.
OpenCraft has been doing a lot of this work over the past year, first through the edX-funded BD-13 initiative, and more recently through the Axim-funded FC-0026. Through both efforts, edX has provided operational support, monitoring, and debugging help. For this latest round of work, I would like to especially call out:
- @Agrendalath for his deep knowledge and painstaking coding work.
- @braden for his insightful reviews during the development process.
- @0x29a for his insightful reviews and testing.
- @jristau1984 and the whole TNL team at 2U for their patience, communication, monitoring, and debugging help.
This latest work was only possible because of the excellent, much larger work done in BD-13. I’m probably going to miss some names here (please correct me), but in addition to the folks already mentioned, there was at least:
This is not glamorous work, but it is absolutely critical to the long term health of the platform. I am so happy to see us making solid progress here, and so incredibly optimistic about what else we can accomplish when Axim, 2U, and community providers collaborate in this way. Thank you all.
Cheers! 