OEP-65 adjacent: A frontend architecture vision

Hi all - this post is a continuation of the OEP-65 next steps/visioning post over in Announcements/Architecture. If this is unfamiliar, there’s some required reading there and in OEP-65.

It’s long. Get comfy. There’s a lot to talk about!

I’m deep in planning migration strategy for OEP-65, trying to get to the point where I can write some ADRs on how we should proceed. But it’s complex, deep, and broadly impactful work, and I think I need to bring some other folks along on the ride with me or the ideas I want to throw around aren’t going to make much sense when I share them.

This post is about more than OEP-65, and is going to challenge the way things have been done in the past. It’s both aspirational and, I think, eminently attainable. The reference implementation of OEP-65 is groundwork for this broader vision, and some of the architectural decisions for OEP-65 will make the most sense when viewed in this context.

So with that foreword, come on a little journey with me!


These fundamental assumptions underpin everything that follows. If we don’t agree on these, then this vision unravels.

Unified platform library

This one is likely the most debatable of the three assumptions. But we’ve had a lot of discussion and affirmation that we believe that in implementing OEP-65 we will want the new shell application to be in the same repository as frontend-platform, frontend-build, the header, footer, plugin framework, etc.

The rationale is that:

  • We want to reduce dependency management overhead.
  • We want to be able to develop MFEs in the shell from their own repo, which implies the MFE can use the shell and frontend-build together as a dev dependency.
  • We expect the lines between these libraries to become more blurry.
  • We expect them to be used together as a cohesive platform, not consumed piecemeal as we suspected 6 years ago.

I think they should be in the same repository and released as one unified platform library. The independent repositories have not proved to be worth it, it will be harder to modify them in place, and they will make it significantly more difficult to work with them in the future as these libraries converge.

Keep it simple. Most operators don’t need micro-frontends and module federation

From the perspective of big companies, micro-frontends make a lot of sense. You divide your frontend up so that independent teams can work with parts of it with autonomy. As we’re discovering now, it’s also helpful - if you need micro-frontends - to be able to compose them together in the page and share resources/dependencies between them, which is the whole point of OEP-65.

But any individual or small group operating an Open edX instance derives virtually zero value from MFEs and module federation. I’d guess that the vast majority of operators fall into this category. MFEs and module federation both significantly increase operational overhead and complexity in favor of ‘autonomy’ these operators don’t care about.

The Open edX platform’s default operating mode should align itself with the needs of these operators, which will increase adoption and stickiness, reduce frustration, and improve time to value.

We still want a small, stable core, open for extension, closed for modification

This is a central tenet of the Open edX architecture which we’ve been aspiring to build for years. We should continue to drive our platform toward this vision, we should reject choices that cause us to drift away from it, and we should seize opportunities to make it a reality.

Background and Context

As we get into this post, I’m going to be suggesting a bunch of big (sounding) changes. I arrived at these proposals by imagining the various alternatives and how they align with the base assumptions above.

How the shell and unified platform library changes things

So imagine the new shell application in its most basic form. If you’ve been following along, it will look very much like the POC ‘host’ app referenced in OEP-65.

  • It acts as a host for other MFEs.
  • Calls frontend-platform’s initialize() once up front so the guest MFEs don’t have to.
  • It owns the header/footer.

Like today’s MFEs, you would git clone its repository somewhere and run the build command on the source code.

We could continue to do this. But remember… we want that unified platform library, and we want it to include the shell. Going along with this, we’ve investigated monorepos enough to know we want to avoid them, and envision the unified platform library as a single package.

That means to build the shell, you would need to check out the entire unified frontend library, which is the small stable core of the Open edX frontend. That flies in the face of our desire for a small extensible core that’s closed for modification, and we’d need to go out of our way to remind operators not to mess with it lest they destabilize their instance and make upgrades more difficult. It’s wild from a DevEx/usability/best practices/encapsulation perspective.

We can and should do better.

By the way, our frontend customization/build workflows are an afterthought

Regardless of where the ‘core’ code is, to work with the Open edX frontend in a general sense, operators need to minimally modify the configuration files that it uses for its dev and prod builds. They optionally need to add customizations, plugins, and a brand.

This is the primary way operators and developers interact with the framework outside of getting it hosted on the internet.

What guidance and best practices do we provide operators on how to do this? No pattern or best practices on how to organize your code that configures and customizes the frontend are provided. We mostly leave operators to figure it out on their own. Big respect to Tutor for tackling this with the Tutor MFE plugin or I’m not sure anyone would ever figure it out.

What edX did

As an operator of the open-source software, edX navigated our current architecture of git clones, configuration and customization by building a fancy, bespoke build process around tubular.

Tubular and the deployment pipeline behind it:

  1. Sets up a workspace in a build job.
  2. Git clones all the frontend code and installs its dependencies.
  3. Pulls config files from a separate private configuration repository and combines them with the code.
  4. Runs npm install a bunch of times to replace dependencies in package.json with any customizations.
  5. Runs the build command to create an artifact with those customizations.
  6. Ships the built dist off to the web to be served.
  7. Deletes the ephemeral ‘dirty’ git checkout after the build finishes.
  8. Starts over for the next build.

I imagine others are doing similarly, and I expect Tutor is also doing a lot of work to abstract away this process.

If we focus on how configuration and customization is done (steps 2 to 5), even implementing this and figuring out how to arrange everything is confusing. Documentation is spotty at best. Since you’ve checked out the platform’s source code and then overridden its dependencies, you’ve created a ‘dirty’ checkout that you can’t reasonably check in anywhere without effectively forking those core MFE repositories.

We treat configuring, extending, building and deploying the code as a poorly supported afterthought when it’s the main thing everyone needs to do!

Again, we can and should do better.

Case studies and prior art - Next.js and Wordpress

To draw a parallel, if you want to make an app with Next.js, you don’t check out the Next.js source code and work with it directly. You certainly don’t fork it and maintain your own copy. You create a project that has next as a dependency. They make this easy with helpers like create-next-app. You put your config and customizations in that project, and you check it in as a cohesive unit that represents “your app” and what makes it unique. Almost all modern frontend meta-frameworks (Next.js, Remix, Astro, etc.) work this way.

Wordpress is a very notable exception. In Wordpress (not the Saas offering) you actually do fork the complete platform and save your own copy of it. But this comes with a big caveat. The first line in Wordpress’ documentation on Why We Make Plugins is:

If there’s one cardinal rule in WordPress development, it’s this: Don’t touch WordPress core.

If they were making Wordpress today, it’d probably make sense to just make that core a dependency so folks aren’t confused. It’s important to remember where Wordpress came from. It was created in 2003, 5 years before Github existed (2008) and 7 years before npm (2010). A lot has changed in the past 21 years, and the ways in which we share and distribute software have evolved significantly. To make up for a lack of package management and package registry options, Wordpress has invested significantly in enabling operators to upgrade via migration scripts and programmatic updates to sites. They presumably have significant staff dedicated to this. Wordpress’s architectural choices have a whole lot of historical baggage which we don’t need to emulate.

I’m a broken record - we can and should do better.

Proposal: Project mentality

All of this is to say, the status quo of how we ask operators to do things today is a bad user experience. Module federation and the architectural choices around the unified platform library are going to exacerbate the problem by exposing even more source code to operators. We need to invest in making this better, and I believe it needs to happen as a fast-follow or in concert with OEP-65.

I believe this looks like shifting our mental model and thinking of the Open edX frontend platform as being akin to Next.js, Remix, Astro, or any other modern meta-framework. We should embrace a “project” mentality instead of a “check out all the source code and modify it” mentality.

The goal of Open edX pluggability and extensibility has always been, implicitly, to enable this. We’ve been talking about “a small stable core, open for extension but closed for modification” for years in our architectural talks. If we don’t expect folks to modify the core, we should stop waving the source code in their face! We should also bring their config and customizations front and center.

What is a project mentality? What do all these frameworks, including Wordpress, have in common?

There’s certainly lot to admire in Wordpress’ plugin framework, as we’ve all noted here on the forums in the past. Let’s step up a level from hooks and filters for the moment, though. If we look at how you create a plugin in Wordpress’ docs, and also think about what you get when you start a new Next.js, Remix, or Astro site… they’re actually pretty similar if you ignore where they put their internal source code.

  • In all of these cases, you create a new project directory that’s yours to do with as you please and check into source control somewhere.
  • You configure the platform with config files (like next.config.js or wp-config.php).
  • You put your “customizations” (i.e., your app) in a sub-directory of that project, whether it’s src or Wordpress’ plugins directory.
  • The framework knows how to find your source files and build them into the underlying platform, extending it with your functionality. In the case of Wordpress, this is about extending a significant body of functionality in terms of its CMS, whereas Next.js and friends are more of a blank canvas.

In all these cases, you create a project and put your customizations and configuration in it, and that’s the thing you check in - it represents your site. I think we can follow this pattern and have Open edX frontend projects, and I don’t think we’re that far off from it.

The unified platform library isn’t the project - we established we don’t want you to have to check out all that internal source code. Instead, it can be a dependency of your project, which is a new thing for us, and brings the primary way operators interact with the platform front and center, rather than leaving it as an afterthought.

Open edX frontend projects

Enter the Open edX frontend project. Hopefully you’re still with me. It represents the frontend of your entire site, and minimally consists of:

  • A directory with a package.json that depends on the unified frontend platform.
  • A set of “scripts” in package.json to build your project via a CLI (command line interface) provided by the unified platform dependency, much like fedx-scripts today, or the CLIs provided by any of these modern meta-frameworks.
  • A few environment-specific configuration files: openedx.prod.config.js, openedx.dev.config.js, etc.
  • An .eslintrc.js, tsconfig.json, and a few other config files for the build/dev tooling. Maybe a Webpack config if you need to customize it.
  • An index.html file you can customize.

As a best practice, you and your organization check this project in somewhere, just like you would any other project you created using a frontend (meta-)framework (Next.js, Wordpress, React, Remix, Angular, whatever). The code is yours, and by default it’s tiny and is pre-configured to build the core product offering. (This is where we’re more like Wordpress than the meta-frameworks. Open edX is not a general-purpose blank canvas, we provide significant functionality and features out of the box.)

You don’t git clone any part of the Open edX frontend core at all. Everything you need is a versioned dependency in package.json (unless you need something more advanced, which we’ll get into). The only things you ever check out are your own code.

As mentioned, by default it’s configured to load the frontend core product apps (i.e., authn, profile, account, learning, course-authoring, learner-dashboard, etc.). It does this as direct plugins via frontend-plugin-framework’s mechanisms. You don’t need to do anything to set this up - it just works, they load seamlessly via one build command, and as part of a single unified deployment to the internet. Again, we are not actually that far off from this!

What does the unified platform library provide?

When you run the build via the CLI, the ‘shell’ application code is used as the entry point, and the shell is responsible for loading the config file in your project which then loads your customizations. The shell lives in the unified platform library, along with what’s currently frontend-platform, frontend-build, the header, footer, and frontend-plugin-framework. The shell is akin to Next.js’s underlying web server or the Wordpress core. The unified platform library also provides the recommended ESLint, Jest, TypeScript, and Webpack configurations to your project much like frontend-build does today.

To summarize, the unified platform library is released as a single npm package with a few exports for different uses:

  • A CLI to use in package.json to run a build, start a dev server, etc., like fedx-scripts today.
  • A ‘runtime’ library for use in your customizations, like frontend-platform/frontend-plugin-framework today.
  • A set of default config files for ESLint, Jest, TypeScript, and you can pull into your project and extend as necessary in the same way that frontend-build works today.

Most of this already exists - we’re just going to package it differently and expand on the functionality by creating some more options.

Working with customizations and extensions

To customize your project beyond the configuration options, you start adding sub-directories to the project directory for your brand, custom plugins, customized replacements for apps (if you really need to), new apps, etc.

You import these into the project via the config files. Many operators won’t need anything more flexible than this, but if your organization needs more independence, you can put them in separate repositories and load them at runtime via module federation or as package.json dependencies.

The CLI will also provide commands to bootstrap your project and these sub-directories, similar to create-react-app or create-next-app:

  • create-site: creates a new project directory.
  • create-plugin: creates a new plugin stub as a sub-directory in your project.
  • create-brand: creates a new brand based on brand-openedx.
  • create-app: creates a new app (MFE in today’s terms), which is a bit different than a plugin because it exists as a top-level route in your site.

And maybe some others as necessary. These will take the place of frontend-template-application and automate the process.

Once you run one of these commands, you write your custom code right there in the project in a sub-directory. As mentioned, we use the config files to import it into the site. If we were to get imaginative, we could even have the build crawl these sub-directories and import them automatically using metadata co-located with the code, which is effectively what Wordpress and Next.js do.

You check it in, along with your environment-specific configuration files, and just run the build as a single command in your build system. No weird npm aliases or combining source code with config files checked in somewhere else.

Advanced customization and development workflows

By default, as mentioned above, you can load your custom code into the site by importing it directly into the config file’s plugin configuration to be loaded as direct plugins. For operators with small teams and a modest amount of custom development, this will be by far the easiest way to work with the Open edX frontend.

But if your organization needs micro-frontends, independent deployments, team autonomy, or is developing a significant amount of custom code, we still got you. I want to note: if you fall into this category you presumably have more resources and developers. You should expect that your more complicated use cases will result in a more complicated build and deployment setup.

The good news is that you’re likely doing this today because of the current micro-frontend architecture. But we’re going to make it more flexible so you can get that power where you need it, and use a simpler setup where you don’t.

Multiple Git repositories

If you’d rather manage your customizations in multiple Git repositories, that works too. They can be checked out as peer directories of the project instead of sub-directories, they can be released as a library and added as a package.json dependency of the project, or they can be fully independently deployed and configured as module plugins to be loaded at runtime via module federation.

The platform’s CLI will provide helpers to build customizations in various ways; it’s all just different webpack configs. The same customization can choose between these deployment mechanisms, it’s not locked into one for all time. The code in the repo is the same.

Big scary breaking changes

Further, if there’s a large breaking change you need to roll out to your custom code iteratively, we can imagine a world where you can temporarily deploy multiple shells and split your apps/plugins between them. Big breaking changes are hard; hopefully they don’t happen very often, but there’s recourse if they do.

How frontend-app-* MFEs evolve

The bulk of an MFE’s code is unchanged. But the trimmings/platform code around them simplifies. Imagine you’re working in an MFE that’s a separate repository from your project - either one of the core product MFEs you’re developing, or a custom one you’ve broken out into its own repo.

The unified platform library provides CLI targets that let you start a development server for your MFE using the shell (we unified the shell with frontend-build so we can do this, remember?). It also turns out that the shell, even though it’s in the unified platform library, is just an MFE that runs initialize and has the header and footer, as shown in the host POC for OEP-65. So if you want to continue to deploy an MFE as a standalone application, you can do this via the shell and giving the MFE a config file, just like you can do today. It’s identical to the “large breaking change” rollout in the previous section, just for one MFE.

This last option will, in fact, be the migration path for the MFEs. We can maintain backwards compatibility with the current “application MFE architecture” by replacing an MFE’s header and footer dependencies, and its index.js file where it calls frontend-platform’s initialize, with a dependency on the new unified platform library. It will effectively build a shell with one app in it - the MFE - which can be deployed as it always has. We’ll be able to maintain backwards compatibility while also fully updating the MFE to the new shell architecture.

Core product MFEs will packaged and released as npm packages so that they can be dependencies of your project and imported as direct plugins. The exports of the package are the same as the exports of the MFE for module federation at runtime.


This sounds like a lot, and it is. But we’re closer than it may seem as we start working through the OEP-65 reference implementation. The majority of the code involved is already written, and many of the more complicated pieces are already figured out (or will be).

My gut feeling is that we can’t responsibly add the composability in OEP-65 without also doing the work to simplify our architecture for operators. Helpfully, there’s strong synergy between the two activities. For OEP-65 we need to turn the MFEs into plugins/modules so they can be loaded by module federation, and it’s an incremental step beyond that to also let them be loaded as direct plugins and providing a project structure around that, drastically simplifying the architecture for most operators.

We’re going from a world where customization means forking source code to one in which we have a plugin framework and a flexible architecture that provides a variety of deployment options according to an operator’s needs. We’re adding even more complexity to a part of the system that was an afterthought - we need to invest in it or it’ll collapse under its own weight. The good news is that once we invest, there’s a world right on the other side of that work that’s way, way simpler, and the investment quickly pays off.

So, like, what’s next?

As mentioned at the beginning, I’m in the process of figuring out how we responsibly migrate the frontend to the new “module MFE” architecture described in OEP-65, enabling frontend composability. For me, that work has always existed in context of this larger vision for how we orient the Open edX frontend around the needs and activities of operators.

It was a bit of a revelation to realize that if we use the shell as an entry point for the entire frontend and express loading MFEs/apps as plugins and modules, we have the opportunity to drastically simplify the platform’s base operational overhead without sacrificing MFE independent/autonomy for those that need it.

I’m going to focus first on building the necessary code to enable module federation, and will be looking to do some intentional architecture that orients that work inside this broader vision. I’ll also be on the lookout for ways to opportunistically enable the vision it where it doesn’t blow my estimates/timeline out of the water.

There’s plenty of devils and details I expect we’ll encounter, but I’m pleased to have what feels like a cohesive mental model for how all this can and should work. I figure I may have some more explaining to do to see if it feels cohesive to others, and that’s okay. Happy to have those conversations!

I don’t think I’ve ever written such a long forum post, thanks for reading if you got to the end! I’m really excited about this work, and hopefully some of that enthusiasm is rubbing off on others. :smiley:


I like this a lot.

1 Like

I like this as well. Count me in

1 Like

That’s great that you have outlined these ideas in a forum post.
We were exploring a similar concept for core shell application, but we didn’t have enough resources to finalize it. We would be happy to participate in this OeX FE transformation!

1 Like

While reading this topic I was a little confused by the different terms. I made a diagram to reflect my understanding of the new architecture that you are proposing:

Am I getting this right? In particular, I’m not sure whether the “shell” is actually just the “project” plus the “platform”.

EDIT: my diagram is incorrect. See David’s answer just below for the correct one :point_down:

@regis A smidge different! I should have made a diagram to begin with, that would have helped. This is kind of one level down from what you drew, and the ‘shell’ is a bit different.

So there are missing arrows from the “App Modules” to the “Unified Platform Library”, but hopefully it conveys the idea. The shell is just part of the unified platform library. It acts as the entry point for the app, much like Next.js’s server acts like the entry point and isn’t actually in your project directory. Unlike the Dev Tool Configs, CLI, and Runtime, it’s not something the library really needs to export either - it’s internal and can be extended via the environment config and runtime.

The only thing I’ll note is that the “imports” from “environment config files” to “App Modules” and “custom modules” is something we could simplify over time so those files aren’t so verbose. In the same way as Wordpress/Next/meta-frameworks don’t force you to manually import your code into some file to be included, but crawl the directories and find it for you, or have sensible defaults.

OK I wasn’t too far out then :slight_smile: A further clarifying points:

  1. Basically, what you are describing is the same as a Django project architecture, but on the frontend: the “frontend project” plays the same role as a Django project. The “unified platform library” is equivalent to the Django API. And the “app modules” are created and loaded the same way as Django reusable apps. Do I get this right?
  2. With this new architecture, does it still make sense to talk about “microfrontends”? Microfrontends are meant to be decoupled. Here we are going to end up with applications that will heavily rely on a rather thick common library. The apps are also not going to be deployed independently anymore. In this context, I think we should drop the “micro” and just call them “frontend applications”.
  3. This architecture is going to resolve the problem of running npm install for every frontend app. Unless I’m mistaken, we’ll then be running npm install once for all apps. (right?) But then…
    a. … how are we going to handle conflicting version requirements between applications? (in particular: conflicting versions of the frontend library and nodejs)
    b. … will we have to run npm install every time a requirement changes in a frontend app? If yes, how long will that take?
  4. What’s the point of separating the platform library and the frontend project in two different repositories? This introduces one extra level of indirection, and I see no immediate benefit.
  5. More crucially, we are going to lose the benefit of autonomous teams. How is this not going back to square one, where all frontend code resided in edx-platform and it was difficult for frontend teams not to step on each others’ toes?

I don’t have a better alternative to suggest. Overall, this proposal looks like a nicer architecture. But I’d like to see more concrete examples of actual benefits for frontend developers. And we should be realistic about the drawbacks as well.


hi, I am very interested in the discussion and have a question about how to use frontend-platform across MFEs.

If the frontend-platform is treated as a singleton and is only initialized by the Host/Shell app during kick start, with the Remote/Child apps using the existing instance, how one configure a different logging service in the Remote/Child apps because it’s never initialize by themself?

+1 to this

I think this can be solved by different shell versions

but I imagine we’d want people to keep them in sync

MFEs had notable drawbacks for operators, and I feel David did address that he was solving for the community of people using our platform, driving adoption, and making education happen. And a plugin architecture absolutely benefits people who are customizing experiences. I take it here you mean that for developers working on core frontend applications (eg, fea-learning), you’re not seeing any benefits, and you would like to see benefits before moving forward? Perhaps that you’re only seeing drawbacks (for core developers) and you’d like to understand if there are any benefits?

Hi @darwin! Thanks for the question. Your mental model sounds right. First, I’ll say that I expect it would be uncommon for someone to want to instrument one of their child apps with Data Dog and another with New Relic, for instance. So in the simplest deployment scenario of a single project, I think that would be unsupported.


The ability to independently deploy apps apart from the main project (either as fully autonomous apps or as a secondary ‘project’) makes this possible; you just need to do some extra work to support it. I talk about this a little bit above in what I called the “Big Scary Breaking Change” section. But fundamentally there are still ways of splitting the project up into independent apps with their own initialization process, either because you need them configured differently, have autonomous teams in your organization, or need to migrate through a big breaking change that can’t be accomplished atomically.

I’m going to try to draw some diagrams of these alternate ‘advanced’ development workflows to clarify what I mean. Between changing the vernacular and the architecture at the same time, I’m sure I haven’t done a great job of explaining everything. Will be doing that shortly in response to Regis above.

  1. Django Comparison: I probably should have used Django as an example instead of Next.js. The point is that we’re building the framework/metaframework that people make projects with. Talking about Django gets a bit muddy since the backend actually uses that framework, of course.
  2. Micro-frontend terminology: I think we need to revise our terminology into some combination of MFE, app, module, and site. The project represents a complete site, and loads the shell app. A frontend-app-* repo represents a module that can be deployed as a micro-frontend or included in a shell app as a dependency. If it’s deployed as a micro-frontend, it can be a standalone app or it can be loaded as a module via module federation.
  3. Dependency management: OEP-65 spends some time talking about maintaining dependency consistency. Projects and module federation don’t solve our dependency management burden alone; managing it is primarily a process, best practices, and tooling problem. Admittedly, install times have not been a focus of this work. What I can say is that frontend-app-* repositories will have fewer dependencies, since they’ll be offloading a bunch of things to the shell.
  4. Separate project / library: I think we must be talking past each other a bit… fundamentally it’s about letting operators and developers work with the Open edX frontend without checking out its actual internal source code from git. Instead, they have a project that represents their code that depends on the platform’s source.
  5. Autonomous teams: We don’t lose the benefits of or ability to have autonomous teams. This is explicitly addressed in the Advanced customizations and development workflows section above, and maintaining it is the focus of OEP-65. Companies that need independently deployed apps can still use a module federation-based webpack config to deploy their MFEs and load them into the shell at runtime. This post is mostly saying that the default way of using the platform shouldn’t assume an operator wants or needs autonomous teams.

The benefits to frontend developers, beyond those described in detail in OEP-65, are:

  • A dramatically simpler site configuration by default. One deployment artifact, no micro-frontends.
  • CLI support for creating new sites, modules (MFEs?), plugins, and brands.
  • CLI support for working with the various deployment options we’re adding this year (module federation, frontend-plugin-framework), which would otherwise be adding complexity without helping folks navigate it.
  • First-class guidance and project structure on how to organize and check in customizations in your Open edX site, rather than leaving this as an afterthought.
  • A familiar project-based code organization that should feel natural to developers who have used almost any other site framework.


  • Complexity for maintainers: We need to acknowledge that we want “our cake and to eat it too” - we want a simple build and deploy process and we also want autonomous teams and independent deployments. As maintainers who need to navigate the guts of our libraries, this is going to add significant complexity. We can’t do both without it, and we can’t continue to do it half-way without (to be blunt) continuing to hurt adoption of the platform.
  • We own a meta-framework now: Making a customizable platform is hard if you don’t want everyone to just fork everything. It requires layers of abstraction and customization mechanisms. We don’t just have a react app; we have a series of inter-related, brandable, extendible, customizable react apps. Like the previous drawback, we can’t continue to do it half way. It takes investment and adds complexity. Our goal as developers and maintainers is to hide that complexity to make a great experience for operators.

Three posts in a row, but three separate thoughts.

Hi all! I’m glad we’re talking about this, and am delighted that folks are willing to come along for the ride.

We can keep talking here, but I also just want to say that I’m going to write this stuff up as ADRs on OEP-65, and will try to include some visualizations of the various ways of setting up a project. The goal is to support everything from “push a button, get a site” all the way up to “independently deploy all the MFEs and link them up at runtime with module federation and plugins”.

I think we can get there. Once I get this written up in a more formal way, I’ll link over to the ADRs here and in Slack (#module-federation, #architecture, and #wg-frontend)

We disagree on the benefits of the proposed architecture, but that’s fine, that’s not a big deal. My understanding is that we can get most of the alleged benefits right now, using the current architecture, provided we implement a small subset of the proposed solutions, which are the following:

  1. Dependency management: you refer to this section of OEP-65, which highlights some of the issues we are facing, but which is not proposing any actual solution to resolve them. Can we implement concrete actions to harmonize dependencies across MFEs right now? We could start by harmonizing header and footer, as their version mismatch are causing big headaches for frontend devs.
  2. MFE de-bloating: trimming 3rd-party dependencies and improving the build tooling is something we’ll have to do anyway to implement your adjacent vision. Thus, can we start working on it now? This is an urgent issue for frontend devs.
  3. To implement items 2. and 3., we’ll need caring maintainers for every frontend repo. For instance, I recently discovered that frontend-build and frontend-learner-dashboard have no assigned maintainer. Maybe we can start with these two repos?
  4. MFE customisation is an urgent and important requirement, and it was decided that Javascript configuration was the way to go. But there is no practical implementation yet because there is no integration with tutor-mfe (issue). Can we complete the implementation of this proposal before moving on to other projects?

These four issues will have to be resolved anyway if we implement your adjacent vision, so I suggest we craft an action plan that starts with those. Once these issues are resolved, then I’ll sign off on any refactoring proposal you like. But I will strongly oppose any plan that pushes back these urgent and important issues until after Sumac.

Honestly, all of this stuff takes time and investment; we can do some short term tactical stuff by Sumac, probably, but making real change in any of these areas is a longer-term thing. In my opinion, solving these issues in a purely tactical way will result in more, different pain, and resolving them in the course of executing the strategic vision will result in a better overall set of outcomes.

  1. I expect meaningfully addressing our dependency maintenance burden should happen in concert with the OEP-65 reference implementation work because the implementation will significantly change the dependency landscape.
  2. If we can find unused dependencies to remove, we should remove them. That shouldn’t be controversial. I’m not sure I believe there’s much we can do about ‘bloat’ in the current MFE architecture, except to rearchitect it to reduce duplication and streamline the build processes (the OEP-65 work does that)
  3. Fostering a community of caring maintainers takes time and as we all know, is much easier said than done. Rolling out maintainers has been ongoing for years, I’m sure we can tactically add a few to the repos you mentioned.
  4. In my opinion, there’s very little in MFE customization that can make a huge difference in the short term. Even if you get the tutor-mfe plugin working with JavaScript file config, the actual extension mechanisms are still nascent and being developed, though we’re making quick progress here w/r/t the header and footer I think. Otherwise, this is a huge long term project which is precisely the point of OEP-65 and the project structure this thread is about.

Taking a step back, though… look, there’s a ton to do in the frontend. We all know this. There are huge gaps, and we’ve spent an equally huge amount of text talking and (frankly) complaining about them (myself very much included). This isn’t easy. I put a slide in an Open edX 2019 conference talk about how we’re trying to rebuild an entire fleet of planes in the air while they’re flying. When you’re doing that, every mis-placed bolt has the potential to be urgent, but no particular bolt alone is going to rebuild the planes on its own. It’s still true, and the planes aren’t rebuilt yet. And to extend the metaphor - mid-build we’ve decided we weren’t satisfied with the blueprints and are modifying them as we go as we learn more.

Any conversation about any part of the frontend invariably brings up other important priorities in the frontend, and somehow we act like any particular solution or idea solves them all. They don’t; this is multi-faceted, novel, and complex. From spending literal years of my life locked in rooms doing more thinking and talking about this frontend, I can confidently say that we’re building something relatively unique in the Open edX platform that has only been accomplished by a limited number of projects/companies, many with vastly more resources than us.

So it’s frustrating to hear that this issue or that issue is more of a priority than another, or that we should stop work on something strategically important because of something else tactically urgent. I’d love to have a macro-prioritization and understand what resources we have available, but I don’t expect we have either today.

As far as this work goes, we’ve been keeping backwards compatibility top of mind since the beginning, and I hope that comes through in the words that have already been written, and in the ADRs to come. I’m hopeful, having spent way too long talking and digging into this, that we can also address some of your tactical concerns in the process of doing this work. I’m doing my best to communicate that out, but as I said at the beginning of this thread: this is complex, deep, and broadly impactful, and even with a huge forum post it’s hard to actually touch on everything, and it’s even more difficult to anticipate everything.

I have, now, what I consider to be a cohesive action plan that addresses both tactical and strategic concerns. It’s based on OEP-65 and these (and other) conversations. That plan is finding its way into a public backlog. As a comprehensive, future state architecture, I feel maybe 80% confident in it and wouldn’t want to try to get higher before getting into the implementation details.

I believe executing on that plan will help us both migrate toward a better future, and actually make headway on some of the dependency and MFE customization issues in the short-ish/medium term. Maybe not for Sumac, but that is what it is. Happy to tweak the plan as more information emerges on priorities and the details of the architecture, but my goodness, after years of trying to figure this out, I’m just glad to have a cohesive plan at all. :sweat_smile: I’m even more excited to have the personal bandwidth and focus to go make it happen.

+1 to this.

While I sympathize with most of @djoy’s post and his response, I think this highlights an orthogonal issue. I’m happy to take on MFE de-bloating with some of my Core Contributor volunteer hours, because it’s an issue that’s important to me. But often I’ll open an issue like this or a dependency removal like this and… nothing happens, at least not for weeks, or worse. As another example, here is a straightforward upgrade PR for frontend-platform (flagged as important) that hasn’t been reviewed in seven months. So even though I want to help and I have some time available to do so, I feel like I can’t, practically - the feedback loop is too slow because we don’t have maintainers.

The OEP-65 re-platforming will solve some issues, but if we don’t have maintainers who are triaging issues, reviewing PRs, and proactively upgrading dependencies now, we’re still not going to have them when that re-architecting is in place, and similar problems will eventually accrue again. So while I’m all for @djoy going full steam ahead on what’s described in this post, I see that we still have some issues around how our frontends are maintained that are urgent to solve now, and the rest of us should be working on figuring that out.

1 Like

We just had a bit of a live conversation with David to decide whether we could address these four issues. The consensus is that the community should be fixing those right now, but they are parallel to what David is currently working on. So even though OEP 65 is supposed to tackle these issues in the long run, if we want them resolved before Sumac then the community should start working on it separately.

If you are reading this and you are going to Cape Town for the conference, then I want to have a conversation with you about becoming a new frontend maintainer :slight_smile: There are multiple companies that have expressed interest in maintaining frontend repos, I think it’s time to make progress and start finding volunteers. I’ll open a separate thread for that.