Frontend Pluggability Summit

Development Architecture
, ,
1

Over the last years we have been migrating our user experiences away from the server side, to micro-frontends based on the React framework. This has allowed us to iterate faster on user experiences and deliver value faster. However, our frontend stack doesn’t provide the same facilities that we have on the backend for extending and customizing the base platform installation. For example, on the backend we now have events and filters and Open edX plugins. Furthermore, XBlock has been available for customizing course content for many years.

I believe that the need for pluggability on the frontend has now become urgent. There have been a number of recent pull requests that have committed features that are specific to a particular user of the platform, rather than general and valuable to the entire community. And while other PRs have pushed the pluggability story forward, without architectural direction and community buy-in these tend to solve for a limited set of use cases, all the while adding to the burden of maintenance and documentation.

We need to be able to support customization of the platform frontends, in support of the needs of independent users, without altering the shared foundation. Unfortunately, we do not currently have a strong, agreed to proposal for moving forward.

Furthermore, I would argue that we need customization to be possible without forking repositories. Doing so is bad for the team forking and the community as contributing back becomes harder. At the very least, we need to minimize forking and enable easy upgrades. We want an approach that minimizes effort to port customizations into future releases.

I’d like to propose a Frontend Pluggability Summit to help us converge on a specific architectural approach to support user specific extension and customization of the Open edX platforms frontend views.

I propose that we do the following before, during and after the summit.

In advance, asynchronously:

  • Discuss and catalog the requirements for frontend pluggability in this thread
  • Capture any prior art either from the Open edX project or other platforms
  • Propose high level approaches and build teams around them to create a high level approach document. This is really like an asynchronous design hackathon, pitch an idea, build a team, produce an artifact.
  • Publish and review the proposed approaches

At the summit, synchronously:

  • Spend some time reviewing and discussing the requirements.
  • Review and discuss proposals that are on the table
  • Discuss the pros and cons of all approaches as a group over the course of several synchronous hours
  • Converge on a plan that will become an ADR or OEP documenting the approach
  • Develop a specific plan to deliver this capability where all stakeholders contribute to completing the work.

After the meeting, asynchronously:

  • Do the work to publish the architectural decision
  • Do a technical spike/POC to validate the selected technical approach
  • Groom the required work, determine who will do which parts, execute and ship it.

I have created a meeting poll to determine the time to meet that works best for interested parties. I am proposing a three hour meeting – with pre-work – so we can make as much progress as possible. Ideally, we would agree on the high-level approach during the workshop. In any event, we would conclude with an agreed set of next steps with owners for each.

There are some existing PRs that should be considered as part of the requirements for what we design and deliver. I’m including them here as a reference and start to the requirements gathering effort.

11 Likes
2

Thanks for organizing this. I completely agree that frontend pluggability is urgent.

FWIW: Last year, I needed to implement frontend plugins in another project of mine so I designed and developed a simple UI plugin framework for React, vaguely similar to the hooks framework in that the UI defines “Slots” which are customizable areas that contain “Widgets”. Plugins can then inject their own widgets into those slots or modify the default ones.

Designing and implementing the plugin framework was surprisingly easy, and it’s been working well ever since. I put the design details and code on my blog: Creating a UI Plugin Framework in React. If anyone has questions about what I learned from that experience, I’m happy to answer them. But the main point I’d like to make is that it doesn’t have to be complicated, and in my experience it can be done quickly and easily, even for relatively complex applications (I had to support white labelling, runtime config, plugin-defined URLs, theming via plugins, etc.).

In fact, if you end up with an approach like I took for that project, rolling it out at first is [almost] as simple as putting <DefaultUiSlot> ... </DefaultUiSlot> around the major elements of your user interface. Ta-da, it’s customization-ready.

Another well-known approach to UI plugins is Swizzling, used by Docusaurus themes to customize their React UI. It has some overlap with my approach, but some major differences. The ways in which you can modify components are more limited, but you can modify virtually any component without having to declare it as a “slot”. (In my case I had an explicit design goal to not support that, and only allow plugins to modify explicit slots, to improve long-term maintainability.)

3 Likes
3

This sounds like a great idea, Ed.

I’ll add that another existing PR that was started by @djoy and that I’m working to finish is for enabling a React-based iFrame plugin that lives in openedx/frontend-platform.

In terms of its abilities, it allows for:

  • Customizable window width/height of plugin slot
  • Multiple child MFEs can live inside a single plugin slot
  • Designed for future plugin types (LTI, Build-time, Module Federation) to be rendered based on the child MFE’s configuration. Linked is where the iFrame case exists and where the other types implementation logic would exist.

As for the PR, what’s left is:

  • Determine the desired fallback for when a plugin fails to render ADR
  • Write a test that ensures the desired fallback works correctly
3 Likes
4

That’s an excellent initiative to host a Frontend Pluggability Summit. The topic is timely and critical for the Open edX community.

At Raccoon Gang, we’ve been dealing with the challenges of frontend customization for some time. As a service provider for Open edX, we often need to customize the platform to meet the unique business needs of our clients. These customizations often extend beyond simply adding new features, they involve modifying existing features and components to align with specific business requirements. Also, one of the key considerations for us is maintaining the system’s upgradeability, we want to be able to easily transfer and apply platform customizations to the newer named releases, as they are ready. The current approach, which involves forking and customizing micro-frontends, makes this incredibly challenging.

We are currently exploring the use of NPM overrides and additional Webpack scripting. These could be integrated into the frontend-build repository without necessitating the forking of other frontend-app-* repositories.

Last month, we initiated an internal discovery with the following objectives:

  • Identify methods to override JavaScript code in Micro-Frontends (MFEs) with minimal changes, possibly by leveraging existing solutions in frontend-build or frontend-platform repositories.
  • Use a single repository to manage all code overrides and new customizations across MFEs.
  • Our customizations should be compatible with the latest MFE versions with minimal adjustments required.
  • Share a working example or even a fully implemented solution, specifically tailored for the Palm release MFEs.

We’ve developed a Proof of Concept (POC) using Webpack’s NormalModuleReplacementPlugin to replace entire files (modules) at build time. You can find it here: mfe-overrides GitHub Repository. We’ve tested this on a live installation, and it’s functional. However, it’s still a work in progress and requires further refinement to serve as a comprehensive solution.

It would be incredibly valuable to discuss the various approaches currently employed within the community. Maybe it would be possible to co-develop a robust framework for extending and customizing Open edX micro-frontends.

4 Likes
5

Thanks all for the great input. I’ve started to catalog some of the proposed directions in Confluence.

1 Like
6

The best day to meet seems to be October 25, 2023. However, we have one person on the west coast of North America and one person in Ukraine.

Do either @braden or @Glib_Glugovskiy have flexibility for an early or late meeting?

1:00pm - 4:00pm

4 votes

Adolfo, Braden MacDonald, George Babey, Jason Wesson

10:00am - 1:00pm

3 votes

Adolfo, George Babey, Glib

7

Yeah I can do 10am Eastern if necessary.

8

Hello folks,

Thanks Ed for organziaing this initiative

I would like to propose/add the following ideas for discussion/ideas, that are think related this topic.

These are rough idea, that I and my employer Asbtact-Technology are interested to be addressed.

1. Create a mock service/server for Open edX frontend Dev

Put in simple term, if FE dev in Open edX want to chagne a color, ceneter a div…etc does it make sense to run the heavy stack?, can we go along with just having a fake service that would response with the typical APIs.

I think this can go hand in hand or to complement (OEP-37 Data Dev)

I think a POOC of this is easy to create, we can just build an app based on Open API spec.

2. Merge FE repos togathers

frontend-app-* repos that are under maintenance, that no features are expected to be added in the in the feasible future could we merged together:

Below is the table I created, that counts the number of commits for frontend-app-* in a window of 6 months

Repo/MFE # of Commits last 6 month[1] Date first commit
frontend-app-programs-dashboard 0 10.2.2021
frontend-app-shell 2 20.7.2023
frontend-app-learner-portal-programs 10 25.3.2019
frontend-app-program-console 12 20.5.2019
frontend-app-gradebook 15 17.10.2018
frontend-app-communications 18 3.12.2021
frontend-app-learner-record 24 19.7.2021
frontend-app-ecommerce 27 12.4.2019
frontend-app-admin-portal 36 10.5.2018
frontend-app-enterprise-public-catalog 45 2.3.2021
frontend-app-ora 47 12.7.2023
frontend-app-payment 49 5.6.2019
frontend-app-support-tools 51 11.2.2020
frontend-app-publisher 56 21.12.2018
frontend-app-ora-grading 63 16.9.2021
frontend-app-learner-dashboard 65 16.9.2021
frontend-app-discussions 66 11.8.2020
frontend-app-learning 70 2.5.2019
frontend-app-course-authoring 96 6.5.2020
frontend-app-account 101 23.4.2019
frontend-app-profile 102 10.6.2019
frontend-app-learner-portal-enterprise 115 23.9.2019
frontend-app-library-authoring 147 30.6.2020
frontend-app-authn 220 28.9.2020

Looking at the table, one could argue that gradebook and communications could be merged togather, of course other input are important to consider, like if the MFEs are laerner or instructor facing…etc. It I can be probably one of the metric to consdier if/when MFEs are expected to be merged.

3. Xblock/Units to utilzie Paragon/React

One way frontend part of Xblock diverge from the microfrontend, that is it’s SSR (Server Side renderd), while micrfronend is CSR (Client side rendered). This difference currently comes with a cost:

  • Because Xblock is using the templates, while the MFE is based on React/Paragon, there would be a notiable difference in the apperance.

Xblocks are very important, because they are at the core of the platform (what is more imporant then the learning expirence…). I think it’s quite ironic that our UX/UI for every other aspect of the platform is more engaging than the most important UX/UI [2],

I don’t have a magic solution for this, I think nonetheless, is important to keep it in mind. If we can answer this question; Is Open edX going to be completely Resutful API in the feasible future and if yes, does this mean at somepoint all Xblocks view endpoints are going to be deprecated?.

If it would be agreed that for the feasible future we would still be hyprid(CSR/SSR), then I think we need to spend a fair amout of time to reduce the delta between views that are rendered through the templates vs. views that are rendered in clinet using Restuful API.

Note: If I would have until the time of the summit, I would add further infos, POOC…etc.

  1. 6 months between 19th March 2023 to 19th Sept 2023 ↩︎

  2. I might make an assumption(s) here. I think a self-motivated learner would stick no matter how UX/UI look like. ↩︎

9

First off, these are all good points, @ghassan, but they’re only tangentially about pluggability, so I’m not sure they have a place at the summit.

Here are my hot takes, though. Back to front:

There’s already a detailed issue written up for this:

Mono-repos have pros and cons. The biggest con right now is that we don’t have a lot of experience with them, and it would require a bunch of retooling. In other words, there should be significant discovery into what exactly we would be gaining.

@AdamStankiewicz may have some insight, as, reportedly, Enterprise has started some experimentation in that direction.

I think that without dev data (OEP-37), it becomes really hard to build a mock service that’s actually useful.

But, in any case, there have been related conversations around Pact, recently. @DawoudSheraz gave a presentation on it on June 1 to the Frontend Working Group. I think this is a good thread to follow, going forward. But, again, not in the Pluggability summit.

1 Like
10

Those might not all be direclty realted Frontend Plugabiliy but iMHO I would consdier at least Xblock/Paragon issue baby step in the right direction. I will keep an eye the issue anyway.

1 Like
11

I wanted to remind folks that we’ll be gathering on 10/25 at 10AM EDT.

There was a substantial homework assignment :slight_smile:

Will folks have time to do some asynchronous work in advance?

@arbrandes was recommending that we have a short coordination meeting before the meeting. He’ll be reaching out to folks who have responded to this thread to schedule.

1 Like
12

What does plugability encompass? I’ve been reading through the thread and been wondering if my concerns as an operator do fall in this category or are they off-topic?

I’ve found myself wondering about ways to reuse the static artifact across environments (in Tutor this takes the form of a container running a webserver that serves the static files). The MFE config API was a huge step forward, but some configurations are still bound at build time (e.g. the new relic plugin).

Is it ok to consider the summit as blanket “let’s join efforts to give the MFEs more flexibility” or is it much more focused?

1 Like
13

@e0d please please add me to ghassan in the summit invitation if possible. THX

14

Hello all,

thanks a lot Ed for organizing this. Definitely this is one of the Areas of the tech landscape in openedx that needs more flexibility and extension options.

We had an small session with the team at edunext where we came up with dev stories of extensions previous to devoting time to investigate the current proposals. I thought it might help to share those stories here. Not all of them might be covered by the current effort, but still they are woth capturing.

  • almost every customer wants to modify the footer content (often also the header)
  • online academies (gathering from our saas offering) very often ask us to add additional fields to the Authn registration page.
  • online academies offering need to present the same additional fields from the Authn registration to the account settings page.
  • university of michigan wants to make redirections from the learner dashboard to a custom policy acknowledgment/acceptance page. (published in the issues of the mfe)
  • spanish universities want add more filtering options to the course emails page (frontend-app-communications)
  • spanish universities authors want to add more buttons to the html editor (tinyMCE) to make iframe and other embeddings easier.
  • spanish universities want to add an extra component to each block in the course content to gather feedback on the quality of the component.
  • pearson uses a side navbar block that want to add to the learning MFE so learners have easy access to the course navigation tree.

I will be in PTO the day of the event so sadly I won’t be able to join, but as a team there will be different developers from edunext taking a deep look at the proposals made before and commenting both here and hopefully at the sync event.

2 Likes
15

Hello there folks.

Regarding what the requirements should be for frontend pluggability, we can start by listing what we can currently extend in the legacy applications by using the available mechanisms here I list the two I’m the most familiar with:

  • Using Open edX Filters, we can modify the template context, change the templates altogether, make redirections happen according to some conditions,
  • Using Plugin Contexts, we can use theming to access new context added by plugins

Those are more backend-focused mechanisms but still affect user interactions with the platform, which can be lost after completely transitioning to MFEs. The MFE Config API covers some of the cases like context customization, but not others.

So, I wonder whether we should consider those existing customization methods as part of the requirements.

Also, I think we would benefit from having a (short) list of places where we want to extend and start from there. I guess that’s also part of the requirements.

3 Likes
16

Hi guys!

Rewriting an MFE is definitely something that we need, instead of forking MFEs for specific use cases. I’ve tested your POC here, and it has a really good approach. I would like to add one thing:

  • A mechanism to generate Docker images for the rewritten MFE or to deploy the rewritten MFE instead of the default MFE. This can help us to have a complete flow with the MFE once the MFE has been rewritten and provide that MFE in a certain way. This is a recommendation that we can co-develop in the process of building this initiative.

Please count on me to help and contribute to this initiative; it has a lot of potential.

17

Hello folks, thanks for the initiative. It is something that as a company and part of this community see the need to improve on the front end.
Reviewing the design approach similar to Docosaurus’s Swizzling mechanism. We found that it is a good approach similar to what comprehensive theme does, it works for simple components, it has a CLI that is easy to use. The two patterns, eject and wrap, are good mechanisms to have the flexibility to completely change a component or add external elements to one.
But we see that it has limitations, there are components that are complex and this approach would not be recommended because those are likely to have broken changes between releases and make it hard work to migrate.
We just leave two questions that we had in our review:

  • How can we define what is extensible, should it be a set of paragon components? should it be a component list provided for MFE? how to merge them?

  • Should it be done in a new package or could it be part of frontend-platform or frontend-build?

18

HI Guys I hope you are doing great!!

I was analyzing Ed’s proposal related to using an approach similar to Backstage. It seems to me that it is an interesting way to complement another proposal since Backstage is a way of managing resources related to certain infrastructure so it could be used as a tool of project management and this would help us to have an organization of all the projects related to the micro-frontends which will bring us management benefits, ease of finding documentation and ease of contributing to the projects (By correctly analyzing the best approach for the ownership thereof) if we have them gathered in the same place.

But I have some doubts, I see that in the proposal you commented about the possibility of creating backstage plugins, the purpose of this is to create plugins so that anyone who wants to propose a new frontend module creates it following the same template. If so, we would have a microfronted core and we should implement an API that allows us to connect these modules. Is there the possibility of connecting multiple modules with this core? How would the connections be made between the modules? You should have a module creation policy to prevent several people with similar approaches from creating the same module.

In conclusion, I think that backstage is a tool that could help us improve many community processes, including the process of extending micro frontends, but it should be accompanied by another approach.

1 Like
19

Hello Team, I hope everyone is well.

Thank you very much for this initiative.
Along with another colleague (@yoiber) we were reviewing the PR that @Jason_Wesson comments on the iframe based on React, and it seems excellent the option of iframe plugins as it could allow customers to customize different messages and notifications on their platforms. Of course it would depend a lot on how much integration would have this add-ons with the different MFE.
We have several questions, regarding the customization capabilities of the plugin slot, such as the width / height of the window, how flexible and configurable are these parameters for developers who implement the plugins? Are there or will there be restrictions or guidelines to maintain a consistent user experience?

20

I was reviewing the UISlots proposal because of the familiarity, and I’m left with two questions:

  • Is there a way we could modify the component’s lifecycle within a slot as well? Eg. what happens before rendering, during, after, or when loading, etc
  • Besides adding new pages using plugins as specified in the blog, could we modify the existing router definitions? eg. let’s say /dashboard now redirects to component X instead of Y

I’m not versed in React so I’m reviewing this from the current things we can do on the legacy frontend as I previously mentioned in this comment. Thanks, folks!