Proposal: Use MFE config API to send waffle flags to MFEs

Development
1

Hi all,

Wanted to share a pre-ADR idea for consideration. This is very nascent and may contain some faulty assumptions!

We have a long-standing gap in our config capabilities where, when we define a waffle flag in the backend (i.e., edx-platform), we have no good way of getting that value up to the micro-frontends. The only way I know of to solve this today is to add the value of the waffle flag into an existing API call the MFE makes, which is somewhat hacky. If you need to send a waffle flag to many MFEs, this can become a lot of work to find appropriate APIs to use.

I was thinking about whether or not we could use the MFE config API to solve this problem, allowing us to atomically update a waffle flag on the backend and have it propagated to the frontend on page load.

Iā€™m making up the details here, but in spirit:

MFE_CONFIG = {
    "BASE_URL": "https://name_of_mfe.example.com",
    "LANGUAGE_PREFERENCE_COOKIE_NAME": "example-language-preference",
    "LOGO_URL": "https://courses.example.com/logo.png"
}

MFE_CONFIG_OVERRIDES = {
    "mymfe": {
        "LANGUAGE_PREFERENCE_COOKIE_NAME": "mymfe-language-preference",
        "LOGO_URL": "https://courses.example.com/mymfe-logo.png",
        "waffle_flags": { # This is new!
            "ENABLE_MY_THING": "mymfe.enable_my_thing"
        }
    },
    "yourmfe": {
        "LANGUAGE_PREFERENCE_COOKIE_NAME": "yourmfe-language-preference",
        "LOGO_URL": "https://courses.example.com/yourmfe-logo.png",
    },
}

ENABLE_MY_THING is the key that would arrive in the client, and 'mymfe.enable_my_thing' is the waffle flag namespace and name. The MFEConfigView.get handler (edx-platform/lms/djangoapps/mfe_config_api/views.py) would be updated to look for waffle_flags and add flag values into the response based on the MFEā€™s overrides.

One potential issue here is how we ā€˜lookupā€™ a waffle flag by its string name, rather than by using a reference to the actual WaffleFlag instance. I donā€™t know enough about their guts to know if thatā€™s even possible, but assuming the MFE_CONFIG_OVERRIDES may be defined in a YAML file, this may hinge on being able to do so.

Thoughts? For reference, the original MFE Config API PR: feat: add mfe config api by MaferMazu Ā· Pull Request #30473 Ā· openedx/edx-platform Ā· GitHub

Attention @mafermazu and the original reviewers: @Alecar @arbrandes @feanil @Felipe @mgmdi

2

I think what are proposing is rational. I just have few points I would be happy if we can address before moving forward.

The issue of how getConfig is used:

Regarding MFE_CONFIG, I think there should be something documented somewhere about possible issue, the issue I personaly face and has to fix for almost every MFE we use in an open release is the following;

const CONFIG_X = getConfig().CONFGI_X
Now suppose this line is written at the top of any JS file, it would get executed before dynamic config middleware update the configuration So the way to resolve, is instead of writing a variable, defineing a fucntion that get the variable: i.e

const getCONFIG_X = ()=> getConfig().CONFiG_X

This would ensure when using the config variable would get the last value, if it was udpated by the config middleware.

Examples of this issues and fixes:

Settings names in edx-platform v MFEs

On the otherhand can this apporach, also be used for settings, i.e. almost all MFEs requires to know the URLs of some other MFEs, and in my observation, these settings can have different name in their .env than in edx-platform, it would be great thus to normialize that.

Other

One last feedback, about settings, is it MICROFRONTEND or MFE, some setting use the former other use the latter, I know this trivial issue, but it can be confusing for new comer.

3

Other considerations:

  1. How does the MFE know which backend service contains the flag details?
  2. How do you keep the backend endpoint from exposing every flag setting in the service? Note that we have a version of the endpoint you are looking for here: https://courses.edx.org/api/toggles/v0/state/, but it is only available as staff, because we donā€™t want to expose all of our settings to everyone.

UPDATE: One idea is to be able to tell the backend when a toggle is safe to expose to non-staff. This could be in code, or via a setting, or maybe both (default/override). Note that this endpoint also gives the possibility of reading a toggle value, whether it is a Django setting, Waffle Flag, Waffle Switch, etc. on the backend.

4

Well, I didnā€™t think in that use case when we created the API, but in the end, the MFE Config API exposes settings, and if you can use it to set waffle flags and is the best way to communicate the waffle_flags to MFEs, go ahead.
I think the key here is how to read the API output and activate the waffle_flags.

I didnā€™t quite understand this part. Could you explain a little more?

5

Hey David!

I have no objections to the general way of doing this. Itā€™s simple and straightforward, assuming the string lookup is doable. Knowing Python, I figure it probably is. :person_shrugging:

I do, however, worry about what waffle flags in MFEs would allow: a bunch of conditional blocks that would be (much) better implemented via a plugin system. I know youā€™re also thinking about how to solve the latter, but I figure it is still important to bring up.

In other words, as long as weā€™re not using these flags to allow organization-specific code into MFEs, instead relying on them for temporary incremental deployment of new features, then I think itā€™s fine.

It wouldnā€™t need to: it just receives any waffle flags as part of the initialize() call.

Runtime config is done via site_configuration, so control is actually in the hands of whoever writes the corresponding JSON string. Iā€™m assuming that anybody with this level of admin access can also be trusted to know what can be exposed.

6

Addendum to my post above: Iā€™ve been seeing Optimizely creep into MFEs, lately. While thereā€™s precedent for including calls to paid services in Open edX (NewRelic, Segment, Datadog, Google Analytics, Stripe), I think we should be aiming torwards making all of them plugins. And in this particular case, waffle flags would be an even better native alternative.

7

I donā€™t quite follow how this data is exposed from the backend, but it sounds like we have a way to pick and choose, and thatā€™s all I was confirming. Thanks.

1 Like
8

Iā€™m caching up to this and I think is a very reasonable proposal. I completely see the need for an api that exposes waffle configs and that every mfe can use.

However I also have the same question as @robrap:

Am I understanding the ergonomics correctly?

Lets make an example. I have the following flag:

image
image740Ɨ286 22.1 KB

As an admin I would write in the mfe-configs or the overrides something like:

MFE_CONFIG_OVERRIDES = {
    "mymfe": {
        ...
        "waffle_flags": {
            "THIS_NAME_CAN_BE_ANYTHING": "grades.writable_gradebook"
        }
    },
    ....
}

and the actual call to /api/mfe_config/v1?mfe=mymfe would yield:

{
    "CSRF_TOKEN_API_PATH": "/csrf/api/v1/token",
    "LANGUAGE_PREFERENCE_COOKIE_NAME": "openedx-language-preference",
    ...
    "waffle_flags": {
        "THIS_NAME_CAN_BE_ANYTHING": true
    },
    ...
}

If so, I think the exposure of flags would not be an issue, but the mapping could become unruly. I donā€™t have any better proposal just now. Iā€™m first trying to fully understand.

10

@ghassan:

const getCONFIG_X = ()=> getConfig().CONFiG_X

This is the reason why getConfig is a function to begin with. As a best practice it may not be well-documented, but no one should be pulling variables from getConfig into a constant at the top of a file to begin with for exactly this reason - it pulls the value too early and also means if it changes for some reason, you donā€™t get the update. getConfig is just a getter, it does no work, and is perfectly reasonable to call each time you want a piece of config from frontend-platform. I donā€™t think thereā€™s any change necessary here except to document why using a constant is a bad idea in general.

@mafermazu:

I think Iā€™m saying that if weā€™re going to be defining MFE_CONFIG_OVERRIDES in a YAML file, weā€™ll want to identify a waffle flag by a string there, since we wonā€™t be able to execute code in it to get the flag.

We define waffle flags something like this:

WAFFLE_NAMESPACE = 'notifications'
ENABLE_NOTIFICATIONS = CourseWaffleFlag(f'{WAFFLE_NAMESPACE}.enable_notifications', __name__)

And then we pass around that ENABLE_NOTIFICATIONS reference to find out the value wherever we need it. So in our YAML file weā€™ll want something like ā€œnotifications.enable_notificationsā€ and then weā€™ll need a way to look up the actual variable so we can find the WaffleFlag instance, get itā€™s value, and dump it into the API. I donā€™t know how weā€™d do that or if itā€™s possible. :grimacing:

@arbrandes:

Agree on making sure weā€™re using them for the right reasons. ā€œtemporaryā€ can sometimes be a few Open edX releases, as Iā€™m pretty sure at time we use flags like these to gate new features as well while theyā€™re experimental. And yeah, in general we shouldnā€™t have any waffle flags for org-specific stuff.

@robrap and @Felipe:

Felipe has the intention right - youā€™d specify in the MFE config API overrides which flags an MFE needs, so itā€™ll only get a boolean value for things weā€™ve explicitly said it can have.

The point is that itā€™d let us have one piece of configuration for a feature that may otherwise span the backend and frontend, which can get awkward to coordinate separately in production at times if you just have two separate flags. Also, frontends donā€™t have a waffle flag framework, just booleans, so producing the same behavior frontend and backend is often impossible without sending the value up somehow.

As for which service contains the flag, well, as envisioned I think this would only work for flags defined in edx-platform, since thatā€™s the service the MFE config API hits. I donā€™t have a good way around that, but itā€™s a general limitation of the MFE config API anyway. Any config we want to send up (even if it deals with some other service) needs to be exposed through edx-platform.

And sorry itā€™s been over two weeks since the original post. My TODO stack got deep. :sweat_smile:

2 Likes