Inter-app communication experience (LMS -> CMS)

Development
1

Recently we worked on a simple Open edX plugin: when a user registers an account, the plugin must automatically create an organisation and give that user staff privileges for it. This is for testing/demo purposes.

There was a problem we found tricky to overcome: user registration and the associated signal (STUDENT_REGISTRATION_COMPLETED) happens on the LMS, but the CourseCreator model is only installed and available on the CMS.

We considered a few approaches and thought our findings may be interesting to the community, especially where it relates to usability of the event bus and celery queues:

Patching openedx-platform

An option was to submit a patch to openedx-platform to make CourseCreator available on the LMS. We considered it, but this would mean adding a CMS app to the list of LMS installed Django apps, which was not preferable.

Consuming the signal on the CMS via the event bus

There is a relatively new message bus system as part of openedx-events, which should provide support for directly consuming this signal from the CMS, even though the signal was sent from the LMS. However this turned out to be rather complex with the current tooling:

  • The message bus system requires a separate server process running (the consume_events management command). This is not configured in Tutor deployments as far as we know (the event bus is deployed, but no consumers), so would require a Tutor plugin, and running extra pods with this event consumer. Note that this is separate to the celery lms/cms workers, and separate to the usual lms/cms backend processes.
  • The signal needs to be added to the EVENT_BUS_PRODUCER_CONFIG setting, for it be published to the event bus (otherwise the signal will remain local to the app). There isn’t a neat way to add to this for a deployment with Tutor yet - you can override the EVENT_BUS_PRODUCER_CONFIG in the lms-env or cms-env patches, which loses pre-existing config, but not add a new entry to it.

We experimented with a proof of concept on a Tutor dev devstack, and got it working with the following set up:

  • A plugin installed on the CMS
    • that registers a receiver for the STUDENT_REGISTRATION_COMPLETED signal.
  • The following Tutor plugin to ensure the signal would be sent to the event bus:
    hooks.Filters.ENV_PATCHES.add_item(
  (
      "lms-env",
      """
EVENT_BUS_PRODUCER_CONFIG:
  org.openedx.learning.student.registration.completed.v1:
    user-lifecycle:
      enabled: true
      event_key_field: user.id
"""
    )
)
  • Manually running the consumer in the cms container: python manage.py cms consume_events -t user-lifecycle -g arbitrarygroup --extra '{"consumer_name": "c1"}'.
    • Note that you would need to be careful to only run a consumer for the related topic (“user-lifecycle” here) on the CMS; running it on the LMS or on both would result in errors, because this code fails if run on the LMS.
    • It appears there is great flexibility for site operators to run consumers how they wish, which also makes them more complex and more specialised than the Celery workers.

For reference, a diagram of what this structure looks like:

diagram of the architecture of using event bus to subscribe to events
diagram of the architecture of using event bus to subscribe to events1082Ă—355 23.4 KB

However we abandoned this approach due to the effort required to set this up cleanly in production.

Celery antics

For the final approach we settled on, we took advantage of per-app Celery queues.

In this method, we install two separate entrypoints of the plugin: one in the LMS and one in the CMS:

  • In the CMS: register a Celery task that takes a user id, and configures the desired organisation and permissions for the user
  • In the LMS: listen for the STUDENT_REGISTRATION_COMPLETED signal, and schedule a Celery task on the edx.cms.core.high queue.

A diagram of how this looks:

diagram of the architecture of the plugin described above
diagram of the architecture of the plugin described above961Ă—384 24.5 KB

The caveat is that it’s trickier to test on a Tutor dev devstack, because it doesn’t configure Celery workers in dev mode. We needed to turn off CELERY_ALWAYS_EAGER and add Celery worker containers. See the example Tutor plugin for how this works.

This works out of the box on a production environment though. :slight_smile:

The result can be browsed at opencraft / dev / openedx-auto-studio · GitLab

2 Likes
2

@dave I mentioned this to you in passing the other day - here are the details from @samuelallan ^

3

Thanks for the ping @braden.

FWIW, I think that this is actually the way to go in platform (not just your plugin). Role information is on its way to getting centralized with the ongoing RBAC project, and I don’t think it makes sense for us to confine the course_creators app to just Studio. We already had to move the student app to common because that’s the central place for role information, and that’s the one other app that course_creators needs to import from. I think this move might feel more wrong than it is because STUDENT_REGISTRATION_COMPLETED is also confusingly named, since it’s really registration for all users.

Yeah, this is the big downside, and it’s really counter intuitive to develop and test when people use cross-process celery tricks like this. The failure mode for mis-configuration can also be really mysterious, where things just don’t happen and people aren’t sure why. An even worse version of this is when the app actually exists on both sides of the CMS/LMS divide but are configured slightly differently (block transformers ran into all kinds of headaches around this).

In any event, I realize this is just a small plugin at the moment. But I would support moving course_creators into common so that the celery antics are unnecessary.

1 Like