Externally considered, the Open edX events suffer from the fact that they eschew industry standards, making interoperability difficult. No event standard is perfect, but adopting something like xAPI would immediately open up interoperability with numerous open-source and proprietary Learner Records Stores (LRS).
Internally considered, Open edX events suffer from:
Not having a schema to detect conformance issues
Not having a mechanism for versioning events to allow graceful evolution
Not having consistent naming conventions or namespaces
Having known cases where events are emitted that are not valid JSON
Having semantically identical events with different names
My belief is that the amount of effort required to improve the native Open edX eventing sub-system would be better spent leap-frogging it and ensuring that documented standards exist and are enforced via schemata, linting, etc., from the beginning.
At the highest level, the proposal is:
Embargo the native Open edX eventing system. This would entail declaring it to be used at one’s own risk and acknowledge that it will not be under active development.
Declare that xAPI will be the official eventing system for learning events in the Open edX platform, leveraging the work done in the event routing backends project.
Declare that any events that are mapped to xAPI events are immutable except to fix bugs in the mapping
Preserve the eventing system as embargoed unless or until xAPI can be generated natively, without mapping from tracking events.
Ensure that all critical learning events are available in the xAPI form
I am sure others will offer more detailed and, likely, more valuable feedback, but here is at least my initial thought. A key element within this proposal is what might be considered a sub-bullet of the last bullet in your proposal: We need to first come up with a contemporary definition of “critical learning events.” I think part of what hampers the current eventing system is that it simply doesn’t provide enough of what we at WGU consider “critical” events or, in some cases, enough detail about the events it does capture. Determining what are critical events and how those should be captured and described could help as we determine the value of something like xAPI as the solution.
I like the idea of standardization of the events in OpenedX. This part looks very attractive.
The other part related to the replacement of the event subsystem is not so clear right now. The event subsystem currently provides an option for processing (processor list) and broadcasting processed events to the backends. We in RG used to use this approach in our customizations and extensions.
@e0d do you think it would be a similar option after replacement to work with standardized and natively generated events, like process events in a “pipeline” and send it to different backends?
I think it would be possible to work with both, but it’s a question of what guarantees we make about the native events and how we handle the maintenance burden of maintaining two things rather than one.
Are you advocating for maintaining the open-ended flexibility of the existing native events?
Would you be OK with interface being declared, “use at your own risk?”
@mikeh1239 Does WGU have a specification of what you consider “critical learning events.” We would love to capture them in our use cases documentation
I have not a strong feeling about it right now, and my question is whether we going to persist event “pipeline” approach or if it is out of the proposal’s scope?
This conversation had been moved into a ticket on the Data WG board, where there was more discussion. Eventually it was decided that there is a lot of value in having a native format that can be transformed into other formats and we should keep the native events. I’m very in favor of versioning the existing events to be cleaner and more consistent, but that’s a whole other project we haven’t even started to look at yet.