A very meaningful and historical context discussion and I like that. The data flow to and from Discovery is confusing for sure. In my opinion, the decisions around the source of truth of marketing/catalog data were not formulated when most of the catalog information was moved to Discovery. For instance, there are remnants of About Pages in Studio (image, description, etc.) and then there are the start and end dates that are editable only in Studio. In Publisher, this is mentioned explicitly to Editors and Project Coordinators on UI. That is one data flow. But when a course run is created by the Publisher, the publisher demands start and end dates at that point. That creates a data inconsistency issue whose resolution is attempted by a very expensive management command refresh_course_metadata in Discovery.
If I look at the Discovery codebase, I feel like it has been treated like a playground or testing area for Catalog information. We need indexing; add ES. ES is doing its job but we need more. Add Algolia but keep ES for other teams and for the open edX community. We need Partner-editor communication flow, add Salesforce. Discovery has become the octopus that has its tentacles everywhere. It is difficult to find out why a certain decision was made. ADRs help but they contain only a part of the context. Then if we think about deprecating/removing components, we would not know the consequences until it is live. There is always some hidden use-case, be it an external tool, data warehouse, or reporting that breaks.
1 Like