While testing juniper.rc3 I encountered a strange problem with course discovery.
The behaviour between and juniper are different when adding the ecommerce API as a source used to retrieve data.
When running refresh_course_metadata in juniper, we get tons of errors for audit courses that also have an honor seat:
2020-06-01 14:23:50,526 WARNING 5071 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:403 - Calculating course type failure occurred for [PolyMtl+CHE101.1: La chimie, en route vers le génie].
2020-06-01 14:24:18,857 WARNING 5071 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:572 - Seat type honor is not compatible with course run type audit for course run course-v1:UMontreal+MUS-001.1+P2018
I could give you tons of these WARNING under juniper.rc3 for all these audit courses with an audit seat.
In the end this causes the following errors:
2020-06-01 14:24:19,840 WARNING 5071 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:326 - Processing failure occurred caused by an exception on at least on of the threads, blocking deletes. 2020-06-01 14:24:19,840 ERROR 5071 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - EcommerceApiDataLoader failed! Traceback (most recent call last): File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader loader_class(*loader_args).ingest() File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py", line 329, in ingest raise CommandError('Max retries exceeded and Ecommerce Data Loader failed to successfully load') django.core.management.base.CommandError: Max retries exceeded and Ecommerce Data Loader failed to successfully load CommandError: One or more of the data loaders above failed.
Because of these errors running update_index does not return or isn’t able to return the right numbers for the course runs:
Indexing 0 People Indexing 357 courses Indexing 151 course runs Indexing 0 programs
Running course discovery on ironwood returns the following numbers on “approximately” the same data set (I have more courses in my Ironwood test environment):
Indexing 0 People Indexing 344 courses Indexing 358 course runs Indexing 0 programs
One thing I have noted is that Honor courses seem to be treated differently in Ironwood:
2020-05-17 22:30:04,946 INFO 14601 [course_discovery.apps.course_metadata.models] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/models.py:888 - Course run [course-v1:PolyMtl+CHE101.1+E2015] is not publishable. 2020-05-17 22:32:06,759 INFO 14601 [course_discovery.apps.course_metadata.models] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/models.py:888 - Course run [course-v1:UMontreal+MUS-001.1+P2018] is not publishable. 2020-05-17 22:32:06,761 INFO 14601 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:165 - Processed course run with UUID [58aceb75-b8a2-430b-914f-41a3fc108807]. 2020-05-17 22:32:06,765 INFO 14601 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:199 - Processed course with key [UMontreal+MUS-001.1]. 2020-05-17 22:32:06,765 INFO 14601 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:140 - Processed course with key [UMontreal+MUS-001.1].
Anyone from the course discovery team able to tell me what is going on?
My first guess is that course discovery no longer knows what to do with honor seats because those are only used in Open edX instances, but I could be wrong.