Course Discovery, juniper and courses with an honor seat in ecommerce

Development
1

While testing juniper.rc3 I encountered a strange problem with course discovery.

The behaviour between and juniper are different when adding the ecommerce API as a source used to retrieve data.

When running refresh_course_metadata in juniper, we get tons of errors for audit courses that also have an honor seat:

2020-06-01 14:23:50,526 WARNING 5071 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:403 - Calculating course type failure occurred for [PolyMtl+CHE101.1: La chimie, en route vers le génie].

or

2020-06-01 14:24:18,857 WARNING 5071 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:572 - Seat type honor is not compatible with course run type audit for course run course-v1:UMontreal+MUS-001.1+P2018

I could give you tons of these WARNING under juniper.rc3 for all these audit courses with an audit seat.

In the end this causes the following errors:

2020-06-01 14:24:19,840 WARNING 5071 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:326 - Processing failure occurred caused by an exception on at least on of the threads, blocking deletes.
2020-06-01 14:24:19,840 ERROR 5071 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - EcommerceApiDataLoader failed!
Traceback (most recent call last):
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader
    loader_class(*loader_args).ingest()
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py", line 329, in ingest
    raise CommandError('Max retries exceeded and Ecommerce Data Loader failed to successfully load')
django.core.management.base.CommandError: Max retries exceeded and Ecommerce Data Loader failed to successfully load
CommandError: One or more of the data loaders above failed.

Because of these errors running update_index does not return or isn’t able to return the right numbers for the course runs:

Indexing 0 People
Indexing 357 courses
Indexing 151 course runs
Indexing 0 programs

Running course discovery on ironwood returns the following numbers on “approximately” the same data set (I have more courses in my Ironwood test environment):

Indexing 0 People
Indexing 344 courses
Indexing 358 course runs
Indexing 0 programs

One thing I have noted is that Honor courses seem to be treated differently in Ironwood:

2020-05-17 22:30:04,946 INFO 14601 [course_discovery.apps.course_metadata.models] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/models.py:888 - Course run [course-v1:PolyMtl+CHE101.1+E2015] is not publishable.
2020-05-17 22:32:06,759 INFO 14601 [course_discovery.apps.course_metadata.models] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/models.py:888 - Course run [course-v1:UMontreal+MUS-001.1+P2018] is not publishable.
2020-05-17 22:32:06,761 INFO 14601 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:165 - Processed course run with UUID [58aceb75-b8a2-430b-914f-41a3fc108807].
2020-05-17 22:32:06,765 INFO 14601 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:199 - Processed course with key [UMontreal+MUS-001.1].
2020-05-17 22:32:06,765 INFO 14601 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:140 - Processed course with key [UMontreal+MUS-001.1].

Anyone from the course discovery team able to tell me what is going on?

My first guess is that course discovery no longer knows what to do with honor seats because those are only used in Open edX instances, but I could be wrong.

2

If needed, I could share with the course discovery development team or the devops team the output for both runs on ironwood and juniper. Let me know where to send them.

3

“Calculating course type failure occurred for” in juniper come from course_discovery/apps/course_metadata/api.py after calling calculate_course_type from course_discovery/apps/course_metadata/data_loaders/course_type.py

calculate_course_type and course_type.py didn’t exist in ironwood

There seems to be a “python manage.py backpopulate_course_type --partner edulib” command available.

I’ve also seen this in the Juniper wiki.

    • We have created a new Microfrontend named Publisher (https://github.com/edx/frontend-app-publisher/) for creating courses and course runs. Although primarily used to keep track of course metadata (such as marketing information), Publisher can also help manage ecommerce products. Publisher integrates between edx-platform, ecommerce, and course-discovery. (Dillon Dumesnil)
      • Note: At this point in time, there is no standard process for installing Microfrontends, but Publisher is still being provided so the community can become more familiar with it and possibly configure and install it on their own.

Are we forced to install Publisher now?

Can someone answer these questions please?

@nedbat Who should we talk to?

4

Problem might originate from here:

Audit courses with an honor seat might not be considered ?

I am trying to figure this out but I would really need some help. I am not sure if there is an “Audit and Honor” or “Honor and Audit” run type or seat allowed.

5

I will ask a simple question.

What is the simplest way to migrate the data coming from course discovery running in Ironwood to course discovery in Juniper?

Under Django Admin, course metadata is very different between the two and I believe some data is “lost in migration” for a lack of a better term.

I am concerned that it does not recognize audit course with an honor seat.

For example, this is an audit course with an honor seat because we wanted to deliver honor certificates:
2020-06-01 19:52:21,883 WARNING 20759 [course_discovery.apps.course_metadata.data_loaders.api] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:572 - Seat type honor is not compatible with course run type audit for course run course-v1:PolyMtl+CHE101.7+H2020

What and where do I need to fix this? Any suggestions would be greatly appreciated.

6

This conversation continued in the Open edX slack in the #build-test-release-wg channel (https://openedx.slack.com/archives/CK94QNCQ0) and also in the Open edX Contributors Discussion meeting. During the Open edX Contributors Discussion meeting, I explained that internally to edX, Honor is not a Seat Type that we use and it remains in the code base for historical reasons. This was central to the reason this issue was showing up for Pierre (and would show up for many others) because we did not add Honor as a default value when setting up Course Types and Course Run Types.

I asked during the meeting for how widespread Honor is used and if it should be supported as a default shipped in our Open edX installations and it was immediately clear that Honor is very widely used in the Open edX community. As a result, we will be updating the defaults for Course Types and Course Run Types to include Honor.

This discussion also helped expose the continued disconnect between edX and the Open edX community. We now want to figure out how we can best communicate between edX and Open edX to ensure misses like this are less frequent in the future.

1 Like
7

For example, if we go back to audit courses that could deliver certificates in Open edX, we also need to consider that we also need audit-verified courses where only the verified track can deliver certificates. I could be wrong, but this could be a need to differentiate between audit and honor modes.

8

It would be great to have a per-course advanced config to allow audit certificates. I don’t know if this would be a problem for edx, to allow course staff to enable these certs. It might be combined with a site-configuration like ALLOW_COURSE_AUDIT_CERT to enable this advanced config.
Over all, having a full honor mode is quite a burden if it’s just for certificates. Specially considering that we need to go to django admin to create the mode and move the audit students manually. I would be happier if we only had audit and the possibility to issue certificates there.

9

Hi all, I am starting work on adding the defaults of Honor to Course Types. I am hoping to get some insight into all of the different formats Honor typically shows up in. Below is my current list and I’d love some feedback on it. Feel free to also pull in others if they may have insights as well.

  1. Honor only (Honor mode with Honor seat)
  2. Audit and Honor (Audit and Honor modes with Audit and Honor seats)
  3. Verified and Honor (Verified and Honor modes with Verified and Honor seats)
  4. Verified, Audit, and Honor (Verified, Audit, and Honor modes with Verified, Audit, and Honor seats)
  5. Credit with Honor (Credit, Verified, Audit, and Honor modes with Credit, Verified, Audit, and Honor seats)

Do these sound right or do some of these situations never occur? Are there other situations that do occur? In our internal system, we saw a Credit, Verified, and Honor (no audit mode or seat). Is that more likely in the Open edX community?

I am going to be running some more tests, but wanted to make sure I check in here as well.

1 Like
10

Hi Dillon,
In our case the only track we only is Honor, but we do not use the ecommerce app. We create the honor mode manually. @sambapete said that they use honor and verified tracks.

11

@Andres.Aulasneo I told @Dillon_Dumesnil that we use audit and honor. That’s the way we make sure the audit courses have an honor seat and are able to generate an honor certificate.

12

@sambapete When you say you use Audit and Honor, does that mean your Course has both the Audit and Honor modes in the LMS as well as both Audit and Honor seats in Ecommerce/Course Discovery?

I tried recreating that type of course locally, but mine only included a Honor Course Mode in the LMS and only a Honor seat in Ecommerce/Course Discovery.

Although, I’m also running into some other issues with saving in the E-Commerce Course Authoring Tool right now so it might just be an issue with my setup which is why I want to verify with you.

13

And good to know Andrés. Given that, hopefully you won’t run into the same issues Pierre was experiencing and it is still helpful to know how others in the community use Honor!

14

@Dillon_Dumesnil What I meant is that the course is created as Audit in Studio. When we add it to ecommerce we add an Honor seat to that course. Before we open the inscriptions to the public, we need to make sure the students will be registered as honor.

Sorry for the later reply, I was in a meeting :slight_smile:

15

@sambapete Can you outline the steps you take to create an Honor Course?

I believe I did

  1. Create Course in Studio
  2. Create Course in E-Commerce as Audit with Honor seat
  3. Create Course in Course Discovery
  4. Run Refresh Course Metadata.

For me, this resulted in a Course with only the Honor seat in Ecommerce and Course Discovery and only the Honor mode in LMS/Studio.

16

@Dillon_Dumesnil That seems similar to what I do, but with a slight difference.

I did not create the Course in Course Discovery. In Ironwood it was created as part of the Refresh Course Metadata where I add added Studio and Ecommerce in the API Configuration of the Core ----> Partners ----> edulib in Discovery’s Django Admin.

I did the same in Juniper and that’s where I am having problems when running Refresh Course Metadata.

17

@sambapete would you mind testing something for me? I’m trying to see if it’s broken for me or everyone. If you go to one of your Honor courses in the E-Commerce Course Authoring Tool and click on the Edit Course and then click the Save button (no changes necessary), do you receive an error about a duplicate entry for a partner_stockrecord?

I believe the Course Authoring Tool tries to add an additional Honor seat (aside from the one that already exists) and it causes this error.

18

An additional question, do Audit and Honor Seats ever co-exist? I know the E-Commerce Course Authoring Tool refers to the Course Type as Audit and creates a Honor Seat, but products-wise, there only is a Honor seat.

I also have tried creating a Verified and Audit Course with an Honor seat in the E-Commerce Course Authoring Tool and it only created Verified and Honor seats (again, missing the Audit seat).

This has led me wonder if Audit and Honor products ever co-exist. Please let me know if you have ever seen/used this.

19

@Dillon_Dumesnil I will only be able to test it again early next week. I’ve shut down (read terminate in AWS terms) my current juniper.rc3 test setup in order to install a new juniper.1 test setup later today, this week or early next week. Sorry for the inconvenience.

I don’t believe Audit and Honor seats were coexisting in Juniper, at least not when I look at course_modes in the LMS Django Admin in Ironwood. Those entries are created by ecommerce normally when you use the Course Authoring Tool.

What I can tell you is that in ecommerce under Ironwood the Course Type for these courses shows as Audit even if they have an Honor seat. But when you access the course before modifying it, the course seat is Honor.

See what I have from Ironwood.

Screen Shot 2020-06-10 at 1.44.24 PM
Screen Shot 2020-06-10 at 1.44.24 PM2994Ă—1250 250 KB

Under Ironwood, trying to edit that course, not making any modifications and saving does not return a visible error in the ecommerce console. I might have to check the ecommerce logs now. it does not seem to produce any errors in Ironwood. I will definitely test it again when I reinstall juniper.1.

20

@sambapete Having you test it in Ironwood should be enough. I don’t believe we have changed much of the functionality on this page so the experience between Ironwood and Juniper should line up. Thank you!

Good to know that it seems to only have an Honor seat and not both Honor and Audit. I will move forward in creating the Course Types (and sub models) to include Honor and without Audit as these seem to be the only types that are created. This will end up looking like:

  1. Honor Only
  2. Verified and Honor
  3. Credit, Verified, and Honor

And the error would have been very visible and block the ability to save so it must be something I did locally. Thank you for taking a look!