We’re running into this issue mentioned above for students to be able to view the MFE learning page when Course Visibility in Catalog is set to none. The students receive the There was an error loading this course. message and nothing loads on the Course Home page. Also the LMS log files indicate no error messages so we were lucky to find this discussion article.
I’m trying to write a management command to update all courses that have this value for Course Visibility in Catalog set to none to about. Also after updating this value do I need to send a republish signal for the course, so that, the openedx.course_overviews_courseoverview updates too?
How can I get advanced settings for a course and override those values, then republish the course after the change if necessary? I don’t want to go through all our courses manually and would like a Django management command to handle this.
So far I can get all field names with this code but not output the existing value or set it.
from opaque_keys.edx.keys import CourseKey
from xmodule.modulestore.django import modulestore
course_module = module_store().get_course(CourseKey.from_string(course_id))
# `str(field) doesn't output the value of the advanced setting.
for field in course_module.fields:
print(f"Field {field} is set to {str(field)}")
This appears to be some help toward that. Please let me know if you think this will work or not.
I suppose it may be easier to just call the PATCH endpoint here since it checks to see if the course exist, permissions to update, and calls that update_course_advanced_settings helper method.
I was in simliar situation the way I did it, was by modulestore().update_item(course_module, user.id) note that this what the update_course_advanced_settings method does, hence the below.
So the way I did is probably the dirty way because it bypasses the checks you mentioned.
So a complete example of the drity way to do it
from opaque_keys.edx.keys import CourseKey
from xmodule.modulestore.django import modulestore
coursemodule = module_store().get_course(CourseKey.from_string(course_id))
course_module.hide_progress_tab = True #Just an example to change something
modulestore().update_item(course_module,staff_id)
I see that the original bug report has been closed because it was “stale”. That doesn’t mean it was fixed, right? Can someone confirm it was fixed in Nutmeg and it isn’t an issue in Olive?
We also have courses that are set to “Course Visibility in Catalog == None” and I would want them to stay that way when we upgrade our production server from Koa Native to Nutmeg Tutor in a couple of weeks.
Unless I misunderstand what is going on, does that mean “Course Visibility in Catalog == None” no longer works under certain circumstances? If that is so, this is a problem that needs to be fixed and not toss under the rug.
I just want to point out something interesting I just found out.
I currently have a test server with Nutmeg.3 with a recent copy of our data from Koa.3. The test server was upgraded from Native Koa.3 to Tutor Koa.3 to Tutor Lilac to Tutor Maple and finally to Tutor Nutmeg.
I may not have observed the behavior mentioned by @Zachary_Trabookis because most of my courses that were with “course_visibility == none” in Native Koa.3 are now with “course_visibility == both” under Tutor Nutmeg.3 and I have absolutely no clue as how they ended up that way…
Is there something on the way from Koa to Nutmeg that changed the course_visibility in the background? This was a complete shock to me.
Speaking for the BTR group (which owns the Nutmeg-targetted issue that I closed last week), our involvement hinges on:
The issue being properly identified, with instructions to reproduce it, and ideally (but not necessarily) for it to be part of the community test plan;
The issue has to affect the latest (and only) supported release: in literally half an hour, that is going to be Olive;
An assignee has to step up to manage the issue, which doesn’t mean they’ll have to fix it: they just chase after somebody to do it.
At this moment, as far as BTR is concerned none of the 3 are satisfied.
That doesn’t mean as a community we can’t fix the issue or draw attention to it. We can start by simply opening an issue to edx-platform with the reproduction instructions.
After trying a new migration from Native Koa to Tutor Koa and up to Tutor Nutmeg, I have discovered that the values for course_visibility changed during the Tutor Lilac to Tutor Maple upgrade.
Meaning it was never properly raised as an issue to edx-platform.
And this brings back a question I frequently had for the BTR.
We do all our tests on a completely newly installed server but never on a server with history or for which the latest release is not the first release installed on that server. We might be missing test cases when we start from scratch that do not pop up when we upgrade from Nutmeg to Olive for example. I am not saying this is the case here, but I did encounter it after reading @Zachary_Trabookis description of the problem and checking on a server I recently upgraded from Native Koa to Tutor Koa up to Tutor Nutmeg.
Helps upstream be aware that there is an issue in the first place: not many people look at the BTR board other than BTR itself;
Makes it clear that while BTR found the issue, they’re not necessarily volunteering to fix it.
I totally agree. I would say from past experience that this is mostly for lack of resources to do upgrade tests. As the group grows, hopefully we can address this. We should bring this up in the next BTR meeting. I’ll try and add it to the next agenda, but if I forget to do it, please remind me, @sambapete.
@ghassan Thanks for letting me know. It seems like after I called the edx-platform/cms/djangoapps/contentstore/views/course.py:update_course_advanced_settings that it didn’t update the advanced field.
I’m going to check out the call to edx-platform/cms/djangoapps/models/settings/course_metadata.py:validate_and_update_from_json then look into the modulestore().update_item(course_module, user.id) to see if that helps. Both of those calls are within the update_course_advanced_settings method.
I don’t see anywhere in validate_and_update_from_json where it actually sets something like you did above with course_module.hide_progress_tab = True. That’s probably why it’s not working.
Consdier the following case of changing the course calcualtor, pelase pick any course and try to change if the calcualtor should show up: given calcualtor is considered advacnted settings
from opaque_keys.edx.keys import CourseKey
from xmodule.modulestore.django import modulestore modulestore().get_course(CourseKey.from_string(string_id))
coursemodule.show_calculator = False # Just toggle it
modulestore().update_item(coursemodule,author_id)
Now after this I was able to see the the changes, I cheked with the learning the MFE: but please note the following:
It doesn’t what I did in the above code, the value of show calcualtor in advacned settings i.e. in https://stuido.example/settings/advanced/course-id` Would not change however the value I change in the above script would take the lead.
The testing is done on olive.master branch.
Side note advnace settings legacy UI, is kinda depcreated or in the process to, for example so of the stuff would be changed through course authoring MFE even from Nutmeg.
My 2 cent is that we can work on maintaing data dump for that cover most of the cases, and this data dumb can be incrementally migrated between each release. Plus it can be used to for developer to do quick migration/pre filling…etc
Since I am using AWS, I took the time last evening and early this morning to do a complete migration from Koa Native to Koa Tutor, then to Lilac Tutor and then to Maple Tutor.
As of now, my courses are still at “course_visibility == none”. In my last migration test up to Nutmeg Tutor, I had discovered that some courses had miraculously switched to “course_visibility == both". And I probably know why now.
On https://openedx.atlassian.net/wiki/spaces/COMM/pages/3205201949/Nutmeg it is mentioned that: * An internal performance improvement called “learning sequences” has been opt-in for a few releases, but is now always-on for Nutmeg. If you have any courses that have not been re-published on Koa or later, run the simulate_publish cms django command on your courses before upgrading, to populate the learning sequence data. @Michael Terry (Deactivated)
Well, I do have a few courses that have not been re-published on Koa or later because they are always opened.
As a result, before upgrading to Nutmeg I ran:
tutor local exec cms bash
python manage.py cms simulate_publish
And suddenly some of my courses switched from “course_visibility == none” to “course_visibility == both”. This is why I never experienced the problem that @Zachary_Trabookis encountered.
I am not that surprised because the output of the simulate_publish command does refer to openedx.core.djangoapps.content.course_overviews.models heavily and most probably updates the database as a result.
Should the simulate_publish cms django command be run while I am on Maple or only when I am in Nutmeg? It does say to run it on your courses before upgrading. So for me, it is while I am on Maple.
At last now, I understand why I never encountered @Zachary_Trabookis problem with my tests.
@sambapete I think no before upgrading to Maple but it sounds like we need to do that when we upgrade to Nutmeg. I think I tried running that after we already upgraded to Maple but nothing happened.
In my case, it ran for a few minutes. And it did change the “course_visibility == none” to “course_visibility == both” for some courses. It probably did so because it is still at DEFAULT_COURSE_VISIBILITY_IN_CATALOG = “both” when I run it.
SEARCH_SKIP_SHOW_IN_CATALOG_FILTERING = False
Long conversation can be found here
I don’t if that exactly related to your case, but sounds relevant.and the above suggested setting is in tutor by default since Nutmeg, probably this is why @sambapete doesn’t have same behavior
I can now confirm that when in Maple, if I run the simulate_publish command, some courses will end up taking whatever is the value of DEFAULT_COURSE_VISIBILITY_IN_CATALOG as their new value for course_visibility. I’ve tried it with both, about and none.
FWIW since it’s my bug that was reference at the beginning[1], I definitely ran simulate_publish on my lilac → nutmeg upgrade, as recommended by you to me here
[1] @arbrandes I was very upset when I saw the email that you unceremoniously closed the bug I took the time to file, without any evidence that it was fixed yet. You should have at least provided the criteria you provided in this thread, in the ticket as it was being closed, so I would have known what was required to re-open it. (I still wouldn’t know the criteria if my thread hadn’t been referenced here for me to come read.) In general after about a year of working with Open edX the distinct impression I’m getting is that bugs filed by the community never get fixed, features that worked before keep breaking, never to re-work (YouTube API, Google Analytics, Anonymous Course access), and the developers keep caring more about adding new features and refactoring code in place just to make themselves happier with the aesthetics, than fixing breakage, if it doesn’t affect the couple commercial vendors. This isn’t quite the straw that broke the camel’s back, yet…but I’m getting very very close to abandoning OEX.
My sincerest apologies for the effect this had. It was definitely not my intention to upset anybody, and least of all to give the impression that developers don’t care about bug reports. I can’t speak for everybody, but as an Open edX developer myself, I very much do, and consider them to be a valuable contribution.
Plus, I agree I should’ve given better reason in the ticket, even if just by linking to an explanation elsewhere (which I did a couple of days after the fact). I know it’s little consolation, but this was for simple lack of time: as BTR chair, I had to make quick judgement calls as to what we would devote our limited resources to fixing as the release approached, and the issue in question had not seen any activity in 4 months.
With that in mind, I take it you understand the reasoning behind the criteria for any particular issue to be made the Build-Test-Release working group’s concern. The group is entirely community supported, and it doesn’t have the manpower to fix everything: no single organization in this community does. If nobody is interested enough to assist in fixing the issue - even if just by volunteering to try and find somebody to do so - there is not much else the group can do.
As I mentioned earlier, however, it doesn’t mean other groups or individuals in the community can’t do something about it. A good strategy, for instance, is to report the issue upstream in the Github repository where the problem is presumed to be found. Another is to try and reproduce the issue on the latest release, which will get it more attention. None of this is guaranteed to get it fixed, but… It’s a start.
All that said, I understand your frustration, and sympathize. We’re all striving to make Open edX a better project that does not lead to such feelings. Perhaps, though, I could convince you to join more BTR meetings or the #wg-build-test-release channel as a way of making your needs heard? I’m sure that if we’d gotten this issue into the Community Test Plan we might have found a more satisfactory conclusion prior to Olive.