After a harrowing journey that took over a year, I’ve finally got the ability to export many libraries from a course on my beta server, and import them to my production server… Since I have over 40 libraries in my next course, this was a must-have.
However, now that I’ve exported and imported the libraries, I can’t actually import the course that used them, because it’s saying that it can’t find the libraries! There seems to be some sort of version number string which is getting changed between export and import, thus nullifying the actual capability to import.
I do:
tutor local run -v /home/tutor/all_course:/tmp/bla cms ./manage.py cms import /tmp/bla/
And I then get
2022-05-08 13:17:12,545 ERROR 1 [xmodule.modulestore.xml_importer] [user None] [ip None] xml_importer.py:484 - Course import course-v1:MyOrg+MyCourse+MyRun: failed to import module location i4x://MyOrg/MyCourse/library_content/04cd1eff20154ce684890373f134b80b
Traceback (most recent call last):
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 474, in depth_first
_update_and_import_module(
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 888, in _update_and_import_module
LibraryToolsService(store, user_id).update_children(
File "/openedx/edx-platform/common/lib/xmodule/xmodule/library_tools.py", line 163, in update_children
raise ValueError(f"Requested library {library_key} not found.")
ValueError: Requested library library-v1:MyOrg+MyLibrary+branch@library+version@625822eb0dcf1a4591bb4903 not found.
2022-05-08 13:17:12,612 WARNING 1 [edx_toggles.toggles.internal.waffle.flag] [user None] [ip None] flag.py:79 - Flag 'cms.export_course_metadata' accessed without a request
2022-05-08 13:17:12,618 INFO 1 [openedx.core.djangoapps.content.course_overviews.models] [user None] [ip None] models.py:260 - Attempting to load CourseOverview for course course-v1:MyOrg+MyCourse+MyRun from modulestore.
2022-05-08 13:17:12,622 INFO 1 [openedx.core.djangoapps.content.course_overviews.models] [user None] [ip None] models.py:177 - Updating course overview for course-v1:MyOrg+MyCourse+MyRun.
2022-05-08 13:17:12,655 INFO 1 [openedx.core.djangoapps.course_date_signals.handlers] [user None] [ip None] handlers.py:85 - Extracting course dates for course-v1:MyOrg+MyCourse+MyRun
2022-05-08 13:17:12,662 INFO 1 [openedx.core.djangoapps.course_date_signals.handlers] [user None] [ip None] handlers.py:108 - Extracting dates from 45 items in course-v1:MyOrg+MyCourse+MyRun
2022-05-08 13:17:12,663 INFO 1 [edx_when.api] [user None] [ip None] api.py:81 - Setting date for BlockUsageLocator(CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None), 'course', 'course'), start, datetime.datetime(2022, 5, 15, 0, 0, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6af3f1d00>)
2022-05-08 13:17:12,666 INFO 1 [edx_when.api] [user None] [ip None] api.py:81 - Setting date for BlockUsageLocator(CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None), 'chapter', 'bb87c99ce3c446399457ac9bf0f73f48'), start, datetime.datetime(2022, 5, 15, 0, 0, tzinfo=<bson.tz_util.FixedOffset object at 0x7ff6af4a1e20>)
2022-05-08 13:17:12,668 INFO 1 [edx_when.api] [user None] [ip None] api.py:81 - Setting date for BlockUsageLocator(CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None), 'chapter', '1544c07c742c4a98b74d34abd1df6383'), start, datetime.datetime(2022, 5, 15, 0, 0, tzinfo=tzlocal())
2022-05-08 13:17:12,705 INFO 1 [celery_utils.logged_task] [user None] [ip None] logged_task.py:25 - Task lms.djangoapps.discussion.tasks.update_discussions_map[d434d912-2267-4c2a-b311-c01267ed55cf] submitted with arguments [{'course_id': 'course-v1:MyOrg+MyCourse+MyRun'}], None
2022-05-08 13:17:12,705 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function listen_for_course_publish at 0x7ff6b227ca60> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,706 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function export_course_metadata at 0x7ff6b2281c10> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,706 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function _listen_for_course_publish at 0x7ff6b1f871f0> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,706 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function _listen_for_course_publish at 0x7ff6b1f874c0> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,707 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function update_block_structure_on_course_publish at 0x7ff6b1f9b820> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,707 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function _listen_for_course_publish at 0x7ff6b1fa2e50> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,707 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function extract_dates at 0x7ff6b1e70b80> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,708 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function trigger_update_xblocks_cache_task at 0x7ff6b1d69820> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
2022-05-08 13:17:12,708 INFO 1 [xmodule.modulestore.django] [user None] [ip None] django.py:199 - Sent course_published signal to <function update_discussions_on_course_publish at 0x7ff6b1d26790> with kwargs {'course_key': CourseLocator('MyOrg', 'MyCourse', 'MyRun', None, None)}. Response was: None
Traceback (most recent call last):
File "./manage.py", line 123, in <module>
execute_from_command_line([sys.argv[0]] + django_args)
File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/openedx/edx-platform/cms/djangoapps/contentstore/management/commands/import.py", line 66, in handle
course_items = import_course_from_xml(
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 762, in import_course_from_xml
return list(manager.run_imports())
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 547, in run_imports
self.import_children(source_courselike, courselike, courselike_key, dest_id)
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 660, in import_children
self.recursive_build(source_courselike, courselike, courselike_key, dest_id)
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 497, in recursive_build
depth_first(source_courselike)
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 495, in depth_first
depth_first(child)
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 495, in depth_first
depth_first(child)
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 495, in depth_first
depth_first(child)
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 474, in depth_first
_update_and_import_module(
File "/openedx/edx-platform/common/lib/xmodule/xmodule/modulestore/xml_importer.py", line 888, in _update_and_import_module
LibraryToolsService(store, user_id).update_children(
File "/openedx/edx-platform/common/lib/xmodule/xmodule/library_tools.py", line 163, in update_children
raise ValueError(f"Requested library {library_key} not found.")
ValueError: Requested library library-v1:MyOrg+MyLibrary+branch@library+version@625822eb0dcf1a4591bb4903 not found.
And then if I go manually delete the usage of the first library that breaks, and then re-try the import, it breaks on the second library (which is used many places, so deletion as a workaround is not an option.)
I think the only thing I could perhaps do is try to use e.g sed to fix up all the “version@625822eb0dcf1a4591bb4903” numbers… But how can I find these numbers for the libraries which are already imported into the production server? I can’t see such numbers in the UI, and I didn’t see them in the CLI at import time either?
And more generally, what’s the correct way to do this so that the sed workaround isn’t necessary?