How to Import multiple Courses/Libraries?

I made multiple libraries and courses in a local VM, and now I want to upload them to the production server. Is there a better way to import multiple classes at once, or do I need to import them all one at a time?

Using the Studio front-end
All you can do is export/import single courses in zip format.

However, if you have access to both servers, you can export all courses using management commands. These commands should be run within the edxapp virtualenv.

# Load the edxapp virtualenv
sudo -Hu edxapp bash
source /edx/app/edxapp_env
cd /edx/app/edx-platform

# Export all courses to a directory
./manage.py cms export_all_courses /your/output/path
# Import all courses from a directory
./manage.py cms import /your/exported/courses/path

Don’t forget to replace paths with ones of your choice.
There are similar commands to for libraries : export_content_library and import_content_library. Unfortunately they have some bugs so you can’t use them untill they are corrected.

1 Like

Are you aware of a ticket that’s tracking the export_content_library bug? Because I have way more libraries than courses, so that’d be the more useful functionality to me. So if there’s not a bug for that, then I should probably file one right?

(I searched for hits for this error signature but couldn’t find anything:
Traceback (most recent call last):
File “./manage.py”, line 123, in
execute_from_command_line([sys.argv[0]] + django_args)
File “/opt/bitnami/apps/edx/venvs/edxapp/lib/python3.8/site-packages/django/core/management/init.py”, line 381, in execute_from_command_line
utility.execute()
File “/opt/bitnami/apps/edx/venvs/edxapp/lib/python3.8/site-packages/django/core/management/init.py”, line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File “/opt/bitnami/apps/edx/venvs/edxapp/lib/python3.8/site-packages/django/core/management/base.py”, line 323, in run_from_argv
self.execute(*args, **cmd_options)
File “/opt/bitnami/apps/edx/venvs/edxapp/lib/python3.8/site-packages/django/core/management/base.py”, line 364, in execute
output = self.handle(*args, **options)
File “/opt/bitnami/apps/edx/edx-platform/cms/djangoapps/contentstore/management/commands/export_content_library.py”, line 65, in handle
shutil.copyfileobj(tarball.file, f)
File “/opt/bitnami/apps/edx/venvs/edxapp/lib/python3.8/shutil.py”, line 205, in copyfileobj
fdst_write(buf)
TypeError: write() argument must be str, not bytes

I could solve the one in export_content_library.py by editing line 64 (put wb instead of w in open mode)

            with open(target, 'wb') as f:

I couldn’t go any further with import_content_library. Unless this is solved, it’s irrelevant to export your libraries without a way to import them

@oedx I’ve filed a pull request here for bugfixes.
You can check it out and use it to correct your own code and do your import while it gets reviewed.

That’s very cool of you. It’s unfortunate they put up such a legal roadblock to providing this fix, that it will take forever to get incorporated :-/

@oedx just make the changes for yourself for the time being

Jumping in to clarify a couple of problems with that assertion:

  1. It’s a fact of life that any serious open source project will have a contributor’s agreement that needs to be signed before a contribution can be accepted. It’s just how copyright (and left :wink:) works. I urge you to take a look at the agreement. It’s pretty reasonable and understandable, as far as these things go.
  2. While many community contributions indeed take a significant amount of time to get merged, this is largely not because of the contributor’s agreement. Most often, the delay is simply due to the maintainers not having enough time to get to them all. The good news is that edX is actively looking to remedy this via the Core Commiters Program - i.e., delegating merge rights to community members that stand up to certain very reasonable criteria.

Which is all to say: worry not, there’s every chance that this fix will get into Lilac, the next Open edX release. :slight_smile:

(And since it looks like you’re a heavy library user, you might want to keep tabs on the new Library Authoring experience, which is currently under development.)

PS: @braden, as core commiter and original author of the libraries code, mind if I propose @ARMBouhali’s PR as an opportunity to demonstrate that Open edX is worthy of contributions? :wink:

Abderraouf, think you can get that agreement signed? Unfortunately, without your consent, there’s little else we can do in the meantime.

Sure, thanks for the nudge. I’d be happy to once it’s cleared for review :).

How does one “reach out to the Open edX team to explore potential contributions and improvements to this experience”? I can’t contribute because I can’t program in your high level languages, but I can at least say what I’m trying to get out of libraries (since using them as they currently exist is a bit of a hack just because there’s apparently no other way to include material in multiple courses.)

Well, in this case, your best bet is myself. :slight_smile: I’m not with the Open edX team, but I am leading this effort on the community’s side. If you manage to try it out, feel free to jot down your thoughts here. Code contributions always carry more weight, of course, but since this is a relatively new project, feedback is very valuable.

@ARMBouhali, have you managed to sign that agreement, yet? This is a slam-dunk contribution. We just need your consent.

Apparently I can’t make this kind of change myself in a Tutor-based environment without forking edx-platform and keeping that fork up to date, which I am loath to do. So it’d really be good if you could get this commit pushed through.

(See @arbrandes, this is how things stay broken. Requiring an IP agreement for a 3 line fix is a massive disincentive for potential contributors. This is why GCC moved away from this recently: GCC Will No Longer Require Copyrights Be Assigned to the FSF - Slashdot)

I agree, but unfortunately I’m (very) far from being the one who makes these rules.

The Developer Certificate of Origin idea is a good one, IMHO. We should bring it to the new Open edX leadership, once the process of acquisition is complete.