MySQL and MongoDB schema—and schema change—documentation?

Site Operators
,
1

I have some old Eucalyptus backups to restore, and bring up to Juniper.

Really I just want to DB migrate MongoDB and MySQL all the way up from Eucalyptus to Juniper.

Attempts thus far, and their success:

  • Official .box files for Vagrant use: deploy of eucalyptus-devstack-2016-09-01.box failed
  • Next attempt with this Vagrant box—this time on Ubuntu host—worked but ended with a vagrant-devstack.yml missing error
  • openfun/openedx-docker: deploy worked, database restore started to show promise, then realised there aren’t that many versions of the images available; and upgrading each within the currently-running container would be quite annoying and error prone
  • So I rewrote openfun/openedx-docker in Python (Fabric), but got errors from proj.settings not being defined
  • Native Ubuntu 16.04 install failed due to this sanity check; removing sanity-check left other errors
  • Native Ubuntu 12.04 install failed due to PGP errors from pgp.mit.edu and id 69464050 not being verifiable (see util/install/ansible-bootstrap.sh)

So it’s time to give up. I’ve just deployed the latest Juniper on a fresh Ubuntu 16.04 [cloud] VM.

My plan is to:

  1. Backup MongoDB and MySQL in latest Juniper [of demo course and demo users]
  2. Restore older backups to the right version of MongoDB and MySQL
  3. Compare schema of older backups to schema of Juniper
  4. Write custom SQL and MongoDB queries to translate from old to new

With this in mind, are there any existent schema-level documentation? - Preferably with changelogs?

(yes, I know of the Django migrations)

Thanks

2

There are 655 tables in the MySQL from Eucalyptus.

All I want to migrate to Juniper are:

  • student {auth, (grade, time) per question per course}
  • staff {auth, edits to course}
  • course {and edits}

Exploring the databases and tables, here are the non-empty non-default ones which look relevant, took some serious monotony to compile this:

  • edxapp.auth_registration
  • edxapp.auth_user
  • edxapp.auth_userprofile
  • edxapp.bookmarks_bookmark
  • edxapp.bookmarks_xblockcache
  • edxapp.course_action_state_coursererunstate
  • edxapp.course_groups_coursecohortssettings
  • edxapp.course_overviews_courseoverview
  • edxapp.course_overviews_courseoverviewtab
  • edxapp.course_structures_coursestructure
  • edxapp.courseware_studentmodule
  • edxapp.courseware_xmodulestudentinfofield
  • edxapp.courseware_xmodulestudentprefsfield
  • edxapp.courseware_xmoduleuserstatesummaryfield
  • edxapp.dark_lang_darklangconfig
  • edxapp.instructor_task_instructortask
  • edxapp.milestones_milestonerelationshiptype
  • edxapp.student_anonymoususerid
  • edxapp.student_courseaccessrole
  • edxapp.student_courseenrollment
  • edxapp.student_courseenrollmentallowed
  • edxapp.student_historicalcourseenrollment
  • edxapp.student_loginfailures
  • edxapp.user_api_userpreference
  • edxapp.util_ratelimitconfiguration
  • edxapp.verify_student_historicalverificationdeadline
  • edxapp.verify_student_verificationdeadline
  • edxapp.wiki_article
  • edxapp.wiki_articleforobject
  • edxapp.wiki_articlerevision
  • edxapp.wiki_urlpath
  • edxapp_csmh.coursewarehistoryextended_studentmodulehistoryextended

Expecting it be straightforward from here to successful migration, just by restricting the comparison between Eucalyptus and Juniper to these 32 tables.

(haven’t looked at the MongoDB side yet, what schema changes are there? - Yes I understand the schema in MongoDB isn’t ‘real’ the way SQL have tables)