Django 2.2 upgrade: statistics and lessons learned

Development
1

All edX production services in active use are now running Django 2.2! This is an incredible milestone for our platform and for the Open edX community as we have moved off of the now unsupported Django 1.11 version, and onto a version which will be supported for the next 2 years.

Thanks to all the people from the edX Architecture squad, Arbisoft BOM squad, eduNEXT, and others who helped get this done! Also thank you all for your cooperation in helping us upgrade the services.

Some statistics for the curious:

  • Production services that had to be updated: 14
  • edX repositories that had to be updated: 62+
  • Total time to finish the upgrade: about 5 months
  • Staff and contractors who worked full time on this for at least a month: 14

Although the primary goal was to ā€œupgrade to the next still-supported Django releaseā€, in order to accomplish that we had to ā€œcomplete remaining tasks from the last 3 major Django release upgrades, do the next 3 upgrades, and do some deferred maintenance from the past 2+ years in most of our codeā€. That is, in addition to upgrading Django, we also:

  • Made dozens of repositories comply with OEP-18, getting their Python package dependencies up to date and making it easier to keep them that way.
  • Switched dozens of repositories from the nose test runner to pytest.
  • Upgraded pylint significantly in most of these repositories, and fixed all new warnings in most cases.
  • Upgraded to Python 3 two services that had been skipped during the original Python 2 to 3 upgrade.
  • Did a major refactoring and cleanup of authentication code in most of our services (fondly known as DOP-recation).
  • Developed new automation and processes to streamline future upgrades.

A summary of problems encountered during the upgrade and lessons learned from it can be found at https://openedx.atlassian.net/wiki/spaces/AC/pages/1525252769/Django+2.2+Upgrade+Issues+and+Lessons+Learned .

In our new approach of continuous upgrades, we proactively moved on to upgrading our codebase from Python 3.5 to Python 3.8. Thanks to the Arbisoft BOM squad and eduNEXT for their ongoing effort. Stay tuned to hear more of our upgrade updates as we enable quality engineering with up-to-date technology standards.

9 Likes