Python 3.11/3.12 upgrade for Tutor and its plugins

The Redwood :evergreen_tree: release, which is cut early April and release early June, will include the upgrade of Open edX services from Python 3.8 to 3.12. As those upgrades begin happening upstream, let’s use this thread to coordinate upgrading Tutor and its plugins’ Dockerfiles to installing Python 3.12.

Rather than switching to 3.12 all at once for Tutor v18 in June, I think we should switch over core and plugins on the Nightly branches as soon as possible. That way, Tutor Nightly users can begin running and testing 3.12 ahead of Redwood release.

The last time a big Open edX Python upgrade happened was 3.5 → 3.8 in Koa. At that time there were fewer plugins and no such thing as Tutor Nightly, so we are trodding a new path :smile:

@tutor-maintainers, could you all confirm that:

  1. If your plugin runs a Python service: you’ll keep an eye on that service’s Python upgrade process, so that you can switch your plugin’s Nightly branch over to installing Python 3.12 as soon as it’s ready.
  2. Separately, for every plugin: the actual Python code in your plugin is compatible with 3.12.
8 Likes

Just wondering - will this include the ancilliary components like the Tubular pipeline for user retirement? That’s currently running Python 3.8.

Hi. For tutor codebase Python 3.12 upgrade, the issues have been created in various repositories. We are tracking the overall progress on Python 3.12 upgrade · Issue #1004 · overhangio/tutor · GitHub. We might use the same issue to track Tutor compatibility with 3.12 versions for Open edX services (once upgraded). For now, our focus is to make the Tutor codebase compatible with Python 3.12.

1 Like

We’ve just moved the user retirement code from tubular to edx-platform, so the edx-platform Python 3.12 upgrade will take care of that bit: Move user retirement scripts code from the tubular repo by farhan · Pull Request #34063 · openedx/edx-platform · GitHub

Do you know of any other parts of tubular or other openedx repos that Tutor relies on?

I honestly know very little about Tutor as we don’t really use it meaningfully at MIT unless the devs are using it.

Thanks for the heads-up. We implemented some local tubular changes at MIT so we’ll need to figure out how to handle this. (ISTR it’s around letting tubular consume environment variables for its required secrets.

The Python 3.12 update has landed on the edx-notes-api repo, any tutor updates that need to happen to make that the default for the edx-notes-api plguin should be made.

Relevant PR on the app side: Add support for Python 3.12 by irtazaakram · Pull Request #392 · openedx/edx-notes-api · GitHub

2 Likes

@jhony_avella , as tutor-notes maintainer, could you confirm that you’ll soon upgrade the plugin to install Python 3.12 in its Dockerfile?

@kmccormick the updates are on my plate. Due to a very tight schedule, I plan to start with this effort next Monday, April 1. Is it OK?

1 Like

Sounds good, thanks @jhony_avella !

UPDATE: For Redwood, edx-platform will be upgraded to Python 3.11, not Python 3.12. In Sumac, we’d go to 3.12.

All other services are still aiming for Python 3.12 in Redwood.

1 Like

@kmccormick for my own education, what’s the reason behind this decision? Are there too many breaking changes in 3.12? Or are there too many dependencies that don’t work on 3.12?

While Kyle and Feanil will have more detailed answer, from what I know, there are some libraries that aren’t working on 3.12 and 3.8 like numpy and scipy which are present in various repos, so we need 3.11.

1 Like

@feanil Do you know the exact reason?

The reason that edx-platform is going to 3.11 and not 3.12 is because it depends on Numpy/SciPy and there is not a version that is compatible with Both 3.11 and 3.8. Once we get to 3.11 we can update the two packages further and be able to go to 3.12 for Sumac.

For edx-platform dependencies, where possible we should be aiming to test on both 3.11 and 3.12 today and identify any other places what that’s not possible today.

3 Likes