Hello! I’m Kyle from tCRIL, and I’ve been working to make it easier to use Tutor as an Open edX development environment.
This post is dual-purpose:
First, I want to share my new Tutor plugin: “quickdev”. It builds upon Tutor development improvements we made for Nutmeg (notably, the
--mount functionality), aiming to further improve performance & reduce complexity. If you’re a Tutor power user, feel free to check it out and let me know what you think! That being said, don’t count on the plugin existing in the long-term, which brings me to the second purpose of this post…
I’d like to propose to incorporate some or all of quickdev into Tutor proper. To be more specific, I’d like to make these changes to
Define named volumes for the LMS/CMS venv, egg-info, node_modules, and generated static assets. Declare them in the Dockerfile so that they’re auto-populated when LMS/CMS are first started.
Do the same in the official Tutor plugins that add services: discovery, notes, mfe, etc.
In order to prevent the contents of these named volumes from becoming stale over time, provide a mechanism for deleting them so that they can be freshly populated from the image. We could have an automatic way, a manual way, or both:
i. Automatic: Delete the named volumes regularly by triggering on a common event An aggressive version of this would be deleting the named volumes whenever the platform is stopped. A less-aggressive idea would be to clear them whenever
launchis run, as well as whenever the related Docker image is pulled or built. We would need to add a few new hooks to implement this:
IMAGE_PULLEDactions, providing a way to to trigger the volume deletion; and
COMPOSE_DEV_IMAGE_NAMED_VOLUMESfilter in order to map images to their named volumes.
ii. Manual: Create commands to clear out the named volmes on-demand. In quickdev, those commands are
tutor quickdev [pip|npm|static]-restore, but they are not extensible. In Tutor, we could have
tutor dev restore <image> <volume>. For example:
tutor dev restore openedx venvwould delete the named volume holding the edx-platform virtual environment. This command could also use the new
COMPOSE_DEV_IMAGE_NAMED_VOLUMESfilter in order to map
"<image> <volume>"to actual Docker volumes.
Automatically mount suspected edx-platform packages to a standard location. In quickdev, the location is
/openedx/mounted-packages, and the recognized prefixes are:
xblock-*, a fairly established pattern for XBlock repos,
platform-plugin-*, which 2U/edX started using recently and I like, and
platform-lib-*, which is not a pattern yet, but I wouldn’t mind encouraging it.
The same thing could be done in plugins if we could figure out any reasonable naming conventions for their packages.
Provide a command for installing all packages mounted in the standard location mentioned above In quickdev, that’s
tutor quickdev pip-install-mounted -m ... -m .... In Tutor, this could be implemented using the upcoming
CLI_DO_COMMANDSfilter, giving us
tutor dev do pip-install-mounted -m ... -m ... -m ...
Proposed benefits of incorporating quickdev:
- Developers would see better performance when working with bind-mounted repos.
- The extra setup steps required after bind-mounting a repository would go away.
- The need for image rebuilds or virtual environment bind-mounting when working on platform packages would go away.
- More state would be persisted between platform restarts, which increases the number of things that could be “wrong”, invisible to the developer.
- More implicitness:
tutor dev runwould now share a virtual environment with other containers, even though the command doesn’t make it clear.
- Increased difference between
tutor dev, since
localmode wouldn’t use these named volumes.
- I think this system could ease frontend development (using tutor-mfe) as well, but I’m not certain yet. I’ll need to do more research here, but I wanted to get this TEP out there first.
@tutor-maintainers @regis , let me know what you think!