The edX support policy is that the latest Open edX release is the only one that is supported. So as of this moment, Ironwood is supported. This means that we will fix problems with installation, security, or data loss.
Hawthorn (and earlier) are not supported. Yet people still want to install those versions. What can we do when those installations stop working? @lpm0073 has opened a pull request to fix Hawthorn in light of numpy dropping support for Python 2. edX doesn’t have the resources to verify this fix, which is why we only support one version. What does the community want to do to keep old releases working?
it’d be judicious to create a community committee for any release which, like hawthorn, has both breaking changes and a base of community members who depend on the repo to continue to at least “work”.
i’d be in favor of:
this committee carrying out its duties under the supervision of a delegate from edX
the committee charter being limited to simply keeping the repo functional. that is, it would generally discourage pull requests for anything other than fixes to breaking changes in the code.
so in other words, its a committee that is staffed by the very people who depend on the repo, and, their mandate is basically to keep the repo on life support.
Definitely not a bad idea. Unfortunately, a lot of people with experience move on from a later release to a newer release. In our case, we’re always up front or ahead of the curve as we were told from the beginning that only the newest release would be supported.
And it is not easy to keep up with prerequisites for older releases as they simply may not be accessible anymore. Not everybody can support multiple releases in order to help others.
And there are a lot of newbies or non sysadmin / non devops people who are asked to manage an Open edX instance. The learning curve is difficult for some of them.
The easier/base way would be to automate the testing/deployment of the released tags, so at least we can have information about if its broken or not, and anyone interested can fix it. This would prevent anything unnecessary like having a committee, and the devs would only need to check the commit and CI to keep the branch working.
I imagine the only need to install older versions would be to migrate data while trying to upgrade instances to the newer versions. Is it possible to consider more number of master releases to support? as @lpm0073 said to ensure at least they continue to work. or maybe it would be easier to have one or two type of virtual machine images, with edx-platform installed, ready to use without any package installation needed which could be used to mount and to load&migrate data.
I love the idea of the community pulling together to help support the older releases they need. Who can put together a proof-of-concept of a continuous integration that installs the old releases?
@nedbat, we at OpenCraft are looking into setting up periodic installs of arbitrary branches and I guess that can be extended to include previous named releases.
I will post an update once we have more details. FYI @xitij2000.
Is there any news from anyone on this? We have a few open pull requests from people trying to patch Ficus, Ginkgo, … But edX doesn’t have the means to verify them.
@nedbat We’ve been working on the system @guruprasad mentioned above. This system will periodically deploy a server with a configured branch and email us if it fails for any reason.
While it’s up and running now there are a few bugs we are ironing out, and we’re hoping it should be ready quite soon!
As for the particular case of testing PRs for older branches, we currently cannot do that in a straightforward manner, but we can test it once it’s merged with the system we have.
Sorry for the delayed response. We actually have an existing system that automatically spins up a sandbox instance for every PR opened by an OpenCraft member. This system also supports reading some of the settings and configuration supplied in the PR description.
In this PR for example, the YAML content in the settings section will be passed along when configuring the instance. This alone should allow using our existing infrastructure to test PRs against older releases, the PR can simply be opened with appropriate versions settings. This is how we internally test PRs against older releases as well.
The main issue here is that currently the system only auto-provisions PRs opened by OpenCraft members. We could look into building a system that would allow edX team members to tag specific PRs and they could also be auto-provisioned with sandboxes. The PR creator would need to specify the correct settings as well.
Would that approach work for this use-case (in addition to regularly testing the release branches)?
Another potentially minor caveat: we use OpenStack, not AWS, so any PRs that are fixing AWS-specific issues, or similar can’t be tested in this manner.
I think for the most part it shouldn’t make a difference, especially for edx-platform itself. The configuration bits differ more, and those PRs would be harder to test.
For edx-platform PRs targetting old releases, we can set up a sandbox manually for now. Are there any current PRs against old releases where a sandbox would be useful?
@nedbat I’ve been looking into this issue a bit and figuring out how we could help support this.
Currently, any PR opened by an OpenCraft team member against edx-platform automatically gets a sandbox created for it no matter what branch it targets. For other PRs this needs to be a manual process. We’re looking into automating this a bit to make it easier to manage.
Even once it’s automated though, here is the process it would need:
Any PR against an older release branch that needs a sandbox can get a specific label for this purpose (mainly to help us track them)
Once it’s ready for review and has a reviewer assigned, someone from OpenCraft can be pinged, and they will go through the process of setting up a sandbox
The sandbox will be linked to the ticket, and the reviewer’s GitHuh ssh key will be added to the sandbox.
@xitij2000 thanks, sorry for the delay. This would be a good scheme for ensuring that changes don’t break the installation, and we should talk more about how to get it in place.
But I guess I’m more concerned about the installation starting to fail due to external changes. That is, nothing changed in our repos at all, but now a package isn’t available, or a key has expired, or something like that. We don’t have a way to know that the world changed around us, and we need to adapt. Do you have any thoughts about how we might approach that?
I think there are two somewhat related issues here:
The first is to ensure that old releases continue to be installable. This would cover issues like expired keys, missing dependencies and other similar external issues. We’ve set up periodic builds of the master branch and will similarly set up periodic builds of other branches as well. Each build installs the platform in a fresh VM from start, and notifies us in case of any error. This will ensure that we know as early as possible when any monitored branches break. @swalladge can share more about this.
The other concern is setting up sandboxes for testing PRs against old branches. For this, I’ve proposed the above process. This would be useful since otherwise there is no way to test if a PR against an old branch fixes the issue discovered by the above process.
Currently, we are not running E2E tests on each install.
