The self-hosted Github workflow runner doesn’t have to be on AWS. We’re just looking for a way to run these unit-tests workflows that the edx-platform runs on a PR. Looking for hosting configuration examples if possible.
Thanks @sarina for including others here. I guess the questions is what additional configuration outside of install requirements for edx-platform needs to occur. What’s a typical system requirements for this CI server? Does edX use Docker containers to run these unit-tests workflow actions in to avoid security issues and conflicts with other developers PR actions running simultaneously?
I noticed that this unit-tests workflow requires the following
The main edx-platform repo uses self-hosted runners because the volume of commits and test shards is large enough to overwhelm the GitHub plan’s allotment of concurrent workers. We’re also using faster machines than the default hosted runners, giving us shorter test run durations.
The original thinking was that we should be able to configure this for forks to just fall back to using the GitHub hosted runners at no cost, just needing to wait longer for tests to complete. What we haven’t figured out yet is how to allow this without needing to customize the unit-tests.yml workflow definition in the fork; if we add hosted runners as a fallback, the main repo will sometimes start using slower hosted workers before new self-hosted workers have time to finish spinning up.
For now, I’d recommend customizing your fork’s worfklow to run in “ubuntu-20.04” instead of “edx-platform-runner”. You could set up a Kubernetes cluster for self-hosted runners like we have, but it’s complex enough that I don’t think it’s worth the effort for most forks. And I welcome suggestions on how to make the default configuration work better for forks; the best idea I have so far is two copies of the workflow with different if conditions, one to run only on the main repo and the other to run only in forks (but I suspect we’d often forget to keep them synchronized).
@jmbowman I see your point in using Github hosted workers to avoid having to setup Kubernetes cluster to run self-hosted runners. Do you have recommendations on that Kubernetes configuration in case we’d like to look into that further?
Jeremy: “… test shards is large enough to overwhelm the GitHub plan’s allotment of concurrent workers. We’re also using faster machines than the default hosted runners, giving us shorter test run durations.”
Will this work for Github hosted runs-on: ubuntu-20.04? It seems like it could take longer but will all the shards run?
Also when @feanil indicated that there was this Docker image Docker Hub that use use for this unit-tests.yml workflow it made me think what additional software do we need to run these tests prior to the edx-platform installing requirements.
Looking at the workflow steps it appears MongoDB would need to start.
It looks like MongoDB is setup in the Docker image here.
I like your idea of using if conditions to factor in using self-hosted runners versus Github hosted runners may work but there doesn’t appear to be an else statement. Wonder the if expression should be a branch name instead of organization/repo.
@jmbowman Whenever you get a second please just look at this latest PR. I setup two separate jobs in the workflow based on the branch reference name. This should allow the upstream Open edX job to use runs-on: [ self-hosted ] and the fork repo to use runs-on: [ubuntu-20.04]. Let me know what you think.
I’m getting errors on install requirements step within the run-tests-fork job, however, it’s probably how the pip cache directory works. If you have any suggestions on that please let me know.
Any way that you can look over my workflow update and give recommendations on how to run this successfully on Github platform runner for the unit-tests.yml workflow?
How can I get past this install requirements error?
ERROR: Cannot uninstall ‘PyYAML’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
WARNING: You are using pip version 20.3.4; however, version 22.0.3 is available.
You should consider upgrading via the ‘/usr/bin/python3 -m pip install --upgrade pip’ command.
Error: Process completed with exit code 1.
As noted in another article it’s related to MongoDB.