Dear Community,
As you know, the next version of Open edX® is named Olive. With the support of the Build-Test-Release Working Group, I am happy to say the Olive Test Sheet is now ready to be used for testing!
The following Test Team is a volunteer group of 17 people from the community. We give a warm welcome to @Fouzia Basharat (Arbisoft) and @Maksim_Sokolskiy (RaccoonGang) who are joining the test team for Olive! We also sadly say goodbye to a test champion @Neo Gonzalo Romero (eduNEXT). The team includes:
Ned Bachelder (2U),
Kaleb Abebe (MIT Open Learning),
Régis Behmo (Tutor),
Pierre Mailhot (EDUlib),
Peter Pinch (MIT Open Learning),
Brian Grossman (MITx xPRO),
Ghassan Maslamani (Zaat.dev),
Maria Grimaldi (eduNEXT),
Kyle McCormick (tCRIL),
Zia Fazal (Arbisoft),
Fayyaz Ahmed (Edly),
Gabriel D’Amours (OpenCraft),
JayRam Nai (DRC Systems),
Dean Jay Mathew (ABC Online Courses),
Carlos Muniz (tCRIL),
Fouzia Basharat (Arbisoft),
Max Sokolski (RaccoonGang).
- TEST SPRINT 1 of 3 will run for 4 weeks from today Friday 21 October 2022 to Friday 18 November.
- TEST SPRINT 2 of 3 will run for 2 weeks from Friday 18 November to Friday 2 December.
- TEST SPRINT 3 of 3 will run for 1 week from Friday 2 December to Friday 9 December.
- Resultantly: The first release of Olive will be made on 9 December 2022.
The Test Plan for Olive includes 229 individual tests and takes the form of a Google Sheet with a number of tabs such as README, SUGGESTIONS, LEADERBOARD, OLIVE TESTS, and LIST OF FAILED TESTS FROM NUTMEG!
ACCESSIBILITY: A COMMUNITY FOR EVERYONE
Screen reader support is turned on.
Braille support is turned on.
Screen magnifier support is turned on.
TEST INSTANCE ACCESS: THANKS TO OUR SPONSOR RÉGIS BEHMO, FOUNDER OF TUTOR
- The demo platform will be deployed WEEKLY on Mondays at 7am UTC and will be available at: https://olive.demo.overhang.io/ and https://studio.olive.demo.overhang.io/. This means the user database is wiped weekly, so you need to re-register your account when the platform redeploys each Monday. The superuser login details won’t change, so just use those.
- The credentials for the superuser/staff account are listed in the project readme (superuser details are, email: admin@overhang.io, password: admin).
- There is a repository to host the deployment scripts here: GitHub - overhangio/openedx-release-demo: Open edX Olive demo platform CD
- Changes to the instance configuration can be made as PRs here
- The SMTP service is enabled, but the server is not configured for email authentication (DKIM, SPF, etc.). This means that emails do get sent, but it’s very likely that they will end up in spam, depending on your email client configuration.
WHAT’S NEW IN THE TEST SHEET? 
The most important addition to the test plan are two new columns:
(1) Previously Failed in Nutmeg:
This column indicates a YES or a NO to let testers know if this test has ever failed before (i.e. it failed in Nutmeg and was then highly likely fixed)
(2) Previously Assigned in Nutmeg:
This column lets testers know who did that test case previously (i.e. in Nutmeg).
TESTING GUIDELINES: HOW TO USE THE SHEET
Step 1: Browse unassigned Test Cases and self-assign from the dropdown list (contact us if your name is not there).
Step 2: When you are ready to perform the test, update the Status column to ‘In Progress’.
Step 3: Perform the test.
Step 4: Update the Status column to ‘Complete’.
Step 5: Update the Result column to ‘Passed’ or ‘Failed’.
Step 6: IF test ‘Failed’, create an issue in the Build-Test-Release GitHub repo and link it in the column. Make sure to add the Test Case ID into GitHub. The correct repo for adding your issues is: Issues · openedx/wg-build-test-release · GitHub. Once created, add a labels by making a comment, by commenting the following string in bold then waiting 10 seconds for the bot to add the label: label: nutmeg testing.
Step 7: IF GitHub issue is resolved, go back to Step 2 and start the testing process again.
DOS AND DONTS: LET’S KEEP IT NEAT AND ERROR FREE
- Testing should happen on the official instance: https://olive.demo.overhang.io/
- Only assign yourself tasks that you will complete within 24 hours. This is to avoid blocking the pipeline for others.
- If you have suggestions, add your suggestions to the sheet named SUGGESTIONS next to this sheet.
- If you test a component in Studio, please check it’s working in the LMS view too.
- Most of the sheets are PROTECTED meaning you cannot alter them, apart from a specially selected range that you will be using as tester.
This video was created earlier in the year to teach testers how to use the test sheet. While it was made for Nutmeg it is still relevant for Olive as the sheet looks almost identical, so please watch this video to learn how to use the sheet if you are new:
If you have any questions or need help to get started testing Olive, reach out to the team. This is also the case if you find any error or issues with our test sheet, please reach out immediately so we can fix it.
| CONTACTS & COMMUNICATION CHANNELS | |||
|---|---|---|---|
| Test Coordinator: | Dean Jay Mathew (dean@abconlinecourses.com) | Release Manager: | Matjaz Gregoric (matjaz@opencraft.com) |
| Test Instance Owner: | Régis Behmo (regis@behmo.com) | Release Notes Manager: | Peter Pinch (pdpinch@mit.edu) |
| Test QA Manager: | Pierre Mailhot (pierre.mailhot@edulib.org) | ||
| Slack: | Slack | ||
| Confluence: | https://openedx.atlassian.net/wiki/spaces/COMM/pages/1022099494/Build+-+Test+-+Release+Working+Group | ||
| Discuss: | Build-Test-Release - Open edX discussions |
I would like to end this post by giving an important shout out to the Test Champions from Nutmeg, who without their support, we could not have launched Nutmeg with confidence:
1st Place: @Fayyaz Ahmed from Edly (Pakistan) with 86 test cases!
Fayyaz really helped kickstart testing by performing test cases as early as day one of the project. But the end of testing, he had completed 86 test cases, which is more than double compared to any other member of the test team. Thank you Fayyaz and I hope you enjoyed the experience.
2nd Place: @mgmdi Maria Grimaldi, eduNEXT (Columbia) with 21 test cases
Maria took the lead in testing SSO test cases and more, and was one of the strongest contributors towards solving the Github Issues created.
3rd Place: @ghassan Maslamani, Independent Zaat.dev (Palestine) with 20 test cases
What can we say but THANK YOU to Ghassan . From manual test cases to coding, and pull requests and basically everything under the sun, Ghassan was EVERYWHERE! Testing would not have been successful without him.
4th Place: @Kaleb_Abebe, MITx (USA) with 11 test cases
Kaleb helped get testing off to a fast start which was super helpful.
5th Place: @Gonzalo Romero, eduNEXT (Columbia) with 10 test cases
Gonzalo was diversified in his test contributions, and is even helping to upgrade a Tutor plugin together with Felipe Montoya.
