[šŸMaple] Retrospective

Working Groups Build-Test-Release
1

Now that Maple is out of the door itā€™s time for us to look back and reflect. As weā€™ve done with the last 2 releases (Koa and Lilac), weā€™ll kickstart the retro asynchronously!

Please share your thoughts about the Maple release process in this thread:

  • What do you think went well?
  • What do you think could be improved?
  • Do you have any ideas about things we should consider?

Letā€™s aim to have everyoneā€™s thoughts by next Monday so we have a week to discuss them here before the next meetup!

CC @regis @nedbat @pdpinch @blarghmatey @jhony_avella @Neo @Maksim_Sokolskiy @dbates @djoy

2 Likes
2

For those of you who are interested here is a link to a Retro board for this release

1 Like
3

Throughout the Maple release process I was alternately 1) not paying too much attention, or 2) deep in PR review or bug fixing mode. So I may have a skewed sense of how things went. With that as a caveat:

What do you think went well?

As always, I was struck with the professionalism and dedication of the team toward what I think of as a daunting task. I really appreciate the BTR GitHub project board and find it a really great resource, and am pleased that it tends to get well-used for keeping track of the release status.

What do you think could be improved?

I think we could use some general education and a bit more support from teams outside the group at times; on some occasions, it feels a bit like screaming into the void (i.e., no response). Iā€™ve occasionally been the one going silent for lack of hours in the day. Iā€™m not 100% sure how to deal with this, but as per my answer above, I think the BTR group handles it with grace.

Our process and work with MFEs continues to evolve, and has definitely been a bit bumpy at times. We had a rush right there at the end to try to close out a bunch of issues all at once, and it wasnā€™t the safest way to work, and I wouldnā€™t be super surprised if we introduced some regressions here and there. I think during this release we got more familiar with what it takes to make an MFE ready for an Open edX release, and Iā€™m hopefully that this will start being a bit more routine.

Do you have any ideas about things we should consider?

Finding ways to enable asynchronous work earlier in the process. Modest ideas could include some more synchronous small group working sessions (easier to schedule), or helping folks schedule dedicated time on their calendars for solo work (perhaps with a check-in to keep honest people honest!)

-David

4

What went well?

Iā€™ve said it before, and I think that we put out the best Open edX release in years. We fixed major Ecommerce issues that were affecting Lilac and we managed to close all important issues that were detected before the cut.

What could be improved?

We resorted to postponing the release from December 9th to the 20th because we did not manage to fix the Ecommerce issues in time. We had to take this extraordinary measure and make a global call to the community to make people interested in fixing the issue. The fact that people responded to our call was a success, but on the other hand I wish we did not have to proceed with such urgency. The Ecommerce issue had been known for a long time and should have been addressed earlier.

The last-minute rush and live coding session on Zoom to fix MFE issues was a first for me. While I did enjoy the group effort and the fact that we manage to close many issues in just a couple hours, it wasnā€™t something that I would like to do again.

Other ideas

Does it make sense to have Zoom meetings with 10+ people? This is not a rhetorical question, Iā€™m not sure whatā€™s the right answer. If we were to meet IRL, we would never schedule meetings that would include more than ~4 people, so why do we do it in the BTR? (or in the core committers group, for that matter)

Do we invite so many people to every meeting becauseā€¦

  1. we are an open source community?
  2. thatā€™s the new normal in a pandemic world?
  3. we (including myself) donā€™t do a great job at organizing meetings?

I suspect itā€™s 3. If we want the BTR to become more inclusive and attract more contributors, then we should not expect them to join a video meeting to make their first contribution.

By default, we make decisions and keep track of ongoing projects during meetings. What changes do we have to implement to switch to asynchronous conversations by default? Iā€™m ruling out the addition of new tools, because I think that we already have all the tools we need (Slack, Discourse, GitHub). I think that itā€™s mostly a matter of better usage of the existing tools ā€“ though Iā€™m not sure how exactly.

5

We cut the maple release on schedule. We tested and fixed things. And we made a final maple release (almost) on schedule. It was a bit stressful at times, but not as much as deploying our system upgrades ā€“ which were able to do on schedule because maple was on schedule. Most importantly, we found ways to work together to get through the challenging bits. Each time we do this, we get help from more people.

  • What do you think could be improved?

I am too deadline-motivated, which meant that although I could have started the release notes when the first candidate was cut, I didnā€™t start them until a week before the final release date.

Collectively, we have some of the same behaviors. We knew that ecommerce was an issue, but we didnā€™t start working on it (collectively) until we had to consider cutting it.

  • Do you have any ideas about things we should consider?

Whenā€™s the next release? Letā€™s get started on it now. I know thatā€™s glib, but I think having some vision into features that are coming, or than the community wants, and assigning responsibilities will make it possible for us to be more timely next time, and also allow for the asynchronous types of coordination others are talking about.

1 Like
6

Iā€™ll echo some things that have already been said:

  • Issue tracking helped us stay organized and focused on what mattered
  • We started the release on the chosen date
  • We finished the released near the chosen date
  • Making progress asynchronously: too often work waited until the next face-to-face meeting.
  • We made master branches two months ahead of the release so that we had time to test, but I donā€™t know if we used those two months fully.
  • Perhaps it was only because i wasnā€™t directly involved, but I didnā€™t get the sense that the testing was coordinated among testers.
  • The fix-fest on release day was an impressive last push, but was the wrong way to get those issues fixed. Why did we wait until release day?
7

Thanks to all those who have shared their thoughts!

Iā€™ve grouped the different points raised and put them in the retrospective board.

Please vote on the cards that you would like us to discuss further (3 votes per person).

@dbates Can we limit the number of votes to 3? At the moment I see that the limit is 10 (not sure if itā€™s per user).

8
  • We managed to release Maple on the promised date
  • Did a good issue tracking job
  • Improve collaboration with projects teams (Platform, MFEs teams)
  • Improve testing efficiency by using automated frameworks
  • More attention to the DevStack repo
  • We can start to use e2e tests to be able to handle regression much early
  • How we can improve our collaboration with the projects teams to be better aware of a Features development plan and to be able to see fixes in master branches early

@BbrSofiane Sorry for the late input.

9

Summary of the Retrospective

Issues identified:

  • Work tends to be done at the last minute close to meetup dates and release dates.
  • As a result, there is not much happening asynchronously and progress is very sporadic which leads to intense crunch time.
  • There is not much coordination in the testing process.

Ideas to improve:

  • Start working on the next release now.
  • Introduce an issue reviewer role. Any BTR member can be an issue reviewer, the goal is to have someone that keeps track of a certain number of issues and make sure people provide updates outside of the meetup.
  • Deploy a testing instance and engage with the community to help with testing.
2 Likes