Interim guidance on the `openedx` GitHub organization

Announcements Architecture
1

The questions

Since last November, these questions have come up a lot:

  • What repositories can go in the openedx GitHub organization?
  • What repositories must go in the openedx GitHub organization?

We’ve been using the OEP process to try to answer these questions collaboratively:

  • OEP-57: Product Offering is in review, check it out!
  • OEP-61 will come next. It’ll build on OEP-57 to propose a criteria for what code should be in named releases, and what code should be in the openedx org.

However, it’s become clear that we need an answer to these questions now in order to do our work, particularly around release time. So, here’s how we (tCRIL) recommend operating from now until when we have a real OEP to guide us. Let us know if you have any questions or concerns.

Our guidance

  1. Repositories can be added to the openedx organization if they are of interest to the Open edX community and are willing to accept community contributions. Anyone can submit a tCRIL request if they want to add a repository to the organization. In that request, we’ll discuss whether the repositories meets those criteria, as well as whether there are any legal constraints around us accepting the code.

  2. Repos must be added to the openedx org before they become critical dependencies of any other repository in the openedx organization.

    • Example: A frontend component must be added to the openedx organization before it can be installed into frontend-app-gradebook.
    • Exception: Third-party libraries upon which openedx code depends do not need to be added to the openedx organization. For example: django, requests, celery, react, etc.
    • Note: Tutor code is hosted in the overhangio organization. Although Tutor is the supported distribution/installation method for Open edX, code in the openedx organization does not (and should not) have critical dependencies on Tutor or any other overhangio code. It should remain possible to run the platform without using Tutor.

  3. Repos must be added to the openedx org if they are to be included in future named releases.

Implications

  • Starting with Palm, we will only consider tagging repositories that are in the openedx GitHub organization. There are certain repositories in the edx organization that we will either need to move in to the openedx organization, or remove from the next release.
  • A number of repositories currently break rule #2. We may move some repositories around to improve things; mostly, though, we just want to avoid having new exceptions. After the OEPs are complete, we’re more likely to proactively make changes.

This all seems a little pedantic. Why does it matter?

  • tCRIL cannot see security alerts for code outside the openedx org.
  • tCRIL cannot manage access to code that is not in the openedx org. Access management is an important part of the Core Contributor and Maintainer programs.
  • tCRIL and the Core Contributors cannot approve & merge code outside the openedx org. Ideally, repository maintainers will be available to review and merge important pull requests, such as security patches, critical bugfixes, and resolutions to release-blocking issues. However, when they are not, tCRIL and CCs need to be able to act as a backstop so we can keep the platform secure and keep releases rolling out on time.
4 Likes
2

How are we ensuring this work gets prioritized? (particularly, do we have any tickets for it, and is this work other teams/Core Contributors can assist with?)

This is another thing that would be good to ticket - at least, producing the list of current exceptions and monitoring so there aren’t more. As well this could be a good CC task.

3

Yup, we have this issue on the BTR WG board Remove the `edx` GitHub organization from the release process · Issue #203 · openedx/wg-build-test-release · GitHub . If anyone wants to get involved, they are welcome to comment on that ticket.

1 Like
4

Wouldn’t “critical dependencies” also include all upstream projects used by Open edX? (eg. django, markdown, etc.) Imho unless we take a “zero dependency” approach like Django, we will always depend critically on third-party dependencies? Maybe we need categories & different rules/process for internal & external dependencies?

That’s an interesting case actually, because even if no other repository on the openedx org depends on Tutor, it remains the only official and supported installation method, so we depend critically on Tutor in any case - that also breaks the spirit of the rule, no?

Not that we necessarily need to move Tutor to the Open edX project (though it would be a good thing imho!), but that shows that we currently have third party dependencies, even for software within the Open edX community, and that we need to better understand and define the difference between a third party dependency, and one internal to the project?

1 Like
5

I don’t think you are suggesting to somehow have Django in the openedx GitHub org, so I’m not sure what you are proposing? There are hundreds of third-party dependencies across the Open edX project, all of which are “critical” in the sense that a bad problem in one of them could be bad for an Open edX installation.

To me, this seems like a special case. Django (as an example third-party library) is widely used beyond the Open edX world, and so is unlikely to pose a particular danger to an Open edX installation. But Tutor is only used by Open edX sites, and is the only supported installation mechanism. If we think “critical components” need to be in the openedx GitHub org for safety, I don’t understand why we’re OK with Tutor being outside the org. Or, we should accept that the openedx org isn’t special, and adopt mechanisms that secure our software regardless of the GitHub org that hosts it.

1 Like
6

@nedbat @antoviaque My original post calls out third-party libraries as an exception. They are managed and maintained separately. Sure, we should be judicious when we add new dependencies to the project, but that’s not what I’m trying to solve for here. With the exception of Tutor and Tutor plugins, I think it’s clear-cut when something is a first-party vs. third-party dependency. If you can think of any other cases where it’s ambiguous, I’m interested to hear them.

Now, with regards to Tutor, you both bring up good points. Maybe it does belong in the openedx organization; I’m not sure. Still, this post is purposefully not trying to cause sweeping changes, nor is it trying to settle philosophical debates on what is Open edX vs extensions to Open edX vs separate from Open edX. Those conversations would be better had on OEP-57, and, when it’s ready, OEP-61; they should also involve @regis and should take into account the roadmap he’s already outlined for bringing the Open edX and Tutor communities closer together.

4 Likes
7

Ahah, that would be fun, but no I’m not proposing that ;p My goal is to acknowledge that as an open source project we routinely use third-party dependencies, and that there is nothing wrong with that – including third-party dependencies that are more specific to the project.

The distinction between being used only within Open edX or beyond is going to be fuzzy. It’s clear enough for something like django, but there is everything in between – i.e. if someone makes a generic python module that is only used by Open edX, they would need to have it on the Open edX repo, but if one other software includes it in its dependencies, then it would be ok to keep it separate? It wouldn’t have magically become more secure just because of that.

If we are going to define the rules for dependencies, it would be more consistent to decide on rules for third-party dependencies as a whole. For example, require that third party dependencies are actively supported, including paying attention to security issues. And if we want to be sure that the work is done, we could setup automation to start monitoring our dependencies – that would be beneficial for all third-party extensions, not just the Open edX specific ones.

On the other side, we can (and should!) still provide incentives for moving third-party dependencies to the project - like benefiting from the project infrastructure (CI, security alerts), volunteer time (core/maintainer programs, prioritization by working groups, etc.), in exchange for moving the code to the openedx org and agreeing to be subject to the project rules & governance (ie OEPs apply, etc.).

@kmccormick I’m definitely up for having those discussions on these OEPs – though I’m not sure where/if it would fit on OEP-57 in its current form? Maybe more for OEP-61? CC @ali_hugo @sarina

Imho it’s not a coincidence that these various discussions happen simultaneously – with the change in the project direction and governance, it forces us to revisit the status quo to cleanup the organizational debt, and we want to get off the right foot. Depending on the path we decide to take in those higher level discussions in the OEPs, they would impact more targeted changes like the one you are bringing up here. It makes sense to still be able to take decisions in the meantime – but it can also make sense to use the opportunity to take note of larger points that need to be addressed, and could end up impacting the change you’re suggesting?

3 Likes
8

Yep, I think these are good discussions to have both before and during OEP-61’s review.

Agreed. I’m glad you and @nedbat brought up your concerns. In the interim, I don’t see us changing the rules; by all means, though, feel free to kick off these conversations in new threads.