We (edX.org) is looking to eventually move our Open edX platform authentication to a vendor/third-party solution. I want to share some notes on that plan, and potential consequences, so we can discuss as a community.
We will be looking into a variety of migration plans, but the third-party solution is where we’d like to end up.
If and when we finally move to a third-party solution, it would no longer make sense for our team to maintain the LMS-hosted authentication system. I can imagine at least two paths here (that aren’t mutually exclusive):
We deprecate/remove the custom authentication system and work together to ensure that there is an open source solution (e.g. Keycloak), and/or
We find a new maintainer of the LMS-hosted authentication system.
Other possible migration steps we’re discussing include:
Moving the LMS-hosted authentication system to a new Identity service inside the Open edX platform, and
Adding features to the Open edX authentication system, like MFA, that we’d ultimately get by moving to a third-party solution, but that we may want sooner than we can get there.
There are of course many details to all of this, but I simply wanted to share these high-level thoughts to get the conversation started with interested parties.
I think that Ory Kratos looks like a very nice open source solution, and they have a cloud-hosted version. If edX.org moves to something like that (a third party hosted service but with a fully open source stack), it would keep the delta between what the community is using and what edX.org is doing very small. I think the main drawback with Kratos is that SAML SP isn’t yet finished, but otherwise it’s one of the most robust and feature-rich that I’ve seen (Passkeys, MFA) and is very easy to integrate as much or as little as you want - HN discussion.
I definitely think it makes more sense to contribute to an open source identity service like Keycloak or Ory Kratos than to maintain a service just for Open edX.
@sambapete: I’m not sure I understand what you mean by this:
Make sure the cloud service has a Canadian service or that the data can stay in Canada.
Are you referring to the cloud service that edx.org chooses to manage its authentication?
Or, are you referring to a cloud service that the Open edX platform chooses? For the Open edX platform, I imagine that an open source solution would be selected, which may or may have a vendor provider, but could be self-hosted.
Maybe you could clarify your request based on my questions?
It’s tough to get into details, because the real answer is that I don’t know. That said, I imagine it would be much of logistration and its associated backend, and all of the OAuth2 code, and creation of access tokens, etc. That said, there’s going to be a lot of tricky details here, and so I really do not know.
Is there a particular area of concern you have, or are you just generally curious?
You perfectly understood. If the authentication code is removed from the Open edX code, all Open edX provider will need to provide a local solution or a cloud solution that may need to be country based for PII information of their learners or learners from the UE because of GDPR for example.
That’s why knowing what information will be shared or stored by a third party may be useful in order to define the solution.
Interesting. Thanks for getting the conversation started early.
It opens so many roads and so many questions.
Does this mean that much like ecommerce had, the lms would then in the future keep copies of the user object that have local information, but in order to edit any of that data one would require to change it in the upstream auth service?
How do the current TPAs fit into this? they would authenticate to the new auth service and the LMS only connects to that auth service?
I would imagine that being the new source of Oauth2, all other apps like Studio, ecom, discovery and so would connect to the hosted auth service. Is that right?
What kind of timeline do you think would be required for extracting this?
I think so. I imagine the Account MFE would work with this new service. I imagine we might need events from this new service, but I’m not certain of that, nor how it would be implemented.
I imagine so.
That would be the goal.
I do not know. We may end up migrating on to the LMS, before extracting out, which would delay this type of effort.
That said, any migration would require a slow rollout on our part, so it is likely that it will be possible to use two solutions (new and old auth) at the same time for different users. It is not clear whether the community would use this, or do big-bang transitions, but it means we could certainly deprecate the old authentication system and delay removal until the appropriate named release, which I imagine wouldn’t be for a couple of years from now. This is just a guess though.
Ultimately, the DEPR process would be more appropriate, but I’m not even ready to commit to deprecating. So, this is just a conversation starter as we consider that possibility in the future.
Thanks for starting this discussion @robrap . Some questions, with the caveat that I am not experienced with auth As I understand it, the current state looks something like this:
[external_idp -> ] lms -> open_edx_microservices
where X -> Y means that X is providing an identity to Y. external_idp could be Google or OneLogin; open_edx_microservices could be Credentials or Discovery.
where authn_service is a 3rd party vendor. Notably, LMS is no longer acting as an identity provider; it’s consuming authn like the other Open edX services. Thus, you are surfacing that, since 2U will no longer use the authn-provider capabilities of LMS, that the community will likely either need to maintain those capabilities, or find/build an authn_service of its own.
Do I have all that right? (I’m assuming I can safely ignore nuances like SAML vs OAuth here, and just talk in the abstract.)
What is the value of eschewing the auth-providing services of LMS rather than just putting the 3rd-party authn service behind LMS? ie:
Thanks @kmccormick. I think what you said is in line with what I am saying.
That said, I’m not sure if I’m following this last point yet. But note that there is a difference with lms_and_other_open_edx_services and what that means for auth in the external_idp and somehow initiating an LMS session in parallel, so it could drive auth in downstream services. This doesn’t seem to be what we are thinking at this time, but maybe you can make a case for it and I’ll see the light?
So currently the LMS has the ability to defer auth to a 3rd party already right(“Login with …”). How would the thing you’re talking about here be different? In the above case, we take the 3rd party verification and then provide the user with a session token and JWT. Is the goal for you to have the 3rd party generate the JWT and have the Open edX ecosystem respect it? What’s the value of that over just doing an OAuth exchange into the Open edX Auth system?
Note: I know that the OAuth workflow right now is not very nice especially for users that don’t already have an account (forcing them to fill out a form to register instead of auto-registering them.) But this seems like it could be solved an the experience could be streamlined.
The main idea is that Authentication is a generic subdomain that we don’t want to maintain it (e.g. deployments, security patches, upgrades, adding new generic authentication related features, etc.). Yes, the result of this would be that the 3rd party system would be the auth provider, creating JWTs, and that the LMS would no longer create JWTs.
The solution you are proposing may allow for SSO with other systems, but it doesn’t solve the larger issue. It adds, rather than replaces, if I am following.
Does this clarify the intent?
Note that one possible migration step or final solution would be moving the LMS authentication system as it exists today out to its own separate service for those who wish to use it, but someone would need to maintain it.
Makes sense, I think my gut response to extracting vs keeping a reference auth system in the platform is that we probably want to keep it there for now to reduce the complexity for small/medium operators with documentation to help larger operators hook in something else instead. Given that we’ll still need user models in our backend django services, I’m skeptical about how much auth logic we’ll be able to ultimately remove.
However, given how complex it is right now, I think any work in this space will help us better understand this domain and hopefully lead us to reducing some of the complexity.
Read you loud and clear that 2U is looking to not remain the maintainer of the auth subsystem in the long term. If we do the work right, I’m hoping the auth bits that remain in the platform are mostly off-the-shelf and we can generally reduce the amount of maintenance needed. We should behave like common Oauth2 services as much as possible.
Tangential to this discussion, but there has long been interest for the platform to support SCIM for user provisioning. Would be great if in the process of 2U moving to an external auth system we could broaden the platform’s support of standards in this space.
For now, I’ll keep open source options listed on this post. As Jeremy noted, these options may just inform finding other options, but all good to keep in mind at this point.
As I understand it, the current state looks something like this: