The Cosmonauts team at edX is embarking on a redesign of our proctoring library edx-proctoring. Our plan is to develop a new special-exams IDA that will eventually replace edx-proctoring and will serve as the backend for the frontend-app-learning MFE, which uses the frontend-lib-special-exams library for its proctoring specific UI.
A key part of the special exams experience is ensuring that learners are authorized to view exam content and that they see content appropriate to their status in said exam. Given this opportunity, we are investigating options for redesigning this mechanism.
Context:
Currently, the learning MFE only renders the unit if a learner is able to actually view the exam content. There are forms of access control rules.
The MFE only calls the LMS render_xblock view if the learner’s exam attempt is in the “started” state or if the exam is a timed exam, the exam due date has passed, and the subsection is not set to “hide after due”.
When the MFE calls to the LMS’s render_xblock view, the LMS double-checks the learner’s access. The LMS imports and calls to the edx-proctoring API directly from _time_limited_student_view; edx_proctoring.get_student_view returns interstitial HTML when the library wants to show a learner a proctoring-related interstitial. It returns None when the learner should have access to content. Likewise, edx-proctoring checks the edx_proctoring.can_take_proctored_exam permission, which is a bridgekeeper permission defined in the LMS, when determining whether to return an interstitial or None in various methods.
In the learning MFE, this logic is used to determine to render the unit or redirect to the sequence.
In the legacy view, this logic is used to return directly to the learner the proctoring specific HTMl to display, if eligible, or None to fallback to the platform view.
Desires and Aspirations:
We need a mechanism to gate access to xBlocks based on a set of rules and for a certain amount of time. The reason we need this is that a learner could grab the iFrame URL and view the xBlock directly if we do not have access control on the backend. Our rules are proctoring-specific, but we want to see if there is a way to genericize, unify, and centralize access rules within the platform in some form of a rules registry. The more we can decouple the platform from the proctoring service, the better.
For example, we imagine that making a blocking call to the special exams IDA before rendering every xBlock will result unacceptable performance implications.
I’m posting this to see whether this sort of proposal has been considered before. Is there any prior art? Are there any upcoming plans in this area? Do you have a recommendation for where to start? Does this idea strike fear into you? All feedback is welcome.
Disclaimer: XBlock permissions checks are painful, and it’s been a long while since I’ve had to really beat my head against it.
100% agree that such a mechanism would be too slow and cause operational issues. We’ve hit other issues like this, where we’ve had to shift over to replicating data across systems so that the local system has the information it needs to calculate things like this quickly without the runtime dependency on a separate service.
I’m going to throw out some random thoughts and ideas, in the hopes that something might be useful to you. I realize you probably already know a lot of this, but just in case there’s something new…
The overloaded 'load' permission
Much of the courseware makes no distinction between “the student is allowed to see this content rendered” vs. “the student is allowed to know that this piece of content exists”, and this causes some painful contortions in the code. You may find that getting your desired proctoring behavior will cause some other system (e.g. grading) to break in unexpected ways.
Useful bugs become undocumented “features”
I seem to recall that there was at least one course (Super-Earths and Life?) that ran using an external adaptive engine that would take advantage of the fact that a student can access content in A/B split_test groups that they are not a part of if they have a direct link to it. If I remember correctly, they’d make assessments in groups that nobody could access (i.e. set it to show to 0% of people), and then launch those pieces of content directly via LTI.
I may be messing up some of the details, and it was a very long time ago, so I don’t know if this is still a use case–it sticks out in my mind mostly because my jaw basically dropped open during this part of that team’s presentation, and probably stayed that way for a few minutes afterward. It was an incredibly clever way to get the functionality that they needed out of a system that didn’t support it, but it’s also really hacky.
I don’t mean to say we have to preserve this particular behavior (though we should message clearly if we’re going to break it). But I highlight it just as a cautionary reminder that there may be folks who rely on the current behavior, however bad it might be.
Can we re-use existing mechanisms?
Would a new user partitioning type help here, where people are swapped into and out of the group that can see the exam? (I’m honestly kinda skeptical of this one, honestly.)
Does the gating logic need to live in courseware rules at all?
I think that decoupling platform from proctoring rules is a commendable goal, and building an access rules system can accomplish that. But there might be other paths that achieve that goal in a way that might even be more generic–especially if the goal is to only prevent unauthorized use of a narrow subset of functionality (like people manually putting in the /xblock/{usage_key} endpoint URL).
For instance, say we had a new inheritable XBlock field that was something like render_token_required (that’s a terrible name, but please bear with me). It’s false by default, but you can set it to true and block off a chunk of content in that way. The render_xblock view checks for this field on whatever top level XBlock it’s rendering. If the value is true, it looks to see if it’s been called with a JWT token passed in as a query parameter to the endpoint. That signed JWT token would be small, and basically encode (user_id, usage_key, until_time).
So then the view layer could use this as a gatekeeping mechanism. If the JWT doesn’t match expectations (e.g. it’s the wrong user), it gives a 403–one which users should never see, since it’s only catching people who try to skirt the rules. It carves out a pretty narrow use-case, but it can also be really decoupled. As long as the entity issuing the JWT knows the right signing key, the restrictions could be set by an external proctoring service, or really almost anything else, using whatever logic it wants. It’s also performant, since there’s just one top level check happening.
Caveats:
This probably has mobile app implications that I haven’t thought through.
This might not block calling render directly on the problem block within a library content module because of the very weird things we do with inheritance and default values there, but I don’t think this would be too hard to work around.
I’ll try to think more about what unifying access rules on the platform could look like, and write more here if I come up with anything that could be useful. It’s a big undertaking. @braden or @kmccormick may have thoughts here as well.
Some random other thoughts about about a revamped permissions system, jotted down before I get sucked into meetings this morning:
We currently conflate permissions with composition in a way that I don’t think we should. Permissions are used to implement “which A/B test content should we show the student”, in addition to “is the student allowed to access the test”. I believe these should be better separated out.
In terms of “who is supposed to be able to access what” role-based permissions, we currently implement it on a very granular level in the LMS, but we only really care about groupings at the sequence level and above (or maybe unit). As a result, the current system does 10-100X the work it really needs to.
If both of those areas are factored out, what remains at the XBlock rendering layer could hopefully be really dumb in terms of permissions and permutations, and just render a given chunk of content that these other systems have composed and declared accessible by a given user.
Thank you for your feedback, @dave! There is a lot to consider here, and I appreciate you laying out these different options and potential roadblocks.
I like your JWT idea, and it would work well for us. I knew that the centralized access control was a lofty idea and that it’s probably not the ideal time to embark on that journey. I do have some other thoughts.
Clarification Questions
I had thought about user partitioning as a way of implementing this. We’d need to create a REST API to call from the special exams service. What are your reservations about this idea?
What do you mean by “composition” here? Which of “which A/B test content should we show the student” and “is the student allowed to access the test” is “permissions”, and which is “composition”?
JWT Token
I have mostly followed your idea.
The one thing I’m unsure of is who issues the JWT. I imagine it would the special exams service. It has the business rules to mostly determine whether a learner should have access to exam content, and it can create and sign a JWT with its private key. As the learning MFE makes requests to the special exams service, the special exams service will send back a JWT to the learning MFE if something happens that grants the learner access to the exam (e.g. the learner starts the exam, etc.). The MFE then forwards the JWT token in a cookie to the LMS’s render_xblock view. The LMS, which will (somehow) have the special exam service’s public key, will verify the signature and the claims in the JWT.
That makes sense to me. However, I thought the LMS was the only OAuth authorization server and would be issuing JWTs, per OEP-42. Has that changed?
Am I missing something here?
Other Options Specific to xBlock Loading/Rendering
There are a few other mechanisms that could get us what we want, I think. If we instead develop special exams as a plugin, then we could leverage the following.
We, theoretically, could add a filter hook to the render_xblock view - something like RENDER_XBLOCK_REQUESTED. Plugins could register pipeline_steps that run to modify the context that’s sent to the template that would show or hide the content or to raise exceptions to block rendering entirely. I think this would similarly be too slow and cause operational issues, though.
I did find references to Plugin Contexts described in edx-django-utils, which seems to implement the general idea above without filters: “This means that a plugin will be able to add context to any view where it is enabled.” Although, it appears to require theming, and I know very little about that.
I came across this thread, titled Pluggable access control both viewing and enrolling in a course. I haven’t read through it in its entirety, and it looks like it concerns courses and not xBlocks. Nevertheless, I wonder if a pipeline of access control processors, defined within individual plugins that describe access controls relevant to the plugin, could be used to determine access to an xBlock. It reminds me of the pipeline_steps in the hooks framework or the third party authentication pipeline.
Plugin or IDA?
Since some solutions lend themselves better to plugins and others to IDAs, what are your thoughts about whether the special exams “service” should be an IDA or a plugin? I cannot find advice on when to consider one over the other. Special exams, which would encompass proctoring and timed exams, are an extension to the core courseware and the learning subdomain. And when I read about the drive for easier extension of the platform, I see mainly investment in plugin architecture. Are “extensions” synonymous with “plugins”? How do IDAs or micro-services fit into this?
I feel that keeping the special exams service a plugin would align better with the Architecture Manifesto, as I understand it. I feel that enabling special exams should be as simple as installing the special exams plugin. Standing up a separate micro-service to enable special exams seems like a big lift. However, your JWT idea lends itself well to a special exams micro-service.
My understanding is that our main justification for an IDA was decoupling the deployment of the special exams service from that of edx-platform.
FWIW, I’m not going to advocate for the JWT approach. I honestly haven’t had time to think it through very well. I mostly just wanted to make sure you folks had an opportunity to weigh other options as you were deliberating.
I was originally thinking that there would be performance issues because of how often this is called, but I guess as long as data is pushed to a model somewhere and lookup at read-time was done locally and cached (i.e. not reaching out over a network anywhere), it would work.
That could also be a very interesting generalized system to reach for (as opposed to the entirety of permissions)–a general user partition grouping model that is time-aware. So in a data model somewhere, you could store something like:
I am sure it’s more complex than this, and it depends on the various query patterns, but I think that having a centralized mechanism like this that works on user partitioning is approachable. It would provide a way for partition schemes to still exist but to push data into a common API/model. The goal would be to do all the interesting, extensible/pluggable logic at the time of group assignment, but to make the read operation stupid simple and involve just the new API and its data models, and not any plugin code.
It’s also not something where you’d have to port over everything at once. You could use the new models and API to fulfill this particular use case, while leaving the door open to convert the other partition types later.
Sorry, this was me writing something in a hurry, and my own sense of the boundary for this is imprecise. When I think about permissions, I think of think of it as applying role based access to larger resources–i.e. “are you allowed to take this exam?” When I think of composition, I think of “what is the permutation of content within this exam that this particular student sees?”.
Two students might have the same role and same permission to take the same test, but they could get different content because of randomized problems from content libraries, A/B tests, etc. Right now, we kinda mash these concerns together because we’re only really interested in generating a set of blocks for the user to see. But I think they’re solving different problems, and we should better tease them apart. User partitioning is a definite step in that direction.
Also, making the distinction between “composition” and “permissions” is probably the wrong wording. I feel like permission and access control is one big umbrella (since we do want to restrict students to the permutation of content that we’ve decided on for them), and “composition” is a distinct component of that.
So maybe this idea breaks down because the thing that needs to issue any such token doesn’t have all the information necessary to determine whether people should be allowed to see the content anyway.
I think it’s a different usage than OEP-42 was intended for. This JWT wouldn’t really be used to determine the user’s identity. It would be something held by the user to prove “the exams service says that I’m allowed to take this exam for the next two hours”. But it likely does open up a separate can of worms.
I agree that these aren’t really feasible, for the reasons you mention.
I think the main tradeoff you’re looking at is between deployment speed and data coupling. If there’s a lot of coupling to content or user data that you would otherwise have to replicate across services, then I think that a plugin is the better bet. If you’re going to be doing a lot of fast iteration on this and really need the deployment independence from edx-platform, a separate service makes more sense. As you point out, if there’s a relatively modest amount of code, plugins will help you avoid all the fixed overhead of a service (deployment pipelines, testing setup, all that not-rocket-science-but-it-still-takes-time setup stuff).
I agree that going in-process would be easier for you in the long run, but I don’t have a great understanding of your requirements, or how big you expect this will grow.