Feedback on Open edX REST APIs

Development Architecture
1

Hey All,

I’ve been spending a bit more time getting familiar with the current state of the REST APIs in edx-platform and I’m seeking some input!

  • What are some issues you’ve run into when building or using REST APIs in the Open edX ecosystem?
  • Are there tools or plugins you’ve developed to help you work around the limitations of the core system?
  • What are things you wish you could do with the REST APIs that you can’t today?
2 Likes
2

These are all very good questions.

We have been using the APIs we can for extending Open edX (specially the lms) for a long time and I think the most common issues we have found are:

  • Administrative access: many of the available APIs were designed for the frontend to so something on the backend on behalf of the currently enrolled user. Sometimes that user is an instructor and can act on behalf of a learner. Very often our use case ends up being a different service trying to do a server-to-server call on behalf of either a learner or an instructor and this is often not supported.

  • Token auth: some APIs still only support session_id as a mean of authentication and this makes it extra hard to call from scripts or other servers.

  • Long lived tokens: authentication for APIs can be done via oauth tokens and more recently JWT tokens. The default life of this tokens is short (1hour) and one must refresh them or create new tokens. For some uses we maintained a change in a fork that allows us to create tokens that are valid for 1 year. It would be great to have this option in main for server-2-server calls.

  • Standardization: the session_id issue points in a more general direction of having all APIs support the latest standards. I know this is easier said than done because some APIs have not been touched in years and it’s a tough sell to go change them when they are not obviously broken, but a general baseline would be nice.

In terms of tools we have created some edxapp plugins to help us contain the maintenance work of some custom APIs.

  • eox-core: we wrote admin APIs for management of base objects such as courses, users, enrollments from external systems.
  • eox-audit-model: this is a dependency for most of our APIs and it keeps a record of what models were changed with them. Specially useful for when third party systems are doing calls to change core objects.

If you want more detail or depth in any of my comments please let me know and I’ll be happy to elaborate.

4 Likes
3

Related to @Felipe’s points, some of the older APIs do not use Django REST Framework at all, and are not picked up by the OpenAPI documentation schema so they don’t appear in the LMS APIs docs nor on the Swagger site. For example, the “v1” XBlock APIs (which must be one of the most frequently called of all the APIs in the LMS) don’t appear there.

I think the auto-generated OpenAPI spec is under-used. Even some long-time developers in the community don’t know about it, nor about how it can auto-generate a fully typed python or TypeScript or JS API client for you.

Personally I know we have things like edx-rest-api-client which is a python library to handle authenticating with OpenEdX APIs, but it doesn’t contain actually typed functions like client.get_xblock(...) or client.get_grades(). I would love to see something more like the PyGitHub library for the GitHub API, where the client has all the models and methods defined for you, with type hints. As I mentioned, such a client can be largely auto-generated from the OpenAPI spec, assuming the spec is accurate.

4 Likes
4

Thanks for the feedback y’all. That’s some good stuff to think about. Some thoughts and questions below.

For this point, I recently identified all the endpoints that override the default authentication classes but don’t support JWT auth and we’ll be working to update them so that they just use the default authentication classes and we’ll be adding JWT auth to the default so that it works with all endpoints by default going forward.

I’d like to understand this a bit more. If you have a client_id and client_secret for a user, or a JWT/Refresh Token pair, why not just use those to auto regenerate tokens if the current one has expired? For server to server calls I would think you would have an OAuth App with a client_id and client_secret and you could just generate new tokens whenever you need them pretty easily.

Thanks for calling this out, I’ll take a closer look and see what it might take to either convert these to DRF or to annotate them such that they show up in the Swagger Site.

5

Sure, I can elaborate a bit more. We have had customers in the past calling the APIs we created in the eox-core plugin that did not have the option to create a token manager class. They asked us to provide a long lived token so they could manually put it in their integration as other APIs do with API_KEYS.
I know this is not ideal and that short-lived access tokens and long-lived refresh tokens are probably more secure, but it is a case that we encountered more than a couple times so I thought I mentioned here.

6

I have some thoughts on validations, since I was just working on them for the experimental CMS API.
At least in studio, my observation is that public APIs have serializers with validations, but internal APIs - that is REST APIs used by the MFEs - don’t. Also, the validations we have are done with the standard drf serializers. Those by default do not use strong parameters.
I don’t consider this quite sufficient. I would expect any REST API - whether it’s accessible only via MFEs or public - to have backend validations, and coming from Rails, I would also expect validation to happen using strong parameters: Securing Ruby on Rails Applications: Part 3 (Use Strong Parameters) - Mintbit…
Since public APIs are especially vulnerable to users wreaking havoc with bad parameters they pass, I feel strong parameters and good field validations to be very important.
(The StrictSerializer class here does the trick, for example: https://github.com/openedx/edx-platform/blob/master/cms/djangoapps/contentstore/rest_api/v1/serializers/common.py.)

If you agree with that take, I would love to get this included in any ADR on future API design.

7

Hi Felipe,

Very useful, could you tell us more about how to implement it ?
Thank you
Amaury

8

Hi @braden

Could you please tell me a bit more about the XBlock APIs, i was not able to query the answers and metadata related to polls and survey. I’m very interested in getting those data and do not understand how to access them.
Thanks
Amaury

9

There is no generic API for such things, but the poll/survey APIs are documented here: xblock-poll/README.md at master · open-craft/xblock-poll · GitHub . I believe if you are an instructor, it also shows a UI for downloading a CSV report of each one, and there is probably some way to access that via an API if you need to as well.

10

Hi.
Yes. There is a GUI. Main aim is to be able to query the data without manual export.
The export feature can be called with API, yes, however don’t know how to properly use it to get the file loaded as a dataset in my notebook

11

As I mentioned, we have maintained this in a fork and you can see the code here:

1 Like