XQueue refactoring project

Announcements
1

Hi all,
I’m glad to announce that we have started a project to refactor XQueue, as part of the Axim request.

What is XQueue?
XQueue is a mechanism that enables site operators to implement external graders for complex assignments. In computing courses for instance, assignments that request the students to write code are complex to evaluate. There is no single correct answer to them, and grading involves executing the code submitted and running many tests on it. In addition to the complexity of supporting multiple programming languages and types of assignments, executing a code provided by a regular user imposes a security risk. That’s why the platform does not include this as part of the core services.

How does it work today?
Today the whole workflow happens in three places:

  1. in edx-platform, where the student submits the response and receives the grade and feedback
  2. in an intermediate server, the XQueue Server, totally independent from edx-platform, which keeps track of the submissions in an internal database and serves the API endpoints for the external grader
  3. in the XQueue Watcher, which implements the external grader functionality.

This diagram shows the current workflow:

image
image3984×2044 374 KB

What are the objectives of the project?
The project has the following objectives:

  • Simplify the global architecture
  • Deprecate XQueue Server, which is an under-maintained repo
  • Provide more scalable mechanisms for large sites

We also consider these design premises:

  • Keep backwards compatibility with existing XQueue Watchers
  • Implement an intermediate phase where both solutions can coexist and allow a seamless transition
  • Leverage on newer, more scalable mechanisms like the event bus and edx-submissions.

The proposal
With this in mind, we are proposing the following architecture for the new service to replace XQueue:

image
image5024×3164 656 KB

Legacy external graders that implemented XQueue Watcher or similar (option A) will still work, although the URL of the endpoints will change.
All the functions of the XQueue Server will be assumed by edx-submissions, specially the endpoints for the external graders.
Site operators might want to implement a Django application that interacts with the event bus to retrieve submitted responses and send grades and feedback.
The intermediate phase is not depicted in the diagram, but will allow connecting both via an existing XQueue Server and edx-submissions. Site operators will be able to transition all pending submissions to the new workflow without impacting the users.

How to contribute?
Your comments are welcome! If you are using XQueue, if you have special use cases that we should consider or you have any comment about the project please feel free to reply to this post.

1 Like
2

I’m curious about how the XQueue Listener will work? Is that going to be in the LMS runtime listening to the event bus? Will operators have to run the listener as a small management command run from the LMS code?

3

@Andres.Aulasneo could you please follow up on Feanil’s question in this thread?

4

In Option B, there will be two listeners: one to catch the new submission, which will have to be implemented by the site operator as part of the Django plugin, and another to catch the grade and feedback, which will reside in the edx-platform and will be implemented as part of this project. The idea is that it should run automatically, without need of any management command.

The latter will be triggered as soon as the grader sends the output in the event bus as a Django signal.

Does it answer your question?

5

Sort of, what do you mean it will reside in the edx-platform? Is the edx-platform going to be listening to the eventbus or do you expect to use a diferent mechanism to recieve grades and feedback?

6

The initial idea is yes, to have a listener actually in edx-submissions (better than edx-platform itself), listening to the event bus. Do you think it might bring problems? Note that the event bus will not replace the legacy REST flow, instead you can implement one or the other.