ORA2 UX users forgetting to submit

Note: creating this thread on behalf of OpenCraft.

One of our clients is having issues where users are consistently forgetting to submit ORA2 assessments. They upload their files and/or save their progress in the text response, but never come back to submit before the due date. We suspect it is because it’s not clear enough that their response will not be recorded unless they click “Submit”.

We would be interested to know if others have had this experience, and if there are any existing proposed changes to the ORA2 UX to help with this? Alternately, what research or changes would edX be interested in pursuing in this direction?

Ideas that we have had so far include:

  • automatically marking an in-progress response as submitted once the due date passes
  • allowing multiple submissions, effectively making every saved response and/or uploaded file event become the latest submission. (we researched this and the previous point and determined that these would be very complex to implement correctly)
  • modifying the UI layout to make the steps required clearer (simple but would require some research to work well)
  • adding behaviour to alert users that they have unsubmitted in-progress responses and that they will not be recorded until manually submitted. This could be an alert course-wide, or popup after uploading a file or saving a text response. (simplest method, possibly tricky to implement in a non-annoying manner)
  • add a button for staff to trigger a submit for all students with in-progress responses (unresearched; similar to auto-submitting but takes out the complexities required for auto-launching jobs at certain times)
  • automatically send email alerts to students when due-dates draw near warning about un-submitted responses (also still in idea phase)

Thoughts?

3 Likes

Gotta say that this is a long running issue for all type of problems in Open edX, and if there’s a way to solve it, it would be great.

Thanks @swalladge for taking the lead and publishing this post.

1 Like

We’ve seen this happen at HarvardX from time to time. If the long-promised notification functionality were part of the platform, this would seem like a perfect use case for them: “You have saved work for a problem at (link). Go there to complete and submit it.”

1 Like

We’ve definitely had customers with this issue, and it’s often caused problems, not just with deadlines but also with timed exams. Due dates aren’t the only issue! Anything that can lock learner access to a unit based on a condition causes this problem to rear its head.

That said, it’s not common en masse - at least not that I’ve seen. I think there’s three things I’d like to see here.

  1. An option for staff to force submission for a single learner whose submission is saved but not submitted
  2. An option for staff to force submission for all learners with saved progress
  3. An optional to enable auto submission when the deadline passes

I don’t like 3 as a solution, not every in-progress submission should pollute the pool of responses. I can see this being a big problem on larger courses where someone types a single word or space, never comes back and all of a sudden you’ve got hundreds of worthless responses in the way of grading. But 1 and 2 provide sorely needed support tools for instructors.