"Live" Development: Learner Backpack / Data Storage (done!)

Wow, this is weird and cool. Thanks for sharing your process here!

1 Like

Yet Further Progress!

That commit does the following:

  • HX-JS now creates an iframe to the backpack problem.
  • If it loads properly, it makes the Get/Set/Clear functions available to the page in general.
  • If not, it still creates those functions but they just return null.

I also switched up the Boilerplate Course so that the problem has the filename backpack.xml. The Course Template Builder now includes the backpack, so anything you make starting from that base will have it.

Programming is the art of reducing the number of bugs in your code.

Previous bug: calling parent.whatever within a function invokes that at the same level at which the function is called. When you boost a function up through some iframes, it looks for the current parent. So all the functions I brought up from the innermost frame (Get/Set/Clear) need to refer back to the original parent frame. I think I’ve got that fixed now.

Current bug: I can only set the state once. Later calls to hxSetState do submit the probloem, but the state is not properly updated. Hmm.

I am now in a fascinating state where the first time I call a function, it runs normally, and the second time I call it, I’m getting an older version. I know because I added a ton of console.log() calls to the function, and those all fire the first time but not the second time.

Honestly I just want to know where it’s getting the old version.

Clearing my cache didn’t fix the problem.

Calling hxClearData runs into the same issue. In fact, for all I can tell it’s calling the same function. Or just, like, only returning “true” and then moving on with other stuff.

Testing out the function calls at the innermost iframe shows me they’re still working there. It’s the connection between them that’s the problem.

I think it’s between hx-js in the outermost window (the edX page) and the iframe in which I load the problem via XBlock URL. hx-js publishes the functions, but it only does that once, when the iframe initially loads. I need it to happen every time.

I should probably have the inner window fire an onload thing, huh. Ok. Let’s see if I can get that working. I’ve got some pieces from the Qualtrics Grader Problem that I think I can use for this.

Oh, postMessage(), why didn’t I use you in the first place?

Now instead of listening for the (first, duh on me) time the backpack loads, I’m picking up every time it loads. The functions are getting reconnected properly and I can store things appropriately.


More things to do:

  • Do I want other functions? Should I have a hxClearAllData() and an hxGetAllData?
  • There’s a potential “simultaneous editing” issue for someone with two browser windows open on two different pages. Do I care?
  • Updating the ReadMe files.
  • Probably an autosave.
  • That LZ compression I mentioned.

Added a GetAllData function.

Updated the ReadMe.

Added compression. It seemed a little broken at one point, but I think that’s because there were old cached versions of some files lying around. Because things are loaded within iframes within iframes, that’s going to be a major concern any time this is updated. We’ll want to update as rarely as possible so learners don’t have to clear their cache.

I realized as I was doing this that auto-saving wasn’t actually necessary, because any use of hxSetData() saves. It’ll be on developers who hook into this to keep a good cadence with their storage - not too often, but not too rare.

I think we may be done! Here are the links to the pieces of this:

Backpack problem and instructions:

HX-JS - get your updated version:

There are still some oddities and potential issues. Some pages send a “Backpack ready” message twice, and I don’t know whether that’s actually problematic. I don’t know whether certain data might be stored in a way that breaks the state - I vaguely remember needing to escape percent signs when storing problem states, and with the compression you can’t even tell whether you’re storing a percent sign. Gonna have to watch for glitches.

Comments and questions are welcome.

HX-JS and the Learner Backpack problem are available under the MIT license. Feel free to use them in whatever way you want.

1 Like


  • One issue with this solution comes if someone does not submit the assignment. With no state stored for the learner, edX never calls the JSInput setState() function, which is where our code makes the get/set/clear functions available to the world. This may not be the worst thing. Preventing learners from accessing parts of the course until they agree to the honor code is something I’m ok with.
  • I had some issues with the compression, which I think were related to Javascript storing all strings in UTF-16 and whatever’s storing it on the edX side using something else. I switched the compression to compressToEncodedURIComponent() instead, which is not nearly as compressy but which should be more reliable.
  • Added the ability to input a whole object worth of data at once, rather than just one key/value pair at a time. This makes it possible to “batch” data input so that the problem doesn’t have to refresh as often.

I’m working on the first application of this. I’ll post here when it’s ready. Who knows, maybe we’ll even fold it into the next version of Studio Advanced.

After some substantial work, I have created the first application for this: the Journaling Assignment!

Learners can write in rich text (thanks to the summernote editor), it gets saved in the backpack, and they can submit it for a grade. The current version just checks submission length, so this is “required activity” rather than “graded essay” but we’ve already got several courses where this would be useful.

There’s one real weird thing about this: you can’t have the summernote editor in the same component as the jsinput problem. If you do, the getState and setState functions for the jsinput problem don’t fire. If you have any idea why, suggestions are definitely welcome - we’re stumped over here. Gonna try to fix this one, so if you want to hold off on implementing this I don’t blame you.

However, I also got something nice on the way to this! If you use hx-js, you can also use the editor on its own without the assignment. Just make a div with the class hx-editor and it’ll automatically get converted into a rich text editor. The data-saveslot attribute will tell hx-js where to store things, so you can show students their own writing again later on.

Quick update: I’m just about done with the journaling assignment. My last hurdle is a focus issue in Safari that comes up while auto-saving. I think I know what’s causing it, but I’m still working on fixing it.

Welp, it’s broken.

No idea what happened. It just stopped working on edx.org one day. Even Partner Support didn’t know what happened, because no one keeps a changelog, even internally. My best guess is that some iframe security setting changed, but who knows.

It was in a course that was going live in a week. We had to strip it out.

Immensely frustrating.

Dang. I’m very interested in being able to access learner data in a question. I’m new to edx and am trying to figure out how to access learner data in a question.

I don’t understand why you’ve had to use the iFrame.

Since you don’t have to deal with CORS could you JS to get and set the data in the server using an API call?

Or fill localStorage if its empty or replace it if the remote “latest update” timestamp is < the locally stored “latest update” timestamp?

To my knowledge there are no API calls I can make to an edX instance to store data in the server, or to retrieve a learner’s answer to a question.

JSInput questions are different: there’s a Javascript object (part of the “problem state”) that is created when the problem loads, and sent to the server when the problem is submitted. Whatever we store in the problem state for this particular kind of problem, we can retrieve later.

Does that help?

Yes that probably helps. Seems like I can’t readily do what I want. I was hoping to be able to change or add to the question content e.g.

"Good job Paul, after a rough start you’ve gotten the past 4 questions right. "

For the JSInput question is the “problem state” object empty or is it initialized with any data?

You can actually decide how to initialize the state yourself - completely blank, empty data structure, some starting data, whatever you want.

If you want to give a particular piece of feedback within one problem, that’s easier - you can use Write-Your-Own-Grader problems and return all kinds of custom stuff.

Sharing data between problems is very difficult if they’re not on the same page. That’s one of the things I had hoped to make easier with this project - providing variables that are keyed to a particular student and letting problems interact with them via javascript. Hopefully I’ll be able to fix this project eventually, but I’m awful busy these days.

If you want some examples of JSInput problems, check out https://github.com/HarvardX/js-input-samples , where there a bunch of problems that I wrote and that some folks at Stanford wrote. The text_logger problem is probably the easiest one to start with.

Thanks for the pointers.

So looking at the “Write Your Own grader” & JSProblemState usage in those examples it seems that they all seem useful enough but its not clear to me how to learn & use the leaners data e.g. first name from any of the documentation I’ve read thus far.

Is there any documentation about the context within which this JSInput code or the Python code in an OLX script executes? I read about the CodeJail. But what objects are accessible during execution that might contain the user’s info or an id to do a lookup somewhere to find it?

I’m interested in doing some personalization of the questions/content. Thanks so much for making time to help me understand better.

You are correct - there’s no easy access to user data from within a problem. There’s no access to it at all from the problem’s python; instead you have to pull it via javascript.

You can get the username via javascript from analytics._user._getTraits(), but only once the page loads completely. Then you have to somehow pass it to the python if you want it in the feedback message. If you have a JSInput problem, you can pass it back to the grading python as part of the state I mentioned before. Otherwise, you need to use Stupid JQuery Tricks™ to insert it after the message appears.

Practically none of this is documented. My colleagues and I had to hunt around to find the analytics object, and there’s no guarantee it will be unchanged in the future. I keep a wiki called edX Undocumented that attempts to track some of these things.

The journaling assignment is working again!

Honestly, the fix made me wonder why it was ever working in the first place… but everything’s better now.

Congrats I’m glad its working again for you. Thanks again for the info