Design tradeoffs for Codejail upgrade darklaunch

[dave (Dave Ormsbee)]

@Tim_McCormack

Ah, yes, thank you for bringing that up – at one point I had been wondering whether we should make sandbox editions a user-visible feature. Then instructors would be able to migrate content at their own pace, and maybe even see dark-launch results for their own courses. I was thinking of how an xblock could indicate whether it takes the default edition, early access, legacy, a specific edition, etc.

I’m in support of that generally, since I do think there are other script backends that make sense. Ultimately, I’d really like to do something with WASM so that the sandbox execution environment becomes data that we can bundle and keep around forever. I think we should be able to add it as an optional attribute to the <script> tag we use to wrap instructor code in ProblemBlock.

@Tim_McCormack

I think that’s compatible with either option, but would tend to align better with Option A (because we’d be potentially looking at simultaneous support of 3+ editions, not just 2.)

Agreed. We might also have options that don’t really use codejail at all, like WASM or CEL.

@Tim_McCormack

And yeah, 3.14 is definitely going to break some things. I suspect we’re just going to have to try the darklaunch and see how bad the mismatch is before deciding how to address it.

So is the idea that the LMS would be making requests to both, using the 3.8 sandbox for the real result, and then comparing with 3.14 results in order to error log any potential behavioral differences?