It looks like the custom response OLX with Python code problem breaks upon submission when codejail service is enabled. I’m posting it here to bring it into the notice of the community and see what they think about this.
Steps to reproduce:
- Create a custom response problem with a script like
<problem>
<script type="text/python" system_path="python_lib">
# Python code here e.g.
def hint_fn(answer_ids, student_answers, new_cmap, old_cmap):
</script>
<customresponse>
<drag_and_drop_input>
</drag_and_drop_input>
<hintgroup hintfn="hint_fn"/>
</customresponse>
</problem>
- Publish changes, And try to test the submissions with any user(admin, staff, or any), NOTE: Make sure that you have enabled codejail service to run the custom Python code.
- You can check the error stacktrace at the end of the post
Probable Cause?
I think I was able to get to the root of this error and here are some of the findings:
- The error is produced because when we try to run the custom Python code through codejail, we build a dictionary followed by a JSON to send as a payload to the request. The code breaks when we try to convert our dictionary to JSON.
- Why point# 1 is happening? We are trying to convert a dictionary to JSON which has a whole binary/bytes
python_libpackage in it(See the example dict below) whichjson.dumps()fails to convert to JSON and it breaks.
What i think, we do with Custom Response Problems?
For every custom Python code response problem, we run the custom script through codejail or local service either when we load an xBlock in the browser or we submit a response.
- But why it fails only on Submit and not load? The answer to this is that when the problem loads we send the extra files e.g.
python_libas part of the parameterextra_filesto oursafe_execfunction call. edX handles this automatically and pops the files separately before dumping JSON. But when we submit a problem, the samepython_libcomes as a part ofglobals_dict, which edX doesn’t handle appropriately and doesn’tpopbefore dumping JSON.
Possible Solution?
We should remove any binaries from globals_dict as well before we try json.dumps() on it. Just like how we are doing it here.
I’m not sure how many things can break with this fix, So just wanted to hear from commuity.
An example of how the data looks with a Binary in it is:
'globals_dict': {'seed': 1, 'anonymous_student_id': '98845cd477b76f4339f46600264a3e83'}, 'python_path': ['python_lib.zip'], 'limit_overrides_context': 'course_id', 'slug': 'course_slug', 'unsafely': False, 'extra_files': [('python_lib.zip', b'PK\x03\x04\n\x00\x00\x00\x00\x00k^]O\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0b\x00\x10\x00grader_lib/UX\x0c\x00\x12`\xb8]\xfa_\xb8]\xf5\x01\x14\x00PK\x03\x04\x14\x00\x08\x00\x08\x00=c\xf8J\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x1c\x00\x10\x00grader_lib/MultiFunction.pycUX\x0c\x00SZ\xb8]\x95\x1fvY\xf5\x01\x14\x00\xddY_s\xdb\xc6\x11_\x10$ER\x94-\xdb\x92l\xd5\xb1\x85\xd8n\xcb\xb6\x8a\xe5\xa93\x93\x99\x8c\xed$\x8d\xc7}p\x93\x07\xa8\x8d\xc7rU\x0cD\x1cI\xc8 @\x03\'\x0b\xecP\x0f\x1d\xe5\xa1\x1f\xa1\xed\xf4\xad\x0f\xfd\x0c}\xee\xa7\xe8[_\xfa\t\xfa\x01\xda\xdd\xbd;\xf0\x8f\x18\xc5I\x98t\xa6\x94\x08-\x0e\x87\xbd\xfd\xf3\xbb\xdd\xbd\x95\xfd\xef\x95\xc6\xbfn\xbe~\x...
Stack trace of the actual error:
Staff debug info: Traceback (most recent call last): File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2297, in execute_check_function safe_exec.safe_exec( File "/usr/local/lib/python3.8/contextlib.py", line 75, in inner return func(*args, **kwds) File "/openedx/edx-platform/xmodule/capa/safe_exec/safe_exec.py", line 159, in safe_exec emsg, exception = get_remote_exec(data) File "/openedx/edx-platform/xmodule/capa/safe_exec/remote_exec.py", line 48, in get_remote_exec return remote_exec_function(*args, **kwargs) File "/openedx/edx-platform/xmodule/capa/safe_exec/remote_exec.py", line 69, in send_safe_exec_request_v0 payload = json.dumps(data) File "/usr/local/lib/python3.8/json/__init__.py", line 231, in dumps return _default_encoder.encode(obj) File "/usr/local/lib/python3.8/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/local/lib/python3.8/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/usr/local/lib/python3.8/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type bytes is not JSON serializable During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/openedx/edx-platform/xmodule/capa_block.py", line 1777, in submit_problem correct_map = self.lcp.grade_answers(answers) File "/openedx/edx-platform/xmodule/capa/capa_problem.py", line 436, in grade_answers new_cmap = self.get_grade_from_current_answers(answers) File "/openedx/edx-platform/xmodule/capa/capa_problem.py", line 494, in get_grade_from_current_answers results = responder.evaluate_answers(self.student_answers, oldcmap) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 314, in evaluate_answers new_cmap = self.get_score(student_answers) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2267, in get_score self.execute_check_function(idset, submission) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2308, in execute_check_function self._handle_exec_exception(err) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2523, in _handle_exec_exception raise ResponseError(text_type(err), traceback_obj) xmodule.capa.responsetypes.ResponseError: ('Object of type bytes is not JSON serializable', )
A thing that can help you, and it’s associated with your approaches, is that when we test codejail we import a 
