Exception when submitting a Custom response Python problem

It looks like the custom response OLX with Python code problem breaks upon submission when codejail service is enabled. I’m posting it here to bring it into the notice of the community and see what they think about this.

Steps to reproduce:

  • Create a custom response problem with a script like
  <script type="text/python" system_path="python_lib">
# Python code here e.g. 

def hint_fn(answer_ids, student_answers, new_cmap, old_cmap):

    <hintgroup hintfn="hint_fn"/>

  • Publish changes, And try to test the submissions with any user(admin, staff, or any), NOTE: Make sure that you have enabled codejail service to run the custom Python code.
  • You can check the error stacktrace at the end of the post

Probable Cause?
I think I was able to get to the root of this error and here are some of the findings:

  1. The error is produced because when we try to run the custom Python code through codejail, we build a dictionary followed by a JSON to send as a payload to the request. The code breaks when we try to convert our dictionary to JSON.
  2. Why point# 1 is happening? We are trying to convert a dictionary to JSON which has a whole binary/bytes python_lib package in it(See the example dict below) which json.dumps() fails to convert to JSON and it breaks.

What i think, we do with Custom Response Problems?

For every custom Python code response problem, we run the custom script through codejail or local service either when we load an xBlock in the browser or we submit a response.

Possible Solution?
We should remove any binaries from globals_dict as well before we try json.dumps() on it. Just like how we are doing it here.

I’m not sure how many things can break with this fix, So just wanted to hear from commuity.

An example of how the data looks with a Binary in it is:

'globals_dict': {'seed': 1, 'anonymous_student_id': '98845cd477b76f4339f46600264a3e83'}, 'python_path': ['python_lib.zip'], 'limit_overrides_context': 'course_id', 'slug': 'course_slug', 'unsafely': False, 'extra_files': [('python_lib.zip', b'PK\x03\x04\n\x00\x00\x00\x00\x00k^]O\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0b\x00\x10\x00grader_lib/UX\x0c\x00\x12`\xb8]\xfa_\xb8]\xf5\x01\x14\x00PK\x03\x04\x14\x00\x08\x00\x08\x00=c\xf8J\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x1c\x00\x10\x00grader_lib/MultiFunction.pycUX\x0c\x00SZ\xb8]\x95\x1fvY\xf5\x01\x14\x00\xddY_s\xdb\xc6\x11_\x10$ER\x94-\xdb\x92l\xd5\xb1\x85\xd8n\xcb\xb6\x8a\xe5\xa93\x93\x99\x8c\xed$\x8d\xc7}p\x93\x07\xa8\x8d\xc7rU\x0cD\x1cI\xc8 @\x03\'\x0b\xecP\x0f\x1d\xe5\xa1\x1f\xa1\xed\xf4\xad\x0f\xfd\x0c}\xee\xa7\xe8[_\xfa\t\xfa\x01\xda\xdd\xbd;\xf0\x8f\x18\xc5I\x98t\xa6\x94\x08-\x0e\x87\xbd\xfd\xf3\xbb\xdd\xbd\x95\xfd\xef\x95\xc6\xbfn\xbe~\x...

Stack trace of the actual error:

Staff debug info: Traceback (most recent call last): File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2297, in execute_check_function safe_exec.safe_exec( File "/usr/local/lib/python3.8/contextlib.py", line 75, in inner return func(*args, **kwds) File "/openedx/edx-platform/xmodule/capa/safe_exec/safe_exec.py", line 159, in safe_exec emsg, exception = get_remote_exec(data) File "/openedx/edx-platform/xmodule/capa/safe_exec/remote_exec.py", line 48, in get_remote_exec return remote_exec_function(*args, **kwargs) File "/openedx/edx-platform/xmodule/capa/safe_exec/remote_exec.py", line 69, in send_safe_exec_request_v0 payload = json.dumps(data) File "/usr/local/lib/python3.8/json/__init__.py", line 231, in dumps return _default_encoder.encode(obj) File "/usr/local/lib/python3.8/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/local/lib/python3.8/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/usr/local/lib/python3.8/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type bytes is not JSON serializable During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/openedx/edx-platform/xmodule/capa_block.py", line 1777, in submit_problem correct_map = self.lcp.grade_answers(answers) File "/openedx/edx-platform/xmodule/capa/capa_problem.py", line 436, in grade_answers new_cmap = self.get_grade_from_current_answers(answers) File "/openedx/edx-platform/xmodule/capa/capa_problem.py", line 494, in get_grade_from_current_answers results = responder.evaluate_answers(self.student_answers, oldcmap) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 314, in evaluate_answers new_cmap = self.get_score(student_answers) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2267, in get_score self.execute_check_function(idset, submission) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2308, in execute_check_function self._handle_exec_exception(err) File "/openedx/edx-platform/xmodule/capa/responsetypes.py", line 2523, in _handle_exec_exception raise ResponseError(text_type(err), traceback_obj) xmodule.capa.responsetypes.ResponseError: ('Object of type bytes is not JSON serializable', )

I was looking at https://github.com/openedx/edx-platform/pull/27795 PR that originally added the options for running the codejail optionally.

Would really appreciate it if someone with experience in codejail could respond. @mgmdi or @eric.herrera any chance you can take a look at this and let us know what you think about the problem? I’ve tried a reasonable amount of details, But definitely let me know if you would need more info from my side.

1 Like

Hello @arslanashraf, thanks for your patience. I couldn’t replicate your error with the script you gave us. Can you put a script example that has enough information to be able to reproduce the error?

Another question, Do you mean Problem > Advanced > Blank Advanced Problem, with custom response OLX? Or is there something I’m missing?

On the other hand, I opened an issue around this to have your problem on our radar: ["BUG"] Exception when submitting a Custom response Python problem · Issue #40 · eduNEXT/tutor-contrib-codejail · GitHub

I’ll be attentive to your answers.

1 Like

@mafermazu Thanks for looking at this and taking the appropriate actions to keep this on priority.

So, Actually, we were seeing this in one of the courses and I’m not sure I could share that exact problem code with you so I tried reproducing this with a locally created course in Studio and finally I was able to do so. Also, I think I missed one point in the detail in my original post but now I have all the steps. Please take a look at the steps below:

So here are the details:

  1. Create Problem > Advanced > Blank Advanced Problem with: (Note: This is a regular drag and drop inline problem XML with just the addition of hint_fn)

    Sample Drag and Drop code (Click Here)
    <script type="text/python" system_path="python_lib">
    def hint_fn(answer_ids, student_answers, new_cmap, old_cmap):
          <h3>Drag and Drop with Outline</h3>
          <p>Label the hydrogen atoms connected with the left carbon atom.</p>
          <drag_and_drop_input img="https://studio.edx.org/c4x/edX/DemoX/asset/ethglycol.jpg" target_outline="true" one_per_target="true" no_labels="true" label_bg_color="rgb(222, 139, 238)">
              <draggable id="1" label="Hydrogen" />
              <draggable id="2" label="Hydrogen" />
              <target id="t1_o" x="10" y="67" w="100" h="100"/>
              <target id="t2" x="133" y="3" w="70" h="70"/>
              <target id="t3" x="2" y="384" w="70" h="70"/>
              <target id="t4" x="95" y="386" w="70" h="70"/>
              <target id="t5_c" x="94" y="293" w="91" h="91"/>
              <target id="t6_c" x="328" y="294" w="91" h="91"/>
              <target id="t7" x="393" y="463" w="70" h="70"/>
              <target id="t8" x="344" y="214" w="70" h="70"/>
              <target id="t9_o" x="445" y="162" w="100" h="100"/>
              <target id="t10" x="591" y="132" w="70" h="70"/>
        <answer type="loncapa/python">
    correct_answer = [{
      'draggables': ['1', '2'],
      'targets': ['t2', 't3', 't4' ],
    if draganddrop.grade(submission[0], correct_answer):
      correct = ['correct']
      correct = ['incorrect']
        <hintgroup hintfn="hint_fn"/>
  2. Add this python_lib.zip to your course assets
    python_lib.zip (345.9 KB)

I noticed that the actual problem is because of this additional python_lib.zip file that we have in our assets and the platform loads this file because of <script type="text/python" system_path="python_lib">. But when it does, It tries to convert this whole library into JSON for code jail payload and fails to do so. Here is the relevant edX code that breaks if a library is there in the assets of a course.

I think we should approach this in ways:

  • We should fix this issue upstream e.g. if there is a lib in the assets we don’t try to convert it into a JSON.
  • If adding a zip library in assets is not the right thing we should add it somewhere in the docs.

Let me know if you are able to reproduce this now or need more details.

1 Like