# Error running AnswerDistributionWorkflow

**URL:** <https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687>\
**Category:** Site Operations Help\
**Created:** [July 9, 2020, 7:30am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687 "2020-07-09T07:30:17Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [July 9, 2020, 7:30am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/1 "2020-07-09T07:30:17Z")

</div>

Hi All,

I recently deployed ironwood lms and managed to install insights on a different server. I am at a point of running analytics tasks but got stuck at the first one. Can anyone help me with understanding this error please.

DEBUG:edx.analytics.tasks.launchers.local:Loading override configuration ‘override.cfg’…  
ERROR: Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
res.update(((param\_name, param\_obj.parse(attr)),))  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/parameter.py”, line 940, in parse  
return list(json.loads(x, object\_pairs\_hook=\_FrozenOrderedDict))  
File “/usr/lib/python2.7/json/ **init**.py”, line 352, in loads  
return cls(encoding=encoding, \*\*kw).decode(s)  
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode  
obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())  
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw\_decode  
raise ValueError(“No JSON object could be decoded”)  
ValueError: No JSON object could be decoded  
ERROR:luigi-interface:Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
res.update(((param\_name, param\_obj.parse(attr)),))  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/parameter.py”, line 940, in parse  
return list(json.loads(x, object\_pairs\_hook=\_FrozenOrderedDict))  
File “/usr/lib/python2.7/json/ **init**.py”, line 352, in loads  
return cls(encoding=encoding, \*\*kw).decode(s)  
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode  
obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())  
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw\_decode  
raise ValueError(“No JSON object could be decoded”)  
ValueError: No JSON object could be decoded  
Connection to localhost closed.  
Exiting with status = 40

Cannot figure out which json file it is talking about. Output.json file is shown below

{

```
    "username": "pipeline001",
    "host": "localhost",
    "password": "passwordxxxx",
    "port": 3306

```

}

Regards,  
Neville

---

<div class="post-metadata">

**Author:** ![David\_Adams](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/david_adams/32/569_2.png) [@David\_Adams](https://discuss.openedx.org/u/David_Adams)\
**Post date:** [July 9, 2020, 9:30pm UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/2 "2020-07-09T21:30:43Z")

</div>

> [@nevilleonline](#):
>
> override.cfg

Looks like it’s trying to read your override.cfg

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [July 10, 2020, 4:45am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/3 "2020-07-10T04:45:06Z")

</div>

If I dont specify override.cfg like below i still get an error  
export UNIQUE\_NAME=$(date +%Y-%m-%dT%H\_%M\_%SZ)   
remote-task AnswerDistributionWorkflow --host localhost --user ubuntu --remote-name analyticstack --wait   
–local-scheduler --verbose   
–src [hdfs://localhost:9000/data]   
–dest hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/dest   
–name $UNIQUE\_NAME   
–output-root hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/course   
–include [“_tracking.log_.gz\*”]   
–manifest hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/manifest.txt   
–base-input-format “org.edx.hadoop.input.ManifestTextInputFormat”   
–lib-jar [hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar]   
–n-reduce-tasks 1   
–marker hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/marker   
–credentials /edx/etc/edx-analytics-pipeline/output.json

Error code is different  
DEBUG:edx.analytics.tasks.launchers.local:Configuration file ‘override.cfg’ does not exist!  
ERROR: Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
res.update(((param\_name, param\_obj.parse(attr)),))  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/parameter.py”, line 940, in parse  
return list(json.loads(x, object\_pairs\_hook=\_FrozenOrderedDict))  
File “/usr/lib/python2.7/json/ **init**.py”, line 352, in loads  
return cls(encoding=encoding, \*\*kw).decode(s)  
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode  
obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())  
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw\_decode  
raise ValueError(“No JSON object could be decoded”)  
ValueError: No JSON object could be decoded  
ERROR:luigi-interface:Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
res.update(((param\_name, param\_obj.parse(attr)),))  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/parameter.py”, line 940, in parse  
return list(json.loads(x, object\_pairs\_hook=\_FrozenOrderedDict))  
File “/usr/lib/python2.7/json/ **init**.py”, line 352, in loads  
return cls(encoding=encoding, \*\*kw).decode(s)  
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode  
obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())  
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw\_decode  
raise ValueError(“No JSON object could be decoded”)  
ValueError: No JSON object could be decoded  
Connection to localhost closed.  
Exiting with status = 4

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [July 15, 2020, 9:13am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/4 "2020-07-15T09:13:30Z")

</div>

Anybody having any comment about this error would greatly help. thanks 🙂

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [July 15, 2020, 9:35am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/5 "2020-07-15T09:35:57Z")

</div>

@nevilleonline I don’t think it’s a JSON file that’s causing the error… I think it’s a JSON-formatted task parameter. I can see two in your command which aren’t valid JSON:

> [@nevilleonline](#):
>
> ```auto
> –src [hdfs://localhost:9000/data]
> –lib-jar [hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar]
> 
> ```

You need to put quotes around those strings:

```auto
–src ["hdfs://localhost:9000/data"]
–lib-jar ["hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar"]

```

See [AnswerDistributionWorkflow task documentation](https://edx-analytics-pipeline-reference.readthedocs.io/en/latest/running_tasks.html?highlight=lib-jar#task).

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [July 20, 2020, 6:53am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/6 "2020-07-20T06:53:34Z")

</div>

Thanks Jill for your response. I tried with the quotes and it seems to give the same error.

export UNIQUE\_NAME=$(date +%Y-%m-%dT%H\_%M\_%SZ)

export UNIQUE\_NAME=$(date +%Y-%m-%dT%H\_%M\_%SZ)   
remote-task AnswerDistributionWorkflow --host localhost --user ubuntu --remote-name analyticstack --skip-setup --wait   
–local-scheduler --verbose   
–src [“hdfs://localhost:9000/data”]   
–dest hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/dest   
–name $UNIQUE\_NAME   
–output-root hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/course   
–include [“_tracking.log_.gz”]   
–manifest hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/manifest.txt   
–base-input-format “org.edx.hadoop.input.ManifestTextInputFormat”   
–lib-jar [“hdfs://localhost:9000/edx-analytics-pipeline/site-packages/edx-analytics-hadoop-util.jar”]   
–n-reduce-tasks 1   
–marker hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/$UNIQUE\_NAME/marker   
–credentials /edx/etc/edx-analytics-pipeline/output.json

Error below

DEBUG:edx.analytics.tasks.launchers.local:Loading override configuration ‘override.cfg’…  
ERROR: Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
res.update(((param\_name, param\_obj.parse(attr)),))  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/parameter.py”, line 940, in parse  
return list(json.loads(x, object\_pairs\_hook=\_FrozenOrderedDict))  
File “/usr/lib/python2.7/json/ **init**.py”, line 352, in loads  
return cls(encoding=encoding, \*\*kw).decode(s)  
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode  
obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())  
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw\_decode  
raise ValueError(“No JSON object could be decoded”)  
ValueError: No JSON object could be decoded  
ERROR:luigi-interface:Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
res.update(((param\_name, param\_obj.parse(attr)),))  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/parameter.py”, line 940, in parse  
return list(json.loads(x, object\_pairs\_hook=\_FrozenOrderedDict))  
File “/usr/lib/python2.7/json/ **init**.py”, line 352, in loads  
return cls(encoding=encoding, \*\*kw).decode(s)  
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode  
obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())  
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw\_decode  
raise ValueError(“No JSON object could be decoded”)  
ValueError: No JSON object could be decoded  
Connection to localhost closed.  
Exiting with status = 40

Is there anything else you can see that i am not seeing?

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [July 22, 2020, 7:16am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/7 "2020-07-22T07:16:07Z")

</div>

> [@nevilleonline](#):
>
> Is there anything else you can see that i am not seeing?

The error logs don’t help, and I can’t see from your command what tripping it up ☹

> [@nevilleonline](#):
>
> File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 133, in \_get\_task\_kwargs  
> res.update(((param\_name, param\_obj.parse(attr)),))

This is really hacky, but you could try modifying this luigi code shown above to get it to print out the `param_name` when the parsing fails, so you can see which variable is causing the issue? e.g.

```python
    def _get_task_kwargs(self):
        """
        Get the local task arguments as a dictionary. The return value is in
        the form ``dict(my_param='my_value', ...)``
        """
        res = {}
        for (param_name, param_obj) in self._get_task_cls().get_params():
            attr = getattr(self.known_args, param_name)
            if attr:
                try:
                    res.update(((param_name, param_obj.parse(attr)),))
                except ValueError as err:
                    print("Error parsing JSON %s, value=%s" % (param_name, param_obj))
                    raise err
        return res

```

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [October 2, 2020, 2:47pm UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/8 "2020-10-02T14:47:09Z")

</div>

Apologies Jill for responding late. Managed to do the change in the luigi code. something new has popped up. Hopefully it makes sense to you.

“Error parsing JSON lib\_jar, value=\<luigi.parameter.ListParameter object at 0x7fd87f304dd0\>”

DEBUG:edx.analytics.tasks.launchers.local:Loading override configuration ‘override.cfg’…  
Error parsing JSON lib\_jar, value=\<luigi.parameter.ListParameter object at 0x7fd87f304dd0\>  
ERROR: Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 137, in \_get\_task\_kwargs  
raise err  
ValueError: No JSON object could be decoded  
ERROR:luigi-interface:Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 137, in \_get\_task\_kwargs  
raise err  
ValueError: No JSON object could be decoded  
Connection to localhost closed.

Regards,  
Neville

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [October 6, 2020, 3:54am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/9 "2020-10-06T03:54:23Z")

</div>

> [@nevilleonline](#):
>
> “Error parsing JSON lib\_jar, value=\<luigi.parameter.ListParameter object at 0x7fd87f304dd0\>”

Well that’s cool, it confirms that the issue is with parsing your `--lib-jar` parameter at least. Unfortunately, the `value` printed doesn’t tell us what’s been parsed from the command line like I’d hoped it would.

I can’t tell from your [previous post](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/6) (because Discourse insists on using “smart quotes”), but can you confirm that you’re only using plain double quotes around the strings in that list?

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [October 6, 2020, 7:04am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/10 "2020-10-06T07:04:24Z")

</div>

Thanks Jill, for your response. I have tried putting quotes like below.

–src [“hdfs://localhost:9000/data”]  
–lib-jar [“hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar”]

I tried deleting and then uploading the lib jar file in hadoop as well. But the error is the same.

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [October 7, 2020, 4:45am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/11 "2020-10-07T04:45:25Z")

</div>

> [@nevilleonline](#):
>
> –lib-jar [“hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar”]

I’ve found one of our client configurations which uses the following instead:

```auto
–lib-jar '"[\"hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar\"]"'

```

I’m sorry it’s so convoluted, but here’s an explanation, from the outside in:

- The single quotes around the whole thing wrap the value for the shell command.
- The double quotes around the array make the array string parseable as a JSON string.
- The double quotes around the `hdfs..` need to be escaped with a `\` to make them read as double quotes once the JSON string is read.

If that works, we can work with edX to get the analytics task documentation updated too.

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [October 8, 2020, 6:38am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/12 "2020-10-08T06:38:50Z")

</div>

Thanks Jill for your response. I still cant shake that error off. Error message is slightly different.

–lib-jar ‘"[“hdfs://localhost:9000/edx-analytics-pipeline/packages/edx-analytics-hadoop-util.jar”]"’ \

Same error shown below.

Error parsing JSON src, value=\<luigi.parameter.ListParameter object at 0x7fbac9a0cd50\>  
ERROR: Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 137, in \_get\_task\_kwargs  
raise err  
ValueError: No JSON object could be decoded  
ERROR:luigi-interface:Uncaught exception in luigi  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/retcodes.py”, line 74, in run\_with\_retcodes  
worker = luigi.interface.\_run(argv)[‘worker’]  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/interface.py”, line 248, in \_run  
return \_schedule\_and\_run([cp.get\_task\_obj()], worker\_scheduler\_factory)  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 116, in get\_task\_obj  
return self.\_get\_task\_cls()(\*\*self.\_get\_task\_kwargs())  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/cmdline\_parser.py”, line 137, in \_get\_task\_kwargs  
raise err  
ValueError: No JSON object could be decoded  
Connection to localhost closed.

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [October 8, 2020, 7:04am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/13 "2020-10-08T07:04:23Z")

</div>

> [@nevilleonline](#):
>
> Error parsing JSON src, value=\<luigi.parameter.ListParameter object at 0x7fbac9a0cd50\>

Ooh this is progress! It’s now complaining about the `--src` parameter, which It means that we did find the magic incantation for `--lib-jar`! Just have to do the same for the other list parameters.

```auto
--src '"[\"hdfs://localhost:9000/data\"]"' 

```

And maybe also:

```auto
--include '"[\"*tracking.log*.gz\"]"' 

```

Shell escaping is painful! The good news is, for all the other analytics task types, you can use an `override.cfg`, which has a much easier syntax, e.g. [luigi\_docker.cfg](https://github.com/edx/edx-analytics-pipeline/blob/ad7b86878f61b6a381c377ecd4ed2ac446089254/config/luigi_docker.cfg)

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [October 9, 2020, 9:54am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/14 "2020-10-09T09:54:18Z")

</div>

Thanks Jill. It worked. I no longer get the JSON error. But I am stuck with another error  
“Cannot overwrite a table with an empty result set.”

Not sure how to deeal with this. Do I need to delete the previous hadoop data and mysql data? Need to know which tables if you can and then I can try to re-run this task.

2020-10-09 18:48:06,588 ERROR 34358 [edx.analytics.tasks.common.mysql\_load] mysql\_load.py:393 - Cannot overwrite a table with an empty result set.  
2020-10-09 18:48:06,591 ERROR 34358 [luigi-interface] worker.py:213 - [pid 34358] Worker Worker(salt=367386724, workers=1, host=insights.millicenttechnologies.co.in, username=hadoop, pid=34358, sudo\_user=millicentr) failed AnswerDistributionToMySQLTaskWorkflow(database=reports, credentials=/edx/etc/edx-analytics-pipeline/output.json, name=2020-10-09T18\_44\_02Z, src=[“hdfs://localhost:9000/data”], dest=hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/2020-10-09T18\_44\_02Z/dest, include=[“_tracking.log_.gz”], manifest=hdfs://localhost:9000/tmp/pipeline-task-scheduler/AnswerDistributionWorkflow/2020-10-09T18\_44\_02Z/manifest.txt, answer\_metadata=None, base\_input\_format=org.edx.hadoop.input.ManifestTextInputFormat)  
Traceback (most recent call last):  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/worker.py”, line 194, in run  
new\_deps = self.\_run\_get\_new\_deps()  
File “/var/lib/analytics-tasks/analyticstack/venv/src/luigi/luigi/worker.py”, line 146, in \_run\_get\_new\_deps  
requires = task\_gen.send(next\_send)  
File “/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/edx/analytics/tasks/common/mysql\_load.py”, line 377, in run  
self.insert\_rows(cursor)  
File “/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/edx/analytics/tasks/common/mysql\_load.py”, line 332, in insert\_rows  
raise Exception(‘Cannot overwrite a table with an empty result set.’)  
Exception: Cannot overwrite a table with an empty result set.

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [October 11, 2020, 4:36am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/15 "2020-10-11T04:36:17Z")

</div>

> [@nevilleonline](#):
>
> “Cannot overwrite a table with an empty result set.”

That error comes from [this parameter](https://github.com/edx/edx-analytics-pipeline/blob/8881afd0af558d16b463c39b1febf95252874354/edx/analytics/tasks/common/mysql_load.py#L61): `allow_empty_insert = False`

A couple of tasks make this configurable ([InsertToMysqlAllVideoTask](https://github.com/edx/edx-analytics-pipeline/blob/3e929815e1aab288b2a6409a1ff29ac60f584a56/edx/analytics/tasks/insights/video.py#L892-L897), [ModuleEngagement](https://github.com/edx/edx-analytics-pipeline/blob/e63292012af075f37b42876c38a155e57291db36/edx/analytics/tasks/insights/module_engagement.py#L287-L290)), but unfortunately, AnswerDistributionWorkflow isn’t one of them.

To work around it, you can change [this line](https://github.com/edx/edx-analytics-pipeline/blob/8881afd0af558d16b463c39b1febf95252874354/edx/analytics/tasks/common/mysql_load.py#L61) to be `allow_empty_insert = True`.

---

<div class="post-metadata">

**Author:** ![nevilleonline](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/nevilleonline/32/1259_2.png) [@nevilleonline](https://discuss.openedx.org/u/nevilleonline)\
**Post date:** [October 12, 2020, 6:12am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/16 "2020-10-12T06:12:21Z")

</div>

Thanks Jill once again. Got it to work finally. I changed it False to True in 2 files.

edx-analytics-pipeline/edx/analytics/tasks/common/mysql\_load.py

allow\_empty\_insert = False

to

allow\_empty\_insert = True

/var/lib/analytics-tasks/analyticstack/venv/local/lib/python2.7/site-packages/edx/analytics/tasks/common/mysql\_load.py

allow\_empty\_insert = False

to

allow\_empty\_insert = True

Finally a good result. Not sure how if it will affect the other tasks.

===== Luigi Execution Summary =====

Scheduled 7 tasks of which:

- 2 present dependencies were encountered:
  - 1 ExternalURL(url=/edx/etc/edx-analytics-pipeline/output.json)
  - 1 PathSetTask(…)

- 5 ran successfully:
  - 1 AnswerDistributionOneFilePerCourseTask(…)
  - 1 AnswerDistributionPerCourse(…)
  - 1 AnswerDistributionToMySQLTaskWorkflow(…)
  - 1 AnswerDistributionWorkflow(…)
  - 1 ProblemCheckEvent(…)

This progress looks 🙂 because there were no failed tasks or missing external dependencies

===== Luigi Execution Summary =====

---

<div class="post-metadata">

**Author:** ![jill](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/jill/32/3188_2.png) [@jill](https://discuss.openedx.org/u/jill)\
**Post date:** [October 12, 2020, 7:10am UTC](https://discuss.openedx.org/t/error-running-answerdistributionworkflow/2687/17 "2020-10-12T07:10:05Z")

</div>

Hallelujah! Sorry it was such a messy journey for you @nevilleonline. Could you mark this post as Solved, so others can have hope of solving this issue too?

A tip for the other analytics tasks: it’s best to create and use an `override.cfg` file to specify the parameters for those tasks which remain the same for every run. For example, to set [`allow_empty_insert` for the aforementioned Video task](https://github.com/edx/edx-analytics-pipeline/blob/3e929815e1aab288b2a6409a1ff29ac60f584a56/edx/analytics/tasks/insights/video.py#L892-L897), your `override.cfg` file needs a stanza like the one below. You can see the section and parameter name in the parameter’s `config_path`, which in this case is `{'section': 'videos', 'name': 'allow_empty_insert'}`

```auto
[videos]
allow_empty_insert = true

```

Similarly, your array parameters get easier to specify in the `.cfg` file, and don’t require all that gnarly escaping:

```auto
[event-logs]
pattern = [".*tracking.log-(?P<date>[0-9]+).*"]
source = ["hdfs://localhost:9000/data/logs/tracking/"]

```