Unable to get Learner Analytics up and running in Insights

Site Operators Site Operations Help
1

Hello again!

We are trying to get Learner Analytics up and running, but we are experimenting some problems.

I believe there is something with ElasticSearch, because there is data in mysql module_engagement tables (and in hive too) but analytics data API cannot find the proper indexes in ElasticSearch. Calling curl -XGET ‘http://localhost:9200/roster/’ returns {“error”:“IndexMissingException[[roster] missing]”,“status”:404} (see screenshot below). Subtask ModuleEngagementRosterIndexTask didn’t run because was not granted run permission by the scheduler (see another screenshot) and I believe this subtask is responsible of creating the indexes

Pipeline log here.
Analytics-API log here

I’ve checked alias in override.cfg and analytics.yml, elasticsearch port, etc… but I believe they are correct (please see the files attached)

There is one more thing: in LMS we have ES version 1.7.3 while in Insight we have 1.5.2 (I’ve attached 2 screenshots of config). I don’t know if this can affect

Thanks very much for any help, anything to check, etc… Any help is appreciated

Kind regards

Insights ES version
LMS ES version

insights.yml (2.3 KB) nohup.out (518.2 KB) override.cfg (3.1 KB)

image (1)
image (1)1794×1490 281 KB

image
image2880×894 213 KB

2

Hi @Yago, looks like you’ve made it really far through the analytics setup process, congratulations!

The ModuleEngagement tasks expect that you’ve already run the Enrollment task successfully. From the Learner Engagement task docs:

Another prerequisite before running the module engagement workflow below is to have run enrollment first. It is assumed that the course_enrollment directory under warehouse_path has been populated by running enrollment with a TO_DATE matching that used for the module engagement workflow (i.e. today).

That will resolve that first error you’re seeing:

ExternalURL(url=hdfs://localhost:9000/edx-analytics-pipeline/warehouse/course_enrollment/dt=2020-06-21/)

There is one more thing: in LMS we have ES version 1.7.3 while in Insight we have 1.5.2 (I’ve attached 2 screenshots of config).

This is totally fine, there’s no ES data from the LMS used in the Insights process. Just MySQL and tracking logs.

I haven’t checked your config files yet, but can do that if you run into more issues. Hope this helps!

1 Like
3

Thanks @jill , it worked!

Its frequently hard for me to understand the logs, but “re-reading” the issue it is fairly clear.

We are just finishing analytics setup process, Geolocation is the last task we have left, hope it works fine :crossed_fingers:

Thank you very much for your support
Have a nice day!

1 Like
4

Congratulations! :tada:

5

Hi @Yago, has this problem been solved, and how? Thanks in advance

6

Hi, currently we have Analytics running, and this issue was solved with @jill 's answers, so yes, this problem was solved with the solution proposed

1 Like
7

@Yago I have encountered this problem when run task ModuleEngagementWorkflowTask, Do you have any advice? Thanks

image
image1773×364 85.1 KB

8

I’m afraid not… It is not the same error I have posted. You can try to post an issue regarding this problem and maybe someone has got the same problem.

Also check in Slack

Good luck!

9

@Yago Thank you so much, have a nice day!!