Unable to get Learner Analytics up and running in Insights

Hello again!

We are trying to get Learner Analytics up and running, but we are experimenting some problems.

I believe there is something with ElasticSearch, because there is data in mysql module_engagement tables (and in hive too) but analytics data API cannot find the proper indexes in ElasticSearch. Calling curl -XGET ‘http://localhost:9200/roster/’ returns {“error”:“IndexMissingException[[roster] missing]”,“status”:404} (see screenshot below). Subtask ModuleEngagementRosterIndexTask didn’t run because was not granted run permission by the scheduler (see another screenshot) and I believe this subtask is responsible of creating the indexes

Pipeline log here.
Analytics-API log here

I’ve checked alias in override.cfg and analytics.yml, elasticsearch port, etc… but I believe they are correct (please see the files attached)

There is one more thing: in LMS we have ES version 1.7.3 while in Insight we have 1.5.2 (I’ve attached 2 screenshots of config). I don’t know if this can affect

Thanks very much for any help, anything to check, etc… Any help is appreciated

Kind regards

Insights ES version
LMS ES version

insights.yml (2.3 KB) nohup.out (518.2 KB) override.cfg (3.1 KB)

Hi @Yago, looks like you’ve made it really far through the analytics setup process, congratulations!

The ModuleEngagement tasks expect that you’ve already run the Enrollment task successfully. From the Learner Engagement task docs:

Another prerequisite before running the module engagement workflow below is to have run enrollment first. It is assumed that the course_enrollment directory under warehouse_path has been populated by running enrollment with a TO_DATE matching that used for the module engagement workflow (i.e. today).

That will resolve that first error you’re seeing:

ExternalURL(url=hdfs://localhost:9000/edx-analytics-pipeline/warehouse/course_enrollment/dt=2020-06-21/)

There is one more thing: in LMS we have ES version 1.7.3 while in Insight we have 1.5.2 (I’ve attached 2 screenshots of config).

This is totally fine, there’s no ES data from the LMS used in the Insights process. Just MySQL and tracking logs.

I haven’t checked your config files yet, but can do that if you run into more issues. Hope this helps!

1 Like

Thanks @jill , it worked!

Its frequently hard for me to understand the logs, but “re-reading” the issue it is fairly clear.

We are just finishing analytics setup process, Geolocation is the last task we have left, hope it works fine :crossed_fingers:

Thank you very much for your support
Have a nice day!

1 Like

Congratulations! :tada: