Attempting to setup Analytics Pipeline on devstack master

Development
1

Hi guys

I have a setup of the devstack make project on my Ubuntu server, and I’m trying to setup a pipeline to stream course data into my Insights service for analytics work. However, it seems like the analytics pipeline services is missing/removed in the master devstack.
So far I have tried to implement the analytic pipeline github to no avail. (They uses vagrant and majority of instructions provided by the analytic pipeline github README doesnt seem applicable to my setup in anyway)

I noticed that for some of the older released devstack branch like open-release/hawton
analytics_pipeline can infact be implemented from the same MAKE project by simple make pull.analytics_pipeline

The same equivalent command make

Is setting up a second devstack on hawthorn’s version, the only way for me to get this analytics pipeline right? Pls help!

Thanks alot!
Wayne

2

Heres what it looks like when I tried to make dev.pull.analytics_pipeline

3

Hi @T_GW,

Unfortunately the analytics pipeline is not really maintained or supported anymore. I would say however that if your instance is on the smaller side, that’s probably a good thing as the analytics pipeline was very expensive to run on AWS and provided fairly minimal stats, helpful if you run thousands of huge courses, but not as helpful if you want a deep dive into analytics of smaller courses.

We are hoping that with the formation of tCRIL, there will be an organized effort to build a new community-developed analytics suite for Open edX, but I don’t believe such an initiative has started yet.

Some alternatives you can consider in the meantime are:

1 Like
4

Which one might be the best for my application? If let say I’m just dealing with the deep analytics of 5 courses and a few hundred students.

The cairn plugin appears to be quite an improved equivalent to the Insights application which I’m trying to deploy and its easy to install with the tutor installation. But It seems like the tutor plugin cairn isn’t free? and that I’m required to purchase the Wizard edition of the Tutor installation?

Do you recommend cairn the most for beginners like me?

5

I may also try the second service called appsembler/figures to stream data from LMS into my own alternative platform to perform analytics, however majority of appsembler/figures setup guides appear to be based on the old vagrant setup. Do you recommend that I use the vagrant devstack setup(GitHub - openedx/edx-platform: The Open edX LMS & Studio, powering education sites around the world!) rather than the docker base environment devstack(GitHub - openedx/devstack: Get up and running quickly to develop or extend Open edX services) to setup such LMS?

Just wondering, is the below setup:

applicable for my application? Currently my server is running an LMS instance of openedx/devstack/master

The requirements implies that the instructions accept all Open edx releases for as long as it has Python 2.7. Yet at the top of the page, there is a notice highlighting that “these instructions are for Ficus and Ginkgo” only

6

If you are willing to implement your own custom solution, you can replace the Segment IO hooks with Rudderstack hooks inside the code and run your own Rudderstack cluster.

I am suggesting Rudderstack as it is a mostly drop-in replacement to Segment and you can use it for free if you deploy your own cluster.

7

I feel that the appassember/Figures might be very relevant to what I want now without much modification to the original open edx source code. Was wondering if anyone of you may have any idea how to go about doing this with an existing devstack LMS or any other recommended.

8

That may be possible in the future when I’m used to the overall of open edx. At this point of time, I’m really just trying to setup something workable and best to have the procedure as simple as possible.
I feel that the appassembler/Figures may have everything I need, real-time data streaming to my own custom solution (based on things i’m familiar with)…