Upgrade Hadoop version in ironwood (Analytics)

Pls, help me, How to upgrade Apache Hadoop 2.7.2 version to the latest Hadoop version for ironwood. master Open edx? My server was created on the Azure portal and uses Ubuntu 16.04. Now my security team informs me of a Malware attack on my server and it depends on the Hadoop version coz of the Hadoop version is out of date and these issues can fix only with the upgrade Hadoop version.
So I want to know How can I upgrade my Hadoop version, it’s important for me coz many students are using my site. I didn’t find any notes of upgrading Hadoop for ironwood in openedx. Pls help me and give me advice. Thanks…

Hello,

I wasn’t able to upgrade the version myself, I tried but it did not work, mostly because I was not aware about hadoop works and how it’s setup is done.

I just kept it same as 2.7.2.

Do u want to see my data files. Pls help me that is red alert for my site. I want to know /edx/var/log/tracking/tracking.log is related or not for this error issue coz of my tracking.log is not updated for many months. How can i fix tracking.log for up to date?

You have to put in fresh logs from lms machines to insights machine.

If you run your tasks on old files it’ll not fetch you any real data.

How can i put fresh logs. Pls documentation

You can refer to scripts here https://openedx.atlassian.net/wiki/spaces/OpenOPS/pages/43385371/edX+Analytics+Installation

Also you just need to copy log files from your lms application machine to your insights machine

It’ll mostly be in /edx/var/log/tracking/ this path and you’ll also be putting it in the same path in insights machine.

You can refer to script in the link it’ll show the steps, only extra thing you need to do is copy logs from your lms machine to your insights machine.

2 Likes

Thanks for your sharing sir. My server is a separate (divided) app server and DB server. All data (mysql) are stored in the DB server. So its means the DB server is an insight machine, right? Do I need to put logs files of the App server (LMS) to the DB server (insight machine). Is correct sir?

1 Like

It doesn’t clear up, but yes you need to put logs where insights is running.

Ok sir, I will try and check this issue. Thanks. I will back and ask if not ok. Pls follow up