Open edX (Koa.master) Native install with OpenJdk

Site Operators Site Operations Help
1

So i have been experimenting with openJdk instead of oraclejdk(beacause of licence)
System is Ubuntu20.4 VirtualMachine (default installation is working fine by the way after mysql 5.32 to 5.33 change for Koa.master Tag)

I am following these docs :

1.https://openedx.atlassian.net/wiki/spaces/OpenOPS/pages/1969455764/Koa+Native+Open+edX+platform+Ubuntu+20.04+64+bit+Installation

2.https://openedx.atlassian.net/wiki/spaces/OpenOPS/pages/153813109/How+to+Override+Default+Configuration+Passwords+and+Verify+Exposed+Services#HowtoOverrideDefaultConfigurationPasswordsandVerifyExposedServices-suk=ff808081465a4dee0146bf8bcdc80053

I have changed the following files to change oraclejdk to openjdk

edx/app/edx_ansible/edx_ansible/playbooks/roles/oraclejdk

Below are my changes :

1. oraclejdk/default/main.yml

---

#ORACLEJDK_VERSION: "8u131"
#oraclejdk_base: "jdk1.8.0_131"
#oraclejdk_build: "b11"
#oraclejdk_platform: "linux"
#oraclejdk_arch: "x64"
#oraclejdk_file: "jdk-{{ ORACLEJDK_VERSION }}-{{ oraclejdk_platform }}-{{ oraclejdk_arch }}.tar.gz"

#oraclejdk_url: "http://download.oracle.com/otn-pub/java/jdk/{{ ORACLEJDK_VERSION }}-{{ oraclejdk_build }}/d54c1d3a095b4ff2b6607d096fa80163/{{ oraclejdk_file }}"

#oraclejdk_link: "/usr/lib/jvm/java-8-oracle"

#oraclejdk_debian_pkgs:
#  - curl
  
  
  
  
openjdk_default: 8
openjdk_pkg_state: latest
oraclejdk_base: 'java-{{openjdk_default}}-openjdk-amd64'
openjdk_version:
  - '{{ openjdk_default }}'

openjdk_ppa: 'ppa:openjdk-r/ppa'

oraclejdk_link: "/usr/lib/jvm/java-8-oracle"

oraclejdk_debian_pkgs:
  - openjdk-{{openjdk_default}}-jre-headless 
  - openjdk-{{openjdk_default}}-jdk-headless 
#  - jre
#  - jdk

2. oraclejdk/task/main.yml

---
# oraclejdk
#
# Dependencies:
#
#   * common
#
# Example play:
#
#   roles:
#   - common
#   - oraclejdk

- name: Install debian needed pkgs
  apt:
	name: "{{ item }}"
  with_items: "{{ oraclejdk_debian_pkgs }}"

#- name: Download Oracle Java
#  get_url:
#    url: "{{ oraclejdk_url }}"
#    headers: 'CookiCreate symlink expected by elasticsearche:oraclelicense=accept-securebackup-cookie'
#    dest: "/var/tmp/{{ oraclejdk_file }}"

- name: Create jvm dir
  file:
	path: /usr/lib/jvm
	state: directory
	owner: root
	group: root

#- name: Untar Oracle Java
#  unarchive:
#    src: "/var/tmp/{{ oraclejdk_file }}"
#    dest: "/usr/lib/jvm"
#    copy: no

- name: Add the openjdk ppa. To have openjdk 8/9 on precise and trusty
  apt_repository: repo='{{ openjdk_ppa }}' update_cache=yes

- name: install OpenJDK JRE 
  apt: pkg={{ item[1] }} state={{ openjdk_pkg_state }} update_cache=yes cache_valid_time=3600
  with_nested:
	- '{{ openjdk_version }}'
	- '{{ oraclejdk_debian_pkgs }}'

- name: Set the default OpenJDK
  shell: update-java-alternatives -s java-1.{{ item }}.0-openjdk-amd64
  with_items: '{{ openjdk_default }}'


- name: Create symlink expected by elasticsearch
  file:
	src: "/usr/lib/jvm/{{ oraclejdk_base }}"
	dest: "{{ oraclejdk_link }}"
	state: link
	force: yes

- name: Update alternatives java
  alternatives:
	name: "{{ item }}"
	link: "/usr/bin/{{ item }}"
	path: "/usr/lib/jvm/{{ oraclejdk_base }}/bin/{{ item }}"
  with_items:
	- java
	- javac
	#- javaws
	- jarsigner

- name: Add JAVA_HOME for Oracle Java
  template:
	src: "java.sh.j2"
	dest: "/etc/profile.d/java.sh"
	owner: root
	group: root
	mode: "0755"

3. Error i am getting is

TASK [forum : install comments service bundle] *************************************************************************************
changed: [localhost]

TASK [forum : initialize elasticsearch] ********************************************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["/edx/app/forum/cs_comments_service/bin/rake", "search:initialize"], "delta": "0:00:02.073510", "end": "2021-02-11 18:14:14.883624", "msg": "non-zero return code", "rc": 1, "start": "2021-02-11 18:14:12.810114", "stderr": "/edx/app/forum/cs_comments_service/lib/tasks/flags.rake:6: warning: already initialized constant ROOT\n/edx/app/forum/cs_comments_service/lib/tasks/kpis.rake:7: warning: previous definition of ROOT was here\n/edx/app/forum/cs_comments_service/lib/tasks/deep_search.rake:7: warning: already initialized constant ROOT\n/edx/app/forum/cs_comments_service/lib/tasks/flags.rake:6: warning: previous definition of ROOT was here\n/edx/app/forum/cs_comments_service/lib/tasks/db.rake:28: warning: already initialized constant COURSE_ID\n/edx/app/forum/cs_comments_service/models/constants.rb:2: warning: previous definition of COURSE_ID was here\nrake aborted!\nElasticsearch::Transport::Transport::Errors::InternalServerError: [500] {\"error\":\"ClassCastException[java.lang.String cannot be cast to java.util.Map]\",\"status\":500}\n/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error'\n/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/transport/base.rb:346:in `perform_request'\n/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/transport/http/faraday.rb:37:in `perform_request'\n/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/client.rb:176:in `perform_request'\n/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-api-7.8.0/lib/elasticsearch/api/namespace/common.rb:38:in `perform_request'\n/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-api-7.8.0/lib/elasticsearch/api/actions/indices/create.rb:48:in `create'\n/edx/app/forum/cs_comments_service/lib/task_helpers.rb:92:in `block in create_indices'\n/edx/app/forum/cs_comments_service/lib/task_helpers.rb:89:in `each'\n/edx/app/forum/cs_comments_service/lib/task_helpers.rb:89:in `create_indices'\n/edx/app/forum/cs_comments_service/lib/task_helpers.rb:198:in `initialize_indices'\n/edx/app/forum/cs_comments_service/lib/tasks/search.rake:30:in `block (2 levels) in <top (required)>'\n/edx/app/forum/.gem/ruby/2.5.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'\nTasks: TOP => search:initialize\n(See full trace by running task with --trace)", "stderr_lines": ["/edx/app/forum/cs_comments_service/lib/tasks/flags.rake:6: warning: already initialized constant ROOT", "/edx/app/forum/cs_comments_service/lib/tasks/kpis.rake:7: warning: previous definition of ROOT was here", "/edx/app/forum/cs_comments_service/lib/tasks/deep_search.rake:7: warning: already initialized constant ROOT", "/edx/app/forum/cs_comments_service/lib/tasks/flags.rake:6: warning: previous definition of ROOT was here", "/edx/app/forum/cs_comments_service/lib/tasks/db.rake:28: warning: already initialized constant COURSE_ID", "/edx/app/forum/cs_comments_service/models/constants.rb:2: warning: previous definition of COURSE_ID was here", "rake aborted!", "Elasticsearch::Transport::Transport::Errors::InternalServerError: [500] {\"error\":\"ClassCastException[java.lang.String cannot be cast to java.util.Map]\",\"status\":500}", "/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error'", "/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/transport/base.rb:346:in `perform_request'", "/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/transport/http/faraday.rb:37:in `perform_request'", "/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-transport-7.8.0/lib/elasticsearch/transport/client.rb:176:in `perform_request'", "/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-api-7.8.0/lib/elasticsearch/api/namespace/common.rb:38:in `perform_request'", "/edx/app/forum/.gem/ruby/2.5.0/gems/elasticsearch-api-7.8.0/lib/elasticsearch/api/actions/indices/create.rb:48:in `create'", "/edx/app/forum/cs_comments_service/lib/task_helpers.rb:92:in `block in create_indices'", "/edx/app/forum/cs_comments_service/lib/task_helpers.rb:89:in `each'", "/edx/app/forum/cs_comments_service/lib/task_helpers.rb:89:in `create_indices'", "/edx/app/forum/cs_comments_service/lib/task_helpers.rb:198:in `initialize_indices'", "/edx/app/forum/cs_comments_service/lib/tasks/search.rake:30:in `block (2 levels) in <top (required)>'", "/edx/app/forum/.gem/ruby/2.5.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'", "Tasks: TOP => search:initialize", "(See full trace by running task with --trace)"], "stdout": "W, [2021-02-11T18:14:14.802735 #941223]  WARN -- : Overwriting existing field _id in class User.\nW, [2021-02-11T18:14:14.823858 #941223]  WARN -- : MONGODB | Unsupported client option 'max_retries'. It will be ignored.\nW, [2021-02-11T18:14:14.823904 #941223]  WARN -- : MONGODB | Unsupported client option 'retry_interval'. It will be ignored.\nW, [2021-02-11T18:14:14.823920 #941223]  WARN -- : MONGODB | Unsupported client option 'timeout'. It will be ignored.", "stdout_lines": ["W, [2021-02-11T18:14:14.802735 #941223]  WARN -- : Overwriting existing field _id in class User.", "W, [2021-02-11T18:14:14.823858 #941223]  WARN -- : MONGODB | Unsupported client option 'max_retries'. It will be ignored.", "W, [2021-02-11T18:14:14.823904 #941223]  WARN -- : MONGODB | Unsupported client option 'retry_interval'. It will be ignored.", "W, [2021-02-11T18:14:14.823920 #941223]  WARN -- : MONGODB | Unsupported client option 'timeout'. It will be ignored."]}

Please let me know if more information is required
Please let me know your ideas, how to go on about it

2

@vicmathur91, welcome to the forum!

So, I’ve done something like this in the past, but haven’t actually tried it of late, particularly not since the Elasticsearch 7 upgrade. If we can get it working, though, I think it would be a welcome change. Please keep us posted on your progress, and you’re welcome to join the #wg-build-test-release channel where we have discussed this issue at least once, already.

Cheers!

3

@arbrandes Did that change worked ??
Because i am still getting error at “initialize elasticsearch” search when forum module is installing

I have tried with your changes as well (installing elastic search with openjdk) but still same error is coming

4

It did, at the time. But I’m thinking that something changed between then and now. We’d have to figure out what, and then see how to fix it.

5

@arbrandes

My ansible script ran through without any error for now
I have changed the oraclejdk (to openjdk) changes in below files paths :

  1. /var/tmp/configuration/playbooks/roles/oraclejdk
  2. /edx/app/edx_ansible/edx_ansible/playbooks/roles/oraclejdk
  3. /edx/app/edx_ansible/edx_ansible/playbooks/roles/edx_ansible/tasks/deploy.yml
  4. /var/tmp/configuration/playbooks/roles/edx_ansible/task/deploy.yml

for 1 and 2 i put the openjdk changes as respect to my original post

for 3 and 4 point i commented out

name: Git checkout edx_ansible repo into edx_ansible_code_dir

reason for change in 3&4 is : as checking out latest config code giving me error as below because of my changes in /edx/…/playbook/role/oraclejdk path :

msg: Local modifications exist in repository (force=no).

Can you please guide me how i can check if all components are working with openjdk and if there is any oraclejdk component is installed ( to be searched inside any specific environment or user )???

also for root user

$java -showversion -verbose 2>&1 | head -1
[Opened /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar]

$java -version
openjdk version “1.8.0_282”
OpenJDK Runtime Environment (build 1.8.0_282-8u282-b08-0ubuntu1~20.04-b08)
OpenJDK 64-Bit Server VM (build 25.282-b08, mixed mode)

6

As far as I know (somebody correct me if I’m wrong), it’s just Elasticsearch. If it’s working… Then you’re good!

Is the Elasticsearch initialization working for the forum, now?

7

@arbrandes Yes all services are restarting and running

$root@vaibhav-VirtualBox:~# /edx/bin/supervisorctl status
analytics_api                    RUNNING   pid 65310, uptime 1:06:56
certs                            RUNNING   pid 117194, uptime 0:45:32
cms                              RUNNING   pid 60574, uptime 1:10:24
discovery                        RUNNING   pid 114283, uptime 0:47:19
edxapp_worker:cms_default_1      RUNNING   pid 54609, uptime 1:18:44
edxapp_worker:cms_high_1         RUNNING   pid 54613, uptime 1:18:43
edxapp_worker:lms_default_1      RUNNING   pid 54619, uptime 1:18:42
edxapp_worker:lms_high_1         RUNNING   pid 54625, uptime 1:18:40
edxapp_worker:lms_high_mem_1     RUNNING   pid 54629, uptime 1:18:39
forum                            RUNNING   pid 121935, uptime 0:44:44
insights                         RUNNING   pid 70935, uptime 1:01:15
lms                              RUNNING   pid 61174, uptime 1:09:30
xqueue                           RUNNING   pid 116257, uptime 0:46:20
xqueue_consumer                  RUNNING   pid 116291, uptime 0:46:18