Run Open edX: Server error, given troubleshooting details

Hi,
I’ve installed openedX natively, using the koa.master branch on a VirtualBox VM with 3GB of RAM and 60GB of VHD.
When starting all services using sudo /edx/bin/supervisorctl start all (and checking all 16 of them are RUNNING with sudo /edx/bin/supervisorctl status), I used a browser on the host machine to access the LMS.

I’m met with the “Server error” page:
Imgur
Following the troubleshooting guide, I did the following steps:

wis20211@wis20211:~$ time sudo service nginx status
● nginx.service - nginx - high performance web server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-05-10 09:54:24 UTC; 26min ago
       Docs: https://nginx.org/en/docs/
    Process: 675 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS)
   Main PID: 828 (nginx)
      Tasks: 5 (limit: 3404)
     Memory: 1.6M
     CGroup: /system.slice/nginx.service
             ├─828 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
             ├─829 nginx: worker process
             ├─830 nginx: worker process
             ├─831 nginx: worker process
             └─832 nginx: worker process

May 10 09:54:12 wis20211 systemd[1]: Starting nginx - high performance web server...
May 10 09:54:24 wis20211 systemd[1]: nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: Operation not permitted
May 10 09:54:24 wis20211 systemd[1]: Started nginx - high performance web server.

real    0m11.738s
user    0m0.016s
sys     0m0.058s
  1. Then sudo service nginx status (which returned nothing);
wis20211@wis20211:~$ sudo /edx/bin/supervisorctl status edxapp:
edxapp: ERROR (no such group)
wis20211@wis20211:~$ sudo service nginx stop
wis20211@wis20211:~$ sudo service supervisor stop
wis20211@wis20211:~$ sudo service supervisor.devpi stop
Failed to stop supervisor.devpi.service: Unit supervisor.devpi.service not loaded.
wis20211@wis20211:~$ sudo pkill -u www-data
wis20211@wis20211:~$ sudo service nginx start
wis20211@wis20211:~$ sudo service supervisor start
Job for supervisor.service failed because the control process exited with error code.
See "systemctl status supervisor.service" and "journalctl -xe" for details.
wis20211@wis20211:~$ sudo service supervisor.devpi start
Failed to start supervisor.devpi.service: Unit supervisor.devpi.service not found.

I’ll stop here.
Thank you for the help, and please tell me if you need any specific logs and/or additional details.

Hey @tapicationer ,

Troubleshooting guide is not updated. By running this sudo /edx/bin/supervisorctl status command you will get status of Open edX services

edxapp is not separated with lms and cms services in supervisor.

For more details, you need to look at request logs and errors from the LMS. Here is the path of LMS log file: /edx/var/log/lms/edx.log

One suggestion, 3GB RAM might not be work, try to increase Memory to 8 GB.

1 Like

Hi @amit,
Thanks for replying.
I’ve rebooted the VM, started all services to ensure a “fresh” session, then visited my site.
I’ve examined /edx/var/log/lms/edx.log, /edx/var/log/cms/edx.log and they both contain no new information.
/edx/var/log/nginx/access.log shows that I did make requests to the server from the host machine:

1. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:51:54 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.005 "-" "python-requests/2.22.0"

2. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:51:59 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.008 "-" "python-requests/2.22.0"

3. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:04 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.006 "-" "python-requests/2.22.0"

4. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:09 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.005 "-" "python-requests/2.22.0"

5. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:14 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.005 "-" "python-requests/2.22.0"

6. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:19 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.007 "-" "python-requests/2.22.0"

7. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:26 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 1.507 "-" "python-requests/2.22.0"

8. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:31 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.007 "-" "python-requests/2.22.0"

9. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:37 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 1.048 "-" "python-requests/2.22.0"

10. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:43 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 1.472 "-" "python-requests/2.22.0"

11. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:52:48 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.155 "-" "python-requests/2.22.0"

12. - - 192.168.56.1 - - - [10/May/2021:10:52:53 +0000] "GET / HTTP/1.1" 504 1448 60.103 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56"

13. - - 192.168.56.1 - - - [10/May/2021:10:53:13 +0000] "GET /favicon.ico HTTP/1.1" 200 30237 0.000 "http://192.168.56.5/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36 Edg/90.0.818.56"

14. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:53:29 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 23.757 "-" "python-requests/2.22.0"

15. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:53:36 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.005 "-" "python-requests/2.22.0"

16. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:53:42 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.479 "-" "python-requests/2.22.0"

17. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:53:47 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.055 "-" "python-requests/2.22.0"

18. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:53:52 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.217 "-" "python-requests/2.22.0"

19. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:53:58 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 1.296 "-" "python-requests/2.22.0"

20. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:04 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.264 "-" "python-requests/2.22.0"

21. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:09 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.294 "-" "python-requests/2.22.0"

22. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:14 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.379 "-" "python-requests/2.22.0"

23. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:19 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.207 "-" "python-requests/2.22.0"

24. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:25 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.156 "-" "python-requests/2.22.0"

25. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:32 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 2.349 "-" "python-requests/2.22.0"

26. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:38 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.798 "-" "python-requests/2.22.0"

27. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:44 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.785 "-" "python-requests/2.22.0"

28. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:50 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 1.523 "-" "python-requests/2.22.0"

29. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:54:56 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.625 "-" "python-requests/2.22.0"

30. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:01 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.458 "-" "python-requests/2.22.0"

31. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:07 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.505 "-" "python-requests/2.22.0"

32. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:12 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.336 "-" "python-requests/2.22.0"

33. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:18 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.684 "-" "python-requests/2.22.0"

34. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:23 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.397 "-" "python-requests/2.22.0"

35. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:29 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.735 "-" "python-requests/2.22.0"

36. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:35 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.771 "-" "python-requests/2.22.0"

37. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:40 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.572 "-" "python-requests/2.22.0"

38. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:46 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.370 "-" "python-requests/2.22.0"

39. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:51 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.339 "-" "python-requests/2.22.0"

40. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:55:57 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.517 "-" "python-requests/2.22.0"

41. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:56:02 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 200 60 0.517 "-" "python-requests/2.22.0"

42. - - 127.0.0.1 - zNH3OfZXZQSxk7DGy58DzJ0QESJ8wSBxkjf - [10/May/2021:10:57:07 +0000] "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1" 504 160 60.060 "-" "python-requests/2.22.0"

While /edx/var/log/nginx/error.log shows a timeout:

2021/05/10 10:52:53 [error] 846#846: *20 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.56.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "192.168.56.5"
2021/05/10 10:57:07 [error] 848#848: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: , request: "GET /xqueue/get_queuelen/?queue_name=certificates HTTP/1.1", upstream: "http://127.0.0.1:8040/xqueue/get_queuelen/?queue_name=certificates", host: "localhost:18040"

Through recent searching, I see that an user has the same timeout message here, however they are not running on the koa.master branch and do not have all the same errors.
I hope this provides needed information to further investigate.
In the meantime, I’ll try to allocate more RAM and restart the services (albeit reluctantly, I’m doing more of a class Web project than establishing a full-fledged edX platform; and my host machine has 8GB RAM).