Tutor local do init failing : ModuleNotFoundError: No module named 'lms.envs.tutor.production'

igmar · February 23, 2023, 1:18pm

Standard tutur install bundled binary, compiled on RHEL8, images are all loaded in the local docker CE instance. Doing a local run / init of tutor gives me :

ModuleNotFoundError: No module named ‘lms.envs.tutor.production’

2023/02/23 13:14:05 Ready: tcp://mysql:3306.
2023/02/23 13:14:05 Ready: tcp://mongodb:27017.
Loading settings lms.envs.tutor.production
Traceback (most recent call last):
File “./manage.py”, line 102, in
startup = importlib.import_module(edx_args.startup)
File “/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/init.py”, line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1014, in _gcd_import
File “”, line 991, in _find_and_load
File “”, line 975, in _find_and_load_unlocked
File “”, line 671, in _load_unlocked
File “”, line 843, in exec_module
File “”, line 219, in _call_with_frames_removed
File “/openedx/edx-platform/lms/startup.py”, line 10, in
settings.INSTALLED_APPS # pylint: disable=pointless-statement
File “/openedx/venv/lib/python3.8/site-packages/django/conf/init.py”, line 82, in getattr
self._setup(name)
File “/openedx/venv/lib/python3.8/site-packages/django/conf/init.py”, line 69, in _setup
self._wrapped = Settings(settings_module)
File “/openedx/venv/lib/python3.8/site-packages/django/conf/init.py”, line 170, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File “/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/init.py”, line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1014, in _gcd_import
File “”, line 991, in _find_and_load
File “”, line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named ‘lms.envs.tutor.production’

The config is generated with

/usr/local/bin/tutor config save --interactive

Any ideas on the cause / how to solve this ?

ghassan · February 23, 2023, 5:15pm

It looks may be from the logs, its encoding/decoding issue something may be to do with utf-8. hence the the single and double colon in the logs are not correct. instead of ” it should be ". and instead of ’ it should be '.

Can you just check the template files to confirm it’s based on the correct format.

igmar · February 23, 2023, 5:39pm

You mean the files on the docker host, right ? The containers are run as-is, I didn’t touch them.
I’ll check the locale on the shell, that might be the issue. What is expected there ? UTF8 ?

ghassan · February 23, 2023, 5:43pm

The .yaml files and other .py essetinaly files in directory tutor config prinroot of which config.yml file and env directory.

I think UTF8 shall work, but might be other coding that would work as well.

igmar · February 23, 2023, 6:05pm

Looks all good to me. It gets created in the Dockerfile, and it should get the data from ./env/apps/openedx/settings

I check all the .py files in there, all looks good to me. LANG is set to en_US.UTF-8, so that is also pretty normal. Any idea on how to start debugging this ?

ghassan · February 23, 2023, 6:16pm

did you check this file in particular

cat `tutor config printroot`/env/local/docker-compose.yml

can you check the value of DJANGO_SETTINGS_MODULE in that file it should be (cms.envs.tutor.production or/and lma.envs.tutor.production) depending on the service . In my case there was no qoutation around it

igmar · February 23, 2023, 6:32pm

Looks all good :

lms:
    image: docker.io/overhangio/openedx:15.3.0
    environment:
      SERVICE_VARIANT: lms
      DJANGO_SETTINGS_MODULE: lms.envs.tutor.production
      UWSGI_WORKERS: 2
    restart: unless-stopped
    volumes:
      - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
      - ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
      - ../apps/openedx/config:/openedx/config:ro
      - ../../data/lms:/openedx/data
      - ../../data/openedx-media:/openedx/media


cms:
    image: docker.io/overhangio/openedx:15.3.0
    environment:
      SERVICE_VARIANT: cms
      DJANGO_SETTINGS_MODULE: cms.envs.tutor.production
      UWSGI_WORKERS: 2
    restart: unless-stopped
    volumes:
      - ../apps/openedx/settings/lms:/openedx/edx-platform/lms/envs/tutor:ro
      - ../apps/openedx/settings/cms:/openedx/edx-platform/cms/envs/tutor:ro
      - ../apps/openedx/config:/openedx/config:ro
      - ../../data/cms:/openedx/data
      - ../../data/openedx-media:/openedx/media

I somewhat get the idea that whatever actually defines lms.envs.tutor.production in Python isn’t properly working. I’ll do some more digging

ghassan · February 23, 2023, 6:35pm

Hummm, yeah If I can guess, it can be file ownership issue, it’s just a guess it might be totally other thing

igmar · February 23, 2023, 6:37pm

Any idea what in the container actually defines it ? A grep yielded nothing.

ghassan · February 23, 2023, 6:39pm

It should be an ENV variable, python/django gets through ENV i.e inside the container you shall be something like

root@4ea0adff9e9e:/openedx/edx-platform# echo $DJANGO_SETTINGS_MODULE 
lms.envs.tutor.production

igmar · February 23, 2023, 7:00pm

Yeah, indeed. Question is more : What python code does it actually load that contains the settings. I can’t get into the container, since it keeps restarting.

ghassan · February 23, 2023, 7:21pm

Humm…
So to my understanding edx-platform is based is a django app, following the logs above, you can trace it to this line

github.com

django/django/blob/722e9f8a38f5b34f2423059a75f8a59bb8eb931a/django/conf/init.py#L170


      
          class Settings:
              def __init__(self, settings_module):
                  # update this dict from global settings (but only for ALL_CAPS settings)
                  for setting in dir(global_settings):
                      if setting.isupper():
                          setattr(self, setting, getattr(global_settings, setting))
          
          
        # store the settings module in case someone later cares
                  self.SETTINGS_MODULE = settings_module
          
          
        mod = importlib.import_module(self.SETTINGS_MODULE)
          
          
        tuple_settings = (
                      "INSTALLED_APPS",
                      "TEMPLATE_DIRS",
                      "LOCALE_PATHS",
                  )
                  self._explicit_settings = set()
                  for setting in dir(mod):
                      if setting.isupper():
                          setting_value = getattr(mod, setting)

The cycle is

edx-platform use django
django load the setting as pointed above
django uses importlib module to load the file

It’s a standard pattern of django to define the settings location as env variable which seems from our checks so far, this is correct.

That file pointed above isn’t part of edx-platform code, hence in the logs the prefix of openedx/venv/lib/python3.8/site-packages/django.

ghassan · February 23, 2023, 7:35pm

Aside from the cycle above, if I were you I would checks the following;

from inside the container can you check if this file exists /openedx/edx-platform/lms/envs/tutor/production.py _This file doesn’t come shipped with the docker image rather its mounted from tutor config printroot/env... So this check suppose to tell us if mounting is working between host and docker
Ensure python runtime has access to /openedx/edx-platform/ i.e. that directory is in sys.path below is what I get running from the container

root@4ea0adff9e9e:/openedx/edx-platform# python

Python 3.8.12 (default, Dec 11 2022, 11:58:22) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/opt/pyenv/versions/3.8.12/lib/python38.zip', '/opt/pyenv/versions/3.8.12/lib/python3.8', '/opt/pyenv/versions/3.8.12/lib/python3.8/lib-dynload', '/openedx/venv/lib/python3.8/site-packages', '/openedx/venv/src/codejail', '/openedx/venv/src/olxcleaner', '/openedx/venv/src/rate-xblock', '/openedx/venv/src/xblock-google-drive', '/openedx/venv/src/django-wiki', '/openedx/venv/src/blockstore', '/openedx/edx-platform']

Notice the last directroy in the array, if it doesn’t exits then django wouldn’t be able to load the module

igmar · February 23, 2023, 7:40pm

Thanks, good pointers. I need to find a way into the container without it restarting. I’ll probably override the compose file for this.

ghassan · February 23, 2023, 8:10pm

Humm, can you try something like this ~~docker run -it overhangio/openedx:15.0.0 /bin/bash.~~
this suppose to override the deafult CMD in the image.

Edit: The above command will fail the first check above because we are not mounting, so run this instead tutor local run lms /bin/bash

igmar · February 24, 2023, 3:20pm

Good… I found it. The host has umask 0027 set for all accounts. That caused the dirs to be created with 750, and files with 640. Since there is no uid mapping done on container level, the process that django runs as can’t open the files → No config.

Changing the perms from 750 → 755 and 640 → 644 fixed the issue. We might want to do a pre-check on this, to just abort if the config can’t be read.

ghassan · February 25, 2023, 12:04pm

Yeah I expected it might be file ownership, it seems if python doesn’t have permission to read it will just say it doesn’t exist. Also typically redhat image are more secured than typical linux image.

igmar · March 8, 2023, 1:58pm

Its indeed the hosts settings. I also got strange permission errors when doing a local do init, but that can be fixed by running the migrate from within the container. I can’t explain the error itself tho.

lceconomy · April 28, 2023, 2:59pm

Hi! What steps did you take to change the perms from 750 → 755 and 640 → 644? I’m facing the same issue.

aracid · February 9, 2024, 12:16am

Hey there,

I’m running into the same issue, any more specifics would be great as to how you solved this issue.

Thanks in advance

Topic		Replies	Views
ModuleNotFoundError: No module named 'celery.decorators' Tutor Help	0	907	February 7, 2023
Tutor dev quickstart initialized, but the local.overhang.io cannot be accessed Tutor Help	1	378	November 10, 2022
Tutor fails on M1 at 'tutor local do init' command Tutor Help	0	404	March 28, 2023
Tutor dev launch fails with "SyntaxError: EOL while scanning string literal" Tutor Help tutor	2	152	June 19, 2023
Tutor E-Commerce: Server error Tutor Help tutor , tutor-plugin	0	186	July 5, 2023

Tutor local do init failing : ModuleNotFoundError: No module named 'lms.envs.tutor.production'

Related Topics