Inconsistent OpenedX Custom Docker Image Updates: Container Restart Loop & Unreflected Changes on tutor local start/launch - Ubuntu 24.04, Docker Hub Integration

Site Operators Tutor Help
1

Hi everyone, I am encountering an issue while attempting to update my local OpenedX development environment (Ubuntu 24.04) with a custom Docker image. My workflow involves the following steps:

  1. Image Build and Push:
  • A custom Docker image is built from my GitHub repository using GitHub Actions, based on the existing OpenedX Dockerfile.
  • The resulting image is pushed to my Docker Hub registry.
  1. Local Server Update:
  • I have a local OpenedX instance running using tutor local launch (intended as a testing environment, not production).
  • I execute a shell script to:
    • Stop the running OpenedX containers (tutor local stop).
    • Prune the Docker build cache (docker builder prune).
    • Remove the existing OpenedX containers (lms, cms, lms-worker1, cms-worker1).
    • Remove the old OpenedX image (docker rmi -f overhangio/openedx:19.0.2-indigo).
    • Update the DOCKER_IMAGE_OPENEDX configuration in Tutor (tutor config save --set DOCKER_IMAGE_OPENEDX=docker.io/(personal image repository)).
    • Pull the new custom image from Docker Hub (docker pull image:tag).
    • save the tutor config.
    • Start the containers with tutor local start -d.
  1. Observed Issues:
  • After running tutor local start, the lms and cms worker containers enter a restart loop.
  • The changes made in my custom Docker image are not reflected after rebuilding the image using tutor images build openedx --no-cache and then running tutor local launch.
  • I have provided the logs from the lms and cms worker containers below, as well as the shell script I used for the update process.
tutor local stop

echo "Removing cache" 
yes "" | docker builder prune 

echo "Removing containers; lms, lms-worker1, cms, cms-worker-1"
docker rm -f tutor_local-cms-worker-1
docker rm -f tutor_local-lms-worker-1
docker rm -f tutor_local-cms-1  
docker rm -f tutor_local-lms-1  

echo "Removing old OpenedX image..."
docker rmi -f overhangio/openedx:19.0.2-indigo || true  
 
echo "Updating Openedx Image..."
tutor config save --set DOCKER_IMAGE_OPENEDX=docker.io/(personal image repostiroy)
 
echo "Pulling the latest OpenedX image..."
docker pull image:tag
 
tutor config save
 
echo "Applying OpenedX configuration..."
# tutor images build openedx --no-cache
yes "" | tutor local start -d

Specific Questions:

  1. Why are the lms and cms worker containers experiencing a restart loop after updating the image and running tutor local start?
  2. Why are the changes in my custom Docker image not being reflected after rebuilding with tutor images build openedx --no-cache and running tutor local launch?
  3. Are there any specific configurations or steps I might be missing in my update process?
  4. Please analyze the shell script to identify potential causes of the issue.

LOGS

2025-03-24 13:01:27 Traceback (most recent call last):
2025-03-24 13:01:27   File "/edx/app/edxapp/edx-platform/./manage.py", line 102, in <module>
2025-03-24 13:01:27     startup = importlib.import_module(edx_args.startup)
2025-03-24 13:01:27               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:27   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:27     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:27            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2025-03-24 13:01:27   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:27   File "/edx/app/edxapp/edx-platform/cms/startup.py", line 10, in <module>
2025-03-24 13:01:27     settings.INSTALLED_APPS  # pylint: disable=pointless-statement
2025-03-24 13:01:27     ^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:27   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
2025-03-24 13:01:27     self._setup(name)
2025-03-24 13:01:27   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
2025-03-24 13:01:27     self._wrapped = Settings(settings_module)
2025-03-24 13:01:27                     ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:27   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
2025-03-24 13:01:27     mod = importlib.import_module(self.SETTINGS_MODULE)
2025-03-24 13:01:27           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:27   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:27     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:27            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:27   File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
2025-03-24 13:01:27 ModuleNotFoundError: No module named 'cms.envs.tutor'
2025-03-24 13:01:29 Traceback (most recent call last):
2025-03-24 13:01:29   File "/edx/app/edxapp/edx-platform/./manage.py", line 102, in <module>
2025-03-24 13:01:29     startup = importlib.import_module(edx_args.startup)
2025-03-24 13:01:29               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:29   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:29     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:29            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2025-03-24 13:01:29   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:29   File "/edx/app/edxapp/edx-platform/cms/startup.py", line 10, in <module>
2025-03-24 13:01:29     settings.INSTALLED_APPS  # pylint: disable=pointless-statement
2025-03-24 13:01:29     ^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:29   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
2025-03-24 13:01:29     self._setup(name)
2025-03-24 13:01:29   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
2025-03-24 13:01:29     self._wrapped = Settings(settings_module)
2025-03-24 13:01:29                     ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:29   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
2025-03-24 13:01:29     mod = importlib.import_module(self.SETTINGS_MODULE)
2025-03-24 13:01:29           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:29   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:29     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:29            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:29   File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
2025-03-24 13:01:29 ModuleNotFoundError: No module named 'cms.envs.tutor'
2025-03-24 13:01:31 Traceback (most recent call last):
2025-03-24 13:01:31   File "/edx/app/edxapp/edx-platform/./manage.py", line 102, in <module>
2025-03-24 13:01:31     startup = importlib.import_module(edx_args.startup)
2025-03-24 13:01:31               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:31   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:31     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:31            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:31   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:31   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:31   File "<frozen importlib._bootstrap>", line 1147, in 

2025-03-24 13:01:26 Traceback (most recent call last):
2025-03-24 13:01:26   File "/edx/app/edxapp/edx-platform/./manage.py", line 102, in <module>
2025-03-24 13:01:26     startup = importlib.import_module(edx_args.startup)
2025-03-24 13:01:26               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:26   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:26     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2025-03-24 13:01:26   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:26   File "/edx/app/edxapp/edx-platform/lms/startup.py", line 10, in <module>
2025-03-24 13:01:26     settings.INSTALLED_APPS  # pylint: disable=pointless-statement
2025-03-24 13:01:26     ^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:26   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
2025-03-24 13:01:26     self._setup(name)
2025-03-24 13:01:26   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
2025-03-24 13:01:26     self._wrapped = Settings(settings_module)
2025-03-24 13:01:26                     ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:26   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
2025-03-24 13:01:26     mod = importlib.import_module(self.SETTINGS_MODULE)
2025-03-24 13:01:26           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:26   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:26     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:26   File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
2025-03-24 13:01:27 ModuleNotFoundError: No module named 'lms.envs.tutor'
2025-03-24 13:01:28 Traceback (most recent call last):
2025-03-24 13:01:28   File "/edx/app/edxapp/edx-platform/./manage.py", line 102, in <module>
2025-03-24 13:01:28     startup = importlib.import_module(edx_args.startup)
2025-03-24 13:01:28               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:28   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:28     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:28            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2025-03-24 13:01:28   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:28   File "/edx/app/edxapp/edx-platform/lms/startup.py", line 10, in <module>
2025-03-24 13:01:28     settings.INSTALLED_APPS  # pylint: disable=pointless-statement
2025-03-24 13:01:28     ^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:28   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
2025-03-24 13:01:28     self._setup(name)
2025-03-24 13:01:28   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
2025-03-24 13:01:28     self._wrapped = Settings(settings_module)
2025-03-24 13:01:28                     ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:28   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
2025-03-24 13:01:28     mod = importlib.import_module(self.SETTINGS_MODULE)
2025-03-24 13:01:28           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:28   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:28     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:28            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:28   File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
2025-03-24 13:01:28 ModuleNotFoundError: No module named 'lms.envs.tutor'
2025-03-24 13:01:30 Traceback (most recent call last):
2025-03-24 13:01:30   File "/edx/app/edxapp/edx-platform/./manage.py", line 102, in <module>
2025-03-24 13:01:30     startup = importlib.import_module(edx_args.startup)
2025-03-24 13:01:30               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:30   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:30     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:30            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2025-03-24 13:01:30   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2025-03-24 13:01:30   File "/edx/app/edxapp/edx-platform/lms/startup.py", line 10, in <module>
2025-03-24 13:01:30     settings.INSTALLED_APPS  # pylint: disable=pointless-statement
2025-03-24 13:01:30     ^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:30   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
2025-03-24 13:01:30     self._setup(name)
2025-03-24 13:01:30   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
2025-03-24 13:01:30     self._wrapped = Settings(settings_module)
2025-03-24 13:01:30                     ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:30   File "/edx/app/edxapp/venvs/edxapp/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
2025-03-24 13:01:30     mod = importlib.import_module(self.SETTINGS_MODULE)
2025-03-24 13:01:30           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:30   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
2025-03-24 13:01:30     return _bootstrap._gcd_import(name[level:], package, level)
2025-03-24 13:01:30            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2025-03-24 13:01:30   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
2

@regis Kindly helps