Tutor `openedx` build error with `sphinxcontrib-openapi[markdown]==0.6.0` with Maple (tutor 13.3.1)

Site Operators
1

Using tutor local: 13.3.1

Received the following Docker openedx image build error after issuing.
tutor images build openedx.

ModuleNotFoundError: No module named 'setuptools.command.build'

Collecting sphinx==4.2.0
  Downloading Sphinx-4.2.0-py3-none-any.whl (3.1 MB)
Collecting sphinxcontrib-applehelp==1.0.2
  Downloading sphinxcontrib_applehelp-1.0.2-py2.py3-none-any.whl (121 kB)
Collecting sphinxcontrib-devhelp==1.0.2
  Downloading sphinxcontrib_devhelp-1.0.2-py2.py3-none-any.whl (84 kB)
Collecting sphinxcontrib-htmlhelp==2.0.0
  Downloading sphinxcontrib_htmlhelp-2.0.0-py2.py3-none-any.whl (100 kB)
Collecting sphinxcontrib-httpdomain==1.8.0
  Downloading sphinxcontrib_httpdomain-1.8.0-py2.py3-none-any.whl (19 kB)
Collecting sphinxcontrib-jsmath==1.0.1
  Downloading sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Collecting sphinxcontrib-openapi[markdown]==0.6.0
  Downloading sphinxcontrib-openapi-0.6.0.tar.gz (173 kB)
    ERROR: Command errored out with exit status 1:
     command: /openedx/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info
         cwd: /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/
    Complete output (68 lines):
    /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/.eggs/setuptools_scm-8.2.0-py3.8.egg/setuptools_scm/_integration/setuptools.py:31: RuntimeWarning:
    ERROR: setuptools==44.1.0 is used in combination with setuptools-scm>=8.x
    
    Your build configuration is incomplete and previously worked by accident!
    setuptools-scm requires setuptools>=61
    
    Suggested workaround if applicable:
     - migrating from the deprecated setup_requires mechanism to pep517/518
       and using a pyproject.toml to declare build dependencies
       which are reliably pre-installed before running the build tools
    
      warnings.warn(
    WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
    Traceback (most recent call last):
      File "/tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/.eggs/setuptools_scm-8.2.0-py3.8.egg/setuptools_scm/_integration/pyproject_reading.py", line 36, in read_pyproject
        section = defn.get("tool", {})[tool_name]
    KeyError: 'setuptools_scm'
    running egg_info
    creating /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info
    writing /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info/dependency_links.txt
    writing namespace_packages to /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info/namespace_packages.txt
    writing requirements to /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info/requires.txt
    writing top-level names to /tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info/top_level.txt
    writing manifest file '/tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/pip-egg-info/sphinxcontrib_openapi.egg-info/SOURCES.txt'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-c_odw0hm/sphinxcontrib-openapi/setup.py", line 16, in <module>
        setup(
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 296, in run
        self.find_sources()
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 303, in find_sources
        mm.run()
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 534, in run
        self.add_defaults()
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 570, in add_defaults
        sdist.add_defaults(self)
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/command/sdist.py", line 226, in add_defaults
        self._add_defaults_python()
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/command/sdist.py", line 135, in _add_defaults_python
        build_py = self.get_finalized_command('build_py')
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/cmd.py", line 299, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/command/build_py.py", line 34, in finalize_options
        orig.build_py.finalize_options(self)
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/command/build_py.py", line 43, in finalize_options
        self.set_undefined_options('build',
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/cmd.py", line 286, in set_undefined_options
        src_cmd_obj = self.distribution.get_command_obj(src_cmd)
      File "/opt/pyenv/versions/3.8.12/lib/python3.8/distutils/dist.py", line 857, in get_command_obj
        klass = self.get_command_class(command)
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/dist.py", line 834, in get_command_class
        self.cmdclass[command] = cmdclass = ep.load()
      File "/openedx/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
        return self.resolve()
      File "/openedx/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve
        module = __import__(self.module_name, fromlist=['__name__'], level=0)
    ModuleNotFoundError: No module named 'setuptools.command.build'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 25.0.1 is available.
You should consider upgrading via the '/openedx/venv/bin/python -m pip install --upgrade pip' command.
The command '/bin/sh -c pip install -r requirements/edx/development.txt' returned a non-zero code: 1
1 Like
2

Hi @Zachary_Trabookis - it may help with debugging to know a bit more about your setup. What version of Tutor are you using? Any other relevant system details?

3

We’re using Maple (tutor v13). We’ve built before with no issues @sarina.

I tried adding the following to a tutor patch but it still tried to install that same version 0.6.0.

# openedx-dockerfile-post-python-requirements 
RUN pip install sphinxcontrib-openapi[markdown]==0.8.4
4

Hmm… my very best guess is the issue is related to being on an old release, and older upstream packages being deprecated. Unfortunately I don’t think I can help any further, this is the line that gets me:

    Your build configuration is incomplete and previously worked by accident!
1 Like
5

v13 installs setuptools==44.1.0

sphinxcontrib-openapi package uses setuptools-scm >= 8.x, which requires setuptools >= 61

on v19 it uses setuptools==69.1.1

You might be able to get away with replacing the setuptools version in your dockerfile, assuming that doesn’t conflict with anything else, being such an older version I’m sure support is going to be somewhat limited

2 Likes
6

Thanks @joel.edwards @sarina

We are trying to upgrade our release now to a later version for cases like this. I appreciate your advice.

2 Likes
7

I encountered the same issue and resolved it by updating setuptools from version 44.1.0 to 69.1.1. It seems to be working fine now.

2 Likes
8

@muneera_salah Thanks for the reply.

With maple it uses the following.

setuptools==44.1.0 pip==20.0.2 wheel==0.34.2

Did you also end up having to update pip and wheel to the respective versions that are used in the sumac release?

setuptools==69.1.1 pip==24.0 wheel==0.43.0
9

So far, I haven’t needed to upgrade pip or wheel, everything has been running smoothly with Maple 3 and tutor: 13.3.1

However, I did add the following lines:

RUN pip install setuptools==69.1.1 pip==20.0.2 wheel==0.34.2

# Install missing py2neo package that was abruptly removed from PyPI  
RUN pip install https://github.com/overhangio/py2neo/releases/download/2021.2.3/py2neo-2021.2.3.tar.gz

also I had to run this command

pip install --upgrade pyyaml
2 Likes
10

@Zachary_Trabookis by the way, the issue might be related to Ubuntu 24. It was resolved when I downgraded to Ubuntu 20.04.6 LTS

1 Like
11

Hi all

I faced the following issue

Collecting XlsxWriter==3.0.3 (from -r requirements/edx/development.txt (line 1568))
  Downloading XlsxWriter-3.0.3-py3-none-any.whl.metadata (2.6 kB)
Requirement already satisfied: qrcode in /openedx/venv/lib/python3.8/site-packages (from -r requirements/edx/development.txt (line 1569)) (7.4.2)
Requirement already satisfied: setuptools>=20.0 in /openedx/venv/lib/python3.8/site-packages (from astroid==2.6.6->-r requirements/edx/development.txt (line 66)) (69.1.1)
error: invalid-installed-package

× Cannot process installed package celery 4.4.7 in '/openedx/venv/lib/python3.8/site-packages' because it has an invalid requirement:
│ Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
│     pytz (>dev)
│          ~^
╰─> Starting with pip 24.1, packages with invalid requirements can not be processed.

hint: To proceed this package must be uninstalled.
app@1c64bc00b2e6:~/edx-platform$

I fixed it by updating the following versions in these files

  • /.local/share/tutor/volumes/edx-platform/requirements/edx/base.txt
  • /.local/share/tutor/volumes/edx-platform/requirements/edx/development.txt
from
celery==4.4.7
to
celery==5.2.7

from
kombu==4.6.11
to 
kombu>=5.2.3,<6.0

form
vine==1.3.0
to
vine>=5.0.0,<6.0

from
amqp==2.6.1
to
amqp>=5.0.0

from
click==7.1.2
to
click>=8.0.3,<9.0
12

@muneera_salah
Are you using tutor for deployment? It appears that you had to modify the edx-platform repo requirements directly. I’m curious to know if you modified the tutor openedx Dockerfile directly or used tutor hooks to make those changes.

13

Hi @Zachary_Trabookis , apologies for the delayed response.

Yes, I applied the changes directly in the Dockerfile:

RUN pip install setuptools==69.1.1 pip==20.0.2 wheel==0.34.2

# Install missing py2neo package that was abruptly removed from PyPI  
RUN pip install https://github.com/overhangio/py2neo/releases/download/2021.2.3/py2neo-2021.2.3.tar.gz

I also made some customizations in the edx-platform and applied changes directly in the requirements directory.

Additionally, I created a new Docker Compose override file named docker-compose.override.yml, which I added to the following paths:

  • ~/.local/share/tutor/env/dev/docker-compose.override.yml
  • ~/.local/share/tutor/env/local/docker-compose.override.yml

The content of the file is:

version: "3.7"

services:
  lms:
    volumes:
      - ../../volumes/edx-platform:/openedx/edx-platform

  cms:
    volumes:
      - ../../volumes/edx-platform:/openedx/edx-platform

  lms-worker:
    volumes:
      - ../../volumes/edx-platform:/openedx/edx-platform

  cms-worker:
    volumes:
      - ../../volumes/edx-platform:/openedx/edx-platform
1 Like
14

@muneera_salah
After applying the fix for the cms, cms-worker, lms, lms-worker containers with this change, I ran into this additional error.

RUN pip install setuptools==69.1.1 pip==20.0.2 wheel==0.34.2

Error

tutor_local-lms-worker-1  |     INSTALLED_APPS.extend(get_plugin_apps(ProjectType.LMS))
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/plugins/plugin_apps.py", line 22, in get_plugin_apps
tutor_local-lms-worker-1  |     for app_config in registry.get_plugin_app_configs(project_type)
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/plugins/registry.py", line 16, in get_plugin_app_configs
tutor_local-lms-worker-1  |     return DjangoAppRegistry.get_available_plugins(project_type).values()  # pylint: disable=dict-values-not-iterating
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/plugins/plugin_manager.py", line 34, in get_available_plugins
tutor_local-lms-worker-1  |     extension_manager = ExtensionManager(
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/stevedore/extension.py", line 133, in __init__
tutor_local-lms-worker-1  |     extensions = self._load_plugins(invoke_on_load,
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/stevedore/extension.py", line 218, in _load_plugins
tutor_local-lms-worker-1  |     for ep in self.list_entry_points():
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/stevedore/extension.py", line 207, in list_entry_points
tutor_local-lms-worker-1  |     eps = list(_cache.get_group_all(self.namespace))
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/stevedore/_cache.py", line 179, in get_group_all
tutor_local-lms-worker-1  |     data = self._get_data_for_path(path)
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/stevedore/_cache.py", line 162, in _get_data_for_path
tutor_local-lms-worker-1  |     data = _build_cacheable_data(path)
tutor_local-lms-worker-1  |   File "/openedx/venv/lib/python3.8/site-packages/stevedore/_cache.py", line 119, in _build_cacheable_data
tutor_local-lms-worker-1  |     item = ep[:]  # convert namedtuple to tuple
tutor_local-lms-worker-1  | TypeError: 'EntryPoint' object is not subscriptable

To fix this, I also installed the following Python package. Did you also have to do this?

RUN pip install stevedore==3.5.0
1 Like
15

@muneera_salah
I also ran into this issue with the pdfXBlock. I fixed this by setting the setup.py version to 1.0.2 instead of v1.0.2-rg due to an invalid version string (not compliant with PEP 440) from the updated setuptools package.

Error

Obtaining xblock-pdf==v1.0.2-rg from git+https://github.com/CUCWD/pdfXBlock.git@1.0.2-maple.3-cucwd#egg=xblock-pdf==v1.0.2-rg
  Cloning https://github.com/CUCWD/pdfXBlock.git (to revision 1.0.2-maple.3-cucwd) to /openedx/venv/src/xblock-pdf
  Running command git clone -q https://github.com/CUCWD/pdfXBlock.git /openedx/venv/src/xblock-pdf
    ERROR: Command errored out with exit status 1:
     command: /openedx/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/openedx/venv/src/xblock-pdf/setup.py'"'"'; __file__='"'"'/openedx/venv/src/xblock-pdf/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
         cwd: /openedx/venv/src/xblock-pdf/
    Complete output (15 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/openedx/venv/src/xblock-pdf/setup.py", line 18, in <module>
        setup(
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
        return distutils.core.setup(**attrs)
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 147, in setup
        _setup_distribution = dist = klass(attrs)
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/dist.py", line 314, in __init__
        self.metadata.version = self._normalize_version(self.metadata.version)
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/dist.py", line 350, in _normalize_version
        normalized = str(Version(version))
      File "/openedx/venv/lib/python3.8/site-packages/setuptools/_vendor/packaging/version.py", line 198, in __init__
        raise InvalidVersion(f"Invalid version: '{version}'")
    setuptools.extern.packaging.version.InvalidVersion: Invalid version: 'v1.0.2-rg'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 25.0.1 is available.
You should consider upgrading via the '/openedx/venv/bin/python -m pip install --upgrade pip' command.
The command '/bin/sh -c pip install -e git+https://github.com/CUCWD/pdfXBlock.git@1.0.2-maple.3-cucwd#egg=xblock-pdf==v1.0.2-rg' returned a non-zero code: 1
1 Like
16

@Zachary_Trabookis Great, I didn’t encounter these errors, but it’s good to know, thank you for sharing

1 Like
17

@Zachary_Trabookis
I encountered another error, which I believe is due to the changes made to Celery

from
celery==4.4.7
to
celery==5.2.7

error:

from celery.task.control import revoke
ModuleNotFoundError: No module named 'celery.task'

code:

from celery import shared_task
from celery.task.control import revoke

def setup_schedule_for_generating_certificate(course_key):
    """
    Create a task to generate a certificate for students on the course
    """
    log.info(f'Course {course_key}. Setup schedule for generating certificate task.')
    extra_setting = ExtraCourseSetting.objects.get(course_id=course_key)
    generated_at = extra_setting.certificate_receive_datetime

    if not generated_at:
        return False

    certificate_meta, created = CertificateMeta.objects.update_or_create(
        course_id=extra_setting.id,
        defaults={
            'generated_at': generated_at,
        }
    )
    if not created:
        log.info(f'Task {certificate_meta.generation_task_id} has been revoked.')
        revoke(certificate_meta.generation_task_id, terminate=True)
        certificate_meta.update_task_result('REVOKED')

    task_result = generate_certificates.apply_async(
        eta=generated_at,
        kwargs={'course_key': str(course_key), 'extra_setting_id': extra_setting.id}
    )
    log.info(f'Task {task_result.id} has been created on {generated_at} with status {task_result.status}')
    CertificateMeta.objects.filter(course_id=extra_setting.id).update(generation_task_id=task_result.id)

what do you thing?

18

I’ll have to check on celery @muneera_salah. Does it occur when you do something specific?

Looks like certificate generation.

19

@muneera_salah
I didn’t have any issues with celery when generating a certificate exception. It should have run this celery task to generate a new certificate.

1 Like
20

@Zachary_Trabookis

I think that I found the issue. It occurred when running the following command in the deployment pipeline:

tutor local run lms ./manage.py lms migrate

The root cause was the way revoke was being imported. It was fixed by updating the import statement from:

from celery import shared_task
from celery.task.control import revoke

to:

from celery import shared_task, current_app
revoke = current_app.control.revoke

The function is a customized Celery task within Open edX, used to generate certificates after the on-site course has ended.