Building on Ubuntu 20 to prepare for Koa

I have been working on trying to run edX master on Ubuntu 20 and using Python 3.8. I have been able to run through the Ansible playbooks to get the code deployed, but I have not yet managed to get to a functional system.

Of note:

  • the static assets do not get generated properly. When I try to run the /edx/bin/edxapp-update-assets script I see a sequence of tracebacks at the beginning of execution.
Unable to load XModuleDescriptor 'course'
Traceback (most recent call last):
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/xblock/plugin.py", line 141, in load_classes
    yield (class_.name, cls._load_class_entry_point(class_))
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/xblock/plugin.py", line 70, in _load_class_entry_point
    class_ = entry_point.load()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2443, in load
    return self.resolve()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2449, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/course_module.py", line 52, in <module>
    EXAM_SETTINGS_HTML_VIEW_ENABLED = settings.FEATURES.get('ENABLE_EXAM_SETTINGS_HTML_VIEW', False)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/conf/__init__.py", line 80, in __getattr__
    val = getattr(self._wrapped, name)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/conf/__init__.py", line 222, in __getattr__
    return getattr(self.default_settings, name)
AttributeError: module 'django.conf.global_settings' has no attribute 'FEATURES'
  • Related to this I see log output that is throwing tracebacks due to a lack of the webpack stats file being generated.
Traceback (most recent call last):
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/core/handlers/exception.py", line 129, in handle_uncaught_exception
    return callback(request, **param_dict)
  File "/edx/app/edxapp/edx-platform/common/djangoapps/util/views.py", line 95, in wrapper
    return func(request, *args, **kwargs)
  File "/edx/app/edxapp/edx-platform/lms/djangoapps/static_template_view/views.py", line 103, in render_500
    return HttpResponseServerError(render_to_string('static_templates/server-error.html', {}, request=request))
  File "/edx/app/edxapp/edx-platform/common/djangoapps/edxmako/shortcuts.py", line 183, in render_to_string
    return template.render(dictionary, request)
  File "/edx/app/edxapp/edx-platform/common/djangoapps/edxmako/template.py", line 83, in render
    return self.mako_template.render_unicode(**context_dictionary)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/mako/template.py", line 481, in render_unicode
    return runtime._render(
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/mako/runtime.py", line 878, in _render
    _render_context(
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/mako/runtime.py", line 920, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/mako/runtime.py", line 947, in _exec_template
    callable_(context, *args, **kwargs)
  File "/tmp/mako_lms/84caac5afec1d5cec18c223971540436/mitx-theme/lms/templates/main.html.py", line 214, in render_body
    __M_writer(filters.decode.utf8(static.webpack(entry='commons')))
  File "/tmp/mako_lms/84caac5afec1d5cec18c223971540436/static_content.html.py", line 282, in render_webpack
    __M_writer(filters.html_escape(filters.decode.utf8(HTML(render_bundle(entry, extension=None, config='DEFAULT', attrs=attrs)))))
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/templatetags/webpack_loader.py", line 12, in render_bundle
    tags = utils.get_as_tags(bundle_name, extension=extension, config=config, attrs=attrs)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/utils.py", line 62, in get_as_tags
    bundle = _get_bundle(bundle_name, extension, config)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/utils.py", line 40, in _get_bundle
    bundle = get_loader(config).get_bundle(bundle_name)
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/loader.py", line 59, in get_bundle
    assets = self.get_assets()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/loader.py", line 36, in get_assets
    self._assets[self.name] = self.load_assets()
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/loader.py", line 28, in load_assets
    raise IOError(
OSError: Error reading /edx/var/edxapp/staticfiles/webpack-stats.json. Are you sure webpack has generated the file and the path is correct?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/webpack_loader/loader.py", line 25, in load_assets
    with open(self.config['STATS_FILE'], encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: Path('/edx/var/edxapp/staticfiles/webpack-stats.json')
  • When I try to run migrations I am seeing exceptions about invalid credentials, because for some reason it is trying to use the values for the separate CSMH database (Courseware Student Module History). If I remove the configuration block in /edx/etc/lms.yml for that database then migrations can run through properly, but there is definitely some confusion happening as far as which credentials to use for which database when executing migrations.

I will update this thread with any other findings or changes in state as I periodically attempt to get this build to succeed.

1 Like

So, I ran a new build trying to deploy edX master on Ubuntu 16.04 but with Python 3.8 and I am seeing the same problems. This points to the fact that it’s the version of Python that is the issue, not the version of Ubuntu (aside from the fact that Ubuntu 20.04 ships with Python 3.8 as default).

FYI: I’m seeing the update_assets issue mentioned above on the latest version of devstack as well. (Note: my devstack is tracking master, not Koa.)