Nutmeg upgrade: outlines.py: "openedx.core.djangoapps.content.learning_sequences.models.LearningContext.DoesNotExist: LearningContext matching query does not exist"

Rohan · July 21, 2022, 5:49pm

I upgraded from tutor 12.0.4 Lilac to tutor 14.0.3 Nutmeg. I was getting errors not being able to access pages, and then I enabled the tutor “mfe” plugin based on some other people’s errors. Now I’m getting a different error when I attempt to access a course as an enrolled learning account rather than admin account:
“There was an error loading this course.”
The logs show the below, but the most relevant line is perhaps:
“openedx.core.djangoapps.content.learning_sequences.models.LearningContext.DoesNotExist: LearningContext matching query does not exist”

I can access the page fine in studio and all the content is still there, it’s only accessing as a student which no longer works.

Full error:

lms_1                        | 2022-07-21 17:38:46,324 ERROR 7 [root] [user None] [ip None] signals.py:22 - Uncaught exception from None
lms_1                        | Traceback (most recent call last):
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 223, in _get_course_context_for_outline
lms_1                        |     LearningContext.objects
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
lms_1                        |     raise self.model.DoesNotExist(
lms_1                        | openedx.core.djangoapps.content.learning_sequences.models.LearningContext.DoesNotExist: LearningContext matching query does not exist.
lms_1                        | 
lms_1                        | During handling of the above exception, another exception occurred:
lms_1                        | 
lms_1                        | Traceback (most recent call last):
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
lms_1                        |     response = get_response(request)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
lms_1                        |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lms_1                        |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
lms_1                        |     return func(*args, **kwds)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
lms_1                        |     return view_func(*args, **kwargs)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
lms_1                        |     return self.dispatch(request, *args, **kwargs)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
lms_1                        |     response = self.handle_exception(exc)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
lms_1                        |     self.raise_uncaught_exception(exc)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
lms_1                        |     raise exc
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
lms_1                        |     response = handler(request, *args, **kwargs)
lms_1                        |   File "/openedx/edx-platform/./lms/djangoapps/course_home_api/outline/views.py", line 295, in get
lms_1                        |     user_course_outline = get_user_course_outline(
lms_1                        |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
lms_1                        |     return func(*args, **kwds)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 268, in get_user_course_outline
lms_1                        |     user_course_outline, _ = _get_user_course_outline_and_processors(course_key, user, at_time)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 316, in _get_user_course_outline_and_processors
lms_1                        |     full_course_outline = get_course_outline(course_key)
lms_1                        |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
lms_1                        |     return func(*args, **kwds)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 112, in get_course_outline
lms_1                        |     course_context = _get_course_context_for_outline(course_key)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 230, in _get_course_context_for_outline
lms_1                        |     raise CourseOutlineData.DoesNotExist(  # lint-amnesty, pylint: disable=raise-missing-from
lms_1                        | openedx.core.djangoapps.content.learning_sequences.data.CourseOutlineData.DoesNotExist: No CourseOutlineData for course-v1:MyOrg+MyClass+Run1
lms_1                        | 2022-07-21 17:38:46,442 ERROR 7 [django.request] [user 5] [ip 1.2.3.4] log.py:224 - Internal Server Error: /api/course_home/outline/course-v1:MyOrg+MyClass+Run1
lms_1                        | Traceback (most recent call last):
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 223, in _get_course_context_for_outline
lms_1                        |     LearningContext.objects
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
lms_1                        |     raise self.model.DoesNotExist(
lms_1                        | openedx.core.djangoapps.content.learning_sequences.models.LearningContext.DoesNotExist: LearningContext matching query does not exist.
lms_1                        | 
lms_1                        | During handling of the above exception, another exception occurred:
lms_1                        | 
lms_1                        | Traceback (most recent call last):
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
lms_1                        |     response = get_response(request)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
lms_1                        |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lms_1                        |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
lms_1                        |     return func(*args, **kwds)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
lms_1                        |     return view_func(*args, **kwargs)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
lms_1                        |     return self.dispatch(request, *args, **kwargs)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch
lms_1                        |     response = self.handle_exception(exc)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception
lms_1                        |     self.raise_uncaught_exception(exc)
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
lms_1                        |     raise exc
lms_1                        |   File "/openedx/venv/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch
lms_1                        |     response = handler(request, *args, **kwargs)
lms_1                        |   File "/openedx/edx-platform/./lms/djangoapps/course_home_api/outline/views.py", line 295, in get
lms_1                        |     user_course_outline = get_user_course_outline(
lms_1                        |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
lms_1                        |     return func(*args, **kwds)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 268, in get_user_course_outline
lms_1                        |     user_course_outline, _ = _get_user_course_outline_and_processors(course_key, user, at_time)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 316, in _get_user_course_outline_and_processors
lms_1                        |     full_course_outline = get_course_outline(course_key)
lms_1                        |   File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
lms_1                        |     return func(*args, **kwds)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 112, in get_course_outline
lms_1                        |     course_context = _get_course_context_for_outline(course_key)
lms_1                        |   File "/openedx/edx-platform/./openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 230, in _get_course_context_for_outline
lms_1                        |     raise CourseOutlineData.DoesNotExist(  # lint-amnesty, pylint: disable=raise-missing-from
lms_1                        | openedx.core.djangoapps.content.learning_sequences.data.CourseOutlineData.DoesNotExist: No CourseOutlineData for course-v1:MyOrg+MyClass+Run1
lms_1                        | [pid: 7|app: 0|req: 8/20] 172.19.0.4 () {54 vars in 3412 bytes} [Thu Jul 21 17:38:43 2022] GET /api/course_home/outline/course-v1:MyOrg+MyClass+Run1 => generated 9552 bytes in 3006 msecs (HTTP/1.1 500) 9 headers in 617 bytes (1 switches on core 0)

sambapete · July 21, 2022, 9:12pm

I would suggest you read the following page:
https://openedx.atlassian.net/wiki/spaces/COMM/pages/3205201949/Nutmeg

An internal performance improvement called “learning sequences” has been opt-in for a few releases, but is now always-on for Nutmeg. If you have any courses that have not been re-published on Koa or later, run the simulate_publish cms django command on your courses before upgrading, to populate the learning sequence data.

Rohan · July 21, 2022, 10:01pm

Thanks for the pointer. But I upgraded from Lilac so the “If you have any courses that have not been re-published on Koa or later” doesn’t apply in my case - they’re all Lilac-published classes.

(A comment on that page also says “Note that publishing a course in Koa, Lilac, or Maple would be good enough to have the course work fine in Nutmeg. (The learning sequences data was generated for those releases.)”)

Just to be on the safe side, I still ran:
tutor local run cms ./manage.py cms simulate_publish
but the error still remains even after I did that

And then based on this saying " The primary path which feeds course outline data into learning sequence models is a signal handler upon a Studio course publish. However, you can also seed data into it by using the update_course_outline management command." I tried the following command:

tutor local run cms ./manage.py cms update_course_outline <coursekey>
But that just ultimately led to the same error after a few encouraging statements:

2022-07-21 22:13:13,937 INFO 1 [openedx.core.djangoapps.content.learning_sequences.api.outlines] [user None] [ip None] outlines.py:389 - Replacing CourseOutline for course-v1:MyOrg+MyClass+MyRun1 (version 62d994bca7bf1b5486ec9997, 99 sequences)
2022-07-21 22:13:13,946 INFO 1 [openedx.core.djangoapps.content.learning_sequences.api.outlines] [user None] [ip None] outlines.py:433 - Created new CourseContext for course-v1:MyOrg+MyClass+MyRun1
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
openedx.core.djangoapps.content.learning_sequences.models.DoesNotExist: CourseSection matching query does not exist.

And finally I tried a manual edit of some course content and publishing it, but that also didn’t resolve the error.

dave · July 22, 2022, 4:38am

Is that the entirety of the stacktrace?

I think it’s dying here:

github.com

openedx/edx-platform/blob/open-release/nutmeg.master/openedx/core/djangoapps/content/learning_sequences/api/outlines.py#L445-L453


      
          sec_model, _created = CourseSection.objects.update_or_create(
              course_context=course_context,
              usage_key=section_data.usage_key,
              defaults={
                  'title': section_data.title,
                  'ordering': ordering,
                  'hide_from_toc': section_data.visibility.hide_from_toc,
                  'visible_to_staff_only': section_data.visibility.visible_to_staff_only,
              }

It’s really weird, because it looks like get_or_create is being called, is not finding the entry, but doesn’t create the entry. The only thing I can think of is that it’s hitting an integrity error underneath, assumes it’s a race condition, and then tries to do the get, which still fails. The only way I can think of that happening is if the connection to MySQL is running with a transaction isolation mode of REPEATABLE_READ instead of READ_COMMITTED, but I thought that the Django defaults for MySQL had moved everything to READ_COMMITTED, so I don’t know how that would happen at this point, unless you had specific connection configuration to set it that way.

I guess another possibility might be some weird case sensitivity issue. Can you look in the learning_sequences_coursesection table to see if it created any entries for your course, and if there are any ones for your course where the usage_key differs only by case from your course data?

@pdpinch: Is this the outline issue you mentioned running into before?

Rohan · July 22, 2022, 11:55am

It wasn’t the entire error message, because the rest of it was “During handling of the above exception, another exception occurred:” that seemed secondary, but here’s the full output that happened:

ubuntu@ip-172-31-72-11:~$ tutor local run cms ./manage.py cms update_course_outline course-v1:MyOrg+MyClass+MyRun1
docker-compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local run --rm cms ./manage.py cms update_course_outline course-v1:MyOrg+MyClass+MyRun1
Starting tutor_local_mongodb-permissions_1       ... done
Starting tutor_local_lms-permissions_1           ... done
Starting tutor_local_elasticsearch-permissions_1 ... done
Starting tutor_local_redis-permissions_1         ... done
Starting tutor_local_cms-permissions_1           ... done
Creating tutor_local_cms_run                     ... done
2022-07-21 22:13:07,623 WARNING 1 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

2022-07-21 22:13:07,695 WARNING 1 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  from collections import Mapping, MutableMapping

2022-07-21 22:13:10,124 WARNING 1 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/edx-platform/openedx/core/types/admin.py:49: DeprecationWarning: Django 3.2+ available: the _admin_display method and the AdminMethodclass should be removed from openedx.core.types
  warnings.warn(

2022-07-21 22:13:10,286 WARNING 1 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto.py:41: DeprecationWarning: The S3BotoStorage backend is deprecated in favor of the S3Boto3Storage backend and will be removed in django-storages 1.8. This backend is mostly in bugfix only mode and has been for quite a while (in much the same way as its underlying library 'boto'). For performance, security and new feature reasons it is _strongly_ recommended that you update to the S3Boto3Storage backend. Please see the migration docs https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#migrating-boto-to-boto3.
  warnings.warn(

2022-07-21 22:13:10,629 WARNING 1 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/swiftclient/client.py:84: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if StrictVersion(requests.__version__) < StrictVersion('2.0.0') \

System check identified some issues:

WARNINGS:
consent.DataSharingConsent.granted: (fields.W903) NullBooleanField is deprecated. Support for it (except in historical migrations) will be removed in Django 4.0.
	HINT: Use BooleanField(null=True) instead.
consent.HistoricalDataSharingConsent.granted: (fields.W903) NullBooleanField is deprecated. Support for it (except in historical migrations) will be removed in Django 4.0.
	HINT: Use BooleanField(null=True) instead.
2022-07-21 22:13:13,937 INFO 1 [openedx.core.djangoapps.content.learning_sequences.api.outlines] [user None] [ip None] outlines.py:389 - Replacing CourseOutline for course-v1:MyOrg+MyClass+MyRun1 (version 62d994bca7bf1b5486ec9997, 99 sequences)
2022-07-21 22:13:13,946 INFO 1 [openedx.core.djangoapps.content.learning_sequences.api.outlines] [user None] [ip None] outlines.py:433 - Created new CourseContext for course-v1:MyOrg+MyClass+MyRun1
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
openedx.core.djangoapps.content.learning_sequences.models.DoesNotExist: CourseSection matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
  File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x9A\\xAB O...' for column 'title' at row 1")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./manage.py", line 106, in <module>
    execute_from_command_line([sys.argv[0]] + django_args)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/openedx/edx-platform/cms/djangoapps/contentstore/management/commands/update_course_outline.py", line 27, in handle
    update_outline_from_modulestore(course_key)
  File "/openedx/edx-platform/cms/djangoapps/contentstore/outlines.py", line 385, in update_outline_from_modulestore
    replace_course_outline(course_outline_data, content_errors=content_errors)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/openedx/edx-platform/openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 405, in replace_course_outline
    _update_sections(course_outline, course_context)
  File "/openedx/edx-platform/openedx/core/djangoapps/content/learning_sequences/api/outlines.py", line 445, in _update_sections
    sec_model, _created = CourseSection.objects.update_or_create(
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 608, in update_or_create
    obj, created = self.select_for_update().get_or_create(defaults, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/openedx/venv/lib/python3.8/site-packages/model_utils/models.py", line 38, in save
    super().save(*args, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
  File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x9A\\xAB O...' for column 'title' at row 1")
ERROR: 1
Error: Command failed with status 1: docker-compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local run --rm cms ./manage.py cms update_course_outline course-v1:MyOrg+MyClass+MyRun1

I haven’t customized MySQL in any way.

When I do select distinct usage_key from learning_sequences_coursesection; I see entries for other courses, but not the specific one I was trying to view in this test. However, when I then try to view those other courses, I don’t see the original post error in the LMS logs, but the student account does see “There was an error loading this course.” (But it’s probably best to diagnose that after the initial error for learning sequences.)

pdpinch · July 22, 2022, 2:07pm

Yes. We’ve definitely seen the “Incorrect string value:” error associated with this before. In one case we tracked it down to emojis in the section title (see Emoji in subsection title not supported since Nutmeg) but I don’t think that’s everything we’re facing.

FWIW, I’ve tested similar content on edge.edx.org and it works fine, so it seems like it could be an issue with our MySQL configuration, and apparently, with Tutor’s as well?

dave · July 22, 2022, 2:22pm

Hmmmm… MySQL’s old UTF-8 encoding (which we still mostly use) doesn’t support emojis, because it’s only three-byte (as opposed to four). And learning_sequences would be the first time we’re storing section titles in MySQL, since they used to be stored only in MongoDB and in picked block structures collection files. MySQL’s utf8mb4 does support that, but I’m not sure how painful a migration would be at this point.

We do need to make that migration eventually. We should at least start making new tables as utf8mb4.

@Rohan: Does this sound like it could be the issue you’re seeing?

Rohan · July 22, 2022, 2:30pm

Arg, yes! I thought there were no emojis in any of the section or subsection titles, but I found one sign in a title, and when I removed it and re-ran
tutor local run cms ./manage.py cms update_course_outline <classkey>
then I could access the class as a student finally!

So for this class, removing the emoji is no big deal, but there’s another class where they’re intentionally used a lot to convey meaning and help with memorization. What’s the long term perspective on a fix for this then? Am I able to do something to the database to temporarily change it to accept utf8mb4 until a Nutmeg fix? Do I need to wait for a nutmeg fix? Do I need to wait for the next release after Nutmeg? (Because basically I can’t upgrade my production site without a fix for this.)

Thanks!

dave · July 22, 2022, 2:42pm

I think you should be able to drop into the database and manually ALTER the learning_sequences tables to use utf8mb4 for the character set and utf8mb4_unicode_ci for the collation, but I don’t know if there are Django settings that also need to change to make that work end-to-end.

It looks like this was discussed by @insad, @regis, and @iamCristYe in the Tutor forums: Utfmb4 instead of utf8mb3? (#442) - Tutor - Overhang.IO

Rohan · July 22, 2022, 2:48pm

That’s a bit beyond my MySQL knowledge. But based on this would I be doing (after a VM snapshot of course):

ALTER DATABASE
    openedx
    CHARACTER SET = utf8mb4
    COLLATE = utf8mb4_unicode_ci;

ALTER TABLE
    learning_sequences_coursesection
    CONVERT TO CHARACTER SET utf8mb4
    COLLATE utf8mb4_unicode_ci;

ALTER TABLE
    learning_sequences_coursesection
    CHANGE usage_key usage_key
    VARCHAR(191)
    CHARACTER SET utf8mb4
    COLLATE utf8mb4_unicode_ci;

ALTER TABLE
    learning_sequences_coursesection
    CHANGE title title
    VARCHAR(191)
    CHARACTER SET utf8mb4
    COLLATE utf8mb4_unicode_ci;

?

Any other tables/columns that would need changing?

dave · July 22, 2022, 3:00pm

I don’t know if altering the whole database will have other side-effects.

You definitely want to include at least:

learning_sequences_contenterror.message
learning_sequences_learningsequence.title

Another weird part of this is that changing the encoding will make the field larger, which will run into InnoDB issues if you have an older database table format (the reason we use 255 as a length everywhere is because 255 * 3-byte-encoding = 765 bytes < 767 byte max index size for old row formats). That limit was raised to something like 3K for newer table formats (DYNAMIC and COMPRESSED), so you might want to check what row format those tables are in before doing the encoding change.

And my apologies, but I’m probably not going to be able to reply to this for the rest of the day. Good luck!

insad · July 22, 2022, 7:30pm

ROW_FORMAT should be DYNAMIC, else InnoDB will choke on indices for varchar(255) columns

Abdess · July 22, 2022, 9:05pm

Just tested and it didn’t work.

Here’s what I’ve tried from a fresh installation:

docker exec -it tutor_local_mysql_1 bash

mysql -uroot -pXXXXXXXX

use openedx;

SHOW TABLE status WHERE Name='learning_sequences_contenterror' OR Name='learning_sequences_learningsequence' OR Name='learning_sequences_coursesection';

+-------------------------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+
| Name                                | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation       | Checksum | Create_options | Comment |
+-------------------------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+
| learning_sequences_contenterror     | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2022-07-22 19:47:12 | NULL        | NULL       | utf8_general_ci |     NULL |                |         |
| learning_sequences_coursesection    | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        32768 |         0 |              1 | 2022-07-22 19:47:11 | NULL        | NULL       | utf8_general_ci |     NULL |                |         |
| learning_sequences_learningsequence | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2022-07-22 19:47:11 | NULL        | NULL       | utf8_general_ci |     NULL |                |         |
+-------------------------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+
3 rows in set (0.00 sec)

ALTER DATABASE openedx CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_contenterror CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_coursesection CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_learningsequence CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_coursesection CHANGE usage_key usage_key varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_coursesection CHANGE title title varchar(1000) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_contenterror CHANGE message message LONGTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE learning_sequences_learningsequence CHANGE title title varchar(1000) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

SHOW TABLE status WHERE Name='learning_sequences_contenterror' OR Name='learning_sequences_learningsequence' OR Name='learning_sequences_coursesection';

+-------------------------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
| Name                                | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation          | Checksum | Create_options | Comment |
+-------------------------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
| learning_sequences_contenterror     | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2022-07-22 20:37:10 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL |                |         |
| learning_sequences_coursesection    | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        32768 |         0 |              1 | 2022-07-22 20:37:10 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL |                |         |
| learning_sequences_learningsequence | InnoDB |      10 | Dynamic    |    0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2022-07-22 20:37:11 | NULL        | NULL       | utf8mb4_unicode_ci |     NULL |                |         |
+-------------------------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
3 rows in set (0.00 sec)

Creation of a TEST course in the studio and then import of a dummy course with emojis: https://transfer.sh/8HPa1U/course.emoji_test.tar.gz

Logs: Nutmeg : Emoji Title Logs - Pastebin.com

Abdess · July 23, 2022, 10:37am

Please excuse me for the double post.

I transmit here the results of another experiment where I create a course without import from a fresh installation.

After creating a course where I put “TEST” everywhere, the logs show this:

cms-worker_1                 | [2022-07-23 10:12:27,777: INFO/ForkPoolWorker-8] Attempting to load CourseOverview for course course-v1:TEST+TEST+TEST from modulestore.
cms-worker_1                 | [2022-07-23 10:12:27,782: INFO/ForkPoolWorker-8] Could not create CourseOverview for non-existent course: course-v1:TEST+TEST+TEST
cms-worker_1                 | [2022-07-23 10:12:27,782: ERROR/ForkPoolWorker-7] Task openedx.core.djangoapps.bookmarks.tasks.update_xblocks_cache[8d60bb47-5ecc-4598-832c-5d03885261de] raised unexpected: AttributeError("'NoneType' object has no attribute 'has_children'")
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
cms-worker_1                 |     R = retval = fun(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
cms-worker_1                 |     return self.run(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
cms-worker_1                 |     return wrapped_function(*args, **kwargs)
elasticsearch_1              | {"type": "server", "timestamp": "2022-07-23T10:12:27,783Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "openedx", "node.name": "c2011c622c67", "message": "[courseware_content] creating index, cause [api], templates [], shards [1]/[1]", "cluster.uuid": "5-p93KviS6Sw0_DcFs7-5g", "node.id": "pSkFQpa6TqayTxas5HhN_Q"  }
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 163, in update_xblocks_cache
cms-worker_1                 |     _update_xblocks_cache(course_key)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 115, in _update_xblocks_cache
cms-worker_1                 |     blocks_data = _calculate_course_xblocks_data(course_key)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 35, in _calculate_course_xblocks_data
cms-worker_1                 |     children = current_block.get_children() if current_block.has_children else []
cms-worker_1                 | AttributeError: 'NoneType' object has no attribute 'has_children'
cms-worker_1                 | [2022-07-23 10:12:27,785: ERROR/ForkPoolWorker-8] Task openedx.core.djangoapps.course_apps.tasks.update_course_apps_status[7919d86a-f7e8-48e4-9104-676b265068ac] raised unexpected: DoesNotExist()
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
cms-worker_1                 |     R = retval = fun(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
cms-worker_1                 |     return self.run(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
cms-worker_1                 |     return wrapped_function(*args, **kwargs)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/course_apps/tasks.py", line 43, in update_course_apps_status
cms-worker_1                 |     is_enabled = course_app.is_enabled(course_key=course_key)
cms-worker_1                 |   File "/openedx/edx-platform/lms/djangoapps/courseware/plugins.py", line 145, in is_enabled
cms-worker_1                 |     return CourseOverview.get_from_id(course_key).show_calculator
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/lib/cache_utils.py", line 74, in decorator
cms-worker_1                 |     result = wrapped(*args, **kwargs)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/content/course_overviews/models.py", line 413, in get_from_id
cms-worker_1                 |     return course_overview or cls.load_from_module_store(course_id)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/content/course_overviews/models.py", line 355, in load_from_module_store
cms-worker_1                 |     raise cls.DoesNotExist()
cms-worker_1                 | openedx.core.djangoapps.content.course_overviews.models.CourseOverview.DoesNotExist
cms-worker_1                 | [2022-07-23 10:12:27,987: INFO/ForkPoolWorker-1] PUT http://elasticsearch:9200/courseware_content [status:200 request:0.222s]
cms-worker_1                 | [2022-07-23 10:12:28,013: ERROR/ForkPoolWorker-1] Indexing error encountered, courseware index may be out of date course-v1:TEST+TEST+TEST - AttributeError("'NoneType' object has no attribute 'id'")
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 251, in index
cms-worker_1                 |     groups_usage_info = cls.fetch_group_usage(modulestore, structure)
cms-worker_1                 |   File "/openedx/edx-platform/cms/djangoapps/contentstore/courseware_index.py", line 396, in fetch_group_usage
cms-worker_1                 |     partitions_info = GroupConfiguration.get_partitions_usage_info(modulestore, structure)
cms-worker_1                 |   File "/openedx/edx-platform/cms/djangoapps/contentstore/course_group_config.py", line 233, in get_partitions_usage_info
cms-worker_1                 |     items = store.get_items(course.id, settings={'group_access': {'$exists': True}}, include_orphans=False)
cms-worker_1                 | AttributeError: 'NoneType' object has no attribute 'id'
cms-worker_1                 | [2022-07-23 10:12:28,015: ERROR/ForkPoolWorker-1] cms.djangoapps.contentstore.tasks.update_search_index[8b6a4a91-b00e-43ab-87ff-6b110cccab0e]: Search indexing error for complete course course-v1:TEST+TEST+TEST - Error(s) present during indexing - ["Une erreur générale d’indexation s'est produite"]
cms-worker_1                 | [2022-07-23 10:12:28,016: INFO/ForkPoolWorker-1] Task cms.djangoapps.contentstore.tasks.update_search_index[8b6a4a91-b00e-43ab-87ff-6b110cccab0e] succeeded in 0.3513652409965289s: None
cms_1                        | 2022-07-23 10:12:28,022 INFO 25 [openedx_events.tooling] [user 4] [ip 83.114.154.223] tooling.py:160 - Responses of the Open edX Event <org.openedx.learning.course.enrollment.changed.v1>:

Then here is the log when I create a section “ Section”, there is no error at this point :

cms_1                        | 2022-07-23 10:19:31,492 INFO 25 [celery_utils.logged_task] [user 4] [ip 83.114.154.223] logged_task.py:25 - Task lms.djangoapps.discussion.tasks.update_discussions_map[bf1683d9-5216-465d-b140-d3d55ed534c3] submitted with arguments [{'course_id': 'course-v1:TEST+TEST+TEST'}], None
cms_1                        | 2022-07-23 10:19:31,492 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function listen_for_course_publish at 0x7f871a186550> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,493 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function export_course_metadata at 0x7f871a1123a0> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,493 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function _listen_for_course_publish at 0x7f871951bd30> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,493 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function _listen_for_course_publish at 0x7f871951bf70> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,493 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function update_block_structure_on_course_publish at 0x7f871952b310> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,494 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function _listen_for_course_publish at 0x7f8719532ee0> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,494 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function extract_dates at 0x7f87193cd3a0> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,494 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function trigger_update_xblocks_cache_task at 0x7f87193baa60> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms-worker_1                 | [2022-07-23 10:19:31,493: INFO/MainProcess] Task lms.djangoapps.discussion.tasks.update_discussions_map[bf1683d9-5216-465d-b140-d3d55ed534c3] received
cms_1                        | 2022-07-23 10:19:31,494 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function update_course_apps at 0x7f87193c1670> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
cms_1                        | 2022-07-23 10:19:31,494 INFO 25 [xmodule.modulestore.django] [user 4] [ip 83.114.154.223] django.py:199 - Sent course_published signal to <function update_discussions_on_course_publish at 0x7f871934e790> with kwargs {'course_key': CourseLocator('TEST', 'TEST', 'TEST', None, None)}. Response was: None
caddy_1                      | {"level":"info","ts":1658571571.5023177,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"172.19.0.1:52136","proto":"HTTP/1.0","method":"POST","host":"xxx.xxxx.xx","uri":"/xblock/block-v1:TEST+TEST+TEST+type@chapter+block@5b6f2788316f4b5592fcb0bc9f77c546"},"user_id":"","duration":0.164858744,"size":169,"status":200}
cms_1                        | [pid: 25|app: 0|req: 22/130] 172.19.0.3 () {70 vars in 3764 bytes} [Sat Jul 23 10:19:31 2022] POST /xblock/block-v1:TEST+TEST+TEST+type@chapter+block@5b6f2788316f4b5592fcb0bc9f77c546 => generated 169 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 295 bytes (1 switches on core 0)
cms-worker_1                 | [2022-07-23 10:19:31,504: INFO/ForkPoolWorker-7] Creating XBlockCache with usage_key: block-v1:TEST+TEST+TEST+type@chapter+block@5b6f2788316f4b5592fcb0bc9f77c546
cms-worker_1                 | [2022-07-23 10:19:31,507: INFO/ForkPoolWorker-7] Ending XBlockCaches update for course_key: course-v1:TEST+TEST+TEST
cms-worker_1                 | [2022-07-23 10:19:31,508: INFO/ForkPoolWorker-7] Task openedx.core.djangoapps.bookmarks.tasks.update_xblocks_cache[f9466a90-b5d4-4854-b165-b30e0cacd07d] succeeded in 0.026728827004262712s: None
cms-worker_1                 | [2022-07-23 10:19:31,530: WARNING/ForkPoolWorker-1] Flag 'course_live.enable_course_live' accessed without a request, which is likely in the context of a celery task.
cms-worker_1                 | [2022-07-23 10:19:31,536: WARNING/ForkPoolWorker-1] Flag 'teams.enable_teams_app' accessed without a request, which is likely in the context of a celery task.

But when I just click on create a new subsection I get this :

cms-worker_1                 | [2022-07-23 10:22:08,684: ERROR/ForkPoolWorker-7] Task openedx.core.djangoapps.bookmarks.tasks.update_xblocks_cache[96ade948-cc31-4e54-9e05-2c8814b2e000] raised unexpected: OperationalError(1366, "Incorrect string value: '\\xF0\\x9F\\x93\\xBA S...' for column 'display_name' at row 1")
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
cms-worker_1                 |     return self.cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
cms-worker_1                 |     return self.cursor.execute(query, args)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
cms-worker_1                 |     res = self._query(query)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
cms-worker_1                 |     db.query(q)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
cms-worker_1                 |     _mysql.connection.query(self, query)
cms-worker_1                 | MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x93\\xBA S...' for column 'display_name' at row 1")
cms-worker_1                 |
cms-worker_1                 | The above exception was the direct cause of the following exception:
cms-worker_1                 |
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
cms-worker_1                 |     R = retval = fun(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
cms-worker_1                 |     return self.run(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
cms-worker_1                 |     return wrapped_function(*args, **kwargs)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 163, in update_xblocks_cache
cms-worker_1                 |     _update_xblocks_cache(course_key)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 131, in _update_xblocks_cache
cms-worker_1                 |     update_block_cache_if_needed(block_cache, block_data)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 124, in update_block_cache_if_needed
cms-worker_1                 |     block_cache.save()
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/model_utils/models.py", line 38, in save
cms-worker_1                 |     super().save(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 739, in save
cms-worker_1                 |     self.save_base(using=using, force_insert=force_insert,
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 776, in save_base
cms-worker_1                 |     updated = self._save_table(
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 858, in _save_table
cms-worker_1                 |     updated = self._do_update(base_qs, using, pk_val, values, update_fields,
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 912, in _do_update
cms-worker_1                 |     return filtered._update(values) > 0
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 802, in _update
cms-worker_1                 |     return query.get_compiler(self.db).execute_sql(CURSOR)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1559, in execute_sql
cms-worker_1                 |     cursor = super().execute_sql(result_type)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cms-worker_1                 |     cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
cms-worker_1                 |     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
cms-worker_1                 |     return executor(sql, params, many, context)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
cms-worker_1                 |     return self.cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
cms-worker_1                 |     raise dj_exc_value.with_traceback(traceback) from exc_value
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
cms-worker_1                 |     return self.cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
cms-worker_1                 |     return self.cursor.execute(query, args)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
cms-worker_1                 |     res = self._query(query)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
cms-worker_1                 |     db.query(q)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
cms-worker_1                 |     _mysql.connection.query(self, query)
cms-worker_1                 | django.db.utils.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x93\\xBA S...' for column 'display_name' at row 1")

The error reappears when putting an emoji in a sub-section but does not appear when I put an emoji in unit.

The error comes back when publishing.

And the course outline does not appear in the course.
Something that disturbs me is that when I had realized this approach before, the course outline had worked.

After updating the collation to utf8mb4, exactly the same thing happens.

UPDATE :
I just tested @insad solution (here: Utfmb4 instead of utf8mb3? (#442) - Tutor - Overhang.IO) by converting the whole DB to utf8mb4 (with all tables and columns). No difference after retrying the experiment.

insad · July 23, 2022, 1:45pm

Did you make the rowtype DYNAMIC while converting to utf8mb4 (see MySQL :: MySQL 5.7 Reference Manual :: 14.11 InnoDB Row Formats)?
If not, all indexed varchar(255) fields will cause a crash (can’t have indexed varchar fields bigger than 192 chars in the COMPACT row format).

dave · July 24, 2022, 12:33am

I think you might have to also change the Django database connection settings to explicitly set charset in options (so DATABASES['default']['OPTIONS']['charset'] = 'utf8mb4').

But I don’t know what kind of side effects this has elsewhere when trying to operate on old tables, or whether it means that you’d have to translate the entire database over to utf8mb4 at the same time.

Abdess · July 25, 2022, 3:51pm

@insad yes I made the rowtype DYNAMIC while converting to utf8mb4

@dave Guess what? It works!

I imported a course to test, 0 error in the logs, wonderful!

I’ll run the test again to check and then make an update.

Rohan · July 25, 2022, 4:38pm

Very good job! I haven’t tested it on my system yet, but once you test again and confirm the steps if you could put them all here, I’ll then test them all on my setup as well.

pdpinch · July 25, 2022, 7:13pm

Hey that’s great. What do we need to commit in tutor and/or edx-platform to address this for everyone?

Abdess · July 25, 2022, 7:52pm

I just finished a whole set of tests.

I confirm that @insad’s method complemented with @dave’s method works.

So here’s what I did in a nutshell:

From @iamCristYe’s message : Utfmb4 instead of utf8mb3? (#442) - #17 by iamCristYe - Tutor - Overhang.IO. In the auth.yml file, I added “charset: "utf8mb4"” at line 21. The file has been modified from this directory:

.local/lib/python3.x/site-packages/tutor/templates/apps/openedx/config/partials/auth.yml

DATABASES:
  default:
    ENGINE: "django.db.backends.mysql"
    HOST: "{{ MYSQL_HOST }}"
    PORT: {{ MYSQL_PORT }}
    NAME: "{{ OPENEDX_MYSQL_DATABASE }}"
    USER: "{{ OPENEDX_MYSQL_USERNAME }}"
    PASSWORD: "{{ OPENEDX_MYSQL_PASSWORD }}"
    ATOMIC_REQUESTS: true
    OPTIONS:
      init_command: "SET sql_mode='STRICT_TRANS_TABLES'"
      charset: "utf8mb4"

Update of the tutor environment:

tutor config save

Start tutor

tutor local start

Backup of MySQL schema:

docker exec tutor_local_mysql_1 /usr/bin/mysqldump -Q -d -uroot -pXXXXXXXX --default-character-set=utf8 --skip-set-charset openedx | sed 's/utf8/utf8mb4/gi' | sed 's/utf8mb4mb4/utf8mb4/gi' > openedx_schema.sql

Backup of MySQL data:

docker exec tutor_local_mysql_1 /usr/bin/mysqldump -Q --insert-ignore -t -uroot -pXXXXXXXX --default-character-set=utf8 --skip-set-charset openedx > openedx_data.sql

Then the database is deleted and restored:

docker exec -it tutor_local_mysql_1 bash

mysql -uroot -pXXXXXXXX

DROP DATABASE openedx;

create database openedx default charset utf8mb4 collate utf8mb4_general_ci;

exit

exit

cat openedx_schema.sql | docker exec -i tutor_local_mysql_1 /usr/bin/mysql -u root --password=XXXXXXXX openedx

cat openedx_data.sql | docker exec -i tutor_local_mysql_1 /usr/bin/mysql -u root --password=XXXXXXXX openedx

And now course outline with emojis works again. The whole thing looks stable. I haven’t encountered any problem yet.

The format of all rows seems to be DYNAMIC according to the feedback of this query:

SELECT * FROM information_schema.INNODB_SYS_TABLESPACES;

Topic		Replies	Views
Problem refreshing content and creating course in Quince Site Operations Help how-to , lms , quince	3	158	June 25, 2024
Error with preview and notes Site Operations Help	1	279	December 14, 2022
Anonymous Course Access in Nutmeg Site Operations Help nutmeg	15	930	December 12, 2022
Tutor: Potential upgrade problem from Maple to Nutmeg with user tours? Site Operations Help tutor	11	1340	September 27, 2022
Juniper devstack installation: course_discovery.apps.course_metadata.models.DoesNotExist: Course matching query does not exist Development juniper	0	347	December 22, 2021

Nutmeg upgrade: outlines.py: "openedx.core.djangoapps.content.learning_sequences.models.LearningContext.DoesNotExist: LearningContext matching query does not exist"

Related topics