Emoji in subsection title not supported since Nutmeg

Hello,

Since the migration from Maple to Nutmeg, the contents with an emoji in the title of the subsection cause a crash in the logs and prevent the course outline from being displayed.

On a new installation the problem remains.

When creating a new course, the problem does not appear. But when exporting the same course, its import causes an error.

I tried to convert the mysql database to utf8mb4 but the error still persists.

Example of creating a test course with and without emoji:

![image|690x383]https://discuss.overhang.io/uploads/default/original/2X/2/21f9dd6e14aa7f06a69b8411967bb46dd9e50fb4.png)

Course Outline does not display the content of the sections:

(as a new user I am only allowed to upload one media, so I’m going to upload the other screenshots elsewhere
EDIT: It seems that this is impossible, as a new user I am limited to 2 links per post. Sorry about that. :sweat_smile:)

What happens after clicking on “start course”:

CMS Logs:

cms-worker_1                 | [2022-07-18 16:44:09,819: ERROR/ForkPoolWorker-8] Task openedx.core.djangoapps.bookmarks.tasks.update_xblocks_cache[54a02550-46e0-48a2-b4ef-0f576eee5c49] raised unexpected: OperationalError(1366, "Incorrect string value: '\\xF0\\x9F\\x93\\xBA S...' for column 'display_name' at row 1")
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
cms-worker_1                 |     return self.get(**kwargs), False
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
cms-worker_1                 |     raise self.model.DoesNotExist(
cms-worker_1                 | openedx.core.djangoapps.bookmarks.models.XBlockCache.DoesNotExist: XBlockCache matching query does not exist.
cms-worker_1                 |
cms-worker_1                 | During handling of the above exception, another exception occurred:
cms-worker_1                 |
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
cms-worker_1                 |     return self.cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
cms-worker_1                 |     return self.cursor.execute(query, args)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
cms-worker_1                 |     res = self._query(query)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
cms-worker_1                 |     db.query(q)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
cms-worker_1                 |     _mysql.connection.query(self, query)
cms-worker_1                 | MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x93\\xBA S...' for column 'display_name' at row 1")
cms-worker_1                 |
cms-worker_1                 | The above exception was the direct cause of the following exception:
cms-worker_1                 |
cms-worker_1                 | Traceback (most recent call last):
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
cms-worker_1                 |     R = retval = fun(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
cms-worker_1                 |     return self.run(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
cms-worker_1                 |     return wrapped_function(*args, **kwargs)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 163, in update_xblocks_cache
cms-worker_1                 |     _update_xblocks_cache(course_key)
cms-worker_1                 |   File "/openedx/edx-platform/openedx/core/djangoapps/bookmarks/tasks.py", line 137, in _update_xblocks_cache
cms-worker_1                 |     block_cache, created = XBlockCache.objects.get_or_create(usage_key=block_data['usage_key'], defaults={
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
cms-worker_1                 |     return getattr(self.get_queryset(), name)(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 588, in get_or_create
cms-worker_1                 |     return self.create(**params), True
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 453, in create
cms-worker_1                 |     obj.save(force_insert=True, using=self.db)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/model_utils/models.py", line 38, in save
cms-worker_1                 |     super().save(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 739, in save
cms-worker_1                 |     self.save_base(using=using, force_insert=force_insert,
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 776, in save_base
cms-worker_1                 |     updated = self._save_table(
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 881, in _save_table
cms-worker_1                 |     results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 919, in _do_insert
cms-worker_1                 |     return manager._insert(
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
cms-worker_1                 |     return getattr(self.get_queryset(), name)(*args, **kwargs)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1270, in _insert
cms-worker_1                 |     return query.get_compiler(using=using).execute_sql(returning_fields)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
cms-worker_1                 |     cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
cms-worker_1                 |     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
cms-worker_1                 |     return executor(sql, params, many, context)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
cms-worker_1                 |     return self.cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
cms-worker_1                 |     raise dj_exc_value.with_traceback(traceback) from exc_value
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
cms-worker_1                 |     return self.cursor.execute(sql, params)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
cms-worker_1                 |     return self.cursor.execute(query, args)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
cms-worker_1                 |     res = self._query(query)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
cms-worker_1                 |     db.query(q)
cms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
cms-worker_1                 |     _mysql.connection.query(self, query)
cms-worker_1                 | django.db.utils.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x93\\xBA S...' for column 'display_name' at row 1")

LMS Logs on course outline page:

caddy_1                      | {"level":"error","ts":1658162672.7590897,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"172.18.0.1:49142","proto":"HTTP/1.0","method":"GET","host":"lms.localhost.fr","uri":"/api/edx_proctoring/v1/user_onboarding/status?is_learning_mfe=true&course_id=course-v1%3ATEST%2BTEST%2BTEST&username=usertest"},"user_id":"","duration":0.048045005,"size":83,"status":404}

Course backup example : hxxps://transfer.sh/8HPa1U/course.emoji_test.tar.gz

Do you have any clues?
I remain available for testing.

Thanks :smile:

Thanks for posting this. We have encountered the same issue.

It seems to be an issue with writing the course overview to SQL. We tried changing the database collation, similar to what you did but it didn’t help.

What’s puzzling is that the emoji seem to work fine in our courses on edx.org. But on our MIT campus server, running open edX nutmeg, it crashes.

1 Like

@abdess it looks like you are running tutor – is that correct?

For future reference: this thread was root caused to being due to lack of emoji support in the mysql DB as well.

1 Like

Sorry for the late. Glad to see I’m not the only one to encounter this issue. :slight_smile:
Indeed, that’s puzzling ! :thinking:

Yes, I’m running tutor 14.0.3.

Thank you @Rohan, I’m going to check this thread.

@jmakowski @dave
I’ve had many instructional designers as if we can add in icons or emojis to the course outline section and subsection headers. This article mentions emojis, however, it appears that this may cause a database issue.

Do you have anything in development on adding in icons via a settings interface for future releases?