Translate the default XBlock data

omar · August 7, 2019, 3:57pm

There’s a problem in the Open edX tooling in which that the XBlocks fields are always initialized with the default data before the i18n tooling is activated.

This is bad for non-English course authors, especially that we already have all of those strings in Transifex. However, there’s no way to use them. I’ll explain more below.

So I’ve created a hack to fix the problem:

https://github.com/appsembler/edx-platform/pull/434

As-is it works well so far. However, I wouldn’t merge it in my fork until I find a better method of doing so, I would love your help and feedback.

The hack does the following:

When creating a XBlock new instance __init__(), for every field, translate the value again.

This only concerns the XBlock fields since the .html templates and JavaScript can already be translated.

What’s Exactly the Issue with XBlocks:

An XBlock is defined as follows:

def _(text):
    """
    A noop underscore function that marks strings for extraction.
    """
    return text

@XBlock.needs("i18n")
class DoneXBlock(XBlock, CompletableXBlockMixin):
    display_name = String(
        help=_("The display name for this component."),
        scope=Scope.settings
    )

    message = String(
        default=_("Mark as done"),
        help=_("The display name for this component."),
        scope=Scope.settings
    )

    done = Boolean(
        scope=Scope.user_state,
        help=_("Is the student done?"),
        default=False
    )

It’s almost the pattern for all XBlocks – to use a dummy function to extract the strings but not to translate them. The reason that a dummy function is used from django.utils.translation import ugettext as _ cannot be used since the XBlocks are initialized at a very early stage in the Django startup functions, causing the ugettext to panic with an Exception. Please correct me if I’m wrong, and do expand if you have a better idea.

braden · August 8, 2019, 4:37pm

@omar Are you aware of this: https://openedx.atlassian.net/wiki/spaces/COMM/pages/753532962/Native+XBlocks+Morning+Session#NativeXBlocksMorningSession-XBlockInternationalization

I was under the impression that what it says there (make sure the XBlock renders templates using ResourceLoader.render_django_template, etc.) is sufficient to get all of an XBlock’s HTML + JS code internationalized and localized.

omar · August 9, 2019, 5:11am

Thanks @braden for chiming in. I’ve checked the wiki page and what it says (thank you for working on t that) still holds true. The HTML/JS code can indeed be i18n’ed, even Python strings can be i18n’ed like below:

class MyXBlock(...):
    ...
    def get_something_for_html(self):
        ugettext = self.runtime.service(self, "i18n").ugettext
        return ugettext("I'm translatable!")

However, there’s no way (yet) to translate this:
ugettext_noop = lambda s: s
class MyXBlock(…):
display_name = String(
display_name=ugettext_noop(“Display name.”),
help=ugettext_noop(“The display name for this component.”),
scope=Scope.settings
)

First, we have to use an actual ugettext function instead of noop, ugettext_lazy can work. But when it’s used the following error shows up:

edx.devstack.studio | Traceback (most recent call last):
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/cms/djangoapps/contentstore/views/preview.py", line 326, in get_preview_fragment
edx.devstack.studio |     fragment = module.render(preview_view, context)
edx.devstack.studio |   File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/xblock/core.py", line 202, in render
edx.devstack.studio |     return self.runtime.render(self, view, context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1903, in render
edx.devstack.studio |     return self.__getattr__('render')(block, view_name, context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1310, in render
edx.devstack.studio |     return super(MetricsMixin, self).render(block, view_name, context=context)
edx.devstack.studio |   File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/xblock/runtime.py", line 810, in render
edx.devstack.studio |     frag = view_fn(context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/vertical_block.py", line 119, in author_view
edx.devstack.studio |     self.render_children(context, fragment, can_reorder=True, can_add=True)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/studio_editable.py", line 26, in render_children
edx.devstack.studio |     rendered_child = child.render(StudioEditableModule.get_preview_view_name(child), context)
edx.devstack.studio |   File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/xblock/core.py", line 202, in render
edx.devstack.studio |     return self.runtime.render(self, view, context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1903, in render
edx.devstack.studio |     return self.__getattr__('render')(block, view_name, context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1310, in render
edx.devstack.studio |     return super(MetricsMixin, self).render(block, view_name, context=context)
edx.devstack.studio |   File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/xblock/runtime.py", line 814, in render
edx.devstack.studio |     updated_frag = self.wrap_xblock(block, view_name, frag, context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1262, in wrap_xblock
edx.devstack.studio |     frag = wrapper(block, view, frag, context)
edx.devstack.studio |   File "/edx/app/edxapp/edx-platform/openedx/core/lib/xblock_utils/__init__.py", line 146, in wrap_xblock
edx.devstack.studio |     template_context['js_init_parameters'] = json.dumps(frag.json_init_args).replace("/", r"\/")
edx.devstack.studio |   File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps
edx.devstack.studio |     return _default_encoder.encode(obj)
edx.devstack.studio |   File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
edx.devstack.studio |     chunks = self.iterencode(o, _one_shot=True)
edx.devstack.studio |   File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
edx.devstack.studio |     return _iterencode(o, 0)
edx.devstack.studio |   File "/usr/lib/python2.7/json/encoder.py", line 184, in default
edx.devstack.studio |     raise TypeError(repr(o) + " is not JSON serializable")
edx.devstack.studio | TypeError: u'Drag and Drop' is not JSON serializable

I think the solution is to use something like this in the XBlock Utils, but I don’t know if that will work for all XBlocks, therefore I’m opening this for discussion:

For now the issue is that because the XBlock Utils is unable to deal with Promise objects resulting from ugettext_lazy. One solution is to use a custom JSON encoder like:

from rest_framework.utils.encoders import JSONEncoder
value = json.dumps(..., cls=JSONEncoder)

What do you think?

braden · August 9, 2019, 5:55pm

Are you using StudioEditableXBlockMixin? Because it should handle that case:

First, strings to translate are marked with a dummy (no-op) function only. We want them extracted into .po files, but we do not want them actually translated (lazily or not) because of the Promise/JSON encoder/Django initialization errors that you mention.
Next, strings that may or may not be marked for dummy translation are explicitly translated using ugettext immediately before render:
```
'display_name': ugettext(field.display_name) if field.display_name else "",
'help': ugettext(field.help) if field.help else "",
```

If you are using StudioEditableMixin, this should already be working. If you aren’t, you just need to wrap your help and display_name fields in an additional call to ugettext inside your studio_view method.

omar · August 10, 2019, 7:48pm

Thanks. That’s understood. The dummy function works well as we can see the strings being extracted and pushed to Transifex.

I haven’t covered all of the details yet but looks like the Promise/JSON issue has a much better method to solve that exception, which is to use a better JSONEncoder

Thanks for the extra info, I wasn’t aware of the StudioEditableMixin. It looks like the Poll XBlock can make use of it since it doesn’t use it.

Yes, the StudioEditableXBlockMixin is used in most XBlocks including DiscussionXBlock, which always shows Week 1 as the default value for discussion_category, because the XBlock utils does not translate that default value.

I’m looking for something that’s more rooted in the XBlocks and not just in Studio, also definitely not just for the field names. The hack I provided definitely does what I want – in the worst possible way!

Ideally I would like to be able to just use uggettext_lazy as we do in Django Forms and other Django-based code. What if we used a JSONEncoder that supports Promise, that should work even for complex default values (example from the Poll XBlock).

What do you think?

Topic		Replies	Views
Free Text Response xblock not translated Site Operations Help i18n	11	1412	January 23, 2021
Translating Strings from DB Fields i18n i18n	2	236	September 13, 2023
Use the translation in Transifex for Ukrainian for my Open edX site Site Operations Help	6	202	February 25, 2024
Translations issues with the frontend-app-learner-dashboard i18n	10	274	February 19, 2024
Override default edx-platform XBlock mappings without forking Development	11	238	August 19, 2024

Translate the default XBlock data

Related Topics