Issues developing a non-core MFE using Tutor

Bringing a discussion from Slack over to the forum at the request of @regis.

My team has been successfully developing an internal, non-core MFE using Tutor v14 and the Nutmeg release. While preparing our upgrade to Olive and the MFE runtime config I discovered that MFE development under Tutor v15 may not be possible, so I jumped to Tutor v16 to see if I could get things working there.

I added my MFE using the method described in the readme.

# plugin_root/

from tutormfe.hooks import MFE_APPS

def _add_my_mfe(mfes):
    mfes["mogc-partners"] = {
        "repository": "",
        "version": "palm-test",
        "port": 1998,
    return mfes

Then, following the instructions in the readme, I added a mount with the appropriate name and ran tutor dev launch, which resulted in Error: Image 'mogc-partners-dev' could not be found. Full output below.

❯ tutor dev launch
        Interactive platform configuration
As you are not running this platform in production, we automatically set the following configuration values:
    LMS_HOST =
    CMS_HOST =
    ENABLE_HTTPS = False
Your platform name/title [My Open edX] 
Your public contact email address [] 
The default language code for the platform [en] 
Configuration saved to /Users/mpwheel/projects/tmp/palm-partner-portal/tutor_root/config.yml
Environment generated in /Users/mpwheel/projects/tmp/palm-partner-portal/tutor_root/env
        Building Docker images
Building image openedx-dev:16.1.7
docker buildx build --tag=openedx-dev:16.1.7 --output=type=docker --target=development --build-arg=APP_USER_ID=501 --cache-from=type=registry,ref=openedx-dev:16.1.7-cache /Users/mpwheel/projects/tmp/palm-partner-portal/tutor_root/env/build/openedx
[+] Building 2.1s (67/67) FINISHED                                                                                                                                    docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                  0.0s
 => => transferring dockerfile: 11.56kB                                                                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                                                                     0.0s
 => => transferring context: 2B                                                                                                                                                       0.0s
 => resolve image config for                                                                                                                          1.0s
 => CACHED docker-image://                                                     0.0s
 => [internal] load metadata for                                                                                                                  0.7s
 => [internal] load metadata for                                                                                                                       0.6s
 => ERROR importing cache manifest from openedx-dev:16.1.7-cache                                                                                                                      0.2s
 => [internal] load build context                                                                                                                                                     0.0s
 => => transferring context: 21.11kB                                                                                                                                                  0.0s
 => [minimal 1/2] FROM                                                         0.0s
 => FROM                                                                  0.0s
 => CACHED [minimal 2/2] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked     apt update &&     apt install -y   0.0s
 => CACHED [production  1/30] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked apt update     && apt install -y  0.0s
 => CACHED [production  2/30] RUN if [ "501" = 0 ]; then echo "app user may not be root" && false; fi                                                                                 0.0s
 => CACHED [production  3/30] RUN useradd --no-log-init --home-dir /openedx --create-home --shell /bin/bash --uid 501 app                                                             0.0s
 => CACHED [production  4/30] COPY --link /usr/local/bin/dockerize /usr/local/bin/dockerize                                                0.0s
 => CACHED [code 1/3] RUN mkdir -p /openedx/edx-platform &&     git clone --branch open-release/palm.4 --depth 1 /openedx/edx-platform    0.0s
 => CACHED [code 2/3] WORKDIR /openedx/edx-platform                                                                                                                                   0.0s
 => CACHED [code 3/3] RUN git config --global ""   && git config --global "Tutor"                                                               0.0s
 => CACHED [edx-platform 1/1] COPY --from=code /openedx/edx-platform /                                                                                                                0.0s
 => CACHED [production  5/30] COPY --chown=app:app --from=edx-platform / /openedx/edx-platform                                                                                        0.0s
 => CACHED [locales 1/1] RUN cd /tmp     && curl -L -o openedx-i18n.tar.gz     && tar xzf /tmp/openedx-i1  0.0s
 => CACHED [production  6/30] COPY --chown=app:app --from=locales /openedx/locale /openedx/locale                                                                                     0.0s
 => CACHED [python 1/4] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked apt update &&     apt install -y libss  0.0s
 => CACHED [python 2/4] RUN git clone /opt/pyenv --branch v2.3.17 --depth 1                                                                            0.0s
 => CACHED [python 3/4] RUN /opt/pyenv/bin/pyenv install 3.8.15                                                                                                                       0.0s
 => CACHED [python 4/4] RUN /opt/pyenv/versions/3.8.15/bin/python -m venv /openedx/venv                                                                                               0.0s
 => CACHED [production  7/30] COPY --chown=app:app --from=python /opt/pyenv /opt/pyenv                                                                                                0.0s
 => CACHED [python-requirements 1/8] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked apt update     && apt ins  0.0s
 => CACHED [python-requirements 2/8] RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared pip install     setuptools==67.6.1 pip==23.0.1. wheel==0.40.0                   0.0s
 => CACHED [python-requirements 3/8] RUN pip install                                           0.0s
 => CACHED [python-requirements 4/8] RUN --mount=type=bind,from=edx-platform,source=/requirements/edx/base.txt,target=/openedx/edx-platform/requirements/edx/base.txt     --mount=ty  0.0s
 => CACHED [python-requirements 5/8] RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared pip install     django-redis==5.2.0     uwsgi==2.0.21                           0.0s
 => CACHED [python-requirements 6/8] COPY ./requirements/ /openedx/requirements                                                                                                       0.0s
 => CACHED [python-requirements 7/8] RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared cd /openedx/requirements/   && touch ./private.txt   && pip install -r ./priva  0.0s
 => CACHED [python-requirements 8/8] RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared pip install 'openedx-scorm-xblock>=16.0.0,<17.0.0'                              0.0s
 => CACHED [production  8/30] COPY --chown=app:app --from=python-requirements /openedx/venv /openedx/venv                                                                             0.0s
 => CACHED [production  9/30] COPY --chown=app:app --from=python-requirements /openedx/requirements /openedx/requirements                                                             0.0s
 => CACHED [nodejs-requirements 1/4] RUN pip install nodeenv==1.8.0                                                                                                                   0.0s
 => CACHED [nodejs-requirements 2/4] RUN nodeenv /openedx/nodeenv --node=16.14.0 --prebuilt                                                                                           0.0s
 => CACHED [nodejs-requirements 3/4] WORKDIR /openedx/edx-platform                                                                                                                    0.0s
 => CACHED [nodejs-requirements 4/4] RUN --mount=type=bind,from=edx-platform,source=/package.json,target=/openedx/edx-platform/package.json     --mount=type=bind,from=edx-platform,  0.0s
 => CACHED [production 10/30] COPY --chown=app:app --from=nodejs-requirements /openedx/nodeenv /openedx/nodeenv                                                                       0.0s
 => CACHED [production 11/30] COPY --chown=app:app --from=nodejs-requirements /openedx/edx-platform/node_modules /openedx/node_modules                                                0.0s
 => CACHED [production 12/30] RUN ln -s /openedx/node_modules /openedx/edx-platform/node_modules                                                                                      0.0s
 => CACHED [production 13/30] WORKDIR /openedx/edx-platform                                                                                                                           0.0s
 => CACHED [production 14/30] RUN pip install -e .                                                                                                                                    0.0s
 => CACHED [production 15/30] RUN mkdir -p /openedx/config ./lms/envs/tutor ./cms/envs/tutor                                                                                          0.0s
 => CACHED [production 16/30] COPY --chown=app:app revisions.yml /openedx/config/                                                                                                     0.0s
 => CACHED [production 17/30] COPY --chown=app:app settings/lms/*.py ./lms/envs/tutor/                                                                                                0.0s
 => CACHED [production 18/30] COPY --chown=app:app settings/cms/*.py ./cms/envs/tutor/                                                                                                0.0s
 => CACHED [production 19/30] RUN mkdir /openedx/locale/user                                                                                                                          0.0s
 => CACHED [production 20/30] COPY --chown=app:app ./locale/ /openedx/locale/user/locale/                                                                                             0.0s
 => CACHED [production 21/30] RUN cd /openedx/locale/user &&     django-admin compilemessages -v1                                                                                     0.0s
 => CACHED [production 22/30] RUN ./ lms --settings=tutor.i18n compilejsi18n                                                                                                 0.0s
 => CACHED [production 23/30] RUN ./ cms --settings=tutor.i18n compilejsi18n                                                                                                 0.0s
 => CACHED [production 24/30] COPY --chown=app:app ./bin /openedx/bin                                                                                                                 0.0s
 => CACHED [production 25/30] RUN chmod a+x /openedx/bin/*                                                                                                                            0.0s
 => CACHED [production 26/30] RUN openedx-assets xmodule     && openedx-assets npm     && openedx-assets webpack --env=prod     && openedx-assets common                              0.0s
 => CACHED [production 27/30] COPY --chown=app:app ./themes/ /openedx/themes/                                                                                                         0.0s
 => CACHED [production 28/30] RUN openedx-assets themes     && openedx-assets collect --settings=tutor.assets     && rdfind -makesymlinks true -followsymlinks true /openedx/staticf  0.0s
 => CACHED [production 29/30] RUN mkdir /openedx/data                                                                                                                                 0.0s
 => CACHED [production 30/30] RUN echo   "This copy of edx-platform was built into a Docker image."   > bindmount-canary                                                              0.0s
 => CACHED [development 1/4] RUN --mount=type=cache,target=/var/cache/apt,sharing=locked     --mount=type=cache,target=/var/lib/apt,sharing=locked apt update &&     apt install -y   0.0s
 => CACHED [development 2/4] RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared pip install -r requirements/edx/development.txt                                         0.0s
 => CACHED [development 3/4] RUN --mount=type=cache,target=/openedx/.cache/pip,sharing=shared pip install ipdb==0.13.13 ipython==8.12.0                                               0.0s
 => CACHED [development 4/4] RUN rm -r /openedx/staticfiles &&     mkdir /openedx/staticfiles &&     openedx-assets webpack --env=dev                                                 0.0s
 => exporting to image                                                                                                                                                                0.0s
 => => exporting layers                                                                                                                                                               0.0s
 => => writing image sha256:be2bef1ff28073784efc0f6df718f7925301942ec41b60053028b86d414052f4                                                                                          0.0s
 => => naming to                                                                                                                                 0.0s
 > importing cache manifest from openedx-dev:16.1.7-cache:
Error: Image 'mogc-partners-dev' could not be found

It may be worth mentioning that tutor images build mogc-partners-dev resulted in the same error.

After digging through the tutor-mfe code, I eventually discovered that I could get the MFE dev image to build by explicitly specifying the details using the IMAGES_BUILD filter in my plugin.

def _add_image(images, _config):
    mfe_name = "mogc-partners"
    name = f"{mfe_name}-dev"
    tag = "{{ DOCKER_REGISTRY }}overhangio/openedx-" + name + ":{{ MFE_VERSION }}"
            os.path.join("plugins", "mfe", "build", "mfe"),
    return images

However, I still run into issues because mogc-partners-dev runs webpack dev server at port 8080 but the docker compose file is forwarding port 1998.

Not sure if I’m doing something wrong, or perhaps running into an issue with tutor-mfe, but I’m thankful for any help folks could offer.

What’s the output of tutor mounts list? Do you manage to run tutor dev launch without this mount?

OK it’s a bug in the tutor-mfe plugin. @michael can you please try this fix? fix: auto-build 3rd-party MFEs by regisb · Pull Request #169 · overhangio/tutor-mfe · GitHub

Thanks @regis that fix allows me to build the mogc-partners-dev image, both when running tutor dev launch and tutor images build mogc-partners-dev :raised_hands:

I’m still running into issues running mogc-partners-dev because webpack-dev-server is listening on port 8080 while the docker-compose.yml forwards 1998:1998, but I’m still digging into this so I’ll make another post if I can’t figure things out.

1 Like

Actually going to just follow up here. I opened a PR that solves my port issue.