[Tutor] Fails when build mfe images

the_tan_Nguyen · April 11, 2024, 2:18pm

hi,

Case 1: not integrated plugin

You can see in the image above, config is saved to C:.…
Open that config.yml file
Search for your custom mfe name. If it does not exist, it means you config the plugin wrong => blank page

Case 2: Failed when build image

After successfully have your custom mfe in your config.yml file
You need to run tutor build images
If you failed in this step => no images was built => blank page

WillEVTdigital · April 11, 2024, 3:10pm

It’s the tutor images build mfe part that fails for me. Currently I haven’t customised the mfe image at all

rediris · April 11, 2024, 3:19pm

You could try building the images with the --no-cache and --no-registry-cache flags, like so: tutor images build mfe --no-cache --no-registry-cache

Sometimes I notice the image build fails depending on network connectivity, or even CPU resources. In the case of network connectivity, I’ve had success restarting the build again, and if you skip the flags on the restart, the build will pick up where you left off.

Regarding CPU resources, take a look at the note towards the bottom of the tutor-mfe repo README:

Note

Docker tries to run as many build processes in parallel as possible, but this can cause failures in the MFE image build. If you’re running into OOM issues, RAM starvation, or network failures during NPM installs, try the following before restarting:

cat >buildkitd.toml <<EOF
[worker.oci]
 max-parallelism = 1
EOF
docker buildx create --use --name=singlecpu --config=./buildkitd.toml

I’ve had way more success running a single cpu build process rather than the default.

the_tan_Nguyen · April 11, 2024, 3:30pm

hi @WillEVTdigital ,

rediris trick is right, but it will not work everytime, it’s just the git server’s traffic too high, you’ll have chance to failed.

so, if you did what rediris said (only use one worker at a time) but still fail, don’t give up, pick another hour and rebuild image

WillEVTdigital · April 11, 2024, 4:01pm

Thanks both, unfortunately I’ve tried the --no-cache --no-registry-cache and max-parallelism buildx with no luck

WillEVTdigital · April 16, 2024, 12:20pm

After retrying multiple times, I finally got the build to run successfully with max-parallelism as 1. I’m not really sure where to go from here as, although successful, it takes so long that it really hampers the dev process.

the_tan_Nguyen · April 16, 2024, 1:03pm

Hi @WillEVTdigital ,

I had a hard time with development process too. Please refer to this link: CORS problem when working with MFE template - #14 by the_tan_Nguyen

In summary:
Step 1: Edit the dev.env file to run with tutor (the default is for devstack), the file content is in the link
Step 2: Run tutor local start docker in the background
Step 3: Go to your MFE and npm start
Step 4: you will see the /login_refresh API has 401
Step 5: hardcode your credentianls and ignore the 401 with the format in the link
Step 6: re run MFE and dev

WillEVTdigital · May 13, 2024, 2:24pm

In the end I found that the problem was my VM. I have a Ubuntu guest on a Windows 10 host using Virtualbox. I had the ‘green turtle’ issue, which means that CPU virtualisation wasn’t set up correctly. Disabling Hyper-V and some other Windows features fixed the problem, and I can now build images on my VM absolutely no problem, even with parallelism.

I followed this video to fix my issue.

When trying to build on a t3-large, max-parallelism at 2 is fine, but only if the tutor instance isn’t running at the same time.

Kush · October 15, 2024, 6:45am

I am also facing the same issue

WillEVTdigital · October 15, 2024, 2:06pm

What error do you get when you try to build the mfe image?

Kush · October 15, 2024, 2:17pm

(venv) kushagraGangwar@Kushs-MacBook-Air code % tutor images build mfe --no-cache

Adding /Users/kushagraGangwar/Documents/code/openedx/frontend-app-authn to the build context ‘authn-src’ of image ‘mfe’
Building image docker.io/overhangio/openedx-mfe:18.0.1-indigo
docker buildx build --tag=docker.io/overhangio/openedx-mfe:18.0.1-indigo --no-cache --output=type=docker --cache-from=type=registry,ref=docker.io/overhangio/openedx-mfe:18.0.1-indigo-cache --build-context=authn-src=/Users/kushagraGangwar/Documents/code/openedx/frontend-app-authn ‘/Users/kushagraGangwar/Library/Application Support/tutor/env/plugins/mfe/build/mfe’
[+] Building 2575.4s (116/137) docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 20.38kB 0.0s
=> resolve image config for docker-image://docker.io/docker/dockerfile:1 2.3s
=> [auth] docker/dockerfile:pull token for registry-1.docker.io 0.0s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:865e5dd094beca432e8c0a1d5e1c465db5f998dca4e439981029b3b81fb39ed 0.0s
=> [context authn-src] load .dockerignore 0.0s
=> => transferring authn-src: 204B 0.0s
=> [internal] load metadata for Docker Hub Container Image Library | App Containerization 2.1s
=> [internal] load metadata for Docker Hub Container Image Library | App Containerization 2.1s
=> [auth] library/caddy:pull token for registry-1.docker.io 0.0s
=> [auth] library/node:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> importing cache manifest from docker.io/overhangio/openedx-mfe:18.0.1-indigo-cache 3.2s
=> => inferred cache manifest type: application/vnd.oci.image.index.v1+json 0.0s
=> CACHED [learner-dashboard-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-learner-dashboard: Learner Dashboard MFE 2.8s
=> CACHED [profile-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-profile: Open edX micro-frontend application for managing user profile information.. 2.9s
=> CACHED [ora-grading-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-ora-grading: Open Response Assessment - Enhanced Staff Grader - Allows graders to easily review, evaluate and grade ORAs 2.8s
=> CACHED [learning-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-learning: Front-end for the Open edX course experience, implemented using React and Paragon. 3.0s
=> CACHED [account-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-account: Open edX micro-frontend application for managing user account information.. 2.9s
=> CACHED [base 1/4] FROM Docker Hub Container Image Library | App Containerization 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 143B 0.0s
=> CACHED [course-authoring-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-authoring: Front-end for the Open edX Studio experience, implemented in React and Paragon. 2.9s
=> [context authn-src] load from client 0.1s
=> => transferring authn-src: 32.79kB 0.1s
=> CACHED [discussions-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-discussions: A React-based micro frontend for the Open edX discussion forums. 2.9s
=> CACHED [gradebook-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-gradebook: Instructor grade book tool 2.8s
=> CACHED [production 1/12] FROM Docker Hub Container Image Library | App Containerization 0.0s
=> CACHED [communications-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-communications: Frontend to manage instructor-learner communications 2.8s
=> [auth] overhangio/openedx-mfe:pull token for registry-1.docker.io 0.0s
=> [base 2/4] RUN apt update && apt install -y git gcc libgl1 libxi6 make autoconf libtool pkg-config zlib1g-dev 69.6s
=> [production 2/12] RUN mkdir -p /openedx/dist 0.4s
=> [base 3/4] RUN mkdir -p /openedx/app /openedx/env 0.1s
=> [base 4/4] WORKDIR /openedx/app 0.0s
=> [authn-common 1/6] COPY --from=authn-src /package.json /openedx/app/package.json 0.1s
=> [communications-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-communications: Frontend to manage instructor-learner communications 0.2s
=> [gradebook-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-gradebook: Instructor grade book tool . 0.3s
=> [ora-grading-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-ora-grading: Open Response Assessment - Enhanced Staff Grader - Allows graders to easily review, evaluate and grade ORAs 0.2s
=> [learner-dashboard-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-learner-dashboard: Learner Dashboard MFE 0.4s
=> [profile-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-profile: Open edX micro-frontend application for managing user profile information. . 0.2s
=> [learning-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-learning: Front-end for the Open edX course experience, implemented using React and Paragon. . 0.4s
=> [account-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-account: Open edX micro-frontend application for managing user account information. . 0.2s
=> [discussions-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-discussions: A React-based micro frontend for the Open edX discussion forums. 0.2s
=> [course-authoring-git 1/1] ADD --keep-git-dir=true GitHub - openedx/frontend-app-authoring: Front-end for the Open edX Studio experience, implemented in React and Paragon. 0.8s
=> [authn-common 2/6] COPY --from=authn-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [authn-common 3/6] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --reg 1354.5s
=> [profile-src 1/1] COPY --from=profile-git /openedx/app / 0.0s
=> [communications-src 1/1] COPY --from=communications-git /openedx/app / 0.1s
=> [profile-common 1/9] COPY --from=profile-src /package.json /openedx/app/package.json 0.1s
=> [discussions-src 1/1] COPY --from=discussions-git /openedx/app / 0.1s
=> [account-src 1/1] COPY --from=account-git /openedx/app / 0.0s
=> [ora-grading-src 1/1] COPY --from=ora-grading-git /openedx/app / 0.1s
=> [communications-common 1/5] COPY --from=communications-src /package.json /openedx/app/package.json 0.1s
=> [gradebook-src 1/1] COPY --from=gradebook-git /openedx/app / 0.1s
=> [profile-common 2/9] COPY --from=profile-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [account-common 1/9] COPY --from=account-src /package.json /openedx/app/package.json 0.1s
=> [communications-common 2/5] COPY --from=communications-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [discussions-common 1/9] COPY --from=discussions-src /package.json /openedx/app/package.json 0.1s
=> [ora-grading-common 1/5] COPY --from=ora-grading-src /package.json /openedx/app/package.json 0.1s
=> [learner-dashboard-src 1/1] COPY --from=learner-dashboard-git /openedx/app / 0.1s
=> [profile-common 3/9] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --r 2254.0s
=> [gradebook-common 1/5] COPY --from=gradebook-src /package.json /openedx/app/package.json 0.1s
=> [account-common 2/9] COPY --from=account-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [learning-src 1/1] COPY --from=learning-git /openedx/app / 0.1s
=> [discussions-common 2/9] COPY --from=discussions-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [communications-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-f 1079.2s
=> [ora-grading-common 2/5] COPY --from=ora-grading-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [gradebook-common 2/5] COPY --from=gradebook-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [learner-dashboard-common 1/8] COPY --from=learner-dashboard-src /package.json /openedx/app/package.json 0.1s
=> [account-common 3/9] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --r 1042.4s
=> [discussions-common 3/9] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund 1029.9s
=> [learning-common 1/9] COPY --from=learning-src /package.json /openedx/app/package.json 0.1s
=> [ora-grading-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund 1346.0s
=> [gradebook-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund - 1112.7s
=> [learner-dashboard-common 2/8] COPY --from=learner-dashboard-src /package-lock.json /openedx/app/package-lock.json 0.0s
=> [learning-common 2/9] COPY --from=learning-src /package-lock.json /openedx/app/package-lock.json 0.1s
=> [learner-dashboard-common 3/8] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --n 1662.2s
=> CANCELED [learning-common 3/9] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --n 2495.7s
=> [course-authoring-src 1/1] COPY --from=course-authoring-git /openedx/app / 0.2s
=> [course-authoring-common 1/5] COPY --from=course-authoring-src /package.json /openedx/app/package.json 0.0s
=> [course-authoring-common 2/5] COPY --from=course-authoring-src /package-lock.json /openedx/app/package-lock.json 0.0s
=> [course-authoring-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no 1443.0s
=> [discussions-common 4/9] RUN npm install ‘@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.0.0’ 82.6s
=> [account-common 4/9] RUN npm install ‘@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.0.0’ 53.1s
=> [communications-common 4/5] COPY --from=communications-src / /openedx/app 1.5s
=> [communications-common 5/5] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=“–repository=openedx/openedx-translations --r 10.9s
=> ERROR [communications-prod 1/1] RUN npm run build 1392.3s
=> [account-common 5/9] RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@~3.0. 225.2s
=> [discussions-common 5/9] RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@~ 371.2s
=> [gradebook-common 4/5] COPY --from=gradebook-src / /openedx/app 1.8s
=> [gradebook-common 5/5] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=”–repository=openedx/openedx-translations --revisio 8.4s
=> CANCELED [gradebook-prod 1/1] RUN npm run build 1363.7s
=> [account-common 6/9] RUN npm install @edly-io/indigo-frontend-component-footer@^2.0.0 74.1s
=> [ora-grading-common 4/5] COPY --from=ora-grading-src / /openedx/app 2.4s
=> [ora-grading-common 5/5] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=“–repository=openedx/openedx-translations --revi 11.1s
=> [authn-common 4/6] RUN npm install ‘@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.0.0’ 45.7s
=> CANCELED [ora-grading-prod 1/1] RUN npm run build 1137.2s
=> [account-common 7/9] COPY indigo/env.config.jsx /openedx/app/ 0.2s
=> [account-common 8/9] COPY --from=account-src / /openedx/app 1.1s
=> [account-common 9/9] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=”–repository=openedx/openedx-translations --revision= 7.3s
=> [authn-common 5/6] COPY --from=authn-src / /openedx/app 1.7s
=> [authn-common 6/6] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=“–repository=openedx/openedx-translations --revision=op 6.9s
=> CANCELED [account-prod 1/1] RUN npm run build 1080.9s
=> CANCELED [authn-prod 1/1] RUN npm run build 1076.4s
=> [course-authoring-common 4/5] COPY --from=course-authoring-src / /openedx/app 5.2s
=> [course-authoring-common 5/5] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=”–repository=openedx/openedx-translations - 10.2s
=> CANCELED [course-authoring-prod 1/1] RUN npm run build 1026.1s
=> [discussions-common 6/9] RUN npm install @edly-io/indigo-frontend-component-footer@^2.0.0 231.1s
=> [learner-dashboard-common 4/8] RUN npm install ‘@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.0.0’ 169.1s
=> [discussions-common 7/9] COPY indigo/env.config.jsx /openedx/app/ 0.5s
=> [discussions-common 8/9] COPY --from=discussions-src / /openedx/app 1.5s
=> [discussions-common 9/9] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=“–repository=openedx/openedx-translations --revi 10.5s
=> CANCELED [discussions-prod 1/1] RUN npm run build 758.0s
=> [learner-dashboard-common 5/8] RUN npm install @edly-io/indigo-frontend-component-footer@^2.0.0 239.7s
=> [learner-dashboard-common 6/8] COPY indigo/env.config.jsx /openedx/app/ 0.6s
=> [learner-dashboard-common 7/8] COPY --from=learner-dashboard-src / /openedx/app 2.4s
=> [learner-dashboard-common 8/8] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS=”–repository=openedx/openedx-translations 16.0s
=> CANCELED [learner-dashboard-prod 1/1] RUN npm run build 396.4s
=> [profile-common 4/9] RUN npm install ‘@edx/brand@npm:@edly-io/indigo-brand-openedx@^2.0.0’ 141.3s
=> [profile-common 5/9] RUN npm install '@edx/frontend-component-header@npm:@edly-io/indigo-frontend-component-header@~3.0. 100.4s
=> CANCELED [profile-common 6/9] RUN npm install @edly-io/indigo-frontend-component-footer@^2.0.0 1.3s

[communications-prod 1/1] RUN npm run build:
4.613
4.613 > @edx/frontend-app-communications@0.1.0 build
4.613 > fedx-scripts webpack
4.613
5.276 Running with resolved config:
5.276 /openedx/app/webpack.prod.config.js
5.276
16.24 Browserslist: caniuse-lite is outdated. Please run:
16.24 npx update-browserslist-db@latest
16.24 Why you should do it regularly: GitHub - browserslist/update-db: CLI tool to update caniuse-lite to refresh target browsers from Browserslist config
311.4 [BABEL] Note: The code generator has deoptimised the styling of /openedx/app/node_modules/tinymce/tinymce.js as it exceeds the max of 500KB.
494.0 [BABEL] Note: The code generator has deoptimised the styling of /openedx/app/node_modules/tinymce/themes/silver/theme.js as it exceeds the max of 500KB.
1389.5 Killed
1390.4 npm notice
1390.5 npm notice New minor version of npm available! 10.2.3 → 10.9.0
1390.5 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.0
1390.5 npm notice Run npm install -g npm@10.9.0 to update!
1390.6 npm notice

Dockerfile:510

508 | FROM communications-common AS communications-prod
509 | ENV NODE_ENV=production
510 | >>> RUN npm run build
511 |

512
ERROR: failed to solve: process “/bin/sh -c npm run build” did not complete successfully: exit code: 137

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/vgwu6mx4251xnhriijtijj2na
Error: Command failed with status 1: docker buildx build --tag=docker.io/overhangio/openedx-mfe:18.0.1-indigo --no-cache --output=type=docker --cache-from=type=registry,ref=docker.io/overhangio/openedx-mfe:18.0.1-indigo-cache --build-context=authn-src=/Users/kushagraGangwar/Documents/code/openedx/frontend-app-authn /Users/kushagraGangwar/Library/Application Support/tutor/env/plugins/mfe/build/mfe

WillEVTdigital · October 15, 2024, 2:30pm

As it shows that the process is killed (137), instead of an NPM TIMEOUT, I would guess that there is a different issue here.

It’s probably that Docker is running out of memory - how much RAM have you got?

Kush · October 15, 2024, 3:37pm

I have allotted 7GB RAM to docker on my mac
and CPU limit : 8

WillEVTdigital · October 16, 2024, 10:41am

How much RAM is the build process using? Is it hitting the 7GB limit?

Kush · October 16, 2024, 11:07am

yes it is hitting someting around 6.69GB- 7GB

i have checked the build details this is what it is showing

npmWARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated@babel/plugin-proposal-unicode-property-regex@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-unicode-property-regex instead.
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npmWARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated domexception@2.0.1: Use your platform's native DOMException instead
npmWARNdeprecatedw3c-hr-time@1.0.2: Use your platform's native performance.now() and performance.timeOrigin.
npm WARN deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm WARNdeprecateddomexception@4.0.0: Use your platform's native DOMException instead
npm WARN deprecated sane@4.1.0: some dependency vulnerabilities fixed, support for node < 10 dropped, and newer ECMAScript syntax/features added
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated @fortawesome/fontawesome-common-types@0.3.0: Please upgrade to 6.1.0. https://fontawesome.com/docs/changelog/
npm WARN deprecated @fortawesome/fontawesome-svg-core@1.3.0: Please upgrade to 6.1.0. https://fontawesome.com/docs/changelog/
npm WARN deprecated superagent@3.8.3: Please upgrade to v7.0.2+ of superagent.  We have fixed numerous issues with streams, form-data, attach(), filesystem errors not bubbling up (ENOENT on attach()), and all tests are now passing.  See the releases tab for more information at <https://github.com/visionmedia/superagent/releases>.
npm WARN deprecated popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
Killed369

Topic		Replies	Views
Npm ERR! code ERR_SOCKET_TIMEOUT when building mfe image Tutor Help tutor	4	1890	June 14, 2023
Issue with Building MFE Docker Image After Forking edx/brand-edx.org Site Operations Help tutor , mfe , quince	2	176	August 1, 2024
Error while building mfe images Tutor Help how-to , tutor	0	41	October 15, 2024
Tutor 16.1.8 "tutor images build all" fails with docker buildx error Tutor Help	1	305	January 17, 2024
Service 'mfe' failed to build: The command '/bin/sh -c npm clean-install --no-audit --no-fund --registry=$NPM_REGISTRY && rm -rf ~/.npm' returned a non-zero code: 1 Site Operators tutor	4	644	October 28, 2022

[Tutor] Fails when build mfe images

Dockerfile:510

Related topics