Tutor mfe build hangs

Site Operators Tutor Help
1

Hello folks!

I’m trying to build the mfe plugin via tutor on my local Mac Silicon laptop, and it appears to hang on [profile-prod 1/1] RUN npm run build . I’ve searched the forums and slack for insight on how to let the build proceed, but haven’t found anything yet that matches my issue. There are options to locally run the MFE outside of tutor, which I’m open to, but would like to understand why the tutor mfe flow is not working.

Any ideas on what might be the issue?

Essentially ran basic setup steps:

python -m venv env
source env/bin/activate
git clone https://github.com/openedx/edx-platform
git clone https://github.com/overhangio/tutor
cd tutor
git checkout main
pip install -e .

Set Docker Desktop to use 16GB of RAM, 10 CPUs, and 50 GB of disk space, and essentially ran:

cd ../
export DOCKER_DEFAULT_PLATFORM=linux/amd64
tutor mounts add ./openedx-platform
tutor images build openedx-dev --no-cache --no-registry-cache
git clone https://github.com:overhangio/tutor-mfe
pip install -e ./tutor-mfe
tutor plugins enable mfe
tutor config save
tutor dev launch <---- this hangs (doesn’t complete the mfe build and doesn’t provide output besides [authoring-prod 1/1] RUN npm run build)

so I upped my Docker Desktop to use 28GB of RAM and 256 of disk space, and added:

export TUTOR_ROOT=/Users/…/tools/tutor

and then ran:

tutor images build mfe 
–no-cache 
–no-registry-cache 
–docker-arg=“–progress=plain” 
2>&1 | tee mfe-build.lo

and it’s been sitting on this output for the last hour:

#121 6.276 Translations pulled successfully!
#121 6.289 ./node_modules/.bin/intl-imports.js frontend-platform paragon frontend-component-header frontend-component-footer frontend-app-profile
#121 DONE 7.7s

#115 [account-prod 1/1] RUN npm run build
#115 …
#122 [profile-common 6/6] COPY env.config.jsx /openedx/app#122 DONE 0.1s

#123 [profile-prod 1/1] RUN npm run build#123 1.969#123 1.969 > @edx/frontend-app-profile@1.0.0-semantically-released build#123 1.969 > fedx-scripts webpack#123 1.969#123 3.594 Running with resolved config:#123 3.594 /openedx/app/node_modules/@openedx/frontend-build/config/webpack.prod.config.js#123 3.594

Docker Desktop is reporting used resources as: RAM 24.52 GB, CPU 100.00%, Disk 67.28 GB.

I tried giving docker another CPU (for 11 total, I have 12 available), and the build still pegs the CPU and sits at the profile-prod build step.

2

Hi @Chris_Beach
I suspect that the issue might be related to how many processes spawn due to the number of cores in your setup, which might be eating up too much RAM and slowing down disk I/O. As far as I know it by default tries to run a parallel task for each core in your system.

Have a look at Resource limiting - Max parallelism in the Docker documentation

On my instance in particular I have 4x CPU and 18GB RAM, but I often found that the build would start fine initially but slow down severely with the RAM fully used and paging to the swapfile. So after limiting to 3 parallel tasks my memory usage improved to the point I didn’t have to worry about it paging to swap, takes slightly longer overall to complete all the builds as there’s less running at the same time, but it did make the process more stable, for me anyway…

Hope this helps :slight_smile:

1 Like
3

Thanks for the advice @joel.edwards !

I’m retrying the tutor dev launch with a couple CPUs and we’ll see how it goes.

When you do a tutor build with just 3 parallel tasks, does that force the build to run a max of 3 npm run buildcommands concurrently? I’m seeing the following in my tutor build run and am curious if I didn’t limit the CPUs correctly.

[+] Building 5279.7s (113/135)                                                               docker:desktop-linux
 => [ora-grading-prod 1/1] RUN npm run build                                                              5270.0s
 => => # Running with resolved config:                                                                           
 => => # /openedx/app/webpack.prod.config.js                                                                     
 => => # Browserslist: browsers data (caniuse-lite) is 10 months old. Please run:                                
 => => #   npx update-browserslist-db@latest                                                                     
 => => #   Why you should do it regularly: https://github.com/browserslist/update-db#readme                      
 => [discussions-prod 1/1] RUN npm run build                                                              5268.3s
 => [communications-prod 1/1] RUN npm run build                                                           5267.4s
 => [learning-prod 1/1] RUN npm run build                                                                 5265.9s
 => [authoring-prod 1/1] RUN npm run build                                                                5265.9s
 => [learner-dashboard-common 4/6] COPY --from=learner-dashboard-src / /openedx/app                          0.1s
 => [learner-dashboard-common 5/6] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS="--repository=openedx/op  10.1s
 => [gradebook-common 4/6] COPY --from=gradebook-src / /openedx/app                                          0.1s
 => [gradebook-common 5/6] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS="--repository=openedx/openedx-tra  9.7s
 => [learner-dashboard-common 6/6] COPY env.config.jsx /openedx/app                                          0.1s
 => [learner-dashboard-prod 1/1] RUN npm run build                                                        5075.6s
 => [gradebook-common 6/6] COPY env.config.jsx /openedx/app                                                  0.0s
 => [gradebook-prod 1/1] RUN npm run build                                                                5071.6s
 => [admin-console-common 4/6] COPY --from=admin-console-src / /openedx/app                                  0.1s
 => [admin-console-common 5/6] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS="--repository=openedx/opened  11.2s
 => [admin-console-common 6/6] COPY env.config.jsx /openedx/app                                              0.0s
 => [admin-console-prod 1/1] RUN npm run build                                                            5036.1s
 => [profile-common 4/6] COPY --from=profile-src / /openedx/app                                              0.1s
 => [profile-common 5/6] RUN make OPENEDX_ATLAS_PULL=true ATLAS_OPTIONS="--repository=openedx/openedx-tran  10.8s
 => [profile-common 6/6] COPY env.config.jsx /openedx/app                                                    0.0s 
 => [profile-prod 1/1] RUN npm run build                                                                  4745.4s

Note, the in progress / completed status doesn’t show nicely in the code block - please see the screenshot below for each task’s status.

image
image1593×818 157 KB

4

@Chris_Beach I updated your trust level so you can post images :slight_smile:

1 Like
5

Thank you @sarina ! I’ve updated my reply above with the screenshot of tutor build.

6

@Chris_Beach I actually will say, you should always paste output in a code block. Images are not searchable and we can’t copy bits out of it. Could you revert to using a code block?

1 Like
7

Sounds good @sarina . I adjusted my reply above.

8

From what I can see in your screenshot, it looks like there are 9 parallel tasks running and its taking 2+ hours to run, this is definitely not ideal. Most of my builds take between 10-30mins depending on what is/isn’t cached already, but don’t think I’ve had it take more than an hour at most for everything uncached (although your connectivity/internet speed may be a factor too, assuming that’s on the slower end)

This is how I did the Max-Parallelism config on my system:
create file ~/.docker/buildkitd3.toml with content:

[worker.oci]
  max-parallelism = 3

run commands:
docker buildx create --name max3cpu --driver docker-container --use --config ~/.docker/buildkitd3.toml
docker buildx inspect max3cpu --bootstrap

if done correctly you should see max3cpu in the build process like this:

image
image1919×336 40.7 KB

At the least that’s how I achieved this, hope it helps :slight_smile:

9

@joel.edwards After following your instructions, I was able to confirm the max parallelism was indeed lowered on my tutor build (thank you).

However, it still just sits on those `npm run build` mfe tasks for hours, without any apparent progress.