Busting Docker cache when building from a github repo

Development
1

Hello forum - I wanted to make a change to a function in one of the javascript files from the git repo at “GitHub - openedx/xblock-in-video-quiz”. So I forked it to my own github account, and pointed my “config.yml” at the fork.

The next time I ran $ tutor dev launch, it pulled in my code successfully. However, I then made a second change to my fork, committed it, and pushed it up. But no matter what I’ve tried, when I do $ tutor dev launch, that javascript file always ends up with only the original change that I first pushed to my forked repo.

I’ve tried stopping and deleting all EdX-related Docker containers, and deleting all dangling Docker images. I tried removing the dependency from my “config.yml”, rebuilding the code without it, then adding it back in and rebuilding the code again. But ‘tutor launch’ always delivers that same original change, even though the code it’s using no longer even exists in the repo on github. So it must be coming from some local cache that I can’t figure out how to clear. It’s maddening. How can I make tutor dev launch build with the latest changes to the repo on github?

2

Hi @mrbun try with the tutor dev images build --no-cache command

3

Hi @leonelkatsikaris - thank you for the suggestion. Unfortunately, tutor doesn’t seem to like this command:

$ tutor dev images build --no-cache
Usage: tutor dev [OPTIONS] COMMAND [ARGS]...
Try 'tutor dev -h' for help.

Error: No such command 'images'.

Is this unexpected?

4

Hello @mrbun :raised_hands:

The command for building dev images is tutor images build openedx-dev. You can use the flag --no-cache if needed.

But being extremely sincere, I don’t build while working in development. I use Sharing directories with containers to make changes in runtime.

  1. I would use a sharing method to have my xblock in the cms and my lms container if needed.
  2. Then I would enter both containers (You can use tutor dev run <lms/cms> bash or something like docker exec -it <container_name> bash)
  3. Perform a pip install -e <path to the xblock inside the container>.
  4. Perform a restart tutor dev restart.
  5. See my changes!

The problem with this approach is if you launch again, you will lose the changes (because the launch creates the container again, and it won’t have your custom installation). So, I recommend using start, stop, and restart after launch.

Note: The building command is probably better if you prefer to keep the changes.

I hope this helps you. :sparkles:

References:

5

Sorry, I forgot openedx/mfe command, depending on what you want:

tutor images build mfe --no-cache for mfe initialization

or

tutor images build openedx --no-cache for theme initialization

6

Make a new branch from your develop branch and reference the new branch name to the config,
–no-cache makes your building process take much longer.

7

Thank you everyone for your help!

8

Hello @mafermazu -

I “solved” my original problem by creating my forked github repository with a different name every time I wanted to make a change. This forced tutor / Docker to ignore its cache since it was building from a “new” respository.

But that is obviously not a good long-term solution. Now I am trying to change a style rule, but I have not been able to make tutor / Docker incorporate the change to my external stylesheet.

You said that you “don’t build while working in development” and recommended using “sharing directories with containers to make changes in runtime.” I tried reading the linked documentation, but I was unable to follow it. The only commands I’ve ever used are tutor dev launch, tutor dev stop, and tutor dev start. I’d prefer to keep doing that if only I could find a way to make tutor / Docker clear its cache. I don’t understand what is so hard about that.

You suggested using tutor images build openedx-dev, but this fails with the error message: “Error: Image ‘openedx-dev’ could not be found”

Similarly, tutor images build dev fails with “Error: Image ‘dev’ could not be found”

I tried tutor dev launch -p. Supposedly the -p flag will “Update docker images”, but this failed with the message “Error response from daemon: pull access denied for openedx-dev, repository does not exist or may require ‘docker login’: denied: requested access to the resource is denied”

I tried tutor images build all but this failed with the message “ERROR: failed to solve: process “/bin/sh -c npm run build” did not complete successfully: exit code: 1”

Can you recommend any changes to any of the approaches above, or do you know any other way to make tutor / Docker discard its cache so that changes to stylesheets will be incorporated?

9

Hi @mrbun - you may need to run tutor dev save after you make a change (this updates the tutor env files with any changes that have been made) and then try running tutor images build openedx mfe. If that still doesn’t work, try building the images without the cache tutor images build openedx --no-cache mfe --no-cache

10

Regarding tutor commands failing, you can always verify the commands and how to use them with the flag -h. Example: tutor -h, tutor config -h, tutor images build -h, etc. You also have all the commands here: Command line interface (CLI) — Tutor documentation.

About your problem with pulling images. I suggest you check if it is in your config.yml file (inside your TUTOR_ROOT, tutor config printroot); you have defined some of these variables listed here about docker images Configuration and customisation — Tutor documentation. If you didn’t modify those variables, they should have the default value, and all those images are public in Docker, so you shouldn’t need a Docker login.

About building images, you should be able to build the openedx-dev and mfe images, as some collaborators explained before. For more info: Open edX development — Tutor documentation.

I know it is a lot of information :see_no_evil:

If someone asks me to try and edit the code of an xblock.
I will do this following the Sharing directories with containers doc:

  1. Using a clean installation of tutor.
    1.1. Uninstalling what I have. Installing Tutor — Tutor documentation
    1.2. Install tutor again
  2. Launch a dev env: tutor dev launch
  3. Stop my containers: tutor dev stop
  4. Clone the repo I will test, modify, or debug, e,g,. git clone git@github.com:openedx/xblock-in-video-quiz.git
  5. I want that in my cms container to use that xblock in Studio, so I mount that xblock in cms: tutor mounts add cms:<your_local_path>:<container_path>, in my case, tutor mounts add cms:xblock-in-video-quiz:/openedx/xblock-in-video-quiz
  6. Run tutor mounts list to verify everything is okay.
  7. Start the env again: tutor dev start
  8. In another terminal, I will run: docker exec -it <cms_container> bash. The cms container could come from docker ps or the tutor dev start logs. I think the default is something like tutor_dev-cms-1; in my particular case, it looks like: docker exec -it dev_redwood_env_new-cms-1 bash.
  9. As in step 5, I said I mount the xblock in /openedx/xblock-in-video-quiz, I’ll run: pip install -e /openedx/xblock-in-video-quiz. This inside the docker container (step 8).
  10. In another terminal, I will run: tutor dev restart cms

That’s it! :sparkles:
That’s the way I use the sharing directories.

Behind that process, the logic is: I am sharing the xblock-in-video-quiz directory I cloned with the cms container in the path /openedx/xblock-in-video-quiz. So, each change I made in my cloned directory will appear inside the container in /openedx/xblock-in-video-quiz. And when I install with pip install -e, I install the xblock in edition mode. So, my changes will affect the platform.

In this particular case, to see my changes…

  1. Prepare dummy data
    1.1. tutor dev do createser --staff --superuser admin admin@example.com
    1.2. tutor dev do importdemocourse
  2. Enter the browser to the Studio: http://studio.local.edly.io:8001/
  3. Login and enter to edit the course.
  4. Add "invideoquiz" in Settings → Advanced Settings. From: GitHub - openedx/xblock-in-video-quiz
  5. Create sections and a unit.
  6. Add an in-video Xblock in the Advance options.
  7. Made a change in the Xblock, example, adding the following in xblock-in-video-quiz/invideoquiz/public/css/invideoquiz.css:
body {
    background-color: #de3011; /* Replace with your desired color */
}
  1. Reload the Studio page: http://studio.local.edly.io:8001/
  2. See my changes

image
image1723×816 55.7 KB

And again: These changes aren’t persistent. If you launch again, you will lose these changes (the only commands you should avoid using are the launch and the init commands related to the edited container).

I use this to test things on the fly, and I use the build for the openedx image, which we use in stage and production.

But it is up to you. If you find a way that works well for you, go ahead.

I hope this and the other answers help you with your problem. :raised_hands: