Mysql initialisation error when upgrading tutor

Hello,

I am trying to update from olive (tutor v15.1.0) to redwood (tutor v18.3.0), and I am encountering a “mysql initialisation error” when running tutor local launch -I (see the whole log below).

I have seen this issue in many other topics, e.g., Tutor fails on tutor local launch - #2 by ih-ffg, Tutor local init using ssh cause mysql error, Tutor (Nutmeg): mysql initialisatino error during quickstart - #6 by seungjoon.kim, but I could not find any solution.

Commands I ran:

sudo rsync -avr "$(tutor config printroot)"/ /tmp/tutor-backup/
pip install --upgrade "tutor[full]"
tutor local stop
tutor plugins disable codejail
pip install git+https://github.com/edunext/tutor-contrib-codejail@v18.0.0
tutor plugins enable codejail
sudo apt install docker-compose-plugin
tutor config save
tutor images build all
tutor local upgrade --from=olive
tutor local do init --limit codejail
tutor local launch -I

Full logs of the last command:

==================================================
        Interactive platform configuration
==================================================
Configuration saved to /home/ubuntu/.local/share/tutor/config.yml
Environment generated in /home/ubuntu/.local/share/tutor/env
======================================
        Building Docker images
======================================
No image to build
==============================================
        Stopping any existing platform
==============================================
docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local stop
[+] Stopping 17/0
 ✔ Container tutor_local-lms-worker-1       Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-caddy-1            Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-xqueue-consumer-1  Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-codejailservice-1  Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-cms-worker-1       Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-mfe-1              Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-discovery-1        Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-xqueue-1           Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-cms-1              Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-lms-1              Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-redis-1            Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-smtp-1             Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-forum-1            Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-mysql-1            Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-mongodb-1          Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-elasticsearch-1    Stopped                                                                                                                    0.0s 
 ✔ Container tutor_local-permissions-1      Stopped                                                                                                                    0.0s 
======================================================
        Starting the platform in detached mode
======================================================
docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/dev/docker-compose.yml --project-name tutor_dev stop
docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local up --remove-orphans -d
[+] Running 17/17
 ✔ Container tutor_local-codejailservice-1  Started                                                                                                                    0.9s 
 ✔ Container tutor_local-mysql-1            Started                                                                                                                    0.6s 
 ✔ Container tutor_local-caddy-1            Started                                                                                                                    1.0s 
 ✔ Container tutor_local-smtp-1             Started                                                                                                                    0.6s 
 ✔ Container tutor_local-xqueue-consumer-1  Started                                                                                                                    1.8s 
 ✔ Container tutor_local-xqueue-1           Started                                                                                                                    1.7s 
 ✔ Container tutor_local-permissions-1      Started                                                                                                                    0.9s 
 ✔ Container tutor_local-mongodb-1          Started                                                                                                                    2.1s 
 ✔ Container tutor_local-elasticsearch-1    Started                                                                                                                    2.1s 
 ✔ Container tutor_local-redis-1            Started                                                                                                                    2.0s 
 ✔ Container tutor_local-forum-1            Started                                                                                                                    2.9s 
 ✔ Container tutor_local-lms-1              Started                                                                                                                    3.7s 
 ✔ Container tutor_local-discovery-1        Started                                                                                                                    4.9s 
 ✔ Container tutor_local-cms-1              Started                                                                                                                    5.1s 
 ✔ Container tutor_local-lms-worker-1       Started                                                                                                                    5.1s 
 ✔ Container tutor_local-mfe-1              Started                                                                                                                    5.0s 
 ✔ Container tutor_local-cms-worker-1       Started                                                                                                                    6.2s 
================================================
        Database creation and migrations
================================================
Initialising all services...
Running init task in lms
docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm lms-job sh -e -c '# The initialization job contains various re-install operations needed to be done
# on mounted directories (edx-platform, /mnt/*xblock, /mnt/<edx-ora, search, enterprise>)
# 1. /mnt/*
# Whenever xblocks or other installable packages are mounted, during the image build, they are copied over to container
# and installed. This results in egg_info generation for the mounted directories. However, the egg_info is not carried
# over to host. When the containers are launched, the host directories without egg_info are mounted on runtime
# and disappear from pip list.
#
# 2. edx-platform
# When a new local copy of edx-platform is bind-mounted, certain build
# artifacts from the openedx image'"'"'s edx-platform directory are lost.
# We regenerate them here.


for mounted_dir in /mnt/*; do
    if [ -f $mounted_dir/setup.py ] && ! ls $mounted_dir/*.egg-info >/dev/null 2>&1 ; then
        echo "Unable to locate egg-info in $mounted_dir"
        pip install -e $mounted_dir
    fi
done

if [ -f /openedx/edx-platform/bindmount-canary ] ; then
        # If this file exists, then edx-platform has not been bind-mounted,
        # so no build artifacts need to be regenerated.
        echo "Using edx-platform from image (not bind-mount)."
        echo "No extra setup is required."
        exit
fi

echo "Performing additional setup for bind-mounted edx-platform."
set -x # Echo out executed lines

# Regenerate Open_edX.egg-info
pip install -e .

# Regenerate node_modules
npm clean-install

# Regenerate static assets.
npm run build-dev

set -x
echo "Done setting up bind-mounted edx-platform."'
[+] Creating 3/0
 ✔ Container tutor_local-permissions-1  Created                                                                                                                        0.0s 
 ✔ Container tutor_local-mongodb-1      Running                                                                                                                        0.0s 
 ✔ Container tutor_local-mysql-1        Running                                                                                                                        0.0s 
[+] Running 1/1
 ✔ Container tutor_local-permissions-1  Started                                                                                                                        1.1s 
Using edx-platform from image (not bind-mount).
No extra setup is required.
Running init task in codejail-apparmor
docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm codejail-apparmor-job sh -e -c 'if [  "$SKIP_INIT" == "True" ]; then
    echo "Skipped AppArmor initialization"
else
    /usr/bin/loader -logtostderr -v=2 /profiles
fi'
W0815 08:58:21.282563       1 loader.go:91] No new profiles found.
Running init task in mysql
docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm mysql-job sh -e -c 'echo "Initialising MySQL..."
mysql_connection_max_attempts=10
mysql_connection_attempt=0
until mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e '"'"'exit'"'"'
do
    mysql_connection_attempt=$(expr $mysql_connection_attempt + 1)
    echo "    [$mysql_connection_attempt/$mysql_connection_max_attempts] Waiting for MySQL service (this may take a while)..."
    if [ $mysql_connection_attempt -eq $mysql_connection_max_attempts ]
    then
      echo "MySQL initialisation error" 1>&2
      exit 1
    fi
    sleep 10
done
echo "MySQL is up and running"

# edx-platform database
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "CREATE DATABASE IF NOT EXISTS openedx;"
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "CREATE USER IF NOT EXISTS '"'"'openedx'"'"';"
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "ALTER USER '"'"'openedx'"'"'@'"'"'%'"'"' IDENTIFIED BY '"'"'P6geSnSu'"'"';"
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "GRANT ALL ON openedx.* TO '"'"'openedx'"'"'@'"'"'%'"'"';"'
[+] Creating 1/0
 ✔ Container tutor_local-mysql-1  Running                                                                                                                              0.0s 
Initialising MySQL...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2003 (HY000): Can't connect to MySQL server on 'mysql:3306' (111)
    [1/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [2/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [3/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [4/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [5/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [6/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [7/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [8/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [9/10] Waiting for MySQL service (this may take a while)...
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-2)
    [10/10] Waiting for MySQL service (this may take a while)...
MySQL initialisation error
Error: Command failed with status 1: docker compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm mysql-job sh -e -c echo "Initialising MySQL..."
mysql_connection_max_attempts=10
mysql_connection_attempt=0
until mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e 'exit'
do
    mysql_connection_attempt=$(expr $mysql_connection_attempt + 1)
    echo "    [$mysql_connection_attempt/$mysql_connection_max_attempts] Waiting for MySQL service (this may take a while)..."
    if [ $mysql_connection_attempt -eq $mysql_connection_max_attempts ]
    then
      echo "MySQL initialisation error" 1>&2
      exit 1
    fi
    sleep 10
done
echo "MySQL is up and running"

# edx-platform database
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "CREATE DATABASE IF NOT EXISTS openedx;"
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "CREATE USER IF NOT EXISTS 'openedx';"
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "ALTER USER 'openedx'@'%' IDENTIFIED BY 'P6geSnSu';"
mysql -u root --password="SM2rmC4b" --host "mysql" --port 3306 -e "GRANT ALL ON openedx.* TO 'openedx'@'%';"

Thank you. Best,
Martí