Tutor: Potential upgrade problem from Maple to Nutmeg with user tours?

sambapete · June 11, 2022, 3:50pm

I had Maple installed on one of my test servers.

After installing Nutmeg, and before running tutor local quickstart I ran * tutor local upgrade --from=maple

I got the following error message when it tried to backpopulate the user tours for Nutmeg:

django.db.utils.ProgrammingError: (1146, "Table 'openedx.user_tours_usertour' doesn't exist")
Error: Command failed with status 1: docker-compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.tmp.yml --project-name tutor_local run --rm lms sh -e -c ./manage.py lms backpopulate_user_tours

Is it possible there is a missing migration somewhere during the upgrade command? Do we still need to run the upgrade command? Maybe that table is not created at the time the upgrade command is run?

I tried running tutor local quickstart afterwards even if I was concerned the upgrade command did not work until the end. It did complete with success.

I had no problem running the ./manage.py lms backpopulate_user_tours afterwards.

Can anyone confirm this behavior?

regis · June 13, 2022, 6:46am

This was reported here: Migration needed before upgrade_from_maple · Issue #689 · overhangio/tutor · GitHub
And is being handled here: fix: upgrade_from_maple issue by ibrmora · Pull Request #692 · overhangio/tutor · GitHub

misilot · June 27, 2022, 8:03pm

@regis has the fix been released?

I am trying to updated 14.0.2 from 13.2.2 and it errors out in the same place when running tutor local quickstart after installing the version with pip.

Error: Command failed with status 1: docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local run --rm lms sh -e -c ./manage.py lms migrate user_tours

Thanks!

regis · June 28, 2022, 8:47am

Yes: tutor/CHANGELOG.md at master · overhangio/tutor · GitHub

Please paste here the output of tutor --version, the command that you are running as well as its full output.

misilot · June 28, 2022, 3:00pm

$ tutor --version
tutor, version 14.0.2

Output after running tutor local quickstart -I this morning.

$ tutor local quickstart -I
==================================================
        Interactive platform configuration
==================================================
Configuration saved to /klib/data/nla-openedx/config.yml
Environment generated in /klib/data/nla-openedx/env
==============================================
        Stopping any existing platform
==============================================
docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local stop
Stopping tutor_local_cms-worker_1    ... done
Stopping tutor_local_lms-worker_1    ... done
Stopping tutor_local_cms_1           ... done
Stopping tutor_local_lms_1           ... done
Stopping tutor_local_mongodb_1       ... done
Stopping tutor_local_redis_1         ... done
Stopping tutor_local_elasticsearch_1 ... done
Stopping tutor_local_mfe_1           ... done
Stopping tutor_local_caddy_1         ... done
Stopping tutor_local_mysql_1         ... done
======================================================
        Starting the platform in detached mode
======================================================
docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/dev/docker-compose.yml -f /klib/data/nla-openedx/env/dev/docker-compose.tmp.yml --project-name tutor_dev stop
docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local up --remove-orphans --build -d
Building mfe
Step 1/112 : FROM docker.io/node:16.14-bullseye-slim AS base
 ---> c49c5d9e27d3
Step 2/112 : RUN apt update   && apt install -y git     gcc git libgl1 libxi6 make     autoconf libtool pkg-config zlib1g-dev     python g++
 ---> Using cache
 ---> 2dc5e64399c6
Step 3/112 : RUN mkdir -p /openedx/app /openedx/env
 ---> Using cache
 ---> a89ff65b2185
Step 4/112 : WORKDIR /openedx/app
 ---> Using cache
 ---> 1512ddf297eb
Step 5/112 : ENV PATH ./node_modules/.bin:${PATH}
 ---> Using cache
 ---> 56c7ba4c4416

Step 6/112 : FROM base AS i18n
 ---> 56c7ba4c4416
Step 7/112 : COPY ./i18n /openedx/i18n
 ---> Using cache
 ---> abcc7c47188a
Step 8/112 : RUN chmod a+x /openedx/i18n/*.js
 ---> Using cache
 ---> 49143e9021aa
Step 9/112 : RUN echo "copying i18n data"   && mkdir -p /openedx/i18n/account   && mkdir -p /openedx/i18n/gradebook   && mkdir -p /openedx/i18n/learning   && mkdir -p /openedx/i18n/profile   echo "done."
 ---> Using cache
 ---> a24b76f69726

Step 10/112 : FROM base AS account-src
 ---> 56c7ba4c4416
Step 11/112 : RUN git clone https://github.com/edx/frontend-app-account --branch open-release/nutmeg.1 --depth 1 .
 ---> Using cache
 ---> ddb8fe2c8875
Step 12/112 : RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)
 ---> Using cache
 ---> 95cceae3eacc

Step 13/112 : FROM base AS account-i18n
 ---> 56c7ba4c4416
Step 14/112 : COPY --from=account-src /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> bae7c8614235
Step 15/112 : COPY --from=i18n /openedx/i18n/account /openedx/i18n/account
 ---> Using cache
 ---> e8e7fcc92ad0
Step 16/112 : COPY --from=i18n /openedx/i18n/i18n-merge.js /openedx/i18n/i18n-merge.js
 ---> Using cache
 ---> 4a69a45caa5d
Step 17/112 : RUN /openedx/i18n/i18n-merge.js /openedx/app/src/i18n/messages /openedx/i18n/account /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 6c930daa06a0

Step 18/112 : FROM base AS account-dev
 ---> 56c7ba4c4416
Step 19/112 : COPY --from=account-src /openedx/app/package.json /openedx/app/package.json
 ---> Using cache
 ---> bc11618c70fd
Step 20/112 : COPY --from=account-src /openedx/app/package-lock.json /openedx/app/package-lock.json
 ---> Using cache
 ---> d94078196660
Step 21/112 : ARG NPM_REGISTRY=https://registry.npmjs.org/
 ---> Using cache
 ---> 53a1f1048dfe
Step 22/112 : ENV CPPFLAGS=-DPNG_ARM_NEON_OPT=0
 ---> Using cache
 ---> facd51fba4b8
Step 23/112 : ENV PACT_SKIP_BINARY_INSTALL=true
 ---> Using cache
 ---> e20480f11f33
Step 24/112 : RUN npm clean-install --no-audit --no-fund --registry=$NPM_REGISTRY   && rm -rf ~/.npm
 ---> Using cache
 ---> 547b8a02e518
Step 25/112 : COPY --from=account-src /openedx/app /openedx/app
 ---> Using cache
 ---> b38d82411d79
Step 26/112 : COPY --from=account-i18n /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 6f63727b5b11
Step 27/112 : ENV PUBLIC_PATH='/account/'
 ---> Using cache
 ---> bbce65a3b96f
Step 28/112 : EXPOSE 1997
 ---> Using cache
 ---> 1181f136073e
Step 29/112 : CMD ["npm", "run", "start"]
 ---> Using cache
 ---> 476828ce602e

Step 30/112 : FROM account-dev AS account
 ---> 476828ce602e
Step 31/112 : COPY ./env/production /openedx/env/production
 ---> Using cache
 ---> c7530560b477
Step 32/112 : RUN touch /openedx/env/production.override   && echo "COACHING_ENABLED=''" >> /openedx/env/production.override   && echo "ENABLE_DEMOGRAPHICS_COLLECTION=''" >> /openedx/env/production.override   && echo "done setting production overrides"
 ---> Using cache
 ---> 3f8b0db994e6
Step 33/112 : RUN bash -c "set -a && source /openedx/env/production && source /openedx/env/production.override && npm run build"
 ---> Using cache
 ---> 8a2bd7127546

Step 34/112 : FROM base AS gradebook-src
 ---> 56c7ba4c4416
Step 35/112 : RUN git clone https://github.com/edx/frontend-app-gradebook --branch open-release/nutmeg.1 --depth 1 .
 ---> Using cache
 ---> fff395ce62f0
Step 36/112 : RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)
 ---> Using cache
 ---> 473c8973d0b3

Step 37/112 : FROM base AS gradebook-i18n
 ---> 56c7ba4c4416
Step 38/112 : COPY --from=gradebook-src /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 24f4300e59ad
Step 39/112 : COPY --from=i18n /openedx/i18n/gradebook /openedx/i18n/gradebook
 ---> Using cache
 ---> b56aeafcc2a1
Step 40/112 : COPY --from=i18n /openedx/i18n/i18n-merge.js /openedx/i18n/i18n-merge.js
 ---> Using cache
 ---> 8c8aa2960994
Step 41/112 : RUN /openedx/i18n/i18n-merge.js /openedx/app/src/i18n/messages /openedx/i18n/gradebook /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 40726dbde090

Step 42/112 : FROM base AS gradebook-dev
 ---> 56c7ba4c4416
Step 43/112 : COPY --from=gradebook-src /openedx/app/package.json /openedx/app/package.json
 ---> Using cache
 ---> ff9755948b05
Step 44/112 : COPY --from=gradebook-src /openedx/app/package-lock.json /openedx/app/package-lock.json
 ---> Using cache
 ---> 4aba054eda01
Step 45/112 : ARG NPM_REGISTRY=https://registry.npmjs.org/
 ---> Using cache
 ---> 429dcc308cf0
Step 46/112 : ENV CPPFLAGS=-DPNG_ARM_NEON_OPT=0
 ---> Using cache
 ---> a88bcee5f643
Step 47/112 : ENV PACT_SKIP_BINARY_INSTALL=true
 ---> Using cache
 ---> 87ccd2e0b840
Step 48/112 : RUN npm clean-install --no-audit --no-fund --registry=$NPM_REGISTRY   && rm -rf ~/.npm
 ---> Using cache
 ---> 7f4db0a7b0e3
Step 49/112 : COPY --from=gradebook-src /openedx/app /openedx/app
 ---> Using cache
 ---> 652271e1dbd7
Step 50/112 : COPY --from=gradebook-i18n /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 7cf7a702683e
Step 51/112 : ENV PUBLIC_PATH='/gradebook/'
 ---> Using cache
 ---> 2dbb03784812
Step 52/112 : EXPOSE 1994
 ---> Using cache
 ---> 87c3eb948f82
Step 53/112 : CMD ["npm", "run", "start"]
 ---> Using cache
 ---> f5ab2f1fc9b2

Step 54/112 : FROM gradebook-dev AS gradebook
 ---> f5ab2f1fc9b2
Step 55/112 : COPY ./env/production /openedx/env/production
 ---> Using cache
 ---> 8279f46b3fea
Step 56/112 : RUN touch /openedx/env/production.override   && echo "done setting production overrides"
 ---> Using cache
 ---> 5b17c9651eb0
Step 57/112 : RUN bash -c "set -a && source /openedx/env/production && source /openedx/env/production.override && npm run build"
 ---> Using cache
 ---> d866bc845fbe

Step 58/112 : FROM base AS learning-src
 ---> 56c7ba4c4416
Step 59/112 : RUN git clone https://github.com/edx/frontend-app-learning --branch open-release/nutmeg.1 --depth 1 .
 ---> Using cache
 ---> 153a3d439f21
Step 60/112 : RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)
 ---> Using cache
 ---> 54c5c16ab796

Step 61/112 : FROM base AS learning-i18n
 ---> 56c7ba4c4416
Step 62/112 : COPY --from=learning-src /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 41de2a555f5e
Step 63/112 : COPY --from=i18n /openedx/i18n/learning /openedx/i18n/learning
 ---> Using cache
 ---> 7984eace1860
Step 64/112 : COPY --from=i18n /openedx/i18n/i18n-merge.js /openedx/i18n/i18n-merge.js
 ---> Using cache
 ---> 63686da4645a
Step 65/112 : RUN /openedx/i18n/i18n-merge.js /openedx/app/src/i18n/messages /openedx/i18n/learning /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 8b341e67b378

Step 66/112 : FROM base AS learning-dev
 ---> 56c7ba4c4416
Step 67/112 : COPY --from=learning-src /openedx/app/package.json /openedx/app/package.json
 ---> Using cache
 ---> 030e21d7a8aa
Step 68/112 : COPY --from=learning-src /openedx/app/package-lock.json /openedx/app/package-lock.json
 ---> Using cache
 ---> 7dadbd7f9e02
Step 69/112 : ARG NPM_REGISTRY=https://registry.npmjs.org/
 ---> Using cache
 ---> 446df309d305
Step 70/112 : ENV CPPFLAGS=-DPNG_ARM_NEON_OPT=0
 ---> Using cache
 ---> efc9a471059c
Step 71/112 : ENV PACT_SKIP_BINARY_INSTALL=true
 ---> Using cache
 ---> a58bee1ffbbe
Step 72/112 : RUN npm clean-install --no-audit --no-fund --registry=$NPM_REGISTRY   && rm -rf ~/.npm
 ---> Using cache
 ---> c6bf536bd65a
Step 73/112 : COPY --from=learning-src /openedx/app /openedx/app
 ---> Using cache
 ---> 8689393205c2
Step 74/112 : COPY --from=learning-i18n /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 808df91f6844
Step 75/112 : ENV PUBLIC_PATH='/learning/'
 ---> Using cache
 ---> 003b6e4300b0
Step 76/112 : EXPOSE 2000
 ---> Using cache
 ---> f128854361dd
Step 77/112 : CMD ["npm", "run", "start"]
 ---> Using cache
 ---> e96fdf07c794

Step 78/112 : FROM learning-dev AS learning
 ---> e96fdf07c794
Step 79/112 : COPY ./env/production /openedx/env/production
 ---> Using cache
 ---> a5df6385a062
Step 80/112 : RUN touch /openedx/env/production.override   && echo "done setting production overrides"
 ---> Using cache
 ---> 64ea62b36f50
Step 81/112 : RUN bash -c "set -a && source /openedx/env/production && source /openedx/env/production.override && npm run build"
 ---> Using cache
 ---> 93c1ba0bb743

Step 82/112 : FROM base AS profile-src
 ---> 56c7ba4c4416
Step 83/112 : RUN git clone https://github.com/edx/frontend-app-profile --branch open-release/nutmeg.1 --depth 1 .
 ---> Using cache
 ---> 47c211795a19
Step 84/112 : RUN stat /openedx/app/src/i18n/messages 2> /dev/null || (echo "missing messages folder" && mkdir -p /openedx/app/src/i18n/messages)
 ---> Using cache
 ---> 2878bf061137

Step 85/112 : FROM base AS profile-i18n
 ---> 56c7ba4c4416
Step 86/112 : COPY --from=profile-src /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> a8e3b785c128
Step 87/112 : COPY --from=i18n /openedx/i18n/profile /openedx/i18n/profile
 ---> Using cache
 ---> b62eb01fe786
Step 88/112 : COPY --from=i18n /openedx/i18n/i18n-merge.js /openedx/i18n/i18n-merge.js
 ---> Using cache
 ---> c5c89d33fbd5
Step 89/112 : RUN /openedx/i18n/i18n-merge.js /openedx/app/src/i18n/messages /openedx/i18n/profile /openedx/app/src/i18n/messages
 ---> Using cache
 ---> 6b38f4526201

Step 90/112 : FROM base AS profile-dev
 ---> 56c7ba4c4416
Step 91/112 : COPY --from=profile-src /openedx/app/package.json /openedx/app/package.json
 ---> Using cache
 ---> 755cd3b6dd19
Step 92/112 : COPY --from=profile-src /openedx/app/package-lock.json /openedx/app/package-lock.json
 ---> Using cache
 ---> b70dc578b5f8
Step 93/112 : ARG NPM_REGISTRY=https://registry.npmjs.org/
 ---> Using cache
 ---> d5f8d06b577e
Step 94/112 : ENV CPPFLAGS=-DPNG_ARM_NEON_OPT=0
 ---> Using cache
 ---> be47348d3dcf
Step 95/112 : ENV PACT_SKIP_BINARY_INSTALL=true
 ---> Using cache
 ---> f7c73f2169b1
Step 96/112 : RUN npm clean-install --no-audit --no-fund --registry=$NPM_REGISTRY   && rm -rf ~/.npm
 ---> Using cache
 ---> e64f7e0492b8
Step 97/112 : COPY --from=profile-src /openedx/app /openedx/app
 ---> Using cache
 ---> fc94a5540ecb
Step 98/112 : COPY --from=profile-i18n /openedx/app/src/i18n/messages /openedx/app/src/i18n/messages
 ---> Using cache
 ---> fdbd86123651
Step 99/112 : ENV PUBLIC_PATH='/profile/'
 ---> Using cache
 ---> 9072d9b1e8a9
Step 100/112 : EXPOSE 1995
 ---> Using cache
 ---> 5442d6400997
Step 101/112 : CMD ["npm", "run", "start"]
 ---> Using cache
 ---> 91757f06792e

Step 102/112 : FROM profile-dev AS profile
 ---> 91757f06792e
Step 103/112 : COPY ./env/production /openedx/env/production
 ---> Using cache
 ---> 3348ece14b3b
Step 104/112 : RUN touch /openedx/env/production.override   && echo "ENABLE_LEARNER_RECORD_MFE='true'" >> /openedx/env/production.override   && echo "done setting production overrides"
 ---> Using cache
 ---> acb93bfa5ddd
Step 105/112 : RUN bash -c "set -a && source /openedx/env/production && source /openedx/env/production.override && npm run build"
 ---> Using cache
 ---> ba618ecc5d6a

Step 106/112 : FROM docker.io/caddy:2.4.6 as production
 ---> 1ace180df3e6
Step 107/112 : RUN mkdir -p /openedx/dist
 ---> Using cache
 ---> f78d365c332b
Step 108/112 : COPY --from=account /openedx/app/dist /openedx/dist/account
 ---> Using cache
 ---> 044da584f497
Step 109/112 : COPY --from=gradebook /openedx/app/dist /openedx/dist/gradebook
 ---> Using cache
 ---> 2a44332afafe
Step 110/112 : COPY --from=learning /openedx/app/dist /openedx/dist/learning
 ---> Using cache
 ---> 2435ad9faeea
Step 111/112 : COPY --from=profile /openedx/app/dist /openedx/dist/profile
 ---> Using cache
 ---> e0f9c35dc39b
Step 112/112 : COPY ./Caddyfile /etc/caddy/Caddyfile
 ---> Using cache
 ---> 6669ef9ebcc9

Successfully built 6669ef9ebcc9
Successfully tagged overhangio/openedx-mfe:14.0.0
Starting tutor_local_cms-permissions_1           ... done
Starting tutor_local_mysql_1                     ... done
Starting tutor_local_mongodb-permissions_1       ... done
Starting tutor_local_lms-permissions_1           ... done
Starting tutor_local_caddy_1                     ... done
Starting tutor_local_redis-permissions_1         ... done
Starting tutor_local_mysql-permissions_1         ... done
Starting tutor_local_elasticsearch-permissions_1 ... done
Starting tutor_local_mfe_1                       ... done
Starting tutor_local_elasticsearch_1             ... done
Starting tutor_local_mongodb_1                   ... done
Starting tutor_local_redis_1                     ... done
Starting tutor_local_lms_1                       ... done
Starting tutor_local_cms_1                       ... done
Starting tutor_local_lms-worker_1                ... done
Starting tutor_local_cms-worker_1                ... done
================================================
        Database creation and migrations
================================================
Initialising all services...
Running init task: hooks/mysql/init
docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local -f /klib/data/nla-openedx/env/local/docker-compose.jobs.yml -f /klib/data/nla-openedx/env/local/docker-compose.jobs.tmp.yml run --rm mysql-job sh -e -c echo "Initialising MySQL..."
mysql_connection_max_attempts=10
mysql_connection_attempt=0
until mysql -u root --password="changemepassword" --host "mysql" --port 3306 -e 'exit'
do
    mysql_connection_attempt=$(expr $mysql_connection_attempt + 1)
    echo "    [$mysql_connection_attempt/$mysql_connection_max_attempts] Waiting for MySQL service (this may take a while)..."
    if [ $mysql_connection_attempt -eq $mysql_connection_max_attempts ]
    then
      echo "MySQL initialisation error" 1>&2
      exit 1
    fi
    sleep 10
done
echo "MySQL is up and running"

# edx-platform database
mysql -u root --password="changemepassword" --host "mysql" --port 3306 -e "CREATE DATABASE IF NOT EXISTS openedx;"
mysql -u root --password="changemepassword" --host "mysql" --port 3306 -e "CREATE USER IF NOT EXISTS 'openedx';"
mysql -u root --password="changemepassword" --host "mysql" --port 3306 -e "ALTER USER 'openedx'@'%' IDENTIFIED BY 'BgWNtbpc';"
mysql -u root --password="changemepassword" --host "mysql" --port 3306 -e "GRANT ALL ON openedx.* TO 'openedx'@'%';"
Starting tutor_local_mysql_1 ... done
Initialising MySQL...
mysql: [Warning] Using a password on the command line interface can be insecure.
MySQL is up and running
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
Running init task: hooks/lms/init
docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local -f /klib/data/nla-openedx/env/local/docker-compose.jobs.yml -f /klib/data/nla-openedx/env/local/docker-compose.jobs.tmp.yml run --rm lms-job sh -e -c dockerize -wait tcp://mysql:3306 -timeout 20s
dockerize -wait tcp://mongodb:27017 -timeout 20s

echo "Loading settings $DJANGO_SETTINGS_MODULE"

./manage.py lms migrate

# Create oauth2 apps for CMS SSO
# https://github.com/openedx/edx-platform/blob/master/docs/guides/studio_oauth.rst
./manage.py lms manage_user cms cms@openedx --unusable-password
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "https://studio.test.online.newliteraciesalliance.org/complete/edx-oauth2/" \
  --client-id cms-sso \
  --client-secret client-secret-value \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso cms
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "http://studio.test.online.newliteraciesalliance.org:8001/complete/edx-oauth2/" \
  --client-id cms-sso-dev \
  --client-secret client-secret-value \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso-dev cms


# Fix incorrect uploaded file path
if [ -d /openedx/data/uploads/ ]; then
  if [ -n "$(ls -A /openedx/data/uploads/)" ]; then
    echo "Migrating LMS uploaded files to shared directory"
    mv /openedx/data/uploads/* /openedx/media/
    rm -rf /openedx/data/uploads/
  fi
fi

# Create waffle switches to enable some features, if they have not been explicitly defined before
# Completion tracking: add green ticks to every completed unit
(./manage.py lms waffle_switch --list | grep completion.enable_completion_tracking) || ./manage.py lms waffle_switch --create completion.enable_completion_tracking on
Starting tutor_local_mongodb-permissions_1 ... done
Starting tutor_local_mysql_1               ... done
Starting tutor_local_mongodb_1             ... done
2022/06/28 14:57:03 Ready: tcp://mysql:3306.
2022/06/28 14:57:03 Ready: tcp://mongodb:27017.
Loading settings lms.envs.tutor.production
Traceback (most recent call last):
  File "./manage.py", line 102, in <module>
    startup = importlib.import_module(edx_args.startup)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/lms/startup.py", line 10, in <module>
    settings.INSTALLED_APPS  # pylint: disable=pointless-statement
  File "/openedx/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/openedx/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 69, in _setup
    self._wrapped = Settings(settings_module)
  File "/openedx/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/lms/envs/tutor/production.py", line 3, in <module>
    from lms.envs.production import *
  File "/openedx/edx-platform/lms/envs/production.py", line 66, in <module>
    __config__ = yaml.safe_load(f)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "/openedx/venv/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/parser.py", line 438, in parse_block_mapping_key
    raise ParserError("while parsing a block mapping", self.marks[-1],
yaml.parser.ParserError: while parsing a block mapping
  in "/openedx/config/lms.env.yml", line 1, column 1
expected <block end>, but found ','
  in "/openedx/config/lms.env.yml", line 94, column 2483
Error: Command failed with status 1: docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local -f /klib/data/nla-openedx/env/local/docker-compose.jobs.yml -f /klib/data/nla-openedx/env/local/docker-compose.jobs.tmp.yml run --rm lms-job sh -e -c dockerize -wait tcp://mysql:3306 -timeout 20s
dockerize -wait tcp://mongodb:27017 -timeout 20s

echo "Loading settings $DJANGO_SETTINGS_MODULE"

./manage.py lms migrate

# Create oauth2 apps for CMS SSO
# https://github.com/openedx/edx-platform/blob/master/docs/guides/studio_oauth.rst
./manage.py lms manage_user cms cms@openedx --unusable-password
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "https://studio.test.online.newliteraciesalliance.org/complete/edx-oauth2/" \
  --client-id cms-sso \
  --client-secret client-secret-value \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso cms
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "http://studio.test.online.newliteraciesalliance.org:8001/complete/edx-oauth2/" \
  --client-id cms-sso-dev \
  --client-secret client-secret-value \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso-dev cms


# Fix incorrect uploaded file path
if [ -d /openedx/data/uploads/ ]; then
  if [ -n "$(ls -A /openedx/data/uploads/)" ]; then
    echo "Migrating LMS uploaded files to shared directory"
    mv /openedx/data/uploads/* /openedx/media/
    rm -rf /openedx/data/uploads/
  fi
fi

# Create waffle switches to enable some features, if they have not been explicitly defined before
# Completion tracking: add green ticks to every completed unit
(./manage.py lms waffle_switch --list | grep completion.enable_completion_tracking) || ./manage.py lms waffle_switch --create completion.enable_completion_tracking on

Output from tutor local upgrade --from=maple

$ tutor local upgrade --from=maple
⚠️  This command only performs a partial upgrade of your Open edX platform. To perform a full upgrade, you should run `tutor local quickstart`.
====================================
        Upgrading from Maple
====================================
Environment generated in /klib/data/nla-openedx/env
docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local run --rm lms sh -e -c ./manage.py lms migrate user_tours
Starting tutor_local_redis-permissions_1         ... done
Starting tutor_local_mongodb-permissions_1       ... done
Starting tutor_local_lms-permissions_1           ... done
Starting tutor_local_elasticsearch-permissions_1 ... done
Starting tutor_local_mysql_1                     ... done
Starting tutor_local_redis_1                     ... done
Starting tutor_local_mongodb_1                   ... done
Starting tutor_local_elasticsearch_1             ... done
Traceback (most recent call last):
  File "./manage.py", line 102, in <module>
    startup = importlib.import_module(edx_args.startup)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/lms/startup.py", line 10, in <module>
    settings.INSTALLED_APPS  # pylint: disable=pointless-statement
  File "/openedx/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/openedx/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 69, in _setup
    self._wrapped = Settings(settings_module)
  File "/openedx/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/lms/envs/tutor/production.py", line 3, in <module>
    from lms.envs.production import *
  File "/openedx/edx-platform/lms/envs/production.py", line 66, in <module>
    __config__ = yaml.safe_load(f)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/openedx/venv/lib/python3.8/site-packages/yaml/composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "/openedx/venv/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/openedx/venv/lib/python3.8/site-packages/yaml/parser.py", line 438, in parse_block_mapping_key
    raise ParserError("while parsing a block mapping", self.marks[-1],
yaml.parser.ParserError: while parsing a block mapping
  in "/openedx/config/lms.env.yml", line 1, column 1
expected <block end>, but found ','
  in "/openedx/config/lms.env.yml", line 94, column 2483
Error: Command failed with status 1: docker-compose -f /klib/data/nla-openedx/env/local/docker-compose.yml -f /klib/data/nla-openedx/env/local/docker-compose.prod.yml -f /klib/data/nla-openedx/env/local/docker-compose.tmp.yml --project-name tutor_local run --rm lms sh -e -c ./manage.py lms migrate user_tours

ghassan · June 28, 2022, 3:13pm

It seems the error is realted to parsing.
I specualte you might have a tutor plugin(s) which has lms-env or cms-env patch. If that is true, make sure you convert the patches to Yaml syntax, instead of JSON.

misilot · June 28, 2022, 3:58pm

@ghassan I saw the release notes about using yml instead of json, however I believe they are all yml syntax already? Or am I missing something obvious?

name: acme
version: 1.0.0
patches:
 caddyfile-global: |
    email {{ ACME_EMAIL }}
    acme_ca {{ ACME_CA | default("https://acme-v02.api.letsencrypt.org/directory") }}
    acme_eab {
       key_id {{ ACME_KEY | default("\"\"") }}
       mac_key {{ ACME_HMAC | default("\"\"")  }}
    }

name: change-user-password
version: 1.0.0
patches:
  common-env-features: |
    "ENABLE_CHANGE_USER_PASSWORD_ADMIN": true

name: googleanalytics
version: 0.1.0
patches:
  openedx-common-settings: |
    # googleanalytics special settings
    GOOGLE_ANALYTICS_ACCOUNT = "{{ GA_ACCOUNT }}"
    GOOGLE_ANALYTICS_TRACKING_ID = "{{ GA_ACCOUNT }}"

name: nla_changes
version: 1.1.0
patches:
 common-env-features: |
    "ENABLE_LTI_PROVIDER": true,
    "ALWAYS_REDIRECT_HOMEPAGE_TO_DASHBOARD_FOR_AUTHENTICATED_USER": false,
    "LTI_AGGREGATE_SCORE_PASSBACK_DELAY": 5 * 60
 nginx-lms: |
    add_header Access-Control-Allow-Origin *;
    proxy_hide_header X-Frame-Options;

name: saml
version: 0.1.0
patches:
 common-env-features: |
    "ENABLE_THIRD_PARTY_AUTH" : true

 openedx-lms-common-settings: |
    # saml special settings
    THIRD_PARTY_AUTH_BACKENDS = ["third_party_auth.saml.SAMLAuthBackend"]

 openedx-auth: |
    "SOCIAL_AUTH_SAML_SP_PRIVATE_KEY" : "{{ SAML_PRIVATE_KEY }}",
    "SOCIAL_AUTH_SAML_SP_PUBLIC_CERT" : "{{ SAML_PUBLIC_CERT }}"

ghassan · June 28, 2022, 4:25pm

That looks like JSON format, for esxample you need to remove the ,I am sorry I forgat to say or wasn’t aware that the changes also apply to openedx-auth patches

ref; tutor/CHANGELOG.md at b8f773a5efde67a769301aecf8434b8c19bbea93 · overhangio/tutor · GitHub

ghassan · June 28, 2022, 4:28pm

Also here is an example where JSON patch was converting to YAML feat: upgrade plugin from v0 to v1 · overhangio/tutor-forum@65dc326 · GitHub just in case you would get too confuse like me while converting.

regis · June 29, 2022, 8:44am

misilot:

common-env-features: |
    "ENABLE_LTI_PROVIDER": true,
    "ALWAYS_REDIRECT_HOMEPAGE_TO_DASHBOARD_FOR_AUTHENTICATED_USER": false,
    "LTI_AGGREGATE_SCORE_PASSBACK_DELAY": 5 * 60
 nginx-lms: |

This is also non-yaml compliant syntax. You should get rid of the trailing commas.

misilot · June 29, 2022, 4:40pm

Thanks @regis, I did change those as well, and it seems to be starting. Now to try and figure out the new method for the indigo theme.

system · September 27, 2022, 4:40pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.