Multiple isolated devstacks on the same host

Synopsis

You can have multiple isolated Devstacks provisioned on a single computer now. This was the result of a big PR, so let us know if bugs arise that seem to be related. If you are blocked by such a bug, check out the juniper.master branch of edx/devstack, which currently does not include this change.

Details

The Change

Hi folks. @ztraboo and I recently collaborated on a change to allow having multiple Devstack instances provisioned on the same host computer. Each instance has an isolated set of databases. This could, for example, be used to quickly switch between versions of Open edX without hitting as many issues with migrations, data integrity, etc.

Unfortunately, this does not currently support running Devstacks simultaneously, because we hard-code host port numbers all over the place, and as far as I know, two running containers cannot share the same host port.

How to Use It

First, bring down any running containers:

make dev.down

Then, edit the file options.local.mk in the root of the edx/devstack repo, creating if it does not exist. Change the Devstack project name by adding the following line:

COMPOSE_PROJECT_NAME=<your-alternate-devstack-name>

For example:

COMPOSE_PROJECT_NAME=secondarydevstack

By default, COMPOSE_PROJECT_NAME is defined as devstack, which is what it has been set to so far.

Now, provisionion and run Devstack like you would normally:

make dev.provision  # Warning: takes 1hr+
make dev.up.<services>

Finally, you can switch between your Devstack projects by doing the following:

  1. Bring down containers.
  2. Edit the project name in options.local.mk
  3. Bring up containers.

Using with OPENEDX_RELEASE

Many of you may use the OPENEDX_RELEASE environment variable to specify the version of services that are checked out during Devstack provisioning.

If you set OPENEDX_RELEASE and do not specify COMPOSE_PROJECT_NAME, then your Devstack project name will be set as devstack-${OPENEDX_RELEASE}.

As a specific example, if OPENEDX_RELEASE is set in your environment as ironwood.master, then COMPOSE_PROJECT_NAME will default to devstack-ironwood.master instead of devstack.

The implication of this is that you can switch between isolated Devstack databases by changing the value of the OPENEDX_RELEASE environment variable.

Questions & troubleshooting

Are there official docs on this?

For now, these are the docs. Hopefully that will change in time.

This broke my existing Devstack!

Try posting on Discourse or Slack to see if you have the same issue as any others.

If you think you have found a bug, file a CR ticket. If the bug blocks you, you may want to check out the juniper.master version of Devstack, which currently does not include this change.

I’m getting errors related to ports already being used.

Make sure you bring down your devstack before changing the value of COMPOSE_PROJECT_NAME. If you forgot to, change the COMPOSE_PROJECT_NAME back to its original value, run make dev.down, and then try again.

I have custom scripts/compose files that integrate with or extend Devstack. Will those still work?

With the default value of COMPOSE_PROJECT_NAME = devstack, they should still work.

If you choose a different COMPOSE_PROJECT_NAME, your extensions will likely break, because the names of containers change along with the project name.

5 Likes

Additional notes when changing release can be found here.

edxenv bash alias to switch between releases.

direnv to handle environmental changes.

2 Likes

This is awesome! Great work, @ztraboo and @kmccormick!

@nedbat, is it not too late to merge this into the juniper.master devstack? This will make it a lot easier to test Juniper, for some of us.

Congrats on merging this! Please please please document this feature in the devstack’s README prior to making a PR on the juniper.master branch!

1 Like

I recommend using direnv over the edxenv bash script that I created above. It is easy to setup and use plus it can do more. I’ve created several files that I used in my setup here https://gist.github.com/ztraboo/d1e9cad3e9a9c03e75f2792922f887f2.

After direnv is configured I ran these steps to make it work. Thanks to @arbrandes for recommending this directory environment application. Using zsh from Mac OS X Catalina but you may prefer bash command.

ztraboo-macbookpro:devstack.ironwood ztraboo$ zsh
direnv: loading ~/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood/.envrc                                                                                                                                                                                                                                                                                                         
direnv: export +DEVSTACK_WORKSPACE +OPENEDX_RELEASE +VIRTUAL_ENV ~PATH
(venv)ztraboo@ztraboo-macbookpro devstack.ironwood % which python
/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood/devstack/venv/bin/python
(venv)ztraboo@ztraboo-macbookpro devstack.ironwood % printenv
TERM=xterm-256color
SHELL=/bin/bash
TMPDIR=/var/folders/kq/gmqwcr_n1jj6pxpnhvvlh9g00000gn/T/
USER=ztraboo
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.FfC6udK6kq/Listeners
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
PATH=/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood/devstack/venv/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:~/Library/Python/3.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
TERMINAL_EMULATOR=JetBrains-JediTerm
PWD=/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
SHLVL=2
HOME=/Users/ztraboo
LOGNAME=ztraboo
LC_CTYPE=en_US.UTF-8
OLDPWD=/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood
DIRENV_DIFF=eJzcU09zqjoU_y5Zq5HKQ2TmLXiAgAoiIDDdMPyJomiCSUSw0372N22n3pk7XfZu7u53Tk4y5_cnL6ABygvw1NACCoCrQ04z2sM5zc7oRmjNoNfziuDR7qsDI0TZgWAGJyMJ5gesvD2ufc4-DuCVUXgiRXb6VX6AR8U-EPttmFUZRbAkHCOuvMHRJ4KckBNTvtvRIZh8t6F2pRRhDjVyPme4ZOB1APA7X92IglDVlmm89peBp2rGO_stQ5TBO6dZTgjUUQuN8lpkHMWE1jtCCwR91BAGSYPwEJUdLFHLeFbUowMl-EZICQZAt33DjVLd9oEChj_7aKyGmmUEQAFo0Y3v9Xwt3dWN6mpTtHjasWiMOfWGSWBXC2ejsVydGffo0s_9oysJ6mQxnu2rtTxLL411cqaXLf6v6ONGdo36Oc7F8M4KOWoC0qFYlrtAW0zJrUrEDWfp7cj20zw-FVcxEdHEPLp6yGvu46TZ5w6bXora2SY8LWzvKnjmUJjnkbD2l8lM2BV01gdBu912tpbYwZNVLiV5Lxx0y3ICs3J1_RlXsmhKw-dNq13JsRCabtHxmWOuL3YbVmJv_nM5WpKJxu20NzNLkidU1FVVT1MxX8W5_S8YgLVnuIaepL6xMtTg3dEvAUfnjHFEweAR9B9w5dGBLcLtZ6b_9v8zAJHth1t1lRpu9Ed0BK-v_wcAAP__noycCw==
DIRENV_WATCHES=eJx0zkFO6zAQANC7eJ2fsV0ntrP-XSIhJMQCsbA9EzVqyFRjN61A3J09ghO89_qpHlM7qUnBcyWp8NEkZWb4Tzsc8VpSoxeW88xSCJ7owhX4Qts_wjsg7bWlcu4X4e3GjNDTtktRnXpgbMs7qckMUXt_cIPu1PG-1FbV1ORKX91fcr9ySSvUUxICXIS2HdK68g1iDHHMSGhNDDZnh84G6-ZQvCuojc1pxJxt9MGOqIvTh4yG5qjH6Ge0v7yGaH683r4DAAD__4bLWbI=
DEVSTACK_WORKSPACE=/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood
DIRENV_DIR=-/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood
OPENEDX_RELEASE=ironwood.master
VIRTUAL_ENV=/Users/ztraboo/Dev/EducateWorkforce/Repos/open-edx/devstack.ironwood/devstack/venv
_=/usr/bin/printenv

Once the environment is setup you can just go to the ./devstack directory and launch make dev.up and your release specific containers will launch.

(venv)ztraboo@ztraboo-macbookpro devstack.ironwood % cd devstack 
(venv)ztraboo@ztraboo-macbookpro devstack % make dev.up
bash -c 'docker-compose -f docker-compose.yml -f docker-compose-host.yml up -d'
Starting edx.devstack-ironwood.master.elasticsearch ... done
Starting edx.devstack-ironwood.master.mysql         ... done
Starting edx.devstack-ironwood.master.memcached     ... done
Starting edx.devstack-ironwood.master.mongo         ... done
Starting edx.devstack-ironwood.master.firefox       ... done
Starting edx.devstack-ironwood.master.devpi         ... done
Starting edx.devstack-ironwood.master.gradebook     ... done
Starting edx.devstack-ironwood.master.chrome        ... done
Starting edx.devstack-ironwood.master.credentials   ... done
Starting edx.devstack-ironwood.master.ecommerce     ... done
Starting edx.devstack-ironwood.master.discovery     ... done
Starting edx.devstack-ironwood.master.forum         ... done
Starting edx.devstack-ironwood.master.lms           ... done
Starting edx.devstack-ironwood.master.studio        ... done
Starting edx.devstack-ironwood.master.edx_notes_api ... done
bash ./programs/provision.sh cache >/dev/null

Added documentation for both releases here.

master

open-release/juniper.master

3 Likes

The README.rst updates (master, open-release/juniper.master) and backport to Juniper release are now merged thanks to review from @regis!

cc: @arbrandes, @kmccormick

3 Likes