Tech talk/demo: Deploying multiple Open edX instances onto a Kubernetes Cluster with Tutor

At OpenCraft, I’ve been doing some exploration about how we might use Tutor for our Open edX hosting services, which can involve us managing hundreds of separate Open edX deployments. (We currently use our in-house Ocim software for managing most of our instances, which works well but is not docker-based.)

My exploration has resulted in a proof of concept implementation, and I’ve put together a video (15 min) showing what I’ve built so far, which lets you use Tutor + Terraform + GitLab CI to manage a Kubernetes cluster that can have many different Open edX instances running on it:

The code is available at on GitLab: opencraft / dev / Tutorraform · GitLab


:arrow_right: The OpenCraft team has already been discussing this quite a bit, and you can read that and join the discussion on this thread: Tech talk/demo: Deploying multiple Open edX instances onto a Kubernetes Cluster with Tutor - Announcements - Public - OpenCraft

I’d love to hear from @regis about this approach, as well as any others who are interested in a similar use case.

11 Likes

This is a great presentation! thank you Braden and Opencraft for putting this together. Here are my notes, both from the video, the Opencraft forum post and the notes in the gitlab repo:

  • Yes, tutor prints an obnoxious warning when you run it as root. If possible, I’d like to keep this warning, because users very frequently make the mistake of running tutor both as root and as a different user, resulting in different env folders, but identical containers, leading to misunderstanding, despair, and mayhem. I’m only slightly exaggerating here. Seriously, it’s a big issue that pops up every now and then on the forums. Also, I’d like to suggest that you do not run tutor as root, even inside containers: this will make it easier for you in the future to transition to rootless containers (which many people want and need).

  • You mentioned that you would like to define tutor settings as environment variables: this is already possible by defining env vars prefixed by TUTOR_: Configuration and customisation — Tutor documentation – But now that I’m reading your wrapper-script I realize that you already know that.

  • When running tutor k8s quickstart, you get the interactive prompts for setting a few platform parameters (url, language, etc.). If for some reason you don’t need this part you can simply run instead tutor k8s start && tutor k8s init.

  • I stronly suggest not creating different kubernetes clusters for every platform. This will make maintenance very difficult in the future. Instead, each platform should run in a different namespace, defined by K8S_NAMESPACE. This should greatly simplify the terraform scripts.

  • In my experience, a single 8Gb DigitalOcean node should be enough to run a full Open edX platform – although this is obviously not recommended in production.

Again, great job!

PS: On an unrelated note: please avoid “Tutor-something” project names, as Tutor is now a registered trademark. “Tutorraform” is perfectly OK if it’s only used internally, but not if it becomes widely used.

Thanks for checking it out, @regis ! And I have to say a big thank you to you - Tutor is really impressive, and it worked flawlessly (and on the first try) for everything I asked it to do :100:

No worries, it’s up to you. I was just hoping that we could make the warning more subtle when Tutor is run inside a docker container; not get rid of it completely.

I can disable the interactive prompts with --non-interactive. But Tutor still rewrites the config file, which is what I’m trying to avoid. From what I can tell, neither start nor init re-generate the env, but maybe I missed something. What I really want for this use case is a command that does this part and can be used after I manually change the config file. That is, it should: re-generate the env, start/restart the platform, and run migrations/upgrades, but not rewrite the config file.

Isn’t that what I’m doing? This project is focused on exactly that, running multiple Open edX instances on the same Kubernetes cluster, and puts each one into a different k8s namespace.

Ah, ok. Well “Tutorraform” is a terrible name anyways, so if we develop this beyond this initial proof of concept, I don’t mind changing it. Open to ideas, if anyone reading this wants to suggest something :slight_smile:

What about unofficial tutor plugins - can we still call them something-tutor-plugin ?

Right. This use case has not occurred, yet – in part because it’s a bit risky. If some of the generated variables (such as passwords) are not correctly set, they will be re-generated later with different values – and lost.

We can certainly add a -C/--skip-config-file option to the config save command to address your use case.

Right! :sweat_smile: Sorry about that, I did not take the time to read in details the scripts and instructions before replying.

Good question: I’d rather avoid “tutor*” names for non-official plugins. However, currently the plugin cookiecutter automatically generates tutor-* packages. I guess I should change that to, for instance, “tutor-contrib-*” packages. I’m open to ideas, too.

tutor-contrib seems fine to me if you aren’t willing to accept ___-tutor-plugin or ____-for-tutor.

there is not any -C/--skip-config-file option for tutor config save
tutor, version 11.2.11

This is correct. This option was never added because no one worked on it. The change is simple enough to implement, but I wouldn’t be sure how to name the option. “–env-only”? “–skip-config”? “–no-config”?
What’s your use case @imantaba? Do you also need this option? Why?

1 Like

Hi @regis ,
Thank you for fast reply. With this change some of our issues will be resolved . For example i want to change this part of config , but i can’t change it with my custom plugin. I think --skip-config is better option.

@imantaba The DATA_DIR part of the configuration has nothing to do with the --skip-config option. I suggest you create a dedicated Tutor plugin that will add content to the settings file. Let’s not derail this thread further: for further Tutor support, you should head to the Tutor forums.