Tech talk/demo: Deploying multiple Open edX instances onto a Kubernetes Cluster with Tutor

Development Collaborative Proposals
, ,
47

Hi all,
I would like to write down a brief summary of our experience about Kubernetes and Tutor implementations. Your comments will be appreciated!

I have created a separate post to discuss about infrastructure reference architecture.

These are some of our findings:

  • Multiple Open edX instances can be deployed in a single cluster, assigning one namespace to each Open edX instance.
  • K8s is straightforward for stateless pods. However, for stateful parts of the application, special care is needed.
  • databases and storage apart, lms, lms-worker, cms, cms-worker, mfe and forum are stateless, so should work fine without much care.
  • databases are complex systems.
    – We have four: MySQL, MongoDB, Redis and ElasticSearch.
    – Each of them have a different and specific dynamics, in terms of escalation and redundancy. An incorrect setup may lead to data loss, downgraded performance, loss of synchronization, etc.
    – Setting up a database properly in K8s may need a careful tuning, different for each technology.
    – Our conclusion is that it is better to rely on the cloud provider’s services for databases, and let them manage redundancy, scalability, fine tuning, updates, etc.
  • File storage can be managed by
    – out-of-the-box MinIO, but in this case files are stored in a block service (see next)
    – rely on the cloud provider’s file storage service (S3 for AWS), which is our option
  • Block storage: Stateful pods need a place to store persistent data. In K8s, it’s the PV (Persistent Volumes). The actual location and driver for each PV is defined in a storage class. The default storage class uses the node’s storage to allocate space to the PV. This is not the best option, as the data will not survive a node recreation, or a pod recreation in another node. If we want a scalable and resilient block storage, we need to use a NFS (network file system) PV, which can be complex to setup and dependent on the cloud provider.
  • However, if the databases are left out of the cluster and lms, cms and plugins are stateless, there is no need to use block storage. With one exception:
  • Caddy, which deserves a separate paragraph.

The next post is about databases.

2 Likes