Open edX Devstack Development with Apple Macbook M1 Silicon (ARM Architecture)

Development
1

Hi All,

I am using Ubuntu/Linux for Open edX Devstack development (Hawthorne, Ironwood, Juniper, Koa, +).

As far as I know, Intel MackBooks based on x86 Architecture are working good with open edX Devstack, but now I am considering to purchase the new Apple MacBook Pro M1 Silicon (ARM Architecture) for Open edX Devstack development.

So if anyone has any experience or knowledge working with M1 Silicon (ARM Architecture) for edX and Docker?

I am asking this because of there are some mixed reviews on Docker running with M1 Silicon chip.

2

We’re starting to think about this, but I’m not aware of anyone in the Open edX community who has an Apple Silicon Mac yet to test it out. You could try running the existing x86 images in a recent Docker for Mac release and they should work thanks to qemu integration, but they might be a bit slow and/or unstable. It’s hopefully not too hard to tweak our existing image build process to also upload suitable ARM images, but it would be useful to have appropriate hardware available to make sure they work correctly.

Relevant Docker docs and blog posts:

2 Likes
3

I have a PR that does that for now, and as expected it seems to work but is slow: Add basic support for Apple Silicon by bradenmacdonald · Pull Request #852 · openedx/devstack · GitHub

4

We also have that issue on Tutor, where users of Apple M1 can’t run the mysql Docker image: Support for ARM64 · Issue #510 · overhangio/tutor · GitHub Also, they currently have to rebuild the Tutor-specific Docker images themselves because we don’t provide ARM-compatible images: Docker

When this issue was first reported I posted the following rant thoughtful remark: https://twitter.com/regisb/status/1451183922830721030

Serious question: given that Docker support sucks on Mac OS, and it’s even worse on ARM, and it’s not changing for the better, why do devs who run Docker keep buying Apple laptops?

To be honest I’m pissed as hell not too excited by the perspective of creating a dedicated ARM-based CI to build and push Docker images for these 5% of users who bought $2k+ laptops and on which even installing Docker is a PITA.

5

Umm, first, I don’t think that attitude is productive here. If you don’t want to support ARM, don’t support ARM - just say it’ll be up to the community to implement or not. Or encourage M1 Mac users to run Tutor on an external/cloud linux server, which can be a very nice option anyways. I may end up doing that myself, if the native situation doesn’t improve. (Some devs at OpenCraft who have x86_64 machines still use an external server for their dev environment out of preference.)

Don’t forget that supporting ARM also brings support to Raspberry Pi, the most affordable computing system of all. (Unless I’m missing something?) I think running Open edX and Tutor on Raspberry Pi would be awesome.

I think that’s old information, I just got a new MacBook and installing Docker was just as easy as on any previous system, literally drag and drop to install then run one command to enable Rosetta 2. I then made that PR with minor config changes to the docker-compse file and got the edx-devstack running.

6

Yes, this is one of the use cases of tutor-machine.

Thus, to answer @Arpit_Gandhi’s original question: no, the M1 is not a great platform to run the Devstack or Tutor.

I think so too. One Tutor user seems to have succeeded at running Open edX on Raspberry Pi, but I don’t have any more information.

It might be old information indeed. My impression was that the only sane way to install Docker on macOS was to install Docker Desktop, and a commercial subscription was necessary for businesses. Is this still valid?

1 Like
7

Ah, I misunderstood and thought you were referring to some ARM-specific difficultly. Yes that’s still the case that Docker Desktop is the easiest tool for MacOS, but it is hardly a “PITA”; it’s extremely convenient. And a commercial subscription is only needed if your business earns more than $10 million per year; it’s free for open source, small business, or education.

8

On a related note, the user who managed to run Tutor on Raspberry Pi has given us some feedback (discussion, report). Here is an excerpt:

(…)

Conversion Docker images from AMD64 to ARM

The Tutor uses nine Docker images to run Open Edx.

The Docker images provided by Tutor are all built on AMD64 architecture, and since the Raspberry Pi is an ARM architecture, the default Docker images cannot be run on the Raspberry Pi.

Therefore, I converted all the Docker images from AMD64 to ARM.

I paid close attention to the software versions and used the same version of the software as much as possible to reduce the complexity of the debugging process. As a result, I was able to eliminate problems caused by differences in software versions.

I was able to convert Nginx, SMTP, Rabbitmq, and Memcache from AMD64 to ARM without any hindrance. I also found a Docker image of MariaDB for the ARM architecture.

The Open Edx and Open Edx Forum Docker images use Dockerize for container configuration; in the process of building the Docker images, both the Open Edx and the Open Edx Forum; the AMD64-based Dockerize will not work on the Raspberry Pi, so we replaced Dockerize with an ARM-based version. Other than that, I was able to build Open Edx and Open Edx Forum Docker images smoothly.

The Elasticserach container uses JAVA containers, and since the architecture of JAVA containers is AMD64, I used the ARM-based Debian to build the JAVA containers. I then built the Elasticsearch container based on the ARM-based JAVA container. The directory structure of Elasticsearch was slightly different between the AMD64-based Elasticsearch container and the ARM-based Elasticsearch container, but other than that, there were no problems.

Since MongoDB versions 3 and 4 do not work on the ARM architecture (they should work on ARM64), I could not find a Docker image of MongoDB version 3 that could be modified for the ARM-based architecture. However, Dominic Chen had created a MongoDB version 3 that works on ARM architecture and published it on his GitHub repository, so I used his binary to create a MongoDB version 3 Docker container. This MongoDB can be used on the ARM architecture, but it does not allow you to use the WiredTiger storage engine due to the 32-bit architecture, so you need to use the MMAPv1 storage engine.

Tutor CLI

I have built an interface to control Open Edx on Treehouses.

Since Treehouses uses the Tor service to host web applications, I modified the Tutor CLI to add the ability to configure Onion addresses for Open Edx and Open Edx Forum.

I then compiled the Tutor CLI and uploaded it to the Treehouses repository.

I then implemented the Treehouses feature to install and configure the Tutor CLI using the Treehouses command. You can use the Tutor CLI or the Treehouses command that wraps the Tutor CLI to operate Open Edx.

Performance and Practicality

The performance of Open Edx on the Raspberry Pi 4 is not very good; the performance is even worse because of using the Tor network.

Also, in the current implementation, the Onion address is used as an email address, so it is not possible to receive email.

Therefore, the current implementation is not practical at all.

1 Like
9

Ah yes, I think that getting it working on a 32-bit Raspberry Pi is going to be too difficult, but there are many 64-bit models of Raspberry Pi available that use ARMv8 64-bit instructions.

10

[EDIT: best to jump to my comment below dated 22-Sep-2022]
Thanks all for digging into this subject a full year before it even hit my radar screen. I recently bought an M2 and decided to give local development a try. On an aside, a mac running Apple Silicon is the low-cost option for freelance developers when you take into consideration all of the software that’s needed in order to avail yourself to would-be customers. Getting it to run a dev-friendly installation of open edx would make it even better.

Now then, following is what i’ve learned so far.

  1. The the official tutor documentation for setting up a local dev environment on ARM based machines is surprisingly easy to follow. I spelunkered around the tutor ecosystem and saw that @regis and @braden in particular worked on this extensively last year, so thanks to both of you for your extra efforts.
  2. [EDIT: see my comment below, 22-Sep-2022] I wrote this bash script for setting up a local dev environment for a client. Anyone reading this could leverage this basic pattern for setting up a local openedx build that includes a custom theme and additional Python requirements. I also wrote this “tutor reset” script for anyone who, like me, ends up making multiple attempts.
  3. The biggest stumbling block for me is making my custom code editable in order to minimize rebuilding the openedx container, at 40+ minutes per iteration. @kmccormick wrote this plugin for auto-mounting a volume, and it seems promising though i’m only just now taking a closer look at it.
  4. The 2nd biggest stumbling block is installing the necessary system software, of which Docker is only one example. The relevant snippets in this install.sh script, though technically written for Ubuntu, are mostly what i used to get my macOS environment setup.
  5. As of this post, MySQL is still fiddly. On my M2, the MariaDB option installs but is buggy. Database migrations froze on my first few attempts, and for now there’s a dearth of searchable info about how to resolve this on google, so I’m going to try the MySQL approach that @braden describes in this thread.

Zooming out. Twenty-five years ago when i was just getting started as a freelance developer i saw a full page magazine advertisement of a guy sitting on a boulder in the wilderness with a laptop staring thoughtfully into the Cascade mountain range, and i thought, “that could be me”. If we can get an Open edX dev environment running locally on a mac laptop then that WILL be me!

2 Likes
11

It’s even easier if you use this plugin: GitHub - open-craft/tutor-contrib-arm64: An unofficial plugin and docker images to make it easier to run Tutor on ARM64 systems :wink:
But the plugin is still unofficial and support is limited as we don’t have a process for keeping the arm64 images up to date on a regular basis. For now, I can updated them manually when someone asks.

What are these, specifically? I generally use homebrew to install such software and it tends to make it quick and easy.

Interesting. I never encountered issues with MariaDB but to be more compatible anyways I updated the arm64 plugin to use MySQL8 since the platform supports it now. I should probably open a PR to make the same change to the docs.

1 Like
12

Ditto re homebrew. it’s more an issue of the quantity of packages, and that these generally need to be installed in a particular order and with particular versions, and inside of particular virtual environments, and that the existence of other installed packages in the os can have unintended consequences that are challenging to diagnose. Per the comment header in this script this is what my vanilla dev environment looks like for Ubuntu, and thus what i’m now installing on my macOS environment:

  • homebrew
  • Python 3.8.12
  • tutor
  • probably also your plugin
  • probably also Kyle’s plugin:
  • edx-platform github repository
  • Open edX system packages for Open edX development
  • All Open edX python requirements for production and development
  • MySQL client software
  • MongoDB client software
  • aws cli
  • kubectl
  • k9s
  • terraform and terragrunt
  • helm

That’s the basic kit that i work with on most projects. i find it more efficient to automatically install all of this at the onset of a project rather than cherry pick the things that i need as i need them. Historically, these are the packages that I’ve added to clients’ bastion servers for access to cloud resources inside the VPC that’s hosting their open edx installation. And then naturally, this is where I’d end up doing most of my ongoing post-deployment code maintenance work for them.

I want to replicate that exact experience inside my local macOS environment.

But then, localizing these packages introduces some new minor challenges in terms of, for example, keeping chinese walls between clients’ aws account credentials, and partitioning custom openedx code packages without actually creating an entire openedx venv for each project. By contrast when developing on an EC2 instance these things are already explicitly taken care of, and so I’m just trying to point out that there’s a lot to think through on this subject.

13

Ah, gotcha. Many Open edX developers may need only the first two on that list installed at the system level, then tutor+plugins in a virtualenv and that’s it. But you’re doing a lot of advanced development and deployment, and so that big list makes sense in that context.

14

Here’s what I came up with. I think it’s generalized to the extent that the couple of scripts that i built will work for both Ubuntu as well as arm64-based macOS machines thanks to OpenCraft/@braden’s most excellent tutor plugin for arm64. The two scripts are:

  • openedx-install-venv.sh. This is a strict interpretation of tutor’s Dockerfile for openedx and seems to work well on both OS’s.
  • tutor-developer-build.sh. This is a codified implementation of this discussion thread combined with the official tutor documentation for creating a development environment. It contains examples and inline documentation for adding more custom code packages to your openedx build.

Screen Shot 2022-09-22 at 19.12.40
Screen Shot 2022-09-22 at 19.12.401884Ă—1070 235 KB

15

@lpm0073 that link isn’t working at the moment.

16

thx. fixed.

2 Likes
17

i spent time this afternoon to try getting tutor dev up on a macOS M2. no luck, for the most part. to be fair, i DID get an instance running; albeit it was one of eight attempts. i ran into a myriad of weird problems: Docker file system corruption, and containers occasionally and randomly dying. I lost count of how many times i rebuilt the openedx container and re-ran tutor dev quickstart.

my one success:

Screen Shot 2022-10-21 at 14.08.16
Screen Shot 2022-10-21 at 14.08.162462Ă—1392 326 KB

Maybe the M3 will be better :frowning:

18

Hello @lpm0073,

Are your problems M2 specific or are they also present in M1?

We managed to get Tutor Dev running on M1, so I am wondering whether we should not upgrade our dev machines to M2.

19

I wonder did you had Rosetta 2 installed/enabled ? This step doesn’t seem to be included in the tutor doc Also @braden mentiond it here and I as well have in my M1 machine. Just specualting it might be the reason…

20

i had similar but different problems with M1. Yesterday with the M2, Docker file system corruption was the most recurring problem. I had to use Docker Desktop’s “Clean / Purge data” trouble shooting utility 3 times because of containers that would somehow break, leaving them in a state where I could not delete them; a different container each time and so there didn’t seem to be any repeatable behavior other than the file corruption itself. I also filled up the 59gb file system once, which was surprising.

  • This unpleasant experience aside, the M2 works great, but not so much that I’d recommend replacing your existing M1.
1 Like