Issue Connecting Cairn to an External PostgreSQL Database in Tutor (Mac M3)

Site Operators Tutor Help
1

Hi everyone,

I’m trying to configure Cairn in Tutor to use an external PostgreSQL database instead of the built-in one. I’m following the instructions from the Tutor Cairn repository, but I haven’t been able to get it working properly.

Goal:

The goal is to connect Cairn Superset to an external PostgreSQL database.

Steps Followed:

  1. Disabled the built-in PostgreSQL in Tutor:

    tutor config save --set CAIRN_RUN_POSTGRESQL=false

  2. Set the connection details for the external PostgreSQL database:

    tutor config save --set CAIRN_POSTGRESQL_HOST=<external_db_host>
tutor config save --set CAIRN_POSTGRESQL_PORT=5432
tutor config save --set CAIRN_POSTGRESQL_DATABASE=<db_name>
tutor config save --set CAIRN_POSTGRESQL_USERNAME=<db_user>
tutor config save --set CAIRN_POSTGRESQL_PASSWORD="<db_password>"

  3. Launched an external PostgreSQL container:

    docker run -d --name <external_db_container> \
  -e POSTGRES_USER=<db_user> \
  -e POSTGRES_PASSWORD=<db_password> \
  -e POSTGRES_DB=<db_name> \
  -v <external_db_volume>:/var/lib/postgresql/data \
  --network=tutor_default \
  postgres:14

  4. Verified that the container is running and the database is accessible:

    docker exec <external_db_container> bash -c "psql -U <db_user> -d <db_name> -c 'SELECT datname FROM pg_database;'"
    • Output:
      datname  
-----------
postgres
<db_name>
template1
template0
(4 rows)

  5. Rebuilt the Tutor environment and restarted services:

    rm -rf "$(tutor config printroot)/env"
tutor config save
tutor local stop
tutor local start -d

Issue:

Even though all configurations are set, Cairn Superset continues to look for cairn-postgresql instead of the new external database. The error message I receive is:

service "cairn-superset" depends on undefined service "cairn-postgresql": invalid compose project

Running:

grep -r "cairn-postgresql" "$(tutor config printroot)/env"

I found references to cairn-postgresql in the following files:

  • superset_config.py
  • docker-compose.yml
  • docker-compose.jobs.yml

I manually replaced all occurrences of cairn-postgresql with <external_db_host>, but when I restart Tutor, it seems to overwrite these files and revert to the default settings.

System Details:

  • Tutor version: (19.0.0)
  • OS: macOS (MacBook with M3 chip)
  • Docker version: (28.0.1)
  • PostgreSQL version: 14 (running as a Docker container)
  • Cairn plugin version: (19.0.0)

Questions:

  • Is there an additional step needed to make Cairn recognise the new PostgreSQL database?
  • Do I need to modify any additional files that I might have missed?
  • Has anyone successfully configured Cairn to use an external PostgreSQL instance instead of the default one?

Any help would be greatly appreciated! Thanks in advance!

2

Hi @anastasiouxrhstos, thank you for pointing out this issue.

This issue has now been resolved in tutor-cairn v19.0.2. Please update to this version to incorporate the fix.

It seems tutor-cairn was adding cairn-postgresql as a dependency even when CAIRN_RUN_POSTGRESQL was set to false. Furthermore, the CAIRN_POSTGRESQL_HOST and CAIRN_POSTGRESQL_PORT configuration settings did not exist. We’ve added those as well now.

2 Likes