Documenting Open edX settings

As part of an ongoing collaboration with edX (more specifically with @nimisha and @robrap), we are in the process of documenting Open edX Django settings. We are talking about the settings located in lms/envs/common.py, production.py, ecommerce/settings/production.py, etc. There are hundreds of those settings in edx-platform alone and most of them are scarcely documented. Understanding what every setting does is a crucial part of operating an Open edX platform. Right now, the most reliable way to understand what a setting does is to grep the source code and reverse-engineer its impact on the application. This requires extensive developer experience about Open edX and is not a sustainable approach.

We are working to produce a human-readable documentation for all Open edX settings. This documentation would be similar in spirit (but likely not in implementation, see why below) to the Django settings documentation: https://docs.djangoproject.com/en/3.0/ref/settings/

However, edx-platform is a very large project with many Python requirements. When generating the documentation in readthedocs.org (the primary documentation building target for all Open edX applications) we cannot afford to install all development requirements. Thus, the Sphinx job responsible for generating the docs should not be importing all python modules from edx-platform. If you are famililar with Sphinx, then you know this is usually how documentation is generated for a Python project: python modules are imported, objects from these modules are listed and the __doc__ attribute for every object is examined. In edx-platform, importing python modules would result in ImportError exceptions because we cannot afford to install all dependencies.

Thus, we need to figure out an alternative approach. The general principle is that we should do our best to keep the documentation as close as possible to the code (see this documentation manifesto by @nedbat). We settled on a tool that directly parses the Open edX source code using regular expressions. This tool is called code_annotations: https://github.com/edx/code-annotations It has a minimal set of dependencies and is quite efficient.

Documenting a repository with code annotations works as follows:

  1. Define an annotation format using yaml. For instance:
...
annotations:
  feature_toggle:
    - ".. toggle_name:":
    - ".. toggle_implementation:":
        choices: [ExperimentWaffleFlag, WaffleFlag, WaffleSample, WaffleSwitch, CourseWaffleFlag, ConfigurationModel, DjangoSetting]
    - ".. toggle_default:":
    - ".. toggle_description:":
   ...
  1. Run code_annotation on the source repo by passing this yaml configuration file as argument. This generates a report.yml file in which all annotations are collected.
  2. Use this machine-readable report to generate a human-readable documentation with Sphinx. To achieve that, we create a relatively simple Sphinx extension. The final result is published on readthedocs.org.

We already have a prototype in place for documenting feature toggles: https://draft-feature-toggles.readthedocs.io/en/latest/
This documentation is generated from this pull request: https://github.com/edx/edx-platform/pull/24344

Now, we want to apply a similar approach to Django settings. We are still in step #1, where we need to define the appropriate documentation format. I suggest the following:

"""
.. setting_name: MY_CUSTOM_SETTING
.. setting_default: "somevalue"
.. setting_description: Enable the foobarizing capability of the BAR microservice 
  which is  used in conjunction with the Klunk package. You should enable the 
  teletransmitter if you set this to a non-real value.
"""
MY_CUSTOM_SETTING = "somevalue"

We would like to collect feedback from the Open edX developer community regarding this approach. What do you think? Is this bad/good/don’t care? Would you be interested in contributing? Please tell us!

2 Likes

This is something that is long overdue and I’m interested in contributing to the effort.

At the risk of creating a distraction, can I ask what is the difference between a “toggle” and a “setting” ?

This is all up for discussion, but I think there are Django Settings that are feature toggles and Django Settings that are non-toggle settings. OEP-17: Feature Toggles is all about feature toggles. An example of a non-toggle Django Setting would be LMS_ROOT_URL.

I think the proposal is that .. toggle would be used to annotate feature toggles, which could include some Django Settings, and .. setting would be used to annotate non-toggle settings.

Note that toggles will likely include metadata to help retire them in the event that they were only necessary for short-term uses cases like a monitored rollout. I imagine non-toggle settings probably won’t need the same level of metadata, but we’ll see as we start documenting.

1 Like