OEP-19: Dev Docs: Clarifying where documentation should live

Development Architecture
1

A previous discussion is being moved here regarding possible updates to OEP-19 to add clarity to when to use docstrings in modules vs README.rst.

A PR was created with a change here OEP-19: Dev Docs: add clarifications by robrap · Pull Request #135 · openedx/open-edx-proposals · GitHub, but there is still a larger conversation below that has not been resolved.

Ned Batchelder 4:25 PM
…

Also, it has this sentence, “The closer the documentation lives with its corresponding code, the more likely it will be kept up-to-date.” and then a few sentences later, “Documentation should not live within the code itself (e.g., in __init__.py modules).” Anyone understand why?

…

Nimisha Asthagiri 5:17 PM

The closer the documentation lives with its corresponding code, the more likely it will be kept up-to-date

I intended this to be a principle we would follow - essentially, co-locate docs with the code it corresponds to. So that means: ADRs, READMEs, HowTos.

Documentation should not live within the code itself (e.g., in init .py modules).

I used to put long docstrings in the init modules to explain the purpose of the containing directory. However, we then converged on putting that information in RST docs in the directory. Either as READMEs, HowTos, or ADRs. So we’d like to follow that as a consistent convention. (Besides, Python3 no longer requires the existence of those init modules.)

Put another way, when a new reader of the code approaches a new app in our codebase, they would have a consistent experience and look for a consistent location for initial onboarding docs (R, H, ADRs).
…

Ned Batchelder 4 hours ago

The readme’s outside of the docs directory could be included in publishing if a .rst in the docs directory includes them with a directive. But they will be “static” pages, not dynamically associated with the module name as autodoc would do.

Robert Raposa 4 hours ago

Interesting and stinky. We should keep in mind the various use cases.

Use cases for discovering/reading docs:

  • Finding/reading published docs.
  • Navigating code in Github
  • Search (where?)

Use cases for maintaining docs:

  • Finding relevant docs outside of code to update.
  • Maintaining appropriate cross-references to docs. (Automate?)

I wonder if we could automate something like pulling in the README when the module’s autodoc is generated, or something like this?

2

I guess in my mind we have to be clear about what we are actually writing about, and who the audience is.

  • “To use this module, invoke it like this…” This is for callers of the module. These are API docs, and for Python, docstrings are the natural place to put them. A module-level docstring seems fine to me.
  • “This directory is for code that blah blah…” This is for developers contributing to the module. A README is appropriate.
3

Just to add a data point from outside the realm of Python and RST - when writing edx/frontend-platform (all JavaScript code), we spent a good chunk of time coming up with a documentation scheme that felt maintainable and usable.

We ended up writing JSDoc (think javadoc) style comments in the code and then used the jsdoc utility to generate a documentation site from those comments. The end result is pretty nice, if we do say so ourselves: https://edx.github.io/frontend-platform.

We were concerned about writing and maintaining API documentation that wasn’t tied to the code itself. We have a large-ish API surface for frontend-platform (~90 entities), and in the previous versions of the frontend runways (frontend-auth, frontend-analytics, frontend-i18n, and frontend-logging) our README.rst-based API doc tended to get out of sync with the code pretty quick. Furthermore, using JSDoc in the code can be used by intellisense/autocomplete in IDEs, which is extremely valuable when working with these libraries in an app.

And as one last aside, we also ended up writing Markdown in our JSDoc comment. The JavaScript community has decent tooling to produce a doc site from it, but there was virtually no JS/node-based tooling to do the same with RST. It may be off topic for this thread, but I’d humbly suggest we use Markdown for frontend code - it’s where the community is and most frontend folks haven’t even heard of RST.

1 Like
4

I completely agree that it makes sense to update OEP-19 to account for the JavaScript-centered part of the Open edX world.

5

Additional questions regarding README.rst files in our OEP:

  • I had thought READMEs would live directly inside the repo/app/folder, which is where I would generally expect this file. I only just realized the OEP puts this inside the docs folder. I’d like to propose we change this, and wondering what other people thought.
  • I imagined that READMEs for top-level folders would be required. Do others think it should be required like I do? If you don’t think it should be required, what would you say to help people understand when and if it should be written?