:information_source: Context: the Paver *assets* module was [accepted for deprec…ation back in May 2023](https://github.com/openedx/edx-platform/issues/31895), but we haven't removed it yet. Now, in May 2024, we are ready to deprecate the entire Paver suite. For simplicity, we are deprecating *all* Paver commands in this single ticket, assets included.
### Timeline
#### Communicated
2024-05-01
#### Target Acceptance
2024-05-21
#### Earliest Open edX Named Release Without This Functionality
Sumac
#### Latest Open edX Named Release With the Replacement Available
Internal CI commands: Sumac
All other commands, including Asset commands: Redwood
(See table below for details)
### Rationale
edx-platform historically handled its build scripts with [paver](https://github.com/paver/paver):
> Paver is a Python-based software project scripting tool along the lines of Make or Rake. It is not designed to handle the dependency tracking requirements of, for example, a C program. It is designed to help out with all of your other repetitive tasks (run documentation generators, moving files about, downloading things), all with the convenience of Python’s syntax and massive library of code.
Using paver has a few problems:
* **Paver adds dependencies to edx-platform:** Paver follows the Python packaging model, so anything that's a dependency of edx-platform's [pavelib suite](https://github.com/openedx/edx-platform/tree/master/pavelib) becomes a dependency of edx-platform itself. In addition to the `paver` package, this means that we install `libsass-python` in production, which takes 60+ seconds (!) to compile and install.
* **Paver makes Python a requirement for all build actions:** When your build tooling runs on Python, then you need a Python environment in order to do anything, even things that don't involve Python (including building static assets, running JS tests, etc.). This harms the cachability of edx-platform Dockerfiles, because it means that any change to the Python requirements list will invalidate any build steps used by Paver.
* **Paver is indirect:** Paver basically just wraps shell commands that you could have run directly, adding a layer of complexity and potential bugs. Build scripts are ideally very simple--unlike application code, repetition is preferable to abstraction; they should be easy to write, easy to read, and easy to throw away and replace if necessary. Standard tools like Makefiles, Bash scripts, and Dockerfiles all encourage and excel at this sort of simplicity; Paver does not.
* **Paver is idiosyncratic:** Unlike Make and Bash, Paver is not at all a standard part of a programmer's toolset, so it makes edx-platform less approachable for new contributors. Even for programmers who want to script in Python, [Click](https://click.palletsprojects.com/en/8.1.x/) is significantly more popular, well-documented, and user-friendly.
For the Paver Asset commands in particular, more depth is provided in the ['Reimplement edx-platform static asset processing' ADR](https://github.com/openedx/edx-platform/blob/master/docs/decisions/0017-reimplement-asset-processing.rst).
### Replacement
**Module** | **Replacement**
-------------|-----------------
*assets* | [New static assets guide](https://github.com/kdmccormick/edx-platform/tree/kdmccormick/assets-final/docs/references/static-assets.rst)
*quality* | Directly invoking tools (`pylint`, etc.) when possible, Makefile targets otherwise
*test_js* | Makefile target
*prereq* | `pip`
*docs* | Makefile target
*i18n* | Makefile targets for common tasks; [i18n_tool](https://github.com/openedx/i18n-tools) for advanced tasks
*servers* | Tutor (or the deprecated Docker-based Devstack, or [bare-metal development](https://github.com/kdmccormick/edx-platform/blob/kdmccormick/assets-final/README.rst#bare-metal-advanced)
### Migration
To ease migration for Paver users, in Redwood, [each Paver Asset command will simply proxy to its replacement command](https://github.com/openedx/edx-platform/pull/34554), and will raise a deprecation warning explaining the new command that it is running.
#### Requirements
In Redwood and earlier, Paver and its dependencies were included in requirements/edx/base.txt
Starting in Sumac, these dependencies will be **removed** from requirements/edx/base.txt. Instead, operators will need to install:
* requirements/edx/assets.txt to build static assets
* requirements/edx/testing.txt to run Python tests and linting
* requirements/edx/base.txt download translations and collect static assets
#### Commands
**Module** | **Known Users** | **Replacement Status** | **Before** | **After**
-------------|----------------------------|------------------------|-----------------------------------------------------------|-----------
*assets* | Ansible, Devstack | Ready to use |`paver update_assets` | `npm run build && ./manage.py lms collecstatic --noinput && ./manage.py cms collectstatic`ns
*assets* | None | Ready to use |`paver process_xmodule_assets` | No longer needed - no replacement
*assets* | Tutor, pavelib | Ready to use |`paver compile_sass` | `npm run compile-sass`
*assets* | pavelib | Ready to use |`./manage.py [lms/cms] compile_sass` | `npm run compile-sass`
*assets* | pavelib | Ready to use |`paver webpack` | `npm run webpack`
*assets* | Devstack | Ready to use |`paver watch_assets` | `npm run watch`
*quality* | edx-platform CI | Need to implement |`paver find_fixme` | `make find_fixme`
*quality* | edx-platform CI | Need to implement |`paver run_eslint` | `eslint`
*quality* | edx-platform CI | Need to implement |`paver run_stylelint` | `make stylelint_js`
*quality* | edx-platform CI | Need to implement |`paver run_xsslint` | `make xsslint`
*quality* | edx-platform CI | Need to implement |`paver run_pii_check` | `make pii_check`
*quality* | edx-platform CI | Need to implement |`paver check_keywords` | `make check_keywords`
*quality* | edx-platform CI | Need to implement |`paver run_quality` | `make quality`
*quality* | edx-platform CI | Ready to use |`paver run_pylint` | `pylint`
*quality* | edx-platform CI | Ready to use |`paver run_pep8` | `pycodestyle`
*js_test* | edx-platform CI | Need to implement |`paver diff_coverage` | `make diff_coverage_js`
*js_test* | edx-platform CI | Need to implement |`paver test_js` | `make test_js`
*js_test* | edx-platform CI | Need to implement |`paver test_js_run` | `make test_js`
*js_test* | edx-platform CI | Need to implement |`paver test_js_dev` | `make test_js MODE=browser`
*prereq* | Ansible, Devstack, pavelib | Ready to use |`paver install_prereqs` | `pip install -r requirements/edx/base.txt -r requirements/edx/assets.txt && npm clean-install`
*prereq* | pavelib | Ready to use |`paver install_node_prereqs` | `npm clean-install`
*prereq* | pavelib | Providers need to impl |`paver uninstall_python_packages` | `pip uninstall -yr $ANSIBLE_UNINSTALL_LIST`
*prereq* | pavelib | Ready to use |`paver install_coverage_prereqs` | `pip install -r requirements/edx/coverage.txt`
*prereq* | pavelib | Ready to use |`paver install_python_prereqs` | `pip install -r requirements/edx/base.txt`
*docs* | None | Ready to use |`paver build_docs` | `make docs`
*i18n* | None | Ready to use |`paver i18n_validate_gettext` | `which xgettext`
*i18n* | None | Ready to use |`paver i18n_extract` | `make extract_translations`
*i18n* | None | Ready to use |`paver i18n_dummy` | `i18n_tool dummy && i18n_tool generate`
*i18n* | None | Ready to use |`paver i18n_generate` | `i18n_tool generate`
*i18n* | None | Ready to use |`paver i18n_generate_strict` | `i18n_tool generate --strict`
*i18n* | None | Ready to use |`paver i18n_clean` | `make clean_translations`
*servers* | Vagrant Devstack | Ready to use |`paver check_settings` | None
*servers* | Vagrant Devstack | Ready to use |`paver lms` | `./manage.py lms runserver`
*servers* | Vagrant Devstack | Ready to use |`paver studio` | `./manage.py cms runserver`
*servers* | Vagrant Devstack | Ready to use |`paver run_all_servers` | `tutor local ...`
*servers* | Vagrant Devstack | Ready to use |`paver devstack` | `tutor dev ...`
Notes:
* 'Ansible' is the deprecated [configuration](https://github.com/openedx/configuration) repo.
* 'Devstack' refers to the [Docker-based devstack](https://github.com/openedx-unsupported/devstack) which was deprecated recently but still in use by some community members.
* 'Vagrant Devstack' refers to the old VM-based devstack which was replaced by Docker-based devstack eight years ago. We do not retain any support for this.
* 'pavelib' is the [pavelib folder in edx-platform](https://github.com/openedx/edx-platform/tree/master/pavelib) that defines Paver tasks. In other words, these commands are only used by other Paver tasks.
* The `paver uninstall_python_packages` command is only used by Ansible and Devstack. Providers who still use those deprecated repositories will need to maintain an `ANSIBLE_UNINSTALL_LIST` file containing packages that need to be removed and add the `pip uninstall` command to their pipeline.
#### Django settings (for Asset commands)
In order to reimplement the Paver Asset commands without Python/Django, several Django settings must now be specified as environment variables. Remove these from your Django setting overrides. Instead, add them to your the shell environment that you use to build and run LMS and CMS.
* `STATIC_ROOT_LMS` (mirrored in LMS as `settings.STATIC_ROOT`)
* `STATIC_ROOT_CMS` (mirrored in CMS as `settings.STATIC_ROOT`)
* `COMPREHENSIVE_THEME_DIRS`, colon-separated (parsed into LMS and CMS as a list, `settings.COMPREHENSIVE_THEME_DIRS`)
* `WEBPACK_CONFIG_PATH` (formerly available in LMS/CMS as `settings.WEBPACK_CONFIG_PATH`; to be removed)
* `JS_ENV_EXTRA_CONFIG` (formerly available in CMS as `settings.JS_ENV_EXTRA_CONFIG`; to be removed)
### Deprecation
In time for Redwood, deprecation warnings will be added to all edx-platform Paver Asset commands.
### Removal
* In edx-platform:
* rm -rf ./pavelib/
* rm ./requirements/edx/paver.in
* rm ./requirements/edx/paver.txt
* Search docs for 'paver' and replace with new commands
* Elsewhere in the openedx GitHub org:
* Search docs for 'paver' and replace with new commands
* In Tutor:
* Replace paver Asset references with new commands
```[tasklist]
### Tasks/PRs
- [ ] https://github.com/openedx/edx-platform/issues/31624
- [ ] https://github.com/openedx/edx-platform/pull/31810
- [ ] https://github.com/openedx/edx-platform/pull/31934
- [ ] https://github.com/openedx/edx-platform/pull/31790
- [ ] https://github.com/openedx/edx-platform/pull/32717
- [ ] https://github.com/openedx/edx-platform/pull/32767
- [ ] https://github.com/openedx/edx-platform/pull/32804
- [ ] https://github.com/openedx/edx-platform/pull/32823
- [ ] https://github.com/openedx/edx-platform/pull/32866
- [ ] https://github.com/openedx/edx-platform/pull/34318
- [ ] https://github.com/openedx/edx-platform/pull/34468
- [ ] https://github.com/openedx/edx-platform/pull/34554
- [ ] https://github.com/overhangio/tutor/pull/1042
- [ ] TODO update all edx-platform documentation
- [ ] TODO update external documentation
- [ ] TODO remove paver scripts and deps from edx-platform
```
I want to seriously suggest that for Sumac we aim to have an option to build+run Open edX with no frontend files in the