Is it possible to inherit from the open edx templates into my own django application. For example I would like to inherit from the courseware/courseware-chromeless.html into a template that I will modify. This template will live in a django app that I will install on open edx.
You can look into creating a comprehensive theme. More information about that is available here.
A comprehensive theme will let you modify most templates used by Open edX and does not need to be installed as a separate django app.