Is it possible to send SQL queries to an external database from inside a Jupyter Notebook (in the context of teaching a course in database/data science), and return query results?
I think that should be possible. You can set up permissions/firewalls for this SQL server to only accept connections from the server running your Jupyter Notebook.
Do you mean the edx.org site itself or another instance of Open edX (edx-platform etc)? In the former case I don’t know, I am not aware of how it is configured. It will need some configuration beyond my understanding to securely set up a connection there.
For your own instance AWS etc it should be a lot more straightforward to set up. Without more knowledge of that setup it’s hard to advise.