I’m using an external SMTP provider for Open EdX, and have been for over a year, and it’s been working reliably. But since upgrading to the latest major Tutor version 14.0.4, no emails are being sent. I tried to send an email (e.g. a password reset email) and the following was logged:
Origin 'https://training.step-data.uk' was not in CORS_ORIGIN_WHITELIST; full referer was 'https://training.step-data.uk/login' and requested host was 'training.step-data.uk'; CORS_ORIGIN_ALLOW_ALL=False
I figure the way to solve this is to set the whitelist. Here’s how I did that:
vim "$(tutor config printroot)/config.yml"
Add a new line with this: CORS_ORIGIN_WHITELIST: ["https://training.step-data.uk"]
tutor config save
tutor local exec lms reload-uwsgi
It didn’t seem to work, the same error is logged with a test email.
When I re-edit the config file (vim "$(tutor config printroot)/config.yml") I can see that it didn’t actually save the new line with the CORS whitelist: that line is missing. I double checked that changes to existing lines persist (such as editing the PLATFORM_NAME: line), but it doesn’t like this CORS whitelist line, and every time I run tutor config save it removes it.
Why won’t it save CORS_ORIGIN_WHITELIST: ["https://training.step-data.uk"]? Is the syntax bad?
You are supposed to use Plugins to add this type of configuration instead of directly editing the config.yml file. config.yml is for configuration of tutor itself rather than Open edX.
The following plugin will do what you require.
from tutor import hooks
hooks.Filters.ENV_PATCHES.add_item(
(
"openedx-lms-production-settings",
'CORS_ORIGIN_WHITELIST.append("https://training.step-data.uk")'
),
)
@uetuluk Thank you so much for that! That makes sense. After implementing that and re-testing, it no-longer gives an error about CORS_ORIGIN_WHITELIST. However, email still won’t send, with an error “Not authorised to send from this header address”. I have searched for solutions to this error but couldn’t find any posts on it.
A redacted error log is below. Any help is greatly appreciated.
... INFO 6 [audit] [user None] [ip xxx ] password_reset.py:610 - Password reset initiated for email xxx@xxx.com.
... ERROR 6 [edx_ace.channel.django_email] [user None] [ip xxx ] django_email.py:70 - (551, b'5.7.1 Not authorised to send from this header address')
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/edx_ace/channel/django_email.py", line 68, in deliver
mail.send()
File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/message.py", line 284, in send
return self.get_connection(fail_silently).send_messages([self])
File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 109, in send_messages
sent = self._send(message)
File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 125, in _send
self.connection.sendmail(from_email, recipients, message.as_bytes(linesep='\r\n'))
File "/opt/pyenv/versions/3.8.12/lib/python3.8/smtplib.py", line 906, in sendmail
raise SMTPDataError(code, resp)
smtplib.SMTPDataError: (551, b'5.7.1 Not authorised to send from this header address')
{"level":"info","ts":1662024121.9064996,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_addr":"XXX:55454","proto":"HTTP/2.0","method":"POST","host":"training.step-data.uk","uri":"/account/password","tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"training.step-data.uk"}},"user_id":"","duration":3.113652072,"size":0,"status":200}
[pid: 6|app: 0|req: 4/21] 172.18.0.9 () {64 vars in 1453 bytes} [Thu Sep 1 09:21:58 2022] POST /account/password => generated 0 bytes in 3110 msecs (HTTP/1.1 200) 7 headers in 268 bytes (1 switches on core 0)
There are a few things that could be going on. I haven’t set up openedx for mail delivery, so I can only comment on general email troubleshooting.
But are you attempting to send emails through the built-in smtp server, or your email server/service?
If using the built in openedx smtp server, your mail could be getting rejected by the recipient mail server if the smtp server isn’t in your domain’s spf record. If you’re using your mail server, you may need to perform additional configuration to actually send smtp messages.
I’m going to assume you’re using a step-data.uk email address. So I queried the domain’s spf records with nslookup (you can use the type flag to get different record types: nslookup -type=txt step-data.uk), and I see you only have spf records for fastmail.
That means you’ve identified fastmail servers as the only authorized sender.
For the smtp server, you could try adding the public ip address of the openedx smtp server to your spf record. It doesn’t look like step-data.uk is using DKIM.
If this doesn’t fix it, I can ask the person who set up emails in our edx environment to take a look. But we would need some more context surrounding your edx mail settings.
@edazzo Thanks so much for helping to to investigate this.
I set up Open EdX to use my fastmail smtp server, as it’s a trusted email provider and so will do substantially better at passing spam filters than any mail I send directly from step-data.uk smtp servers.
Yes, that’s right.
Yes, that’s right. I’ve done this with several sites, including Open EdX which was working until recently. I hadn’t changed any smtp configuration and suddenly it’s stopped working.
Right you are! Well, step-data.uk was already set up, but I hadn’t done training.step-data.uk. I’ve set that up now and according to this test it is connect to the mail server correctly for both the root domain and now for the training subdomain, and the SPF and DKIM seem to be working fine.
Where I’m at now
I’ve double-checked I’ve followed the instructions at the tutor use gmail for smtp tutorial, modifying the values to work for fastmail. At the end it gets you to do a test:
tutor local run --no-deps lms ./manage.py lms shell -c \
"from django.core.mail import send_mail; send_mail('test subject', 'test message', 'redacted@step-data.uk', ['me@example.com'])"
This message sends just fine and gets to my inbox.
But when I test whether Open EdX can send a forgot password email, I get the same error in the log as before: Not authorised to send from this header address
This message sends just fine and gets to my inbox.
But when I test whether Open EdX can send a forgot password email, I get the same error in the log as before: Not authorised to send from this header address
The issue you’re seeing now sounds similar to something my team was troubleshooting this morning. We were able to send mail in tests, but not from openedx services.
It turned out openedx was sending from a different email address than the one we configured. We’re still troubleshooting the issue, but right now we’re trying to set BULK_EMAIL_DEFAULT_FROM_EMAIL to the correct address in a tutor plugin.
We originally thought a different parameter controlled the from address.
I can let you know what worked for us if the change doesn’t work for you.
But the fact that your test runs shows it’s not an issue with your provider sending emails from redacted@step-data.uk. tutor local run will run a command from the local lms container. Since that sends, it’s safe to assume this is an openedx configuration problem.
This sounds promising. Could you let me know how you changed that please? I’ve just figured out how to control this kind of setting in Tutor using a custom plugin, for example using the following snippet:
But as for BULK_EMAIL_DEFAULT_FROM_EMAIL, I don’t know whether this is part of common-env-features (as in that snippet), or is it something else like openedx-lms-production-settings (as in other snippets I’ve found).
I’ve figured it out. I was doing some other work on another server and got exactly the same error message while trying to send an email: Not authorised to send from this header address. I realised this is not an Open EdX-generated message, in fact it’s from Fastmail, which is the smtp host.
The problem was that actually no emails were being sent from this server to any address that wasn’t controlled by Fastmail. Once I set up the server to reliably send emails (using ssmtp), then Open EdX could also send emails without any problems.
I really appreciate everyone’s suggestions and help on this one.
