IMPORTANT UPDATE
Patching MINIO with MINIO_DOMAIN parameter as described, will have as side effect that none of the other assets (images, translations, avatar etc) will be available, and new uploads will go the wrong buckets.
There is a much simpler solution without changing tutor-minio, please follow these new instructions:
1. Patch, configure and rebuild the openedx image
- Create a plugin (e.g. “video_upload.py” in your plugins folder (tutor plugins printroot), with following content:
from tutor import hooks
hooks.Filters.ENV_PATCHES.add_item(
(
"openedx-dockerfile-post-git-checkout",
"RUN curl -fsSL https://github.com/insad/edx-platform/commit/74f6ef4efbe06839cb574479166ec8c9fb20cad8.patch | git am"
)
)
hooks.Filters.ENV_PATCHES.add_item(
(
"openedx-cms-common-settings",
"FEATURES['ENABLE_VIDEO_UPLOAD_PIPELINE'] = True"
)
)
hooks.Filters.ENV_PATCHES.add_item(
(
"openedx-cms-common-settings",
"VIDEO_UPLOAD_PIPELINE['VEM_S3_BUCKET'] = 'openedxvideos'"
)
)
hooks.Filters.ENV_PATCHES.add_item(
(
"openedx-cms-common-settings",
"VIDEO_UPLOAD_PIPELINE['ROOT_PATH'] = 'upload'"
)
)
- Enable the plugin:
tutor plugins enable video_upload
tutor config save
- Rebuild your openedx image:
tutor images build openedx
- Stop and start tutor:
tutor local stop
tutor local start -d
Explanation
The file edx-platform/videos.py at master · openedx/edx-platform · GitHub is being patched in function storage_service_bucket() as follows:
if waffle_flags()[ENABLE_DEVSTACK_VIDEO_UPLOADS].is_enabled():
params = {
'aws_access_key_id': settings.AWS_ACCESS_KEY_ID,
'aws_secret_access_key': settings.AWS_SECRET_ACCESS_KEY,
'security_token': settings.AWS_SECURITY_TOKEN
}
else:
params = {
'host': settings.AWS_S3_ENDPOINT_URL.replace('https://', ''),
'calling_format': s3.connection.OrdinaryCallingFormat(),
'aws_access_key_id': settings.AWS_ACCESS_KEY_ID,
'aws_secret_access_key': settings.AWS_SECRET_ACCESS_KEY
}
‘host’ will be equal to {{ MINIO_HOST }} (without https, i.e. “files.lmsdomain.tld”), and the calling format will use path-style calling format, instead of the default virtual-host-style calling format - ref. Making requests using the REST API - Amazon Simple Storage Service
2. Patch your reverse proxy configuration and restart your webserver
The proxy server must unescape both x-amz-meta-* parameters from the presigned upload url query string, and add them to the header.
I can only indicate here how to do it in Apache2, people who use NGINX as proxy, please post here below the relevant config settings for your webserver.
The relevant part in my Apache2 configuration is:
RewriteEngine On
RewriteMap ue int:unescape
RewriteCond %{QUERY_STRING} (?:^|&)x-amz-meta-client_video_id=([^&]+)
RewriteRule (.*) - [E=VIDEO_ID:${ue:%1}]
RequestHeader set X-Amz-Meta-Client_video_id %{VIDEO_ID}e env=VIDEO_ID
RewriteCond %{QUERY_STRING} (?:^|&)x-amz-meta-course_key=([^&]+)
RewriteRule (.*) - [E=COURSE_KEY:${ue:%1}]
RequestHeader set X-Amz-Meta-Course_key %{COURSE_KEY}e env=COURSE_KEY
and for being redundant, my complete relevant virtual host setting for Apache2 (with my domain replaced by “lmshost.tld”) is:
<VirtualHost *:80>
ServerName lmshost.tld
Redirect / https://lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName apps.lmshost.tld
Redirect / https://apps.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName courses.lmshost.tld
Redirect / https://courses.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName discovery.lmshost.tld
Redirect / https://discovery.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName ecommerce.lmshost.tld
Redirect / https://ecommerce.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName files.lmshost.tld
Redirect / https://files.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName grades.lmshost.tld
Redirect / https://grades.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName mail.lmshost.tld
Redirect / https://mail.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName minio.lmshost.tld
Redirect / https://minio.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName mobile.lmshost.tld
Redirect / https://mobile.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName notes.lmshost.tld
Redirect / https://notes.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName preview.lmshost.tld
Redirect / https://preview.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName studio.lmshost.tld
Redirect / https://studio.lmshost.tld/
</VirtualHost>
<VirtualHost *:80>
ServerName xqueue.lmshost.tld
Redirect / https://xqueue.lmshost.tld/
</VirtualHost>
<VirtualHost *:443>
ServerName lmshost.tld
ServerAlias *.lmshost.tld
SSLEngine on
LogLevel info
RequestHeader set X-Forwarded-Proto https
RequestHeader set X-Forwarded-SSL on
RewriteEngine On
RewriteMap ue int:unescape
RewriteCond %{QUERY_STRING} (?:^|&)x-amz-meta-client_video_id=([^&]+)
RewriteRule (.*) - [E=VIDEO_ID:${ue:%1}]
RequestHeader set X-Amz-Meta-Client_video_id %{VIDEO_ID}e env=VIDEO_ID
RewriteCond %{QUERY_STRING} (?:^|&)x-amz-meta-course_key=([^&]+)
RewriteRule (.*) - [E=COURSE_KEY:${ue:%1}]
RequestHeader set X-Amz-Meta-Course_key %{COURSE_KEY}e env=COURSE_KEY
ProxyPreserveHost On
ProxyRequests Off
ProxyVia Block
<Proxy *>
Require all granted
</Proxy>
ProxyPass / http://localhost:444/
ProxyPassReverse / http://localhost:444/
ErrorLog /var/log/apache2/openedx_error.log
CustomLog /var/log/apache2/openedx_access.log combined
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/lmshost.tld/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/lmshost.tld/privkey.pem
</VirtualHost>
Don’t forget to restart your webserver:
systemctl restart apache2
3. Test video upload
Your file should be in bucket “openedxvideos”, inside the folder “upload”:
