Properly configure Openedx to use AWS S3 storage

Site Operators
1

Hi

I deployed a native instance of Openedx [Juniper] and I am at the stage of configuring storage. I want to use AWS S3. It seems that there are two phases: 1) configuring S3 and 2) configuring Openedx.

I think i properly configured S3 [ARN and Policy]. So, I got the following:

AWS_SECRET_ACCESS_KEY
AWS_STORAGE_BUCKET_NAME
AWS_ACCESS_KEY_ID
AWS_S3_CUSTOM_DOMAIN

So what are the steps forward to configure Openedx to use S3 for ORA, ImportExportCourses and Grades? The Docs seems to be very scattered on the web. Others are incomplete and stored in old Google Forum discussions…

So, which files should be changed and what commands should be applied?
I think answers to this thread, will benefit many developers who newly got their hands into Openedx

Thanks
HN

2

Hi @nachha,

Did you try following the steps mentioned here and here and configure all the required variables with the right values?

Below are the configuration variables that our OpenCraft Instance Manager generates for using S3 as the storage backend for an instance. You can explore the code in the repository to check how the buckets are provisioned and these variables are configured. If doing all this appears to be complicated and you’d rather have someone else run and manage it for you, you can check out our hosting plans.

AWS_ACCESS_KEY_ID: <access key>
AWS_S3_LOGS: true
AWS_S3_LOGS_ACCESS_KEY_ID: <access key>
AWS_S3_LOGS_SECRET_KEY: <secret key>
AWS_SECRET_ACCESS_KEY: <secret key>
COMMON_ENABLE_AWS_INTEGRATION: true
COMMON_ENABLE_AWS_ROLE: true
COMMON_OBJECT_STORE_LOG_SYNC: true
COMMON_OBJECT_STORE_LOG_SYNC_BUCKET: <bucket name>
COMMON_OBJECT_STORE_LOG_SYNC_PREFIX: <bucket name with hyphens replaced by underscore>/logs/tracking/
EDXAPP_AUTH_EXTRA:
  AWS_STORAGE_BUCKET_NAME: <bucket name>
EDXAPP_AWS_ACCESS_KEY_ID: <access key>
EDXAPP_AWS_LOCATION: <bucket name with hyphens replaced by underscore>
EDXAPP_AWS_S3_CUSTOM_DOMAIN: <bucket name>.s3.<s3 region>.amazonaws.com
EDXAPP_AWS_SECRET_ACCESS_KEY: <secret>
EDXAPP_AWS_STORAGE_BUCKET_NAME: <bucket name>
EDXAPP_DEFAULT_FILE_STORAGE: storages.backends.s3boto.S3BotoStorage
EDXAPP_FILE_UPLOAD_BUCKET_NAME: <bucket name>
EDXAPP_FILE_UPLOAD_STORAGE_PREFIX: <bucket name with hyphens replaced by underscore>/submissions_attachments
EDXAPP_GRADE_BUCKET: <bucket name>
EDXAPP_GRADE_ROOT_PATH: <bucket name with hyphens replaced by underscore>/grades-download
EDXAPP_GRADE_STORAGE_CLASS: storages.backends.s3boto.S3BotoStorage
EDXAPP_GRADE_STORAGE_KWARGS:
  bucket: <bucket name>
  location: <bucket name with hyphens replaced by underscore>/grades-download
EDXAPP_GRADE_STORAGE_TYPE: s3
EDXAPP_IMPORT_EXPORT_BUCKET: <bucket name>
EDXAPP_PROFILE_IMAGE_BACKEND:
  class: storages.backends.s3boto.S3BotoStorage
  options:
    headers:
      Cache-Control: max-age-{{ EDXAPP_PROFILE_IMAGE_MAX_AGE }}
    location: <bucket name with hyphens replaced by underscore>/profile-images
XQUEUE_AWS_ACCESS_KEY_ID: <access key>
XQUEUE_AWS_SECRET_ACCESS_KEY: <secret key>
XQUEUE_UPLOAD_BUCKET: <bucket name>
XQUEUE_UPLOAD_PATH_PREFIX: <bucket name with hyphens replaced by underscore>/xqueue

For ORA2, I’d recommend following the steps mentioned in its official documentation.

Please note that these S3 configuration values have been tested only for buckets created in the us-east-1 region. Other regions may use different authentication/signature versions and hence might require additional configuration that is not specified in the sample configuration above.

1 Like
3

hi @nachha , I’ve also deployed open edx(Juniper release).Now I’m trying to configure aws s3, for videos learners can see in the course?
Have you figured out on how to do it on Juniper version of open edx.
Thanks.