ORA2 S3 Backend Configuration

Site Operators
1

Hey everyone - I am having some trouble with the ORA module using S3. There are a few things at play here but let’s start with my configurations.

We are using @lpm0073 's cookie-cutter and configurations. This includes using tutor-contrib-s3 which is working across the board except for ORA where it only partially works.

The issue I ran into is that it seems the ORA library is not configuring S3 in the same way as the rest of openedx. This results in failures to upload files when using pre-signed URLs. The error from AWS is:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256

Specifically, ORA is not using the AWS_S3_SIGNATURE_VERSION and AWS_S3_REGION_NAME environment variables when setting up boto3.

To get around this, I have forked edx-ora2 and made these changes.

def _connect_to_s3():
    """Connect to s3
    Creates a connection to s3 for file URLs.
    """
    # Try to get the AWS credentials from settings if they are available
    # If not, these will default to `None`, and boto3 will try to use
    # environment vars or configuration files instead.
    aws_access_key_id = getattr(settings, "AWS_ACCESS_KEY_ID", None)
    aws_secret_access_key = getattr(settings, "AWS_SECRET_ACCESS_KEY", None)
    endpoint_url = getattr(settings, "AWS_S3_ENDPOINT_URL", None)
+   signature_version = getattr(settings, "AWS_S3_SIGNATURE_VERSION ", None)
+   region_name = getattr(settings, "AWS_S3_REGION_NAME", None)

    return boto3.client(
        "s3",
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key,
        endpoint_url=endpoint_url,
+       config=Config(
+          signature_version=signature_version, 
+          region_name=region_name
           )
    )

Do these changes need to be pulled into edx-ora2? Or are these changes not necessary if I change my configuration?

1 Like
2

i think the problem, or at least part of the problem, is that you’ll need to define a proxy service for the AWS S3 storage bucket, and then route traffic to this service from a dedicated ingress. eduNEXT worked out this problem a while back, and @Donato_Bracuto has been guiding me thru how their solution works on another project.

Note the following example manifests taken from this Stack Overflow post:
To define the service:

kind: Service
apiVersion: v1
metadata:
  name: ora-proxy-service
  namespace: my-openedx
spec:
  type: ExternalName
  externalName: s3://my-openedx-storage/ora/
selector: {}

To route traffic to this service:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-openedx-ora-proxy-service
  namespace: my-openedx
  annotations:
    # add sticky sessions
    # ---------------------
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "openedx_sticky_session"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

    # how it works in Caddy:
    # ----------------------
    #  @scorm_matcher {
    #    path /scorm-proxy/*
    #  }
    #  route @scorm_matcher {
    #    uri /scorm-proxy/* strip_prefix /scorm-proxy
    #    reverse_proxy https://codlp-global-pre-staging-storage.s3.amazonaws.com {
    #      header_up Host codlp-global-pre-staging-storage.s3.amazonaws.com
    #    }
    #  }

spec:
  rules:
  - host: courses.my-openedx.edu
    https:
      paths:
      - path: /ora-proxy/
        pathType: Prefix
        backend:
          service:
            name: ora-proxy-service
            port:
              number: 443
  - host: studio.courses.my-openedx.edu
    https:
      paths:
      - path: /ora-proxy/
        pathType: Prefix
        backend:
          service:
            name: ora-proxy-service
            port:
              number: 443

these were JUST merged to main in the cookiecutter earlier today

3

Additionally and perhaps more to the point, you might need to redeploy in order to pickup the latest changes to tutor plugin tutor-contrib-k8s-deploy-tasks. Here’s a snippet of what was added last week:

def scorm_xblock_storage(xblock):
  from django.conf import settings
  from storages.backends.s3boto3 import S3Boto3Storage

  if SERVICE_VARIANT == "lms":
    domain = settings.LMS_BASE
  else:
    domain = settings.CMS_BASE

  return S3Boto3Storage(
    bucket=AWS_STORAGE_BUCKET_NAME,
    access_key=AWS_ACCESS_KEY_ID,
    secret_key=AWS_SECRET_ACCESS_KEY,
    querystring_expire=86400,
    custom_domain=f"{domain}/scorm-proxy"
  )

XBLOCK_SETTINGS["ScormXBlock"] = {
  "STORAGE_FUNC": scorm_xblock_storage,
}
4

@lpm0073 the issue occurs when attempting to generate the pre-signed URL for upload. The SDK doesn’t seem to use the correct region or signature version, which I don’t think would be resolved via a proxy. Even with a proxy, the signature and region would still be incorrect.

Regarding the XBLOCK_SETTINGS, it doesn’t appear that ORA has any such settings or the possibility to override the _connect_to_s3 function. If it did, then that I think that would work as a solution.

1 Like