S3 Configuration with Hawthorn (S3ResponseError: 403 Forbidden)

Site Operators Site Operations Help
1

Hello Everyone,
I’m trying to configure Amazon S3 bucket to use it as a storage for course instructor generated reports, I’ve added my S3 access key and secret key and bucket info in lms.auth.json and cms.auth.json, as seen below:

"AWS_ACCESS_KEY_ID": "my-access-key",
  "AWS_QUERYSTRING_AUTH": false,
  "AWS_S3_CUSTOM_DOMAIN": "bucket-name.s3.amazonaws.com",
  "AWS_SECRET_ACCESS_KEY": "my_sevret_key",
  "AWS_STORAGE_BUCKET_NAME": "bucket-name",

and the grade downloads block in lms.env.json and cms.env.json as seen below:

"GRADES_DOWNLOAD": {
    "BUCKET": "bucket-name",
    "ROOT_PATH": "lms-grades",
    "STORAGE_CLASS": "storages.backends.s3boto.S3BotoStorage",
    "STORAGE_KWARGS": {
      "location": "/tmp/edx-s3/grades"
    },
    "STORAGE_TYPE": "s3"
  },

when i try to generate the reports from the instructor data downloads dashboard, i get the below error:

“”"""""""
Traceback (most recent call last):
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/celery/app/trace.py”, line 240, in trace_task
R = retval = fun(*args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/celery/app/trace.py”, line 438, in protected_call
return self.run(*args, **kwargs)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks.py”, line 187, in calculate_grades_csv
return run_main_task(entry_id, task_fn, action_name)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/runner.py”, line 113, in run_main_task
task_progress = task_fcn(entry_id, course_id, task_input, action_name)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/grades.py”, line 207, in generate
return CourseGradeReport()._generate(context)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/grades.py”, line 222, in _generate
self._upload(context, success_headers, success_rows, error_headers, error_rows)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/grades.py”, line 277, in _upload
upload_csv_to_report_store([success_headers] + success_rows, ‘grade_report’, context.course_id, date)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/tasks_helper/utils.py”, line 40, in upload_csv_to_report_store
report_store.store_rows(course_id, report_name, rows)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/models.py”, line 279, in store_rows
self.store(course_id, filename, output_buffer)
File “/edx/app/edxapp/edx-platform/lms/djangoapps/instructor_task/models.py”, line 266, in store
self.storage.save(path, buff)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/files/storage.py”, line 54, in save
return self._save(name, content)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/storages/backends/s3boto.py”, line 417, in _save
self._save_content(key, content, headers=headers)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/storages/backends/s3boto.py”, line 428, in _save_content
rewind=True, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/s3/key.py”, line 1293, in set_contents_from_file
chunked_transfer=chunked_transfer, size=size)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/s3/key.py”, line 750, in send_file
chunked_transfer=chunked_transfer, size=size)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/s3/key.py”, line 951, in _send_file_internal
query_args=query_args
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/s3/connection.py”, line 665, in make_request
retry_handler=retry_handler
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/connection.py”, line 1071, in make_request
retry_handler=retry_handler)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/connection.py”, line 940, in _mexe
request.body, request.headers)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/s3/key.py”, line 884, in sender
response.status, response.reason, body)
S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
AccessDeniedAccess Denied82F5667F28D84275QGz8JvykjcDih8mfX+yWLo4j19VyvYcIF5IKIEAK0WuUv2XlzJfLFTWk0fUP0PXUA7hof4wyR90=
“”’’’’’’’’’’’’’’
fetching the reports works fine, but when i try to generate the report i get the above error.
please note iv’e granted the IAM user the “amazons3fullaccess” policy.
I’m using Hawthorn release.

Regards.

1 Like
2

Hi @SaudH,

Check this google group post. This will help you with the settings if you are missing anything.

You are not able to write to your s3 bucket so check on the access of the bucket, most probably issue will be on s3 side.

And don’t post same question multiple time.

3

Thank you @jramnai for your replay,

my setting is exactly like the one in the link, I’ve also given the IAM user “amazons3fullaccess”.

also, I was able to upload to the bucket from amazon command line using the same user but when I try to generate the grade report I get “S3ResponseError: S3ResponseError: 403 Forbidden”.

4

Have you added Bucket policy and CORS configuration under Permissions tab in s3 bucket?

5

yes, below is the bucket policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-name/*"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "IAM user arn"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::bucket-name"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "IAM user arn"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::bucket-name/*"
        }
    ]
}

I’ve also added the below CORS configuration:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

and for the Block public access section I switched it off for both

“Block public access to buckets and objects granted through new public bucket or access point policies”
and
" Block public and cross-account access to buckets and objects through any public bucket or access point policies"

6

Hello Everyone.

I am facing the same issue, Can you please suggest any solution or give us reference to a documentation that would help to resolve the issue.

Stay safe.