Failed to execute refresh metadata course in discovery service

Site Operators
,
1

Hi;
I want to run the refresh refresh_course_metadata in discovery service with a production environment.
I created an Oauth2 client in lms. I created a partner in discovery admin interface and I set the lms url and the client_id and the client sercret.
When I executed this command I got the following error

Traceback (most recent call last):
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py”, line 31, in execute_loader
loader_class(*loader_args, loader_kwargs).ingest()
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py”, line 82, in ingest
response = self._make_request(initial_page)
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py”, line 120, in _make_request
return self.api_client.courses().get(page=page, page_size=self.PAGE_SIZE, username=self.username)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/slumber/init.py”, line 155, in get
resp = self._request(“GET”, params=kwargs)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/slumber/init.py”, line 103, in _request
raise exceptions.HttpServerError(“Server Error %s: %s” % (resp.status_code, url), response=resp, content=resp.content)
slumber.exceptions.HttpServerError: Server Error 500: http://*********/api/courses/v1/courses/

In the lms logs I found:

Traceback (most recent call last):
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/exception.py”, line 41, in inner
response = get_response(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py”, line 249, in _legacy_get_response
response = self._get_response(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py”, line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py”, line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/utils/decorators.py”, line 185, in inner
return func(*args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/views/decorators/csrf.py”, line 58, in wrapped_view
return view_func(*args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/views/generic/base.py”, line 68, in view
return self.dispatch(request, *args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 489, in dispatch
response = self.handle_exception(exc)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 477, in dispatch
self.initial(request, *args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 394, in initial
self.perform_authentication(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 320, in perform_authentication
request.user
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/request.py”, line 378, in getattribute
return super(Request, self).getattribute(attr)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/request.py”, line 196, in user
self._authenticate()
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/request.py”, line 342, in _authenticate
user_auth_tuple = authenticator.authenticate(self)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/authentication.py”, line 47, in authenticate
return super(JwtAuthentication, self).authenticate(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework_jwt/authentication.py”, line 33, in authenticate
payload = jwt_decode_handler(jwt_value)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/decoder.py”, line 57, in jwt_decode_handler
_verify_jwt_signature(token, jwt_issuer)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/decoder.py”, line 137, in _verify_jwt_signature
key_set = _get_signing_jwk_key_set(jwt_issuer)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/decoder.py”, line 189, in _get_signing_jwk_key_set
key_set.load_jwks(signing_jwk_set)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/jwkest/jwk.py”, line 776, in load_jwks
self.load_dict(json.loads(jwks))
File “/usr/lib/python2.7/json/init.py”, line 339, in loads
return _default_decoder.decode(s)
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw_decode
raise ValueError(“No JSON object could be decoded”)

It seems that the discovery service calls the url: http://**************/api/courses/v1/courses/?page_size=50&page=1&username= with an access token set as an Authorization header

2

Hi @nablisoft,

From your error logs, it seems like the JWK Signing keys aren’t correctly set.
Since Ironwood, edx-platform uses asymmetric JWT cryptographic keys for signing JWTs (you can read more about this here).

To fix that issue, you’ll need to generate a JWK key pair and set both EDXAPP_JWT_PRIVATE_SIGNING_JWK and COMMON_JWT_PUBLIC_SIGNING_JWK_SET on your instance settings.

Generating the keys

To generate these keys, you can use this guide.
You’ll need two values:

  • serialized_public_keys_json
  • serialized_keypair_json

Instance settings

If you are deploying a new instance

Add these two Ansible variables to the configuration file (pick the values from the previous step):

COMMON_JWT_PUBLIC_SIGNING_JWK_SET: 'contents of serialized_public_keys_json'
EDXAPP_JWT_PRIVATE_SIGNING_JWK: 'contents of serialized_keypair_json'

If you instance is already up and running

You’ll need to set these values directly on lms.env.json:

// Look for the JWT_AUTH key
{
    ...
    "JWT_AUTH": {
        ...,
        "JWT_PRIVATE_SIGNING_JWK": "contents of serialized_keypair_json",
        "JWT_PUBLIC_SIGNING_JWK_SET": "contents of serialized_public_keys_json",
    },
    ...
}

And you also need to set JWT_PUBLIC_SIGNING_JWK_SET on the configuration files of services you use (discovery, ecommerce, etc). You’ll need to look at the configuration files and search for the JWT_AUTH key, and then set JWT_PUBLIC_SIGNING_JWK_SET.

3 Likes
3

Thank you for your response.
It resolves my problem :slight_smile:

4

@nablisoft @giovannicimolin
I followed the below steps and got into error prompted on the terminal.

ssh -i  user@openedx_instance 
sudo -H -u discovery bash 
source ~/discovery_env 
cd ~/discovery 
./manage.py refresh_course_metadata --partner_code=openedx

here value assigned to partner_code is the Short code created in creating Partners in Discovery Service Administration Panel.

Error prompted are

2020-04-24 10:02:26,698 INFO 2867 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:89 - Retrieving access token for partner [openedx]
2020-04-24 10:02:26,916 ERROR 2867 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:99 - No access token acquired through client_credential flow.
Traceback (most recent call last):
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py”, line 96, in handle
token_type=token_type
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py”, line 66, in get_oauth_access_token
data = response.json()
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py”, line 808, in json
return complexjson.loads(self.text, **kwargs)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/init.py”, line 525, in loads
return _default_decoder.decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 370, in decode
obj, end = self.raw_decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 9 column 1 (char 8)
Traceback (most recent call last):
File “./manage.py”, line 15, in
execute_from_command_line(sys.argv)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/init.py”, line 364, in execute_from_command_line
utility.execute()
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/init.py”, line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/base.py”, line 283, in run_from_argv
self.execute(*args, **cmd_options)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/base.py”, line 330, in execute
output = self.handle(*args, **options)
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py”, line 96, in handle
token_type=token_type
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py”, line 66, in get_oauth_access_token
data = response.json()
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py”, line 808, in json
return complexjson.loads(self.text, **kwargs)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/init.py”, line 525, in loads
return _default_decoder.decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 370, in decode
obj, end = self.raw_decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 9 column 1 (char 8)

5

Hii @chinmaybhatk , try setting “enable_oauth2_provider” flag to true in both lms.env.json and cms.env.json, then restart lms and cms.

1 Like
6

I have done this configuration afer that error is coming…

2020-11-07 11:41:43,866 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:116 - Command is not using threads to write data.
2020-11-07 11:41:43,868 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:153 - Executing Loader [https://example.com/api/courses/v1/]
2020-11-07 11:41:44,692 ERROR 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - CoursesApiDataLoader failed!
Traceback (most recent call last):
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader
    loader_class(*loader_args).ingest()
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 31, in __init__
    self.username = self.get_username_from_client(self.api_client)
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 41, in get_username_from_client
    token = client.get_jwt_access_token()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 270, in get_jwt_access_token
    self._ensure_authentication()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 254, in _ensure_authentication
    timeout=self._timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 170, in get_and_cache_oauth_access_token
    timeout=timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 115, in get_oauth_access_token
    response.raise_for_status()  # Raise an exception for bad status codes.
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://example.com/oauth2/access_token
2020-11-07 11:41:44,700 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:116 - Command is not using threads to write data.
2020-11-07 11:41:44,701 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:153 - Executing Loader [https://example.com/api/courses/v1/]
2020-11-07 11:41:44,727 ERROR 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - CoursesApiDataLoader failed!
Traceback (most recent call last):
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader
    loader_class(*loader_args).ingest()
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 31, in __init__
    self.username = self.get_username_from_client(self.api_client)
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 41, in get_username_from_client
    token = client.get_jwt_access_token()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 270, in get_jwt_access_token
    self._ensure_authentication()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 254, in _ensure_authentication
    timeout=self._timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 170, in get_and_cache_oauth_access_token
    timeout=timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 115, in get_oauth_access_token
    response.raise_for_status()  # Raise an exception for bad status codes.
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://example.com/oauth2/access_token
CommandError: One or more of the data loaders above failed.

can you help me in this…

7

Hey,
Did you find any solution to the problem?

8

@Himanshu_Pandit @AbdullahDagia

Did anyone of you found a solution for the above issue?

Please suggest.

Thanks.

9

Yes, Create discovery partner with correct configurations-

  • LMS URL:
  • Courses API URL:
  • Site:
  • Organizations API URL:
    then run refresh_course_metadata and update_index