Failed to execute refresh metadata course in discovery service

nablisoft · October 14, 2019, 11:37am

Hi;
I want to run the refresh refresh_course_metadata in discovery service with a production environment.
I created an Oauth2 client in lms. I created a partner in discovery admin interface and I set the lms url and the client_id and the client sercret.
When I executed this command I got the following error

Traceback (most recent call last):
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py”, line 31, in execute_loader
loader_class(*loader_args, loader_kwargs).ingest()
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py”, line 82, in ingest
response = self._make_request(initial_page)
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/api.py”, line 120, in _make_request
return self.api_client.courses().get(page=page, page_size=self.PAGE_SIZE, username=self.username)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/slumber/init.py”, line 155, in get
resp = self._request(“GET”, params=kwargs)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/slumber/init.py”, line 103, in _request
raise exceptions.HttpServerError(“Server Error %s: %s” % (resp.status_code, url), response=resp, content=resp.content)
slumber.exceptions.HttpServerError: Server Error 500: http://*********/api/courses/v1/courses/

In the lms logs I found:

Traceback (most recent call last):
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/exception.py”, line 41, in inner
response = get_response(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py”, line 249, in _legacy_get_response
response = self._get_response(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py”, line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/handlers/base.py”, line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/utils/decorators.py”, line 185, in inner
return func(*args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/views/decorators/csrf.py”, line 58, in wrapped_view
return view_func(*args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/views/generic/base.py”, line 68, in view
return self.dispatch(request, *args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 489, in dispatch
response = self.handle_exception(exc)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 477, in dispatch
self.initial(request, *args, **kwargs)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 394, in initial
self.perform_authentication(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/views.py”, line 320, in perform_authentication
request.user
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/request.py”, line 378, in getattribute
return super(Request, self).getattribute(attr)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/request.py”, line 196, in user
self._authenticate()
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework/request.py”, line 342, in _authenticate
user_auth_tuple = authenticator.authenticate(self)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/authentication.py”, line 47, in authenticate
return super(JwtAuthentication, self).authenticate(request)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/rest_framework_jwt/authentication.py”, line 33, in authenticate
payload = jwt_decode_handler(jwt_value)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/decoder.py”, line 57, in jwt_decode_handler
_verify_jwt_signature(token, jwt_issuer)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/decoder.py”, line 137, in _verify_jwt_signature
key_set = _get_signing_jwk_key_set(jwt_issuer)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_rest_framework_extensions/auth/jwt/decoder.py”, line 189, in _get_signing_jwk_key_set
key_set.load_jwks(signing_jwk_set)
File “/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/jwkest/jwk.py”, line 776, in load_jwks
self.load_dict(json.loads(jwks))
File “/usr/lib/python2.7/json/init.py”, line 339, in loads
return _default_decoder.decode(s)
File “/usr/lib/python2.7/json/decoder.py”, line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib/python2.7/json/decoder.py”, line 382, in raw_decode
raise ValueError(“No JSON object could be decoded”)

It seems that the discovery service calls the url: http://**************/api/courses/v1/courses/?page_size=50&page=1&username= with an access token set as an Authorization header

giovannicimolin · November 1, 2019, 11:32pm

Hi @nablisoft,

From your error logs, it seems like the JWK Signing keys aren’t correctly set.
Since Ironwood, edx-platform uses asymmetric JWT cryptographic keys for signing JWTs (you can read more about this here).

To fix that issue, you’ll need to generate a JWK key pair and set both EDXAPP_JWT_PRIVATE_SIGNING_JWK and COMMON_JWT_PUBLIC_SIGNING_JWK_SET on your instance settings.

Generating the keys

To generate these keys, you can use this guide.
You’ll need two values:

serialized_public_keys_json
serialized_keypair_json

Instance settings

If you are deploying a new instance

Add these two Ansible variables to the configuration file (pick the values from the previous step):

COMMON_JWT_PUBLIC_SIGNING_JWK_SET: 'contents of serialized_public_keys_json'
EDXAPP_JWT_PRIVATE_SIGNING_JWK: 'contents of serialized_keypair_json'

If you instance is already up and running

You’ll need to set these values directly on lms.env.json:

// Look for the JWT_AUTH key
{
    ...
    "JWT_AUTH": {
        ...,
        "JWT_PRIVATE_SIGNING_JWK": "contents of serialized_keypair_json",
        "JWT_PUBLIC_SIGNING_JWK_SET": "contents of serialized_public_keys_json",
    },
    ...
}

And you also need to set JWT_PUBLIC_SIGNING_JWK_SET on the configuration files of services you use (discovery, ecommerce, etc). You’ll need to look at the configuration files and search for the JWT_AUTH key, and then set JWT_PUBLIC_SIGNING_JWK_SET.

nablisoft · February 27, 2020, 10:02pm

Thank you for your response.
It resolves my problem

chinmaybhatk · April 24, 2020, 10:11am

@nablisoft @giovannicimolin
I followed the below steps and got into error prompted on the terminal.

ssh -i  user@openedx_instance 
sudo -H -u discovery bash 
source ~/discovery_env 
cd ~/discovery 
./manage.py refresh_course_metadata --partner_code=openedx

here value assigned to partner_code is the Short code created in creating Partners in Discovery Service Administration Panel.

Error prompted are

2020-04-24 10:02:26,698 INFO 2867 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:89 - Retrieving access token for partner [openedx]
2020-04-24 10:02:26,916 ERROR 2867 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:99 - No access token acquired through client_credential flow.
Traceback (most recent call last):
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py”, line 96, in handle
token_type=token_type
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py”, line 66, in get_oauth_access_token
data = response.json()
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py”, line 808, in json
return complexjson.loads(self.text, **kwargs)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/init.py”, line 525, in loads
return _default_decoder.decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 370, in decode
obj, end = self.raw_decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 9 column 1 (char 8)
Traceback (most recent call last):
File “./manage.py”, line 15, in
execute_from_command_line(sys.argv)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/init.py”, line 364, in execute_from_command_line
utility.execute()
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/init.py”, line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/base.py”, line 283, in run_from_argv
self.execute(*args, **cmd_options)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/django/core/management/base.py”, line 330, in execute
output = self.handle(*args, **options)
File “/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py”, line 96, in handle
token_type=token_type
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py”, line 66, in get_oauth_access_token
data = response.json()
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py”, line 808, in json
return complexjson.loads(self.text, **kwargs)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/init.py”, line 525, in loads
return _default_decoder.decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 370, in decode
obj, end = self.raw_decode(s)
File “/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/simplejson/decoder.py”, line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 9 column 1 (char 8)

KomalRajput · May 24, 2020, 2:37pm

Hii @chinmaybhatk , try setting “enable_oauth2_provider” flag to true in both lms.env.json and cms.env.json, then restart lms and cms.

Himanshu_Pandit · November 7, 2020, 11:47am

I have done this configuration afer that error is coming…

2020-11-07 11:41:43,866 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:116 - Command is not using threads to write data.
2020-11-07 11:41:43,868 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:153 - Executing Loader [https://example.com/api/courses/v1/]
2020-11-07 11:41:44,692 ERROR 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - CoursesApiDataLoader failed!
Traceback (most recent call last):
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader
    loader_class(*loader_args).ingest()
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 31, in __init__
    self.username = self.get_username_from_client(self.api_client)
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 41, in get_username_from_client
    token = client.get_jwt_access_token()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 270, in get_jwt_access_token
    self._ensure_authentication()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 254, in _ensure_authentication
    timeout=self._timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 170, in get_and_cache_oauth_access_token
    timeout=timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 115, in get_oauth_access_token
    response.raise_for_status()  # Raise an exception for bad status codes.
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://example.com/oauth2/access_token
2020-11-07 11:41:44,700 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:116 - Command is not using threads to write data.
2020-11-07 11:41:44,701 INFO 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:153 - Executing Loader [https://example.com/api/courses/v1/]
2020-11-07 11:41:44,727 ERROR 11334 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - CoursesApiDataLoader failed!
Traceback (most recent call last):
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader
    loader_class(*loader_args).ingest()
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 31, in __init__
    self.username = self.get_username_from_client(self.api_client)
  File "/edx/app/discovery/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 41, in get_username_from_client
    token = client.get_jwt_access_token()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 270, in get_jwt_access_token
    self._ensure_authentication()
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 254, in _ensure_authentication
    timeout=self._timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 170, in get_and_cache_oauth_access_token
    timeout=timeout,
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 115, in get_oauth_access_token
    response.raise_for_status()  # Raise an exception for bad status codes.
  File "/edx/app/discovery/venvs/discovery/lib/python3.5/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://example.com/oauth2/access_token
CommandError: One or more of the data loaders above failed.

can you help me in this…

AbdullahDagia · March 31, 2021, 7:21am

Hey,
Did you find any solution to the problem?

mohan.krishna · April 13, 2021, 8:16am

@Himanshu_Pandit @AbdullahDagia

Did anyone of you found a solution for the above issue?

Please suggest.

Thanks.

Himanshu_Pandit · May 10, 2021, 2:50pm

Yes, Create discovery partner with correct configurations-

LMS URL:
Courses API URL:
Site:
Organizations API URL:
then run refresh_course_metadata and update_index

Topic		Replies	Views
Unable to access Discovery API Site Operations Help api , how-to	1	750	November 29, 2022
Discovery course_metadata error Development	0	289	August 24, 2021
Tutor local launch Internal Server Error for url: http://lms:8000/oauth2/access_token Development	2	150	April 17, 2024
Refresh metadata course in discovery service: data too long for column Site Operators koa	0	267	October 24, 2022
Tutor: Upgrade to Nutmeg - Discovery plugin init finishes with error Site Operations Help tutor	15	871	September 25, 2022

Failed to execute refresh metadata course in discovery service

Generating the keys

Instance settings

If you are deploying a new instance

If you instance is already up and running

Related Topics