ElasticSearch not working in Hindi

I’m working on a Hindi language site that was recently upgraded to nutmeg from maple, runs on Kubernetes and was built with Cookiecutter Openedx. I’m trying to enable the Discovery service or barring that, simply to get the search box on the landing page to work with the ENABLE_DASHBOARD_SEARCH feature set to true.

I would expect the query in this screen shot to return exactly one course card; the 3rd from the left on the first row. but instead i get a run-time error.


I get this error regardless of whether or not I enable the Discovery service. The error seems to originate from ElasticSearch, however, there is no error information in the ElasticSearch pod. The lms stack is below, but, the salient information from the trace is the following:

Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

2023-08-07 20:05:14,660 ERROR 7 [search.elastic] [user 4] [ip 192.168.5.109] elastic.py:647 - error while searching index - RequestError(400, 'search_phase_execution_exception', {'error': {'root_cause': [{'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.'}], 'type': 'search_phase_execution_exception', 'reason': 'all shards failed', 'phase': 'query', 'grouped': True, 'failed_shards': [{'shard': 0, 'index': 'course_info', 'node': '4Y9AzFxmREqlE4tnWBVIPQ', 'reason': {'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.'}}], 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.'}}}, 'status': 400}) Traceback (most recent call last): File "/openedx/venv/lib/python3.8/site-packages/search/elastic.py", line 645, in search es_response = self._es.search(index=self.index_name, body=body, **kwargs) File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/client/utils.py", line 168, in _wrapped return func(*args, params=params, headers=headers, **kwargs) File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/client/__init__.py", line 1670, in search return self.transport.perform_request( File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/transport.py", line 415, in perform_request raise e File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/transport.py", line 381, in perform_request status, headers_response, data = connection.perform_request( File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 277, in perform_request self._raise_error(response.status, raw_data) File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/connection/base.py", line 330, in _raise_error raise HTTP_EXCEPTIONS.get(status_code, TransportError)( elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.') 2023-08-07 20:05:14,661 ERROR 7 [search.views] [user 4] [ip 192.168.5.109] views.py:217 - Search view exception when searching for for user 4: RequestError(400, 'search_phase_execution_exception', {'error': {'root_cause': [{'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.'}], 'type': 'search_phase_execution_exception', 'reason': 'all shards failed', 'phase': 'query', 'grouped': True, 'failed_shards': [{'shard': 0, 'index': 'course_info', 'node': '4Y9AzFxmREqlE4tnWBVIPQ', 'reason': {'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.'}}], 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.'}}}, 'status': 400}) Traceback (most recent call last): File "/openedx/venv/lib/python3.8/site-packages/search/views.py", line 186, in course_discovery results = course_discovery_search( File "/openedx/venv/lib/python3.8/site-packages/search/api.py", line 108, in course_discovery_search results = searcher.search( File "/openedx/venv/lib/python3.8/site-packages/search/elastic.py", line 645, in search es_response = self._es.search(index=self.index_name, body=body, **kwargs) File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/client/utils.py", line 168, in _wrapped return func(*args, params=params, headers=headers, **kwargs) File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/client/__init__.py", line 1670, in search return self.transport.perform_request( File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/transport.py", line 415, in perform_request raise e File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/transport.py", line 381, in perform_request status, headers_response, data = connection.perform_request( File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 277, in perform_request self._raise_error(response.status, raw_data) File "/openedx/venv/lib/python3.8/site-packages/elasticsearch/connection/base.py", line 330, in _raise_error raise HTTP_EXCEPTIONS.get(status_code, TransportError)( elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [org] in order to load field data by uninverting the inverted index. Note that this can use significant memory.') [pid: 19|app: 0|req: 25/48] 192.168.5.189 () {64 vars in 4148 bytes} [Mon Aug 7 20:05:14 2023] GET /media/profile-images/f70cff94eafe7e44b9bfcef3698d14bb_50.jpg?v=1655654431 => generated 10201 bytes in 143 msecs (HTTP/1.1 404) 7 headers in 514 bytes (1 switches on core 0) 2023-08-07 20:05:14,673 ERROR 7 [django.request] [user None] [ip None] log.py:224 - Internal Server Error: /search/course_discovery/ [pid: 7|app: 0|req: 24/49] 192.168.5.189 () {74 vars in 4279 bytes} [Mon Aug 7 20:05:14 2023] POST /search/course_discovery/ => generated 54 bytes in 176 msecs (HTTP/1.1 500) 8 headers in 436 bytes (1 switches on core 0)

One additional piece of information that might bear on this problem. I created the MySQL database with the following special-use collation:

CREATE DATABASE openedx CHARACTER SET utf8 COLLATE utf8_general_ci;

i wonder if there’s anything about utf8 and/or utf8_general_ci collation that might cause varchar fields to be treated as text?

I am not sure about the reason of your error, but I have had some gotcha moments with the /courses page

and my understanding it’s not related to the discovery service, but the discovery featuer, the discovery service, is the service that has it’s own tutor plugin, it’s own IDA…etc, while the discvoery featuer is a library that edx developed, that is called edx-search, Looking through the logs of your error.

So regarding this error,

So regarding this I want just to make sure if you used the correct settings for disabling it, which is ENABLE_COURSE_DISCOVERY that tutor sets true by default. setting ref

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.