XAPI Bulk Dispatch Failure due to LRS 500 Error

Site Operators Site Operations Help
, ,
1

Hi all :waving_hand:

We have experienced an error in our OpenEdx instance (https://uamx.uam.es) related with xAPI events not being dispatched.

I am not sure whose dispatching the error and why is it happening (is it Ralph, Redis, Clickhouse, ¿Mongo maybe?…) The fact is that MongoDB was down, but I believe that Mongo is not related with xAPI management… We have Clickhouse, Redis, Mongo and MySQL running natively in the same external machine. In the other hand, we have an instance of OpenEdx running with tutor local in another machine, where there is Superset, Ralph and a small Redis instance for Aspects…

The error is the following (as it was captured by Sentry):

EventNotDispatched**
Issue ID:** 6899679375
Project: uamx
Date: 13/12/2025, 13:24:02

Issue Summary

XAPI Bulk Dispatch Failure due to LRS 500 Error
What’s wrong: Bulk dispatch of 5 xAPI events failed via LrsClient.
In the trace: The underlying HTTP POST to the LRS returned a 500 Internal Server Error.
Possible cause: The remote LRS responded with “Statements bulk indexation failed”, suggesting a server-side issue.

Tags

  • celery_task_id: d5d0f8f5-b41b-4f16-8681-78714887d0fb
  • environment: produccion
  • handled: yes
  • level: error
  • logger: event_routing_backends.tasks
  • mechanism: logging
  • release: abbc7a21c384b131b3aba1a799db84b8ef45bf4c
  • runtime: CPython 3.11.8
  • runtime.name: CPython
  • server_name: f1b884d9fdac
  • transaction: event_routing_backends.tasks.dispatch_bulk_events

Exception

Exception 1

Type: EventNotDispatched

Stacktrace

 bulk_send in event_routing_backends/utils/xapi_lrs_client.py [Line 103] (Not in app)
            else:
                logger.warning(f"Failed request: {response.request.content}")
                logger.warning('{} request failed for sending xAPI statement of edx events to {}. '
                               'Response code: {}. Response: {}'.format(response.request.method, self.URL,
                                                                        response.response.code, response.data))
                raise EventNotDispatched  <-- SUSPECT LINE

    def send(self, statement_data, event_name):
        """
        Send the xAPI statement to configured remote.

---
Variable values:
{
  "response": "<tincan.lrs_response.LRSResponse object at 0x7fb495f160d0>",
  "self": "<event_routing_backends.utils.xapi_lrs_client.LrsClient object at 0x7fb49c331fd0>",
  "statement_data": [...... ]
}

=======
 bulk_send_events in event_routing_backends/tasks.py [Line 127] (Not in app)
        logger.error('Unsupported routing strategy detected: {}'.format(router_type))
        return

    try:
        client = client_class(**host_config)
        client.bulk_send(events)  <-- SUSPECT LINE
        logger.debug(
            'Successfully bulk dispatched transformed versions of {} events using client: {}'.format(
                len(events),
                client_class
            )
---
Variable values:
{
  "client": "<event_routing_backends.utils.xapi_lrs_client.LrsClient object at 0x7fb49c331fd0>",
  "client_class": "<class 'event_routing_backends.utils.xapi_lrs_client.LrsClient'>",
  "events": [....... ]
}

=======

Thanbks very much in advance!

2

Hello!

This would be a 500 error being thrown by Ralph, so you should be able to find out more about what’s happening in the Ralph logs. Ralph doesn’t do very much so it’s likely to be an issue with Ralph communicating to Clickhouse, but hopefully you’ll get enough information to debug it from the logs.

3

Thanks @TyHob, I’ll check it out. I always forgot about Ralph logs… :grinning_face_with_smiling_eyes:

Regards!