It looks like that doc is out of date, there is a simpler form now. You can see all of the options with:
tutor local do transform-tracking-logs --help
A working example is:
tutor local do transform-tracking-logs --source_provider LOCAL --source_config '{"key": "/openedx/data/", "prefix": "tracking.log", "container": "logs"}' --destination_provider LRS --transformer_type xapi --dry_run
That example will run against the local tracking log file in Tutor local, but just as a test to make sure the file can be found. Nothing will be sent to Ralph or ClickHouse until you remove the --dry_run
option.
It’s very likely that your Tutor LMS worker image doesn’t mount the logs directory and only an empty file will be found. I’m hoping to address this with the Tutor maintainers soon, but in the meantime you can temporarily edit this file:
$TUTOR_ROOT/env/local/docker-compose.jobs.yml
To add this line to the lms-job
→ volumes
key right above the depends_on
line. Then simply run the command I gave you above and it should find the file. This change will be overwritten the next time you tutor config save
.
With --dry_run
on you should see a bunch of standard Django warnings then output like:
Found 1 files in logs/tracking.log*
Found 1 source files:
tracking.log - 61888649 bytes
Looking for log files in logs/tracking.log*
Streaming file <Object: name=tracking.log, size=61888649, hash=90ced65510b0c9eb7b1be588e1bed9c0, provider=Local Storage ...>...
Dry run, skipping, but still clearing the queue.
...
Dry run, skipping, but still clearing the queue.
Finalizing 180 events to LRS
Dry run, skipping final storage.
Queued 100180 log lines, could not parse 0 log lines, skipped 5372 log lines, sent 0 batches.
Without --dry_run
if will print many, many statements to the screen by default, but the final output will look like:
Queued 100180 log lines, could not parse 0 log lines, skipped 5372 log lines, sent 11 batches.
I’ll need to ask you a bunch of questions to try to understand your needs for getting custom data. Please contact me on Slack and then we can move the conversation back here if that makes sense.
The short version is that some options are:
- Override the default xAPI statements to include more data, if that makes sense for your case
- Create new xAPI statements and dbt transforms if the data can be represented as xAPI or you want to use the LRS functionality for this data
- Make your MySQL data queryable and either call it directly in Superset or make the data available to ClickHouse
The first two options are outlines here and here. The last one has some potential tradeoffs and performance implications on the LMS.