Timeout with randomized contents with large libraries

On our Lilac installation, we have a course with a subsection that has 30 problems. When we try to load the course outline on /courses/course-v1:{…}/course we receive a 504 timeout error and even using the “Live version” button to see the content it returns the same error.

The unit on this subsection has 30 randomized components using the same library with 200 problems.

We try different combinations and here are the results:
image

We know library components may not have the best performance, but someone else has experienced this kind of problem or has been working on an update to improve the performance of randomized content?

2 Likes

Thanks @Ian2012. Just to give a bit more context, we created this course as a benchmark for the practical limits of randomized components. We thought it was a good idea to share, so if anyone encountered limits of library size that affects performance has this data.

Also for context, app servers are 16GB memory and 4vCPU (m5.xlarge) with a mongo cluster self hosted on 3 VMs of the same type.

2 Likes

@Ian2012 and @Felipe, I don’t have specific info on libraries v1 performance, but I’m not surprised it’s not… great. It would be interesting to see if the issues are similar to the ones that affect sections with many units and XBlocks, as @omar once laid out in this post. The idea would be to mimic what @dave did to profile the issue.

2 Likes

Thanks for posting this folks. It’s really good to have these data points. Is the block structures collect phase being cached in this scenario?

Thanks @dave for your help here. Yes, the block structures phase is being cached. I’m collecting more info that I will leave in a comment with some statistics.
What more information could be helpful?