I have a particular function in which I am wrapping all of the data updates within a transaction.atomic so as to rollback everything if any of the steps fail. However, what I have noticed is that this only encompasses the changes in mysql, while the mongodb updates are persisted. Is there a way to ensure that the two data stores remain in sync and rollback mongodb changes if the mysql transaction gets rolled back?
In general I don’t think it’s possible. You can try to put in manual double-transaction closing in your exception handling code, and can probably get something that mostly works but at some point you have to pick one database to commit the changes to first, and any errors that occur between that point and the point where you commit the changes to the second database are going to roll back the second database’s transaction and give you an inconsistent state. That’s a (small) part of the motivation for removing MongoDB.
Are you talking about mongo in the context of course content? Because if so, I think you can get the same effect because of how modulestore now works.
When you make a change to a course, the Split Modulestore writes the course data into mongo as new “versions” in the structures and definitions collections. The existing structures and definitions aren’t modified in any way. So the new content won’t get used until the active_versions data is updated to say that the draft/published course is now using the new version of those structures/definitions. But last year, I moved active_versions into MySQL as SplitModulestoreCourseIndex, and that’s what is now used to determine the version of the course.
So: the upside should be that even if you make some course changes in a MySQL transaction and those changes get persisted to MongoDB, when you roll back the transaction it should roll back the update to SplitModulestoreCourseIndex in MySQL, and the new data in MongoDB will just be ignored and have no effect. In essence, you’ve rolled back both, although you technically have some orphaned rows in MongoDB, so you wouldn’t want to do that in something that’s going to roll back with high frequency.
2 Likes
Thank you for this really helpful explanation @braden. I’ll accept the orphans for now and rely on that versioning as my mongo rollback strategy for the time being.
1 Like