# MongoDB - Find Block Types in Modulestore

**URL:** <https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743>\
**Category:** Site Operations Help\
**Tags:** how-to\
**Created:** [August 23, 2024, 3:52am UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743 "2024-08-23T03:52:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zachary\_Trabookis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zachary_trabookis/32/753_2.png) [@Zachary\_Trabookis](https://discuss.openedx.org/u/Zachary_Trabookis)\
**Post date:** [August 23, 2024, 3:52am UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743/1 "2024-08-23T03:52:47Z")

</div>

I’m trying to figure out how I can search the MongoDB `openedx` collections and traverse my way up the parents (vertical → sequential → chapter → course) from a given `html` block\_type that I found in the modulestore.definitions collection.

For this example I’ve located the `html` block\_type of `_id: ObjectId('66ba230533475de206c3a260')` using this query.

```auto
{
block_type: "html", 
"fields.data": RegExp(/The video below shows the simulation of a differential drive robot implementing a line-following controller in CoppeliaSim using Python./)
}

```

 ![image](https://us1.discourse-cdn.com/flex020/uploads/openedx/original/2X/8/88f73c519170d7ab3052f0701f2380b2e88d26c5.png)

Searching the `modulestore.structures` collection for the `block.definition` it comes back with timeout. Am I doing this correctly to find an object with this html block in it?

```auto
{
"blocks.block_type": "html",
 "blocks.definition": ObjectId('66ba230533475de206c3a260')
}

```

 ![image](https://us1.discourse-cdn.com/flex020/uploads/openedx/original/2X/7/7f57e740dfda48aff23468634c473a40cd125e85.png)

cc @dave @braden

---

<div class="post-metadata">

**Author:** ![Andres.Aulasneo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/andres.aulasneo/32/984_2.png) [@Andres.Aulasneo](https://discuss.openedx.org/u/Andres.Aulasneo)\
**Post date:** [August 23, 2024, 8:15pm UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743/2 "2024-08-23T20:15:33Z")

</div>

Hi @Zachary_Trabookis!

If I understood well, we’ve done this for Panorama to extract the [course structures](https://github.com/aulasneo/panorama-elt/blob/main/panorama_elt/course_structures_datasource/course_structures_datasource.py) in tabular form from MongoDB. We first extracted all the blocks, and then implemented a recurrent function to find all the parents of each one. Take a look at the code, it might help you work it out.

---

<div class="post-metadata">

**Author:** ![dave](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/dave/32/263_2.png) [@dave](https://discuss.openedx.org/u/dave)\
**Post date:** [August 25, 2024, 7:22pm UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743/3 "2024-08-25T19:22:53Z")

</div>

@Zachary_Trabookis: You have the data relationship correct, so the query you’re talking about will work in that sense. But it’s going to be extremely expensive, since it’s going to full-scan every structure document, which means you’re looking at every historical version of every course in order to see which ones have a pointer to that definition. And it’s possible that many, many of them will. If you added that three months ago to a course, then every unpruned version of that course since that time will have a pointer to it. If you re-ran a course that had that, both the old and new runs have references to it. If it was a in a v1 library and you used it in a course with a `LibraryContentBlock`, then both the library and borrowing course will have references.

Do you mind stepping back a bit and explaining what your end goal is? There might be easier ways to go about it.

---

<div class="post-metadata">

**Author:** ![Zachary\_Trabookis](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/zachary_trabookis/32/753_2.png) [@Zachary\_Trabookis](https://discuss.openedx.org/u/Zachary_Trabookis)\
**Post date:** [August 26, 2024, 1:32pm UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743/4 "2024-08-26T13:32:46Z")

</div>

@dave  
We’re trying to figure out what course this `html` content resides in. When we went to search for it manually in the CMS course where the content should be included, we noticed that the content had disappeared. I wasn’t editing the course, so I was in charge to see if maybe the course uploader had put the content in a different course by mistake or deleted it.

They were last editing the course at the beginning of August 2024, so maybe we could limit the search to `edited_on` to a date range. I believe this still won’t work because it’s checking the values of `edited_on` for all documents like you mentioned.

```auto
  "blocks.edit_info.edited_on": { 
      $gte: ISODate("2024-08-01T00:00:00.000Z"),
      $lt: ISODate("2024-08-11T00:00:00.000Z"),
   }

```

---

<div class="post-metadata">

**Author:** ![dave](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.openedx.org/dave/32/263_2.png) [@dave](https://discuss.openedx.org/u/dave)\
**Post date:** [August 27, 2024, 6:39pm UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743/5 "2024-08-27T18:39:19Z")

</div>

I believe there are some ways you can optimize this, though I don’t remember the specific functions/syntax. I find myself re-reading through the MongoDB docs literally every time I want to query MongoDB for anything.

The ObjectIDs that MongoDB create have a [timestamp encoded into them](https://www.mongodb.com/docs/manual/reference/method/ObjectId.getTimestamp/). Structure documents are never modified after they are written, except as part of the pruning process. So if you know the date range, you should be able to craft a range query for your ObjectIds, which will be indexed and won’t require inspecting the actual contents to narrow down the records.

Also, limiting your query to stop at the first hit might help, if you’re not doing that already.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/openedx/original/2X/e/e0015f89e879bc75925b71fd11133ab15fe98297.png) [@system](https://discuss.openedx.org/u/system)\
**Post date:** [November 25, 2024, 6:39pm UTC](https://discuss.openedx.org/t/mongodb-find-block-types-in-modulestore/13743/6 "2024-11-25T18:39:20Z")

</div>

Closed on November 25, 2024, 6:39pm UTC

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.