.delay` with `task.apply_async` to apply the `countdown` (I’ve chosen a 5-sec countdown randomly) fixes the issue permanently:
```python
# update_search_index.delay(course_key_str, datetime.now(UTC).isoformat())
update_search_index.apply_async((course_key_str, datetime.now(UTC).isoformat()), countdown=5)
```
- the issue isn’t reproducible if to call the task as a regular function (without `.delay` or `.apply_async`)
- Using `time.sleep` and trying to get the course object with the delay (I’ve tried up to 15sec) has no effect - the course image is always outdated
- the issue isn’t reproduced if using debug (for the `CourseDetails.update_from_json` method) and walking the code step-by-step. The `course_published` signal is fired twice at some moment (didn’t find the reason yet)
- I didn’t find any caching issues according to getting the course data from mongo for the index update
* * *
Does anyone have any clues about the issue?
---
### Author:  [@chintan](https://discuss.openedx.org/u/chintan)
#### Post date: [February 14, 2023, 6:17am UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/2 "2023-02-14T06:17:25Z")
I am not sure about the newer releases, but in older releases, that and about page were cached if user was not logged in.
Also as you mentioned the re-index was not happening properly.
What I did to work around is exactly the same thing but with cron jobs instead of a task.
---
### Author:  [@Evgen\_Dyudyunov](https://discuss.openedx.org/u/Evgen_Dyudyunov)
#### Post date: [February 14, 2023, 7:30am UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/3 "2023-02-14T07:30:03Z")
Hello @chintan
Thanks for the response! Yeah, I’m aware of the `@cache_if_anonymous()` but I had my user logged in.
Anyway, its good to know it is not my exclusive issue))
What release you were working with?
---
### Author:  [@chintan](https://discuss.openedx.org/u/chintan)
#### Post date: [February 14, 2023, 8:00am UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/4 "2023-02-14T08:00:30Z")
I did the cron in KOA and in Nutmeg, haven’t added them or tested them yet with Olive.
---
### Author:  [@regis](https://discuss.openedx.org/u/regis)
#### Post date: [February 14, 2023, 9:18am UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/5 "2023-02-14T09:18:33Z")
I did not know about this issue, but it very much looks like a race condition is happening. The following are very strong clues:
> [@Evgen\_Dyudyunov](#):
>
> in some cases (e.g. workers just restarted) the issue is not reproducible on the first try
Race conditions typically don’t occur 100% of the time.
> [@Evgen\_Dyudyunov](#):
>
> replacing `.delay` with `task.apply_async` to apply the `countdown` (I’ve chosen a 5-sec countdown randomly) fixes the issue permanently
Running the task is run with a 5s delay fixes the race condition.
> [@Evgen\_Dyudyunov](#):
>
> the issue isn’t reproducible if to call the task as a regular function (without `.delay` or `.apply_async`)
The race condition seems to come from the asynchronous worker.
> [@Evgen\_Dyudyunov](#):
>
> Using `time.sleep` and trying to get the course object with the delay (I’ve tried up to 15sec) has no effect - the course image is always outdated
`sleep` does not fix the race condition because of the GIL. (I think?)
> [@Evgen\_Dyudyunov](#):
>
> the issue isn’t reproduced if using debug
Pausing causes a delay, which fixes the race condition.
---
### Author:  [@ghassan](https://discuss.openedx.org/u/ghassan)
#### Post date: [February 14, 2023, 12:23pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/6 "2023-02-14T12:23:25Z")
> [@Evgen\_Dyudyunov](#):
>
> - I didn’t find any caching issues according to getting the course data from mongo for the index update
I think the index is probably stored in Redis or/and ES, to ensure it’s from Redis side you can try to reproduce the probelm and then flush everything in Redis.
I think it’s [edx-search repo](https://github.com/openedx/edx-search) that is responsible for the content `/courses` page _The default case in tutor_.
---
### Author:  [@Evgen\_Dyudyunov](https://discuss.openedx.org/u/Evgen_Dyudyunov)
#### Post date: [February 14, 2023, 1:13pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/7 "2023-02-14T13:13:35Z")
@ghassan yes, the index is stored in the ES
> I think it’s [edx-search repo](https://github.com/openedx/edx-search) that is responsible for the content `/courses` page
This is also true ^ (if `FEATURES['ENABLE_COURSE_DISCOVERY'] = True`)
From what I’ve got from the logs/debugging I see that the problem is that the wrong data is sent **to** the ES for storing. The task responsible for that sending receives outdated course data from mongo.
I’m wondering now how I could confirm or refute @regis clues about the race condition
---
### Author:  [@ghassan](https://discuss.openedx.org/u/ghassan)
#### Post date: [February 14, 2023, 1:27pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/8 "2023-02-14T13:27:52Z")
> [@Evgen\_Dyudyunov](#):
>
> I’m wondering now how I could confirm or refute @regis clues about the race condition
My 2 cent, is that the task to update to ES index is triggerd before the content is updated \_i.e. there are two async tasks of which one depends on the other, so it might be that those tasks are running in parallel, instead in a chain or pipeline \_ I am just speculating here
---
### Author:  [@Evgen\_Dyudyunov](https://discuss.openedx.org/u/Evgen_Dyudyunov)
#### Post date: [February 14, 2023, 1:33pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/9 "2023-02-14T13:33:28Z")
> [@ghassan](#):
>
> task to update to ES index is triggerd before the content is updated
I excluded this because the reindex task is fired on the `course_published` signal which is sent after the actual course update in the mongo
---
### Author:  [@ghassan](https://discuss.openedx.org/u/ghassan)
#### Post date: [February 14, 2023, 1:41pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/10 "2023-02-14T13:41:02Z")
Aha I am not really that familiar about the sequence of events. I think @dave might have something to say about this
---
### Author:  [@dave](https://discuss.openedx.org/u/dave)
#### Post date: [February 27, 2023, 5:41pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/11 "2023-02-27T17:41:16Z")
It might be a race condition as @regis indicated, where workers aren’t seeing the data that’s modified by the web view. This is especially possible if it’s reading from CourseOverviews.
Another race-condition-adjacent possibility that might be worth looking into is whether or not there is clock drift between the server running the web frontend, the server running the celery workers, and _possibly_ MongoDB (I don’t remember exactly how the modified timestamp is created). The current timestamp is getting passed to the indexing function, which implies that it’s using the timestamp to narrow down the range of content changes it has to index. If the clocks are different, the workers may be looking at the wrong time window and not see the change.
---
### Author:  [@dave](https://discuss.openedx.org/u/dave)
#### Post date: [February 27, 2023, 5:51pm UTC](https://discuss.openedx.org/t/automatic-course-reindex-after-course-publishing/9347/12 "2023-02-27T17:51:39Z")
Another potential race condition can happen if MongoDB is configured to read from secondaries–that’s usually the best thing to do for spreading load, but there could be propagation delay that prevents the workers from seeing it. It’s possible you’re not seeing that effect when everything is run in process because Modulestore’s internal caching is preventing it from re-fetching the data.