I assume this is an issue with the beta cluster job queue infrastructure. Locally, I can see that a job is enqueued to reset weighted tags for an article.

However, even if the job queue is functioning normally, you could still expect a delay of up to a few minutes, at least, before the search index is updated.

There's a proposed patch in T290315: Add Link: Ensure user does not see invalid tasks in task queue to filter out invalid link recommendations by checking to see if the link recommendations still exist in the database. We don't store image recommendation in the database, but the patch for T295032: Add image: Support loading recommendation data from cache stores image recommendation data for each task in the taskset in a cache. One idea would be to change AddImageSubmissionHandler to purge items from the cache at the same time the job is enqueued to update the search engine. Then the same filtering mechanism proposed in T290315: Add Link: Ensure user does not see invalid tasks in task queue could be extended to filter out image recommendation tasks that don't have corresponding entries in the image recommendation metadata cache.

Change 737916 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] TaskSet: Add ImageRecommendationFilter

https://gerrit.wikimedia.org/r/737916

I was curious about job queue performance (in production, as monitoring in beta is usually broken in some way); apparently mean job delay can be up to 25 minutes, same for p99 job delay. So yeah that's pretty significant and definitely needs some a workaround.

Notes for testing:

• check for the client error

Fetching task suggestions failed: badinteger badinteger 
Object { error: {…}, servedby: "deployment-mediawiki11" }

Change 737916 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] TaskSet: Add ImageRecommendationFilter

https://gerrit.wikimedia.org/r/737916

In T295410#7507069, @Etonkovidova wrote:

Notes for testing:

• check for the client error

Fetching task suggestions failed: badinteger badinteger 
Object { error: {…}, servedby: "deployment-mediawiki11" }

@Etonkovidova that seems unrelated. Are you able to reproduce it reliably? What does the error object show when you expand it?

In T295410#7499419, @Tgr wrote:

I was curious about job queue performance (in production, as monitoring in beta is usually broken in some way); apparently mean job delay can be up to 25 minutes, same for p99 job delay. So yeah that's pretty significant and definitely needs some a workaround.

In the patch, we ended up going with a TTL of 10 minutes per feedback from Discovery-ARCHIVED team.

Change 741135 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@wmf/1.38.0-wmf.9] TaskSet: Add ImageRecommendationFilter

https://gerrit.wikimedia.org/r/741135

Change 741135 abandoned by Kosta Harlan:

[mediawiki/extensions/GrowthExperiments@wmf/1.38.0-wmf.9] TaskSet: Add ImageRecommendationFilter

Reason:

on second thought, will just let this ride the train next week

https://gerrit.wikimedia.org/r/741135

In T295410#7525951, @kostajh wrote:

In T295410#7507069, @Etonkovidova wrote:

Notes for testing:

• check for the client error

Fetching task suggestions failed: badinteger badinteger 
Object { error: {…}, servedby: "deployment-mediawiki11" }

@Etonkovidova that seems unrelated. Are you able to reproduce it reliably? What does the error object show when you expand it?

Yes, the error is not present anymore (checked in betalabs).

Checked for removal add image tasks

• when an image was accepted
• when an image was rejected (with the reason that should remove an image from the pool)
• when skipped - an image stays in the pool

Moving to Test in Production.

@RHo following up from T290315: Add Link: Ensure user does not see invalid tasks in task queue. Currently, if the user rejects a task with the reasons of unfamiliar or foreignlanguage, the task is not removed from the pool of image recommendation tasks available to all users. That means the user could possibly encounter it in their suggested edits feed again.

One option is to refresh the user's feed of tasks after they've submitted an accept/reject (or just reject) action for an image recommendation. If the user's filters are sufficiently broad (i.e. they don't have a single topic for example that for some reason has only a few tasks) then it's unlikely they'd see that specific task again.

Change 741948 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] AddImage: Refresh user's task feed after undecided rejection

https://gerrit.wikimedia.org/r/741948

In T295410#7529904, @kostajh wrote:

@RHo following up from T290315: Add Link: Ensure user does not see invalid tasks in task queue. Currently, if the user rejects a task with the reasons of unfamiliar or foreignlanguage, the task is not removed from the pool of image recommendation tasks available to all users. That means the user could possibly encounter it in their suggested edits feed again.

One option is to refresh the user's feed of tasks after they've submitted an accept/reject (or just reject) action for an image recommendation. If the user's filters are sufficiently broad (i.e. they don't have a single topic for example that for some reason has only a few tasks) then it's unlikely they'd see that specific task again.

To be consistent with the other tasks (unstructured, and add link), in this patch I am resetting the user's task set cache only if they do a rejeciton action using the "undecided" reasons.

Wrong task again! T296491: Add an image: Investigate removing tasks from a user's queue after they have rejected them looks like the right place to move forward this discussion. I've attached the patch to that one, and am moving this back into the Test in Production column.