filing as a security task since this deals with suppression of nonpublic information and in case more information is needed about the suppression

I can't find any case related to this request in logstash. Can you try again. Is all oversights broken at wikidata atm?

I can reproduce it with my admin revdel. It's not an issue, it's just really slow to respond. The revdel goes in really fast. I think it's due the work @Lucas_Werkmeister_WMDE, @ItamarWMDE and @hoo did on the revdel actually deleting revision from client wikis too (T242164: Retract revdel'ed Wikidata edits from Wikibase client watchlists)

Can you take a look? I can help on anything needed.

Hm, do you have any data on what exactly is being slow (maybe a flame graph or something)? ArticleRevisionVisibilitySetHookHandler already schedules jobs to do the heavy lifting on the client wikis; does scheduling the jobs (and determining the job parameters) still take so long?

XHGui says out of 1 minute it spends on the request, almost all of it is being spent on curl to push 700 jobs: https://performance.wikimedia.org/xhgui/run/view?id=6006c8550b368568dd253a1f

(The xhgui is basically the xray of the request, it has lots of information)

Okay, I was looking a bit at how to optimize the data collection part (there are some queries that could probably be coalesced into one), but I guess that’s not the problem.

…schedule a single repo job to schedule those 700 client jobs?

In T272334#6757454, @Lucas_Werkmeister_WMDE wrote:

Okay, I was looking a bit at how to optimize the data collection part (there are some queries that could probably be coalesced into one), but I guess that’s not the problem.

…schedule a single repo job to schedule those 700 client jobs?

Yes, that's how ChangeNotification job works.

Thanks for filling this task @DannyS712, was considering filling one about the general slowness as well (it's...annoying). Looking forward to a fix!

In T272334#6757435, @Lucas_Werkmeister_WMDE wrote:

Hm, do you have any data on what exactly is being slow (maybe a flame graph or something)? ArticleRevisionVisibilitySetHookHandler already schedules jobs to do the heavy lifting on the client wikis; does scheduling the jobs (and determining the job parameters) still take so long?

Perhaps the scheduling of jobs itself needs to be a job

Got it again

Request from [redacted] via cp1077 cp1077, Varnish XID 268277460
Error: 503, Backend fetch failed at Fri, 05 Feb 2021 20:02:27 GMT

@ItamarWMDE are you working on this? You assigned it to yourself but there hasn't been any update on it since then

@Ladsgroup yes I am, I was at a conference this Friday.

Releasing this ticket to be picked up by another dev in the WMDE team, due to issues with local dev environment

Now even deletions of ordinary pages started to be notably slower. According to https://w.wiki/zNW, it looks like same/similar core issue. Can we prioritize this?