https://meta.wikimedia.org/wiki/Special:GAll of the last 15 globalR renameProgress displays that there are 14 global renames stucks got stuck on metawiki (or never even scheduled there, allthere are no relevant logs, at meta.wikimedia.
When the job tried to run on Meta it seems that it has failed there for some reason and didn't continued on the rest of the projectsneither errors nor the rename start message). Running `fixStuckGlobalRename.php` on meta works fine.
Requesting that the logs be checked to see if the jobs are still trying to restart themselves,The last successful non-CLI rename on meta [[https://logstash.wikimedia.org/goto/eefd5dc5925a12c135cf53b53fe321bf|was at]] 2018-04-26T15:24:07, the next one [[https://logstash.wikimedia.org/goto/e2de83f58e90a111987e671448f72f39|should have been]] around 18:00 but did not get triggered. The only possibly relevant [[https://tools.wmflabs.org/sal/production?d=2018-04-26|SAL event]] in that time frame is [[https://gerrit.wikimedia.org/r/c/428970/|428970: Enable all jobs for test, test2, testwikidata and mediawiki]]. if notmediawikiwiki directly precedes metawiki, requeue the jobs and investigate why this has happenedand rename jobs run in a chain (a successful job schedules the job for the next wiki) so maybe that breaks with Kafka somehow ([[https://github.com/wikimedia/mediawiki-extensions-CentralAuth/blob/master/includes/LocalRenameJob/LocalRenameJob.php#L154|job scheduling from `LocalRenameJob::scheduleNextWiki`]] fails).
Link to guide:See https://wikitech.wikimedia.org/wiki/Stuck_global_renames for the workaround.