Page MenuHomePhabricator

Clean up old change dispatching code
Closed, ResolvedPublic

Description

  • Remove dispatchChanges maint script
  • Remove ChangeDispatcher
  • Remove ChangeDispatchCoordinator
  • Remove pruneChanges maint script (after 2021-10-07)
  • Remove ChangePruner
  • Remove PidLock
  • Remove dispatchLag and the places where it feeds into maxLag
  • Remove old statistics API integration
    • Think about making new statistics available via API? not until we need it T292604#7456480
  • Remove onApiMaxLagInfo hook? => 732361
  • T291846: New Content for Special:DispatchStats
  • Remove all puppet config for old maintenance jobs
  • T292622: decommission or change existing change dispatching dashboards
  • Remove specialized analytics scripts for old dispatching
  • Remove Config related to old change dispatching
    • dispatchLagToMaxLagFactor
    • consider keeping localClientDatabases as it is also used in the new implementation

Details

ProjectBranchLines +/-Subject
operations/mediawiki-configmaster+0 -2
operations/mediawiki-configmaster+0 -18
operations/mediawiki-configmaster+0 -18
operations/mediawiki-configmaster+0 -5
mediawiki/extensions/Wikibasemaster+9 -0
mediawiki/extensions/Wikibasemaster+0 -47
mediawiki/extensions/Wikibasewmf/1.38.0-wmf.5+3 -12
mediawiki/extensions/Wikibasewmf/1.38.0-wmf.5+2 -8
mediawiki/extensions/Wikibasewmf/1.38.0-wmf.5+3 -14
mediawiki/extensions/Wikibasemaster+0 -18
mediawiki/extensions/Wikibasemaster+0 -821
mediawiki/extensions/Wikibasemaster+3 -12
mediawiki/extensions/Wikibasemaster+2 -8
analytics/wmde/scriptsproduction+0 -43
mediawiki/extensions/Wikibasemaster+3 -14
mediawiki/extensions/Wikibasemaster+0 -9
analytics/wmde/scriptsmaster+0 -43
mediawiki/extensions/Wikibasemaster+0 -2 K
operations/puppetproduction+0 -7
mediawiki/extensions/Wikibasemaster+39 -439
operations/puppetproduction+1 -0
operations/puppetproduction+1 -0
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 726746 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/puppet@production] mediawiki: Absent wikibase_repo_prune2 systemd timer

https://gerrit.wikimedia.org/r/726746

Ladsgroup updated the task description. (Show Details)

Change 726748 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/extensions/Wikibase@master] Remove all the old dispatching code

https://gerrit.wikimedia.org/r/726748

Change 731014 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/mediawiki-config@master] Unconditionally enable Wikibase dispatching via jobs

https://gerrit.wikimedia.org/r/731014

Change 731015 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/mediawiki-config@master] Remove wmg variables for dispatch via jobs

https://gerrit.wikimedia.org/r/731015

Change 731027 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/puppet@production] mediawiki: Absent wikidatawiki change pruning

https://gerrit.wikimedia.org/r/731027

Change 731028 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/puppet@production] mediawiki: Drop absented wikidatawiki change pruning

https://gerrit.wikimedia.org/r/731028

Change 731045 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Empty dispatchChanges.php and pruneChanges.php

https://gerrit.wikimedia.org/r/731045

Change 731027 abandoned by Lucas Werkmeister (WMDE):

[operations/puppet@production] mediawiki: Absent wikidatawiki change pruning

Reason:

duplicate of I785714c00a5

https://gerrit.wikimedia.org/r/731027

Change 726746 merged by RLazarus:

[operations/puppet@production] mediawiki: Absent wikibase_repo_prune2 systemd timer

https://gerrit.wikimedia.org/r/726746

Change 731045 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Empty dispatchChanges.php and pruneChanges.php

https://gerrit.wikimedia.org/r/731045

Change 731028 merged by RLazarus:

[operations/puppet@production] mediawiki: Drop absented wikibase_repo_prune2 systemd timer

https://gerrit.wikimedia.org/r/731028

Change 726748 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove all the old dispatching code

https://gerrit.wikimedia.org/r/726748

Change 732277 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[analytics/wmde/scripts@master] Remove dispatch.php

https://gerrit.wikimedia.org/r/732277

Change 732282 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove traces of dispatchingLockManager repo option

https://gerrit.wikimedia.org/r/732282

Change 732283 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove dispatchViaJobsAllowedClients repo setting

https://gerrit.wikimedia.org/r/732283

Change 732284 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting

https://gerrit.wikimedia.org/r/732284

Change 732285 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove dispatchViaJobs repo setting

https://gerrit.wikimedia.org/r/732285

Change 732277 merged by jenkins-bot:

[analytics/wmde/scripts@master] Remove dispatch.php

https://gerrit.wikimedia.org/r/732277

Change 732282 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove traces of dispatchingLockManager repo option

https://gerrit.wikimedia.org/r/732282

Change 732283 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove dispatchViaJobsAllowedClients repo setting

https://gerrit.wikimedia.org/r/732283

Change 732329 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[analytics/wmde/scripts@production] Remove dispatch.php

https://gerrit.wikimedia.org/r/732329

Change 732329 merged by jenkins-bot:

[analytics/wmde/scripts@production] Remove dispatch.php

https://gerrit.wikimedia.org/r/732329

Change 732361 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove old DispatchStats code

https://gerrit.wikimedia.org/r/732361

Change 732284 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting

https://gerrit.wikimedia.org/r/732284

Change 732285 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove dispatchViaJobs repo setting

https://gerrit.wikimedia.org/r/732285

Change 732664 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove dispatchLagToMaxLagFactor repo setting

https://gerrit.wikimedia.org/r/732664

Change 732667 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@wmf/1.38.0-wmf.5] Remove dispatchViaJobsAllowedClients repo setting

https://gerrit.wikimedia.org/r/732667

Change 732668 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@wmf/1.38.0-wmf.5] Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting

https://gerrit.wikimedia.org/r/732668

Change 732669 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@wmf/1.38.0-wmf.5] Remove dispatchViaJobs repo setting

https://gerrit.wikimedia.org/r/732669

Change 732361 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove old DispatchStats code

https://gerrit.wikimedia.org/r/732361

Change 732664 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove dispatchLagToMaxLagFactor repo setting

https://gerrit.wikimedia.org/r/732664

Change 732684 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Remove no-longer-used repo dispatch settings

https://gerrit.wikimedia.org/r/732684

Change 732667 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@wmf/1.38.0-wmf.5] Remove dispatchViaJobsAllowedClients repo setting

https://gerrit.wikimedia.org/r/732667

Mentioned in SAL (#wikimedia-operations) [2021-10-21T15:11:46Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/includes/: Backport: [[gerrit:732667|Remove dispatchViaJobsAllowedClients repo setting (T292604)]] (1/3) (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2021-10-21T15:13:06Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/config/: Backport: [[gerrit:732667|Remove dispatchViaJobsAllowedClients repo setting (T292604)]] (1/3) (duration: 00m 54s)

Mentioned in SAL (#wikimedia-operations) [2021-10-21T15:14:38Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/tests/: Backport: [[gerrit:732667|Remove dispatchViaJobsAllowedClients repo setting (T292604)]] (3/3) (duration: 00m 56s)

Change 732724 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Mention removed config settings in 1.38 release notes

https://gerrit.wikimedia.org/r/732724

Change 732668 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@wmf/1.38.0-wmf.5] Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting

https://gerrit.wikimedia.org/r/732668

Mentioned in SAL (#wikimedia-operations) [2021-10-21T15:58:10Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/includes/: Backport: [[gerrit:732668|Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting (T292604)]] (1/3) (duration: 00m 57s)

Mentioned in SAL (#wikimedia-operations) [2021-10-21T15:59:55Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/config/: Backport: [[gerrit:732668|Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting (T292604)]] (2/3) (duration: 00m 55s)

Mentioned in SAL (#wikimedia-operations) [2021-10-21T16:01:14Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/tests/: Backport: [[gerrit:732668|Remove dispatchViaJobsPruneChangesTableInJobEnabled repo setting (T292604)]] (3/3) (duration: 00m 56s)

Change 732669 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@wmf/1.38.0-wmf.5] Remove dispatchViaJobs repo setting

https://gerrit.wikimedia.org/r/732669

Mentioned in SAL (#wikimedia-operations) [2021-10-21T16:04:51Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/includes/: Backport: [[gerrit:732669|Remove dispatchViaJobs repo setting (T292604)]] (1/3) (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2021-10-21T16:06:05Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/config/: Backport: [[gerrit:732669|Remove dispatchViaJobs repo setting (T292604)]] (2/3) (duration: 00m 54s)

Mentioned in SAL (#wikimedia-operations) [2021-10-21T16:07:14Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.5/extensions/Wikibase/repo/tests/: Backport: [[gerrit:732669|Remove dispatchViaJobs repo setting (T292604)]] (3/3) (duration: 00m 56s)

Change 732684 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Remove no-longer-used repo dispatch settings

https://gerrit.wikimedia.org/r/732684

Change 732724 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Mention removed config settings in 1.38 release notes

https://gerrit.wikimedia.org/r/732724

Change 732949 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/mediawiki-config@master] Remove dispatchChanges.php-related Wikibase settings

https://gerrit.wikimedia.org/r/732949

Change 732950 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/mediawiki-config@master] Remove wmg variables for dispatchChanges.php Wikibase settings

https://gerrit.wikimedia.org/r/732950

Change 732951 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/mediawiki-config@master] Remove wikibaseDispatchRedisLockManager config

https://gerrit.wikimedia.org/r/732951

Change 732969 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/mediawiki-config@master] Remove dispatchLagToMaxLagFactor Wikibase setting

https://gerrit.wikimedia.org/r/732969

Change 732949 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove dispatchChanges.php-related Wikibase settings

https://gerrit.wikimedia.org/r/732949

Mentioned in SAL (#wikimedia-operations) [2021-10-25T11:09:13Z] <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:732949|Remove dispatchChanges.php-related Wikibase settings (T292604)]] (duration: 00m 55s)

Change 732950 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove wmg variables for dispatchChanges.php Wikibase settings

https://gerrit.wikimedia.org/r/732950

Mentioned in SAL (#wikimedia-operations) [2021-10-25T11:14:29Z] <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:732950|Remove wmg variables for dispatchChanges.php Wikibase settings (T292604)]] (duration: 00m 55s)

Change 732951 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove wikibaseDispatchRedisLockManager config

https://gerrit.wikimedia.org/r/732951

Mentioned in SAL (#wikimedia-operations) [2021-10-25T11:18:00Z] <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:732951|Remove wikibaseDispatchRedisLockManager config (T292604)]] (duration: 00m 54s)

Change 732969 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove dispatchLagToMaxLagFactor Wikibase setting

https://gerrit.wikimedia.org/r/732969

Mentioned in SAL (#wikimedia-operations) [2021-10-25T11:22:48Z] <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:732969|Remove dispatchLagToMaxLagFactor Wikibase setting (T292604)]] (duration: 00m 54s)

Basically, all that is left here is "Think about making new statistics available via API?", which is probably a product decision in the end.
But given that the statistics stopped being available in the API for a while now and AFAIK nobody even mentioned it, I think that doesn't have a high priority.

@Lydia_Pintscher thoughts?

Yeah I think we're ok. If it turns out we'll need it again we can handle it then I think.