Page MenuHomePhabricator
Create Task
Maniphest T406318

rest.php via rest-gateway production rollout
Closed, ResolvedPublic
Actions

Description

Before this work can be completed, T401396 must be resolved.

Assuming we get the all-clear from testing on test2wiki, our rollout plan is

  • group0 at 10%
  • group0 at 50% (optional)
  • group0 at 100%
  • group1 at 10%
  • group1 at 50%
  • group1 at 100%
  • Pause to examine capacity and scale up if needed
  • all non-enwiki at 10%
  • all non-enwiki at 50%
  • all non-enwiki at 100%
  • capacity check
  • enwiki at 10%
  • enwiki at 50%
  • enwiki at 100%

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
trafficserver: Fix logic if groupmatch is emptyClément Goubertoperations/puppetproduction+2 -1
trafficserver: remove gateway-check group-specific routes for rest.phpHnowlanoperations/puppetproduction+15 -12
trafficserver: Default rest.php routing to rest-gwClément Goubertoperations/puppetproduction+15 -12
trafficserver: rest-gateway routes for rest.php: enwiki 100%Clément Goubertoperations/puppetproduction+0 -4
trafficserver: rest-gateway routes for rest.php: enwiki 50%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: rest-gateway routes for rest.php: enwiki 10%Clément Goubertoperations/puppetproduction+4 -6
trafficserver: rest-gateway routes for rest.php: group2 100%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: rest-gateway routes for rest.php: group2 50%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: rest-gateway routes for rest.php: group2 10%Clément Goubertoperations/puppetproduction+8 -2
trafficserver: rest-gateway routes for rest.php: group1 100%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: rest-gateway routes for rest.php: group1 50%Clément Goubertoperations/puppetproduction+2 -2
rest-gateway: add wikibase rest.php matchHnowlanoperations/deployment-chartsmaster+3 -0
trafficserver: rest-gateway routes for rest.php: group1 10%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: rest-gateway routes for rest.php: group0 100%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: rest-gateway routes for rest.php: group0 50%Clément Goubertoperations/puppetproduction+2 -2
trafficserver: clean up testing exceptionsClément Goubertoperations/puppetproduction+0 -8
trafficserver: rest-gateway routes for rest.php: group0 50%Hnowlanoperations/puppetproduction+8 -0
trafficserver: rest-gateway routes for rest.php: group0 100%Hnowlanoperations/puppetproduction+6 -6
trafficserver: rest-gateway routes for rest.php: group1 1%Hnowlanoperations/puppetproduction+28 -0
trafficserver: rest-gateway routes for rest.php: group1 50%Hnowlanoperations/puppetproduction+14 -14
trafficserver: rest-gateway routes for rest.php: group1 100%Hnowlanoperations/puppetproduction+14 -14
gateway-check: Introduce regex matchingClément Goubertoperations/puppetproduction+106 -21
gateway-check: Group-based routing approachClément Goubertoperations/puppetproduction+1 K -29
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedhnowlanT400130 Central REST gateway for APIs
 ResolvedhnowlanT406318 rest.php via rest-gateway production rollout

Event Timeline

hnowlan created this task.Oct 3 2025, 11:23 AM
gerritbot added a comment.Oct 6 2025, 9:52 AM

Change #1193805 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 50%

https://gerrit.wikimedia.org/r/1193805

gerritbot added a project: Patch-For-Review.Oct 6 2025, 9:52 AM
gerritbot added a comment.Oct 6 2025, 10:02 AM

Change #1193808 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 100%

https://gerrit.wikimedia.org/r/1193808

gerritbot added a comment.Oct 6 2025, 10:02 AM

Change #1193809 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 1%

https://gerrit.wikimedia.org/r/1193809

gerritbot added a comment.Oct 6 2025, 10:02 AM

Change #1193810 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 50%

https://gerrit.wikimedia.org/r/1193810

gerritbot added a comment.Oct 6 2025, 10:02 AM

Change #1193811 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 100%

https://gerrit.wikimedia.org/r/1193811

gerritbot added a comment.Oct 6 2025, 3:04 PM

Change #1193882 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] gateway-check: Introduce regex matching

https://gerrit.wikimedia.org/r/1193882

gerritbot added a comment.Oct 6 2025, 4:42 PM

Change #1193903 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] gateway-check: Group-based routing approach

https://gerrit.wikimedia.org/r/1193903

HCoplin-WMF moved this task from Now to Supporting on the MediaWiki-API-Platform-Team (MWI-Roadmap) board.Oct 7 2025, 4:02 PM
Stashbot added a comment.Oct 8 2025, 10:43 AM

Mentioned in SAL (#wikimedia-operations) [2025-10-08T10:43:08Z] <claime> Disabling puppet on cp nodes - 1193903: gateway-check: Group-based routing approach | https://gerrit.wikimedia.org/r/c/operations/puppet/+/1193903 - T406318

gerritbot added a comment.Oct 8 2025, 10:46 AM

Change #1193903 merged by Clément Goubert:

[operations/puppet@production] gateway-check: Group-based routing approach

https://gerrit.wikimedia.org/r/1193903

Stashbot added a comment.Oct 8 2025, 11:26 AM

Mentioned in SAL (#wikimedia-operations) [2025-10-08T11:26:21Z] <claime> Enabling puppet on cp nodes - 1193903: gateway-check: Group-based routing approach | https://gerrit.wikimedia.org/r/c/operations/puppet/+/1193903 - T406318

gerritbot added a comment.Oct 8 2025, 11:27 AM

Change #1193882 abandoned by Clément Goubert:

[operations/puppet@production] gateway-check: Introduce regex matching

Reason:

I9bd23e96095f921d762349d7a6f37ca1871e75d2 was preferred

https://gerrit.wikimedia.org/r/1193882

gerritbot added a comment.Oct 8 2025, 11:32 AM

Change #1193811 abandoned by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 100%

Reason:

new group approach makes these irrelevant

https://gerrit.wikimedia.org/r/1193811

gerritbot added a comment.Oct 8 2025, 11:33 AM

Change #1193810 abandoned by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 50%

Reason:

new group approach makes these irrelevant

https://gerrit.wikimedia.org/r/1193810

gerritbot added a comment.Oct 8 2025, 11:33 AM

Change #1193809 abandoned by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 1%

Reason:

new group approach makes these irrelevant

https://gerrit.wikimedia.org/r/1193809

gerritbot added a comment.Oct 8 2025, 11:33 AM

Change #1193808 abandoned by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 100%

Reason:

new group approach makes these irrelevant

https://gerrit.wikimedia.org/r/1193808

gerritbot added a comment.Oct 8 2025, 11:33 AM

Change #1193805 abandoned by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 50%

https://gerrit.wikimedia.org/r/1193805

Maintenance_bot removed a project: Patch-For-Review.Oct 8 2025, 11:36 AM
gerritbot added a comment.Oct 8 2025, 11:36 AM

Change #1194592 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 50%

https://gerrit.wikimedia.org/r/1194592

gerritbot added a project: Patch-For-Review.Oct 8 2025, 11:36 AM

Change #1194593 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 100%

https://gerrit.wikimedia.org/r/1194593

gerritbot added a comment.Oct 8 2025, 11:36 AM

Change #1194594 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 10%

https://gerrit.wikimedia.org/r/1194594

gerritbot added a comment.Oct 8 2025, 11:37 AM

Change #1194595 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 50%

https://gerrit.wikimedia.org/r/1194595

gerritbot added a comment.Oct 8 2025, 11:37 AM

Change #1194596 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 100%

https://gerrit.wikimedia.org/r/1194596

gerritbot added a comment.Oct 8 2025, 11:37 AM

Change #1194597 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group2 10%

https://gerrit.wikimedia.org/r/1194597

gerritbot added a comment.Oct 8 2025, 11:37 AM

Change #1194598 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group2 50%

https://gerrit.wikimedia.org/r/1194598

gerritbot added a comment.Oct 8 2025, 11:37 AM

Change #1194599 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group2 100%

https://gerrit.wikimedia.org/r/1194599

Clement_Goubert changed the task status from Open to In Progress.Oct 8 2025, 11:50 AM
Clement_Goubert triaged this task as High priority.
Clement_Goubert updated the task description. (Show Details)
gerritbot added a comment.Oct 8 2025, 12:07 PM

Change #1194608 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: clean up testing exceptions

https://gerrit.wikimedia.org/r/1194608

gerritbot added a comment.Oct 8 2025, 12:07 PM

Change #1194609 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: enwiki 10%

https://gerrit.wikimedia.org/r/1194609

gerritbot added a comment.Oct 8 2025, 12:08 PM

Change #1194610 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: enwiki 50%

https://gerrit.wikimedia.org/r/1194610

gerritbot added a comment.Oct 8 2025, 12:08 PM

Change #1194611 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: enwiki 100%

https://gerrit.wikimedia.org/r/1194611

Clement_Goubert added a comment.Oct 8 2025, 12:08 PM

group0 is now routed at 10% to the rest-gateway

gerritbot added a comment.Oct 8 2025, 12:21 PM

Change #1194608 merged by Clément Goubert:

[operations/puppet@production] trafficserver: clean up testing exceptions

https://gerrit.wikimedia.org/r/1194608

gerritbot added a comment.Oct 8 2025, 12:21 PM

Change #1194592 merged by Clément Goubert:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 50%

https://gerrit.wikimedia.org/r/1194592

Clement_Goubert updated the task description. (Show Details)Oct 8 2025, 12:33 PM
Clement_Goubert added a comment.Oct 8 2025, 12:36 PM

Note that we rolled test2wiki and testwikidata back into the normal rollout.

  • test2wiki is now back to direct routing to mw-api-ext and will be rolled out with group1
  • testwikidata is being rolled out with group0

group0 is now routed at 50% to the rest-gateway

gerritbot added a comment.Oct 8 2025, 12:41 PM

Change #1194593 merged by Clément Goubert:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group0 100%

https://gerrit.wikimedia.org/r/1194593

Clement_Goubert updated the task description. (Show Details)Oct 8 2025, 12:47 PM
gerritbot added a comment.Oct 8 2025, 2:46 PM

Change #1194594 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 10%

https://gerrit.wikimedia.org/r/1194594

gerritbot added a comment.Oct 8 2025, 3:19 PM

Change #1194655 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/deployment-charts@master] rest-gateway: add wikibase rest.php match

https://gerrit.wikimedia.org/r/1194655

Clement_Goubert updated the task description. (Show Details)Oct 8 2025, 3:20 PM
gerritbot added a comment.Oct 8 2025, 3:31 PM

Change #1194655 merged by jenkins-bot:

[operations/deployment-charts@master] rest-gateway: add wikibase rest.php match

https://gerrit.wikimedia.org/r/1194655

gerritbot added a comment.Oct 8 2025, 3:47 PM

Change #1194595 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 50%

https://gerrit.wikimedia.org/r/1194595

‱ Scott_French mentioned this in T405955: MediaWiki on PHP 8.3 production workload migration.Oct 8 2025, 5:58 PM
gerritbot added a comment.Oct 9 2025, 10:55 AM

Change #1194596 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group1 100%

https://gerrit.wikimedia.org/r/1194596

hnowlan updated the task description. (Show Details)Oct 9 2025, 10:56 AM
gerritbot added a comment.Oct 9 2025, 2:20 PM

Change #1194597 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group2 10%

https://gerrit.wikimedia.org/r/1194597

Clement_Goubert updated the task description. (Show Details)Oct 9 2025, 2:57 PM
gerritbot added a comment.Oct 9 2025, 3:00 PM

Change #1194598 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group2 50%

https://gerrit.wikimedia.org/r/1194598

gerritbot added a comment.Oct 9 2025, 3:15 PM

Change #1194599 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: group2 100%

https://gerrit.wikimedia.org/r/1194599

gerritbot added a comment.Oct 9 2025, 3:55 PM

Change #1194609 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: enwiki 10%

https://gerrit.wikimedia.org/r/1194609

hnowlan updated the task description. (Show Details)Oct 9 2025, 4:04 PM

We're rolling out 10% of enwiki at the moment, and we will leave things there until next week.

CDanis subscribed.Oct 9 2025, 4:06 PM
gerritbot added a comment.Oct 13 2025, 10:55 AM

Change #1194610 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: enwiki 50%

https://gerrit.wikimedia.org/r/1194610

gerritbot added a comment.Oct 13 2025, 12:59 PM

Change #1195679 had a related patch set uploaded (by Hnowlan; author: Hnowlan):

[operations/puppet@production] trafficserver: remove gateway-check group-specific routes for rest.php

https://gerrit.wikimedia.org/r/1195679

gerritbot added a comment.Oct 13 2025, 1:00 PM

Change #1194611 merged by Hnowlan:

[operations/puppet@production] trafficserver: rest-gateway routes for rest.php: enwiki 100%

https://gerrit.wikimedia.org/r/1194611

hnowlan closed this task as Resolved.Oct 13 2025, 2:33 PM
hnowlan claimed this task.
hnowlan updated the task description. (Show Details)

Migration complete. Impact on rest-gateway was minimal, no scaling up required.

gerritbot added a comment.Oct 14 2025, 12:18 PM

Change #1196045 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: Default rest.php routing to rest-gw

https://gerrit.wikimedia.org/r/1196045

gerritbot added a comment.Oct 14 2025, 12:37 PM

Change #1196045 abandoned by Clément Goubert:

[operations/puppet@production] trafficserver: Default rest.php routing to rest-gw

Reason:

Dupe Idb9deadb317460f601e63b6adb23f761d7ce76db

https://gerrit.wikimedia.org/r/1196045

Stashbot added a comment.Oct 15 2025, 11:06 AM

Mentioned in SAL (#wikimedia-operations) [2025-10-15T11:06:58Z] <claime> disabling puppet on cp nodes for 1195679: trafficserver: remove gateway-check group-specific routes for rest.php - T406318

gerritbot added a comment.Oct 15 2025, 11:11 AM

Change #1195679 merged by Clément Goubert:

[operations/puppet@production] trafficserver: remove gateway-check group-specific routes for rest.php

https://gerrit.wikimedia.org/r/1195679

Stashbot added a comment.Oct 15 2025, 11:12 AM

Mentioned in SAL (#wikimedia-operations) [2025-10-15T11:12:49Z] <claime> Enabling puppet on cp6015 for 1195679: trafficserver: remove gateway-check group-specific routes for rest.php - T406318

Stashbot added a comment.Oct 15 2025, 11:16 AM

Mentioned in SAL (#wikimedia-operations) [2025-10-15T11:16:52Z] <claime> Enabling puppet on all cp nodes for 1195679: trafficserver: remove gateway-check group-specific routes for rest.php - T406318

Maintenance_bot removed a project: Patch-For-Review.Oct 15 2025, 11:30 AM
gerritbot added a comment.Oct 15 2025, 12:13 PM

Change #1196416 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: Fix logic if groupmatch is empty

https://gerrit.wikimedia.org/r/1196416

gerritbot added a project: Patch-For-Review.Oct 15 2025, 12:13 PM
gerritbot added a comment.Oct 15 2025, 12:29 PM

Change #1196416 merged by Clément Goubert:

[operations/puppet@production] trafficserver: Fix logic if groupmatch is empty

https://gerrit.wikimedia.org/r/1196416

Stashbot added a comment.Oct 15 2025, 12:29 PM

Mentioned in SAL (#wikimedia-operations) [2025-10-15T12:29:24Z] <claime> disabling puppet on cp nodes for T406318

Maintenance_bot removed a project: Patch-For-Review.Oct 15 2025, 12:30 PM
Stashbot added a comment.Oct 15 2025, 12:44 PM

Mentioned in SAL (#wikimedia-operations) [2025-10-15T12:44:49Z] <claime> enabling puppet on cp nodes for T406318

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits