Page MenuHomePhabricator
Create Task
Maniphest T437461

Implement brownout logic for dumps v1 on dumps web interface
Closed, ResolvedPublic
Actions

Description

To allow a graceful transition from dumps v1 to v2, we'd like to (broadly announce and then) introduce partial unavailability of v1.

This would be at the end of an upcoming month, when fewer downloads happen, then progressing towards the start of the month, and repeating as long as the migration lasts.

This ticket tracks the development of the Nginx config changes to enable that, for the v1 dumps in scope to be replaced by v2.

Acceptance criteria

  • The dumps web interface can serve HTTP 404 responses on particular days of the month, with at least 3 possible configurations: the last 25 days of the month, or last 15, or last 5.
  • The 404 response has a link to a (to be published) Wikitech documentation page.
  • This only applies to the dumps in scope for migration to v2.

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
dumps: Enable brownout for dumps v1 history from 2026-10-26Btullisoperations/puppetproduction+1 -0
dumps: web: Add a brownout for v1 meta-current and meta-historyBtullisoperations/puppetproduction+59 -1
Customize query in gerrit

Event Timeline

GGoncalves-WMF created this task.Sep 9 2026, 2:13 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 9 2026, 2:13 PM
GGoncalves-WMF moved this task from Incoming (new tickets) to Tag with Radar on the Data-Engineering board.Sep 9 2026, 2:13 PM
GGoncalves-WMF added a project: Data-Engineering-Radar.
bking subscribed.Sep 9 2026, 2:46 PM
Gehel edited projects, added: Data-Platform-SRE (2026-08-28 - 2026-09-18); removed: Data-Platform-SRE.Tue, Sep 15, 9:20 AM
GGoncalves-WMF added a comment.Tue, Sep 15, 9:20 AM

We'd like this to be done ideally still in September, but early October is reasonable.

The intention is to have this logic applied at the end of October, that is, blocking v1 downloads for the last N days of that month.

Gehel added a project: Essential-Work.Wed, Sep 16, 9:43 AM
BTullis moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2026-08-28 - 2026-09-18) board.Thu, Sep 17, 11:00 AM
Gehel edited projects, added: Data-Platform-SRE (2026-09-18 - 2026-10-09); removed: Data-Platform-SRE (2026-08-28 - 2026-09-18).Fri, Sep 18, 1:20 PM
Gehel moved this task from Backlog - project to In Progress on the Data-Platform-SRE (2026-09-18 - 2026-10-09) board.
BTullis claimed this task.Tue, Sep 22, 8:45 AM
BTullis triaged this task as High priority.
gerritbot added a comment.Tue, Sep 29, 1:06 PM

Change #1346007 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] dumps: web: Add a brownout for v1 meta-current and meta-history

https://gerrit.wikimedia.org/r/1346007

gerritbot added a project: Patch-For-Review.Tue, Sep 29, 1:06 PM
gerritbot added a comment.Wed, Oct 7, 8:56 AM

Change #1346007 merged by Btullis:

[operations/puppet@production] dumps: web: Add a brownout for v1 meta-current and meta-history

https://gerrit.wikimedia.org/r/1346007

Maintenance_bot removed a project: Patch-For-Review.Wed, Oct 7, 9:31 AM
gerritbot added a comment.Thu, Oct 8, 8:32 AM

Change #1353019 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] dumps: Enable brownout for dumps v1 history from 2026-10-26

https://gerrit.wikimedia.org/r/1353019

gerritbot added a project: Patch-For-Review.Thu, Oct 8, 8:32 AM
gerritbot added a comment.Thu, Oct 8, 8:56 AM

Change #1353019 merged by Btullis:

[operations/puppet@production] dumps: Enable brownout for dumps v1 history from 2026-10-26

https://gerrit.wikimedia.org/r/1353019

Maintenance_bot removed a project: Patch-For-Review.Thu, Oct 8, 9:31 AM
BTullis closed this task as Resolved.Thu, Oct 8, 10:08 AM
BTullis moved this task from In Progress to Done on the Data-Platform-SRE (2026-09-18 - 2026-10-09) board.

I checked the CPU/load for the servers after merging the regex patch to the nginx.conf and any increase is negligible and could be background noise.

I'll resolve this ticket now, as we believe it to be done, but I will make a mental note to check on the 404 responses from October 26th.

Gehel moved this task from Done to Reported on the Data-Platform-SRE (2026-09-18 - 2026-10-09) board.Fri, Oct 9, 7:13 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits