Page MenuHomePhabricator
Create Task
Maniphest T390556

Decommission dumps job `download_enterprise_htmldumps`
Closed, ResolvedPublic
Actions

Description

As per T389542, we are deprecating the HTML dumps that we have historically done via the Dumps 1.0 infrastructure.
Enterprise will continue to offer free HTML dumps via the respective interface.

We should then remove this job from the set of miscellaneous dumps and clean up code.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Removing WM Enterprise downloader Puppet configurationoperations/puppetproduction+0 -1 K
Set the remaining Enterprise WM Downloader job to absentoperations/puppetproduction+3 -3
Absent systemd timers to stop attempting to generate enterprise HTML dumpsoperations/puppetproduction+2 -2
Customize query in gerrit

Related Objects

Event Timeline

xcollazo created this task.Mar 31 2025, 1:47 PM
xcollazo added a comment.Apr 8 2025, 2:18 PM

Looks like we need to (cleanly) revert https://github.com/wikimedia/operations-puppet/commit/ced0ecef68b3d72ecedeb183701ed3d8d2e560f7 to get this done.

For now, perhaps we can just absent the systemd timers?

gerritbot added a comment.Apr 8 2025, 2:26 PM

Change #1135042 had a related patch set uploaded (by Xcollazo; author: Xcollazo):

[operations/puppet@production] Absent systemd timers to stop attempting to generate enterprise HTML dumps

https://gerrit.wikimedia.org/r/1135042

gerritbot added a project: Patch-For-Review.Apr 8 2025, 2:26 PM
Ahoelzl moved this task from Incoming (new tickets) to Q4 2025 April 1st - June 30th on the Data-Engineering board.Apr 8 2025, 5:43 PM
Ahoelzl edited projects, added Data-Engineering (Q4 2025 April 1st - June 30th); removed Data-Engineering.
Ahoelzl assigned this task to amastilovic.Apr 8 2025, 6:28 PM
Ahoelzl moved this task from Next Up to In progress on the Data-Engineering (Q4 2025 April 1st - June 30th) board.Apr 8 2025, 11:55 PM
xcollazo mentioned this in T390839: The 20250401 dumps haven't started on time because the mediawikiwiki dump from 20250320 is looping.Apr 21 2025, 2:26 PM
gerritbot added a comment.Apr 25 2025, 5:51 PM

Change #1135042 merged by Brouberol:

[operations/puppet@production] Absent systemd timers to stop attempting to generate enterprise HTML dumps

https://gerrit.wikimedia.org/r/1135042

xcollazo added a subscriber: brouberol.Apr 25 2025, 5:59 PM

@brouberol helped us figure the patch was returning an empty PPC run (we were targeting the wrong set of hosts).

Now that we have deployed https://gerrit.wikimedia.org/r/1135042, the behavior of publishing empty dumps should stop.

Next step:

In T390556#10721807, @xcollazo wrote:

Looks like we need to (cleanly) revert https://github.com/wikimedia/operations-puppet/commit/ced0ecef68b3d72ecedeb183701ed3d8d2e560f7 to get this done.
...

Maintenance_bot removed a project: Patch-For-Review.Apr 25 2025, 6:30 PM
creynolds mentioned this in T298450: Invalid description of Wikimedia Enterprise HTML Dumps.May 6 2025, 12:16 AM
Mitar subscribed.May 6 2025, 6:20 AM
Ahoelzl subscribed.May 6 2025, 10:10 PM

@amastilovic can you target this for this week? Thanks.

Mitar added a comment.May 6 2025, 10:12 PM

Please also delete empty dirs which are currently presented there, so that it is easy to find the latest existing archived dir.

Ahoelzl updated the task description. (Show Details)May 6 2025, 10:12 PM
gerritbot added a comment.May 6 2025, 11:27 PM

Change #1142712 had a related patch set uploaded (by Aleksandar Mastilovic; author: Aleksandar Mastilovic):

[operations/puppet@production] Removing WM Enterprise downloader Puppet configuration

https://gerrit.wikimedia.org/r/1142712

gerritbot added a project: Patch-For-Review.May 6 2025, 11:27 PM
gerritbot added a comment.May 7 2025, 4:41 PM

Change #1143134 had a related patch set uploaded (by Aleksandar Mastilovic; author: Aleksandar Mastilovic):

[operations/puppet@production] Set the remaining Enterprise WM Downloader job to absent

https://gerrit.wikimedia.org/r/1143134

amastilovic added a comment.May 7 2025, 6:56 PM

Please also delete empty dirs which are currently presented there, so that it is easy to find the latest existing archived dir.

@brouberol could you please help with this? I don't seem to have the permissions to delete these folders.

Ahoelzl renamed this task from Decomission dumps job `download_enterprise_htmldumps` to Decommission dumps job `download_enterprise_htmldumps`.May 9 2025, 4:37 PM
gerritbot added a comment.May 13 2025, 7:32 AM

Change #1143134 merged by Brouberol:

[operations/puppet@production] Set the remaining Enterprise WM Downloader job to absent

https://gerrit.wikimedia.org/r/1143134

brouberol added a comment.May 13 2025, 7:51 AM

Which directories exactly?

Mitar added a comment.May 13 2025, 10:45 AM

These:

brouberol added a comment.May 13 2025, 3:58 PM

All done!

amastilovic added a comment.May 13 2025, 3:59 PM

Thank you!

amastilovic closed this task as Resolved.May 13 2025, 4:00 PM
xcollazo added a comment.May 13 2025, 5:37 PM

@amastilovic do we still need to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1142712 ?

Mitar added a comment.May 13 2025, 5:57 PM

Thank you.

gerritbot added a comment.Jun 13 2025, 9:10 AM

Change #1142712 merged by Brouberol:

[operations/puppet@production] Removing WM Enterprise downloader Puppet configuration

https://gerrit.wikimedia.org/r/1142712

Maintenance_bot removed a project: Patch-For-Review.Jun 13 2025, 9:30 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits