Maniphest T273266

[L] Commons mediainfo dumps retention
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	dcausse
	Jan 29 2021, 9:02 AM

Tags

Referenced Files

None

Subscribers

Description

The dataset https://dumps.wikimedia.org/other/wikibase/commonswiki/ seems to be keeping all the history of previous runs.

Similar to https://dumps.wikimedia.org/wikidatawiki/entities/ should we be keeping 1 month of history?

Details

	Subject	Repo	Branch	Lines +/-
	Cleanup old mediainfo dumps	operations/puppet	production	+2 -2

Customize query in gerrit

Event Timeline

dcausse created this task.Jan 29 2021, 9:02 AM

Yes, we must have missed this when going live. See https://github.com/wikimedia/puppet/blob/production/modules/dumps/manifests/web/cleanups/miscdumps.pp for how we do cleanup for other dumps.

ArielGlenn moved this task from Backlog to Up Next on the Dumps-Generation board.Feb 23 2021, 1:25 PM

Hey folks, whenever you have a few minutes to throw a patch my way, I'm happy to merge it through. (This is part of my super secret campaign to get everyone who is involved in dumps production, to be able to manage as many parts of their dumps as possible ;-) )

Pinging SD folks as they've worked on this dump IIRC.

CBogen edited projects, added Structured-Data-Backlog (Current Work); removed Structured-Data-Backlog.Mar 15 2021, 4:31 PM

CBogen moved this task from Incoming to Ready for Estimation on the Structured-Data-Backlog (Current Work) board.

CBogen renamed this task from Commons mediainfo dumps retention to [L] Commons mediainfo dumps retention.Mar 24 2021, 4:51 PM

CBogen moved this task from Ready for Estimation to Ready for Development on the Structured-Data-Backlog (Current Work) board.Mar 24 2021, 4:58 PM

ArielGlenn moved this task from Up Next to Other teams on the Dumps-Generation board.Apr 14 2021, 5:59 AM

matthiasmullie claimed this task.Jun 28 2021, 12:58 PM

matthiasmullie moved this task from Ready for Development to Doing on the Structured-Data-Backlog (Current Work) board.Jun 30 2021, 11:47 AM

Change 702413 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[operations/puppet@production] Cleanup old mediainfo dumps

https://gerrit.wikimedia.org/r/702413

gerritbot added a project: Patch-For-Review.Jun 30 2021, 3:24 PM

matthiasmullie moved this task from Doing to Code Review on the Structured-Data-Backlog (Current Work) board.Jun 30 2021, 3:24 PM

Change 702413 merged by ArielGlenn:

[operations/puppet@production] Cleanup old mediainfo dumps

https://gerrit.wikimedia.org/r/702413

Change now live. I'll leave this open for a day to make sure some old dumps disappear.

Maintenance_bot removed a project: Patch-For-Review.Jul 1 2021, 5:10 AM

@ArielGlenn Can we close this now?

In T273266#7224153, @Cparle wrote:

@ArielGlenn Can we close this now?

Yes! I let this fall off the radar, sorry about that.

Thank you!

ArielGlenn moved this task from Other teams to Done on the Dumps-Generation board.Feb 19 2022, 6:47 AM