Page MenuHomePhabricator

PageImages maintenance script should be more flexible
Closed, ResolvedPublic

Description

The current maintenance script in PageImages will run against all titles in the database.
During a code review it was suggested we would benefit from only generating page images for articles who had been touched since a given date to avoid re-generating all page images in the event of incidents that impact a certain timeframe.

Any changes to page images algorithm are extremely risky without such a script as it provides us no way to respond to problems easily.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 3 2016, 12:09 AM
bmansurov triaged this task as Normal priority.Dec 7 2016, 5:35 PM
bmansurov set the point value for this task to 2.
bmansurov moved this task from Incoming to Triaged but Future on the Readers-Web-Backlog board.

@ovasileva given this was an actionable from a Wikimedia incident should we consider this for the sprint after next?

Yes, I think sprint after next makes sense

Jdlrobson raised the priority of this task from Normal to High.Jun 23 2017, 8:35 PM

Maybe this should be high. Let's talk about it in next grooming....

Jdlrobson lowered the priority of this task from High to Normal.Jun 28 2017, 5:40 PM
Jdlrobson added a project: patch-welcome.
Jdlrobson lowered the priority of this task from Normal to Low.Oct 31 2017, 1:01 AM

Reflecting reality...

Legoktm renamed this task from Maintenance script should be more flexible to PageImages maintenance script should be more flexible.Oct 31 2017, 1:05 AM
Jdlrobson raised the priority of this task from Low to Normal.Mar 19 2018, 3:45 PM

Given the talk in T91683 this should be a blocker for that.

Jdlrobson removed the point value for this task.
Jdlrobson added a project: User-Jdlrobson.

We attempted to estimate this, but estimates ranged from 2-8 based on the experience levels of the team. I suggest @pmiazga and myself have a go at expanding the description here to make this more clear and estimatable for next time.

Change 423742 had a related patch set uploaded (by Pmiazga; owner: Pmiazga):
[mediawiki/extensions/PageImages@master] Add later-than option to PageImages InitImageData maitenance script

https://gerrit.wikimedia.org/r/423742

Change 423742 merged by jenkins-bot:
[mediawiki/extensions/PageImages@master] Add later-than option to PageImages InitImageData maitenance script

https://gerrit.wikimedia.org/r/423742

@pmiazga and I sat down and just did this. If anyone is curious in the how take a look at the patch - it was super simple. This can skip design and QA review as it's purely technical.
@ovasileva I'll let you sign this off so that you're aware this has been done and this no longer blocks T91683.

ovasileva closed this task as Resolved.Apr 3 2018, 5:56 PM

sounds good to me.