Page MenuHomePhabricator

Increase parsercache keys TTL from 22 days back to 30 days
Closed, ResolvedPublic

Description

Once all the old parsercache are fully decommissioned (T209858 T210969) we should increase the TTL of their keys back to 30 days.
It was reduced to address disk space issues (T167784).
The new hosts have 4.4TB instead of 2.4TB so we should be good to back to the original TTL

I believe the files to revert are:

Details

Related Gerrit Patches:
operations/mediawiki-config : masterInitialiseSettings.php: Restore TTL back to 30 days
operations/puppet : productionparsercachepurging.pp: Increase TTL from 24 days to 30
operations/mediawiki-config : masterInitialiseSettings.php: Increase parsercache TTL
operations/puppet : productionparsercachepurging: Increase keys TTL

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 3 2018, 10:03 AM
Marostegui triaged this task as Medium priority.Dec 3 2018, 10:04 AM
Marostegui moved this task from Triage to Next on the DBA board.

@Krinkle can you confirm whether those are the two reverts we have to do?
I have been looking around and I haven't found anything else to revert
Thank you!

Imarlier assigned this task to aaron.Dec 10 2018, 9:06 PM
Imarlier moved this task from Inbox to Doing on the Performance-Team board.
Imarlier added a subscriber: Imarlier.

@aaron to provide feedback, will assign back once he has.

aaron added a comment.Dec 11 2018, 3:50 PM

@Krinkle can you confirm whether those are the two reverts we have to do?
I have been looking around and I haven't found anything else to revert
Thank you!

That looks like all there is.

Thanks @aaron - I will get a patch out after the code freeze

So we are close to get the last host ready T207258: rack/setup/install pc1007-pc1010, once that is done I will revert everything back to 30 days.

Marostegui added a subscriber: Joe.

I have been talking to @Joe and we have decided, just be on the safe side. We will increase the TTL just by 2 days, and make sure there are no strange behaviours on the hit ratio and things like that. If that works fine, we will go back to 30 as planned.

Change 485582 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] InitialiseSettings.php: Increase parsercache TTL

https://gerrit.wikimedia.org/r/485582

Change 485583 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] parsercachepurging: Increase keys TTL

https://gerrit.wikimedia.org/r/485583

@aaron @Joe @jcrespo
I have made the first small change, to go from 22 to 24 days:

https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/485582/
https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/485583/

The idea would be to push that, leave it for a few days, monitor the parsercache hit ratio and if nothing odd shows up, we can fully go to 30 days.

Marostegui moved this task from Next to In progress on the DBA board.Jan 21 2019, 7:32 AM
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.

Change 485582 merged by jenkins-bot:
[operations/mediawiki-config@master] InitialiseSettings.php: Increase parsercache TTL

https://gerrit.wikimedia.org/r/485582

Mentioned in SAL (#wikimedia-operations) [2019-01-23T06:16:48Z] <marostegui@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Increase parsercache TTL keys from 22 to 24 days T210992 (duration: 01m 06s)

Change 485583 merged by Marostegui:
[operations/puppet@production] parsercachepurging: Increase keys TTL

https://gerrit.wikimedia.org/r/485583

I have merged both changes after the review from @Krinkle (thanks!).
Let's see how it goes

In a couple of days there it will be a month since I switched the TTL from 22 days to 24. There has not been any issues with this, so on Monday I think I will go from 24 to 30 as planned unless someone has any objection.
Thanks!

jcrespo added a comment.EditedFeb 21 2019, 9:43 AM

@Marostegui Did the hit rate increase? What about disk space usage?

Change 492599 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] parsercachepurging.pp: Increase TTL from 24 days to 30

https://gerrit.wikimedia.org/r/492599

Change 492600 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] InitialiseSettings.php: Restore TTL back to 30 days

https://gerrit.wikimedia.org/r/492600

Change 492599 merged by Marostegui:
[operations/puppet@production] parsercachepurging.pp: Increase TTL from 24 days to 30

https://gerrit.wikimedia.org/r/492599

Change 492600 merged by jenkins-bot:
[operations/mediawiki-config@master] InitialiseSettings.php: Restore TTL back to 30 days

https://gerrit.wikimedia.org/r/492600

Mentioned in SAL (#wikimedia-operations) [2019-02-25T09:07:26Z] <marostegui@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Increase ParserCache TTL from 24 days to 30 - T210992 (duration: 00m 46s)

I have increased the TTL back to 30 days.
Going to monitor the graphs for a few days before closing this.

Marostegui closed this task as Resolved.Mar 1 2019, 8:33 AM