Page MenuHomePhabricator

[l10n] "Refreshing ResourceLoader caches" is really really slow
Closed, ResolvedPublic

Description

03:20 logmsgbot: LocalisationUpdate ResourceLoader cache refresh completed at Fri Mar 14 03:20:12 UTC 2014 (duration 20m 11s)
02:47 logmsgbot: LocalisationUpdate completed (1.23wmf18) at 2014-03-14 02:47:13+00:00
02:12 logmsgbot: LocalisationUpdate completed (1.23wmf17) at 2014-03-14 02:12:53+00:00

It takes at least 30 minutes now... Wasn't ever that slow before...


Version: wmf-deployment
Severity: minor

Details

Reference
bz62705

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:54 AM
bzimport added a project: Deployments.
bzimport set Reference to bz62705.
bzimport added a subscriber: Unknown Object (MLST).
Reedy created this task.Mar 16 2014, 3:18 AM
Reedy added a comment.Mar 16 2014, 3:19 AM

[03:18:57] <logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Sun Mar 16 03:18:53 UTC 2014 (duration 53m 29s)

53.5!?

Reedy added a comment.Mar 16 2014, 3:41 AM

[03:33:53] <Reedy> 40 minutes 2nd feb 2014
[03:34:30] <Reedy> 24 minutes 2nd january

greg added a comment.Mar 16 2014, 3:45 AM

All the l10n updates since I added the duration output:

#wikimedia-operations.03-07.log:21:55 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Sat Mar 8 02:55:55 UTC 2014 (duration 55m 54s)
#wikimedia-operations.03-09.log:22:57 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Mon Mar 10 02:57:32 UTC 2014 (duration 57m 31s)
#wikimedia-operations.03-10.log:22:57 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Tue Mar 11 02:57:11 UTC 2014 (duration 57m 10s)
#wikimedia-operations.03-11.log:22:43 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Wed Mar 12 02:42:59 UTC 2014 (duration 42m 58s)
#wikimedia-operations.03-12.log:23:23 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Thu Mar 13 03:23:13 UTC 2014 (duration 23m 12s)
#wikimedia-operations.03-13.log:23:20 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Fri Mar 14 03:20:12 UTC 2014 (duration 20m 11s)
#wikimedia-operations.03-15.log:23:18 <+logmsgbot> !log LocalisationUpdate ResourceLoader cache refresh completed at Sun Mar 16 03:18:53 UTC 2014 (duration 53m 29s)

This data *should* also be in graphite (that was also in my patch), but I have yet to find it...

The duration will vary depending on how many languages had changed l10n that day, by checking the modification timestamp of each language's .cdb file. Did that get broken by the various l10n deployment changes that other people have been doing?

greg moved this task from To Triage to Backlog (Tech) on the Deployments board.Dec 1 2014, 11:50 PM