Page MenuHomePhabricator

MW-1.29.0-wmf.11 deployment blockers
Closed, ResolvedPublic

Event Timeline

greg created this task.Jan 17 2017, 7:40 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 17 2017, 7:40 PM
greg triaged this task as Medium priority.Jan 17 2017, 7:40 PM
greg raised the priority of this task from Medium to Needs Triage.
greg updated the task description. (Show Details)
greg assigned this task to thcipriani.Jan 27 2017, 6:54 PM
greg triaged this task as Medium priority.

The logspam that we've accrued over the past few months has made it difficult to glance at error logs after a deployment and reason about a deployment's impact.

We've talked about pausing deploys or the branch cut or the train in some attempt to get logspam under control (I've even suggested it), but pausing deployments over logspam feels punitive and I feel like it will be frustrating for all involved. In lieu of pausing deployments I'm going to try to do a better job of pointing at some tasks that I think are either easy to address or would take a great deal of pressure off of deployers. The three that spring to mind are:

  1. T125735: Warning: timed out after 0.2 seconds when connecting to rdb1001.eqiad.wmnet [110]: Connection timed out - This one would be a huge win if we could find somewhat to control the volume of messages that we get from this message.
  2. T138987: Notice: Undefined index: width in /srv/mediawiki/php-1.28.0-wmf.7/includes/Linker.php - has been around 8 months and seems like a fix or workaround may be trivial. If we don't care about this message then silencing it would be cool.
  3. T157392: Warning: Cannot modify header information - headers already sent in /srv/mediawiki/php-1.29.0-wmf.10/includes/WebResponse.php on line 45 - this one is kinda vague but it just started happening with the last deployment and it would be nice to not introduce new errors.
greg closed this task as Resolved.Feb 21 2017, 5:46 PM