Production Excellence #21: May 2020
Monthly update on our strive for operational excellence.

How’d we do in our strive for operational excellence last month? Read on to find out!

📊   Month in numbers
  • 5 documented incidents in May. [1]
  • 28 new production error tasks filed in May. [2] [3]
  • 68 recent production errors currently open (up from 61).
  • 193 currently open Wikimedia-prod-error tasks (up from 178). [4]

For more about recent incidents see Incident documentation on Wikitech, or Preventive measures in Phabricator.


📉   Outstanding reports

Take a look at the workboard and look for tasks that could use your help.
→  https://phabricator.wikimedia.org/tag/wikimedia-production-error/

Breakdown of recent months:

  • July 2019: One task closed, 7 of 18 tasks left. ⚠️
  • August: 2 of 14 tasks left (unchanged).
  • September: 7 of 12 tasks left (unchanged).
  • October: 4 of 12 tasks left (unchanged).
  • November: 4 of 5 tasks left (unchanged).
  • December: 4 of 9 tasks left (unchanged).
  • January 2020: 5 of 7 tasks left (unchanged).
  • February: Two tasks closed, 4 of 7 tasks left. ⚠️
  • March: 2 of 2 tasks left (unchanged).
  • April: 14 of 14 tasks left (unchanged).
  • May: 14 new tasks survived the month of May.

At the end of April the total of open production errors over recent months was 61. Of those, 7 got closed, but with 14 new tasks from May still open, the total has grown to 68.

The workboard had 178 open tasks in April, which saw a steep increase to now 192 open tasks (this includes June 2020 so far, and pre-2019 tasks).


🎉   Thanks!

Thank you to everyone else who helped by reporting, investigating, or resolving problems in Wikimedia production. Thanks!

Until next time,

– Timo Tijhof


Footnotes:
[1] Incidents. – https://wikitech.wikimedia.org/wiki/Incident_documentation#2020
[2] Tasks created. – https://phabricator.wikimedia.org/maniphest/query/7Z4Us2BS02Uo/#R
[3] Tasks closed. – https://phabricator.wikimedia.org/maniphest/query/FoIFMu5UO8pw/#R
[4] Open tasks. – https://phabricator.wikimedia.org/maniphest/query/Fw3RdXt1Sdxp/#R

Written by Krinkle on Jun 24 2020, 8:00 PM.
Principal Engineer (Wikimedia Performance)
Projects
None
Subscribers
None

Event Timeline