Page MenuHomePhabricator

(Semi)automatically close Phabricator tickets with status "stalled" after a while
Open, LowestPublic

Description

Note: This task is not about (semi)automatically asking task authors to re-test an issue after a while and in the same action (semi)automatically setting the "stalled" status. That would be a different topic and ticket, maybe T248034.

Stalled is defined as "If a report is waiting for further input (e.g. from its reporter or a third party) and can currently not be acted on, the Stalled status is temporarily given."

This can mean several things:

  1. "stalled" task status due to being blocked on an external upstream task outside of Wikimedia. Hence should be tagged Upstream.
  2. "stalled" task status due to a subtask which needs to be resolved first inside of Wikimedia. Hence should have a subtask.
  3. "stalled" task status due to feedback required from the task author. (Which in itself is also a blurry concept as the task author could also be set as assignee instead or in addition?)
  4. "stalled" task status due to misuse in order to just hide a task on a workboard or in a list of tasks.
  5. (maybe something else that is not coming to my mind?)

I would not want to add more statūs / having more field values to differentiate between #1, #2 and #3. Keep it simple.

I'd rather want to clean up #4. When identifying a list of old "stalled" tasks without updates to potentially (semi)automatically decline, I'd also want to exclude #1 and #2 (DB query?).

In any case, this will require a person who sets the "stalled" status to also always explicitly ask for resetting the task status and explain how to reset the task status. Hence this would require being more aggressive in resetting the task status whenever "stalled" status is set without a clear explanation who (person or upstream project) to provide which information or perform which action.

Event Timeline

Aklapper triaged this task as Lowest priority.

Basically: SELECT CONCAT("https://phabricator.wikimedia.org/T", mt.id) AS taskID, mt.title, FROM_UNIXTIME(mt.dateModified) AS creation FROM phabricator_maniphest.maniphest_task mt WHERE mt.status = "stalled" AND mt.phid NOT IN (SELECT e.src FROM phabricator_maniphest.edge e WHERE e.type = 41 AND e.dst="PHID-PROJ-bchzb6qpl3jhgs2oq6um") AND mt.phid NOT IN (SELECT e.src FROM phabricator_maniphest.edge e WHERE e.type = 3) ORDER BY mt.dateModified;

I'm slightly fearful of this because I believe for so many projects, the automatic closure would likely not be seen by anyone and thus dissapear from boards.

I would suggest that if we go forward with this, to split the automated process in two steps: 1) notify of upcoming automatic closure, 2) actual closure.

For example, if we were to pick automatic closure at 90 days, the bot could leave a comment at 30 days briefly describing what the bot is seeing (task is stalled with no status change or comment for 30 days, link to docs about our conventions around this, announce upcoming closure date, and briefly describe how to prevent it).

This way removes the pressure from those setting things to "Stalled" to have to explain (and remember to explain) stuff upfront. It also takes care of most accidental or unintended uses of the status.

Tentatively adding FY2020-21 Q1 tag for planning

For the records, right now we have 317 stalled tasks which have been stalled for longer than 12 months.

I agree with everything that Krinkle wrote above.

  1. Query [1] for tasks that currently have stalled status and do not have a subtask (could be blocked on that) and do not have an Upstream tag (could be blocked on that) and got stalled status set before a certain time X (INTERVAL below is set to 36 MONTHS) and did not get stalled status set after a certain time X:
  2. Mass-add comment to tasks above that they have been stalled since time X, why that's not good, that task status will be set to declined in three months, that this can be prevented by updating the task status OR adding the Upstream tag if appropriate OR adding a subtask if the task is blocked on some other work }.
  3. Three or six months later, run same query but bump the two INTERVAL values by three or six months.
  4. Mass-decline tasks in query result with an explanation how to reopen.

[1] SELECT CONCAT("https://phabricator.wikimedia.org/T", mt.id) AS taskID, u.userName AS author, mt.title, FROM_UNIXTIME(mta1.dateCreated) AS stalled_date FROM phabricator_maniphest.maniphest_task mt JOIN phabricator_user.user u ON u.phid=mt.authorPHID INNER JOIN maniphest_transaction mta1 WHERE mt.status = "stalled" AND mt.phid NOT IN (SELECT e.src FROM phabricator_maniphest.edge e WHERE e.type = 41 AND e.dst="PHID-PROJ-bchzb6qpl3jhgs2oq6um") AND mt.phid NOT IN (SELECT e.src FROM phabricator_maniphest.edge e WHERE e.type = 3) AND mt.phid = mta1.objectPHID AND mta1.transactionType = "status" AND mta1.newValue = "\"stalled\"" AND FROM_UNIXTIME(mta1.dateCreated) <= (NOW() - INTERVAL 36 MONTH) AND mt.phid NOT IN (SELECT mta2.objectPHID FROM maniphest_transaction mta2 WHERE mta2.transactionType = "status" AND mta2.newValue = "\"stalled\"" AND FROM_UNIXTIME(mta2.dateCreated) >= (NOW() - INTERVAL 36 MONTH)) ORDER BY mta1.dateCreated;

And before doing that, likely ask on wikitech-l@ etc if/why people think this is a potentially bad idea.

Change 638163 had a related patch set uploaded (by Aklapper; owner: Aklapper):
[operations/puppet@production] phabricator weekly changes email: List stalled task stalled for years

https://gerrit.wikimedia.org/r/638163

Using the SQL query above, as of early 2020-11-01 there were
100 stalled tasks for more than 36 months, 137 for more than 24 months, 279 for more than 12 months.
Procastinating, I went through a bunch of them and now reduced numbers to
47 stalled tasks for more than 36 months, 77 for more than 24 months, 217 for more than 12 months.

My low-level idea (to start somewhere™) is to add that query to the weekly Phab reports email, sometimes triage old ones, slowly tighten the timeframe. In theory.

1+-------------------------------------------+-----------------------------------------------------------------------------------------------------------+---------------------+
2| taskID | taskTitle | stalled_date |
3+-------------------------------------------+-----------------------------------------------------------------------------------------------------------+---------------------+
4| https://phabricator.wikimedia.org/T62395 | VisualEditor: Unclosed <dl> and <div> tags caused by absent {{refend}} template cannot be repaired | 2015-02-02 19:24:14 |
5| https://phabricator.wikimedia.org/T92983 | Add IATE's TBX dictionary | 2015-03-17 17:10:42 |
6| https://phabricator.wikimedia.org/T93392 | Make sure async wrapping for Javascript error logging only happens when the call is truly async | 2015-03-25 23:39:38 |
7| https://phabricator.wikimedia.org/T44594 | Stop using rel=nofollow on all external links | 2015-05-16 17:01:32 |
8| https://phabricator.wikimedia.org/T101017 | Early security release access for Lcawte (ShoutWiki) | 2015-07-28 09:00:57 |
9| https://phabricator.wikimedia.org/T1 | Get puppet runs into logstash | 2015-08-14 23:23:02 |
10| https://phabricator.wikimedia.org/T115692 | Flash of unstyled content (FOUC) on Login page and Preferences (and possibly other highly visible pages?) | 2015-11-19 22:46:51 |
11| https://phabricator.wikimedia.org/T119145 | Copy-Paste: VE should add rev-id attributes to content pasted from other wikipages | 2015-11-24 20:24:50 |
12| https://phabricator.wikimedia.org/T101915 | Move private wikis to a dedicated cluster | 2015-12-05 15:21:19 |
13| https://phabricator.wikimedia.org/T70468 | Images with border parameter should display captions | 2015-12-21 14:12:03 |
14| https://phabricator.wikimedia.org/T40186 | Allow autoconfirmed users to move files on Wikimedia wikis by default | 2016-01-12 14:37:13 |
15| https://phabricator.wikimedia.org/T123128 | Support Bibtext format on VE when citing a reference | 2016-01-26 20:13:23 |
16| https://phabricator.wikimedia.org/T66710 | Embed should use localized wikitext | 2016-02-04 21:48:26 |
17| https://phabricator.wikimedia.org/T706 | Requests for addition to the #acl*Project-Admins group (in comments) | 2016-03-01 22:38:00 |
18| https://phabricator.wikimedia.org/T114454 | [RFC] Visual Templates: Authoring templates with Visual Editor | 2016-03-02 22:49:19 |
19| https://phabricator.wikimedia.org/T128573 | VisualEditor single tab popup should keep appearing until you use it | 2016-03-07 19:37:06 |
20| https://phabricator.wikimedia.org/T127350 | Convert T28213 Strip marker issues (tracking) to a "Strip-Marker" tag | 2016-03-09 12:36:00 |
21| https://phabricator.wikimedia.org/T49344 | Internal nodes should eventually be in a separate document ("sub-documents") | 2016-06-29 16:40:08 |
22| https://phabricator.wikimedia.org/T137342 | Trying to replace all contents of a table cell doesn't work, causes loss of focus | 2016-07-19 19:26:25 |
23| https://phabricator.wikimedia.org/T145239 | Media settings > Advanced settings > Image size: Those numbers are confusing | 2016-09-13 19:23:57 |
24| https://phabricator.wikimedia.org/T146128 | Reconsider hiding slugs between two content (text) nodes | 2016-09-20 19:28:21 |
25| https://phabricator.wikimedia.org/T149152 | Diffusion 'view raw file' policy and behavior is unclear and inconsistent | 2017-01-27 01:46:37 |
26| https://phabricator.wikimedia.org/T124020 | Duplicate icons: Bigger/Smaller and LargerText/SmallerText | 2017-03-14 21:20:29 |
27| https://phabricator.wikimedia.org/T160284 | Clean up duplicate coordinate claims | 2017-03-15 15:28:53 |
28| https://phabricator.wikimedia.org/T163370 | Automatic edit used as section heading in &section=new edit | 2017-04-24 17:30:50 |
29| https://phabricator.wikimedia.org/T82831 | ---restricted--- | 2017-04-25 23:58:42 |
30| https://phabricator.wikimedia.org/T25216 | Move the Nourmande Wikipedia from nrm to nrf | 2017-04-30 03:35:26 |
31| https://phabricator.wikimedia.org/T27522 | language code change for Samogitian: "bat-smg" to "sgs" | 2017-04-30 03:42:57 |
32| https://phabricator.wikimedia.org/T17988 | Rename 'roa-rup' wikis to 'rup' | 2017-04-30 03:54:51 |
33| https://phabricator.wikimedia.org/T31186 | Rename Võro Wikipedia, fiu-vro -> vro | 2017-04-30 03:55:31 |
34| https://phabricator.wikimedia.org/T36217 | Rename emlwiki -> eglwiki | 2017-04-30 03:56:01 |
35| https://phabricator.wikimedia.org/T64717 | Move oldwikisource on www.wikisource.org to mul.wikisource.org | 2017-04-30 05:13:13 |
36| https://phabricator.wikimedia.org/T110190 | Rename "simple" wikis to "en-simple" | 2017-04-30 05:13:20 |
37| https://phabricator.wikimedia.org/T124657 | Rename cbk-zamwiki to cbkwiki | 2017-04-30 05:13:33 |
38| https://phabricator.wikimedia.org/T127680 | Rename Serbo-Croatian Wikipedia and Wiktionary from sh.wiki* to hbs.wiki* | 2017-04-30 05:13:46 |
39| https://phabricator.wikimedia.org/T147164 | Site identifier and domain prefix for nowiki should be changed | 2017-04-30 05:13:51 |
40| https://phabricator.wikimedia.org/T161596 | [EPIC] Address vandalism issues with showing Wikidata descriptions on mobile | 2017-06-06 11:48:34 |
41| https://phabricator.wikimedia.org/T114445 | [RFC] Balanced templates | 2017-06-07 19:21:16 |
42| https://phabricator.wikimedia.org/T171157 | Monitor internal CA expirations | 2017-08-21 15:13:35 |
43| https://phabricator.wikimedia.org/T6793 | Rename Alemannic Wikipedia from als.wikipedia to gsw.wikipedia | 2017-08-24 23:08:17 |
44| https://phabricator.wikimedia.org/T146070 | Review progress of dumps rewrite | 2017-08-31 21:46:25 |
45| https://phabricator.wikimedia.org/T142897 | Heritage Monuments database/API seems to be lacking Wiki page links | 2017-09-01 19:05:37 |
46| https://phabricator.wikimedia.org/T71607 | Split 'protect' right in two distinct permissions (instead of using '$wgCascadingRestrictionLevels') | 2017-10-14 14:36:58 |
47| https://phabricator.wikimedia.org/T139052 | Portal Page Update: wiktionary.org | 2017-10-19 16:43:34 |
48| https://phabricator.wikimedia.org/T26413 | Consider DNSSec | 2017-10-23 18:39:27 |
49| https://phabricator.wikimedia.org/T116027 | Make selenium users use botflags at beta-cluster | 2017-10-30 14:05:37 |
50+-------------------------------------------+-----------------------------------------------------------------------------------------------------------+---------------------+
5146 rows in set (0.063 sec)

100 stalled tasks for more than 36 months, 137 for more than 24 months, 279 for more than 12 months.

As of today that's down to 18 stalled tasks for more than 36 months, 33 for more than 24 months, 160 for more than 12 months.

Change 638163 merged by Dzahn:
[operations/puppet@production] phabricator weekly changes email: List stalled task stalled for years

https://gerrit.wikimedia.org/r/638163