Page MenuHomePhabricator

Decide whether we need to add a severity (impact) field to match Bugzilla's
Closed, DeclinedPublic

Description

I don't see a tracking of severity distinct from priority. Considering that priority has no universal meaning, but only a per-component meaning (at best), this seems very inappropriate for a disperse project like ours.
http://c2.com/cgi/wiki?DifferentiatePriorityAndSeverity

Moreover, the main use we make of "Priority" in bugzilla, i.e. how hard/expensive a bug is, is already covered by "Points".


See also: T93499: Add support for task types (subtypes)

Details

Reference
fl168

Event Timeline

flimport raised the priority of this task from to Medium.Sep 12 2014, 1:29 AM
flimport set Reference to fl168.

qgil wrote on 2014-04-19 16:14:25 (UTC)

If we need a field for severity, we can create it through Phabricator's configuration options.

qgil wrote on 2014-04-22 00:59:21 (UTC)

Marked as resolved for the purpose of this RfC. If we want Severity field we can have it.

aklapper wrote on 2014-04-22 07:03:29 (UTC)

I'd like to see a broader discussion first if anybody would like to have a "Severity" field (or maybe "Impact" which would be a better name). Priority, Severity and Target Milestone in Bugzilla often feel like unneeded complication.

Two examples refering to potential shortcomes of Bugzilla's concept:
JamesF once mentioned

  • ​Timing-priority == "this needs to be done soon"
  • Importance-priority == "this is affecting lots of people / lots of actions / has major impact"
  • Difficulty-priority == "this is really hard to do"​

RobLa once thought of merging Target Milestone and Priority - both should describe when / how urgent an issue should get fixed, so (simplified) high priority would be TM = next release, normal priority = release after, low priority = not planned.

aklapper wrote on 2014-04-22 07:05:18 (UTC)

.... so I don't consider "No severity field" a bug per se. Missing functionality to describe or categorize something would be a bug if it's considered important enough to categorize. :)

qgil wrote on 2014-04-23 21:03:14 (UTC)

For what is worth, there is an interesting discussion about Important vs Urgent in the context of Phabricator at https://secure.phabricator.com/T4807

jdforrester wrote on 2014-05-10 15:06:41 (UTC)

The proposal in a meeting at the Zürich Hackathon was to mirror Bugzilla's existing severity field in Phabricator when we set it up, alongside the priority one.

qgil wrote on 2014-05-10 23:26:56 (UTC)

To be precise, this was "a proposal" and not "the proposal". I the topic of priority/severity/importance deserves a good community discussion, in my opinion. We have a chance to improve the current system, if needed. After the migration will be too late.

If we discuss and reach to the conclusion that changing the current approach is not needed, fine. However, deciding after a few minutes of discussion in an afternoon doesn't look sensible to me.

jdforrester wrote on 2014-05-11 07:39:32 (UTC)

To be precise, this was "a proposal" and not "the proposal".

It was, however, the sole proposal moved, and reached unanimous approval (if not agreement), especially that this shouldn't be a block for Day 1 and could instead be changed later.

qgil wrote on 2014-05-11 09:25:00 (UTC)

It was, however, the sole proposal moved

Because I wasn't paying enough attention. :)

If we have a Severity field in Day 1, we will have it in Day 10.000. If there is a time to challenge our current classification of bug/task status, this time is now.

Then again, while I do have a strong opinion about the need to revisit this topic and find an agreement now, I don't have any strong opinion about the decision itself. Whatever works for Andre will work for me.

TheDJ wrote on 2014-05-23 18:15:22 (UTC)

Can we set the visibility of fields ? Then we can at least import it and initial hide it for everyone, then argue about how to follow up :)

aklapper wrote on 2014-05-28 17:00:37 (UTC)

Can we set the visibility of fields ?

Admins can add custom fields at http://fab.wmflabs.org/config/edit/maniphest.custom-field-definitions/

Docs in https://secure.phabricator.com/book/phabricator/article/custom_fields/ say "The fields will then appear in the other config option for the application (for example, in maniphest.fields) and you can enable, disable, or reorder them." so disabling should be possible. Haven't tried; if it does not fit our needs a hackish workaround would be using "display: none" CSS which I'd like to avoid.

jdforrester wrote on 2014-05-29 15:33:57 (UTC)

In T168#30, @Aklapper wrote:

Can we set the visibility of fields ?

Admins can add custom fields at http://fab.wmflabs.org/config/edit/maniphest.custom-field-definitions/

Docs in https://secure.phabricator.com/book/phabricator/article/custom_fields/ say "The fields will then appear in the other config option for the application (for example, in maniphest.fields) and you can enable, disable, or reorder them." so disabling should be possible. Haven't tried; if it does not fit our needs a hackish workaround would be using "display: none" CSS which I'd like to avoid.

This seems a little limited unless I'm missing something – http://fab.wmflabs.org/config/edit/maniphest.fields/ lets you adjust what fields tasks have, but the only field shown (presumably, the only optional field?) is story points – I don't immediately see a way to remove other default fields. Also, it's not clear if removing the field disables it or just fails to show it, and of course it might behave differently for the built-in field for story points compared to a new field we added…

BTW, editing what values the built-in fields have is trivial – see for example http://fab.wmflabs.org/config/edit/maniphest.statuses/ where (if you're an admin) you can edit the task status values available, or http://fab.wmflabs.org/config/edit/maniphest.priorities/ for the task priority values.

TheDJ wrote on 2014-05-29 16:15:27 (UTC)

I'm trying this on my own instance, and i'm thus not sure if we are looking at the exact same thing, but the options is intended to be as flexible as https://fab.wmflabs.org/config/edit/differential.fields/ I presume, so I think the maniphest.fields option is currently awaiting to get the same treatment.

https://secure.phabricator.com/T5011 indicates as much.

qgil wrote on 2014-05-30 00:29:56 (UTC)

In T168#28, @TheDJ wrote:

Can we set the visibility of fields ? Then we can at least import it and initial hide it for everyone, then argue about how to follow up :)

I'd argue before importing the data. What is stopping us from having this discussion?

If we create the custom field, place there the severity data, and then hide it, such data will be "lost" from the users point of view. If we decide not to have it beforehand, or to have something else, the such data can appear in the task description as plain text.

@Aklapper and @jdforrester, I think you two should weigh in with an official position from the Phabmaster & PM team, if possible at all.

Edited: a casual call for opinions at wikitech-l might be interesting as well, especially if we plan to change the current Bugzilla status quo.

swalling wrote on 2014-05-30 19:19:29 (UTC)

I'd really really like to try leaving the Phabricator defaults alone in terms of priority. When prioritizing Growth's last sprint in Phabricator, I found the list of current values to be very useful. Any tinkering we do is kind of bikeshedding here, IMO.

One of the great problems of Bugzilla is that there is just too much bullshit to deal with. Priority and severity fields are just too fine-grained to be useful in seriously prioritizing tasks. The same goes for Target Milestone etc... almost zero people seriously use these fields or rely on them now.

qgil wrote on 2014-05-30 19:33:55 (UTC)

I also think we can keep the default values and then use projects for whatever else (Easy, Security, Regressions, Sprint Candidates, etc). Custom fields bring unnecessary clutter for all tasks, most of them not needing such level of detail.

jdforrester wrote on 2014-05-30 21:05:04 (UTC)

In T168#34, @Qgil wrote:

a casual call for opinions at wikitech-l might be interesting as well, especially if we plan to change the current Bugzilla status quo.

I agree that we should definitely set out in detail what we plan to do in an e-mail to wikitech-l once we've got an idea. Maybe it's a bit premature now? Many people find it easier to find issues with a strawman/proposal than just to answer in a vacuum.

jdforrester wrote on 2014-05-30 21:08:00 (UTC)

In T168#36, @swalling wrote:

I'd really really like to try leaving the Phabricator defaults alone in terms of priority. When prioritizing Growth's last sprint in Phabricator, I found the list of current values to be very useful. Any tinkering we do is kind of bikeshedding here, IMO.

I agree. I don't think a severity field per se is a useful addition, and instead we should probably import the data on severity into the "points" field in some fashion (Blocker -> 13, Critical -> 8, Major -> 5, … or whatever).

One of the great problems of Bugzilla is that there is just too much bullshit to deal with. Priority and severity fields are just too fine-grained to be useful in seriously prioritizing tasks. The same goes for Target Milestone etc... almost zero people seriously use these fields or rely on them now.

Remember that the milestones will be replaced by (sub)projects, which will be used to recording sprint activity. I'm assuming you do want to continue having sprints in Growth? Certainly, other teams are going to. :-)

aklapper wrote on 2014-06-02 13:02:14 (UTC)

I don't think a severity field per se is a useful addition,

+1.

and instead we should probably import the data on severity into the "points" field in some fashion (Blocker -> 13, Critical -> 8, Major -> 5, … or whatever).

So in Phab we'd have priority, points, and columns to express 'things'.

For general understanding and refering to the Severity field values in Bugzilla: Currently a bug report in Bugzilla about a crasher is critical by definition, but would be low priority if circumstances to reproduce required special rights in some rather uncommon volunteer-supported database backend (or such; to come up with one vague example).
Bugzilla's critical severity could also be expressed by a crash project or so; enhancement is already expressed via priority (instead of severity) in Phabricator. Bugzilla severities like trivial or minor don't make sense (a trivial typo could still trigger a critical crasher) and feel like sufficiently expressed by Bugzilla's easy keyword which would end up as another project in Phabricator.

Nemo_bis wrote on 2014-06-15 09:29:26 (UTC)

Priority and severity fields are just too fine-grained to be useful in seriously prioritizing tasks.

I agree on this, but mostly because the recent addition of "Highest", "Immediate" and "Lowest" is disputable.

data on severity into the "points" field in some fashion (Blocker -> 13, Critical -> 8, Major -> 5, … or whatever).

This doesn't fit my (very limited) understanding of what agile points are. Is there a good concise document summarising what's the current expectation of their meaning for most people in Wikimedia/MediaWiki?

jdforrester wrote on 2014-06-16 18:24:58 (UTC)

In T168#42, @Nemo_bis wrote:

data on severity into the "points" field in some fashion (Blocker -> 13, Critical -> 8, Major -> 5, … or whatever).

This doesn't fit my (very limited) understanding of what agile points are. Is there a good concise document summarising what's the current expectation of their meaning for most people in Wikimedia/MediaWiki?

"Most" people at WMF Engineering don't use points; I'd imagine that third party and volunteer developers have even less reason to have particular expectations of their use in the MediaWiki world, so I think we should feel free to innovate.

qgil wrote on 2014-06-16 18:39:23 (UTC)

In T168#43, @jdforrester wrote:

"Most" people at WMF Engineering don't use points; I'd imagine that third party and volunteer developers have even less reason to have particular expectations of their use in the MediaWiki world, so I think we should feel free to innovate.

Or we could just remove them until we find a good use for them. This is a custom field that someone added, not a Phabricator feature out of the box.

jdforrester wrote on 2014-06-16 18:59:59 (UTC)

In T168#44, @Qgil wrote:
In T168#43, @jdforrester wrote:

"Most" people at WMF Engineering don't use points; I'd imagine that third party and volunteer developers have even less reason to have particular expectations of their use in the MediaWiki world, so I think we should feel free to innovate.

Or we could just remove them until we find a good use for them. This is a custom field that someone added, not a Phabricator feature out of the box.

"Most" !== all; the two teams that use them have repeatedly said they are a critical day-0 blocker for them.

Waldir wrote on 2014-06-25 19:49:40 (UTC)

My two cents: I've been thinking a lot recently about how to define task priorities (in general) in an objective way. I looked at bugzilla's model for inspiration, and found the terms used (importance, priority and severity) to be somewhat vague. Thinking of "priority" as "urgency", and of "severity" as "impact", allowed me to grasp their meaning (and how they relate to each other) more intuitively.

Granted, "severity" is good enough a term, but "impact" is more common and can stand for both positive and negative impact, while severity is mostly negative thing. "Priority" definitely feels more objective when defined as "urgency". "Priority" and "importance" (and "points") are terms that should be avoided, IMO, since their meaning is not universal.

Essentially, I agree with what @Aklapper said in T168#14, except that I'd use the terms urgency, impact and effort, to mean "timing-priority", "importance-priority" and "difficulty-priority". Merging target milestone and (timing) priority sounds like a good idea in combination with the above, since it'd provide a concrete, straightforward way to define urgency. Impact can be defined in terms of how many people are affected, and how often (e.g. readers/editors/admins, vs. on every page view/only on a specific subset of pages/only when using parameters x and y). Effort can be just a rough estimate of hours needed to code up the required change (anything too big to be estimated should become a tracking issue anyway).

jdforrester wrote on 2014-06-30 19:46:13 (UTC)

In T168#46, @Waldir wrote:

Merging target milestone and (timing) priority sounds like a good idea in combination with the above, since it'd provide a concrete, straightforward way to define urgency.

I completely and absolutely disagree. "Milestone" is something like "Alpha release 244: 2014-06-30" or "1.23", and is used for short-term planning (what are we working on this week?) and retrospective listing (what did we work on in week x?). It's not a long-term planning thing at all, and trying to merge the priority concept with a short-term planning framework is doomed to failure.

Nemo_bis wrote on 2014-07-02 20:08:33 (UTC)

I think we should feel free to innovate.

For instance by removing this field: I agree with comment #44

the two teams that use them have repeatedly said they are a critical day-0 blocker for them.

Ok, then I rephrase my question: is there a document (or more) summarising what's the current expectation of their meaning for those who use them?

swalling wrote on 2014-07-02 20:16:52 (UTC)

Ok, then I rephrase my question: is there a document (or more) summarising what's the current expectation of their meaning for those who use them?

Points are used in the same way they're understood in other Agile contexts. If you're looking for documentation, all you need do is Google around.

TL;DR summary: You assign points to a task to estimate the time and effort involved in a task. You can then track how many points you completed over a given sprint (i.e. a two week period), and adjust expectations about the team's capacity in the next sprint. In other words, it helps you more accurately guess how much you can reasonably get done.

qgil wrote on 2014-07-10 17:39:30 (UTC)

Revisiting this discussion under the assumption that we would have these levels of Priority (see T436: Weekly assessment of WMF Eng Team migration to Phab proj management after release):

  • Needs Triage
  • Unbreak Now!
  • High
  • Medium
  • Needs Volunteer

Do we really gain clarity by adding a parameter of "Severity"? A "Normal" problem annoying many people will get a "Medium" / "High" priority and will be schedule in a foreseeable sprint. A "Major" or even "Critical" problem around a corner case of the likes of "only in PostgreSQL" will in practice fall under "Needs Volunteer" no matter what.

In other words, maybe what happens is that "severity" is not as objective and informative as it sounds, and what really counts is how relevant it is for those who actually can complete a task. I'd rather have a simple axis (Priority) driving what is scheduled in forthcoming sprints, and then community discussion and negotiation on whether e.g. a "Needs Volunteer" task will be taken by a team as "Medium", and from there to a sprint.

Otherwise we end up in situations where "critical" and "blocker" bugs sit around for years, diluting the meaning of those terms.

Waldir wrote on 2014-07-15 17:08:44 (UTC)

In T168#50, @Qgil wrote:

Do we really gain clarity by adding a parameter of "Severity"? A "Normal" problem annoying many people will get a "Medium" / "High" priority and will be schedule in a foreseeable sprint. A "Major" or even "Critical" problem around a corner case of the likes of "only in PostgreSQL" will in practice fall under "Needs Volunteer" no matter what.

In other words, maybe what happens is that "severity" is not as objective and informative as it sounds, and what really counts is how relevant it is for those who actually can complete a task.

After thinking about this some more, I'd say that "severity" is actually both objective and informative (it means essentially "how much this breaks things"). On the contrary, I'd argue that what's not objective is Bugzilla's "priority" (i.e. timing-priority, or "this needs to be done soon"). That should essentially be the final arbiter of what to do next, but it should be defined based on objective components, not as a subjective single-axis. Your comment goes to the core of this: the "how many people are annoyed by this" factor (i.e. "impact") is central to how we make decisions, yet it's nowhere represented in the systems we use.

So perhaps instead of a single, subjective "priority" measure, we should keep bugzilla's two-axis system, but replacing "priority" with "impact", with impact having a stronger weight in the priority calculation. That would solve the problems you point out: a low-severity, high-impact problem will have higher priority than a high-severity, low-impact one, which is what we already observe today. And an issue with a "blocker" severity could then be untackled for years without diluting the meaning of severity, because we're explicitly acknowledging it's a low-impact one.

mmodell wrote on 2014-07-15 22:35:23 (UTC)

Phabricator's tokens are a perfectly good way to represent "impact" - people can individually assign tokens to express the impact that a given bug is having on them and the tokens are prominently displayed on the task with representative icons. We just need to assign some meanings to the tokens, more explicitly than they currently have in the UI.

aklapper wrote on 2014-07-15 23:51:33 (UTC)

<tl;dr>: My current position is that we do not need a general Severity or Impact field, but I'm welcome to hear more arguments, and I'd be fine with defining clear, isolated usecases for queries (and how they support our software development processes) that we might want to implement in some way (e.g. a "crasher" project, or via defining tokens as mentioned by Mukunda).

Long version:

Severity is only objective and informative if its values are clearly defined (see https://www.mediawiki.org/wiki/Bugzilla/Fields#Severity for our attempt which isn't too great) and if it is consistently used. Both is often not the case.

Currently in Bugzilla, "normal" severity can mean anything. For "major", "Major loss of function in an important area" already has two subjective terms - what's important to you and me? An issue can be "minor" or "trivial" severity (what's the difference?) to fix when it's a typo, still a typo can trigger a crasher (defined as "critical" severity in Bugzilla and hence conflicting).

While Wikimedia Bugzilla is one of those instances which set "unspecified" severity by default, it's been shown that reporters often keep the default value set by ignoring yet another field (see http://ansymo.ua.ac.be/~alamkan/wp-content/papercite-data/pdf/lamkanfi2010msr.pdf page 2), hence up to 60% of reports in some Bugzillas are misclassified as people did not find the "enhancement" value hidden in there and instead write "RFE" in the bug summary (see page 4 of https://www.st.cs.uni-saarland.de/softevo/bugclassify/paper/icse2013-bugclassify.pdf ).

Currently I don't even believe currently that differentiating feature requests/enhancement and bugs make much sense because it's often impossible to draw the line (I won't drop the four quotes from papers now). An uncommon feature request won't happen and will be lowest priority as long as nobody works on it, and if somebody considers a feature crucial, s/he will assign it to her/him and its priority will be raised. So I guess I believe that impact (severity? frequency?) influences priority, among other things like "cost of fixing" and "risk of fixing" (which we didn't express via fields so far either in Bugzilla, but we got points in Phabricator). For the records, I stole these criteria from http://www.ericsink.com/articles/Four_Questions.html.

If you like querying for critical and blocker bugs not just for academical reasons (though again I must warn that any bug report numbers depend first on "the number of users using the software, and the convenience with which those users can register new bugs" and only to lesser extend on "the absolute number of bugs present in the software", to quote page 26-27 from Karl Fogel's "Producing Open Source Software" from 2006), I would honestly love to know the usecase, and we could for example set up a "crasher" project, cross-codebase, or play with tokens.

(PS: Yes, I like to base decisions to some extend on science as we're not the first people running into these questions.)

aklapper wrote on 2014-07-28 20:33:10 (UTC)

I take the silence as people agreeing on my last comment. If not, please raise your voice before it's too late.

Waldir wrote on 2014-07-29 20:45:25 (UTC)

tl;dr I stand by my proposal to have a severity/impact dual-axis priority measure, with the highest priority items at the top right of the resulting space.

I agree the current definition of severity isn't ideal, but it's certainly a good start. Nothing prevents us from working up it. For example, the "important area to whom" question can be defined, say, in terms of the WMF's strategic goals, or the Engineering team; and "trivial" could be removed since it's a measure of effort, not severity. Other improvements are certainly possible.

Then there's the issue of us actually making use of it; getting rid of the "unspecified" value and assigning a mid-range value by default (maybe "enhancement", based on the 60% number cited above?) could help here.

So I guess I believe that impact (severity? frequency?) influences priority

I proposed impact as a measure of how many people the issue annoys. Severity can be measured separately, as mentioned above, and frequency isn't really a factor in our case: bugs that appear to occur randomly are usually just those that occur under a (yet unknown) set of conditions. Instead, a simple scale based on an rough estimation of how many users are affected could work well.

An uncommon feature request won't happen and will be lowest priority as long as nobody works on it, and if somebody considers a feature crucial, s/he will assign it to her/him and its priority will be raised.

These (mostly the first) seem to me like effects of an "impact" metric. The second scenario is true whatever the priority defined for the bug so it's irrelevant if we want a measure that's as objective as possible (i.e. predicts how soon the issue is addressed). Using the criteria that are already implicitly in use, as Quim mentions above, seems like the best way to do this.

Note that I don't mean to simply cling to my previous position; I just think two reasonably objective measures with sane defaults are better than a single, subjective priority measure that we often won't be able to agree on, and tokens which depend strongly on the number of stakeholders that actually participate on the bug tracker (which historically has been very low and biased toward the more tec).

jdforrester wrote on 2014-07-29 21:32:21 (UTC)

I also completely disagree with @Aklapper about getting rid of some measure of severity/impact, as already stated.

As a product manager, I need an orthogonal way to represent how major an issue/problem/improvement a task is for users, distinct from whatever priority I have currently given it ("priority") and how difficult it is for engineers to do ("points").

This is vital when considering which tasks should go in each sprint, and reconsidering longer-term prioritisation. It is particularly important for triaging tasks reported by users and others, to gauge which ones to triage first.

aklapper wrote on 2014-07-29 21:51:47 (UTC)

In T168#56, @Waldir wrote:

I stand by my proposal to have a severity/impact dual-axis priority measure, with the highest priority items at the top right of the resulting space.

I'd like to see this in practice somewhere. It could work but I haven't see any tool implementing this.

I agree the current definition of severity isn't ideal, but it's certainly a good start.

I don't see how it's "a good start".

For example, the "important area to whom" question can be defined, say, in terms of the WMF's strategic goals, or the Engineering team

If the several stakeholders can somehow agree on something, okay. Again, designers might have a different interpretation of "major issue" than a performance engineer (maybe a bad example if it's unlikely that one bug report will touch both design and performance).

getting rid of the "unspecified" value

(The Severity field in Bugzilla has no "unspecified" value - that's the Priority field.)

and assigning a mid-range value by default (maybe "enhancement", based on the 60% number cited above?) could help here.

Enhancement will be expressed by the "Priority" value "Wishlist" in Phabricator. Also, "enhancement" is unrelated to any Severity/Impact. Instead it is a boolean value on its own. (That enhancement request to replace the joystick by a steering wheel in the car that we produce can be pretty severe if all other competitors ship that already and if we'd go bankrupt). It is wrong that Bugzilla has "enhancement" as a value in its Severity field.

I proposed impact as a measure of how many people the issue annoys.

I don't see how "many people" can be defined. Estimation must be based on something, otherwise I'd call it wild guessing.

I just think two reasonably objective measures with sane defaults are better than a single, subjective priority measure that we often won't be able to agree on

I miss how to concretely achieve such "objective measures" because currently we don't have them. If you can draw such concrete values and add them to the discussion here, that would be awesome. Right now it feels like "things aren't great, we can improve them" but I don't see a clear and specific proposal. And I miss why it's "better" - usecases.

aklapper wrote on 2014-07-29 22:02:35 (UTC)

In T168#57, @jdforrester wrote:

As a product manager, I need an orthogonal way to represent how major an issue/problem/improvement a task is for users, distinct from whatever priority I have currently given it ("priority") and how difficult it is for engineers to do ("points").

This is vital when considering which tasks should go in each sprint, and reconsidering longer-term prioritisation. It is particularly important for triaging tasks reported by users and others, to gauge which ones to triage first.

How often do tasks go into a sprint which have lower impact (minor or trivial, in current Bugzilla terms)?

With regard to longer-term prioritisation: Would that mean it's more likely that you add tasks to a future sprint which had a lower priority but a higher impact/severity? If so, I'd interpret that as using Severity/Impact to "subclassify" tasks sharing the same lower Priority, to bump Priority later based on Severity/Impact. It's a pattern that can also be observed in Eclipse Bugzilla (according to page 4 in http://herraiz.org/papers/english/msr84cp-herraiz.pdf ).

jdforrester wrote on 2014-07-29 22:14:35 (UTC)

In T168#60, @Aklapper wrote:
In T168#57, @jdforrester wrote:

As a product manager, I need an orthogonal way to represent how major an issue/problem/improvement a task is for users, distinct from whatever priority I have currently given it ("priority") and how difficult it is for engineers to do ("points").

This is vital when considering which tasks should go in each sprint, and reconsidering longer-term prioritisation. It is particularly important for triaging tasks reported by users and others, to gauge which ones to triage first.

How often do tasks go into a sprint which have lower impact (minor or trivial, in current Bugzilla terms)?

All the time. Normally large-impact changes are intentionally rationed per release – both because there's often a strong correlation between size of impact and size of work, but also because we don't want to make too many large changes for users at once. Also, there's a need for balance for both scale and type of workload between individual team members, and within each engineer's workload.

With regard to longer-term prioritisation: Would that mean it's more likely that you add tasks to a future sprint which had a lower priority but a higher impact/severity?

Yes, though I'd be likely (but not guaranteed) to raise the priority based on impact as well. In general, higher-priority items would be expected to be put into the sprint ahead of equivalent lower-priority ones.

If so, I'd interpret that as using Severity/Impact to "subclassify" tasks sharing the same lower Priority, to bump Priority later based on Severity/Impact. It's a pattern that can also be observed in Eclipse Bugzilla (according to page 4 in http://herraiz.org/papers/english/msr84cp-herraiz.pdf ).

That's a possible analysis, though I think that it's used more as a signal rather than a raw sub-priority field.

qgil wrote on 2014-07-30 11:21:22 (UTC)

tl;dr

Priority + Impact Tags + Effort points + Sprints is a better combination to plan our work. I would not migrate Bugzilla's Severity field as a custom field in our Phabricator.

In detail:

Considering that we are discussing the steps required before Day 1.

Considering that Severity is supposed to be helpful to plan future sprints or the next task that you will work on.

Considering that we have these priority options agreed for Day 1 (T436):

  • Needs Triage
  • Unbreak Now!
  • High
  • Medium
  • Needs Volunteer

And considering that Bugzilla has these Severity options:

  • Blocker: Blocks further development and/or testing work.
  • Critical: Crashes, loss of data (internally, not your edit preview!) in a widely used and important component.
  • Major: Major loss of function in an important area.
  • Normal: Default/average.
  • Minor: Minor loss of function, or other problem that does not affect many people or where an easy workaround is present.
  • Trivial: Cosmetic problem like misspelled words or misaligned text which does not really cause problems.
  • Enhancement: Request for a new feature or change in functionality for an existing feature.

I still think that a Severity/Impact field forced onto all tasks is an overkill that will create more noise than usefulness. Our Bugzilla is a tangible proof. Think also that our Phabricator will not only be open for bugs but for all kinds of tasks. I would not migrate Severity as a custom field, but as a searchable text string in the task description.

If a bug is a "Blocker", causes "Crash" or "Data Loss" (Blocker / Critical in Bugzilla) then we can project-tag them that way, being more specific and not making assumptions like "It crashes, therefore it has a severe impact on many people". If a bug blocks development, crashes applications, and destroys data in a bad way, it will be "Unbreak Now" or "High" priority, a candidate in any sprint planning meeting until it is solved.

Discussing severity/impact Major-Normal, Normal-Minor, and Minor-Trivial is not very different from discussing priority High-Medium or Medium-Needs Volunteer. Basically you are discussing whether a problem is annoying enough to be considered by you / your team soon, some day in the future, or never.

And Enhancement is out this equation anyway, as Andre says.

mmodell wrote on 2014-08-07 14:19:50 (UTC)

Also, FWIW: Phabricator has task dependencies to express "blockers" in much more useful and informative way... "T1 blocks T2" is much more useful than "severity: blocker" - that just says it's blocking _something_ but what's blocked?

aklapper wrote on 2014-08-10 19:50:00 (UTC)

Remaining question: Is there interest in converting the "crasher" severity to a project/tag in Phabricator, similar to a keyword now in Bugzilla? Do people query explicitly for crashers in Bugzilla by querying for critival severity?

aklapper wrote on 2014-08-18 12:50:42 (UTC)

No replies to last comment, hence assuming that searching for the terms "crash" or "NPE" (favorized by the mobile app developers) seems to be sufficient for the "critical severity" case.

Closing as DECLINED as we don't plan to copy the Severity field from Bugzilla to Maniphest.

I've read this discussion before, but I still don't understand the decision: either I forgot, or it was uncomprehensible. I'd like a summary of the reasoning to be added to the description of this task and/or documented, because I suspect some of the reasons may have been superseded by other decisions in the meanwhile.

In its current setup, AFAICS, this instance has *three* manual ways to define priority: addition to a project (meaning a group is in theory working on it), workboard (which often says things like "on hold", "next in our plate" etc.) and points (how expensive is it). This is very confusing and inconsistent with the stated goals of this whole exercise, in particular with the idea of getting rid of the divide between priority field in bugzilla and sprint/roadmap assignments in mingle/trello.

It's not even internally consistent, for instance "Need volunteer" is not a level (can't be sorted by any ordering method together with the others https://en.wikipedia.org/wiki/Total_order ), rather a tag/flag i.e. a project, or a workboard within an existing project, or just an expensive task which has no resources yet. So we already have a "priority" field which cannibalises other areas of maniphest.

Moreover, it's a huge culture shift. For countless years we've stressed how priority has a relative value (very subject to interpretations and context-sensitive) while severity has at least some very clear and relevant boundaries like feature request vs. bug. If the intention is to change this culture, just renaming or removing a field will not work: people will just talk past each other, each fitting the tool to their own view. Besides, I don't see why we'd need or want to go this direction yet: for instance, there's been a lot of talk lately (mostly around https://meta.wikimedia.org/wiki/Community_Engagement_%28Product%29/Process_ideas and the countless related threads) about regressions vs. newly introduced bugs vs. feature request, and about *increasing* our ability to distinguish among them and measure them. And now we go in the opposite direction, dropping even "enhancement" vs. [trivial/minor/normal/major/critical] bug? Again, confusing.

I'm not saying decision-makers here didn't know or consider all of this, they probably did: but I'm unable to tell, or to understand the decision. If you can't, or don't, explain the decision addressing all the topics above, then whatever you decided will not be applied in the reality of the issue tracker.

As Priority was mentioned (though this is about Severity): Story points express estimated complexity and are not directly related to priority. I agree that priority is relative and that severity in theory has boundaries if well defined but I don't see a "change to this culture". I rather see an overcategorization via many many fields in Bugzilla that are only used by a small number of users hence I challenge the usefulness. One of the fields I consider pretty much useless is "Severity", explained above. In this ticket we discussed usecases for the severity field and didn't find many.

Giving up on the Severity field has caused all sorts of negative consequences, in my opinion. Perhaps this decision should be revised in light of new superstructure which is being added (partly) to compensate the lack of such information, like T109492.

T109492 is rather unrelated to any "severity" (except for when you interpret new functionality or maintenance as always either less or more severe than the other one).

For the records, T204138 added a Risk field.

Moreover, it's a huge culture shift. For countless years we've stressed how priority has a relative value (very subject to interpretations and context-sensitive) while severity has at least some very clear and relevant boundaries like feature request vs. bug. If the intention is to change this culture, just renaming or removing a field will not work: people will just talk past each other, each fitting the tool to their own view.

It only took five years but eventually it happened: T93499, T224417#5215068. Maybe in five more years we'll be back to the place where we started, with all the bugzilla functions restored although in a messier way. :)

@Nemo_bis: You seem to mix up Severity (which had unhelpful non-discrete inconsistent values in Bugzilla like "blocker", "critical", "major", "normal", "minor", trivial" and even "enhancement") and Task Types for categorization (think "bug report" vs "feature request").