Page MenuHomePhabricator

Create Needs-user-research tag
Closed, ResolvedPublic

Description

It would be good to have a tag to mark issues which are blocked on user research.

Event Timeline

Tgr created this task.Nov 25 2014, 10:10 PM
Tgr updated the task description. (Show Details)
Tgr raised the priority of this task from to Needs Triage.
Tgr added a project: Project-Admins.
Tgr changed Security from none to None.
Tgr added subscribers: Tgr, Pginer-WMF, Gilles.
Qgil triaged this task as Low priority.Nov 26 2014, 8:48 AM
Qgil added subscribers: Aklapper, Qgil.

@Aklapper, this is one of those "Open, Stalled" flavors, right?

"needs-user-research" may be preferred as a name. That provides the user Research team some freedom to choose the research method (e.g., user testing, surveys).

Plese elaborate on "It would be good".

Would such a tag imply that there cannot be any progress on the ticket until user research has happened?

Who is responsible for performing user research? If there is a user research team where is its page on mediawiki.org or such?

Tgr added a comment.Nov 26 2014, 12:10 PM

Plese elaborate on "It would be good".

There are two basic use cases:

  • help people doing user research to find the tasks where their help is most needed
  • tell coders that this task is blocked and they should not work on it (e.g. it might be an easy bug but still not suitable for new volunteers)

Would such a tag imply that there cannot be any progress on the ticket until user research has happened?

I imagine so.

Who is responsible for performing user research? If there is a user research team where is its page on mediawiki.org or such?

For the WMF, that would be https://www.mediawiki.org/wiki/Design/Research I think. Others do user research in WMF though (I CCd Pginer since he did most of the user research for our team), and other organizations or volunteers might do it - need-user-research and need-wmf-user-research-team is not the same thing.

Tgr added a comment.Nov 26 2014, 12:15 PM
In T75933#787841, @Tgr wrote:

Who is responsible for performing user research? If there is a user research team where is its page on mediawiki.org or such?

For the WMF, that would be https://www.mediawiki.org/wiki/Design/Research I think. Others do user research in WMF though (I CCd Pginer since he did most of the user research for our team), and other organizations or volunteers might do it - need-user-research and need-wmf-user-research-team is not the same thing.

Also, apparently there is a WMF Research team now: T75807

Tgr added a comment.Nov 26 2014, 12:18 PM

There should be a naming convention for such tags. This proposal and T75932 and T75880 use needs-X, plus there is need-volunteer and community-consensus-needed. All of those should use the same format (need-X or needs-X or X-needed).

I find this tag a bit obscure and perhaps too specialized for a tag.Could you provide examples of tasks blocked on user research that would not to be useful to tag with #wmf_design_research ?

Tgr added a comment.Nov 28 2014, 5:56 PM
In T75933#792841, @Qgil wrote:

I find this tag a bit obscure and perhaps too specialized for a tag.Could you provide examples of tasks blocked on user research that would not to be useful to tag with #wmf_design_research ?

In general I think using team tags to denote what kind of work is needed is a poor approach that creates an appearance of ownership which is something that should be avoided as much as possible in an open-source project, otherwise it becomes a bottleneck. It might be case right now that the WMF Design team is the only one capable of doing user research (I don't know if that's actually the case), but what happens when e.g. WMDE hires a UX researcher? Or an experienced volunteer joins the volunteer community? Or one of the WMF researchers decides to work on some tickets in their free time, as a volunteer?

Some examples of tickets I would work on as a volunteer but are blocked by research questions right now: T64266, T71609.

I'm not sure what you mean by obscure - if you think the concept that significant changes the user workflows should be verified by user tests is obscure within the MediaWiki community, I hope you are wrong, but if you aren't, we should work on changing that, as it is a standard best practice in the IT industry (and also common sense). Creating a tag might be a good first step towards that.

As for being too specialized, how would you generalize? Design is not a meaningful generalization IMO - the two tickets linked above do not need design as such - the first does not have any visual elements, the second is about which CSS rule to apply. Generalizing into "needs research" would make some sense - qualitative (user testing) and quantitative (metrics) research can often be applied to the same problem, but the skillsets needed are completely different, and sometimes only one of the two is practical to do.

Qgil raised the priority of this task from Low to Normal.Dec 1 2014, 11:54 AM

There are several mentions to Design-Research in this task. What is the opinion of this team?

Some examples of tickets I would work on as a volunteer but are blocked by research questions right now

These tasks could be marked with the "stalled" status. When somebody (Design-Research team or anybody else) is planning to pick up such a task and perform the required user testing, the status should be changed and the task should be assigned to a team or person responsible to perform the user testing.

The question boils down to "will somebody query specifically for tasks in Maniphest which require user testing first and then act upon some of them based on the query results".

Would love to have input from Design-Research too on which workflows would be helpful. Obviously I'd like to avoid tagging for the sake of tagging.

Aklapper lowered the priority of this task from Normal to Low.Jan 20 2015, 6:56 PM

Pinging Design-Research...

There are several mentions to WMF-Design-Research in this task. What is the opinion of this team?

I would prefer not to create this tag if there's no basic followup process defined who to pick up those tickets. Don't want tagging for the sake of tagging.

@aripstra, thoughts? we might just want an alias, but your call.

Aklapper changed the task status from Open to Stalled.Feb 12 2015, 4:50 PM

Setting status to stalled as per last comments. Please reset the status to open once input has been provided.

aripstra changed the task status from Stalled to Open.Mar 4 2015, 7:34 PM

I think having a tag "needs-user-research" is a good idea. This way we can monitor incoming requests and queue and prioritize them. We would like to work toward broadening our capabilities into the community, maybe encouraging some volunteer design researchers to participate too.

Sorry for my delay in responding. The Design Research department is new, and we all switched over to Phabricator just when I got used to the other tools. I am in catch up / learning mode here.

Could someone please provide a project description that includes clear criteria what the tag is supposed to mean, and when (and who?) to set this tag. There are good bits and pieces and previous comments here already.

The clearer the description, the less likely someone could / will use it incorrectly, and the less likely the project gets too noisy and less valuable for everybody.

After that, I am happy to create #Need-user-testing.

Aklapper renamed this task from Create needs-user-testing tag to Create Need-user-testing tag.Mar 19 2015, 1:45 PM

criteria for using needs-user-research tag:

when there are known usability issues for a functionality or product and usability testing is needed.
when there are questions / confusion / arguments about what users want or need
when it is not known if it will be easy for a user to accomplish a task or goal with the feature or product
when generative design research is needed in very early stage product development (to discover user's needs and provide data to use in concept development)
To validate that concepts are useful and usable by a certain user groups

when there are known usability issues for a functionality or product and usability testing is needed.

Playing a pessimistic bad cop, I could imagine that a number of users might set this tag on basically every slightly controversial functionality or UI change, as "usability issues" can mean a lot of different things to different people.
(In a previous project I managed a bug tracker that had a "usability" keyword and basically every reporter set it on everything because every software bug or design change might make a software less "usable" in some user's eyes. A "design" keyword can have similar problems though its understanding is slightly more shared. All entirely personal impressions only.)
Phabricator is a public place: Anybody can add tags to tasks in Phabricator. I am still concerned that with the proposed description and how "interpretable" it is, providing such a tag could require spending a lot of time removing the tag from tasks and explaining/discussing why the tag was removed from a task (what a "usability issue" is and what it is not).

Links to wiki pages in the tag description which explain certain terms ("usability", "user testing", "generative design research"; I have no idea either what the last term means) and provide background information what's covered by it and what's NOT covered might be helpful to save you some time in the long run (less things to explain again and again, less noise devaluing the usefulness of such a tag).

So I'll create this tag next week.
And I cross fingers that I'll realize in a few months that I was too pessimistic. :)

@Aklapper Thanks for your thoughts here. I agree with your concerns, and when the tag is created, and there is a place for a description, I will add links to wikipages defining the various terms, text about what this tag covers and what it does not cover. Ping me when it is a good time to do that.

Also, this little design research team will grow a bit, and part of our responsibilities will be to educate about design research methods, the value of design research, and to collaborate with people who have these skills. Triaging phab tickets with this tag will be a part of educating those who add the tag. It will also help us to better understand the amount of work that we need to do / support, and the shape collaborations with volunteer researchers and others. We would like to teach people who are interested, to do some basic usability at some point, and then qualified people could take on stuff we can't get to as the WMF DR team. Again, thank you for brining your experience to the table here. I am crossing my fingers also. :)

I agree with your concerns, and when the tag is created, and there is a place for a description, I will add links to wikipages defining the various terms, text about what this tag covers and what it does not cover. Ping me when it is a good time to do that.

@aripstra: Any time, like now. :)
Please just edit the task description here to provide an actual project description (including potential links) so I can re-use that when creating this project. :)

@Aklapper Here is a start. This is the design research page on mediawiki, and there are some definitions toward the bottom and a high level summary at the top. Also, you can use the list I created in my comment in this Phab ticket from March 20. Unfortunately, I don't know how to edit the task description here...I tried a few things just now, and there is no one around me to ask (talk about usability issues ;) I can look into it editing the task description tomorrow or next time I am around people who know Phabricator better than I do.

Thanks. Could you provide the full project description text for the time being here, by clicking "Edit Task" in the upper right corner, so I could just copy that description when creating the project? That would be awesome. :)

Could you provide the full project description text for the time being here, by clicking "Edit Task" in the upper right corner, so I could just copy that description when creating the project? That would be awesome!

Asking one more time:

Could you provide the full project description text for the time being here, by clicking "Edit Task" in the upper right corner, so I could just copy that description when creating the project?

ggellerman added a subscriber: ggellerman.EditedJun 15 2015, 11:22 PM

@Aklapper How is this for project description?

Criteria for using needs-user-research tag (any of the following):

  1. When there are known usability issues for a functionality or product and usability testing is needed to inform resolution of those issues
  2. To evaluate the value of features for specific users
  3. To test usability of features or products with specific users
  4. When design research is needed in early stage product development (to discover user's needs and provide data to use in concept development)
  5. To validate that concepts are useful and usable by a certain user groups

I might repeat myself, but if I was a potential reader or editor who's been unhappy with recent Wikimedia developer decisions, I could tag basically every task in Phabricator with that new tag, given the criteria in the previous comment.

There are no links to information how to find out whether "design research is needed", or who is supposed to "validate that concepts are useful", or who is supposed to tag tasks, so anybody or nobody might do that who might not share the same understanding of the terms used in the proposed description.

Feel free to create if you plan to maintain and triage tasks tagged with that new tag. I remain pessimistic without a better project scope, based on past experience with highly subjective terms like "usability" in public bug trackers. And I'd love to be proven wrong.

Tgr renamed this task from Create Need-user-testing tag to Create Needs-user-research tag.Jun 19 2015, 8:03 PM
Tgr updated the task description. (Show Details)
Tgr added a comment.Jun 19 2015, 10:45 PM

How about just

Used for tasks where further work is blocked by questions answerable by user research.

There is hardly any feature that wouldn't benefit from more user research; I would keep this tag for cases when it is actually necessary for moving forward. (That would also keep it consistent with other similarly named tags like community-consensus-needed which are also blocking.)

When work is blocked is necessarily a subjective question but there is nothing specific to this tag about that. In general I don't think it's realistic to expect the description of a tag to prevent abuse; existing tags and functions already provide lots of opportunities if someone wants to play obstructionist (e.g. regression, community-consensus-needed, Stalled/Declined, the priority field...), but in practice people don't, or at least not often.

Feel free to create it.

ggellerman closed this task as Resolved.Jun 22 2015, 6:47 PM
Danny_B removed a subscriber: Design-Research.