On the German Community Wishlist on https://de.wikipedia.org/wiki/Wikipedia:Umfragen/Technische_W%C3%BCnsche/Top_20 each item has points associated. You probably want to talk to WMDE-TechWish to get info how they came up with scoring criteria.

@Bmueller: How are point calculated for each task at https://de.wikipedia.org/wiki/Wikipedia:Umfragen/Technische_W%C3%BCnsche/Top_20? Is that based solely on votes?

For the TCB team's internal evaluation (after they have been narrowed down to the top 20 requests by the community), they apparently focus on feasibility and effort. Further, they check if the requests will possibly be covered by another teams' work in the future (e.g. by Visual Editor...) or if there is a massive blocker (e.g. SUL finalisation needs to be done first).

@kaldari: The points are based solely on votes. The number of votes shows which requests are most important to the community. However, if the request with the most votes can't be implemented for whatever reason straight away, you would start with another request first. Note that the evaluation after the community's vote is only a very first estimation. It was also not for internal use only, but for everyone interested in the wishlist and published on-wiki ("Aufwand und Realisierungsmöglichkeiten"/"effort and implementation options"). Through this, it was possible to give a rough overview and to structure the wishes in categorys (possible next tasks, long-term projects ...). A much more detailed estimation follows once the team starts to work on the respective request.

@Bmueller, has there been any discussion / criticism about using a voting system in the German community?

Voting has many flaws (we discussed a lot about this in the context of Bugzilla). However, any system not creating a huge research overhead will have some flaws as well. At least the voting criteria is clear: more votes gets a request higher in the list. As Birgit explains, the ranking of most voted requests is not a direct mandate to the developers, but a discrete sorting priority of tasks to be analyzed.

If tasks 1st, 2nd and 4th are not implementable for reasons that are documented, then starting to work on the 3rd is fine. Knowing that 1, 2, 4 are blocked and why should help solving those blockers in a way or another in the mid term, or should help discarding a request altogether for good reasons.

For what is worth Phabricator has tokens, although I'm not sure how easy would be to make useful queries via web or API, and the tokens are reacher than +1/-1. Still, if a user wants to give a thumbs up to a task, they already can.

@Qgil: I agree that voting has flaws, but on the other hand it's crucial to find a way to prioritize requests. We discussed the voting system question (in this case: the pro's and con's of running a technical wishes poll) as well as alternative options at Tech on Tour (https://de.wikipedia.org/wiki/Wikipedia:Technische_W%C3%BCnsche/Tech_on_Tour). The feedback on using on-wiki-polls as a method was in general good to excellent. However we also found out that polls have barriers and exclude some people (e.g. editors without tech knowledge) or do not provide enough room to discuss the technical needs of a specific user group. Thats why we plan to organize user- or topic-specific workshops (e.g. admin workshop, fotografers' workshop, workshop for people who support new editors ...) in addition. The technical wishes resulting from these workshops will be addressed by the dev team, too.
About using Phab to collect/discuss/vote on technical wishes: Imho the main question is: Who we want to reach, which of them is using Phabricator at the moment, who might be excluded by using Phabricator as the central hub?

One possible criteria for evaluating tasks could be: Number of wikis that this request will be beneficial to. Because not all requests will be beneficial to all wikis, and we want to impact as many wikis as we can.

I had an idea that when we are collecting requests, we should allow the proposers to make the case for it. Like:

• Request task:
• Why this task is important/needed:
• Number of wikis that this will be helpful to:
• Possible cons:
• Existing discussions that support this request: (Talk pages in the community so we don't have to go around looking if there is consensus on this)

etc.

Then we can let everyone vote upon these requests.

So the ideas we have so far for potential criteria are:

• Support
  • How many votes did it get?
  • Are there discussions showing consensus for the request?
• Feasibility
  • How much work is involved?
  • Are there any blockers?
  • Does our team have the necessary knowledge to accomplish the task in a timely manner?
• Impact
  • How many wiki projects will this benefit?
  • How many editors will this benefit?
  • How much will this improve the efficiency and happiness of the communities?
  • Is there existing software that can cover this need? (or software that is already being developed)
• Risk
  • Are there any potential cons?
  • Is the task well defined with a clear scope? (i.e. does it have defined acceptance criteria)

Under the Support heading, I would like to add one more question:

• If the task involves working on an existing codebase, are the current maintainers open to us modifying or forking their code?

I realize this isn't legally necessary in most cases (since it's all open source software, hopefully), but we don't want to burn any bridges with volunteer developers by stepping on their toes. Sorry about the mixed metaphors!

If no one has any suggestions for modifying this list further, I would like to publish it on our MediaWiki.org page and close the task. We can always revise it later as well.

This does not seem to take account of the discussion at https://meta.wikimedia.org/wiki/Talk:Community_Tech_project_ideas#Determining_what_projects_get_Tech_time_in_what_order -- was it envisaged that you would involve the community in setting priorities, or is it something that you see as being entirely for the team to do? I mention this because it seems to be the consensus at that discussion, and in the immediately preceding one at https://meta.wikimedia.org/wiki/Talk:Community_Tech_project_ideas#Process_questions that the community should lead on setting the priorities. However, whichever decision you ome to, it would probably be helpful to at least announce them to the community at Meta rather than leaving it to them to discover by scanning the list of tasks at this board.

@Rogol_Domedonfors: The community will definitely be taking the lead in prioritizing our work. Of course there will be some requests from the community that are not actually feasible or appropriate for our team to work on. For example, there may be a technical issue that blocks implementation or it may make more sense for us to refer a task to a different development team. We need a systematic and transparent way of making those evaluations and communicating them back to the community. The entire process for how we decide what to work on will be posted on-wiki in the near future. I'll definitely make sure that the folks on Meta are made aware of it. Also, I'll try to make it more clear that Community Support is the most important of the criteria.

The potential criteria above captures most of what is on meta. I would clarify the "are there any potential cons?" to specifically include "does this negatively affect any group of editors"?

@Doc_James: Good suggestion. I'll add that to the list.