Complete Phabricator transition's measures of success are ready from my point of view. Feedback is welcome.

@Awjrichards Any input from your end on the Phabricator success measures? Will chat w/ Damon about it in a bit.

@Eloquence yes - caveat though, I just got pulled into this conversation by @Qgil yesterday, so my thoughts are not necessarily well-informed nor well-formed ;)

Developers→Project management infrastructure and documentation for best practices.
I am a little unclear on what exactly this means - particularly in regards to 'project management infrastructure'. If I'm not mistaken, it could be said that Phabricator provides project management infrastructure out of the box. It would be valuable to have scope of this and the end-state of this goal more clearly articulated. As for the documentation of best practices (which I understand I am on the hook for) my initial reaction is that we're going to need to keep it fairly high level (eg "create a new workboard for every new sprint") since different teams will have different needs for the tool, but I think this will be valuable and I am very happy to help flesh that out :)

Developers→Burndown charts.
Similar to above, what does this mean exactly?

Developers→Eliminate most uses of Trello/Mingle (Fundraising Tech exempt)
I've already heard some feedback about this particular goal, which seems to be making some teams uneasy. Is this intended to be a mandate that every usage of Trello/Mingle is eliminated (with the exception of FR Tech)? Or is this intended to be a goal of the project to convince all the teams to abandon Trello/Mingle and flock to Phabricator? How do non-team-specific based usages of Trello/Mingle fit into this rubric (eg Scrum of Scrums which uses a matrix view to represent work dependencies that can't be recreated in Phabricator)? Also, I get uneasy when I see the word 'most' in a goal. How do you know when this goal is met - what do you mean by 'most'? For instance, do you mean all projects BUT fr-tech? Or does this mean a simple majority of projects using Phabricator rather than Trello/Mingle is sufficient? It would be good to get specific here.

Stretch goal:
Developers→Basic plan for Phabricator as code review tool
I think this makes sense, though I think it would be valuable to more clearly articulate the goal (eg Basic plan for replacing Gerrit with Phabricator as a code review tool for all WMF-managed code repositories). As an aside, I am not a fan of stretch goals - it's hard for me to grok how they are valuable. I personally would prefer to see this as either a goal, or not a goal, but I understand that some folks do indeed find the 'stretch goal' concept' useful.

As a general rule (and this goes for all the projects, really), I think it would be great to have the goals (or 'measures of success') be more specific (in such a way as to clearly define scope) and measurable, and written in such a way that the end state can be generally understood by anyone glancing the list of goals (rather than intrinsically understood by project participants).

I edited my previous comment, but realized the edit may get swallowed by email notifications or whatever, so figured it would be worth re-articulating in a separate comment. I added the following to Developers→Eliminate most uses of Trello/Mingle (Fundraising Tech exempt):
Also, I get uneasy when I see the word 'most' in a goal. How do you know when this goal is met - what do you mean by 'most'? For instance, do you mean all projects BUT fr-tech? Or does this mean a simple majority of projects using Phabricator rather than Trello/Mingle is sufficient? It would be good to get specific here.

@Tnegrin @howief to help make sure mobile app readership metrics are consistent, documented.

In T667#13578, @Awjrichards wrote:

Developers→Project management infrastructure and documentation for best practices.
I am a little unclear on what exactly this means

You are right. I have removed the infrastructure because it is implicit in the goal of teams migrating (no infrastructure, no team will migrate). And then I have based the measure of success not in the existence of the document (which is just a tool) but about the effect of this tool (good collaboration between teams and most active individual contributors, sharing a common protocol). I will fine tune the measures of success at T558: Goal: Common project management guidelines to be followed by teams and individual contributors are published.

Developers→Burndown charts.
Similar to above, what does this mean exactly?

Removed. Not because we will not have this feature (we will have it) but because, again, it is a feature contributing to the actual goal of teams migrating to Phabricator.

Developers→Eliminate most uses of Trello/Mingle (Fundraising Tech exempt)
I've already heard some feedback about this particular goal, which seems to be making some teams uneasy.

Yeah, you are again right. Pushing teams to change tools "because we have a deadline agreed with management" rarely helps, and frequently creates even more resistance. Then again, it is reasonable to say that yes, one day WMF teams will be all using Phabricator instead of Trello / Mingle. A realistic and measurable goal in the spirit of this top priority is T825: Goal: The majority of WMF developer teams and sprints have moved to Phabricator. This assures a critical mass in terms of volume of activity and diversity of teams, a trend with no return that the minority will end up following.

Is this intended to be a mandate that every usage of Trello/Mingle is eliminated (with the exception of FR Tech)? Or is this intended to be a goal of the project to convince all the teams to abandon Trello/Mingle and flock to Phabricator?

With the goal reformulated, the mandate is still there, just not tied to end of 2014, and not needing to name any specific exceptions.

How do non-team-specific based usages of Trello/Mingle fit into this rubric (eg Scrum of Scrums which uses a matrix view to represent work dependencies that can't be recreated in Phabricator)?

E v e r y b o d y will fit.

If Scrum of Scrums can't make it now because there is a feature missing, please create the tasks required and join the minority that will stay longer out of Phabricator with a justified reason.

Also, I get uneasy when I see the word 'most' in a goal.

Yep, now it says "more than half" of teams and ongoing sprints.

Stretch goal:
Developers→Basic plan for Phabricator as code review tool
I personally would prefer to see this as either a goal, or not a goal

Not a goal, then. I will keep it in the ECT goals for this quarter, but we will play the regular game for the rest of the projects. There is a critical mass of code review champions that have a declared personal interest in pushing Phabricator in this front -- deprecating gitblit and Gerrit sooner than later. There is an ongoing request of one of our developer for a part-time allocation to this project, and others are already contributing at their own risk, on their own time. In fact, not being an official top priority seems to be an incentive for some. :)

Cool Quim, this generally sounds great to me and I think your reasoning is totally sound :) I've got a thought about the goal articulated in T558, though I'll leave it there.

In T667#13798, @Eloquence wrote:

@Tnegrin @howief to help make sure mobile app readership metrics are consistent, documented.

ok -- I'll make this explicit in the tasks themselves.

My comments on the goals themselves:

1. I don't like stretch goals. Let's keep them short and attainable and under-commit per Lila.
2. Mobile app goals are qualitative only? This doesn't seem right.
3. Mobile goals don't have goals, just measurements
4. In general, I don't see any targets -- we need actual numbers (of libraries, etc)

In T667#16960, @Tnegrin wrote:

My comments on the goals themselves:

1. I don't like stretch goals. Let's keep them short and attainable and under-commit per Lila.

I don't like under-committing :)

1. Mobile app goals are qualitative only? This doesn't seem right.

There are a number of resourcing issues at play in the apps right now (noted in the goals page). First of all, there's currently only one full-time iOS engineer and there has, until just this week, been only one full-time dedicated designer. That's just not enough bodies to spec, design, test, build, and ship a new reader engagement feature in 2 months across both platforms.

Second, it's a month into Q2 and we don't have any new apps readership data since the quarterly review. This has been flagged as a high priority mobile ask on a number of docs/etherpads/Trello boards now, so at this point I'm just being realistic – we'd need a lot more dedicated support in helping us decide on and continually monitor app readership metrics over the next two months if we wanted to commit to a quantitative goal on apps, and right now we just don't have that support. Let's keep this in mind for next quarter so we don't run into this problem again.

1. Mobile goals don't have goals, just measurements

Still working on that. I agree it's still a little mushy.

1. In general, I don't see any targets -- we need actual numbers (of libraries, etc)

Do we? Why? Not every project's plans fit well with a numeric target (including mobile web, for instance).