Marking T116370: Community health survey for Wikimedia technical spaces tentatively as a blocker, just in case such survey would have anything to do with our KPIs.

The metrics that CLs are collecting are not ideal, as they measure community activity rather than the team's. The challenge faced is that we don't have easy tools for measuring staff activity. One example of possible KPI is "time to first response on Talk page" - that isn't easily measurable without a script (that I'm aware of).

I'll continue poking at this in the next few weeks.

Just copy&pasting some notes we had in some internal doc:

• Number of volunteer developers and contributors involved in software projects led by the Wikimedia Foundation.
• Number of active tech ambassadors and wiki projects covered by them.
• Number of developers contributing their first 1 / 5 / 10 patches to Wikimedia projects.
• Number of Community Tech backlog projects completed by others.

Let me paste our stated goals here:

1. We want to measure results, not activity. However, we recognize that results are not fully under our control. For example, our results are affected by the actions of other WMF teams and by the phase that each product is in.
2. We want something that is meaningful, even if it requires thoughtful interpretation.
3. We want something that can be measured now, with no new tools.
4. We want to change these later, if we find better indicators or get better tools.
5. At this time, we are focusing on our community-facing roles, rather than our staff-facing roles. Consequently, our KPI will only measure part of our work.

If you're aware of anything that changed since our first iteration (like new tools making it possible for us to measure different things now), or any teams which have been able to prove (not just "correlate") that the impact they see in community metrics is actually caused by work they and they alone did (which TBH is not what I think when I read KPIs for teams in Product), then LMK?

T124041: Technical Collaboration narratives and budget for core work are still missing KPIs for the work that Community Liaisons currently do supporting WMF teams directly. Since that is a draft, should we list the KPIs that you are currently using? While we define better ones (if we find better ones).

For Core work, consistency around the Liaisons' work in product development (Project/initiative 1) will add cohesion. The pages listed here are not inclusive or consistent with the engagement that's actually happening across the products.

In talking with Quim yesterday, we keep coming to "the one thing" that can be measured. Given the nature of the work that the team does, it's a bit complex to find one *place* where something can be measured, but on thing can perhaps be delineated. I'm doing some reading this week and next and will try to throw a few ideas out there.

"Time to first reply" is a customer service metric, not a community management one. Also, the best proof of my success is when the local community members can answer each other's questions. That means that I've built a sub-community with knowledge, arranged for useful and organized documentation, etc.

In terms of "supporting WMF teams directly", we could always just ask the teams.

Our current KPIs don't cover our interactions with staff at all.

@Whatamidoing-WMF - you are raising a good point around the customer service metric, thank you for always bringing your ideas to this! There's more information the KPI page. Might be better to hold the conversation there?

In a couple of session some team members gathered to recommend a new CL KPI. The current thinking is that we may count "major conversations" that the team participates in for individual products, but the definition of what constitutes a major conversation needs to be clarified.

CLs updated KPIs: https://meta.wikimedia.org/wiki/Community_Liaisons/KPIs

The current thinking is that we do a flat count for Q4, and while doing so consider ways of qualifying the discussions had, with a proposal of how to qualify these conversations beginning Q1'16/17.