I'm curious which metrics you are after. Note that there is also https://secure.phabricator.com/fact/ with https://secure.phabricator.com/T1562 as its background.

Hey @Aklapper
I'm interested in measuring leadtime and throughput of specific tasks for this story. Ones with Bug sub-type.

I checked already the Fact thing but they all don't seem to work (e.g. https://phabricator.wikimedia.org/fact/chart/?y1=tasks.count.create). The existing charts/plots in here https://phabricator.wikimedia.org/fact/ don't seem to fit my need here.

(Indeed, they all don't work. Meh. Facts is still a Prototype according to upstream Phab developers.)

@darthmon_wmde here's what we agreed on to measure as outcome of this:

• average/median time in states
  • needs work
  • stalled
  • in progress

=> to know when to panic when a task is so off

• Lead Time (absolute and median) Closed - Created

• Throughput nr. of tasks closed per duration

=> to see a trend for campsite progress

Not sure if this comment will be helpful, but anyway:

I see that your script uses maniphest.search to get related tasks. Then for changes in the tasks themselves it's probably transaction.search to use.
For example to get the Epoch(s) when a task got resolved, declined, invalid for the task Txxxxxx, this would be something like:

echo '{"objectIdentifier":"Txxxxxx"}' | /var/www/html/phab/arcanist/bin/arc call-conduit --conduit-uri https://phabricator.wikimedia.org --conduit-token "cli-xxxxxxxxxxxxxxxxxxxxxxxxxxxx" transaction.search | jq '.response.data[] | select(.type == "status") | select(((.fields.old == "open") or (.fields.old == "stalled")) and ((.fields.new == "resolved") or (.fields.new == "declined") or (.fields.new == "invalid"))).dateModified'

If I understand correctly, @WDoranWMF is interested in similar stats for CPT's Clinic Duty so maybe there's some room for collaboration here. Or maybe not. :)

hey @Aklapper

that's very nice that you went over the code, thanks!

transaction.search sounds indeed a better endpoint for the throughput caluculation. For lead/cycle-times, it can as well be used, but then I need to do two calls to that endpoint to get when the task was created, and to get when it was closed. using maniphest.search kinda was easier here as it gives me both in the response, and it must be as well less overhead on the backend as there's no need to search transaction history for specific changes, I reckon.

@WDoranWMF hey Will, I was only planning on drafting some scripts to use them for a while within Wikidata team and see if the numbers bring any value to our process. But if there's an intersection in the numbers we want to calculate, I'd be very happy to collaborate and turn those scripts into a more mature project.

For lead/cycle-times, it can as well be used, but then I need to do two calls to that endpoint to get when the task was created, and to get when it was closed. using maniphest.search kinda was easier here as it gives me both in the response, and it must be as well less overhead on the backend as there's no need to search transaction history for specific changes, I reckon.

@Aklapper ignore and forgive that :) I couldn't be more wrong ;) I don't need two calls, just need to select more fields with transaction.search most likely.. I'll give it a try in throughput first and then check for cycle-times

Thank @Aklapper for linking me to the task.

hi @alaa_wmde, I think there would definitely be some overlap. We're hoping to get a sense of the history of a task and how it moves through our maintenance team, and in particular how long it stays in different states. I would be super interested in collaborating. I'll review the scripts today.

Hi @WDoranWMF ,

Sorry for the late reply.

esp. the bit "particular how long it stays in different states" is definitely going to be an overlap. We will want to calculate that, and then probably aggregate some of these states into higher level view, so that we can calculate avg. lead times for our tasks in ideation, design, implementation, delivery phases. Having those averages will then help us know how we improve, and when to be alert about outlier tasks.

As for the script, I will try to put it in a final state for the current use-case of it within Wikidata team. Let's sync at somepoint soon on where/how to build a more generic script that can be used by different teams? do we want to team up on it and spend few couple of hours every week on that topic?

hi @alaa_wmde, thank you so much for getting back to me! I would definitely be interested in syncing up and discussing to how we can move it forward. I'll send an email.