Page MenuHomePhabricator

Identify Wikimedia's most important/used info panels in korma.wmflabs.org
Closed, ResolvedPublic

Description

Bitergia will go for Kibana (which is just the visualization layer, there is a simplified software stack behind compared to the current stack behind korma.wmflabs.org), and the software stack currently behind korma.wmflabs.org will go into maintenance mode.
To see Kibana, there is a test instance at http://s.bitergia.com/db-mediawiki-gerrit and http://s.bitergia.com/db-mediawiki-git which is for general purpose testing and does not have the specific stuff we have. https://kibana.bitergia.com/xen/app/kibana#/dashboard/Time_Focused_Dashboard is an actual example of a dashboard with a bunch of panels.

Kibana is based on indexes, coming from Differential, Gerrit, Git, Differential (once a backend has been created, see T118753: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential) etc.
If Wikimedia went for Kibana, panels, indexes and widgets need to be created based on Wikimedia's needs and what Wikimedia has in korma. (TODO: Creation should be way simpler and more standardized then for korma.)

We (@Aklapper, input from @Qgil welcome) need to discuss in order to evaluate and to provide feedback to Bitergia:

  • What are Wikimedia's most critical/important/used panels/pages in korma.wmflabs.org?
  • Which parts are "nice to have"?

First round by @Aklapper:

What those symbols mean:

✔ = Exists in Kibana (e.g. by applying filters)
✖ = Shrug / we don't care --- potentially discuss
✚ = Wanted in Kibana, or need to check whether exists in Kibana --- make sure they exist (check; file tasks; etc)

General remarks:

  • ✖ I don't care about Countries data: incomplete and we don't care/use it currently
  • ✔ Sources (e.g. list of repos, lists, etc) are always a panel in Kibana

Pages and panels on korma.wmflabs.org (to be updated):

  • ✖✔ Mailing lists
  • ✖✔ IRC
  • ✖✔ Wiki
    • ✖✔ Those sections all follow "Created items"; "Most active people in certain time frame"; etc pattern. Low priority and mostly available in Kibana.
  • ✔ Data sources | see Data-Status panel
  • ✖ Metrics definition | TODO: Explanation of terms; could be on-wiki docs in T132323
  • More links:
    • KPI: Overview
      • (✖✔) Age of open changesets (monthly snapshots): Time from last patchset in days | see above under SCR
      • (✖✔) Backlog of open changesets (monthly snapshots): Open changesets waiting for review | see above under SCR
      • (✖✔) New changesets submitted per month | see above under SCR
      • (✖✔) Active Gerrit code review users per month | see above under SCR
    • (✖✔) KPI: Who contributes code? | covered above already under SCR>Contributors
    • ✚ KPI: Gerrit review queue > Ranking of repositories; sorted by oldest median age of unreviewed changesets. | TODO: T151488: List Git/Gerrit repositories with 'worst' performance on unreviewed (CR0) changesets / changesets waiting for merge (CR1)
    • ✚ KPI: Code contributors new and gone > New + first patch(es) + Leaving + last patch: T151161: Have "Last Attracted Developers" information for Gerrit automatically updated / Integrate new demography panels in GrimoireLab product (process how to use data within WM to be further discussed/handled in T137214)
      • ✖ People Intake: Number of people with 1 submission, 2-5 submissions, 6-10 submissions. | Interesting but not used; needs more thought and maybe better data representation format.
    • ✖ KPI: Maniphest response time: 20 tickets for: Longest time without action; without comment; without resolution | Some of this could be queried in Maniphest; in any case Andre does not consider this data helpful. Median response time per project or such could be more interesting, to deal with underlying problems instead of single tasks listed here. (Plus patches always await a response. Tasks do not necessarily.)
    • ✖ Top contributors > Ranking | We do not use this data currently. If anyone wants to keep it, good arguments and a better algorithm are needed. We can already filter for the past 12months who uploaded patches and filter out orgs like WMF and WMDE.

Related Objects

Mentioned In
T163441: Technical Collaboration quarterly check in for April - June 2017
T150658: Technical Collaboration quarterly check in for January – March 2017
T151560: Provide equivalent of "SCR: Oldest open Gerrit changesets without code review" in Kibana
T151559: Provide equivalent of "SCR: Reviewers per month" in Kibana
T151558: Provide equivalent of "SCR: Code review users vs. Code review committers" in Kibana
T151557: Provide equivalent of "SCR: Age of open changesets (monthly snapshots)" in Kibana
T151556: Provide equivalent of "SCR: Distribution of open changesets (by date of submission)" in Kibana
T151555: Provide equivalent of "SCR: Open changesets vs. Open changesets waiting for review (CR0 / CR+1)" in Kibana
T151488: List Git/Gerrit repositories with 'worst' performance on unreviewed (CR0) changesets / changesets waiting for merge (CR1)
T137997: Visualization/data regressions after moving from korma.wmflabs.org to wikimedia.biterg.io
T137214: A process to use available information about new and leaving technical contributors
T127078: Play with Bitergia's Kabana UI (which might potential replace our current UI on korma.wmflabs.org)
T118753: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential
Mentioned Here
T161926: Maniphest Backend: Index which user resolved a task
T137997: Visualization/data regressions after moving from korma.wmflabs.org to wikimedia.biterg.io
T151555: Provide equivalent of "SCR: Open changesets vs. Open changesets waiting for review (CR0 / CR+1)" in Kibana
T151556: Provide equivalent of "SCR: Distribution of open changesets (by date of submission)" in Kibana
T151557: Provide equivalent of "SCR: Age of open changesets (monthly snapshots)" in Kibana
T151558: Provide equivalent of "SCR: Code review users vs. Code review committers" in Kibana
T151559: Provide equivalent of "SCR: Reviewers per month" in Kibana
T151560: Provide equivalent of "SCR: Oldest open Gerrit changesets without code review" in Kibana
T151501: Demographics: Show attracted vs. retained contributors in Git (not Gerrit) in Kibana
T151488: List Git/Gerrit repositories with 'worst' performance on unreviewed (CR0) changesets / changesets waiting for merge (CR1)
T151161: Have "Last Attracted Developers" information for Gerrit automatically updated / Integrate new demography panels in GrimoireLab product
T138002: Deployment of Maniphest panel
T146631: Data in "The Newest Authors" widget in Kibana's "Git-Demographics" is not updated
T137214: A process to use available information about new and leaving technical contributors
T118753: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential
T127078: Play with Bitergia's Kabana UI (which might potential replace our current UI on korma.wmflabs.org)
T132323: Create basic/high-level Kibana (dashboard) documentation for users

Event Timeline

Aklapper created this task.Apr 12 2016, 5:55 AM
Qgil awarded a token.Apr 12 2016, 11:58 AM
Qgil added a comment.Apr 12 2016, 12:03 PM

Yes, I think this is a very important task. Korma has been growing organically during years, our priorities about metrics have been changing, and now we have some kind of a monster. A simpler dashboard focusing on the metrics that really matter and based on an upstream OSS technology like Kibana (which is being used by other Wikimedia teams) is a very good progress.

Andre, what about scheduling an open meeting to discuss this? Meeting to assure that we discuss this promptly. Open just in case anyone else wants to join, also good if it's just the two of us. No need to make Tech Talk - like promotion either.

Aklapper triaged this task as High priority.Apr 13 2016, 12:47 PM

@Qgil , during the last meeting we already had some introductory time with Andre related to the new functionalities of the dashboard.

I guess we can provide extra feedback and ideas if needed.

By the way, what do you think about opening a 'meta task' where all of the Kibana-related tasks are blocking that task?

Aklapper lowered the priority of this task from High to Normal.Jul 15 2016, 5:48 PM
Aklapper moved this task from Backlog to Doing on the wikimedia.biterg.io board.
Aklapper raised the priority of this task from Normal to High.Jul 23 2016, 2:27 PM
Aklapper raised the priority of this task from Normal to High.
Aklapper raised the priority of this task from Normal to High.
Aklapper updated the task description. (Show Details)Aug 22 2016, 10:20 AM

Updated the task description, to be further edited...
To compare, go to https://dashboard.bitergia.com/wikimedia/ (and click "Gerrit" for SCR).

Aklapper updated the task description. (Show Details)Aug 30 2016, 12:18 PM

Moving from Aug to Sep - this needs another round of comparison, as the "real" Kibana-based installation to compare to got moved to a different domain ( https://wikimedia.biterg.io ) that I've only been made aware of this week. (But hey, I've already spotted 'new' panels that I want!)

Qgil added a comment.Sep 6 2016, 12:20 PM

What about you and me meeting half hour during some European morning in order to go through the list? Alone, that list looks daunting...

Aklapper updated the task description. (Show Details)Sep 26 2016, 11:27 AM
Aklapper updated the task description. (Show Details)Sep 29 2016, 9:40 AM
Aklapper updated the task description. (Show Details)Sep 29 2016, 12:05 PM
Aklapper updated the task description. (Show Details)Sep 29 2016, 4:49 PM

Updated the task desc after meeting with Qgil today.

Next steps:
10 ✚ (non-Maniphest) items to discuss with Bitergia in next meeting.
Other 2 ✚ Maniphest items to check / discuss once T138002 is fixed.

Aklapper updated the task description. (Show Details)Nov 21 2016, 4:24 PM
Aklapper updated the task description. (Show Details)Nov 21 2016, 5:28 PM
Aklapper updated the task description. (Show Details)Nov 21 2016, 5:31 PM
Aklapper updated the task description. (Show Details)Nov 23 2016, 3:37 PM
Aklapper updated the task description. (Show Details)Nov 23 2016, 11:13 PM
Aklapper updated the task description. (Show Details)Nov 24 2016, 2:09 PM
Aklapper changed the task status from Open to Stalled.Nov 24 2016, 2:11 PM

Created dedicated tasks for regressions and linked them from this task's description.
I'm going to mark regressions I'd call "real blockers" as subtasks of T137997: Visualization/data regressions after moving from korma.wmflabs.org to wikimedia.biterg.io.

Now this task is blocked on T138002. Once that's done, this task needs a last evaluation regarding Maniphest dashboard functionality in Kibana. Then it can be closed.

Aklapper lowered the priority of this task from High to Low.Dec 31 2016, 6:37 PM
Aklapper moved this task from Doing to Backlog on the wikimedia.biterg.io board.

Lowering priority as long as task is stalled; moving to Q1/2017 (meh).

Aklapper raised the priority of this task from Low to Normal.Feb 28 2017, 10:31 AM
Aklapper moved this task from Ready to Go to March on the Developer-Advocacy (Jan-Mar-2017) board.
Aklapper updated the task description. (Show Details)Jun 30 2017, 4:08 PM
Aklapper closed this task as Resolved.Jun 30 2017, 4:12 PM

This task has served its purpose:

  • All items marked as ✚ above in the task summary ("Wanted in Kibana,") have been checked and all these items have dedicated tasks with corresponding priorities.
  • It has helped me to understand https://wikimedia.biterg.io and our needs wrt metrics a lot better.
  • We do not aim for complete feature parity between defunct korma and https://wikimedia.biterg.io. (Nothing new.)

Hence I am resolving this task.

(Related: For the bigger picture in WMF's Technical Collaboration team, recently focus has been on reconsidering our KPIs in general. Results can be seen on meta.)