Page MenuHomePhabricator

Technical development statistics (Git and Gerrit) on https://wikimedia.biterg.io/
Closed, ResolvedPublic

Description

A short introduction (10-15min?) to https://wikimedia.biterg.io/ for anyone interested in statistics about our technical community who use our technical infrastructure (Wikimedia Git; Gerrit; Phabricator; IRC; Mediawiki.org).
Followed by questions, (hopefully) answers, discussion.

This is a rather spontaneous session and I'm probably only going to showcase some very basics.
I'm looking forward to comments, feedback, and some questions.

Website: https://wikimedia.biterg.io/
Upstream project: https://grimoirelab.github.io/
Upstream repositories and bug reports: https://github.com/chaoss/
Downstream documentation: https://www.mediawiki.org/wiki/Community_metrics
Downstream bugs and feature requests: https://phabricator.wikimedia.org/tag/analytics-tech-community-metrics/?order=priority
List of database indices (to use in the advanced search field on the website): https://wikimedia.biterg.io/app/kibana#/discover


Notes/script used by @Aklapper during the session:

  • Goals of this session:
    • 1. Create awareness that this exists
    • 2. Show some basics what you can do (from a user point of view, not admin stuff); about 15 minutes
    • 3. Get feedback, is this useful or not, etc
    • 4. This session will not answer your developer questions about Kibana or Elasticsearch code.
  • Grimoirelab platform/suite, under the CHAOSS umbrella of the Linux Foundation, written in Python and all FOSS, ran by Bitergia (company spin-off from a research group at the University of Madrid), also providing services for Mozilla and other organizations
  • Gathering activity data from different sources (see the left pane of https://wikimedia.biterg.io ) like Git, Gerrit, Phabricator, IRC, Mailing Lists, mediawiki.org -> indexing -> Database. Does not mean all and any data from those sources is indexed, though (e.g. not GitHub, see T186736).
  • UI: if you are used to Kibana, no news for you:
    • Overview, click items like names, repositories, organizations -> filtered view applied. Enable/disable them below text field.
    • Timeframe in upper corner.
    • Exporting short URLs possible via "Share"
    • Textfield for advanced search but need to know exact index names; see "Discover" link in docs. Standard Lucene search syntax, example: `author_bot:false AND author_name:"Foo"
  • Questions and answers; discussion

Event Timeline

Aklapper triaged this task as Medium priority.May 15 2018, 8:38 PM
Aklapper created this task.
Aklapper moved this task from Backlog to Session on the Wikimedia-Hackathon-2018 board.
Tgr added a subscriber: Tgr.May 18 2018, 9:09 AM
Aklapper raised the priority of this task from Medium to High.May 18 2018, 3:44 PM
Aklapper updated the task description. (Show Details)May 19 2018, 1:25 PM
Aklapper updated the task description. (Show Details)May 19 2018, 3:02 PM
Aklapper updated the task description. (Show Details)May 19 2018, 3:21 PM

Jayvdb brought up https://www.openhub.net/orgs/wikimedia/outside_projects for dependencies of our projects. Plus potentially data in personal repositories out there.

Aklapper closed this task as Resolved.May 19 2018, 5:02 PM

5 people, 15-20min basic overview, after that interesting discussions. Thanks everyone!

Aklapper updated the task description. (Show Details)May 29 2018, 10:44 AM

I added the notes/script which I used during the session to the task description, so that's not anymore only a piece of paper on my desk...

Vvjjkkii renamed this task from Technical development statistics (Git and Gerrit) on https://wikimedia.biterg.io/ to 8vcaaaaaaa.Jul 1 2018, 1:09 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed Aklapper as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from 8vcaaaaaaa to Technical development statistics (Git and Gerrit) on https://wikimedia.biterg.io/.Jul 2 2018, 2:02 AM
CommunityTechBot closed this task as Resolved.
CommunityTechBot assigned this task to Aklapper.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added a subscriber: Aklapper.