Page MenuHomePhabricator

Visualization(s) to see retention rates for newcomers
Closed, ResolvedPublic

Description

Visualization(s) to see retention rates for newcomers specifically: After first patch, how long are they around? Percentage of those newcomers who still stick around after a certain time frame?

Non-public upstream support ticket: https://gitlab.com/Bitergia/c/Wikimedia/support/-/issues/84

Event Timeline

Aklapper triaged this task as Medium priority.

Initial version at https://wikimedia.biterg.io/app/kibana#/dashboard/7663a1e0-e16c-11ea-b2a9-cfa8a0efe987 . See internal ticket for details.
Looking for some feedback (heads-up to @greg, @thcipriani, @Jrbranaa - feel free to unsub if not interested). I'll probably ask Bitergia for a walkthrough with us.

Reminder ping for feedback. :) Adding some more context:

<snip>
The draft version shows info about newcomers, their retention rate and their activity: https://wikimedia.biterg.io/app/kibana#/dashboard/7663a1e0-e16c-11ea-b2a9-cfa8a0efe987 .

Please have into account this is a preliminary version, as the dashboard is based on a different date field than it should be: it is using the date of the first contribution (demography_min_date), no matter its type (approval, comment, patchset or changeset). This field will be updated accordingly to the date of the first patchset as you requested, once this data is available.
Having said this, it would be great if you can provide us some feedback, so we can update and adjust the different visualizations and the information we are showing so the dashboard is fully ready when the new date field arrives.
Some notes about how this dashboard works:

  • We consider as newcomer every Gerrit user whose first contribution was made within a date range defined by a certain date and the current date.
    • Now this contribution could be of any type because of the reason explained above, but once we have the new date field it will refer only to patchsets.
  • The period of time for considering users as newcomers is defined with the time-picker (top-right corner).
    • This means the dashboard will show the activity from people whose first contribution was made during the selected date range (the end date from the selected date range must be the current date Now).
  • About considering newcomers as "Active" or "Leaving" within the community, we can say a newcomer is leaving when its last contribution was made before a defined time period, taking the current date as the end of this range. By default, we are taking 90 days as threshold but this could be modified according to your needs.

The dashboard itself contains these and more details about how the dashboard works and which fields have been used and their purpose.
</snip>

The draft of the custom dashboard implementing has been reviewed in a meeting with Bitergia and WMF on 2020-10-08.

Update:

We were also wondering about an option to exclude GSoC/Outreachy activity, to optionally list only non-Outreach program activity when wanted:

  • We were originally thinking of creating a new "GSoc/Outreachy" affiliation and adding that affiliation for Outreach program students in the database, but they are obviously also at the same time individual volunteers ("Individual" is an affiliation!)
  • Andre then wondering whether folks with Outreach program affiliation could still be listed also when searching for "Individual" affiliation.
  • Hatstall affiliation system by default assumes that any new contributor indexed in the system is "Individual". Hatstall allows to set two affiliations at the same time (multivalue field); but it is impossible to set Affiliation to Individual (because it is the default assumption and not available to set as an explicit option) and also have a GSoC/Outreachy affiliation at the very same time.
  • Multi-affiliation might get simpler with Hatstall version 2 in the year 2021; need further feedback from / discussion with Bitergia about their Hatstall plans.

Hence keeping this task opened for a few more days.

This is implemented at https://wikimedia.biterg.io/app/kibana#/dashboard/7663a1e0-e16c-11ea-b2a9-cfa8a0efe987 and the dashboard is now also linked in the main navigation on top, under "Gerrit 🡒 Retention of Newcomers".

We'll check in again next year with Hatstall 2 for the special case of filtering outreach program students.