Page MenuHomePhabricator

Analyze Impact of Event Invitations Experiment
Closed, DuplicatePublic

Description

Background: As part of our Event Discovery work for WE 1.3, we are experimenting with a prospective feature called Event Invitations, which allows organizers to identify editors who may be interested in their event due to their edit history. We do this by requesting worklists from organizers (which need to include existing Wikipedia articles). We then generate an Invitation List that has a scoring system (see T353459), which finds all editors who contributed to the article in the past three years and assigns a score to the editors based on their level of contributions to the article and their recent editing history. Once we have the Invitation List, we give it to the organizers, who can choose to invite the editors on the list via wikimail or talk page messages.

Note that we originally planned to only give the organizers a list of editors who have high scores. However, we plan to now indicate which editors have higher scores, but we will also give the full list of all the editors, including those with lower scores, in case the organizer wants to invite a wider group of people, since some of the editors with lower scores may still have interest in the topic or the event.

We want to analyze the impact of Event Invitations by looking at the results from a certain minimum number of events (perhaps 10 events, which we should discuss as a team). This way, we can minimize risk if, for example, one or two events experience issues (such as being on less popular topics, being at a time that is inconvenient for many people, etc). From these events, we would like collect data on the following (for each event and the total events):

Once the organizers have invited the editors, we want to know:

  • Who they invited
  • The delivery method for how they invited them (talk page or wikimail)

From there, we can assess the impact of these invitations by looking at:

  • Who from the final invitation list registered for the event?
  • Who from the final invitation list made contributions at the event?

With this information, we can answer some of the following questions:

  • What overall results are we seeing from Event Invitations – in other words, to what degree is Event Invitations bringing in new people to events so that they can make campaign contributions?
  • Who is/not registering from the invitation list?
  • Who is/is not making campaign contributions from the invitation list?
  • What is the quality of the campaign contributions made by those from the invitation list?
    • Do they create or improve articles that reach the acceptable quality score?

With the answers to these questions, we can determine if we want to update the model and its associated scoring system and if there are any other features/product pivots that we should or should not consider for this project.

Event Timeline

@ifried there is metrics/analytics/question overlap between this ticket and ticket T349561 and it's related in progress work.
Will this ticket be using data from that ticket?
How do the two tickets relate? Is T349561 the analytics sub ticket of this one?