Page MenuHomePhabricator

Deprecate/undeploy the GettingStarted extension
Open, LowestPublic

Description

From the description of the Newcomer Tasks project (see T227728: [EPIC] Growth: Newcomer tasks 1.0) and the initial implementation of the underlying API (see T234426: Newcomer tasks: GrowthExperiments API for template tasks), it appears that the project supersedes the GettingStarted project.

The GettingStarted project was deployed by the old Growth Team in 2013 (?) and active development ceased circa August 2014. Because the team disbanded quickly, however, and the project was never undeployed. Newly registered users still receive the GettingStarted UX to this day.

It's great that an extension not under active development still impacts people's experiences on the Wikipedias and largely a testament to the contributors who have fixed compatibility issues and/or bugs after the extension's authors moved on. Simultaneously, all deployed code is a maintenance burden and the GettingStarted project is no exception. We should discuss whether the Newcomer Tasks project does indeed supersede GettingStarted and whether the latter can be undeployed.

Additional places to undeploy/update references:

Event Timeline

phuedx created this task.Oct 17 2019, 11:30 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 17 2019, 11:30 AM
phuedx updated the task description. (Show Details)Oct 17 2019, 11:34 AM
phuedx renamed this task from Deprecate/undeploy the gettingstartedgettasks API to Deprecate/undeploy the GettingStarted extension.Oct 17 2019, 11:38 AM

Wikis with MediaWiki-extensions-GettingStarted :

'wmgUseGettingStarted' => [
	'default' => false,
	'testwiki' => true,
	'test2wiki' => true,
	'astwiki' => true,
	'bswiki' => true,
	'cawiki' => true,
	'dawiki' => true,
	'dewiki' => true,
	'elwiki' => true,
	'enwiki' => true,
	'eswiki' => true,
	'fawiki' => true,
	'frwiki' => true,
	'fowiki' => true,
	'glwiki' => true,
	'hewiki' => true,
	'huwiki' => true,
	'iswiki' => true,
	'itwiki' => true,
	'jawiki' => true,
	'kowiki' => true,
	'lbwiki' => true,
	'mkwiki' => true,
	'mlwiki' => true,
	'nlwiki' => true,
	'plwiki' => true,
	'ptwiki' => true,
	'ruwiki' => true,
	'simplewiki' => true,
	'svwiki' => true,
	'viwiki' => true,
	'ukwiki' => true,
	'zhwiki' => true,
],

We should discuss whether the Newcomer Tasks project does indeed supersede GettingStarted and whether the latter can be undeployed.

As a practical matter the Newcomer Tasks project is only deployed to a handful of mid-sized wikis, and it also approaches the way of recommending tasks in a different way (via Special:Homepage, and not as a GuidedTour). So I don't know if we could make this a like-for-like replacement, not any time in the short term.

Some other questions:

  • IIRC there is some research showing that GettingStarted had no measurable impact on retention/activation (I might be making this up – @MMiller_WMF @nettrom_WMF do you know?), if so, it seems like it could be undeployed without consideration for its potential replacement by GrowthExperiments-NewcomerTasks
  • does there need to be a community consultation (or per wiki consultation? @Trizek-WMF ) before undeploying it?

@kostajh, do you have a way to know if it is broadly used?

IIRC there is some research showing that GettingStarted had no measurable impact on retention/activation (I might be making this up – @MMiller_WMF @nettrom_WMF do you know?), if so, it seems like it could be undeployed without consideration for its potential replacement by

@kaldari pointed me to this link which seems to suggest otherwise: https://meta.wikimedia.org/wiki/Research:Onboarding_new_Wikipedians#Overview_of_results

@kostajh, do you have a way to know if it is broadly used?

@Trizek-WMF can you please clarify what you mean by "broadly used"? Like, do you mean what proportion of new users see GettingsStarted dialogs and what percentage of those interact with them?

phuedx added a comment.EditedOct 18 2019, 2:11 PM

@kostajh, do you have a way to know if it is broadly used?

Firstly, it's been a long time (over 5 years) since I've looked at this instrumentation. Now…

With the data that we're still collecting, I think we can answer the following question: How many users of those users that are delivered the GettingStarted UX go on to start a GuidedTour? Per [0], ~4.54%.

[0]
+--------+-----------------+
|  n     | n_w_guided_tour |
+--------+-----------------+
| 185488 | 8429            |
+--------+-----------------+

select
    sum(1) as n,
    sum(if(
        gtgi.event is not null

        -- TODO Make the gettingstartedtasktoolbar* tours share step names.
        and (
            (gtgi.event.tour = 'gettingstartedtasktoolbar' and gtgi.event.step = 'editIntro')
            or (gtgi.event.tour = 'gettingstartedtasktoolbarve' and gtgi.event.step = '1')
        ),
        1,
        0
    )) as n_w_guided_tour
from
    gettingstartedredirectimpression gsri
    left outer join guidedtourguiderimpression gtgi
        on gsri.event.userid = gtgi.event.userid
        and gsri.year = gtgi.year
        and gsri.month = gtgi.month
        and gsri.day = gtgi.day
        and gsri.hour = gtgi.hour
where
    gsri.year = 2019
    and gsri.month = 9
;
kostajh updated the task description. (Show Details)Oct 21 2019, 10:50 AM

@Trizek-WMF can you please clarify what you mean by "broadly used"? Like, do you mean what proportion of new users see GettingsStarted dialogs and what percentage of those interact with them?

Yes. @phuedx gave some data above, but that's only the number of impressions. We don't know at the moment what is the percentage of people still using it to start editing.

If we put all findings we have for now together, we can say that:

  • the feature had a low results on the experiment
  • the feature has a low number of people getting it
  • the feature requires maintenance we can't provide
  • the ratio of maintenance versus usefulness is apparently low
  • Growth team works on task suggestions and, overall, more up-to-date and dedicated features to onboard newcomers.

Based on this, I would say that we are good to decommission this feature, with a simple information on Tech News explaining the rationale.

@Jdforrester-WMF -- are you planning on working on this task? The current status is actually that @marcella and company and working on an audit of what all popups are used in different contexts for newcomers so that we know what kind of impact removing this will have. I don't think we remove it until we hear from her about the status of that work.

@Jdforrester-WMF -- are you planning on working on this task? The current status is actually that @marcella and company and working on an audit of what all popups are used in different contexts for newcomers so that we know what kind of impact removing this will have. I don't think we remove it until we hear from her about the status of that work.

I assigned it to myself administratively to work with your team to make a task hierarchy that has the right bits in the right places.

Jdforrester-WMF triaged this task as Low priority.Feb 11 2020, 12:18 AM
Jdforrester-WMF lowered the priority of this task from Low to Lowest.Apr 8 2020, 4:16 PM

@MMiller_WMF - The audit (T229887) has been abandoned, so please feel free to move ahead accordingly.