Change Details

Starting on 2020-10-21, we noticed that in arwiki, cswiki, and testwiki that the "Newcomer task" edit tag is not being applied to all suggested edits. Though this was first observed on 2020-10-21, we don't know if that's when the issue started. According to tests so far, the tag may be being applied to half or fewer edits. This spreadsheet contains a list of diffs made to investigate this, along with whether the tag was applied: https://docs.google.com/spreadsheets/d/1PNVojslCoIYd9wNKYSlJqG7O4vFhcf8llQnqoT3ij_I/edit#gid=0 These diffs are all edits originating from clicking tasks on the newcomer homepage or in the post-edit dialog after completing an edit from the newcomer homepage. Once we fix this, we'll want to know: * when it began * In production on September 15, 2020 * whether it affected all wikis equally * yes * whether it affects both desktop and mobile * both * what share of edits it probably affects * we would have to analyze pre-September 15 data to figure out which proportion of edits originate from clicks to the first card seen when a user arrives at Special:Homepage. * whether those edits are recoverable so we can count them in our reporting * we can't retroactively apply edit tags but maybe we can reconstruct from other instrumentation data we have.

Starting on 2020-10-21, we noticed that in arwiki, cswiki, and testwiki that the "Newcomer task" edit tag is not being applied to all suggested edits. Though this was first observed on 2020-10-21, we don't know if that's when the issue started. According to tests so far, the tag may be being applied to half or fewer edits. This spreadsheet contains a list of diffs made to investigate this, along with whether the tag was applied: https://docs.google.com/spreadsheets/d/1PNVojslCoIYd9wNKYSlJqG7O4vFhcf8llQnqoT3ij_I/edit#gid=0 These diffs are all edits originating from clicking tasks on the newcomer homepage or in the post-edit dialog after completing an edit from the newcomer homepage. Once we fix this, we'll want to know: * when it began * In production on September 15, 2020 * whether it affected all wikis equally * yes * whether it affects both desktop and mobile * both * what share of edits it probably affects * we would have to analyze pre-September 15 data to figure out which proportion of edits originate from clicks to the first card seen when a user arrives at Special:Homepage. * whether those edits are recoverable so we can count them in our reporting * we can't retroactively apply edit tags but maybe we can reconstruct from other instrumentation data we have. For example, we could use `is_oversample` from https://meta.wikimedia.org/wiki/Schema:EditAttemptStep, to find edits where `is_oversample` is true, where the `newcomer task` edit tag isn't present, and where the user ID belongs to a user who has suggested edits enabled. That wouldn't be 100% accurate but might be a good place to start.

Starting on 2020-10-21, we noticed that in arwiki, cswiki, and testwiki that the "Newcomer task" edit tag is not being applied to all suggested edits. Though this was first observed on 2020-10-21, we don't know if that's when the issue started. According to tests so far, the tag may be being applied to half or fewer edits. This spreadsheet contains a list of diffs made to investigate this, along with whether the tag was applied: https://docs.google.com/spreadsheets/d/1PNVojslCoIYd9wNKYSlJqG7O4vFhcf8llQnqoT3ij_I/edit#gid=0 These diffs are all edits originating from clicking tasks on the newcomer homepage or in the post-edit dialog after completing an edit from the newcomer homepage. Once we fix this, we'll want to know: * when it began * In production on September 15, 2020 * whether it affected all wikis equally * yes * whether it affects both desktop and mobile * both * what share of edits it probably affects * we would have to analyze pre-September 15 data to figure out which proportion of edits originate from clicks to the first card seen when a user arrives at Special:Homepage. * whether those edits are recoverable so we can count them in our reporting * we can't retroactively apply edit tags but maybe we can reconstruct from other instrumentation data we have. For example, we could use `is_oversample` from https://meta.wikimedia.org/wiki/Schema:EditAttemptStep, to find edits where `is_oversample` is true, where the `newcomer task` edit tag isn't present, and where the user ID belongs to a user who has suggested edits enabled. That wouldn't be 100% accurate but might be a good place to start.