Page MenuHomePhabricator

Investigate whether blocks have increased since August
Closed, ResolvedPublic

Assigned To
Authored By
kzimmerman
Dec 23 2021, 9:39 PM
Referenced Files
F34952857: image.png
Feb 15 2022, 11:25 PM
F34952837: image.png
Feb 15 2022, 10:53 PM
F34940916: image.png
Feb 2 2022, 10:18 PM
F34940904: image.png
Feb 2 2022, 10:18 PM
F34940908: image.png
Feb 2 2022, 10:18 PM
F34940906: image.png
Feb 2 2022, 10:18 PM
F34940914: image.png
Feb 2 2022, 10:18 PM
F34940929: image.png
Feb 2 2022, 10:18 PM

Description

Name for main point of contact and contact preference
Kate Zimmerman, phab

What teams or departments is this for?
Product

What are your goals? How will you use this data or analysis?
to understand whether the decline in internal referrers is a result of a Product/Tech release, community actions, or something else, so we can adjust accordingly

What are the details of your request? Include relevant timelines or deadlines
@nettrom_WMF hypothesized that increased blocking of VPNs could decrease internal referrers:

When thinking about this, I did wonder if community members have been focusing more on blocking VPNs, IPs used for pooling users, and things like that. See for example this thread on Reddit about blocks in Finland , and in the discussion on meta about the relay there's a thread about of railroad travellers in Europe. I’m not sure what the extent of global blocks is, whether it’s increased or decreased over the past few years, and whether it would affect our numbers.

The declines in internal referrers are most obvious looking at changes from October to November, but it's possible the shifts started between August and September.

Can you see whether there are increases in blocks that would impact large populations of internet users?

Is this request urgent or time sensitive?
Time sensitive, as conversations about this will continue in early January.

Event Timeline

kzimmerman triaged this task as High priority.

@jwang [in the new year] let me know if you have any dashboards that we could look at before the January 5 discussion

@kzimmerman , we have trend of blocks on IP masking dashboard. On enwiki, it did surged since August. Meanwhile on eswiki, blocks surged since November.

image.png (1,292×764 px, 139 KB)

@jwang those look like huge changes! Do you have a sense of how something like this might impact our top-level metrics (pageviews or edits?)

@kzimmerman , in theory, blocks will directly impact edits top-level metric. It might not be noticeable if blocks are only applied on small portion of total users.
It possibly impacts new account creation from the blocked IP. But user still can create account by emailing admin (ref: Global block FAQ)
It is mentioned on Global IP block that "Global blocks will remove talk page access when the block is encountered". If the access means pageview access, then it might impact the pageviews on talk page. I checked trend on enwiki from en6C. Pageviews on talk page is not declining 21/22 Q1 and Q2. ( superset link )

image.png (1,520×946 px, 100 KB)

Documents about block

Discussion about blocking on enwiki is ongoing. Re-open this ticket to document findings.

It is mentioned on Global IP block that "Global blocks will remove talk page access when the block is encountered". If the access means pageview access, then it might impact the pageviews on talk page.

Confirmed that the access here means editing access. Global blocks won't impact pageview access. (source: slack)

@DannyH confirmed with enwiki community that the uptick in blocks is due to P2P proxy blocks, done by a bot called ST47ProxyBot. They're copied over to Spanish too. There's an announcement and discussion of the bot's activity.

We want to understand whether the blocks have any impact on our core metrics. We looked into the metrics related to the admin actions and editor actions.

Admin actions

Usually, the admin use flow of vandalism is : revert edits --> protect pages --> block user/IP. We have general metrics of revert edits, page protections, blocks on IP masking dashboard. Looking at these metrics on enwiki, they are not alarming at the first 2 stages until reaching block stage.

  1. Reverted Edits on enwiki 2021 vs 2020 vs 2019

image.png (1,104×808 px, 77 KB)
,

  1. Protected pages on enwiki 2021vs 2020 vs 2019

image.png (1,356×844 px, 81 KB)
,

  1. Block actions on enwiki 2021 vs 2020

image.png (1,306×808 px, 76 KB)

Edits metrics

The edit use flow in vandalism is: edit -> edit reverted-> blocked -> cannot edit. @Iflorez and I looked into edits, revert rate, and active editors on enwiki.

  1. Edits on enwiki 2021 vs 2020 vs 2019 , grouped by user_is_anonymous. Edits by bots are excluded.

Edits from anonymous editors decreased YoYoY. Edits from non-bot logged-in user increased comparing to 2019 but is lower than 2020. superset link

image.png (1,412×832 px, 104 KB)

  1. Revert rate on enwiki 2021 vs 2020 vs 2019 . Revert rate= reverted edits/total edits, edits by bot are excluded.

Revert rate dropped from 10.7% in July to 8.8% in December, 2021. superset link.

image.png (1,560×1,156 px, 168 KB)

  1. Active editor on enwiki 2021 vs 2020 vs 2019

Active editor of 2021 is comparable to 2020, and higher than 2019. superset link

image.png (1,572×1,144 px, 149 KB)

  1. New active editors on enwiki 2021 vs 2020 vs 2019

Although active editors do not show a decrease, the new active editors on enwiki decreased significantly compared to 2020 and 2019. superset

image.png (1,454×958 px, 121 KB)

@jwang can you please mention which wikis did you look at for each of the graphs?

@jwang can you please mention which wikis did you look at for each of the graphs?

They are all enwiki.

Note for further investigations (if any) around revert rate: we should limit the revert window to a set time period, as edits continue to be reverted over time. The baselines will look worse because more time has passed since the edit was made, so there has been a larger window of opportunity to revert the edit.

This work has been done for some time - thank you @jwang ! There will likely be follow-ups that we'll file as separate tasks.