Page MenuHomePhabricator

Data analysis request to identify patterns and weights for user account reputation score
Closed, ResolvedPublic

Description

In this task, we want to find answers to questions like the following:

  1. Given a set of blocked users on a wiki (e.g. enwiki in the month of July 2024), using publicly available data, are there patterns that these accounts have in common?
    1. Can we further segment these users based on block reason (e.g. vandalism, spam, long term abuse, sock puppetry, etc) and see different common patterns in public data points known about these accounts?
    2. Do the distributions change if we look at permanently blocked versus temporarily blocked accounts?
    3. Hypothetical example:
      • We might find that of permanently blocked accounts in July 2024, a high percentage do not have a confirmed email address and a high percentage received no Thanks from any users, and the users have a high ratio of reverted to unreverted edits.
  2. Given a set of data points that are associated with blocked accounts (e.g. no confirmed email, no Thanks, high ratio of reverts to unreverted edits), can we say anything about the relative importance of each one of those data points, to know how to weight them in calculating an overall reputation score for an account?
    1. Given a data point, does its importance change if we look at it over time or in combination with other data points, e.g. number of Thanks received doesn't indicate anything on its own, but combined with age of account and number of edits, it is a strong signal about an account activity in good or bad faith
    2. Hypothetical example:
      • We might find that having < N number of Thanks for accounts with more than X edits and older than Y days is associated with all permanently blocked accounts in the dataset, and based on this, we would say that number of Thanks should be weighted by Z% when calculating an overall reputation score

Background

  • We are interested to be able to provide an account reputation score to present to functionaries/administrators/moderators on the wikis to assist in anti-abuse work.
  • The project overview page is located here
  • The project overview contains a table with data points we have identified as being of interest. There may be other data points to look at; this list is not exhaustive.
    • The primary goal of the reputation score is to make it easier for humans to quickly identify accounts that may need further review by surfacing a score based on various data points known about the account.
    • A secondary, longer-term goal is to make this score available to software (like AbuseFilter) where it could be used for mitigations
  • We are primarily interested in accounts that are < 90 days old, in the assumption that accounts with persistent bad faith activity older than that will have already been blocked.
  • The algorithm to work out a reputation score needs to be "good enough" and not perfect. We should be able to help patrollers and functionaries identify accounts that are likely problematic while their edits are being reviewed, and help decrease the amount of time that patrollers/functionaries need to spend manually building a mental model of an account's reputation
  • The reputation score should be transparent in that those with access to it should be able to understand how various components contributed to an overall score
  • Looking at the score in three tiers ("low risk", "neutral", "high risk"), we are most interested in the "high risk" category

Related tasks

Event Timeline

Update:

  • Ran query and collected initial sample of blocked users on English Wikipedia for the month of July 2024.
    • Limited dataset to users that created accounts less than 90 days prior to the time they were blocked.
    • Included data on block reason and block duration, which can be used to segment different user groups during data exploration.
  • Began exploratory analysis to understand the distribution of accounts across different data points. I'm starting with editing data such as number of edits, articles created, and reverts. I will share the initial results and propose some additional data points that I believe would be useful to explore next week.

@kostajh - Are there any specific data points from the project overview doc you'd like me to prioritize for the exploratory analysis?

  • Also identified a possible approach to determine the relevant importance of different variables on block rates, which relies on random forest (a classification algorithm) and logistic regression models. Inspired by work completed in T356765 to understand variables that impact the deletion rate of content translated articles. I'll learn more about what types of approaches might be needed based on the patterns discovered during the exploratory analysis.

Update:

  • Ran query and collected initial sample of blocked users on English Wikipedia for the month of July 2024.
    • Limited dataset to users that created accounts less than 90 days prior to the time they were blocked.
    • Included data on block reason and block duration, which can be used to segment different user groups during data exploration.

Sounds good! One thing I was thinking about: it would be interesting to look at users who were blocked temporarily, and then later issued a permanent block.

  • Began exploratory analysis to understand the distribution of accounts across different data points. I'm starting with editing data such as number of edits, articles created, and reverts. I will share the initial results and propose some additional data points that I believe would be useful to explore next week.

Sounds good.

@kostajh - Are there any specific data points from the project overview doc you'd like me to prioritize for the exploratory analysis?

Sorting by the "How useful it is column", and choosing just a few, I'd suggest:

  • Number of thanks a user has received
  • Reverted edit count and/or ratio of unreverted/reverted edits
  • If the user has a confirmed email
  • Also identified a possible approach to determine the relevant importance of different variables on block rates, which relies on random forest (a classification algorithm) and logistic regression models. Inspired by work completed in T356765 to understand variables that impact the deletion rate of content translated articles. I'll learn more about what types of approaches might be needed based on the patterns discovered during the exploratory analysis.

👍🏻

@MNeisler I had some discussion with @cwylo, in which she suggested that we consider looking at blocked accounts eswiki. The reason being that administrative activity looks quite different on enwiki than on eswiki. However, for this quarter, I think it would be fine if we restrict the scope to looking at enwiki only, so this is just a note that we might want to consider it for the future.

@MNeisler I had some discussion with @cwylo, in which she suggested that we consider looking at blocked accounts eswiki. The reason being that administrative activity looks quite different on enwiki than on eswiki. However, for this quarter, I think it would be fine if we restrict the scope to looking at enwiki only, so this is just a note that we might want to consider it for the future.

@kostajh Sounds good and I agree it might make sense to compare trends across a couple different wikis.

Here's my progress update for this week:

  • Completed initial exploratory analysis reviewing blocked account distributions across several data points to help identify common patterns. Data points reviewed so far include the number of thanks received, if the user has a confirmed email address, number of articles created, number of reverted edits, and ratio of unreverted to reverted edits.
  • Compiled a summary of initial results. Some highlights:
    • 60% of blocked user accounts do not have a confirmed email address.
    • Only 1% of all reviewed blocked accounts received 1 or more thanks. If a blocked user did receive a thanks, they typically received just one. 75% of all blocked accounts received fewer than 4 thanks total.
    • There are different patterns in the number of reverts depending on if the user was issued a temporary or permanent block. Almost all (96%) of temporary blocked users have made at least one reverted edit compared to 51% of permanently blocked users.
    • For temporary blocked accounts, the median number of reverted edits is 16 edits.
    • There is a median unreverted/reverted ratio of 2.5 across all blocked user accounts. The median unreverted/reverted ratio for temporary accounts is 0.89.
  • Next steps include updating the exploratory analysis to review additional data points and review blocked account patterns by block reason.

Note: I will be OOO next week so no updates for next week. I can resume work on this when I return the week of Sept 30th.

Progress update for this week:

  • Continued work on exploratory analysis to identify common patterns associated with blocked accounts.
  • Updated the exploratory analysis to review blocked account distributions across the following data points:
    • Revert time.
    • Number of edits (by content and non-content edits) .
    • Size of diff in edit.
    • Recently created account and high amount of activity.
    • Check User Reports.
  • Next week: Finalize the exploratory analysis portion of this task. This will include updating to include a few additional data points, reviewing patterns by block reason, and summarizing key data points associated with blocked accounts. I will then share findings for review and comment.

Once we confirm a set of key data points for review, I'll work on running a model to determine the importance of each data point on an account being blocked.

Progress update for this week:

  • Finalized the exploratory analysis to identify common patterns associated with blocked accounts. This included reviewing a few additional data points including (user rights, If the user’s “User:” namespace exists) and reviewing patterns by block reason.
  • The exploratory analysis identified a number of key attributes associated with blocked accounts, which I think are sufficient to begin work on modeling to determine their importance in an account being blocked. More data points can still be added if needed.
  • Key data points and findings are currently summarized in the exploratory analysis report. For several data points, block duration and block reason impact the patterns observed for blocked accounts
  • Next week. I will start work on modeling the data to identify importance of the currently reviewed data points. Based on the exploratory analysis findings, patterns for blocked accounts are impacted by both block duration and block reason for several data points. This will need to be accounted for in the model.

@kostajh - In the meantime, let me know if there are any additional data points you want to make sure are reviewed. I marked all of the reviewed data points identified in the project doc with a green check mark.

Progress update for this week:

  • Finalized the exploratory analysis to identify common patterns associated with blocked accounts. This included reviewing a few additional data points including (user rights, If the user’s “User:” namespace exists) and reviewing patterns by block reason.
  • The exploratory analysis identified a number of key attributes associated with blocked accounts, which I think are sufficient to begin work on modeling to determine their importance in an account being blocked. More data points can still be added if needed.
  • Key data points and findings are currently summarized in the exploratory analysis report. For several data points, block duration and block reason impact the patterns observed for blocked accounts
  • Next week. I will start work on modeling the data to identify importance of the currently reviewed data points. Based on the exploratory analysis findings, patterns for blocked accounts are impacted by both block duration and block reason for several data points. This will need to be accounted for in the model.

@kostajh - In the meantime, let me know if there are any additional data points you want to make sure are reviewed. I marked all of the reviewed data points identified in the project doc with a green check mark.

Sorry for the delay in responding -- perhaps some ones to prioritize would be "Blocks on other wikis", "Current active blocks applied to the user" and "Quick rate of editing from new accounts, especially after a while since account creation (e.g. sleeper account)". "Warnings left on talk page" would be interesting but there's not a good cross wiki way to analyze this.

Progress update

  • Updated the exploratory analysis to include blocks on other wikis, historical blocks, and quick rate of editing from new accounts, especially after a while since account creation (e.g. sleeper account). Based on initial results, quick rate of editing seems strongly associated with blocked accounts.
  • Gathered dataset with all identified key features and completed preprocessing steps needed to run the model.
  • Started work on modeling the data to identify the importance of the currently reviewed data points.

I've completed modeling the data to identify the importance of the currently reviewed data points. Please see details in the draft report for review.

Based on the results, the following data points were identified as having a high impact on determining whether a user was blocked on English Wikipedia by both model outputs.

  • Ratio of reverted edit to all edits completed
  • Maximum edit size
  • Number of edits reverted within 12 hours
  • Has completed a non-content edit
  • Has created a new article
  • Has a user page

The logistic regression model also identified these as high-impact features:

  • Has other wiki blocks
  • Has check user reports
  • Has sleeper account activity

The report includes details on the model probabilities, which can be used to help determine weights to assign each data point.

Let me know if you have any questions or suggestions. Happy to review some of the details on our check-in call this week as well.

cc @kostajh

Code repo

Note: I've made a few updates to the report, including adding text to help clarify trends and removing some of the highly correlated or redundant variables to review impact on the model's output.

There were no significant changes to the key attributes determined to impact the probability of a user being blocked (as summarized in T373478#10385643).

Some follow-up analysis suggestions:

  • Extend analysis to other Wikipedias where the type of user attributes as well as the magnitude and direction of their impact on a user's block status may vary due to different moderation and administrative practices.
  • Consider additional user attributes. The analysis revealed that a quick rate of editing especially if a large portion of those edits were reverted increases the likelihood of a user being blocked. We can explore a few more definitions of quick editing rates to see if we can find a more precise indicator.
MNeisler closed this task as Resolved.EditedJan 28 2025, 3:41 PM

Resolving as the final report for this task is complete and the hypothesis (WE4.2.15) has now been closed.

Please reopen or file a new task if any additional analysis is needed.