Page MenuHomePhabricator

Disproportionate number of anonymous users in likely damaging results on rowiki
Open, Needs TriagePublic

Description

I'm not sure if this is a bug, a training bias or some other kind of issue, but I believe it warrants an investigation. I'm looking into ORES scores on rowiki and out of ~3300 unpatrolled recent changes, 502 are marked as "likely damaging" (score >= 0.847). Just 6 (1%) of these 502 are from logged-in users.

Extendind the search to all recent changes, From the last 5000, 463 are marked as "likely damaging" (score >= 0.847). Just 10 (2%) of these 463 are from logged-in users.

I'm using this pywikibot code to extract the data:

for p in recentchanges(site, namespaces=[0,4,6,8,10], total=5000, top_only=True, changetype="edit|new|categorize", patrolled=False, reverse=True):
        if type(p['oresscores']) == dict:
                score = p['oresscores']['damaging']['true']
                if score >= 0.847:
                        print(p['user'], score)