Tue, Oct 15
Fri, Oct 11
I've updated the task details with some high-level reasoning for the access. If it's not evident, I approve of this request as Kevin's manager and the lead for ORES/Scoring Platform.
Thu, Oct 10
Wed, Oct 9
T232855: ORES query with many statistics results in 503 might be an interesting task once we have covered T223788 as it'll show some of the deeper features of revscoring
Thanks for reporting! This is interesting and weird. We'll look into it.
It looks like the "Issues" link for PetScan points to https://github.com/magnusmanske/petscan_rs/issues Maybe this bug should be filed there instead.
I'm not sure this has anything to do with articlequality-modeling. It seems like this is a UI issue with PetScan. So, I'm going to move this to our "Monitoring" column. Let me know if I've missed something.
Tue, Oct 8
OK I just replicated this on ores-worker-04:
After a bunch more digging in the logs, I'm finding some revids that never seem to finish scoring. E.g. https://ores.wmflabs.org/v3/scores/enwiki/559565916/damaging?features never seems to finish.
From IRC while I have been thinking through this:
Thanks Theklan. I looked through https://eu.wikipedia.org/wiki/Lantanoide but my stupid American monolingual eyes couldn't see any clear instances of English or Spanish words. :) Are there a lot of Basque language typos in the article? I'm trying to think through how we'll make the best use of these new features.
Mon, Oct 7
@Theklan, do you think it's likely we'll find examples of articles that are long enough to be high quality but contain English/Spanish language and are thus lower quality in our labeled data?
Yes. Just a heads up. We might come back to you for advice when we can get this prioritized. Thanks!
Sun, Oct 6
Fri, Oct 4
As of right now, revscoring.datasources.session_oriented is complete and works as expected. revscoring.features.bytes.session is complete and works as expected. I'm hoping to have what exists now reviewed before I move forward with applying list_of_tree() to everything else.
I'm way overdue in updating this task.
Thu, Oct 3
Wed, Oct 2
@Groceryheist helped formulate this idea. He's got a process to develop "debiased" thresholds for ORES filters.
@ACraze, could you let us know what bits are ready for review right now? Alternatively, we could have @EvanProdromou review the higher level API description (see P8830) in the meantime if that still reflects what was implemented.
I talked to @EvanProdromou about getting some review for this. He said that he could do some design review and maybe someone from his team could do the code review for us. As I understand it, we're very close to having a complete patchset for review. We're reaching out now because it's going to be a *big* review so we want to get it on Core Platform Team's radar.
@Volker_E, I see we have a patch merged. Is this done?
Tue, Oct 1
This is live. I'm hoping to make this one of the intro tasks for our new engineer, @kevinbazira. He's still just getting his accounts and access together though.
I wonder if there is a place that patrollers in jawiki gather and discuss issues of counter-vandalism. That seems like a good place to reach out.
Thanks @Joe and no worries. I'm happy to move this one off our backlog :) Thank you both for getting it done.
Mon, Sep 30
I think we can resolve this one as we got a couple of publications out the door. See https://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/viewPaper/13077 and https://www.aclweb.org/anthology/D17-1213/
Wed, Sep 25
As a first pass, I'd love to just have someone try to use what we have deployed and tell us what it tends to get wrong. The best way to do that is to use the filters in Special:RecentChanges to do patrolling work. If we get some interest, I would like to produce some specific sample data with ORES scores that people can label and give us notes on. From that I think it'll become apparent what types of things we're missing.
@abian, ORES models directly a measure "completeness". However, it turns out that accuracy and consistency strongly correlate to these measures of "completeness" so it also a good and useful proxy measure of "consistency" and "accuracy". I'd like to know when and where that breaks down so that we can model that better but in the meantime, I think it is good and useful for measuring "quality" as a general concept. Consuming any measurements comes with caveats and I think this is a good one to highlight so thank you for raising it.
Wait a moment. Is this just due to summer time? We should expect to see an increase in vandalism in September and a drop in May. This is common. It appears and seems to happen every year. I think it is related to students in the west going back to school/leaving on summer vacation. There's a lot of vandalism of the form "<school official of some sort> is <derogatory term>" that just doesn't happen in the summer.
Thank you for the update. Do you have a link to your analysis work?
@akosiaris, do you have experience with trimming the aof files and limiting their size? I'm starting to realize that I'm getting in deep here and a big reason we chose a redis backend is "ops has experience with redis". So I thought maybe you would have an idea.
Looks like we already have:
We probably need to make some changes here: https://github.com/wikimedia/puppet/blob/b347052863d4d2e87b37d6c2d9f44f833cfd9dc2/modules/role/manifests/labs/ores/redis.pp
It looks like we can fix this issue in redis config. Why is this only a problem now? Who knows.
Mon, Sep 23
Fri, Sep 20
Thanks @Keegan and no worries. We'll be ready to go when you are. There's always other work to do in the meantime ^_^
@kostajh, it looks like we missed this one in the last round of updates. I expect this one will be as straightforward as the rest of the updates.
Thu, Sep 19
Awesome. Merged. Looks like we're good here.
Sep 18 2019
it is risky to rely on another team to build models for our target wikis.
Re. using the ORES predictions, you probably don't want to use the "prediction" field directly. I would suggest instead pulling in any predicted class that is above 0.05 probability. This seems to work really well.