@jberkel, I'd just noticed that we never set up a labeling campaign for English Wiktionary. By engaging in this labeling work, we can train smarter models for catching vandalism and other damaging edits in Wiktionary. Would you be interested in organizing a group of editors to label edits? For your reference, here's a page on English Wikipedia that I put together to organize the labeling campaign there: https://en.wikipedia.org/wiki/Wikipedia:Labels/Edit_quality

@Halfak yes i'm interested to help with this. what would i need to do?

Awesome! So right now, I think we need a page on en.wiktionary that looks like https://en.wikipedia.org/wiki/Wikipedia:Labels/Edit_quality describing why we're asking people to label edits as "damaging" and "goodfaith". Then I'd ask you to make an announcement and try to recruit some editors to help out. This is worth peoples' time because an investment of a few hours now will result in a substantially decreased workload WRT counter-vandalism later.

I'm happy to help with drafting things. Most wikis just copy/translate from the enwiki page I linked to, but others go much farther.

Ok sounds good. If I understand it correctly you can update labels from within Wiktionary (with a gadget), without leaving the site? What's the difference to just setting entries as "patrolled"?

Hi @jberkel, sorry for the delayed reply. I somehow missed your Q and am checking on things now.

The major difference with simple patrolling is that Wiki Labels captures more nuanced information. We've tried to use the patrolled flag in the past and it seems that patrolled+reverted != damage as well as patrolled+!reverted != !damage. By explicitly labeling edits "damaging"/not and "goodfaith"/"badfaith", we can build models that target those qualities.

@Halfak ok, so it's separate from a user's normal patrolling activity, meaning you work on an older sample instead of labeling recent changes? I think the participation rate could be a lot higher if it is integrated into the "normal" patrolling activity (opt-in, of course).

Anyway, I can give it try and see if I can recruit volunteers.

@jberkel, when looking at recent changes, you are not reviewing a pure random sample of activity. When we're training a model, it doesn't start with any knowledge of the world at all, so we need to give it the most representative picture that we can. The sample you are looking at in Wikilabels is sampled uniformly from exactly a year of activity on wiktionary to make sure that any seasonal effects (e.g. winter holidays, september and the start of school, etc.) are normalized out.

@Halfak ah ok, that makes sense. how many edits need to be labeled? how sensitive is the whole approach to template/code level changes?