We're mostly looking to catch "spam", "vandalism", and other types of problematic drafts that are obvious from the content itself. In other words, you don't need any subject matter expertise or external references to figure out that something is wrong. This is due to limitations in what the model can be expected to figure out.

So copyright is hard because the only way to know for sure is to find where it was copied from.

Would we be likely to find vandalism under ER20 "Unsuitable content"? It seems like ER6 is the only item that covers spam and advertising. Is that right?

Yes, you would tag vandalism with ER20. ER20 may also be used for spam and advertising.

There are some additional standard deletion reasons that can be used for spam or vandalism, available in the dropdown menu:

• Spam / propaganda / proselitismo religioso ou político ([[WP:SPAM|detalhes]])
• Ofensivo / baixo calão
• Disparate / incompreensível / sem contexto

Some follow up questions:

1. How many of the above categories are we targeting?
2. Do we require the model to classify the probable of quality of an article from the draft?
3. Are we only trying to weed out vandalism and other unwanted content through this model?

Clearly defining the outcomes would help accelerate the task

@GoEThe @Halfak

@Chtnnh, it looks like we want to target a few of these deletion reasons. Essentially, we want to product vandalism, spam, and other types of clear nonsense. First, we'll need to find the deletions and try to associate the deletion *reasons* from the log of deleted articles.

Understood. That will be our training and testing data and then upon satisfactory performance we can try deploying the model, am I right?

Sounds like a plan.

From https://quarry.wmflabs.org/query/43261 we can see that we're able to track about 19k deleted articles from the past year. About 4.2k of them have a speedy deletion reason (Has "WP:ER#"). 1582 were deleted with ER20 ("unsuitable") and 770 were deleted with ER6 ("spam"). Do you think it is safe for us to assume that any deletion without either ER20 or ER6 is probably not spam or vandalism?

@Chtnnh, grab the text dataset from /home/halfak/projects/draftquality/datasets/ptwiki.draft_quality.balanced_3k.with_text.json.bz2

https://github.com/wikimedia/draftquality/pull/36

Model has been built 😄

In T246667#5999648, @Halfak wrote:

From https://quarry.wmflabs.org/query/43261 we can see that we're able to track about 19k deleted articles from the past year. About 4.2k of them have a speedy deletion reason (Has "WP:ER#"). 1582 were deleted with ER20 ("unsuitable") and 770 were deleted with ER6 ("spam"). Do you think it is safe for us to assume that any deletion without either ER20 or ER6 is probably not spam or vandalism?

If I'm not mistaken, there are also ~700 SPAM articles which were deleted under the following reason mentioned by Goethe in T246667#5940917:

Spam / propaganda / proselitismo religioso ou político

Here is an updated version of @Halfak 's script, css, and loader code:

• Main script: https://meta.wikimedia.org/w/index.php?title=User:He7d3r/Tools/DraftAndArticleQualityCore.js&type=revision&diff=20008347&oldid=20006169
• CSS: https://meta.wikimedia.org/w/index.php?title=User:He7d3r/Tools/DraftAndArticleQuality.css&type=revision&diff=20008333&oldid=20008331
• Loader: https://meta.wikimedia.org/w/index.php?title=User:He7d3r/Tools/DraftAndArticleQuality.js

With the script enabled, the page Special:NewPages (as suggested at T250809#6075300) looks like this for now:

@GoEThe : I see you've installed the version of the script I mentioned at T246667#6079484. Did you have the chance to test it on Special:Newpages? Is it good enough for us to publicize it for other users?

Apparently, you are the only one who installed it so far:
https://global-search.toolforge.org/?q=%22DraftAndArticleQuality.js%22&namespaces=2%2C8
while there are many other users testing the original version:
https://pt.wikipedia.org/w/index.php?sort=relevance&search=insource%3A%22EpochFail%2FArticleQuality.js%22&title=Especial:Pesquisar&profile=advanced&fulltext=1&advancedSearch-current=%7B%7D&ns2=1&ns8=1

@He7d3r , I tried it briefly. I think the script works fine. The symbols are not immediately intuitive, but I guess for this phase that is not what is important. I think we can publicize it yes, and see what they think about the new draftquality model.

@GoEThe: in case you have any suggestions on better images for this purpose, we can try changing them. @Halfak suggested the https://commons.wikimedia.org/wiki/Category:OOUI_icons as a good source of icons we could use.

@He7d3r , perhaps https://commons.wikimedia.org/wiki/File:OOjs_UI_icon_alert-warning.svg and https://commons.wikimedia.org/wiki/File:OOjs_UI_icon_block-destructive.svg ?

I was thinking... Would it be better to print the ORES prediction on the page somewhere so that it is easier for people to copy and paste that for misclassification reports?