How do sentences from Spam, Vandalism and Attack articles score in the FA sentence model? How do FA sentences score in the Spam, Vandalism and Attack models?
This task is done when a basic analysis shows how well the models are able to differentiate sentences trained from the 4 grammars.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Halfak | T148038 [Epic] Build draft quality model (spam, vandalism, attack, or OK) | |||
| Open | None | T144636 [Epic] Implement PCFG features for editquality and draftquality | |||
| Resolved | Halfak | T151819 Analyze differentiation of FA, Spam, Vandalism, and Attack models/sentences. | |||
| Resolved | Halfak | T148037 Generate PCFG sentence models | |||
| Resolved | Halfak | T148034 Sentence bank for vandalism | |||
| Resolved | Halfak | T148035 Sentence bank for personal attacks | |||
| Resolved | Halfak | T148033 Sentence bank for Featured Articles | |||
| Resolved | Halfak | T148032 Sentence bank for spam | |||
| Resolved | Halfak | T148867 Implement sentences datascources |
I have some progress to show here, but I found an issue in the spacy parser. So I'm working on that first.
TL;DR: We don't differentiate that well when it comes to vandalism but we do a pretty good job for spam and attack sentences. Looking at the dataset qualitatively. I think that we'll want to look at the data qualitatively next. E.g. what are the most vandalismy sentences in :en:Biology? Or what are the most FA sentences in some of the deleted articles?
OK all done for today. It looks like we're doing a good job of differentiating sentences. I think we're just about ready to start experimenting with live data.
It might even be time to start experimenting with editquality, articlequality, and draftquality.