In past work we've seen that many labels provided by Wikipedians are mistakes. We'd like to have new labels be applied in some circumstances and possibly a discussion happen about the Right(TM) label.
How would this work? How should we store additional labels? And should there be a way to make any particular label as "primary" like you can for statements in Wikidata. E.g. see the gender property in https://www.wikidata.org/wiki/Q298423
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Invalid | None | T148700 JADE: UI/API for reviewing/refuting how ORES classifies you and your stuff | |||
| Resolved | awight | T153145 Meta ORES: API data storage and querying | |||
| Resolved | Halfak | T166045 Scoring platform team FY18 Q1 | |||
| Resolved | awight | T166053 [Spike] potential technical implementations of JADE | |||
| Resolved | Halfak | T153152 Design JADE data storage schema | |||
| Resolved | awight | T171496 Design a collaborative judgement pattern for JADE | |||
| Resolved | Natalia | T171497 Review training set to check strange examples of labels | |||
| Resolved | Halfak | T178101 Post about judgments/endorsements/preference |
@awight, it seems like our training sets really need a re-review pattern. @Natalia's analysis shows that some of the labels are pretty dubious. I think this has implications for meta-ORES. For example, we could support a consensus-based "primary" label like Wikidata supports a consensus based "preferred rank" statement for a claim.
See the Chelsea Manning's "sex or gender" example in the description.
It looks like we should adopt both "preferred" rank and also "deprecated", plus "normal" cos it will be implicit regardless. This would be consistent with how Wikidata uses ranks, for example when an author posts an updated judgement, any conflicting scale scores are marked deprecated. https://www.wikidata.org/wiki/Help:Ranking
This looks good. For the first iteration, we're providing: