Page MenuHomePhabricator

Edit quality campaign for es.wikiquote
Closed, ResolvedPublic

Description

Hello. At es.wikiquote we'd love to have ORES support if at all possible. Since I saw that Spanish is already in the list of supported languages, I didn't request a language support campaign. Sorry if that is still necessary here.

By the way "Edit quality (20k sample)" translates to "Calidad de las ediciones (muestra de 20 000 ejemplos)".

  • Confirm translations are ready
  • List of trusted user groups
  • Translate "Edit quality (20k sample)"
  • Run prelabeling script
  • Load revisions into labels.wmflabs.org

Event Timeline

Restricted Application added a project: artificial-intelligence. · View Herald TranscriptOct 9 2017, 12:29 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Zppix moved this task from Backlog to New campaigns on the Wikilabels board.
Halfak triaged this task as Medium priority.Oct 9 2017, 9:06 PM
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 3 2017, 12:33 PM

While I'm here I made reverted model as well:

Model Information:
	 - type: GradientBoosting
	 - version: 0.4.0
	 - params: {'verbose': 0, 'presort': 'auto', 'learning_rate': 0.1, 'max_leaf_nodes': None, 'label_weights': {True: 10}, 'n_estimators': 500, 'min_samples_leaf': 1, 'max_depth': 3, 'min_weight_fraction_leaf': 0.0, 'center': True, 'random_state': None, 'labels': [True, False], 'loss': 'deviance', 'min_samples_split': 2, 'max_features': 'log2', 'warm_start': False, 'scale': True, 'subsample': 1.0, 'population_rates': None, 'init': None}
	Environment:
	 - revscoring_version: '2.0.11'
	 - platform: 'Linux-4.9.0-4-amd64-x86_64-with-debian-9.2'
	 - machine: 'x86_64'
	 - version: '#1 SMP Debian 4.9.51-1 (2017-09-28)'
	 - system: 'Linux'
	 - processor: ''
	 - python_build: ('default', 'Jan 19 2017 14:11:04')
	 - python_compiler: 'GCC 6.3.0 20170118'
	 - python_branch: ''
	 - python_implementation: 'CPython'
	 - python_revision: ''
	 - python_version: '3.5.3'
	 - release: '4.9.0-4-amd64'
	
	Statistics:
	counts (n=11829):
		label        n         ~True    ~False
		-------  -----  ---  -------  --------
		True      1042  -->      913       129
		False    10787  -->     1119      9668
	rates:
		              True    False
		----------  ------  -------
		sample       0.088    0.912
		population   0.09     0.91
	match_rate (micro=0.769, macro=0.5):
		  False    True
		-------  ------
		  0.827   0.173
	filter_rate (micro=0.231, macro=0.5):
		  False    True
		-------  ------
		  0.173   0.827
	recall (micro=0.894, macro=0.886):
		  False    True
		-------  ------
		  0.896   0.876
	!recall (micro=0.878, macro=0.886):
		  False    True
		-------  ------
		  0.876   0.896
	precision (micro=0.939, macro=0.72):
		  False    True
		-------  ------
		  0.987   0.454
	!precision (micro=0.501, macro=0.72):
		  False    True
		-------  ------
		  0.454   0.987
	f1 (micro=0.909, macro=0.769):
		  False    True
		-------  ------
		  0.939   0.598
	!f1 (micro=0.628, macro=0.769):
		  False    True
		-------  ------
		  0.598   0.939
	accuracy (micro=0.894, macro=0.894):
		  False    True
		-------  ------
		  0.894   0.894
	fpr (micro=0.122, macro=0.114):
		  False    True
		-------  ------
		  0.124   0.104
	roc_auc (micro=0.944, macro=0.944):
		  False    True
		-------  ------
		  0.944   0.944
	pr_auc (micro=0.965, macro=0.835):
		  False    True
		-------  ------
		  0.993   0.678
	
	 - score_schema: {'type': 'object', 'title': 'Scikit learn-based classifier score with probability', 'properties': {'prediction': {'description': 'The most likely label predicted by the estimator', 'type': 'bool'}, 'probability': {'description': 'A mapping of probabilities onto each of the potential output labels', 'type': 'object', 'properties': {'false': 'number', 'true': 'number'}}}}

Mentioned in SAL (#wikimedia-cloud) [2017-12-09T17:38:51Z] <Amir1> ladsgroup@wikilabels-01:/srv/wikilabels/config$ sudo -u www-data /srv/wikilabels/venv/bin/wikilabels new_campaign eswikiquote "Editar calidad (20k muestra aleatoria, 2017)" damaging_and_goodfaith DiffToPrevious 1 50 (T177762)

Mentioned in SAL (#wikimedia-cloud) [2017-12-09T17:38:55Z] <Amir1> ladsgroup@wikilabels-01:/srv/wikilabels/config$ less ~/eswikiquote.revisions_for_review.5k_2017.json | sudo -u www-data ../venv/bin/wikilabels task_inserts 64 (T177762)

Ladsgroup moved this task from Incoming to Done on the User-Ladsgroup board.Dec 28 2017, 11:04 PM
Halfak closed this task as Resolved.Jan 30 2018, 8:31 PM