(An Untitled Masterwork)
ActivePublic
Actions

Authored by Ladsgroup on Sep 9 2020, 12:53 AM.

Tags

None

Referenced Files

	F32251726: raw.txt
	Sep 9 2020, 12:53 AM

Subscribers

None

	import mwapi
	from revscoring import Model
	import time
	import json
	import random
	from revscoring.extractors.api.extractor import Extractor
	import requests
	import sys
	r = requests.get('https://quarry.wmflabs.org/run/495231/output/0/json')
	val = r.json()
	with open("models/wikidatawiki.item_quality.gradient_boosting_old.model") as f:
	old_model = Model.load(f)

	with open("models/wikidatawiki.item_quality.gradient_boosting_new_data_with_ps.model") as f:
	new_model = Model.load(f)

	with open("models/wikidatawiki.item_quality.gradient_boosting_new_data_without_ps.model") as f:
	new_model_without_ps = Model.load(f)

	extractor = Extractor(mwapi.Session(host="https://www.wikidata.org",
	user_agent="revscoring demo"))
	labeleds = []
	boo = val['rows']
	random.shuffle(boo)
	for i in boo:
	try:
	feature_values_old = list(extractor.extract(i[0], old_model.features))
	feature_values_old = old_model.score(feature_values_old)
	feature_values_new_ps = list(extractor.extract(i[0], new_model.features))
	feature_values_new_ps = new_model.score(feature_values_new_ps)
	feature_values_new_without_ps = list(extractor.extract(i[0], new_model_without_ps.features))
	feature_values_new_without_ps = new_model_without_ps.score(feature_values_new_without_ps)
	except KeyboardInterrupt:
	sys.exit()
	except:
	continue
	predictions = [
	feature_values_old['prediction'],
	feature_values_new_ps['prediction'],
	feature_values_new_without_ps['prediction']
	]
	if len(set(predictions)) == 1:
	#print('The same', set(predictions))
	continue
	print(
	'\|' + '\|'.join([
	'[[https://www.wikidata.org/wiki/Special:PermaLink/{}\|{}]]'.format(i[0], i[0]),
	feature_values_old['prediction'],
	feature_values_new_ps['prediction'],
	feature_values_new_without_ps['prediction'],
	json.dumps({
	'old': feature_values_old['probability'],
	'new_ps': feature_values_new_ps['probability'],
	'new_without_ps': feature_values_new_without_ps['probability']
	})
	])
	)

Event Timeline

Ladsgroup created this paste.Sep 9 2020, 12:53 AM

Ladsgroup mentioned this in T261849: Benchmark old and new model accuracy on new labeled data.

(An Untitled Masterwork)ActivePublicActions

Event Timeline

(An Untitled Masterwork)
ActivePublic
Actions