Maniphest T162617

Use 'informals', 'badwords', etc. in Wikidata feature set
Closed, ResolvedPublic
Actions

Description

Add features like:

matching_lang_badwords
english_lang_badwords
matching_lang_dictwords

Careful to not punish langs that we don't have assets for.

Consider using known alphabet of a lang as a feature.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• johl	T127047 Collection of topics for HPI hackathon
		Resolved		Lydia_Pintscher	T127473 Increase signal of feature set for Wikidata model
		Resolved		Halfak	T171505 Late-July 2017 ORES deploy
		Resolved		Ladsgroup	T162617 Use 'informals', 'badwords', etc. in Wikidata feature set
		Resolved		Ladsgroup	T170834 Add basic bad word check to Wikidata feature set
		Resolved		Ladsgroup	T170835 Add entropy-related and uppercase-related measures to comments

Event Timeline

See https://github.com/wiki-ai/bwds/blob/master/dump_based_detection.py#L71 for known alphabets

Halfak moved this task from Unsorted to New development on the Machine-Learning-Team board.Apr 13 2017, 2:38 PM

Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.May 5 2017, 1:56 PM

Halfak edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.Jun 15 2017, 2:42 PM

Restricted Application added a project: artificial-intelligence. · View Herald TranscriptJun 15 2017, 2:42 PM

Halfak assigned this task to Ladsgroup.Jun 15 2017, 2:42 PM

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJun 15 2017, 2:42 PM

Ladsgroup moved this task from Incoming to In progress on the User-Ladsgroup board.Jul 6 2017, 1:03 PM

• AnotherLadsgroup created subtask T170834: Add basic bad word check to Wikidata feature set.Jul 17 2017, 3:39 PM

• AnotherLadsgroup created subtask T170835: Add entropy-related and uppercase-related measures to comments.Jul 17 2017, 3:43 PM

Should we call this done? or set it back for more signals in later tries?

Halfak closed subtask T170834: Add basic bad word check to Wikidata feature set as Resolved.Jul 20 2017, 7:15 PM

Halfak closed subtask T170835: Add entropy-related and uppercase-related measures to comments as Resolved.

We're not getting badwords signal from labels or descriptions are we?

Yes we are. When people edit Wikidata using GUI, it adds what they changed as edit summary. See https://www.wikidata.org/w/index.php?title=Special:RecentChanges&hidenondamaging=1 for example.

Oh! I see! Your approach is a very interesting solution. I'm OK with calling this done :)

Ladsgroup moved this task from Parked to Completed on the Machine-Learning-Team (Active Tasks) board.Jul 21 2017, 3:09 PM

Restricted Application added a subscriber: PokestarFan. · View Herald TranscriptJul 21 2017, 3:09 PM

Ladsgroup moved this task from In progress to Done on the User-Ladsgroup board.Jul 21 2017, 3:09 PM

Halfak closed this task as Resolved.Jul 24 2017, 3:40 PM

Ladsgroup added a parent task: T171505: Late-July 2017 ORES deploy.Jul 24 2017, 6:20 PM

awight mentioned this in Blog Post: Status update (October 6, 2017).Oct 6 2017, 12:18 AM