Greek language assets
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ToAruShiroiNeko
	Jan 1 2016, 4:36 PM

Description

generate TFiDF badword lists
review and aggregation of badwords/informal words by native speaker
implement revscoring.language.Language (Language utility)

Related Objects
Search...

Status	Assigned	Task
Resolved	Halfak	T166045 Scoring platform team FY18 Q1
Resolved	Catrope	T170723 Deploy ORES Review Tool & ORES-based RCFilters for Romanian & Albanian Wikipedia
Resolved	awight	T170485 ORES deployment - Mid July, 2017
Resolved	Halfak	T170491 Train reverted model for Greek Wikipedia
Resolved	Halfak	T166049 Deploy reverted model for elwiki
Resolved	Halfak	T166050 Train/test reverted model for elwiki
Resolved	Halfak	T122727 Greek language assets

Event Timeline

ToAruShiroiNeko created this task.Jan 1 2016, 4:36 PM

ToAruShiroiNeko raised the priority of this task from to Needs Triage.

ToAruShiroiNeko updated the task description. (Show Details)

ToAruShiroiNeko added a project: Machine-Learning-Team (Active Tasks).

ToAruShiroiNeko moved this task to Backlog on the Machine-Learning-Team (Active Tasks) board.

ToAruShiroiNeko subscribed.

Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptJan 1 2016, 4:36 PM

Halfak moved this task from Backlog to Parked on the Machine-Learning-Team (Active Tasks) board.Apr 3 2016, 8:15 AM

Halfak edited projects, added Machine-Learning-Team; removed Machine-Learning-Team (Active Tasks).Apr 3 2016, 8:15 AM

@ToAruShiroiNeko, did you have a contact from elwiki who can help us push this forward?

Looks like the tfidf box got checked, but I don't see evidence of a Bad-Words-Detection-System run. https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Word_lists/el

Halfak added a project: revscoring.Apr 4 2016, 2:43 PM

Halfak moved this task from Unsorted to Blocked on community input on the Machine-Learning-Team board.

Ladsgroup edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.Jun 30 2016, 12:23 AM

Ladsgroup moved this task from Parked to Backlog on the Machine-Learning-Team (Active Tasks) board.

Ladsgroup edited projects, added Machine-Learning-Team; removed Machine-Learning-Team (Active Tasks).Jul 11 2016, 10:45 AM

Halfak triaged this task as Lowest priority.Jul 28 2016, 2:15 PM

Halfak set Security to None.

Halfak added a parent task: T166050: Train/test reverted model for elwiki.May 22 2017, 12:07 PM

geraki subscribed.May 23 2017, 6:08 AM

I'd like to help with this.
The bot generated list seems to cut words in half, separated by their diacritics: almost all greek words contain diacritics (some people omit them only when they type in touch screen or capital letters)
So the list is currently useless, most strings there are parts of common words.

Hi @geraki, we just had a pull request merged this data. https://github.com/wiki-ai/revscoring/pull/317/files

How does it look?

We can certainly work on the greek diacritics issues. I'll do some work on our tokenizer to handle that right now.

Halfak renamed this task from Greek language utilities to Greek language assets.May 23 2017, 7:00 AM

Halfak edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.

Just checked and it looks like our tokenizer handles it but the elwiki words came from an old run of BWDS. @Ladsgroup, do you think we could re-run BWDS with the most recent version of deltas and then use that to extend what we have for Greek language assets? I'll file a new task.

Halfak mentioned this in T166105: Extend greek language assets.May 23 2017, 7:17 AM

T166105: Extend greek language assets :)

Halfak updated the task description. (Show Details)May 23 2017, 7:21 AM

Halfak removed a project: Machine-Learning-Team (Active Tasks).May 23 2017, 7:48 AM

Restricted Application added a project: Machine-Learning-Team. · View Herald TranscriptMay 23 2017, 7:48 AM

Halfak edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.May 23 2017, 7:48 AM

Halfak moved this task from Backlog to Completed on the Machine-Learning-Team (Active Tasks) board.May 23 2017, 7:51 AM

Halfak mentioned this in Blog Post: Status update (June 3rd, 2017).Jun 3 2017, 8:24 PM

Halfak closed this task as Resolved.Jun 5 2017, 5:07 PM

Halfak claimed this task.

Restricted Application added a project: artificial-intelligence. · View Herald TranscriptJun 5 2017, 5:07 PM

Halfak added a parent task: T170490: Train reverted model for Bengali Wikipedia.Jul 12 2017, 9:41 PM