Page MenuHomePhabricator
Create Task
Maniphest T162620

Add language support for Bengali
Closed, ResolvedPublic
Actions

Description

Related Objects
Search...

Event Timeline

Aftabuzzaman created this task.Apr 10 2017, 5:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 10 2017, 5:22 PM
Halfak assigned this task to Ladsgroup.Apr 12 2017, 10:41 PM
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptApr 12 2017, 10:41 PM
Halfak edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.Apr 13 2017, 2:37 PM
Ladsgroup added a comment.Apr 14 2017, 12:00 AM

Started the bot to analyze Bengali Wikipedia:

tools.dexbot@tools-bastion-03:~/pywikibot-core$ jsub -once -N bn_bwds -mem 7g -l release=trusty /data/project/dexbot/pywikibot-core/p3_2/bin/python /data/project/dexbot/pywikibot-core/pwb.py /data/project/dexbot/pywikibot-core/scripts/dump_based_detection_beta.py /public/dumps/public/bnwiki/20170301/bnwiki-20170301-pages-meta-history.xml.bz2 
Your job 3887310 ("bn_bwds") has been submitted

It will be in https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Word_lists/bn in several hours to review.

Ladsgroup moved this task from Incoming to Blocked on others on the User-Ladsgroup board.Apr 14 2017, 12:01 AM
Halfak subscribed.Apr 20 2017, 2:36 PM

Hey @Aftabuzzaman, it looks like our proposed lists are ready. https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Word_lists/bn

See https://meta.wikimedia.org/wiki/Objective_Revision_Evaluation_Service/BWDS_review for instructions about what we need you to do next.

Aftabuzzaman added a comment.Apr 22 2017, 4:14 PM

@Halfak done

Bodhisattwa subscribed.Apr 28 2017, 3:18 PM
Ladsgroup added a comment.Apr 29 2017, 9:34 AM

https://github.com/wiki-ai/revscoring/pull/309

The badword detection doesn't work the way I expect.

Halfak moved this task from Parked to Review on the Machine-Learning-Team (Active Tasks) board.May 1 2017, 4:57 PM
Halfak added a comment.May 8 2017, 4:57 PM

I think we need to work on https://github.com/halfak/deltas/blob/master/deltas/tokenizers/wikitext_split.py#L53 in order to fix this.

Halfak created subtask T164767: Fix bengali tokenization in deltas.May 8 2017, 4:58 PM
Halfak moved this task from Review to Completed on the Machine-Learning-Team (Active Tasks) board.May 8 2017, 7:12 PM
Halfak moved this task from Completed to Review on the Machine-Learning-Team (Active Tasks) board.

This has been fixed in deltas. Please update deltas dependency to 0.4.6 and try again.

Peachey88 reopened subtask T164767: Fix bengali tokenization in deltas as Open.May 9 2017, 10:12 AM
Halfak moved this task from Review to Completed on the Machine-Learning-Team (Active Tasks) board.May 9 2017, 6:32 PM
Restricted Application added a project: artificial-intelligence. · View Herald TranscriptJun 3 2017, 3:46 AM
Halfak mentioned this in Blog Post: Status update (June 3rd, 2017).Jun 3 2017, 8:24 PM
Halfak closed this task as Resolved.Jun 5 2017, 5:07 PM
Halfak closed subtask T164767: Fix bengali tokenization in deltas as Resolved.
Halfak added a parent task: T170490: Train reverted model for Bengali Wikipedia.Jul 12 2017, 9:41 PM
Bodhisattwa moved this task from Backlog to Closed on the Bengali-Sites board.Nov 19 2020, 4:06 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL