Page MenuHomePhabricator

Urdu language utilities
Open, LowestPublic

Description

  • generate TFiDF badword lists
  • review and aggregation of badwords/informal words by native speaker
  • implement revscoring.Language (Language utility)

Event Timeline

ToAruShiroiNeko assigned this task to Ladsgroup.
ToAruShiroiNeko raised the priority of this task from to Medium.
ToAruShiroiNeko updated the task description. (Show Details)
ToAruShiroiNeko added a subscriber: ToAruShiroiNeko.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 3 2015, 6:20 AM
Halfak set Security to None.

These files seem to have wrongly encoded, so we cant see any proper text except meaningless characters.

you need to either download them and open it with notepad (gedit, or anything suitable) or in your browser check for encoding option (probably in view menu) and choose "UTF-8" or "Unicode"

There's nothing wrong with these files regarding encoding.

Halfak added a subscriber: Halfak.Feb 24 2016, 8:12 PM

I fixed the encoding and copied it to the wiki here: https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Word_lists/ur

@MuhammadShuaib, could you take another look? We need you to move words from the "list-generated" to list-badwords(racial slurs, curse words, offensive language) and list-informals(causal talk: e.g., "hello", "haha", "lol", "wat"). Please let me know if you have any questions.

Looks like we need a new contact to help us with this work. Moving it back to the backlog until we can find someone.

Halfak lowered the priority of this task from Medium to Lowest.Jul 5 2016, 3:11 PM

I fixed the encoding and copied it to the wiki here: https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service/Word_lists/ur

@MuhammadShuaib, could you take another look? We need you to move words from the "list-generated" to list-badwords(racial slurs, curse words, offensive language) and list-informals(causal talk: e.g., "hello", "haha", "lol", "wat"). Please let me know if you have any questions.

Bad words have been moved under bad words.

Looks like we need a new contact to help us with this work. Moving it back to the backlog until we can find someone.

I am here @Halfak.

@MuhammadShuaib Thanks. Please tell us when you're done with informal words.

This comment was removed by Ladsgroup.
Restricted Application added a project: artificial-intelligence. · View Herald TranscriptApr 17 2019, 7:16 PM
Aklapper removed ToAruShiroiNeko as the assignee of this task.Jun 19 2020, 4:31 PM

This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!

For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)