Page MenuHomePhabricator

Create A Utility For Informal Words Check
Closed, ResolvedPublic8 Estimated Story Points

Description

Once we have a set of word tokens identified in the wikitext, we want to know if any of them are informal words. This will allow us to build the credibility signals related to informal words.

Implementation is mostly regex pattern matching.

Screen Shot 2022-01-19 at 3.52.45 PM.png (1×1 px, 175 KB)

Implementation details:
[1] This utility will live under structured-data/packages.

[2] is_informal_word(word_token) -> True/ False
We return true if the word token matches any informal word pattern in any language.
For this, you will need to copy/paste the list of informal patterns for each language. Refer to informal patterns for english in the link. Similarly, browse through the directory for other languages.

Event Timeline

Anribolon changed the task status from Open to In Progress.Mar 11 2022, 8:57 PM
Anribolon claimed this task.
Lena.Milenko changed the task status from In Progress to Open.Mar 17 2022, 12:00 PM
Lena.Milenko changed the task status from Open to In Progress.Mar 21 2022, 2:34 PM
Lena.Milenko changed the task status from In Progress to Open.Apr 13 2022, 9:28 PM