Redirect in the search box for Arabic projects
Closed, ResolvedPublic
Actions

Description

This task is a split in 3 tasks of a major task T115561 .

Approval :
We voted in Arabic Wikipedia for the exclusive use of Western Arabic numerals (WAN) (0, 1, 2 ) instead of allowing the use of Eastern Arabic numerals (EAN) (٠‎ - ١‎ - ٢) when editing.
21 for / none against
https://ar.wikipedia.org/wiki/%D9%88%D9%8A%D9%83%D9%8A%D8%A8%D9%8A%D8%AF%D9%8A%D8%A7:%D8%A7%D9%84%D9%85%D9%8A%D8%AF%D8%A7%D9%86/%D8%B3%D9%8A%D8%A7%D8%B3%D8%A7%D8%AA/10/2015

Task :
Actually, if someone make a search about an article that contains numerals and words but uses EAN, the article in WAN will not appear.
As an example : when we make a search for an indian movie called https://en.wikipedia.org/wiki/Love_Story_2050 in Arabic wikipedia with EAN, and may be the problem is in other sister projects, the article doesn't appear in the search results.
https://ar.wikipedia.org/w/index.php?search=%D9%82%D8%B5%D8%A9+%D8%AD%D8%A8+%D9%A2%D9%A0%D9%A5%D9%A0+%28%D9%81%D9%8A%D9%84%D9%85+%D9%87%D9%86%D8%AF%D9%8A%29&title=%D8%AE%D8%A7%D8%B5%3A%D8%A8%D8%AD%D8%AB&go=%D8%A7%D8%B0%D9%87%D8%A8

Even in the article about the movie really exist but with WAN.
https://ar.wikipedia.org/wiki/%D9%82%D8%B5%D8%A9_%D8%AD%D8%A8_2050_%28%D9%81%D9%8A%D9%84%D9%85_%D9%87%D9%86%D8%AF%D9%8A%29

Details

	Subject	Repo	Branch	Lines +/-
	Fold Eastern Arabic Numerals to Western in the Completion Suggester for Arabic	mediawiki/extensions/CirrusSearch	master	+63 -13

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Invalid		None	T218613 {EPIC} Search: Local Impact. Making making bigger improvements for smaller (or underrepresented) communities
		Resolved		TJones	T117217 Redirect in the search box for Arabic projects

Event Timeline

Helmoony created this task.Oct 30 2015, 3:04 PM

Helmoony raised the priority of this task from to Needs Triage.

Helmoony updated the task description. (Show Details)

Helmoony added a project: MediaWiki-Internationalization.

Helmoony subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 30 2015, 3:04 PM

Helmoony mentioned this in T115561: Set Western Arabic numerals on Arabic Wikipedia.Oct 30 2015, 3:09 PM

Meno25 subscribed.Oct 31 2015, 9:25 AM

something new ?

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 20 2015, 12:26 AM

In T117217#1819508, @Helmoony wrote:

something new ?

If there were news they were listed here. This task is in under "MediaWiki-Internationalization (Backlog)" and not prioritized. Anybody is welcome to provide patches to speed up fixing.

• FShbib subscribed.Mar 26 2016, 1:02 PM

Helmoony mentioned this in T148603: Limit the Quality version of the flagged revision in Arabic Wikipedia to ns=0.Oct 19 2016, 3:58 AM

Mohamed_Ahmed_AF subscribed.Jan 17 2018, 6:55 PM

Restricted Application added a subscriber: alaa. · View Herald TranscriptApr 23 2018, 1:00 PM

Helmoony added a project: Wikimania-Hackathon-2018.Jul 18 2018, 8:19 AM

Amire80 added projects: I18n, MediaWiki-Search.Jul 18 2018, 9:07 AM

Restricted Application added a project: Discovery-Search. · View Herald TranscriptJul 18 2018, 9:07 AM

Amire80 moved this task from Untriaged to Search on the I18n board.Jul 18 2018, 9:08 AM

Maybe @TJones has some insight?

• EBjune moved this task from needs triage to watching / waiting on the Discovery-Search board.Jul 26 2018, 5:29 PM

I think the primary part of this is fixed—the example query works now—probably as the result of an update from Elasticsearch that included the automatic transformation. I checked my local installation with Arabic configured, and if I put in the text 2050 ٢٠٥٠, I get back two tokens: 2050 and 2050 again. Same for 1984 and ١٩٨٤.

Searching with quotes only matches the exact characters you typed, so searching for "قصة حب ٢٠٥٠" gets no results because it is looking for exactly that string. Searching without quotes works fine. I think that's probably the desired behavior, because it lets you readily find EAN characters in the text.

The only other place where this isn't happening is the completion suggester (which provides suggestions as you type in the search box). It's possible to equate the characters there, too, for all Arabic language wikis. (We've done something similar for Russian.) I can do that if it would be useful there, too.

@TJones , yes please the completition suggester would be usefull in Arabic projects. Thank you.

What we did now is to create redirects by bot from eastern to western for articles related to years (https://ar.wikipedia.org/w/index.php?title=%D9%A2%D9%A0%D9%A5%D9%A0&redirect=no), but probably we need to do it for all articles with wan.

I'll add updating the completion suggester for Arabic to my list. I may not get to it for a while.

I don't think you will need to add more redirects. The internal representation of the numbers is WAN, for articles, titles, and queries. So searching for ٥5٥ finds 555, even without a redirect for "٥5٥".

Once the change is made for the completion suggester, ٥5٥ will match both 555 and ٥٥٥ for suggestions.

TJones claimed this task.Jul 26 2018, 7:19 PM

TJones moved this task from watching / waiting to Language Stuff on the Discovery-Search board.Jan 29 2019, 7:30 PM

TJones edited projects, added Discovery-Search (Current work); removed Discovery-Search.Apr 18 2019, 12:59 PM

TJones triaged this task as Medium priority.Apr 26 2019, 5:29 PM

TJones updated the task description. (Show Details)May 6 2019, 5:03 PM

Change 510953 had a related patch set uploaded (by Tjones; owner: Tjones):
[mediawiki/extensions/CirrusSearch@master] Fold Eastern Arabic Numerals to Western in the Completion Suggester for Arabic

https://gerrit.wikimedia.org/r/510953

gerritbot added a project: Patch-For-Review.May 17 2019, 6:00 PM

TJones moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.May 17 2019, 6:02 PM

Change 510953 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Fold Eastern Arabic Numerals to Western in the Completion Suggester for Arabic

https://gerrit.wikimedia.org/r/510953

EBernhardson moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.May 23 2019, 5:11 PM

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.7; 2019-05-28).May 25 2019, 3:01 AM

Ladsgroup removed a project: Patch-For-Review.May 28 2019, 3:37 PM

TJones added a parent task: T218613: {EPIC} Search: Local Impact. Making making bigger improvements for smaller (or underrepresented) communities.May 28 2019, 5:41 PM

debt closed this task as Resolved.May 28 2019, 11:42 PM

Meno25 unsubscribed.May 31 2019, 11:49 AM

Meno25 removed a subscriber: • FShbib.Jan 14 2023, 10:10 PM