Maniphest T22097

TitleKey should drop special characters, like diacritics
Open, MediumPublic
Actions

Assigned To

None

Authored By

	Platonides
	Aug 6 2009, 7:58 PM

Description

+++ This bug is a superset of Bug #19882 +++

TitleKey does a normalization for titles comparing just their uppercase values (Language::caseFold).
However, there're also many redirects done just for making users arrive at the page with the diacritics from an ASCII title.

The steps to take would be like these:
*Create a normalizing function at Language.
*Make TitleKey::normalize use it before caseFolding.
*Add some basic scoring to TitleKey::exactMatch instead of just returning the frst one, since the options are now much larger.
*Add a hook to also plug it into MediaWiki:Noarticletext

Version: unspecified
Severity: normal

Details

Reference: bz20097

	Subject	Repo	Branch	Lines +/-
	Further normalise keys to find titles with special characters	mediawiki/extensions/TitleKey	master	+8 -0
	Further normalise keys to find titles with special characters	mediawiki/extensions/TitleKey	master	+6 -1

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T22097 TitleKey should drop special characters, like diacritics
		Resolved		• brion	T21882 Case insensitivity in the search box does not function for Titles with Mixed Case

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:52 PM

• bzimport added a project: MediaWiki-extensions-TitleKey.

• bzimport set Reference to bz20097.

Platonides created this task.Aug 6 2009, 7:58 PM

Scoring could indeed be interesting... certainly we want a "more exact" match before a "less exact" one, especially for languages where diacritics and such frequently distinguish between minimal pairs.

Update component, assign to maintainer.

_bugzilla wrote:

Some discussion regarding this issue here:

https://www.mediawiki.org/wiki/Thread:Extension_talk:TitleKey/Accent_and_special_characters_independent_search

Removing assignment from some tasks I'm not actively working on. Volunteers welcome, I'm happy to help if pinged!

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 2 2015, 8:04 PM

Change 286580 had a related patch set uploaded (by Luis Felipe Schenone):
Further normalise keys to find titles with special characters

https://gerrit.wikimedia.org/r/286580

gerritbot added a project: Patch-For-Review.May 3 2016, 3:15 AM

Sophivorus subscribed.May 3 2016, 3:16 AM

Rbirmann subscribed.Sep 13 2016, 9:27 PM

Change 286580 abandoned by Sophivorus:
Further normalise keys to find titles with special characters

Reason:
I haven't been able to rebase this, so I'm submitting another change for it.

https://gerrit.wikimedia.org/r/286580

Change 382968 had a related patch set uploaded (by Sophivorus; owner: Sophivorus):
[mediawiki/extensions/TitleKey@master] Further normalise keys to find titles with special characters

https://gerrit.wikimedia.org/r/382968

@brion Hi brion! Many months ago I submitted a patch that fixed this issue, but it went unnoticed. Now I submitted another patch because the previous one couldn't be merged any longer. It doesn't do everything that is requested in the task description, but it does at least part of it. Could you check it, please? Thanks!

Sophivorus claimed this task.Oct 7 2017, 11:37 AM

Sophivorus removed Sophivorus as the assignee of this task.

Another year gone... the lack of normalization contributes to "Illegal mix of collations" errors, which results in user-facing database query error messages due to search term encoding.

Mon Dec 10 23:07:08 UTC 2018    test    wikidb-mw_      TitleKey::exactMatch    localhost       1267    Illegal mix of collations (latin1_bin,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '=' (localhost)    SELECT  page_namespace,page_title  FROM `mw_titlekey`,`mw_page`   WHERE (tk_page = page_id) AND tk_namespace = '0' AND tk_key = 'KINGMAKER ZARZĄDZANIE'  LIMIT 1

Applying @Sophivorus 's patch to the master branch avoids the error.

Anomie unsubscribed.Dec 11 2018, 5:29 PM

Change 382968 abandoned by Sophivorus:
Further normalise keys to find titles with special characters

Reason:
Per Brion

https://gerrit.wikimedia.org/r/382968

Maintenance_bot removed a project: Patch-For-Review.Apr 8 2020, 1:11 PM

TitleKey should drop special characters, like diacriticsOpen, MediumPublicActions

Description

Details

Related ObjectsSearch...

Event Timeline

TitleKey should drop special characters, like diacritics
Open, MediumPublic
Actions

Related Objects
Search...