Page MenuHomePhabricator

[EPIC] Support language variant conversion in Parsoid
Open, In Progress, LowPublic

Description

This is the top-level tracker bug for LanguageConverter support in Parsoid.

Plan of record, roughly:

  • Phase 1: Parse all LC constructs into DOM (and round-trip them).

    This is sufficient to allow VE to edit LC wikis in same fashion as wikitext editor, w/ mix of variants displayed during editing.
  • Phase 3 (speculative): Use selective serialization to allow VE to operate on the converted text.

    This allows "single variant" editing, without the chaotic mix of variants shown in wikitext editing, and uses selective serialization to preserve the original variant of unedited text.
  • Phase 4 (speculative): Introduce new LC syntax or Glossary features which are a better match for future plans.

    This would avoid the "from this point forward" behavior of LC rules, which complicates incremental update, as well as avoiding the use of templates as a workaround for per-page glossaries. We might also introduce more pervasive language tagging in the source, to better match LC uses where character set can't be used to distinguish variant (toy example: pig latin -vs- english).

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
ResolvedJdforrester-WMF
Resolvedcscott
InvalidNone
DuplicateNone
ResolvedMarkTraceur
ResolvedJdlrobson
Resolved Pchelolo
ResolvedJdlrobson
DeclinedNone
OpenNone
OpenNone
In ProgressNone
ResolvedDiskdance
Resolvedcscott
Invalid GWicke
Resolvedliangent
OpenNone
DuplicateBUG REPORTNone
Resolvedcscott
OpenNone
OpenNone
ResolvedBUG REPORTJgiannelos
OpenNone
DuplicateBUG REPORTNone
ResolvedBUG REPORTcscott
ResolvedBUG REPORTNone
Resolvedcscott

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ssastry raised the priority of this task from Low to Needs Triage.Sep 20 2018, 4:01 PM
ssastry triaged this task as High priority.

In my opinion, it would be the best to dump the whole LanguageConverter -{ }- markup, which is used to define specific variant translation for one term, and used the data from Wikidata instead. Wikidata can store language variant info and can be used across all wikimedia project, rather than volunteers maintaining the same CGroup across multiple project manually.

LGoto lowered the priority of this task from High to Medium.Mar 13 2020, 4:19 PM
LGoto moved this task from Missing Functionality to Future Ideas on the Parsoid board.

In my opinion, it would be the best to dump the whole LanguageConverter -{ }- markup, which is used to define specific variant translation for one term, and used the data from Wikidata instead. Wikidata can store language variant info and can be used across all wikimedia project, rather than volunteers maintaining the same CGroup across multiple project manually.

See also the Glossary RFC (T484). Unfortunately glossaries tend to be topic-specific --- the dictionary you'd use for a pop culture article about movies may not be appropriate for a science article -- but the glossary could certainly source the variants from wikidata. It would be useful to be able to reference glossaries in a global manner as well, so that pages in zh in places other than zhwiki (ie, commons or mediawiki.org or wikimania.org etc) can use the constructed glossaries.

ssastry renamed this task from Support language variant conversion in Parsoid to [EPIC] Support language variant conversion in Parsoid.Jul 15 2020, 5:55 PM
ssastry added a project: Parsoid-Rendering.

@cscott: Hi, I'm resetting the task assignee due to inactivity. Please feel free to reclaim this task if you plan to work on this - it would be welcome! Also see https://www.mediawiki.org/wiki/Bug_management/Assignee_cleanup for more information - thanks!

This ticket is important for the openZIM/Kiwix community and in particular its Chinese audience, see https://github.com/openzim/mwoffliner/issues/840

MSantos lowered the priority of this task from Medium to Low.Jun 26 2023, 3:16 PM
mikez-WMF changed the task status from Open to In Progress.Feb 23 2026, 11:03 AM
mikez-WMF moved this task from Backlog to In Progress on the Community-Wishlist board.