Page MenuHomePhabricator
Create Task
Maniphest T190689

FY17/18 Q4 Program 7 Services Goal: Language variants support
Closed, ResolvedPublic
Actions

Description

Please do not insert Chinese-Sites to this task, there's nothing that more or less related to those wikis.

The FY17/18 Q4 goal, of the Services team centres around providing support for language variants for page content in the REST API. Concretely, the goal consists of:

  • resolving the on-going disk I/O performance issues
  • incorporating the use case for language variants into the storage design
  • exposing the language variants for Parsoid HTML and related content to clients via the REST API

Details

SubjectRepoBranchLines +/-
Config: Skip schema updatesmediawiki/services/restbase/deploymaster+1 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 StalledNoneT324931 Clean up open RESTBase related tickets
 In ProgressNoneT262315 <CORE TECHNOLOGY> API Migration & RESTbase Sunset
 ResolvedDAlangi_WMFT324678 Migrate proton (chromium-render) away from restbase
 OpenNoneT167603 Any Chinese Wiki's projects about "Download as PDF" can not auto change to Simplified Chinese or Traditional Chinese
ResolvedovasilevaT147553 [EPIC] Page previews broken on many projects
 OpenovasilevaT244262 [Epic] Enable page previews and reference previews as a beta feature on all projects
 OpenNoneT111231 Page previews for Wikidata
 InvalidNoneT148854 Use RESTBase for zhwiki
 Resolved• PcheloloT188164 Popups don‘t support language variant conversion and {{lang}} template
 Resolved• mobrovacT190689 FY17/18 Q4 Program 7 Services Goal: Language variants support
 ResolvedEevansT192112 Consider using default JVM G1GC settings in the RESTBase Cassandra cluster
 ResolvedEevansT186751 Restablish RESTBase dev environment with Cassandra 3.11.2
 ResolvedDzahnT185494 Degraded RAID on restbase-dev1006
Unknown Object (Task)
 ResolvedEevansT224260 restbase-dev1006 has a broken disk
Unknown Object (Task)
 ResolvedEevansT192689 Unchecked storage growth
 OpenNoneT197018 Upstream changes to CQLSSTableWriter
 DeclinedNoneT197019 Publish bulk-import revision culling utility
 ResolvedEevansT189057 Understand (and if possible, improve) cluster performance under the new storage strategy
 ResolvedfgiunchediT186562 Reimage JBO-RAID0 configured RESTBase HP machines
 ResolvedEevansT189822 Replace 5 Samsung SSD 850 devices w/ 4 1.6T Intel or HP SSDs
Unknown Object (Task)
 Resolved• PcheloloT159985 Implement language variant support in the REST API
 OpenNoneT43716 [EPIC] Support language variant conversion in Parsoid
 OpenNoneT21044 Document LanguageConverter
 OpenNoneT53587 Parsoid needs to run findVariantLink or some equivalent thing
 Invalid• GWickeT48658 Tpl-style encapsulation for <*include*> and lang-variant conversions
 ResolvedliangentT45547 MediaWiki needs a fictitious variant for English for easier variant development work
 ResolvedthiemowmdeT156280 Wikibase assumes English doesn't have a variant
 OpenNoneT54661 Preprocessor/Parser irregularities with -{...}- variant constructs.
 ResolvedcscottT146304 Preprocessor should handle -{...}- variant constructs in template arguments
 ResolvedcscottT153761 Incorrect parser output if -{{ appears in wikitext
 Resolved• ElitreT165175 Support communications around the preprocessor fixups
 ResolvedcscottT146305 Parser should protect -{...}- variant constructs in links
 ResolvedcscottT54192 Markups in alt param of <gallery> are "eaten" during parsing
 ResolvedcscottT54190 <gallery> with |link=<external link> doesn't work on wikis with LanguageConverter
 ResolvedcscottT153135 doBlockLevels breaks with embedded language converter markup
 ResolvedcscottT153140 -{ ... }- markup breaks tables
 OpenNoneT153265 Language converter source text and language names cannot use <nowiki> escaping.
 DuplicateBUG REPORTNoneT353501 new Parsoid cannot parse the converter wikitext syntax
 ResolvedcscottT153341 Export LanguageConverter enabled status in page info from core
 OpenNoneT204966 Production use of LanguageConverter for read views of Phase 2A languages
 OpenNoneT204968 Production use of LanguageConverter for read views of Phase 2B languages
 OpenNoneT204969 Production use of LanguageConverter for read views of Phase 2C languages
 OpenNoneT222328 [extlink] parsing - link cannot contain language variant or extension tags
 ResolvedBUG REPORTJgiannelosT305383 [BUG] Kazakh Wikipedia Character mapping
 OpenNoneT320733 Support and document how language conversion work with multidirectional wikitext <=> HTML conversion on language-conversion-supported extensions.
 Resolved• mobrovacT195327 Normalise the Accept-Language header for REST API requests
 ResolvedJdforrester-WMFT195948 MCS should respect Accept-Language header
 ResolvedJdforrester-WMFT197792 MCS should proxy Vary and Content-Language headers header it gets from Parsoid
 ResolvedEevansT178905 Upgrade RESTBase cluster to Cassandra release: 3.11.2
 ResolvedEevansT196745 Add Cassandra 3.11.2 package to internal APT repository
 ResolvedEevansT197082 Cassandra schema migrations to add page_language
 Resolved• mobrovacT197789 Schema upgrades to add headers field
 ResolvedcscottT197949 Parsoid dies if trying to transform to not-yet-supported language variant
 Resolved• PcheloloT198186 VisualEditor should explicitly set Accept-Language
 Resolved• PcheloloT198465 Enable language variants support for summary

Event Timeline

• mobrovac triaged this task as High priority.Mar 26 2018, 2:43 PM
• mobrovac created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 26 2018, 2:43 PM
• mobrovac edited projects, added Services (doing); removed Services (next).Apr 12 2018, 9:01 PM
• mobrovac added a subtask: T192112: Consider using default JVM G1GC settings in the RESTBase Cassandra cluster.
• mobrovac added a subtask: T186751: Restablish RESTBase dev environment with Cassandra 3.11.2.
• mobrovac added a subtask: T192689: Unchecked storage growth.Apr 23 2018, 12:24 PM
• mobrovac added a subtask: T189057: Understand (and if possible, improve) cluster performance under the new storage strategy.
• mobrovac added a subtask: T194190: Infinite rerender loop in RESTBase.May 8 2018, 6:19 PM
• mobrovac added parent tasks: T185139: Popups should respect user's language variant preferences, T167603: Any Chinese Wiki's projects about "Download as PDF" can not auto change to Simplified Chinese or Traditional Chinese , T111231: Page previews for Wikidata, T148854: Use RESTBase for zhwiki.May 22 2018, 7:31 PM
• mobrovac added a subtask: T159985: Implement language variant support in the REST API.
Liuxinyu970226 updated the task description. (Show Details)May 23 2018, 9:50 AM
Liuxinyu970226 subscribed.
• mobrovac added a subtask: T178905: Upgrade RESTBase cluster to Cassandra release: 3.11.2.Jun 5 2018, 9:16 AM
• bearND closed subtask T195948: MCS should respect Accept-Language header as Resolved.Jun 7 2018, 4:48 PM
• bearND reopened subtask T195948: MCS should respect Accept-Language header as Open.Jun 7 2018, 5:02 PM
Eevans closed subtask T178905: Upgrade RESTBase cluster to Cassandra release: 3.11.2 as Resolved.Jun 12 2018, 2:21 PM
gerritbot subscribed.Jun 13 2018, 3:47 PM

Change 440079 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Config: Skip schema updates

https://gerrit.wikimedia.org/r/440079

gerritbot added a project: Patch-For-Review.Jun 13 2018, 3:47 PM

Change 440079 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Config: Skip schema updates

https://gerrit.wikimedia.org/r/440079

• Pchelolo closed subtask T197082: Cassandra schema migrations to add page_language as Resolved.Jun 13 2018, 4:54 PM
Eevans closed subtask T186751: Restablish RESTBase dev environment with Cassandra 3.11.2 as Resolved.Jun 20 2018, 2:52 PM
• mobrovac added a parent task: T188164: Popups don‘t support language variant conversion and {{lang}} template.Jun 22 2018, 3:35 PM
cscott closed subtask T197949: Parsoid dies if trying to transform to not-yet-supported language variant as Resolved.Jun 22 2018, 10:04 PM
• mobrovac closed subtask T197789: Schema upgrades to add headers field as Resolved.Jun 25 2018, 1:12 PM
• Vvjjkkii reopened subtask T197949: Parsoid dies if trying to transform to not-yet-supported language variant as Open.Jul 1 2018, 1:02 AM
• Vvjjkkii reopened subtask T197789: Schema upgrades to add headers field as Open.
• Vvjjkkii reopened subtask T197082: Cassandra schema migrations to add page_language as Open.
ssastry closed subtask T197949: Parsoid dies if trying to transform to not-yet-supported language variant as Resolved.Jul 1 2018, 3:35 AM
• mobrovac closed subtask T197082: Cassandra schema migrations to add page_language as Resolved.Jul 1 2018, 10:20 AM
• mobrovac closed subtask T197789: Schema upgrades to add headers field as Resolved.
• Pchelolo closed subtask T198186: VisualEditor should explicitly set Accept-Language as Resolved.Jul 5 2018, 3:36 PM
• Jhernandez mentioned this in T192999: Support language variants in ReadingLists.Jul 6 2018, 11:35 AM
Stashbot subscribed.Jul 9 2018, 9:07 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-09T09:07:27Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@794d6ee]: Enable language variants for HTML T190689

Stashbot added a comment.Jul 9 2018, 9:22 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-09T09:22:58Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@794d6ee]: Enable language variants for HTML T190689 (duration: 15m 32s)

• Jhernandez added a subtask: T197009: MCS should respect Accept-Language header for MW API requests.Jul 16 2018, 2:02 PM
• mobrovac removed a subtask: T197009: MCS should respect Accept-Language header for MW API requests.Jul 16 2018, 2:57 PM
cscott mentioned this in T198970: Epic: Implement SEO improvements suggested by Go Fish Digital.Jul 19 2018, 2:39 PM
• Jhernandez closed subtask T195948: MCS should respect Accept-Language header as Resolved.Jul 20 2018, 10:50 AM
• Pchelolo closed subtask T198465: Enable language variants support for summary as Resolved.Jul 30 2018, 8:00 PM
• Pchelolo closed subtask T159985: Implement language variant support in the REST API as Resolved.
Eevans closed subtask T192689: Unchecked storage growth as Resolved.Aug 1 2018, 7:17 PM
• Pchelolo removed a subtask: T194190: Infinite rerender loop in RESTBase.Sep 7 2018, 5:48 PM
• mobrovac added a project: Platform Team Workboards (Doing).Dec 20 2018, 12:43 PM
• mobrovac closed this task as Resolved.Apr 12 2019, 11:07 PM
• mobrovac claimed this task.
• mobrovac edited projects, added Services (done), Platform Team Workboards (Done with CPT); removed Platform Team Workboards (Doing), Patch-For-Review, Services (doing).

The RESTBase part has been done long ago.

• mobrovac closed subtask T189057: Understand (and if possible, improve) cluster performance under the new storage strategy as Resolved.Apr 29 2019, 7:48 PM
WDoranWMF closed subtask T192112: Consider using default JVM G1GC settings in the RESTBase Cassandra cluster as Resolved.May 2 2019, 2:49 PM
Winston_Sung removed a parent task: T185139: Popups should respect user's language variant preferences.Jan 27 2024, 3:33 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL