Maniphest T198455

Unify WikibaseLexeme vue templates
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Pablo-WMDE
	Jun 29 2018, 8:18 AM

Description

In WikibaseLexeme the javascript tests do not use the same source of truth for vue templates as the production code.
This can lead to issues and incidents if the templates are changed and the tests are not updated.

The full process in production is: PHP code (SSR) spits out a component from a template which is they hydrated by the client side JS. So if the JS tests are not using the same templates as PHP, then bad things can happen.

As a programmer, I want one canonical definition per vue template in order to make tests more realistic and avoid mistakes caused by templates out-of-sync with the code working on them.

Acceptance criteria

only one definition is used per template for both production and testing
templates are shipped in a way that ensures they match the frontend code they were checked in with (may already be done but should be checked)

Info

Extensions involved: Wikibase, WikibaseLexeme

Currently templates can be found in various places, e.g.
- src/Presentation/View/*
- tests/jasmine/* getTemplate()
- tests/qunit/* getRepresentationWidgetTemplate()

A concrete example would be:

ResourceLoader templates could be used to load the templates from files on the client side.

In T198455#5782277, @Jakob_WMDE wrote:

I believe that this is achievable with ResourceLoader HTML templates. We can extract the vue templates into individual files, and then share them between the PHP and JS code. It looks like MediaWiki currently expects this type of template to be mustache templates, but also provides a mechanism to provide alternative formats via custom template compilers.

this ticket probably also allows us to close T197228

Details

Due Date: Oct 22 2020, 10:00 PM

Subject	Repo	Branch	Lines +/-
Revert "Reinstate template script injections"	mediawiki/extensions/WikibaseLexeme	master	+6 -87
Reinstate template script injections	mediawiki/extensions/WikibaseLexeme	master	+87 -6
Replace script injections with ResourceLoader template loading	mediawiki/extensions/WikibaseLexeme	master	+38 -108
Remove duplicated templates from test files	mediawiki/extensions/WikibaseLexeme	master	+7 -23
Register extracted vue template as resourceloader template for Qunit tests	mediawiki/extensions/WikibaseLexeme	master	+29 -56
Deduplicate vue templates between php implementation and js tests	mediawiki/extensions/WikibaseLexeme	master	+354 -432

Customize query in gerrit

Related Objects

Mentioned In: T266364: Determine cause of 20% increase in minify/req since 15 Oct 2020 (alert)
T197228: Ensure JS templates and code in sync
T211749: Invalidate parser cache when Lexeme templates are updated
T211866: LanguageAndLexicalCategoryWidget Vue Test and Prod Template are not in sync
T209931: Glosses: display the language name also in edit mode
Mentioned Here: T264207: Lexeme: Most QUnit tests are not actually running
T197228: Ensure JS templates and code in sync

Event Timeline

• Pablo-WMDE created this task.Jun 29 2018, 8:18 AM

Restricted Application added a project: Wikidata. · View Herald TranscriptJun 29 2018, 8:18 AM

• Pablo-WMDE updated the task description. (Show Details)Jun 29 2018, 8:24 AM

• Pablo-WMDE added a subscriber: Tarrow.

Lydia_Pintscher moved this task from incoming to other stuff on the Wikidata Lexicographical data board.Jun 29 2018, 9:20 AM

Jan_Dittrich awarded a token.Jun 29 2018, 9:32 AM

• Pablo-WMDE updated the task description. (Show Details)Oct 16 2018, 1:04 PM

Tarrow awarded a token.Dec 12 2018, 10:21 AM

Tarrow mentioned this in T209931: Glosses: display the language name also in edit mode.Dec 12 2018, 10:33 AM

Tarrow mentioned this in T211866: LanguageAndLexicalCategoryWidget Vue Test and Prod Template are not in sync.Dec 13 2018, 10:41 AM

Addshore triaged this task as Medium priority.Dec 13 2018, 12:18 PM

Addshore added projects: Technical-Debt, Wikidata-Campsite.

Addshore moved this task from Incoming to Needs Work on the Wikidata-Campsite board.

Tarrow mentioned this in T211749: Invalidate parser cache when Lexeme templates are updated.Dec 14 2018, 3:46 PM

Lydia_Pintscher added a project: [DEPRECATED] wdwb-tech.Jan 4 2019, 10:20 AM

Lydia_Pintscher moved this task from incoming to needs discussion or investigation on the Wikidata board.

Addshore lowered the priority of this task from Medium to Low.Feb 21 2019, 10:35 AM

Addshore removed a project: Wikidata-Campsite.

Addshore moved this task from needs discussion or investigation to in progress on the Wikidata board.Jun 21 2019, 11:30 PM

Addshore moved this task from in progress to needs discussion or investigation on the Wikidata board.Jun 22 2019, 10:39 PM

Addshore added a project: Wikidata-Trailblazing-Exploration.Dec 3 2019, 3:53 PM

Jakob_WMDE updated the task description. (Show Details)Dec 4 2019, 12:39 PM

Addshore assigned this task to Jakob_WMDE.Dec 18 2019, 11:43 AM

Addshore moved this task from Backlog to In Investigation (cost/impact) on the Wikidata-Trailblazing-Exploration board.

I believe that this is achievable with ResourceLoader HTML templates. We can extract the vue templates into individual files, and then share them between the PHP and JS code. It looks like MediaWiki currently expects this type of template to be mustache templates, but also provides a mechanism to provide alternative formats via custom template compilers.

Addshore removed Jakob_WMDE as the assignee of this task.Jan 29 2020, 11:24 AM

Addshore moved this task from In Investigation (cost/impact) to Campsite Candidates on the Wikidata-Trailblazing-Exploration board.

Addshore added a project: Wikidata-Campsite.

Addshore moved this task from Needs Work to Needs Tech Work on the Wikidata-Campsite board.

Tarrow renamed this task from Unify vue templates to Unify WikibaseLexeme vue templates.Sep 14 2020, 10:06 AM

Addshore moved this task from Needs Tech Work to Prioritized Wikidata Tech Backlog (prioritised from top to bottom) on the Wikidata-Campsite board.Sep 15 2020, 8:57 AM

Addshore updated the task description. (Show Details)Sep 15 2020, 9:42 AM

WMDE-leszek edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.Sep 22 2020, 12:24 PM

Rosalie_WMDE claimed this task.Sep 23 2020, 7:30 AM

Rosalie_WMDE moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

After looking into this for a bit and considering @Jakob_WMDE 's comment I believe this boils down to two questions:

Which templates are required in order for both JS and PHP unit tests to pass?

I have collected the following non-exhaustive list of templates needed for each. Please add or remove stuff from this list that may or may not belong there.

Jasmin/Qunit tests:
- Raw gloss widget
- Language lexical category widget
- Lemma widget
- Lexeme header widget
- Representation widget .
  
  Unit tests:
- Lexem form view
- Lexeme gramatical features form view
- Senses view

How do we integrate ResourceLoader calls in PHP + JS?

The way to use it on the client side is already clearly stated in the ResourceLoader HTML templates link shared above.
This file contains some use of the ResourceLoaderModule, although it's more about injecting templates into JS but maybe it's a good start:

https://github.com/wikimedia/mediawiki-extensions-WikibaseLexeme/blob/6aac433df2db00aad47c8a782109a684730c94c9/src/Presentation/View/TemplateModule.php

If I understand correctly, the templates are currently loaded by injecting them into the parser-cached HTML:

LexemeView.php

    private function getLemmaVueTemplate() {
        return <<<HTML                                                                                                                                                       
<script id="lemma-widget-vue-template" type="x-template">                                                                                                                    
    {$this->getRawLemmaVueTemplate()}                                                                                                                                        
</script>                                                                                                                                                                    
HTML;
    }
    private function getRawLemmaVueTemplate() {
        return <<<'HTML'                                                                                                                                                     
<div class="lemma-widget">                                                                                                                                                   
    <ul v-if="!inEditMode" class="lemma-widget_lemma-list">                                                                                                                  
...
    </div>                                                                                                                                                                   
</div>                                                                                                                                                                       
HTML;
    }

I think this task can be split into two parts:

Extract those templates into files, make the PHP code read them from there, and make the tests (QUnit or Jasmine – I think they both run in Node and have access to the file system) read them from those files as well. getLemmaVueTemplate() would still exist, but instead of calling $this->getRawLemmaVueTemplate(), it would call something to the effect of file_get_contents(). This would, I believe, not solve T197228 yet, but it should be a useful first step, and already eliminate the duplicate templates in the test.
Now that we have the templates in files, export them as ResourceLoader modules, and then make the JavaScript load them from there instead of from the <script> elements which the PHP code added to the parser-cached HTML. Later, remove the corresponding PHP code, so that we no longer embed the templates in the HTML. The QUnit and Jasmine tests can probably continue to load the templates directly from disk, and skip any ResourceLoader integration.

Change 630639 had a related patch set uploaded (by Rosalie Perside (WMDE); owner: Rosalie Perside (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] [WIP] Move LexemeView Templates to individual files.

https://gerrit.wikimedia.org/r/630639

gerritbot added a project: Patch-For-Review.Sep 28 2020, 4:56 PM

while working on this we realised that almost all Qunit test do not run in WikibaseLexeme. See more detail in the task T264207

Change 630901 had a related patch set uploaded (by Rosalie Perside (WMDE); owner: Tobias Andersson):
[mediawiki/extensions/WikibaseLexeme@master] Remove duplicated templates from test files

https://gerrit.wikimedia.org/r/630901

Change 631445 had a related patch set uploaded (by Rosalie Perside (WMDE); owner: Rosalie Perside (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] [WIP]Register extracted vue templates as resourceloader templates

https://gerrit.wikimedia.org/r/631445

Change 630639 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Deduplicate vue templates between php implementation and js tests

https://gerrit.wikimedia.org/r/630639

Change 630901 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Remove duplicated templates from test files

https://gerrit.wikimedia.org/r/630901

Change 631445 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Register extracted vue template as resourceloader template for Qunit tests

https://gerrit.wikimedia.org/r/631445

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.12; 2020-10-05; NEVER DEPLOYED).Oct 2 2020, 1:00 PM

Maintenance_bot removed a project: Patch-For-Review.Oct 2 2020, 1:10 PM

Change 631771 had a related patch set uploaded (by Noa wmde; owner: Noa wmde):
[mediawiki/extensions/WikibaseLexeme@master] Replace script injections with ResourceLoader template loading

https://gerrit.wikimedia.org/r/631771

gerritbot added a project: Patch-For-Review.Oct 2 2020, 1:26 PM

noarave moved this task from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 5 2020, 5:43 AM

Change 631771 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Replace script injections with ResourceLoader template loading

https://gerrit.wikimedia.org/r/631771

noarave moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 6 2020, 9:37 AM

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.13; 2020-10-12); removed MW-1.36-notes (1.36.0-wmf.12; 2020-10-05; NEVER DEPLOYED).Oct 6 2020, 10:00 AM

Change 632486 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Reinstate template script injections

https://gerrit.wikimedia.org/r/632486

Change 632487 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Revert "Reinstate template script injections"

https://gerrit.wikimedia.org/r/632487

Change 632486 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Reinstate template script injections

https://gerrit.wikimedia.org/r/632486

This is complete apart from the final removal of template HTML (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexeme/+/632487/) which is to happen once Wikidata has got 1.36.0-wmf.13 branch code,

WMDE-leszek moved this task from Test (Verification) to Stalled/Waiting on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 7 2020, 3:57 PM

WMDE-leszek set Due Date to Oct 22 2020, 10:00 PM.

WMDE-leszek moved this task from Stalled/Waiting to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 21 2020, 5:48 AM

Change 632487 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Revert "Reinstate template script injections"

https://gerrit.wikimedia.org/r/632487

Lucas_Werkmeister_WMDE moved this task from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 21 2020, 10:30 AM

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.16; 2020-11-03); removed MW-1.36-notes (1.36.0-wmf.13; 2020-10-12).Oct 21 2020, 11:00 AM

WMDE-leszek closed this task as Resolved.Oct 22 2020, 8:18 AM

WMDE-leszek moved this task from Test (Verification) to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

• Pablo-WMDE mentioned this in T197228: Ensure JS templates and code in sync.Nov 19 2020, 9:00 AM

Krinkle mentioned this in T266364: Determine cause of 20% increase in minify/req since 15 Oct 2020 (alert).Mar 11 2021, 6:41 PM