Page MenuHomePhabricator

Update web clients to source Wikidata descriptions from local wiki where available
Closed, ResolvedPublic5 Story Points

Description

In T184000 a mechanism was provided for wikidata descriptions to be updated by editors via a magic word. A prop 'description' and prop 'descriptionsource' was added to pages. Currently the mobile web client does not make use of these.

Acceptance criteria

  • Ensure MobileFrontend/Minerva/RelatedArticles source from description rather than Wikidata if available in all PHP and API calls.

Developer notes

https://en.wikipedia.org/w/api.php?action=query&prop=description&titles=Wagon_Train

returns

{
    "batchcomplete": "",
    "query": {
        "normalized": [
            {
                "from": "Wagon_Train",
                "to": "Wagon Train"
            }
        ],
        "pages": {
            "1071760": {
                "pageid": 1071760,
                "ns": 0,
                "title": "Wagon Train",
                "description": "Western television series aired 1957-1965",
                "descriptionsource": "local"
            }
        }
    }
}

In the server side we'll need to get page props

$out->getProperty('description')

Testing

You'll need to use the beta cluster for this.

https://en.m.wikipedia.beta.wmflabs.org/wiki/Albert_Einstein has a wikidata description that should match https://wikidata.beta.wmflabs.org/wiki/Special:EntityPage/Q356553

https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF&venotify=created&mobileaction=toggle_view_mobile
is associated with the Wikidata article https://wikidata.beta.wmflabs.org/wiki/Q450237.

Testing overriden descriptions

Edit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF and ensure the following text is present:

{{SHORTDESC:Woof woof}}

Verify

  • When searching for "Woof woof" the description shows as Woof woof.
  • When viewing the page "Woof woof" the description under the heading is "woof woof"
  • When you visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Monkey and look at related articles, the card "Woof woof" has the description "woof woof"

Testing the blanking behaviour

Edit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF and ensure the following text is present:

{{SHORTDESC:}}

Verify

  • When searching for "Woof woof" the description doesn't show from wikidata shows
  • When viewing the page "Woof woof" the description under the heading is blank~ from wikidata shows
  • When you visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Monkey and look at related articles, the card "Woof woof" has no description.~ from wikidata shows

[JR: Misunderstood how this works]

Testing the wikidata behaviour

Edit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF and ensure SHORTDESC is NOT present in the wikitext.

Verify

  • When searching for "Woof woof" the description shows with the value on wikidata (currently Meow)
  • When viewing the page "Woof woof" the description under the heading shows with the value on wikidata (currently Meow)
  • When you visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Monkey and look at related articles, the card "Woof woof" has the description "Meow"

Wikis without local override

  • We'll want to verify this is also working for wikis which do not have a local override.

I've linked up Spain articles on the beta cluster to a Wikidata entry

Details

Related Gerrit Patches:
mediawiki/extensions/MobileFrontend : masterShort description can now come from wikitext
mediawiki/extensions/RelatedArticles : masterReplace pageterms with description API
mediawiki/extensions/MobileFrontend : masterReplace "pageterm" wikidata API calls with "description" API

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 28 2018, 6:14 PM
Tgr added a subscriber: Tgr.Mar 28 2018, 6:56 PM

In the server side we'll need to get page props

$out->getProperty('description')

The property name is wikibase-shortdesc. Querying that will only work when there is a local override, though. I suppose Wikibase should provide some sort of helper method... how are you handling it on the server side currently?

bearND added a subscriber: bearND.Mar 28 2018, 9:50 PM

Is this task for web only or should we fan out from here to other clients as well?

@bearND This one is just for web!

Jdlrobson renamed this task from Update client to source Wikidata descriptions from local wiki where available to Update web clients to source Wikidata descriptions from local wiki where available.Mar 28 2018, 11:15 PM
ovasileva triaged this task as Medium priority.Mar 29 2018, 3:08 PM
ovasileva moved this task from Triaged but Future to Upcoming on the Readers-Web-Backlog board.
ovasileva raised the priority of this task from Medium to High.Apr 3 2018, 11:43 AM

Discussed in estimation. Note we should request both description (new) and the pageterms (already queried) in server and client usages in case the project that is running the code doesn't have description props.

Jdlrobson set the point value for this task to 5.Apr 3 2018, 4:22 PM

We're hitting 3 repos and setting up wikidata descriptions locally makes testing difficult so we've estimated by 5.

Tgr added a comment.Apr 3 2018, 4:49 PM

I suppose Wikibase should provide some sort of helper method...

Done in https://gerrit.wikimedia.org/r/c/423244/ (well, done once it gets merged).
See https://gerrit.wikimedia.org/r/c/423245/ for an example of using it.

Note we should request both description (new) and the pageterms (already queried) in server and client usages in case the project that is running the code doesn't have description props.

MobileFrontend is documented as using the release branch compatibility model, in which case that shouldn't be possible.

@Tgr I'm not sure I'm following are you saying we can ignore pageterms now? It was my understanding that pageterms will refer to the value inside Wikidata and pageprops the local version. Is that not correct? Can we ignore pageterms altogether now?

Tgr added a comment.Apr 3 2018, 6:05 PM

If you call the description API, it will look for a local override, and fall back to the Wikidata description if not found. This is true with or without the pending patch.

The patch makes the functionality of the API available from PHP, so you can do something like

$wikibaseClient = WikibaseClient::getDefaultInstance();
$descriptionLookup = $wikibaseClient->getStore()->getDescriptionLookup();
$description = $descriptionLookup->getDescription( $title, [ DescriptionLookup::SOURCE_LOCAL, DescriptionLookup::SOURCE_CENTRAL ] );

(where the last parameter tells the description service to use the local description and fall back to the Wikidata one).

Tgr added a comment.EditedApr 3 2018, 6:09 PM

(FWIW the description API and the pageterms API have different behavior for language fallback: if you call them on the German Wikipedia, and the page has a Wikidata description in English, but not in German, the pageterms API will return the English description and the description API won't return anything, which is IMO the more correct behavior for use cases like showing the description as a navigation aid.)

Jdrewniak moved this task from To Do to Doing on the Readers-Web-Kanbanana-Board-Old board.

Change 425261 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/extensions/MobileFrontend@master] Replace "pageterm" wikidata API calls with "description" API

https://gerrit.wikimedia.org/r/425261

Change 425261 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Replace "pageterm" wikidata API calls with "description" API

https://gerrit.wikimedia.org/r/425261

pmiazga moved this task from To Do to Doing on the Readers-Web-Kanbanana-Board-Old board.
Jdlrobson updated the task description. (Show Details)Apr 10 2018, 3:51 PM
Jdlrobson updated the task description. (Show Details)Apr 10 2018, 3:53 PM

Added testing instructions. Looks like we're 1/3 of the way to doing this! w00t!

Change 425768 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/extensions/RelatedArticles@master] Replace pageterms with description API

https://gerrit.wikimedia.org/r/425768

Jdrewniak added a comment.EditedApr 12 2018, 8:09 AM

The patches above replace client-side calls to pageterms API with the description API in mobilefrontend and RelatedArticles.

I wasn't able to find any occurrences of the pageterms API in Minerva, also, I wasn't able to find any server-side usages of the pageterms API in these codebases.

Am I missing something?

https://gerrit.wikimedia.org/r/425261 would have taken care of Minerva and MobileFrontend client side code.
The server side usage is inside includes/MobileFrontend.body.php :

ExtMobileFrontend::getWikibaseDescription

We'll probably want to update the function signature and the caller (since the new description is not coming from the Wikidata item) so that it also looks in the ParserOutput.

public static function onOutputPageParserOutput( $outputPage, ParserOutput $po ) {

I can take a look at that today if that helps.. I'm pretty familiar with this code and it's hard to test.

Change 425911 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Short description can now come from wikitext

https://gerrit.wikimedia.org/r/425911

Change 425768 merged by jenkins-bot:
[mediawiki/extensions/RelatedArticles@master] Replace pageterms with description API

https://gerrit.wikimedia.org/r/425768

Jdrewniak reassigned this task from Jdrewniak to Jdlrobson.Apr 18 2018, 5:11 PM
Jdrewniak added a subscriber: Jdrewniak.
Jdlrobson reassigned this task from Jdlrobson to pmiazga.Apr 23 2018, 5:36 PM

Piotr to provide guidance on the PHPy bits.

Just out of curiosity: Note that T184000 only applies to enwiki. What are the implications of this for the implementation and testing of the present task in all the other languages (and non-Wikipedia wikis)?

Change 425911 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Short description can now come from wikitext

https://gerrit.wikimedia.org/r/425911

Jdlrobson reassigned this task from pmiazga to ABorbaWMF.Apr 24 2018, 9:02 PM
Jdlrobson updated the task description. (Show Details)
Jdlrobson added subscribers: ABorbaWMF, pmiazga.

Over to you @ABorbaWMF !
Note testing should happen on the beta cluster, but as well as https://en.m.wikipedia.beta.wmflabs.org/wiki/Spain you will also need to test on the wikivoyage version of the beta cluster (https://en.m.wikivoyage.beta.wmflabs.org/wiki/Spain) - testing steps should be concise but let me know if anything is not clear.

Things seem to be working with the exception of the blanking portion. I am still seeing the description on screen after blanking. @Jdlrobson, I'm not sure if you want to kick the whole ticket back or just break out the blanking portion in a separate bug.

Jdlrobson added a comment.EditedApr 25 2018, 4:36 PM

@Tgr is blanking supporting?

When I set {{SHORTDESC:}} in the wikitext it doesn't seem to clear the description. Is that expected?
https://en.m.wikipedia.beta.wmflabs.org/w/api.php?action=query&format=json&prop=description&generator=prefixsearch&ppprop=displaytitle&piprop=thumbnail&pithumbsize=80&pilimit=15&wbptterms=description&redirects=&gpssearch=Spain&gpsnamespace=0&gpslimit=15

EDIT: Per T184000 this is the correct behaviour "If the magic word isn't used on the page, or if the description is blank, then it should show the description from Wikidata."

Jdlrobson updated the task description. (Show Details)

Do you want to take a look at this and sign this off given Olga is away?

Oh cool. I tried the above scenarios on a few devices and browsers. I am seeing the descriptions in the right places.


alexhollender added a subscriber: alexhollender.

Went through the testing steps as well. Looks good to me.

ovasileva closed this task as Resolved.Apr 30 2018, 7:32 PM