Page MenuHomePhabricator

articleinfo: Wikidata label and description cannot be retrieved due to removal of wb_terms
Closed, ResolvedPublicBUG REPORT

Description

Steps to reproduce:
Visit articleinfo for a page, eg https://xtools.wmflabs.org/articleinfo/en.wikipedia.org/Alabama_v._North_Carolina
In the bugs section, observe:

Edit
2	Wikidata	Label for language en is missing	See: Help:Label		Edit
3	Wikidata	Description for language en is missing	See: Help:Description	Edit

Click edit to go to https://www.wikidata.org/wiki/Q60400606, where the label and description are indeed provided

Event Timeline

QEDK added a subscriber: QEDK.Apr 24 2020, 7:52 AM

@BEANS-X2 What "Bugs" section, can you link to the page?

QEDK moved this task from Inbox to Tracking on the User-QEDK board.
DannyS712 renamed this task from Bugs section misbehaving to articleinfo: Wikidata label and section exist, listed as missing.Apr 24 2020, 8:07 AM
DannyS712 triaged this task as High priority.
DannyS712 updated the task description. (Show Details)
DannyS712 moved this task from Backlog to Other on the XTools board.
DannyS712 changed the subtype of this task from "Task" to "Bug Report".
DannyS712 removed a project: User-QEDK.
DannyS712 added a subscriber: DannyS712.EditedApr 24 2020, 8:14 AM

The wikidata information is retrieved via PageRepository::getWikidataInfo

PageRepository::getWikidataInfo
public function getWikidataInfo(Page $page): array
{
    if (empty($page->getWikidataId())) {
        return [];
    }

    $wikidataId = 'Q'.ltrim($page->getWikidataId(), 'Q');
    $lang = $page->getProject()->getLang();

    $sql = "SELECT term_type AS term, term_text
            FROM wikidatawiki_p.wb_terms
            WHERE term_full_entity_id = :wikidataId
            AND term_type IN ('label', 'description')
            AND term_language = :lang";

    return $this->executeProjectsQuery($sql, [
        'lang' => $lang,
        'wikidataId' => $wikidataId,
    ])->fetchAll();
}

And then processed in Page::getWikidataErrors

Page::getWikidataErrors
public function getWikidataErrors(): array
{
    $errors = [];

    if (empty($this->getWikidataId())) {
        return [];
    }

    $wikidataInfo = $this->getRepository()->getWikidataInfo($this);

    $terms = array_map(function ($entry) {
        return $entry['term'];
    }, $wikidataInfo);

    $lang = $this->getLang();

    if (!in_array('label', $terms)) {
        $errors[] = [
            'prio' => 2,
            'name' => 'Wikidata',
            'notice' => "Label for language <em>$lang</em> is missing", // FIXME: i18n
            'explanation' => "See: <a target='_blank' " .
                "href='//www.wikidata.org/wiki/Help:Label'>Help:Label</a>",
        ];
    }

    if (!in_array('description', $terms)) {
        $errors[] = [
            'prio' => 3,
            'name' => 'Wikidata',
            'notice' => "Description for language <em>$lang</em> is missing", // FIXME: i18n
            'explanation' => "See: <a target='_blank' " .
                "href='//www.wikidata.org/wiki/Help:Description'>Help:Description</a>",
        ];
    }

    return $errors;
}

However, the wb_terms query no longer works, since that table is being removed and is no longer available on the replicas. See T248592: Move wb_terms data in cloud replicas to wb_terms_no_longer_updated for the table missing in the replicas, and T248086: Drop wb_terms in production from s4 (commonswiki, testcommonswiki), s3 (testwikidatawiki), s8 (wikidatawiki)/T208425: [EPIC] Kill the wb_terms table for its removal

The query will need to be rewritten for the new scheme

DannyS712 renamed this task from articleinfo: Wikidata label and section exist, listed as missing to articleinfo: Wikidata label and description cannot be retrieved due to removal of wb_terms.Apr 24 2020, 8:20 AM
Restricted Application added a project: User-DannyS712. · View Herald TranscriptApr 24 2020, 8:40 AM
MusikAnimal moved this task from Other to Pending deployment on the XTools board.Apr 25 2020, 10:32 PM
MusikAnimal added a subscriber: MusikAnimal.

Merged. Thank you very much!

Jony added a subscriber: Jony.Apr 26 2020, 2:25 PM
DannyS712 closed this task as Resolved.Apr 27 2020, 11:21 PM
DannyS712 moved this task from Pending deployment to Complete on the XTools board.

It would probably make a lot more sense for this code to make this query via the API rather than directly to the DB.
This avoid binding to the storage layer which is not really a supported data access method (though we do try to not break it too much).
This would also mean using db servers that are much larger and ready to answer these sorts of queries, you might even find that the API call ends up being faster than the DB query because of this is some cases.