Page MenuHomePhabricator

NewPagesFeed not working when sorted oldest first
Closed, ResolvedPublicBUG REPORT

Description

NewPagesFeed does not display any articles when sorted Oldest first.

Steps to reproduce

  • Navigate to Special:NewPagesFeed on en.wikipedia.org
  • Tap on the "Oldest" radio button near the top right

Expected behavior : The NewPagesFeed displays articles oldest first
Current behavior : The articles in the feed disappear and nothing occurs

Notes

Console displays the following new error :

Uncaught ReferenceError: author_byline_html is not defined
    at child.eval (eval at _.template (Special:NewPagesFeed:22), <anonymous>:3:3373)
    at child.template (<anonymous>:22:874)
    at child.render (<anonymous>:197:1336)
    at child.addOne (<anonymous>:217:449)
    at Array.forEach (<anonymous>)
    at Function._.each._.forEach (<anonymous>:8:69)
    at child.Collection.(anonymous function) [as forEach] (<anonymous>:37:683)
    at child.addAll (<anonymous>:217:939)
    at child.trigger (<anonymous>:25:695)
    at child.reset (<anonymous>:35:440)

Details

Related Gerrit Patches:

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2019, 4:31 AM
Force_Radical triaged this task as Unbreak Now! priority.Feb 21 2019, 4:33 AM
Force_Radical added a project: Growth-Team.
Restricted Application added subscribers: Liuxinyu970226, TerraCodes. · View Herald TranscriptFeb 21 2019, 4:33 AM

@SBisson @kostajh @Catrope -- I am able to reproduce this issue. I'm tagging you in case the problem and solution are immediately obvious to you. We can discuss this in an upcoming team meeting.

FYI to @Niharika, whose team will soon be working on the New Pages Feed.

This is caused by the API retuning incomplete data for the latest revision of Battle of Choirokoitia, which was made anonymously on Feb 20, 2019. At the time of writing this, it's the first result returned by this API query.

SBisson lowered the priority of this task from Unbreak Now! to High.Feb 21 2019, 2:20 PM

I have fix this specific page's metadata by running the following on a maintenance server.

$acp = MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileProcessor::newFromPageId( [ 4306709 ] );
$acp->compileMetadata();

That fixes Special:NewPagesFeed in the short term but there's certainly a bug that caused this metadata to be compiled incomplete in the first place.

MMiller_WMF moved this task from Inbox to Revisit on the Growth-Team board.Feb 21 2019, 11:09 PM

Thanks, @SBisson. I'm going to put this in our team's "Revisit" column, so that we can keep an eye on how often it happens. Then we can decide whether to dig into the underlying issues. @Niharika's team may also discover this in their work.

Thanks for the quick fix, @SBisson.

I am currently experiencing this issue.

Thanks, @Barkeep49.

I am moving this back onto our sprint board so our team can discuss tomorrow. Hopefully we can do a quick fix to make the feed work again and then start on fixing the underlying issue.

kostajh added a comment.EditedFeb 27 2019, 2:07 AM

The JSON which is breaking the feed, as returned by this query:

{
                "ores_draftquality": "",
                "ores_articlequality": "B-class",
                "creator_contribution_page_url": "//en.wikipedia.org/wiki/Special:Contributions/",
                "creator_contribution_page": "Special:Contributions/",
                "creator_user_talk_page_exist": false,
                "creator_user_talk_page_url": "//en.wikipedia.org/wiki/User_talk:",
                "creator_user_talk_page": "User talk:",
                "creator_user_page_exist": false,
                "creator_user_page_url": "//en.wikipedia.org/wiki/User:",
                "creator_user_page": "User:",
                "creation_date_utc": "20051117201903",
                "user_name": "",
                "user_id": "",
                "user_experience": "",
                "user_editcount": "",
                "user_creation_date": "",
                "user_bot": "",
                "user_block_status": "",
                "user_autoconfirmed": "",
                "snippet": "",
                "rev_count": "",
                "reference": "",
                "prod_status": "",
                "page_len": "",
                "linkcount": "",
                "csd_status": "",
                "category_count": "",
                "blp_prod_status": "",
                "afd_status": "",
                "afc_state": "",
                "title": "Merrill Lynch & Co.",
                "reviewer": null,
                "ptrp_reviewed_updated": "20190226224641",
                "ptrp_last_reviewed_by": "0",
                "is_redirect": "0",
                "patrol_status": "0",
                "creation_date": "20051117201903",
                "copyvio": "885254066",
                "pageid": "3185291"
            }

The data as it exists in the DB:

> $articles = MediaWiki\Extension\PageTriage\ArticleMetadata::getMetadataForArticles( [ 3185291 ] );

> print_r($articles);
Array
(
    [3185291] => Array
        (
            [copyvio] => 885254066
            [creation_date] => 20051117201903
            [patrol_status] => 0
            [is_redirect] => 0
            [ptrp_last_reviewed_by] => 0
            [ptrp_reviewed_updated] => 20190226224641
            [reviewer] =>
            [title] => Merrill Lynch & Co.
        )

)

What this and T216674#4971774 have in common is the removal of a redirect.

kostajh added a comment.EditedFeb 27 2019, 2:43 PM

The feed is fixed now; the article in question had two edits which triggered metadata recompilation.

My suggestions for this are

  1. add a more robust check here:
if ( !isset( $metaData[$page] ) ) {
  // If metadata is missing for a page, add this to the API output.
  $result['pages_missing_metadata'][] = $page;
  $result['result'] = 'warning';
  continue;
}

We could establish a list of the required metadata fields, without which the UI breaks. Below this block of code, we could have another statement to check that metadata exists and that required fields are populated, and if not, the page is reported as missing metadata. This time we could add some logging so we have a better idea of the frequency of the problem.

  1. In looking at the logs, there's an error on metadata compilation, with the message Using cached lag value for {db_server} due to active transaction. Backtrace:
#0 /srv/mediawiki/php-1.33.0-wmf.18/includes/libs/rdbms/database/DatabaseMysqlBase.php(719): Wikimedia\Rdbms\DatabaseMysqlBase->getLagFromPtHeartbeat()
#1 /srv/mediawiki/php-1.33.0-wmf.18/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1841): Wikimedia\Rdbms\DatabaseMysqlBase->getLag()
#2 /srv/mediawiki/php-1.33.0-wmf.18/includes/specials/SpecialContributions.php(240): Wikimedia\Rdbms\LoadBalancer->safeGetLag()
#3 /srv/mediawiki/php-1.33.0-wmf.18/includes/specialpage/SpecialPage.php(569): SpecialContributions->execute()
#4 /srv/mediawiki/php-1.33.0-wmf.18/includes/specialpage/SpecialPageFactory.php(558): SpecialPage->run()
#5 /srv/mediawiki/php-1.33.0-wmf.18/includes/specialpage/SpecialPageFactory.php(616): MediaWiki\Special\SpecialPageFactory->executePath()
#6 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(3317): MediaWiki\Special\SpecialPageFactory->capturePath()
#7 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Preprocessor_Hash.php(1114): Parser->braceSubstitution()
#8 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(3036): PPFrame_Hash->expand()
#9 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(1354): Parser->replaceVariables()
#10 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(482): Parser->internalParse()
#11 /srv/mediawiki/php-1.33.0-wmf.18/includes/content/WikitextContent.php(369): Parser->parse()
#12 /srv/mediawiki/php-1.33.0-wmf.18/includes/content/AbstractContent.php(555): WikitextContent->fillParserOutput()
#13 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RenderedRevision.php(266): AbstractContent->getParserOutput()
#14 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached()
#15 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RevisionRenderer.php(193): MediaWiki\Revision\RenderedRevision->getSlotParserOutput()
#16 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RevisionRenderer.php(142): MediaWiki\Revision\RevisionRenderer->combineSlotOutput()
#17 (): Closure$MediaWiki\Revision\RevisionRenderer::getRenderedRevision#2()
#18 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RenderedRevision.php(197): call_user_func()
#19 /srv/mediawiki/php-1.33.0-wmf.18/includes/poolcounter/PoolWorkArticleView.php(194): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#20 /srv/mediawiki/php-1.33.0-wmf.18/includes/poolcounter/PoolCounterWork.php(123): PoolWorkArticleView->doWork()
#21 /srv/mediawiki/php-1.33.0-wmf.18/includes/page/WikiPage.php(1244): PoolCounterWork->execute()
#22 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileInterface.php(126): WikiPage->getParserOutput()
#23 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileCategoryCount.php(16): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileInterface->getParserOutputByPageId()
#24 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileProcessor.php(259): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileCategoryCount->compile()
#25 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileProcessor.php(190): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileProcessor->process()
#26 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/Hooks.php(201): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileProcessor->compileMetadata()
#27 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/MWCallableUpdate.php(34): Closure$MediaWiki\Extension\PageTriage\Hooks::onLinksUpdateComplete()
#28 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/DeferredUpdates.php(270): MWCallableUpdate->doUpdate()
#29 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/DeferredUpdates.php(228): DeferredUpdates::runUpdate()
#30 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/DeferredUpdates.php(140): DeferredUpdates::execute()
#31 /srv/mediawiki/php-1.33.0-wmf.18/extensions/EventBus/includes/JobExecutor.php(90): DeferredUpdates::doUpdates()
#32 /srv/mediawiki/rpc/RunSingleJob.php(77): JobExecutor->execute()
#33 {main}

This code is triggered by a change made in September rEPTR812a42f379a4: ArticleCompileInterface: Use Article::getParserOutput() directly. We should investigate this further as a possible cause of the metadata compilation failure.

kostajh claimed this task.Feb 28 2019, 3:27 PM
kostajh changed the subtype of this task from "Task" to "Bug Report".

Change 493505 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/PageTriage@master] Check required metadata properties in list API and compilation job

https://gerrit.wikimedia.org/r/493505

I've +2 the patch above. AFAICT it is not trying to fix the problem but only prevent it from breaking the page and add some logs.

Should it go back to "in progress" for the root issue?

Change 493505 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Check required metadata properties in list API and compilation job

https://gerrit.wikimedia.org/r/493505

@SBisson -- in case that's a question for me, I just want to say that I think that's a better question for @kostajh. I don't have a preference; it depends on the consequences of not fixing the root issue. What are they?

AFAICT it is not trying to fix the problem but only prevent it from breaking the page and add some logs.
Should it go back to "in progress" for the root issue?

I would like to see what we learn from the logs about the frequency of the problem. I didn't see any obvious solutions when I worked through the backtrace. If the patch successfully prevents the page from breaking then IMO this task could be moved to a queue of tasks that we or someone else can work on when there's time, but not necessarily in an upcoming sprint.

Moving to QA per the above.

Etonkovidova closed this task as Resolved.Mar 9 2019, 2:01 AM
Etonkovidova added a subscriber: Etonkovidova.

The patch fixed the problem with that page revision and it was deployed to wmf.20. I looked into logstash - no recent logs for the problem.
Only one WARNING about missing metadata:

Article metadata not found in DB, will attempt to save to DB via the job queue.

{
  "_index": "logstash-2019.03.09",
  "_type": "mediawiki",
  "_id": "AWlgJZGc8aQffZ3HH9xi",
  "_version": 1,
  "_score": null,
  "_source": {
    "server": "en.wikipedia.org",
    "phpversion": "7.2.8-1+0~20180725124257.2+stretch~1.gbp571e56",
    "wiki": "enwiki",
    "channel": "PageTriage",
    "program": "mediawiki",
    "type": "mediawiki",
    "trace": "#0 /srv/mediawiki/php-1.33.0-wmf.20/extensions/PageTriage/includes/ArticleMetadata.php(212): MediaWiki\\Extension\\PageTriage\\ArticleCompile\\ArticleCompileProcessor->compileMetadata(2)\n#1 /srv/mediawiki/php-1.33.0-wmf.20/extensions/PageTriage/includes/Api/ApiPageTriageList.php(86): MediaWiki\\Extension\\PageTriage\\ArticleMetadata->getMetadata()\n#2 /srv/mediawiki/php-1.33.0-wmf.20/includes/api/ApiMain.php(1595): MediaWiki\\Extension\\PageTriage\\Api\\ApiPageTriageList->execute()\n#3 /srv/mediawiki/php-1.33.0-wmf.20/includes/api/ApiMain.php(531): ApiMain->executeAction()\n#4 /srv/mediawiki/php-1.33.0-wmf.20/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()\n#5 /srv/mediawiki/php-1.33.0-wmf.20/api.php(87): ApiMain->execute()\n#6 /srv/mediawiki/w/api.php(3): require('/srv/mediawiki/...')\n#7 {main}",
    "http_method": "GET",
    "host": "mw1289",
    "@version": 1,
    "shard": "s1",
    "timestamp": "2019-03-09T01:51:08+00:00",
    "severity": "warning",
    "unique_id": "XIMcDApAEDYAAHQS1igAAACI",
    "level": "WARNING",
    "ip": "10.64.48.103",
    "mwversion": "1.33.0-wmf.20",
    "logsource": "mw1289",
    "message": "Article metadata not found in DB, will attempt to save to DB via the job queue.",
    "normalized_message": "Article metadata not found in DB, will attempt to save to DB via the job queue.",
    "url": "/w/api.php?action=pagetriagelist&format=json&page_id=60177002",
    "reqId": "XIMcDApAEDYAAHQS1igAAACI",
    "tags": [
      "input-kafka-rsyslog-udp-localhost",
      "rsyslog-udp-localhost",
      "kafka",
      "es"
    ],
    "referrer": "https://en.wikipedia.org/wiki/Sons_of_Antiochus_VIII",
    "@timestamp": "2019-03-09T01:51:08.556Z",
    "raw_query_string": "action=pagetriagelist&format=json&page_id=60177002",
    "facility": "user",
    "articles_without_metadata": "60177002"
  },
  "fields": {
    "@timestamp": [
      1552096268556
    ]
  },
  "sort": [
    1552096268556
  ]
}