Page MenuHomePhabricator

NewPagesFeed not working when sorted oldest first
Closed, ResolvedPublicBUG REPORT


NewPagesFeed does not display any articles when sorted Oldest first.

Steps to reproduce

  • Navigate to Special:NewPagesFeed on
  • Tap on the "Oldest" radio button near the top right

Expected behavior : The NewPagesFeed displays articles oldest first
Current behavior : The articles in the feed disappear and nothing occurs


Console displays the following new error :

Uncaught ReferenceError: author_byline_html is not defined
    at child.eval (eval at _.template (Special:NewPagesFeed:22), <anonymous>:3:3373)
    at child.template (<anonymous>:22:874)
    at child.render (<anonymous>:197:1336)
    at child.addOne (<anonymous>:217:449)
    at Array.forEach (<anonymous>)
    at Function._.each._.forEach (<anonymous>:8:69)
    at child.Collection.(anonymous function) [as forEach] (<anonymous>:37:683)
    at child.addAll (<anonymous>:217:939)
    at child.trigger (<anonymous>:25:695)
    at child.reset (<anonymous>:35:440)

Event Timeline

Force_Radical triaged this task as Unbreak Now! priority.Feb 21 2019, 4:33 AM
Force_Radical added a project: Growth-Team.

@SBisson @kostajh @Catrope -- I am able to reproduce this issue. I'm tagging you in case the problem and solution are immediately obvious to you. We can discuss this in an upcoming team meeting.

FYI to @Niharika, whose team will soon be working on the New Pages Feed.

This is caused by the API retuning incomplete data for the latest revision of Battle of Choirokoitia, which was made anonymously on Feb 20, 2019. At the time of writing this, it's the first result returned by this API query.

SBisson lowered the priority of this task from Unbreak Now! to High.Feb 21 2019, 2:20 PM

I have fix this specific page's metadata by running the following on a maintenance server.

$acp = MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileProcessor::newFromPageId( [ 4306709 ] );

That fixes Special:NewPagesFeed in the short term but there's certainly a bug that caused this metadata to be compiled incomplete in the first place.

Thanks, @SBisson. I'm going to put this in our team's "Revisit" column, so that we can keep an eye on how often it happens. Then we can decide whether to dig into the underlying issues. @Niharika's team may also discover this in their work.

I am currently experiencing this issue.

Thanks, @Barkeep49.

I am moving this back onto our sprint board so our team can discuss tomorrow. Hopefully we can do a quick fix to make the feed work again and then start on fixing the underlying issue.

The JSON which is breaking the feed, as returned by this query:

                "ores_draftquality": "",
                "ores_articlequality": "B-class",
                "creator_contribution_page_url": "//",
                "creator_contribution_page": "Special:Contributions/",
                "creator_user_talk_page_exist": false,
                "creator_user_talk_page_url": "//",
                "creator_user_talk_page": "User talk:",
                "creator_user_page_exist": false,
                "creator_user_page_url": "//",
                "creator_user_page": "User:",
                "creation_date_utc": "20051117201903",
                "user_name": "",
                "user_id": "",
                "user_experience": "",
                "user_editcount": "",
                "user_creation_date": "",
                "user_bot": "",
                "user_block_status": "",
                "user_autoconfirmed": "",
                "snippet": "",
                "rev_count": "",
                "reference": "",
                "prod_status": "",
                "page_len": "",
                "linkcount": "",
                "csd_status": "",
                "category_count": "",
                "blp_prod_status": "",
                "afd_status": "",
                "afc_state": "",
                "title": "Merrill Lynch & Co.",
                "reviewer": null,
                "ptrp_reviewed_updated": "20190226224641",
                "ptrp_last_reviewed_by": "0",
                "is_redirect": "0",
                "patrol_status": "0",
                "creation_date": "20051117201903",
                "copyvio": "885254066",
                "pageid": "3185291"

The data as it exists in the DB:

> $articles = MediaWiki\Extension\PageTriage\ArticleMetadata::getMetadataForArticles( [ 3185291 ] );

> print_r($articles);
    [3185291] => Array
            [copyvio] => 885254066
            [creation_date] => 20051117201903
            [patrol_status] => 0
            [is_redirect] => 0
            [ptrp_last_reviewed_by] => 0
            [ptrp_reviewed_updated] => 20190226224641
            [reviewer] =>
            [title] => Merrill Lynch & Co.


What this and T216674#4971774 have in common is the removal of a redirect.

The feed is fixed now; the article in question had two edits which triggered metadata recompilation.

My suggestions for this are

  1. add a more robust check here:
if ( !isset( $metaData[$page] ) ) {
  // If metadata is missing for a page, add this to the API output.
  $result['pages_missing_metadata'][] = $page;
  $result['result'] = 'warning';

We could establish a list of the required metadata fields, without which the UI breaks. Below this block of code, we could have another statement to check that metadata exists and that required fields are populated, and if not, the page is reported as missing metadata. This time we could add some logging so we have a better idea of the frequency of the problem.

  1. In looking at the logs, there's an error on metadata compilation, with the message Using cached lag value for {db_server} due to active transaction. Backtrace:
#0 /srv/mediawiki/php-1.33.0-wmf.18/includes/libs/rdbms/database/DatabaseMysqlBase.php(719): Wikimedia\Rdbms\DatabaseMysqlBase->getLagFromPtHeartbeat()
#1 /srv/mediawiki/php-1.33.0-wmf.18/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1841): Wikimedia\Rdbms\DatabaseMysqlBase->getLag()
#2 /srv/mediawiki/php-1.33.0-wmf.18/includes/specials/SpecialContributions.php(240): Wikimedia\Rdbms\LoadBalancer->safeGetLag()
#3 /srv/mediawiki/php-1.33.0-wmf.18/includes/specialpage/SpecialPage.php(569): SpecialContributions->execute()
#4 /srv/mediawiki/php-1.33.0-wmf.18/includes/specialpage/SpecialPageFactory.php(558): SpecialPage->run()
#5 /srv/mediawiki/php-1.33.0-wmf.18/includes/specialpage/SpecialPageFactory.php(616): MediaWiki\Special\SpecialPageFactory->executePath()
#6 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(3317): MediaWiki\Special\SpecialPageFactory->capturePath()
#7 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Preprocessor_Hash.php(1114): Parser->braceSubstitution()
#8 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(3036): PPFrame_Hash->expand()
#9 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(1354): Parser->replaceVariables()
#10 /srv/mediawiki/php-1.33.0-wmf.18/includes/parser/Parser.php(482): Parser->internalParse()
#11 /srv/mediawiki/php-1.33.0-wmf.18/includes/content/WikitextContent.php(369): Parser->parse()
#12 /srv/mediawiki/php-1.33.0-wmf.18/includes/content/AbstractContent.php(555): WikitextContent->fillParserOutput()
#13 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RenderedRevision.php(266): AbstractContent->getParserOutput()
#14 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached()
#15 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RevisionRenderer.php(193): MediaWiki\Revision\RenderedRevision->getSlotParserOutput()
#16 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RevisionRenderer.php(142): MediaWiki\Revision\RevisionRenderer->combineSlotOutput()
#17 (): Closure$MediaWiki\Revision\RevisionRenderer::getRenderedRevision#2()
#18 /srv/mediawiki/php-1.33.0-wmf.18/includes/Revision/RenderedRevision.php(197): call_user_func()
#19 /srv/mediawiki/php-1.33.0-wmf.18/includes/poolcounter/PoolWorkArticleView.php(194): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#20 /srv/mediawiki/php-1.33.0-wmf.18/includes/poolcounter/PoolCounterWork.php(123): PoolWorkArticleView->doWork()
#21 /srv/mediawiki/php-1.33.0-wmf.18/includes/page/WikiPage.php(1244): PoolCounterWork->execute()
#22 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileInterface.php(126): WikiPage->getParserOutput()
#23 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileCategoryCount.php(16): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileInterface->getParserOutputByPageId()
#24 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileProcessor.php(259): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileCategoryCount->compile()
#25 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/ArticleCompile/ArticleCompileProcessor.php(190): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileProcessor->process()
#26 /srv/mediawiki/php-1.33.0-wmf.18/extensions/PageTriage/includes/Hooks.php(201): MediaWiki\Extension\PageTriage\ArticleCompile\ArticleCompileProcessor->compileMetadata()
#27 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/MWCallableUpdate.php(34): Closure$MediaWiki\Extension\PageTriage\Hooks::onLinksUpdateComplete()
#28 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/DeferredUpdates.php(270): MWCallableUpdate->doUpdate()
#29 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/DeferredUpdates.php(228): DeferredUpdates::runUpdate()
#30 /srv/mediawiki/php-1.33.0-wmf.18/includes/deferred/DeferredUpdates.php(140): DeferredUpdates::execute()
#31 /srv/mediawiki/php-1.33.0-wmf.18/extensions/EventBus/includes/JobExecutor.php(90): DeferredUpdates::doUpdates()
#32 /srv/mediawiki/rpc/RunSingleJob.php(77): JobExecutor->execute()
#33 {main}

This code is triggered by a change made in September rEPTR812a42f379a4: ArticleCompileInterface: Use Article::getParserOutput() directly. We should investigate this further as a possible cause of the metadata compilation failure.

kostajh changed the subtype of this task from "Task" to "Bug Report".

Change 493505 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/PageTriage@master] Check required metadata properties in list API and compilation job

I've +2 the patch above. AFAICT it is not trying to fix the problem but only prevent it from breaking the page and add some logs.

Should it go back to "in progress" for the root issue?

Change 493505 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Check required metadata properties in list API and compilation job

@SBisson -- in case that's a question for me, I just want to say that I think that's a better question for @kostajh. I don't have a preference; it depends on the consequences of not fixing the root issue. What are they?

AFAICT it is not trying to fix the problem but only prevent it from breaking the page and add some logs.
Should it go back to "in progress" for the root issue?

I would like to see what we learn from the logs about the frequency of the problem. I didn't see any obvious solutions when I worked through the backtrace. If the patch successfully prevents the page from breaking then IMO this task could be moved to a queue of tasks that we or someone else can work on when there's time, but not necessarily in an upcoming sprint.

Etonkovidova added a subscriber: Etonkovidova.

The patch fixed the problem with that page revision and it was deployed to wmf.20. I looked into logstash - no recent logs for the problem.
Only one WARNING about missing metadata:

Article metadata not found in DB, will attempt to save to DB via the job queue.

  "_index": "logstash-2019.03.09",
  "_type": "mediawiki",
  "_id": "AWlgJZGc8aQffZ3HH9xi",
  "_version": 1,
  "_score": null,
  "_source": {
    "server": "",
    "phpversion": "7.2.8-1+0~20180725124257.2+stretch~1.gbp571e56",
    "wiki": "enwiki",
    "channel": "PageTriage",
    "program": "mediawiki",
    "type": "mediawiki",
    "trace": "#0 /srv/mediawiki/php-1.33.0-wmf.20/extensions/PageTriage/includes/ArticleMetadata.php(212): MediaWiki\\Extension\\PageTriage\\ArticleCompile\\ArticleCompileProcessor->compileMetadata(2)\n#1 /srv/mediawiki/php-1.33.0-wmf.20/extensions/PageTriage/includes/Api/ApiPageTriageList.php(86): MediaWiki\\Extension\\PageTriage\\ArticleMetadata->getMetadata()\n#2 /srv/mediawiki/php-1.33.0-wmf.20/includes/api/ApiMain.php(1595): MediaWiki\\Extension\\PageTriage\\Api\\ApiPageTriageList->execute()\n#3 /srv/mediawiki/php-1.33.0-wmf.20/includes/api/ApiMain.php(531): ApiMain->executeAction()\n#4 /srv/mediawiki/php-1.33.0-wmf.20/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()\n#5 /srv/mediawiki/php-1.33.0-wmf.20/api.php(87): ApiMain->execute()\n#6 /srv/mediawiki/w/api.php(3): require('/srv/mediawiki/...')\n#7 {main}",
    "http_method": "GET",
    "host": "mw1289",
    "@version": 1,
    "shard": "s1",
    "timestamp": "2019-03-09T01:51:08+00:00",
    "severity": "warning",
    "unique_id": "XIMcDApAEDYAAHQS1igAAACI",
    "level": "WARNING",
    "ip": "",
    "mwversion": "1.33.0-wmf.20",
    "logsource": "mw1289",
    "message": "Article metadata not found in DB, will attempt to save to DB via the job queue.",
    "normalized_message": "Article metadata not found in DB, will attempt to save to DB via the job queue.",
    "url": "/w/api.php?action=pagetriagelist&format=json&page_id=60177002",
    "reqId": "XIMcDApAEDYAAHQS1igAAACI",
    "tags": [
    "referrer": "",
    "@timestamp": "2019-03-09T01:51:08.556Z",
    "raw_query_string": "action=pagetriagelist&format=json&page_id=60177002",
    "facility": "user",
    "articles_without_metadata": "60177002"
  "fields": {
    "@timestamp": [
  "sort": [