Page MenuHomePhabricator

Text extracts empty for some articles
Closed, ResolvedPublic

Description

For the featured Articles on enwiki today & yesterday:

http://en.wikipedia.org/api/rest_v1/page/summary/Belgium_national_football_team

http://en.wikipedia.org/api/rest_v1/page/summary/Kuiper_belt

Testing criteria:

Event Timeline

Fjalapeno subscribed.

@JoeWalsh suspects it may have to do with image placement in the article, but it's not entirely clear. Has anyone seen husband before?

I wonder if this is getting broken by the parser change adding a new mw-parser-output div (T37247).

Other known fallout: T164733 T165115 (really the same issue).

Change 353565 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/extensions/TextExtracts@master] Suppress parser output wrapper div

https://gerrit.wikimedia.org/r/353565

Change 353566 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] ParserOptions: Include wrapping class in options hash

https://gerrit.wikimedia.org/r/353566

I guess the ones are empty that have had to be re-rendered since the MW DOM change was deployed. The other ones that still show the extract have not been re-rendered yet.

Yeah… I am thinking the same thing… looks like we need to get all this fixed an then the cache will need to purge before this will work

I reverted the parser change and deployed it just now https://gerrit.wikimedia.org/r/#/c/353584/1 the cache will need a purge still

Change 353593 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/extensions/TextExtracts@master] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353593

Change 353594 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/extensions/TextExtracts@wmf/1.30.0-wmf.1] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353594

Change 353594 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@wmf/1.30.0-wmf.1] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353594

Change 353593 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@master] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353593

Change 353596 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/mediawiki-config@master] Add RejectParserCacheValue handler for mw-parser-output invalidation

https://gerrit.wikimedia.org/r/353596

Change 353596 merged by jenkins-bot:
[operations/mediawiki-config@master] Add RejectParserCacheValue handler for mw-parser-output invalidation

https://gerrit.wikimedia.org/r/353596

Mentioned in SAL (#wikimedia-operations) [2017-05-12T19:02:37Z] <thcipriani@tin> Synchronized wmf-config/CommonSettings.php: [[gerrit:353597|Add RejectParserCacheValue handler for mw-parser-output]] T165161 (duration: 00m 40s)

Mentioned in SAL (#wikimedia-operations) [2017-05-12T19:20:51Z] <thcipriani@tin> Synchronized php-1.30.0-wmf.1/extensions/TextExtracts/includes/ApiQueryExtracts.php: [[gerrit:353593|API: Change memcache key to clear cache]] T165161 (duration: 00m 39s)

mobrovac claimed this task.
mobrovac triaged this task as Unbreak Now! priority.
mobrovac subscribed.

We have truncated the summary tables, so things should now be back to normal. Expect slightly elevated latencies until the storage has been repopulated. Closing as there is nothing else to be done here.

Jdlrobson subscribed.

I'm still seeing this on the beta cluster and in master so let's not resolve this quite yet..
It seems like TextExtracts that feeds the endpoint is returning empty text extracts..
https://en.wikipedia.beta.wmflabs.org/wiki/Special:ApiSandbox#action=query&format=json&prop=extracts&titles=San+Francisco

https://gerrit.wikimedia.org/r/#/c/353565/2 needs to be merged to master still. That looks great to me, but the commit message confused me a little.

Jdlrobson lowered the priority of this task from Unbreak Now! to High.May 15 2017, 2:09 PM

Given it appears fixed in production.

Change 353566 merged by jenkins-bot:
[mediawiki/core@master] ParserOptions: Include wrapping class in options hash

https://gerrit.wikimedia.org/r/353566

Here is a summary my understanding the current status of this issue. I was out last Friday, so mostly pieced this together from the incident report, SAL & this task. Corrections would be appreciated.

Friday

  • The original MW change introducing <div> wrappers was backed out, and cache busting code was deployed.
  • Slightly later, RB summary and feed_aggregated tables were purged.

From this point on, RB returned proper summaries, but old copies were potentially still in Varnish.

Sunday

Varnish TTL ceiling (recently lowered to 2 days) elapses, and all broken copies were removed from Varnish caches. From the production client's perspective, the issue was fully resolved at this point.

Ongoing

A new, fixed version of the original change are being prepared in https://gerrit.wikimedia.org/r/#/c/353565 and https://gerrit.wikimedia.org/r/353566.

Change 353565 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@master] Suppress parser output wrapper div

https://gerrit.wikimedia.org/r/353565

Tested on staging. All the link previews worked.

Systems - Browser
Mac Sierra - Safari 10
Mac El Capitan - Firefox 53
Mac Sierra - Chrome 58
Mac Sierra - Opera 45
Mac Mountain Lion - Firefox 15

Windows 10 - Edge 15
Windows 10 - Chrome 58
Windows 8.1 - Firefox 53
Windows 8.1 - Opera 40