Page MenuHomePhabricator

Text extracts empty for some articles
Closed, ResolvedPublic

Description

For the featured Articles on enwiki today & yesterday:

http://en.wikipedia.org/api/rest_v1/page/summary/Belgium_national_football_team

http://en.wikipedia.org/api/rest_v1/page/summary/Kuiper_belt

Testing criteria:

Details

Related Gerrit Patches:
mediawiki/extensions/TextExtracts : masterSuppress parser output wrapper div
mediawiki/core : masterParserOptions: Include wrapping class in options hash
operations/mediawiki-config : masterAdd RejectParserCacheValue handler for mw-parser-output invalidation
mediawiki/extensions/TextExtracts : wmf/1.30.0-wmf.1API: Change memcache key to clear cache
mediawiki/extensions/TextExtracts : masterAPI: Change memcache key to clear cache

Event Timeline

JoeWalsh created this task.May 12 2017, 2:38 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 12 2017, 2:38 PM
Fjalapeno added a subscriber: Fjalapeno.

@JoeWalsh suspects it may have to do with image placement in the article, but it's not entirely clear. Has anyone seen husband before?

I wonder if this is getting broken by the parser change adding a new mw-parser-output div (T37247).

Other known fallout: T164733 T165115 (really the same issue).

Change 353565 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/extensions/TextExtracts@master] Suppress parser output wrapper div

https://gerrit.wikimedia.org/r/353565

Change 353566 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] ParserOptions: Include wrapping class in options hash

https://gerrit.wikimedia.org/r/353566

bearND added a subscriber: bearND.May 12 2017, 4:36 PM

I guess the ones are empty that have had to be re-rendered since the MW DOM change was deployed. The other ones that still show the extract have not been re-rendered yet.

Yeah… I am thinking the same thing… looks like we need to get all this fixed an then the cache will need to purge before this will work

I reverted the parser change and deployed it just now https://gerrit.wikimedia.org/r/#/c/353584/1 the cache will need a purge still

Change 353593 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/extensions/TextExtracts@master] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353593

Change 353594 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/extensions/TextExtracts@wmf/1.30.0-wmf.1] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353594

Change 353594 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@wmf/1.30.0-wmf.1] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353594

Change 353593 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@master] API: Change memcache key to clear cache

https://gerrit.wikimedia.org/r/353593

Change 353596 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/mediawiki-config@master] Add RejectParserCacheValue handler for mw-parser-output invalidation

https://gerrit.wikimedia.org/r/353596

Change 353596 merged by jenkins-bot:
[operations/mediawiki-config@master] Add RejectParserCacheValue handler for mw-parser-output invalidation

https://gerrit.wikimedia.org/r/353596

Mentioned in SAL (#wikimedia-operations) [2017-05-12T19:02:37Z] <thcipriani@tin> Synchronized wmf-config/CommonSettings.php: [[gerrit:353597|Add RejectParserCacheValue handler for mw-parser-output]] T165161 (duration: 00m 40s)

Mentioned in SAL (#wikimedia-operations) [2017-05-12T19:20:51Z] <thcipriani@tin> Synchronized php-1.30.0-wmf.1/extensions/TextExtracts/includes/ApiQueryExtracts.php: [[gerrit:353593|API: Change memcache key to clear cache]] T165161 (duration: 00m 39s)

mobrovac closed this task as Resolved.May 13 2017, 1:05 AM
mobrovac claimed this task.
mobrovac triaged this task as Unbreak Now! priority.
mobrovac added a subscriber: mobrovac.

We have truncated the summary tables, so things should now be back to normal. Expect slightly elevated latencies until the storage has been repopulated. Closing as there is nothing else to be done here.

Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptMay 13 2017, 1:05 AM
Jdlrobson reopened this task as Open.May 15 2017, 11:09 AM
Jdlrobson added a subscriber: Jdlrobson.

I'm still seeing this on the beta cluster and in master so let's not resolve this quite yet..
It seems like TextExtracts that feeds the endpoint is returning empty text extracts..
https://en.wikipedia.beta.wmflabs.org/wiki/Special:ApiSandbox#action=query&format=json&prop=extracts&titles=San+Francisco

https://gerrit.wikimedia.org/r/#/c/353565/2 needs to be merged to master still. That looks great to me, but the commit message confused me a little.

Jdlrobson lowered the priority of this task from Unbreak Now! to High.May 15 2017, 2:09 PM

Given it appears fixed in production.

Change 353566 merged by jenkins-bot:
[mediawiki/core@master] ParserOptions: Include wrapping class in options hash

https://gerrit.wikimedia.org/r/353566

Here is a summary my understanding the current status of this issue. I was out last Friday, so mostly pieced this together from the incident report, SAL & this task. Corrections would be appreciated.

Friday

  • The original MW change introducing <div> wrappers was backed out, and cache busting code was deployed.
  • Slightly later, RB summary and feed_aggregated tables were purged.

From this point on, RB returned proper summaries, but old copies were potentially still in Varnish.

Sunday

Varnish TTL ceiling (recently lowered to 2 days) elapses, and all broken copies were removed from Varnish caches. From the production client's perspective, the issue was fully resolved at this point.

Ongoing

A new, fixed version of the original change are being prepared in https://gerrit.wikimedia.org/r/#/c/353565 and https://gerrit.wikimedia.org/r/353566.

Change 353565 merged by jenkins-bot:
[mediawiki/extensions/TextExtracts@master] Suppress parser output wrapper div

https://gerrit.wikimedia.org/r/353565

Anomie added a subscriber: Anomie.May 16 2017, 4:05 PM

Looks fixed now, e.g. https://en.wikipedia.beta.wmflabs.org/w/api.php?action=query&titles=Asian+black+bear&prop=extracts&exsentences=5&explaintext=true&exintro=true. TextExtract's caches may still be polluted, but a purge of the article fixes it.

ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)
ovasileva reassigned this task from mobrovac to ABorbaWMF.May 16 2017, 5:14 PM
ovasileva added subscribers: ABorbaWMF, ovasileva.

over to you @ABorbaWMF

Tested on staging. All the link previews worked.

Systems - Browser
Mac Sierra - Safari 10
Mac El Capitan - Firefox 53
Mac Sierra - Chrome 58
Mac Sierra - Opera 45
Mac Mountain Lion - Firefox 15

Windows 10 - Edge 15
Windows 10 - Chrome 58
Windows 8.1 - Firefox 53
Windows 8.1 - Opera 40

Jdlrobson closed this task as Resolved.May 17 2017, 8:24 AM

Resolving.