Page MenuHomePhabricator

[BUG] Old page revision shown when using Content Service
Closed, ResolvedPublic1 Estimated Story Points

Description

Steps to reproduce

The MediaWiki and Content Service show different revisions and lastmodified times. I've made the requests a few times and the Content Service doesn't seem to be updating.

Summary:

# Mobileview API:
$ curl -s https://de.wikipedia.org/w/api.php\?action\=mobileview\&format\=json\&formatversion\=2\&prop\=text%7Csections%7Clanguagecount%7Cthumb%7Cimage%7Cid%7Cnamespace%7Crevision%7Cdescription%7Clastmodified%7Cnormalizedtitle%7Cdisplaytitle%7Cprotection%7Ceditable\&onlyrequestedsections\=1\&sections\=0\&sectionprop\=toclevel%7Cline%7Canchor\&noheadings\=true\&page\=Terroranschlag_in_Orlando\&thumbsize\=640 | jq '.mobileview.lastmodified'
"2016-06-15T17:34:11Z"
# MCS endpoint:
$ curl -s https://de.wikipedia.org/api/rest_v1/page/mobile-sections-lead/Terroranschlag_in_Orlando | jq '.lastmodified'
"2016-06-15T17:07:14Z"

Full responses (ran at another time than summary):

https://de.wikipedia.org/api/rest_v1/page/mobile-sections-lead/Terroranschlag_in_Orlando

{
  "id": 9485856,
  "revision": "155323295",
  "lastmodified": "2016-06-15T14:23:06Z",
  "displaytitle": "Terroranschlag in Orlando",
  "normalizedtitle": "Terroranschlag in Orlando",
  "description": "Schießerei in einem Nachtclub in Orlando, Florida",
  "protection": {},
  "editable": true,
  "languagecount": 47,
  "image": {
    "file": "1912 S Orange Ave 2.png",
    "urls": {
      "320": "//upload.wikimedia.org/wikipedia/commons/e/e6/1912_S_Orange_Ave_2.png",
      "640": "//upload.wikimedia.org/wikipedia/commons/e/e6/1912_S_Orange_Ave_2.png",
      "800": "//upload.wikimedia.org/wikipedia/commons/e/e6/1912_S_Orange_Ave_2.png",
      "1024": "//upload.wikimedia.org/wikipedia/commons/e/e6/1912_S_Orange_Ave_2.png"
    }
  },
  "sections": [
    {
      "id": 0,
      "text": "<span><p>Bei dem <b>Terroranschlag in <a href=\"/wiki/Orlando_(Florida)\">Orlando</a></b> im <a href=\"/wiki/Vereinigte_Staaten\">US</a>-<a href=\"/wiki/Bundesstaat_der_Vereinigten_Staaten\">Bundesstaat</a> <a href=\"/wiki/Florida\">Florida</a> am 12. Juni 2016 wurden 49 Menschen getötet und 53 verletzt.<span class=\"mw-ref\"><a href=\"#cite_note-1\" style=\"counter-reset: mw-Ref 1;\"><span class=\"mw-reflink-text\">[1]</span></a></span> Damit ist er das folgenschwerste Attentat in den Vereinigten Staaten seit den <a href=\"/wiki/Terroranschläge_am_11._September_2001\">Anschlägen vom 11. September 2001</a>,<span class=\"mw-ref\"><a href=\"#cite_note-2\" style=\"counter-reset: mw-Ref 2;\"><span class=\"mw-reflink-text\">[2]</span></a></span> der gravierendste einzelne <a href=\"/wiki/Antischwule_Gewalt\">Gewaltakt gegen Homosexuelle</a> und eines der verheerendsten <a href=\"/wiki/Massaker\">Massaker</a> in der Geschichte des Landes.<span class=\"mw-ref\"><a href=\"#cite_note-3\" style=\"counter-reset: mw-Ref 3;\"><span class=\"mw-reflink-text\">[3]</span></a></span> Der Attentäter Omar Mateen wurde durch Polizisten getötet.<span class=\"mw-ref\"><a href=\"#cite_note-4\" style=\"counter-reset: mw-Ref 4;\"><span class=\"mw-reflink-text\">[4]</span></a></span></p></span><figure class=\"mw-default-size\"><a href=\"/wiki/Datei:1912_S_Orange_Ave_2.png\" class=\"image\"><img src=\"//upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/220px-1912_S_Orange_Ave_2.png\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/440px-1912_S_Orange_Ave_2.png 2x, //upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/330px-1912_S_Orange_Ave_2.png 1.5x\" height=\"146\" width=\"220\"></a><figcaption>Nachtclub <i>Pulse</i> (2006)</figcaption></figure>\n\n\n"
    },
    {
      "id": 1,
      "toclevel": 1,
      "anchor": "Tathergang",
      "line": "Tathergang"
    },
    {
      "id": 2,
      "toclevel": 1,
      "anchor": "T.C3.A4ter",
      "line": "Täter"
    },
    {
      "id": 3,
      "toclevel": 1,
      "anchor": "Reaktionen",
      "line": "Reaktionen"
    },
    {
      "id": 4,
      "toclevel": 1,
      "anchor": "Siehe_auch",
      "line": "Siehe auch"
    },
    {
      "id": 5,
      "toclevel": 1,
      "anchor": "Weblinks",
      "line": "Weblinks"
    },
    {
      "id": 6,
      "toclevel": 1,
      "anchor": "Einzelnachweise",
      "line": "Einzelnachweise"
    }
  ]
}
https://de.wikipedia.org/w/api.php?action=mobileview&format=json&formatversion=2&prop=text%7Csections%7Clanguagecount%7Cthumb%7Cimage%7Cid%7Cnamespace%7Crevision%7Cdescription%7Clastmodified%7Cnormalizedtitle%7Cdisplaytitle%7Cprotection%7Ceditable&onlyrequestedsections=1&sections=0&sectionprop=toclevel%7Cline%7Canchor&noheadings=true&page=Terroranschlag_in_Orlando&thumbsize=640

{
  "mobileview": {
    "normalizedtitle": "Terroranschlag in Orlando",
    "ns": 0,
    "lastmodified": "2016-06-15T16:49:29Z",
    "revision": 155327348,
    "languagecount": 47,
    "displaytitle": "Terroranschlag in Orlando",
    "id": 9485856,
    "description": "Schießerei in einem Nachtclub in Orlando, Florida",
    "image": {
      "file": "1912 S Orange Ave 2.png",
      "width": 887,
      "height": 590
    },
    "thumb": {
      "url": "//upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/640px-1912_S_Orange_Ave_2.png",
      "width": 640,
      "height": 426
    },
    "sections": [
      {
        "id": 0,
        "text": "<div class=\"thumb tright\">\n<div class=\"thumbinner\" style=\"width:222px;\"><a href=\"/wiki/Datei:1912_S_Orange_Ave_2.png\" class=\"image\"><img alt=\"\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/220px-1912_S_Orange_Ave_2.png\" width=\"220\" height=\"146\" class=\"thumbimage\" srcset=\"//upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/330px-1912_S_Orange_Ave_2.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/e/e6/1912_S_Orange_Ave_2.png/440px-1912_S_Orange_Ave_2.png 2x\" data-file-width=\"887\" data-file-height=\"590\"></a>\n<div class=\"thumbcaption\">\n<div class=\"magnify\"><a href=\"/wiki/Datei:1912_S_Orange_Ave_2.png\" class=\"internal\" title=\"vergrößern und Informationen zum Bild anzeigen\"></a></div>\nNachtclub <i>Pulse</i> (2006)</div>\n</div>\n</div>\n<p>Bei dem <b>Terroranschlag in <a href=\"/wiki/Orlando_(Florida)\" title=\"Orlando (Florida)\">Orlando</a></b> im US-Bundesstaat <a href=\"/wiki/Florida\" title=\"Florida\">Florida</a> am 12. Juni 2016 wurden 49 Menschen getötet und 53 verletzt.<sup id=\"cite_ref-1\" class=\"reference\"><a href=\"#cite_note-1\">[1]</a></sup> Damit ist er das folgenschwerste Attentat in den Vereinigten Staaten seit den <a href=\"/wiki/Terroranschl%C3%A4ge_am_11._September_2001\" title=\"Terroranschläge am 11. September 2001\">Anschlägen vom 11. September 2001</a>,<sup id=\"cite_ref-2\" class=\"reference\"><a href=\"#cite_note-2\">[2]</a></sup> der gravierendste einzelne <a href=\"/wiki/Antischwule_Gewalt\" title=\"Antischwule Gewalt\">Gewaltakt gegen Homosexuelle</a> und eines der verheerendsten <a href=\"/wiki/Massaker\" title=\"Massaker\">Massaker</a> in der Geschichte des Landes.<sup id=\"cite_ref-3\" class=\"reference\"><a href=\"#cite_note-3\">[3]</a></sup> Der Attentäter Omar Mateen wurde durch Polizisten getötet.<sup id=\"cite_ref-4\" class=\"reference\"><a href=\"#cite_note-4\">[4]</a></sup></p>\n<p></p>"
      }
    ],
    "protection": {},
    "editable": true
  }
}

Tilman reported this issue from https://de.wikipedia.org/wiki/Wikipedia:Fragen_zur_Wikipedia#veralteter_Artikelstand_bei_Abruf_.C3.BCber_Android-App

Event Timeline

Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJun 15 2016, 5:26 PM
Tbayer edited subscribers, added: Tbayer; removed: HaeB.Jun 15 2016, 5:39 PM
bearND updated the task description. (Show Details)

We have been experiencing some issues with Change Propagation lately (which is in charge of sending MCS re-rendering requests). However, the issue raised here is about impatience :) That page has been updated a couple of times in a short period of time. Alas, one should allow a reasonable window of time to pass before reporting it as an issue.

A note on Change Propagation issues. When there are issues with the service, that does not mean that updates to MCS will get lost, only delayed for the time the service is down. Coincidentally, @Niedzielski you picked the wrong timing for the tests. The service was down for 45mins during the time you issued the requests due to Kafka node reboots.

bearND added a subscriber: bearND.EditedJun 15 2016, 6:28 PM

I just ran the comparison between the two again, and see a difference of 58 minutes. My earlier run it was 24 minutes. That seems big, especially for pages with news-related content.

$ curl -s https://de.wikipedia.org/w/api.php\?action\=mobileview\&format\=json\&formatversion\=2\&prop\=text%7Csections%7Clanguagecount%7Cthumb%7Cimage%7Cid%7Cnamespace%7Crevision%7Cdescription%7Clastmodified%7Cnormalizedtitle%7Cdisplaytitle%7Cprotection%7Ceditable\&onlyrequestedsections\=1\&sections\=0\&sectionprop\=toclevel%7Cline%7Canchor\&noheadings\=true\&page\=Terroranschlag_in_Orlando\&thumbsize\=640 | jq '.mobileview.lastmodified'
"2016-06-15T18:05:38Z"
$ curl -s https://de.wikipedia.org/api/rest_v1/page/mobile-sections-lead/Terroranschlag_in_Orlando | jq '.lastmodified'
"2016-06-15T17:07:14Z"

I would say that from the reader's perspective, different platforms showing differing revisions of the same page at the same time looks like a data integrity issue, not one of impatience ;) Also note that the user reporting this on the dewiki village pump (permalink) saw a discrepancy of more than 20 hours.

A correction is in order on my part. As of yesterday's stop of Change Propagation, we have been witnessing a growing backlog. It is currently at 5h (meaning that re-renders are 5h behind live updates). We are working on getting it down to zero as we speak.

Change 294726 had a related patch set uploaded (by Mobrovac):
Change Prop: increase concurrency to 50

https://gerrit.wikimedia.org/r/294726

mobrovac claimed this task.Jun 16 2016, 4:00 PM
mobrovac triaged this task as High priority.

Change 294726 merged by Faidon Liambotis:
Change Prop: increase concurrency to 50

https://gerrit.wikimedia.org/r/294726

The backlog reached the peek of 6h and is now steadily decreasing thanks to increased concurrency. Change Propagation should go through it and completely catch up in the next 6h max.

mobrovac closed this task as Resolved.Jun 17 2016, 7:05 AM

CP is now up to date with the latest titles that need to be re-rendered for MCS, so resolving.