Page MenuHomePhabricator

Continuous errors on several REST API resources (probably related to MCS release)
Closed, ResolvedPublic

Description

Right after MCS was released yesterday (per SAL)

17:15 	<bsitzmann@deploy1001> 	Started deploy [mobileapps/deploy@b04c397]: Update mobileapps to 3edfcad (T220045 T219411 T219667)

we've seen a sudden increase in the number of errors reported by our edge servers. Upon inspecting the issue this morning, I found that indeed some urls return errors consistently, see for instance

https://fr.wikipedia.org/api/rest_v1/page/media/Nikolai_Gorbachev

this must be fixed ASAP, I am even tempted to do a MCS rollback right now, but I am unsure what that would mean for anything relying on it.

Event Timeline

Joe created this task.Apr 10 2019, 6:23 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Joe triaged this task as Unbreak Now! priority.Apr 10 2019, 6:23 AM
Restricted Application added subscribers: Liuxinyu970226, Mholloway, TerraCodes. · View Herald TranscriptApr 10 2019, 6:23 AM
Joe added a comment.Apr 10 2019, 6:33 AM

I've looked around a bit, and while the number of errors is in general below the SLO we have for restbase:

  • they all come from URLs pertaining to MCS AFAICS
  • some resources consistently do not render

So I would consider this a UBN! ticket even if the error rate is not that significant.

Change 502686 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/services/mobileapps@master] Revert "Bifurcate imageinfo queries to improve performance"

https://gerrit.wikimedia.org/r/502686

Change 502686 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Revert "Bifurcate imageinfo queries to improve performance"

https://gerrit.wikimedia.org/r/502686

Mentioned in SAL (#wikimedia-operations) [2019-04-10T07:18:40Z] <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@efd5bd5]: Revert "Bifurcate imageinfo queries to improve performance" (T220574)

Mentioned in SAL (#wikimedia-operations) [2019-04-10T07:22:45Z] <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@efd5bd5]: Revert "Bifurcate imageinfo queries to improve performance" (T220574) (duration: 04m 05s)