Page MenuHomePhabricator

Geoshape service fails to deliver geoshapes from OSM
Closed, ResolvedPublic

Description

Geoshape service fails to deliver most of the map boundaries from OSM. For example,
https://maps.wikimedia.org/geoshape?getgeojson=1&ids=Q3734
returns "headers is not defined", even though Q3734 is linked to the boundary 869555482 on OSM (and it did return the correct geoshape yesterday)

Is it the problem on the Wikimedia side or on the OSM side?

Event Timeline

Atsirlin created this task.Mar 8 2019, 2:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 8 2019, 2:14 PM
jcrespo added a subscriber: jcrespo.Mar 8 2019, 4:07 PM

The people in the know are working on this, trying to figure the best way to fix the issue. They will know more and update when they can (they are working on a fix right now).

There is an issue in the code checking the HTTP headers. The issue is already fixed in https://github.com/kartotherian/geoshapes/releases/tag/v1.0.4. @MSantos is working on packaging and deploying the fix ASAP.

For the record:

We are in the middle of the migration from jessie to stretch. Some application changes were needed, which means we have different versions running on jessie vs stretch and a mixed cluster at this point.

This specific issue was monkey patched before, so as to not render the upgrade more chaotic than it already way (in retrospect, that was probably not a great idea, or at least not followed as closely as it should have been).

This monkey patch was erased by a recent deployment.

Gehel added a comment.Mar 8 2019, 4:16 PM

Note the previous patch was tracked on T214350

Mentioned in SAL (#wikimedia-operations) [2019-03-08T16:17:17Z] <mbsantos@deploy1001> Started deploy [kartotherian/deploy@d71df87] (stretch): UBN geoshapes services (T217898)

Mentioned in SAL (#wikimedia-operations) [2019-03-08T16:19:17Z] <mbsantos@deploy1001> Finished deploy [kartotherian/deploy@d71df87] (stretch): UBN geoshapes services (T217898) (duration: 02m 00s)

jcrespo added a project: User-notice.EditedMar 8 2019, 4:21 PM

More context: This didn't affect the tile service, but some maps shown on wikis may fail to load if using the geoshape service.

Mentioned in SAL (#wikimedia-operations) [2019-03-08T16:22:55Z] <mbsantos@deploy1001> Started deploy [kartotherian/deploy@cc302de] (stretch): UBN geoshapes services on maps2004.codfw.wmnet (T217898)

Gehel added a comment.Mar 8 2019, 4:23 PM

This issue should now (2019-03-08T16:19:17Z) be fixed with the latest deployment by @MSantos

Mentioned in SAL (#wikimedia-operations) [2019-03-08T16:23:19Z] <mbsantos@deploy1001> Finished deploy [kartotherian/deploy@cc302de] (stretch): UBN geoshapes services on maps2004.codfw.wmnet (T217898) (duration: 00m 24s)

Trizek-WMF added a subscriber: Trizek-WMF.

Possible summary for Tech News:

The Maps service has been unavailable during a few hours. It has been fixed.

More information here in the task would be welcomed. :)

Yes, it seems to be OK now. Thanks to all for your quick response!

Atsirlin closed this task as Resolved.Mar 8 2019, 4:30 PM

Mentioned in SAL (#wikimedia-operations) [2019-03-08T16:47:51Z] <mbsantos@deploy1001> Started deploy [kartotherian/deploy@acf2694] (stretch): UBN geoshapes services on maps1004.eqiad.wmnet (T217898)

Mentioned in SAL (#wikimedia-operations) [2019-03-08T16:48:12Z] <mbsantos@deploy1001> Finished deploy [kartotherian/deploy@acf2694] (stretch): UBN geoshapes services on maps1004.eqiad.wmnet (T217898) (duration: 00m 22s)

Gehel added a comment.Mar 8 2019, 4:48 PM

@Trizek-WMF incident report is being written. The very short summary:

Starting 2019-03-07 18:32Z geoshape service was returning errors for some requests. A fix was deployed 2019-03-08 16:33Z.

Trizek-WMF reopened this task as Open.Mar 11 2019, 10:36 AM

I'm still experiencing that issue on a particular case. I don't know if it is due to the same causes, if not, please let me know so that I can open a new ticket.

I was visiting https://fr.wikipedia.org/wiki/Remparts_romains_de_Grenoble on Firefox and the map in-text is not displayed. Displaying the map from the infobox works well. I don't have that issue on Chromium though.

TheDJ added a subscriber: TheDJ.Mar 11 2019, 10:59 AM

@Trizek-WMF if it's only on Firefox, then it is likely due to browser-side caching. The map seems to work for me in Chrome, Safari and Firefox.

@Trizek-WMF are you able to confirm what @TheDJ suggested? It seems normal to me too and it would be strange to have different behaviors between browser because the bug was caused in the geoshapes services and its communication with the Wikidata Query Service.

Trizek-WMF closed this task as Resolved.Mar 11 2019, 6:14 PM

@Trizek-WMF are you able to confirm what @TheDJ suggested?

Yes. I think it was cached at some point, even if it was the first time I was accessing the article I gave as an example (so as other articles where I've seen the issue). I've rechecked on those articles and some others and it works fine.

Sorry for the noise.