Page MenuHomePhabricator

Create a pipeline for supplying data to static map service
Closed, ResolvedPublic

Description

Just like graphs, maps can be constructed on the client by javascript and loading all the needed data, or they can be rendered on the server and given to the client as a simple image without any javascript (static service). In order for the static service to function, it needs to be able to get all the required data (geojson), which is stored in page-props of the article. This task discusses how to do it.

Requirements:

  • Should support previous revisions of a page
  • Should allow for a non-insane URL schema
  • Should permit a fast change propagation, not depending on LinksUpdate (and hence job queue)
  • Should allow for map refreshes on page purge, even if not implemented immediately

Event Timeline

MaxSem created this task.Apr 21 2016, 12:13 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 21 2016, 12:13 AM
Yurik added a subscriber: Yurik.EditedApr 21 2016, 12:55 AM

Proposal

In pageprops, store map-related geojson data as an object:

{
  "groups": {
    "groupID1": [ {...geojson1...} ],
    "groupID2": [ {...geojson2...},  {...geojson3...} ],
    ...
  },
  "maps": {
    "hash1": [ "groupID1" ],
    "hash2": [ "groupID1", "groupID2" ],
    ...
  }
}

In the above, "hash2" is calculated by first concatenating into the same array all groups and their content, converting to a string, running a hash function.

hash( JSON.stringify( [ "groupID1", {...geojson1...}, "groupID2", {...geojson2...},  {...geojson3...} ] ))

URL to get a static map - params: style, zoom, latitude, longitude, width x height. The hash could be made shorter to make it a bit less scary, e.g. first 4 hash symbols. Together with the page title, this should be more than enough to identify the map's data.

https://maps.wikimedia.org/img/osm-intl,15,37.8040,-122.4098,1000x1000.png?site=en.wikipedia.org&title=My_page&id=hash1

Alternatively, we could expose static service via restbase, so that wiki can load static map from the same domain, without creating a separate https connection:

https://en.wikipedia.org/api/rest_v1/page/map/png/My_page/hash1/osm-intl,15,37.8040,-122.4098,1000x1000.png
Restricted Application added a project: Discovery. · View Herald TranscriptApr 21 2016, 1:00 AM
Yurik moved this task from Unsorted to General on the Maps (Kartographer) board.Apr 21 2016, 12:55 PM

Can we get a little bit more context here? This smells like something that we can achieve using the EventBus and the change-propagation service we have in production, but there are too many missing details (like, what is this about, what do you want to achieve, etc etc)

MaxSem updated the task description. (Show Details)Apr 21 2016, 6:21 PM
Yurik updated the task description. (Show Details)Apr 21 2016, 7:17 PM

We'll need to add revision id into play for support of past revisions. The API would retrieve groups for current revid from page_props while for past revisions it would have to rely on parser output.

Yurik moved this task from Backlog to Imager on the Maps (Kartotherian) board.Apr 21 2016, 8:45 PM
Yurik added a comment.Apr 21 2016, 9:59 PM

I would prefer to keep it simple without revids, until we implement T119043

Do you want/need to pre-generate the maps or render them on demand? Would this flow describe what you want to achieve: page edit -> get page-props -> render the map -> store it now, serve later ?

Yurik added a comment.Oct 13 2016, 5:34 PM

Done:
http://maps.wikimedia.org/img/osm-intl,1,0,0,1000x1000.png?domain=www.mediawiki.org&title=Help:Extension:Kartographer&groups=_925bf4f20f98fdda374476b902f702d080492748

Yurik closed this task as Resolved.Oct 13 2016, 5:34 PM
Yurik claimed this task.