Change Details

Currently, the Kartographer extension converts ExternalData in `mapframe` tags to a maps.wikimedia.org URL with the appropriate geoshape request to fetch the shape. Each dynamic map or snapshot tile request will query the geoshape service to expand the shape. It's possible that it would be more efficient to immediately query the geoshape and stuff into the extension ParserCache data. * [ ] Investigate parser cache storage requirements for a typical shape. * Initial results from T322351 found an average of 3kB per shape for a small sample. * Take a larger sample, eg. for all of Germany or Europe data. * Show the difference between "native", binary geometry data and seralized GeoJSON. * How many articles have a geoshape on all wikis? (maybe using search_insource) * [ ] Estimate the impact this might have on the infrastructure, internal and external traffic, storage, and processing. * Parse time: how long do these geoshape requests take? * Internal traffic is increased by average geoshape payload, multiplied by roughly number of pages with geoshapes. * External traffic changes somehow because clients get the geoshape through ResourceLoader in wgKartographerLiveData. * How many fewer calls will the geoshape endpoint receive? * [ ] Implement in Kartographer, behind a feature flag in parallel with the existing behavior. * POC is done in SimpleStyleParser.php but this normalization function should be moved out of this class anyway. * Should be performed before simplestyle transforms, for the Commons "Data" .map path. * Set "expensive" parser flag. * Inlined geoshape will go directly into the parser cache along with other geometry, probably no changes needed there. * Put it behind a feature flag, old code should still be run by default. * Failed requests can fall back to the old behavior gracefully after logging a warning. * [ ] MediaWiki must be able to route requests to kartotherian directly. * Development environment requires some tweaks to docker-compose.yml to allow local MediaWiki to contact local kartotherian. * Production might not allow this until we set it up explicitly. * [ ] Must work for ids and SPARQL query, for geoshape, geoline, geomask, geopoint. * [ ] Must work for [[ https://www.mediawiki.org/wiki/Help:Extension:Kartographer#Map_data_from_Commons | maps from commons ]]. * [ ] Keep kartotherian code for expanding geoshapes--although this can go away in the long term.

Currently, the Kartographer extension converts ExternalData in `mapframe` tags to a maps.wikimedia.org URL with the appropriate geoshape request to fetch the shape. Each dynamic map or snapshot tile request will query the geoshape service to expand the shape. It's possible that it would be more efficient to immediately query the geoshape and stuff into the extension ParserCache data. * [ ] Investigate parser cache storage requirements for a typical shape. * Initial results from T322351 found an average of 3kB per shape for a small sample. * Take a larger sample, eg. for all of Germany or Europe data. * Show the difference between "native", binary geometry data and seralized GeoJSON. * How many articles have a geoshape on all wikis? (maybe using search_insource) * [ ] Estimate the impact this might have on the infrastructure, internal and external traffic, storage, and processing. * Parse time: how long do these geoshape requests take? * Internal traffic is increased by average geoshape payload, multiplied by roughly number of pages with geoshapes. * External traffic changes somehow because clients get the geoshape through ResourceLoader in wgKartographerLiveData. * How many fewer calls will the geoshape endpoint receive?

Currently, the Kartographer extension converts ExternalData in `mapframe` tags to a maps.wikimedia.org URL with the appropriate geoshape request to fetch the shape. Each dynamic map or snapshot tile request will query the geoshape service to expand the shape. It's possible that it would be more efficient to immediately query the geoshape and stuff into the extension ParserCache data. * [ ] Investigate parser cache storage requirements for a typical shape. * Initial results from T322351 found an average of 3kB per shape for a small sample. * Take a larger sample, eg. for all of Germany or Europe data. * Show the difference between "native", binary geometry data and seralized GeoJSON. * How many articles have a geoshape on all wikis? (maybe using search_insource) * [ ] Estimate the impact this might have on the infrastructure, internal and external traffic, storage, and processing. * Parse time: how long do these geoshape requests take? * Internal traffic is increased by average geoshape payload, multiplied by roughly number of pages with geoshapes. * External traffic changes somehow because clients get the geoshape through ResourceLoader in wgKartographerLiveData. * How many fewer calls will the geoshape endpoint receive? * [ ] Implement in Kartographer, behind a feature flag in parallel with the existing behavior. * POC is done in SimpleStyleParser.php but this normalization function should be moved out of this class anyway. * Should be performed before simplestyle transforms, for the Commons "Data" .map path. * Set "expensive" parser flag. * Inlined geoshape will go directly into the parser cache along with other geometry, probably no changes needed there. * Put it behind a feature flag, old code should still be run by default. * Failed requests can fall back to the old behavior gracefully after logging a warning. * [ ] MediaWiki must be able to route requests to kartotherian directly. * Development environment requires some tweaks to docker-compose.yml to allow local MediaWiki to contact local kartotherian. * Production might not allow this until we set it up explicitly. * [ ] Must work for ids and SPARQL query, for geoshape, geoline, geomask, geopoint. * [ ] Must work for [[ https://www.mediawiki.org/wiki/Help:Extension:Kartographer#Map_data_from_Commons | maps from commons ]]. * [ ] Keep kartotherian code for expanding geoshapes--although this can go away in the long term.