Page MenuHomePhabricator

[OSM] Benchmark data with a bigger portion of the planet
Closed, ResolvedPublic5 Estimated Story Points

Description

Background information

imposm3 investigation was performed on a small chunk of the planet, to proceed with final conclusions, we need to test with more data.

What

Determine if imposm3 behaves accordingly with a bigger portion of the planet, ideally the full planet.

How

  • Use @mapbox/pixelmatch to identify changes in the new implementation of the OSM sync with imposm3
  • Document possible changes

Open questions

  • Is it possible to verify data coverage and spot differences between osm2pgsql and imposm3? Yes, with @mapbox/pixelmatch

Acceptance criteria

  • Benchmark use enough planet data: PS: Africa continent was chosen
  • Benchmark weights on the decision regarding imposm3 data reliability.

Results

Benchmark confirmed that the switch from imposm3 and osm2pgsql does not have major side-effects. The difference between the tiles generated using the two sources, does not impact on any negative results.

Here is the final report for this work: https://thesocialdev.github.io/tile-comparison-report/

In the proposed change, there is a slight change: maritime admin borders now are showed up and before they were stripped out with the populate_admin script. The proposal change remove the need for the admin script though.

Proposed decision: continue with imposm3 switch.

Event Timeline

MSantos raised the priority of this task from Medium to High.Sep 28 2020, 2:03 PM

@sdkim bumping priority to reflect reality. This benchmark is important to invest in the efforts to apply imposm3 in production.

Update: I found a way to identify map style changes through image comparison. I'm now finishing a tool to compare a sample of tiles of a specific region of the planet, you can see it here, still developing, once it's done I'll update documentation. The script will generate a report with the percentage of difference found with a sample of the tiles between current implementation with osm2pgsql and the implementation with imposm3 and show image examples comparison side-by-side.

The challenges so far are: I'm having trouble with imposm3 in my docker environment running in my macOS machine, it doesn't work well with big chunks of data. I'm switching at this moment to my Linux machine to see if my assumption is correct.

MSantos set the point value for this task to 5.Oct 6 2020, 11:07 AM

@sdkim and @Jgiannelos here is the result:

Benchmark confirmed that the switch from imposm3 and osm2pgsql does not have major side-effects. The difference between the tiles generated using the two sources does not impact any negative results.

Here is the final report for this work: https://thesocialdev.github.io/tile-comparison-report/

In the proposed change, there is a slight change: maritime admin borders now are showed up and before they were stripped out with the populate_admin script. The proposal change removes the need for the admin script though.

Source code:

Proposed decision: continue with imposm3 switch.