Page MenuHomePhabricator

Investigate missing highways
Closed, ResolvedPublic

Description

pasted_file (415×867 px, 47 KB)

http://www.openstreetmap.org/way/332692000 is missing from the roads table in the database

Related Objects

StatusSubtypeAssignedTask
OpenNone
InvalidNone
OpenFeatureNone
InvalidNone
OpenNone
ResolvedPnorman
DeclinedNone
DeclinedNone
DeclinedNone
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DuplicateNone
ResolvedPnorman
ResolvedNone
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
Resolved debt
ResolvedPnorman
ResolvedPnorman
ResolvedNone
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman
DeclinedPnorman
ResolvedPnorman
DeclinedPnorman
ResolvedPnorman
ResolvedPnorman
ResolvedPnorman

Event Timeline

Works locally, so I'm going to wait until my test server catches up with diffs and try again

Diffs didn't help, and I found some more cases in Merrit.

pasted_file (462×823 px, 284 KB)

Some ways missing are 429682967, 41939409. The last of these looks the best for debugging, being the simplest.

Nothing is obvious on the way that would cause this problem in isolation, but it's been updated recently, with only one change and nothing for a long time before that. Changeset 46886351 removed some tags and changed the position of node 519568901. The changeset made it out in the planet.osm.org minutely replication diff 002359569 (diff, state)

Manually filtering for the involved objects gives the diff

<?xml version='1.0' encoding='UTF-8'?>
<osmChange version="0.6" generator="Osmosis 0.43.1">
  <modify>
    <node id="519568901" version="2" timestamp="2017-03-16T03:04:42Z" uid="355617" user="pnorman" changeset="46886351" lat="50.1054581" lon="-120.7934351"/>
  </modify>
  <modify>
    <way id="41939409" version="4" timestamp="2017-03-16T03:04:42Z" uid="355617" user="pnorman" changeset="46886351">
      <nd ref="519566009"/>
      <nd ref="519567886"/>
      <nd ref="519567978"/>
      <nd ref="600677627"/>
      <nd ref="519568901"/>
      <nd ref="519568767"/>
      <tag k="attribution" v="GeoBase®"/>
      <tag k="geobase:acquisitionTechnique" v="GPS"/>
      <tag k="highway" v="tertiary"/>
      <tag k="is_in" v="Merritt,British Columbia"/>
      <tag k="lanes" v="2"/>
      <tag k="name" v="Voght Street"/>
      <tag k="source" v="Geobase_Import_2009"/>
    </way>
  </modify>
</osmChange>

The next step is to see if I can reproduce it. It's easy enough to find a road somewhere that needs editing for both geometry and tags.

Turns out to be from the update script which saved diffs with a .osm.gz suffix instead of .osc.gz. This caused it to be detected as the wrong file type.

Fixed in https://gist.github.com/pnorman/17b17e2d76505c7365278cbc1c7af80f/revisions#diff-01ef5096d4c7c0fe62a8e38dc274053b on my server, https://github.com/openstreetmap/chef/pull/118 for changes to the OpenStreetMap Chef script mine is based on

I'm seeing this again, so need to do more debugging.

I tested with a pure tag change in diff 2391111 with way 23480609 and it persisted. With a geometry change done by deleting some nodes, moving others, and adding some new ones the object vanished from the rendering tables. This was with way 127018391 which appeared in diff 2391120. Checking the planet.osm.org replication diff against the changes-2391119.osc.gz file found no differences, so it looks like the osmosis --read-replication-interval --write-xml-change file="-" | osmium merge-changes --simplify -F osc --fsync -o "${file}" command line worked.

Still debugging.

So the cause is different, and I've isolated the issue and reported it upstream with https://github.com/openstreetmap/osm2pgsql/issues/735

I'm not sure how to fix it, but will continue tomorrow. My next step is to try osm2pgsql master, osm2pgsql 0.90.0, and if either of those changes the results, 0.92.0 compiled locally instead of packaged.

Fixed upstream, and backported. I believe this will get picked up for debian stretch. If not we can look at if we need to package it ourselves.

Picked up for stretch, built, and updated on my server. I'm asking about jessie-backports.

Pnorman changed the task status from Open to Stalled.Apr 19 2017, 10:31 PM

Confirming that 0.92.0+ds-2 is in jessie-backports, so this is resolved.