Blog

A few days ago we have added a couple of new road-related layers to our OSM Inspector.

Relations with highway=*

In the past, highway=* was sometimes added to route relations (type=route + route=road). This is not in line with current tagging standards any more, and can even lead to duplicated line geometries when importing OSM data into PostgreSQL with Osm2pgsql.

The layer shows these old-style relations in dark red.

Not all relations with highway=* are a problem – for example, pedestrian areas and rest areas mapped as multipolygons relations (type=multipolygon) are totally fine and hence not shown as errors. This exception applies to multipolygons with a highway=* value of pedestrian, footway, service, rest_area or services.

Multipolygon relations with other highway=* values are likely candidates for changing to area:highway=*. See the OSM Wiki page aboutarea:highway=* for details about mapping roads as areas.

Out-of-use Roads

Even though OSM records “facts on the ground” and not historic or future data, it is generally accepted to map roads and paths which are not in use any more (or are not completed yet).

In recent years, the so-called Lifecycle Tagging Schema was adopted by many mappers and is used to tag feature which are not actively used any more, are (partially) removed or expected to be created in the future. The schema works by adding the lifecycle state as a colon-separated prefix to main key of the feature. If a road tagged as highway=secondary becomes disused, the tag is changed to disused:highway=secondary. Other tags of the feature remain unchanged.

(An older, but equally valid, way of describing lifecycle states is to put highway=construction together with construction=secondary. The new OSMI layers treat both methods in the same way.)

The following lifecycle sates are in use:

• razed or dismantled or removed (note that the use of these prefixes is often subject to discussion and may be discouraged in your region)
• abandoned
• disused
• in use (normal state)
• construction
• proposed (note that mapping proposed features may contravene the on-the-ground mapping rule and may be discouraged in your region)

OSM Inspector now has four new layers displaying all linear highway=* features which are abandoned, disused, under construction, or proposed.

Two new layers render frequent mapping mistakes:

• Red: If a way has tags of contradicting lifecycle states, it is rendered on the Multiple lifecycle states layer in red. There can be valid reasons for such tagging, for example when a minor road is reconstructed to be a larger road (or a track in the forest is not maintained any more and becomes a narrow path). But in most cases this is an error.
• Orange: If the old method of lifecycle tagging is used in an incomplete way, e.g. highway=construction without a construction=* tag), this is considered an error. Sometimes the correct tag value can be discovered from the object history; otherwise a survey is required.

Use quality assurance tools responsibly

Please use quality assurance tools responsibly. They are often wrong. And even when they point you to poor mapping, simply repairing the mistakes is not a good choice. Poor mapping is often a sign of lack of knowledge (e.g. newbies), bad intentions, or mechanical edits, organised editing or imports gone wrong. It is worth having a look at an object’s history and other edits by the mappers involved to avoid simply “cleaning” the map and therefore possibly hiding a systemic issue. When contacting others about mistakes they have made, always remember that we all make mistakes and we can only become better by supporting each other.

We have made a few small improvements to the OSM Inspector user interface, providing better responsiveness to different display sizes, using tiled WMS access for better performance, and improving the presentation of feature detail information.

Give it a spin at https://tools.geofabrik.de/osmi and tell us what you think!

In the past night, a problem on the download server caused us to publish truncated data extracts for the following countries:

In Africa:

• Benin
• Cameroon
• Chad
• Comores
• Kenya
• Mali
• Mauritius
• Namibia
• Niger
• Nigeria
• Saint Helena/Ascension/Tristan da Cunha
• Sao Tome and Principe
• Senegal
• Seychelles
• Swaziland
• Togo
• Zambia
• Zimbabwe

In Asia:

• Bangladesh
• Cambodia
• China
• India
• Indonesia
• Iran
• Iraq
• Malaysia
• Maldives
• Nepal
• North Korea
• Philippines
• South Korea
• Sri Lanka
• Syria
• Taiwan
• Vietnam

The extracts have been reset to yesterday’s version but if you have downloaded an extract between 2021-09-05 01:00 UTC and 2021-09-05 10:00 UTC then you have got a truncated file. Also, if you have an automatic process that loads daily updates, and you have loaded and processed an update for any of these countries in that timespan, you now have a truncated database and need to re-import data. (This will not automatically fix itself with the next update.)

If you have not loaded updates during this timespan, or if you have loaded updates but not for the countries mentioned, then you are not affected. (You are also not affected if you have used the full Africa or Asia files, these were correct.)

In the last couple of weeks there have been two events that may have caused tile (or Nominatim) servers to stop updating. Here’s a quick outline on how to identify and repair the problems.
More …

We recently rewrote the OSM Inspector Routing View. The old views were used quite a lot by mappers to find potential routing errors. Unfortunately, they made use of PostGIS to find nearby, unconnected roads which meant having to import the whole road network into a PostGIS database every day. To make matters worse, the island and duplicate segment detection was implemented using OSRM which also has relatively heavy system requirements, especially if you do not limit yourself to automobile routing. This made us have a closer look at using GraphHopper to generate the routing view.

The new backend code was implemented from scratch and does not aim to be a one-by-one reimplementation. It uses GraphHopper to read the planet, build up the graph, analyse it and find nearby edges. Our code uses a forked version of GraphHopper 0.12 with the following changes:

• The original GraphHopper lacks some callback functions in its OSMReader class. Our changes allow us to build additional indexes and populate maps mapping from internal edge or node IDs to OSM object IDs.
• The PrepareRoutingSubnetworks class which removes islands from the graph is not extensible but we need to write the edges being removed to a GeoJSON file.

In addition to forking, we implemented a new routing profile (“flag encoder” in GraphHopper speak) accepting any road and ignoring access restrictions. It is used to find duplicated edges, unconnected roads, and islands.

The backend code is available on GitHub.

The new routing view has the following groups of layers:

Duplicated Edges

These layers show edges which exist twice in the graph. The blue ones are almost always real errors and should be fixed by comparing the tags and history of both ways. The purple ones are locations where two edges have equal geometry and at least one of them is an area. Opinions whether areas should share nodes with neighbouring roads are diverse among the OSM community. Therefore, these cases are shown in a separate layer at high zoom levels only.

The results of the duplicated edges layer (not involving areas) are similar to those of OSRM. Minor differences are possible where OSRM excluded a road due to access restrictions while our new implementation includes it.

Islands

The use of GraphHopper allows us to identify routing islands for multiple types of vehicles in one go. The OSM Inspector now provides islands for cars and bicycles using slightly modified version of the original profiles of GraphHopper (we use fewer bits to store the speed because island detection does not take travel times or distances into account). In addition, a layer with islands inaccessible to any vehicle is provided. This layer contains fewer entries in total but those contained are more serious because they are not caused by correct or incorrect access restriction tagging in OSM.

Unconnected Nodes

Most development time was spent finding a set of unconnected nodes layers with a proper separation of very likely, likely, potential errors and likely false positives. We are grateful to the folks on the German OSM forum who pointed out many bugs and too high rates of false positives.

Nodes where a single edge ends but with other edges within 15 meters are assigned to one of 6 priority classes. A distance below 2 meters makes a unconnected node appear in the top layer. The further composition of the layers depends on the road class, the distance, and access restrictions. The priority of unconnected nodes involving private access roads is reduced by 1. Service roads and footpaths get low default levels. In addition, the following rules avoid too many false positives:

• noexit=yes and entrance=* make unconnected nodes disappear.
• Nearby edges on different layers/levels are ignored.
• Linear barriers (fences, walls, embankments) are taken into account.
• Unconnected open ends of parking aisles are ignored if they snap to a parallel edge.
• If the distance between the open end node and its snap point on the graph is less than twice as long as the beeline distance, no point is shown.

The “snap points” layer shows the snap points of open ends helping to understand the situation.

Not all entries in the unconnected nodes layers are mistakes. Often, adding a linear barrier is helpful. The more blueish a point is, the less likely it is a mistake at all. Don’t feel forced to add noexit=yes to every point the OSM Inspector complains about. You are not mapping for the validator 😉