20.04.2020 | Frederik Ramm
In the last couple of weeks there have been two events that may have caused tile (or Nominatim) servers to stop updating. Here’s a quick outline on how to identify and repair the problems.
How old is my data?
A relatively fail-safe way to get an idea of hold old your data is, independent of the update mechanism you’re using (if any), is to run the PostgreSQL query
select max(osm_id) from planet_osm_point;
and then open the URL http://www.openstreetmap.org/node/XXXX (replacing XXXX with the number you got from PostgreSQL). The date that node was created (if the node is at version 2 or higher, you will need to look at the history to find out) reflects the currency of your database. (If you have a Nominatim database, then the corresponding query would be select max(osm_id) from placex where osm_type='N'; .)
If your database is older than a month,
then you will probably have to do a full new planet file import with osm2pgsql. Dropping the existing database and loading it afresh means that you will not be able to render tiles in the mean time. If this is not acceptable then you can import a planet file into a new database and rename that when it is done; to save disk space while doing this, some parts of the old database can be removed without impacting the tile rendering. The tables that can be removed are planet_osm_ways, planet_osm_rels, and planet_osm_nodes along with the “flatnodes” file if present.
If your replication got stuck around the 11th of April 2020,
then you probably suffer from an old bug in Osmosis, the software frequently used to control the downloading and preprocessing of updates. This bug triggers a Xerces parsing error if the compressed file has certain properties, which accidentally happened around noon UTC on 11th April. This bug can only affect you if you have “minutely” and world-wide updates from planet.openstreetmap.org, not if you run hourly updates or if your server doesn’t have world-wide data. You can get a current version of Osmosis without the bug at https://github.com/openstreetmap/osmosis/releases/download/0.47.4/osmosis-0.47.4.tgz and install it in the appropriate location depending on where your update process expects it. Replication should then continue normally.
Read more on help.openstreetmap.org.
If your replication got stuck around the 17th of April 2020,
then you likely have an osm2pgsql that uses a buggy libosmium version for polygon building which will enter an endless loop upon encountering certain very complex and invalid multipolygons, and such a polygon was created in the late afternoon on 17th April. If this is the case you will notice an osm2pgsql process running on your system since 17th April, using 100% CPU. There are several ways to fix this. If you are currently running osm2pgsql 0.94 or newer, or an older version but without “flatnode” storage, then you can build and install a recent osm2pgsql with a current libosmium version and install that; kill your running osm2pgsql process, and things should pick up nicely from there. If you are running a version of osm2pgsql that is older than 0.94 with “flatnode” storage, then you cannot simply install a current osm2pgsql because the format of the “flatnode” storage has changed. Installing a new osm2pgsql will mean you have to re-load the full planet database. If you want to avoid that, you have the option of building an old osm2pgsql matching your installed version but using a new libosmium version without the bug.
Read more on the OSM dev mailing list and GitHub.
Another option for fixing this particular replication problem is making sure that your replication process consumes updates up to and including at least 19th April, and processes them all on one go with osm2pgsql (after having done change simplification, i.e. the --simc parameter in osmosis) which would then mean that the broken data of 17th April gets overwritten by the fixed data of 19th April and you’ll be fine.
We can help
If you would like to enlist expert help in fixing your tile or Nominatim server – whether or not it has originally been configured by us – we’re happy to solve the issue for you.
