4.05.2018 | Frederik Ramm
As mentioned two months ago, we have made some changes to our download server at download.geofabrik.de. The publicly available files now don’t contain user or changeset information any longer (see below for details). You can still get everything that was there before, but files that do contain user information were moved to a server that requires you to have an OSM login.
The public server continues to be download.geofabrik.de, and the URLs and paths have been unchanged. The new server that requires a login is called osm-internal.download.geofabrik.de, and while it looks largely the same as the public server, the files it offers will have a name component of -internal in order not to confuse them with the public files.
Here’s a table that explains which files to get where:
| public server | internal server | |
|---|---|---|
| .osm.pbf files for all regions (original OSM data) | yes, without user data | yes, with user data |
| .osc.gz files (diff updates) | yes, without user data | yes, with user data |
| .osh.pbf files for all regions (original OSM data with full history) | no | yes, with user data |
| .osm.bz2 files (old XML OSM data format) | yes, without user data | no |
| .shp.zip files for all regions (ESRI shape files) | yes | no |
The change was originally made on 03 May. Unfortunately osmosis and older versions of osmium, and all programs depending on them (e.g. osm2pgsql < 0.96.0) don’t work with PBF files lacking metadata fields. They fail to read PBF files which don’t have any username, user and changeset field set. To work around that, starting 05 May, we’re now writing empty strings to all user fields and zeros to all user and uid fields.
OSM extracs and diffs with full metadata are still available but you have to log in with your OSM account.
For security reasons, it is not enough to pick the cookie out of the developer console of your web browser. The cookie will become invalid after 48 hours and the server will redirect you to authorization form of openstreetmap.org if you send that cookie after it has expired. That’s why you have to retrieve a new cookie every 48 hours or any other time the server forces you to do so.
We have published a client program written in Python (read the documentation) automating the cookie retrieval. The repository containing the client also contains the server software if you are curious or want to use it on your own server providing sensitive data.
This blog post may be updated as things develop.
