Blog

This month we’ve ramped up the infrastructure behind the download server, and downloads should now be available earlier
and faster. There’s also a small technical change in that requests for a “…latest” file will now be answered with a
HTTP redirect to the specific latest version (see previous blog post).

I would like to use this opportunity to appeal to users of the download server to “download responsibly”. We want
to continue offering this service as powerful and as convenient as possible within our means. We want everyone
to have easy access to the latest OSM data in a form that is useful to them.

Every now and then, people break things for others. There have been individual clients downloading the exact same
20-GB file 100s of times per day, for several days in a row. (Just the other day, one user has managed to download almost 10,000 copies of the italy-latest.osm.pbf file in 24 hours!) Others download every single file we have on the
server, every day. There’s a limit to the outgoing network bandwidth, and behaviour like this means that
things are slowing down for everyone. Also, when we block an IP range for abuse, innocent third parties can be affected.

Here’s three concrete appeals to users of the download server:

1. If you want data for the whole planet, don’t download it piecemeal from us – simply get the planet file from planet.openstreetmap.org and you’re done!
2. If you want a large region (like Europe or North America) updated daily, use the excellent pyosmium-up-to-date program which will automatically determine the age of your local file and update it by downloading the latest changes; this saves something like 98% of network traffic compared to a fresh download, and is faster.
3. If you automate anything with regard to our download server, monitor what your script is doing or build in appropriate catches so that you don’t end up downloading the same file 1000 times just because your disk is full or something like that.

Happy downloading!

The Geofabrik dowload server is a wildly popular service that each month pushes out about 400 TB of OpenStreetMap data to users around the world – most of that in the form of country or regional .osm.pbf files. We publish these files with a date in the filename (e.g. germany-250723.osm.pbf) but for convenience we also have a downloadable file like “germany-latest.osm.pbf” that always gives you the most recent version.

Keeping these two as separate files does, however, interfere with caching; the proxies we use to speed up the download offering don’t know that “something-250723” and “something-latest” are the same file so they need to retrieve them both.

Starting 01 September, we’ll change this in line with best practices, and send a HTTP redirect if you request a “latest” file. The redirect then points the client to the appropriate URL with a data embedded in the file name. This should be transparent for most use cases, however if you have some sort of setup that does not follow HTTP redirects you might have to modify that. wget will follow redirects by default, whereas curl requires the command line flag -L.

Geofabrik is pleased to support the OpenStreetMap community as a bronze sponsor of two upcoming conferences in 2025:

• State of the Map Global – Manila, Philippines
• State of the Map Europe – Dundee, Scotland, UK

We have been regular sponsors of SotM events for many years, including regional conferences such as FOSSGIS. Our staff have also contributed by speaking at various sessions over time.

Events like SotM offer a useful opportunity for exchange within the OpenStreetMap ecosystem. While much of the work in the OSM world happens remotely, occasional in-person meetings remain valuable for maintaining collaboration and perspective.

Sponsorship opportunities are still available for both events for those interested in supporting the community.

In der letzten Woche waren drei von uns – Amanda, Michael und Frederik – auf der FOSSGIS-Konferenz in Münster. (Die Geofabrik ist auch Sponsor der FOSSGIS-Konferenz.) Die Veranstaltung, die jedes Jahr interessante Vorträge und Gespräche rund um alles im Open-Source-GIS-Bereich bietet, hatte dieses Mal rund 750 Besucher:innen und fand in den denkmalgeschützten Räumen im Schloss statt.

Alle Vorträge wurden vom CCC-Videoteam aufgezeichnet und können auf https://media.ccc.de/c/fossgis2025 angesehen werden. Amanda hat über ihr Projekt WaterwayMap gesprochen; Michaels Thema war, wie man in Deutschland noch neue Mapping-Aufgaben findet, und Frederik hat einen neuen OSM-Datenabfragedienst namens Postpass vorgestellt.

Der FOSSGIS e.V. hat auf seiner Mitgliederversammlung beschlossen, dass er Unternehmen und andere Organisationen, die OpenStreetMap nutzen, als „Förderer“ gewinnen will, um mehr Planungssicherheit für seine OpenStreetMap-Aktivitäten zu erreichen. Die Geofabrik hat natürlich gleich ja gesagt – wir sind die ersten! Unterstützen auch Sie die wertvolle OSM-Arbeit des FOSSIGS e.V., indem Sie Förderer werden.

We’re happy to announce that we now have a nice public transport map, courtesy of Melchior Moos who runs the original öpnvkarte.de web site. Geofabrik now hosts its own version of that original “ÖPNVKarte” (“ÖPNV” is the German acronym for public transport), and we’ll be offering it to international customers under the somewhat simpler name “Public Transport Map”.

We’ll also approach the OpenStreetMap operations team and recommend this map for inclusion on www.openstreetmap.org which has featured ÖPNVKarte for a long time and only recently dropped it due to operational concerns.

This public transport map currently relies on a proprietary database importer written by Melchior Moos, but using the work contributed by our recent Geofabrik intern Max Hoogerdijk, we’re planning to publish import scripts compatible with osm2pgsql’s Themepark import mechanism under an open source license soon.

We frequently receive requests from researchers about “old” OpenStreetMap data. In one instance we’ve been asked whether we could provide data for a city road grid – for the year 1980! So let’s take a minute to explain what is possible and what isn’t.

In general, you can download older data files – for the 1st of January of each year – from our download server. Just click on the “raw directory index” link and you’re presented with a list of available files – either .osm.pbf or shape files. Sometimes when we’ve split up a country in smaller regions at some point, you might only find more recent files for the small regions, but older files for the whole country.

OpenStreetMap keeps a full history of everything, so it is possible to extract even more fine-grained snapshots – say for the 1st of each month since 2014! This will require the “osmium” utility (with its –time-filter option), as well as a “full history” file. Such files are available (for the whole world) from planet.openstreetmap.org, but also for individual continents or countries from our project-internal download server. (Because these files contain metadata that may be personal data, access is only allowed with an OSM login.)

There are a number of caveats that apply to working with historic OSM data though. First and foremost: An OSM data set of time X tells you what OSM knew about the world at that time – and not what the world was like at that time. If a building has been there for 100 years but was only added to OSM in 2020, then the 2018 data set will not contain that building. You can get an idea of what to expect in an old OSM data set from the “chronology” tab of the taginfo service (globally at taginfo.openstreetmap.org, or for individual regions at taginfo.geofabrik.de). Here’s a graph of the “building” tag for Spain:

As you can see, OSM currently has about 4.5 million building objects in Spain, and in 2014 we had about 450,000. Does that mean that 90% of Spain’s buildings have been erected in the last 10 years? No – it only means that OSM was much smaller in 2014 than it is today!

Because OpenStreetMap was invented in 2004, obviously any data export for any time before 2004 will be completely empty. In the early years we also had a few incompatible data model changes, and the license change in 2012 meant that some data had to be removed again. So, going back further than the license change will really give you very patchy results, and is not recommended for most use cases.

Another thing to consider is that in some areas the mapping methods have changed, and something that was mapped as X in 2014 might today be Y. Depending on what research you are doing, it might be necessary to adapt your filters to that.

If you would like to enlist our help in extracting historic information from OpenStreetMap, we’re happy to make you an offer.

We’re now offering Shortbread vector tile packages on our download server. We have been doing this experimentally for a while, serving .tar.gz files, but have now switched to the more popular “mbtiles” format.

At present you will find download links for country-wide vector tile packages, except for large countries like the US where the packages are at state level. We’re looking into providing continent-wide and planet-wide downloads but these will likely utilize BitTorrent to keep some of the load off our servers.

Data is currently updated infrequently but we’ll sooner or later establish automatic, regular updating.

Our project-internal download server which supplies OSM data with full metadata has been upgraded to use OAuth2, in anticipation of OSMF switching off their support for OAuth1.0a.

(For those who are not aware – the public “no login required” download server doesn’t publish the author and timestamp of modifications in order to be squeaky clean when it comes to data protection; the project-internal server requires an OSM login, thereby ensuring that anyone downloading data from this server has consented to OSM’s policies which, among others, include a data protection clause.)

In fact, as some of you may have noticed, we had been using OAuth1.0, not 1.0a, until now, which was unceremoniously given the boot last weekend, leading to a service interruption of a few days on the internal download server.

It’s all fixed now, and those of you who use some sort of automated download with a cookie will have to update their oauth_cookie_client.py and obtain a new cookie as documented on the GitHub page.

The regional taginfo server at taginfo.geofabrik.de has been updated with the newest taginfo codebase which means, among other things, that you can get a “chronology” view for many tags telling you how the use of that tag changed over time.

For those of you who have used the service in the past and who were probably used to frequent “internal server errors” forcing a reload, those toothing pains should now be a thing of the past, and we’ve added direct links to the respective regions on download.geofabrik.de – taginfo should have pages for every region supported by the download server. Feel free to report any issues you still encounter.

For quite a while now we’ve been offering a region called “Alps” on our download server and simply assigned it a rectangular bounding box. There have been some legitimate complaints about this box missing some bits in the Southwest (notably the Maritime Alps). We’re therefore changing the boundary polygon, but in order to keep the size manageable, we’re also getting rid of some of the areas that were included until now but are definitely not part of the “Alps” by any definition. Here’s the old (rectangular) and the new (potato-shaped) areas:

Notably the old file contained all of Bern, Zürich, Munich, Linz, Venezia, Zagreb, and Milano; these cities are not in the new “Alps” any more.

We’ll be making the change on September 25 which will lead to a relatively large diff, deleting all the stuff that isn’t in the file any more and adding all that is new. If you somehow rely on the old polygon, you’ll have to stop updating now.