GEOFABRIK

Suche // Archiv

Das Buch

OpenStreetMap
Die freie Weltkarte nutzen und mitgestalten.
Von Frederik Ramm
und Jochen Topf.

Events

Karlsruhe Hack Weekend 18.-19. Februar 2017

FOSSGIS 2017 in Passau, 22.-25. März 2017

Blog

The OpenStreetMap Foundation is an English non-profit organisation created to support the OpenStreetMap project. It has about 500 individual members. The Foundation pays for the servers on which OSM is run, organises the yearly “State of the Map” conference, and last not least also has to deal with the legal aspects of running a world-wide, crowd-sourced mapping project. Foundation work is done by volunteers entirely (one of which is this author), but of course money is needed for hardware and hosting, accounting, legal fees, and various bits and pieces that help the volunteers do their jobs.

Since the Foundation’s last Annual General Meeting in September 2013, the Foundation is open to corporate members.

You can now be among the first who publicly show their support for the OpenStreetMap project by signing up as a corporate OSMF member. The membership fee is £1,000 per year (at current rates, that’s €1,200 and US$1,650). Your membership fees will help to keep OpenStreetMap’s servers running and ensure the continued success of the project.

Read more on the OSM Foundation blog.

Geofabrik, of course, has joined already, and we encourage our friends and clients to do likewise.

WMS Relaunch

13.01.2014 | Frederik Ramm

(Diesen Beitrag auf Deutsch lesen)

WMS servers are quite old-fashioned in many respects. The WMS specification was created in 2000, and the basics haven’t changed much – WMS essentially specifies what was then mostly a HTTP CGI request. The client says “give me this map, in this projection, for this extent, in this resolution”, and the server hands over a raster image. This is how web mapping was done before the age of tiles, or even vector data. Nonetheless, WMS is a key component in many professional GIS workflows to this day.

In fact, a long time ago when OpenStreetMap started, it did offer a WMS itself, but that was soon replaced by a tile-based map which is better suited for high-exposure situations.

At Geofabrik, we’re doing both – we have tile servers for public-facing, high-volume services, and we offer WMS for those who have different needs. We have been running our OpenStreetMap WMS for a while now (we’re selling the service starting at €35/month, see product web page). We have developed the Mapnik-based WMS server ourselves and it’s open source.

Setting up a WMS isn’t rocket science, but because the WMS needs to compute a new image for each request, you have to carefully adapt the map style and database queries to work fast enough. This is where we’ve spent quite some time optimising things, by creating suitable database indexes or computing simplified geometries for landuse areas or road networks. We’re now soft-launching our overhauled WMS, with new clients getting access to the new services and existing clients being upgraded one by one. Here’s a couple things that our WMS does:

True on-demand rendering – any projection

Every single image served by our WMS is rendered from fresh OSM data when you make the request. This means that the data you see is no older than a few minutes (apart from the simplified major roads and landuse areas used on large map scales). It also means that we will never do bitmap reprojection of pre-rendered data because the images is produced using the requested projection.

Germany in UTM32N - left, bitmap reprojection with tilted labels; right, Geofabrik WMS

If you are using other projections than the standard “Google Mercator” projection – say, UTM or Gauss-Krüger projections – then you will probably be familiar with maps where the labels are anything but straight. This happens when the WMS serves your image as a raster reprojection rather than a fresh vector rendering. It is faster but compromises image quality. We don’t do that.

High-resolution images for printing

We had offered a high-resolution “print” WMS for a while, but now we’re supporting user-set DPI so you can request a map in any resolution. Some desktop GIS systems, like Quantum GIS, will automatically re-request an image in a higher resolution if you make a printout, leading to excellent results.

Different resolutions served from the Geofabrik WMS.

Selectable layers

Because we render each image on demand, we can also switch individual layers from the OSM style on and off. This is something that we had to configure specifically for clients until now, but the new WMS uses the standard WMS layer selection methods to expose a couple of layers (like world boundaries, place names, the road network, landcover, buildings, POIs and so on). Your GIS client will let you decide which layers to request.

WMS layer selection - left, the Karlsruhe region without landuse; right, Düsseldorf without POIs

This feature is particularly useful for displaying a non-cluttered base map (e.g. without buildings and POIs). You can even use it to generate a transparent map of roads and place names for overlaying on aerial imagery.

WMS-Neuauflage

13.01.2014 | Frederik Ramm

(Read this in English)

WMS-Server sind eine ziemlich altmodische Sache. Die WMS-Spezifikation wurde im Jahr 2000 erstellt, und viel hat sich seitdem nicht verändert: Der Client sagt “gib mir diese Karte, in dieser Projektion, für diesen Bereich, in dieser Auflösung”, und der Server schickt ein Rasterbild zurück. So lief das mit Karten im Web, bevor es Tiles oder sogar Vektorkarten gab. Nichtsdestotrotz sind WMS-Dienste auch heute noch ein wichtiger Bestandteil professioneller Arbeit im GIS-Umfeld.

In den Anfangstagen von OpenStreetMap gab es sogar für einige Zeit einen “offiziellen” Projekt-WMS, der allerdings bald durch die heutige Tile-basierte Karte ersetzt wurde, die für vielbesuchte Webseiten besser geeignet ist.

Bei der Geofabrik haben wir beides – Tileserver für öffentliche, vielbesuchte Seiten, und WMS für Nutzer mit speziellen Anforderungen. Unseren OpenStreetMap-WMS gibt es schon einige Jahre (wir bieten ihn ab €35/Monat ab, Details siehe Produktseite). Der Mapnik-basierte WMS ist eine Eigenentwicklung; der Quelltext ist Open Source.

Einen WMS aufzusetzen, ist keine Zauberei, aber weil der WMS für jede hereinkommende Anfrage ein neues Bild berechnen muss, gilt es, Kartenstil und Datenbank-Abfragen so anzupassen, dass die Antworten schnell genug kommen. Hier wir einiges an Arbeit investiert und beispielsweise spezielle Datenbank-Indizes eingerichtet oder vereinfache Geometrien für das Straßennetz und die Landnutzungsflächen berechnet. Diesen generalüberholten WMS führen wir jetzt schrittweise ein – neue Kunden erhalten gleich den neuen Server, und Bestandskunden werden einer nach dem anderen umgestellt. Hier ein paar Dinge, die unser WMS kann:

Echtes Rendern auf Anfrage – in jeder Projektion

Jedes einzelne Bild, das unser WMS herausgibt, wird aus aktuellen OSM-Daten in dem Augenblick berechnet, in dem die Anfrage hereinkommt. Das heisst, dass die Daten nie älter als ein paar Minuten sind (ausgenommen die vereinfachten Straßen und Landnutzungsflächen, die wir auf den großen Maßstäben einsetzen). Es heisst aber auch, dass wir keine Bitmap-Umprojektion vornehmen müssen, weil wir die Bilder gleich in der angeforderten Projektion ausrechnen.

Deutschland in UTM32N - links eine Bitmap-Umprojektion mit schrägen Beschriftungen, rechts der Geofabrik-WMS

Wenn Sie eine andere Projektion als die Standard-”Google Mercator” einsetzen, zum Beispiel UTM oder Gauß-Krüger, dann kennen Sie sicherlich die typischen Kartenbilder, auf denen die Beschriftungen alles mögliche sind, nur nicht waagerecht. Das passiert, wenn der WMS das Bild als Reprojektion aus vorberechneten Rasterdaten herstellt, anstatt es neu aus Vektordaten zu berechnen – es geht schneller, ist aber zum Schaden der Bildqualität. Wir tun das nicht.

Hochauflösende Bilder für den Druck

Einen hochauflösenden Druck-WMS hatten wir schon eine Weile im Angebot, aber der neue WMS unterstützt jetzt direkt benutzerdefinierte DPI, so dass die Karte in beliebiger Auflösung abgerufen werden kann. Einige Desktop-GIS-Programme, wie zum Beispiel Quantum GIS, fragen das Kartenbild für den Druck automatisch erneut in hoher Auflösung ab und bieten so ausgezeichnete Druckqualität.

Unterschiedliche Auflösungen vom Geofabrik WMS.

Wählbare Layer

Weil wir jedes Bild auf Anfrage ausrechnen, können wir auch einzelne Layer ein- und ausschalten. Bisher mussten wir das für Kunden mit Sonderanforderungen einzeln konfigurieren, aber der neue WMS benutzt dafür einfach die Standard-Layerauswahl, die im WMS-Protokoll vorgesehen ist. So lassen sich Layer wie zum Beispiel Grenzen, Ortsnamen, Straßennetz, Landnutzung, Gebäude, POIs und so weiter getrennt an- oder abwählen. Die GIS-Software zeigt i.d.R. bei der Einbindung des WMS eine Layerauswahl an.

WMS-Layerauswahl - links die Region Karlsruhe ohne Landnutzung ,rechts die Düsseldorfer Altstadt ohne POIs

Diese Funktion ist besonders nützlich, wenn man eine etwas aufgeräumtere Basiskarte (zum Beispiel ohne POIs) anzeigen möchte. Man kann sie sogar nutzen, um ein transparentes Straßen- und Ortsnamen-Overlay über ein eigenes Luftbild zu legen.

The other day we were asked to quickly throw together an “offline map” for laptop use by crisis responders in the Philippines. We had a couple of hours to create something that was simple and rugged, something that ideally runs on any operating system without having to install software.

We knew that you can easily use OpenLayers or Leaflet plus a directory of pre-produced tile PNG images and then access that with a “file://” type URL from your local browser. This solution can even be run from an USB stick without any software installation, and any halfway recent browser works – be it OSX, Windows, Linux, or even more exotic operating systems. There are many more sophisticated solutions – storing vector data on the device gives you more interactivity and detail at a fraction of the disk space, or if using raster tiles, storing them in MBTiles format is less clumsy because it’s just a single file. All these options require some software installation though. So we decided to go with the old-style technology – but we made one minor tweak that made our solution much more useful on certain systems.

Because our set of tiles might be used on computers that use FAT file systems – crucially, most USB sticks or external USB drives will come formatted with that -, we had to take into account that FAT allocates space in full clusters. Even a small 500-byte PNG will consume one full cluster; and the cluster size on FAT systems can go up to 32 kB. With many high-zoom tiles being near empty, this means we’re wasting a lot of space.

The area we were exporting was about 2 GB of image data. The red and orange lines in the above image show how much disk space this would have used had we stored single 256×256 pixel tiles (the standard size) on a FAT system with 8 kB or 32 kB cluster size. Zoom level 18 tiles for the area in question would have taken 25 GB on a volume with an 8kB cluster size, and almost 100 GB (shooting out of the graph) with a 32kB cluster size.

Instead, we opted for making non-standard tiles with 1024×1024 pixels, 16 times as big as standard tiles. This puts more of the map in every single image, and reduces the waste incurred by “rounding up” to the nearest cluster size dramatically. OpenLayers accepts a “tileSize” parameter and processes the non-standard tiles flawlessly.

Unpacking our tiles down to z18 now only takes 6 GB on a volume with a 32 kB cluster size, and 3 GB if the cluster size is 8 kB – perfect for a “map-on-a-stick”!

If you find yourself in a similar situation – there are probably many ways to generate tiles like these, but we used Tirex which we configured to use large tiles (config option tilesize=1024), and then converted the resulting meta tiles to simple PNGs with the meta2tile utility. Remember that when doing this, you’ll only compute tiles down to a nominal zoom level of 16 which, because each tile has 1024×1024 pixels, gives you the same amount of detail that you’d normally see at zoom level 18. Then, in OpenLayers, you’ll have something like var map = new OpenLayers.Map('map', { tileSize: new OpenLayers.Size(1024, 1024) }) to work with these tiles.

(Hat tip to Matt Amos and Ian Dees on whose ideas we based this.)

(Deutsche Version in eigenem Artikel.)

Geofabrik is a household name in OpenStreetMap circles, but even insiders often ask us: What is it that you do at Geofabrik, apart from all the handy things for the community? What’s your business model?

We like to respond like this: At Geofabrik, we’re essentially craftspeople. Something like carpenters or cabinetmakers. Clients come to us and say “can you build me a cupboard”, and then we discuss the details with them, make them an offer, and when they like what we suggest, then we’ll build the cupboard. Of course the analogy only works so far – at Geofabrik, we might be able to build a standard cupboard for a client, which makes things easier. For another client we might, after a longer discussion, suggest that they get a nightstand instead because it serves them better, or we might offer to store their stuff in one of our cupboards. Or maybe we find out that they’re rather good at woodworking themselves and we just give them a blueprint for building their own cupboard… but what we like about the analogy is the the “honest work” aspect. We’re not “in the cloud”, we’re a solid engineering company, and we’re masters of our trade.

Since its inception in 2007, Geofabrik has done lots of different projects, sold products or services, and we’ve finally come up with a web page that presents a selection of them so that you can see for yourself what it is that we do.

For those that are interested further, Frederik has held a talk at the State of the Map US with an inside view of what it is like to do business in OSM.

And no, there aren’t any cupoards on that page, or in that talk.

Meta