{"id":268,"date":"2013-11-19T14:55:16","date_gmt":"2013-11-19T14:55:16","guid":{"rendered":"http:\/\/blog.geofabrik.de\/?p=268"},"modified":"2013-11-19T14:55:16","modified_gmt":"2013-11-19T14:55:16","slug":"map-on-a-stick-but-always-mind-the-cluster-size","status":"publish","type":"post","link":"https:\/\/blog.geofabrik.de\/index.php\/2013\/11\/19\/map-on-a-stick-but-always-mind-the-cluster-size\/","title":{"rendered":"Map on a stick &#8211; but always mind the cluster size"},"content":{"rendered":"<p>The other day we were asked to quickly throw together an &#8220;offline map&#8221; for laptop use by crisis responders in the Philippines. We had a couple of hours to create something that was simple and rugged, something that ideally runs on any operating system without having to install software.<\/p>\n<p>We knew that you can easily use OpenLayers or Leaflet plus a directory of pre-produced tile PNG images and then access that with a &#8220;file:\/\/&#8221; type URL from your local browser. This solution can even be run from an USB stick without any software installation, and any halfway recent browser works &#8211; be it OSX, Windows, Linux, or even more exotic operating systems. There are many more sophisticated solutions &#8211; storing vector data on the device gives you more interactivity and detail at a fraction of the disk space, or if using raster tiles, storing them in MBTiles format is less clumsy because it&#8217;s just a single file. All these options require some software installation though. So we decided to go with the old-style technology &#8211; but we made one minor tweak that made our solution much more useful on certain systems.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.geofabrik.de\/wp-content\/uploads\/2013\/11\/tilesize-1.png\" alt=\"\" title=\"Tile Size and Cluster Size\" width=\"504\" height=\"360\" class=\"alignnone size-full wp-image-270\" srcset=\"https:\/\/blog.geofabrik.de\/wp-content\/uploads\/2013\/11\/tilesize-1.png 504w, https:\/\/blog.geofabrik.de\/wp-content\/uploads\/2013\/11\/tilesize-1-300x214.png 300w\" sizes=\"(max-width: 504px) 100vw, 504px\" \/><\/p>\n<p>Because our set of tiles might be used on computers that use FAT file systems &#8211; crucially, most USB sticks or external USB drives will come formatted with that -, we had to take into account that FAT allocates space in full clusters. Even a small 500-byte PNG will consume one full cluster; and the cluster size on FAT systems can go up to 32 kB. With many high-zoom tiles being near empty, this means we&#8217;re wasting a <em>lot<\/em> of space.<\/p>\n<p>The area we were exporting was about 2 GB of image data. The red and orange lines in the above image show how much disk space this would have used had we stored single 256&#215;256 pixel tiles (the standard size) on a FAT system with 8 kB or 32 kB cluster size. Zoom level 18 tiles for the area in question would have taken 25 GB on a volume with an 8kB cluster size, and almost 100 GB (shooting out of the graph) with a 32kB cluster size.<\/p>\n<p>Instead, we opted for making non-standard tiles with 1024&#215;1024 pixels, 16 times as big as standard tiles. This puts more of the map in every single image, and reduces the waste incurred by &#8220;rounding up&#8221; to the nearest cluster size dramatically. OpenLayers accepts a &#8220;tileSize&#8221; parameter and processes the non-standard tiles flawlessly.<\/p>\n<p>Unpacking our tiles down to z18 now only takes 6 GB on a volume with a 32 kB cluster size, and 3 GB if the cluster size is 8 kB &#8211; perfect for a &#8220;map-on-a-stick&#8221;!<\/p>\n<p>If you find yourself in a similar situation &#8211; there are probably many ways to generate tiles like these, but we used <a href=\"http:\/\/wiki.openstreetmap.org\/wiki\/Tirex\">Tirex<\/a> which we configured to use large tiles (config option <code>tilesize=1024<\/code>), and then converted the resulting meta tiles to simple PNGs with the <a href=\"https:\/\/github.com\/openstreetmap\/mod_tile\/tree\/master\/extra\">meta2tile<\/a> utility. Remember that when doing this, you&#8217;ll only compute tiles down to a nominal zoom level of 16 which, because each tile has 1024&#215;1024 pixels, gives you the same amount of detail that you&#8217;d normally see at zoom level 18. Then, in OpenLayers, you&#8217;ll have something like <code>var map = new OpenLayers.Map('map', { tileSize: new OpenLayers.Size(1024, 1024) })<\/code> to work with these tiles.<\/p>\n<p><em>(Hat tip to Matt Amos and Ian Dees on whose ideas we based this.)<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The other day we were asked to quickly throw together an &#8220;offline map&#8221; for laptop use by crisis responders in the Philippines. We had a couple of hours to create something that was simple and rugged, something that ideally runs on any operating system without having to install software. We knew that you can easily [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/posts\/268"}],"collection":[{"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/comments?post=268"}],"version-history":[{"count":0,"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/posts\/268\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/media?parent=268"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/categories?post=268"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.geofabrik.de\/index.php\/wp-json\/wp\/v2\/tags?post=268"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}