Open Street Map Data

Posted on 27th April 2017

I'm currently working on using some address information from open street map to augment other open data sources. Here are some notes on using data from open street map, in Python.

Getting and using Open StreetMap Data

It seems like this is a bit of a pain. Open StreetMap (OSM) uses a custom, XML based, format which is hard/impossible for standard GIS software to read.

Data sources

  • http://wiki.openstreetmap.org/wiki/Planet.osm Gives links to download the world and various extracts of it.
  • http://wiki.openstreetmap.org/wiki/Overpass_API Details the "Overpass API" which allows targeted querying of data from the OSM database. (The OSMnx package seems great for making direct, small-scale queries.)

GeoFabrik

http://download.geofabrik.de/ Offer downloads of regions of the world, either in OSM format, or converted to shape-file format. The latter can be loaded into geopandas, QGIS etc. etc. but appears, sadly, to be missing useful information. For example, I can locate a building and find the polygon for this (by it's ID) in the shapefile, but there is no meta-data attached to give me e.g. the address!

However, for off the shelf analysis using existing tools, this is perhaps where to start.

Format

The standard format is XML, either compressed, or packaged using Protobuf.

  • http://wiki.openstreetmap.org/wiki/OSM_XML Gives the XML format
  • http://overpass-api.de/output_formats.html Gives details of the JSON format which the Overpass API can return.
  • http://wiki.openstreetmap.org/wiki/Map_Features Gives details of the "features" the returned data can contain.

Libraries

I should probably be using pyOsmium but there is no conda build, and being stuck with either a windows box, or a locked-down linux box, I am scared to try to get it to build.

GDAL / OGR can be easily installed (or come for free with geopandas).

  • http://www.gdal.org/drv_osm.html Gives some brief details
  • A cookbook for GDAL.
  • I didn't have a great deal of luck finding the details I wanted.

OSMnx

It seems that others have had similar thoughts to me. OSMnx looks to be a great tool for network analysis of road networks. The review paper is also very informative, with some nice background information.

With Anaconda, it's a one-line install:

conda install -c conda-forge osmnx

This seems very nice for targeted downloads, but it's a bit too "magic" for my tastes. The source code is very readable however, and I've been learning a lot from it.

Categories
Recent posts