Slashdot Mirror


Getting Started Developing With OpenStreetMap Data

Nerval's Lobster writes In 2004, Steve Coast set up OpenStreetMap (OSM) in the U.K. It subsequently spread worldwide, powered by a combination of donations and volunteers willing to do ground surveys with tools such as handheld GPS units, notebooks, and digital cameras. JavaScript libraries and plugins for WordPress, Django and other content-management systems allow users to display their own maps. But how do you actually develop for the platform? Osmcode.org is a good place to start, home to the Osmium library (libosmium). Fetch and build Libosmium; on Linux/Unix systems there are a fair number of dependencies that you'll need as well; these are listed within the links. If you prefer JavaScript or Python, there are bindings for those. As an alternative for Java developers, there's Osmosis, which is a command-line application for processing OSM data.

39 comments

  1. Nice to see a story about Technology by Crashmarik · · Score: 2

    And actually hacking. I think I may give this a try.

    1. Re:Nice to see a story about Technology by Anonymous Coward · · Score: 0

      The power of linguistic equivocation:

      'THINK' 'MAY' 'TRY
      vs
      'AM' 'GOING TO' 'NOW'

      Does anyone think he's going to do anything?

    2. Re:Nice to see a story about Technology by oodaloop · · Score: 1

      'THINK' 'MAY' 'TRY
      vs
      'AM' 'GOING TO' 'NOW'

      I am I going to give this a now?

      --
      Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
    3. Re:Nice to see a story about Technology by Crashmarik · · Score: 1

      The power of linguistic equivocation:

      'THINK' 'MAY' 'TRY
      vs
      'AM' 'GOING TO' 'NOW'

      Does anyone think he's going to do anything?

      I see why you made the effort to post anonymously

    4. Re:Nice to see a story about Technology by NoNonAlphaCharsHere · · Score: 2

      'HACK' or do not. There is no 'THINK'.

  2. Re:Seriously by Anonymous Coward · · Score: 0

    I would like to know how this story relates to Systemd

  3. Re:Sounds like dependency hell. by oodaloop · · Score: 1

    The 70's called looking for it...

    Yeah, I often misplace things 40 years in the future, or more sometimes! You know how it is, it's a busy day, my mind is somewhere else, and poof! I can't find my command line application. So I just look up the number for every year in the future and start calling. I've called a lot over the years. If someone told you the 80s called and want something back, I'm sorry for bothering you.

    --
    Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
  4. Re:Seriously by richlv · · Score: 1

    this site also used to have technical stories. like this one. so its actually a welcome change from "what would benet..." series.

    --
    Rich
  5. Re: Seriously by Anonymous Coward · · Score: 0

    Please make him contribute again.

  6. Work in the right direction by ADRA · · Score: 4, Informative

    For a project I'm working on, I started to play around with the OpenStreetMap data as a source for locations (from a guy who's never used GIS info systems), so I think I'd be a good insight into getting started with using this great resource.

    Notes:
    - I develop in Java mostly, but I have a generally well rounded skill set.

    Firstly, I had to make the jump to Postgress and PostGIS, which are annoying to setup if you're not familiar with them. I had a MySQL instance running, but for the life of me, I couldn't get osmosis to import before getting the setup just right, which unfortunately wasnt' as simple and stright forward as I'd have liked to see in any docs. So after finally banging PostGIS over the head enough to accept the import, I was hit with a huge knowledge gap on how to actually access spatial and hstore based data. Admittedly, once you get the handle of them, the SQL access the data is quite expressive and powerful.

    For DB imports, I used Osmosis for data import. I couldn't find any stand-alone Java based libaries for actually using the DB data which would help a lot (maybe I'll end up writing an open source one if it doesn't already exist). So, I basically dropped down to writing PostGIS based SQL queries, which is really quite expressive and well structured when the data is good (depends on the world region, mostly good for North America from what I found so far).

    Secondly, there was the OpenStreetMap data itself. As someone who primarily wants to work on geographic barriers and political boundaries, there's a big disconnect between the polygons of the system and the political ones. Generally, there's always a node (think of a pin on a map) to represent a proper place name (New york city for instance) and a polygon that encompass what New York's political boundaries are, but quite often there won't be explicit ties between the two, so you're left with bridging the two yourself constructing queries for where nodes are within city / state / country / etc.. Anyways, thats as far as I've gotten so far, so good luck!

    Some links that helped me:
    http://www.postgresql.org/docs...
    http://postgis.net/docs/manual...
    http://wiki.openstreetmap.org/... (Make sure to read carefully, becase its rather unforgiving and terse about bad environment setups)

    --
    Bye!
    1. Re:Work in the right direction by richlv · · Score: 1

      wouldn't those place properties be usually grouped by a relation like this one ?
      http://wiki.openstreetmap.org/...

      --
      Rich
    2. Re:Work in the right direction by tom17 · · Score: 1

      Sounds like we have gone through similar. I too am working on a project and had to take a 'detour' while I learned about GIS. Also ending up with PostGIS, osm2pgsql and huge scads of time looking around the OSM data trying to work out route data & relations.

      It's been a taxing ride, but I think I have what I need from the GIS side of things now and can get back to the main part of the app. GIS is a tricky subject field and learning which areas you need to learn is a daunting task in itself when you are a complete newb.

    3. Re:Work in the right direction by ADRA · · Score: 1

      Yeah, relations capture the net of the geographic region, but I find the individual node's contain better hierarical town/region/state/country layering which was more correct (at least in Canada where my test data was running against). Politically based relations are great when they exist, but far too often you'd have a relation representing X but no explicit connection besides the fact that the node was geographically found in the region. You may have a relation, but you often don't have relations that point to containing relations which is part of what my specific use case requires. If it isn't done upstream, it means more work for people like me to piece together a hierarical relation based on political or simple associative connections. Eg. The Bronx is in New York, but there's no explicit link in the data to describe that association.

      --
      Bye!
    4. Re:Work in the right direction by Anonymous Coward · · Score: 0

      In a spatial sense, containment is explicit.

      From a social POV, there is quite a lot of resistance to injecting things that can be easily mechanically determined (such as containment) into the OSM database. I think this resistance is sensible. The alternative is that you build the mechanical system anyway to make sure that the data you have is reasonable.

      There is certainly room for people to extract and process such things into convenient formats, but there are too many people using geo data to just stick everything that all of them need into the database, it is a good thing to stick to data that benefits from human attention.

    5. Re:Work in the right direction by Glassman33 · · Score: 1

      Boundless has a simple install for PostGIS along with Postgresql. I just reinstalled PostGIS using Boundless OpenGeo Suite on a Macbook Pro after replacing the hard drive. An easy installation. Place names are a problem. OSM continues to debate whether admin boundaries should be included. Right now we are good down to city level. Small localities such as neighborhoods and unincorporated hamlets are often just a node. Since OSM attempts to document ground truth, it is hard to create a polygon around an area that no one knows where it extends to. The good news is that OSM is open to anyone to edit. If information is missing, you can add it. Just make sure not to copy other peoples work.

    6. Re:Work in the right direction by Anonymous Coward · · Score: 0

      > from a guy who's never used GIS info systems

      This is more difficult that using PIN identification numbers in ATM teller machines but you may be able to figure it out.

    7. Re:Work in the right direction by nostreettype · · Score: 2

      After 16 years of lurking I've finally created an id :-) I'm the primary author of Osmosis. The parent mentioned that there are no standalone java libraries available. This isn't strictly true. Osmosis exposes a command line interface and this is how most people use it, however it is built as a set of libraries that can be included as required in other Java applications. These libraries are all published to Maven Central. http://search.maven.org/#searc... Osmosis documentation is available on the Open Street Map wiki. http://wiki.openstreetmap.org/...

  7. Useless by Anonymous Coward · · Score: 0

    Their bus stops are all out of date for where I live

    1. Re:Useless by gbjbaanb · · Score: 1

      Their bus stops are all out of date for where I live

      its very easy to go in and update them. So you can't blame anyone else but yourself for failing to help yourself with a resource someone else provided to you for free.

    2. Re:Useless by Anonymous Coward · · Score: 1

      What, this is free? Then I must be the product. Yuck. Screw this scam.

    3. Re:Useless by harrywood · · Score: 1

      Well yeah it's a wiki approach, so you can always fix problems yourself. This is always true although there's a bit of learning curve.

      An easier thing is to simply report a problem with the data. There's a neat feature on OpenStreetMap.org called "notes" to let you do this. Read about it here http://wiki.openstreetmap.org/... Maybe some active OpenStreetmap community folks will spot this and resolve the problem.

      An even easier thing is ...while grumbling about useless data "where I live" ...you might actually mention where you live, so that somebody here might pass on the information or tackle the problem. Where are these out-of-date bus stops?

      OpenStreetMap is a not-for-profit collaborative project to create a free and open map of the world. If you're going to grumble about it, at least do so in a vaguely useful way.

    4. Re:Useless by gbjbaanb · · Score: 1

      I was referring the OP to it. I have updated OSM data for my local area when I found it was lacking absolutely vital information on the location of the local pubs. I also helped my work colleague fix a road section as it wasn't giving him the correct satnav details for his drive home.

      OSM is awesome because I can do these things. Its not difficult to do either, just enough of an impediment to deter those who might want to deface the data.

  8. Re: Seriously by Anonymous Coward · · Score: 0

    Hear hear! Bring back Bennett! It was the only thing keeping this website interesting.

  9. GIS Developers by BasilBrush · · Score: 0

    As a developer of GIS systems you can earn very good money indeed. Why would anyone want to do it for free?

  10. Pi by idji · · Score: 1

    I was doing some development with OSM and was using my own formulae to convert from latitude/longitude to OSM tile coordinates. I spent days getting the wrong results until I realised the problem was pi. i had been using pi=3.1415926535. It turns out that that is not GOOD enough to zoom into house level. I added 5 extra 5 or so digits of PI and then all my tiles were correct to the house level. I was awed that for a simple, practical task i needed 15 digits of pi!

    1. Re:Pi by ncc74656 · · Score: 2

      I spent days getting the wrong results until I realised the problem was pi. i had been using pi=3.1415926535

      Why are you trying to represent an irrational number with a rational number of unnecessarily limited precision? If pi isn't defined as a constant in whatever language you're using, calculate it yourself and store it in a variable for future reference. 4*atan(1) is fairly common and simple for this purpose, and you'll get as many digits as the underlying datatype will support.

      --
      20 January 2017: the End of an Error.
    2. Re:Pi by Anonymous Coward · · Score: 0

      There's actually code snippets to do that calculation in a lot of different languages on the OpenStreetMap wiki: http://wiki.openstreetmap.org/wiki/Slippy_map_tilenames ...and yeah these code snippets all use the language's built in constant for PI, or for languages which don't have that, they use a lot of digits of precision.

  11. OSM viewing widget. by Rufty · · Score: 1

    I'm playing around with viewing OSM data. I like the MarbleWidget. (And despite the KDE link, it only needs Qt, not the whole KDE Stack.)

    --
    Red to red, black to black. Switch it on, but stand well back.
  12. It's data by Anonymous Coward · · Score: 0

    No need to converge on some phantom OSM paradigm. It is just data, you can use anything you want for the .dev end of it. OSM has lots of interesting tolls, most of which could use a lot more mature approach of an enterprise, mostly on the data quality verification and maintenance end, as wholesale breakage is not prevented just as in Wikipedia.

  13. OSM for practical navigation by sonic71789 · · Score: 2

    I'm all for using open source software and information but it has to be at least 90% as good as the commercial alternatives especially if those commercial alternatives are reasonably priced. I'm going on a trip to Las Vegas, a city with extremely complex infrastructure including multiple monorail systems, a public bus system, complex streets, taxi and bus specific lanes, and a high density of shops and restaurants residing on multiple levels in extremely large buildings. I figured it would be the perfect opportunity for OSM to show off the best it has to offer.

    Google appears so overwhelmingly better at handling all of those complex mapping scenarios to the point where OSM can't be seriously considered for people trying to use it to navigate that city. Most business data is out of date, public transit stops aren't as detailed, and it doesn't have the internal structures of buildings that take up multiple city blocks.

    Interestingly enough my own city's OSM data is top notch, easily surpassing Google's data. Clearly there seems to be wide range of map data in the database, but I would think large cities that people travel to often seems like the place you really want high quality data and visualization if you want non-ideological users to switch. Had I not looked at my own community I would have written off OSM as a totally underdeveloped open source project with no chance of competing, and it seems like that's the exact opposite of the potential it has, but clearly there needs to be more outreach in some larger cities.

    1. Re:OSM for practical navigation by Rudisaurus · · Score: 1

      Google appears so overwhelmingly better at handling all of those complex mapping scenarios to the point where OSM can't be seriously considered for people trying to use it to navigate that city.

      Interestingly, for one of my favourite mapping application -- establishing coastal sea-kayaking routes -- Google Maps is worse than useless. Google fuzzes out the offshore regions, leaving large blank areas where I know that islands/archipelagos exist. OSM does not do this, making it a much better tool for the job.

      --
      licet differant, aequabitur
    2. Re:OSM for practical navigation by healyp · · Score: 1
      Part of the reason that Public Transit is so good on Google Maps is that Google developed, pushed and helped municipalities adopt the standards (GTFS) they came up with to represent transit data: https://developers.google.com/...

      NYC MTA developer resources link: http://web.mta.info/developers...

  14. try and helpfull by zhouchibo · · Score: 0

    I think i will try this,maybe it will be helpfull to my site http://www.bagswalletsforsale....

  15. OSM needs a usable web front end by Anonymous Coward · · Score: 0

    The current one is awful. Most features are not clickable and don't have tooltips or any other discoverable way to access metadata. Icons are not explained. Layers are not customizable at all. I know it's all open source, but then why is nothing happening on the front end?

  16. you are the product? by harrywood · · Score: 1

    "if something is free, you are the product" ... is a pretty good rule of thumb. It's only sensible to have a healthy level of cynicism about websites and companies operating them, especially websites which are asking to volunteer your time towards contributing to a dataset. There are countless websites operated by profit-making companies, which do this, and then don't behave well. The most basic cardinal sin here is not giving the data back under an open license. This applies to geo datasets with things like Google Map Maker and Waze. "you are the product" is very much true in this case. So yes, be cynical. Before spending time on these things, look into it carefully, and don't contribute to projects which are not treating contributors fairly.

    But if you look into OpenStreetMap carefully, you'll find a similar set-up to Wikipedia. You'll find everything is released with an open license. You'll find a not-for-profit organisation with oversight over servers and funding. You'll find a large open community of volunteers who are just trying to make the world a better place by building something great.

    Be cynical. Be suspicious. Contribute your time wisely. Contribute to OpenStreetMap.