2012-01-17

02:20 Wraithan joined #geoloqi
02:20 <Wraithan> aaronpk: I really like it so far, was going through it some more since I got home
02:42 <@ssf3dymn> ボンバーマンの爆弾再現て意外に難しいな。Mapattack仕込むと攻撃に付随するイベントスッ飛ばされるし。
07:39 Loqi joined #geoloqi
08:57 blk joined #geoloqi
09:47 kyledrake joined #geoloqi
10:07 patrickarlt joined #geoloqi
10:21 caseorganic joined #geoloqi
10:27 caseorganic joined #geoloqi
11:17 caseorganic joined #geoloqi
13:24 Schuyler_ joined #geoloqi
13:25 <Schuyler_> hellloooo
13:25 <Schuyler> aaronpk: sorry to have not answered your emails sooner
13:25 <Schuyler> I was on vacation
13:25 <Schuyler> how are you kids doing?
13:25 <aaronpk> hey!
13:25 <aaronpk> no worries
13:26 <Schuyler> I was going to send a writeup via email of how to deal with the OSM intersections but then I wondered if it mightn't be easier to describe on IRC
13:26 <Schuyler> probably not
13:26 <Schuyler> but I figured I'd come say hi anyway
13:26 <aaronpk> hehe well hello. hope you had a good vacation.
13:27 <Schuyler> so for all that the topological representation of OSM is a pain in the ass, it helps us here
13:27 <Schuyler> at SimpleGeo we did the computation of intersections using Hadoop
13:27 <Schuyler> basically, it's not even a geographic query
13:27 <Schuyler> you just create a big map of node ID -> way ID
13:28 <Schuyler> you could do it as a single-threaded in-memory process in Python or whatever
13:28 <Schuyler> or if you have some way of chunking up the OSM way data, you could do it with a bunch of processes and a Redis server, say
13:28 <Schuyler> in fact I think that's what I'd do
13:28 <Schuyler> using osmium
13:29 <aaronpk> I'm actually relatively unfamiliar with the raw OSM data. So a Way is a collection of Nodes?
13:29 <aaronpk> that together describe a street?
13:29 <Schuyler> an ordered set, exactly
13:29 <Schuyler> and the nodes themselves define teh coordinates
13:30 <aaronpk> so the Way is the thing that has the street name then?
13:30 <Schuyler> yep
13:30 <aaronpk> is looking at http://wiki.openstreetmap.org/wiki/Way for an example
13:30 <Schuyler> yep
13:30 <Schuyler> so the intersections are defined by node IDs
13:30 <Schuyler> which means it's a data processing problem, not a GIS problem
13:31 <Schuyler> anyway while you're at it, you also map way IDs to names
13:31 <Schuyler> then for each node, take the set of unique way names that it maps to
13:31 <Schuyler> if that set is larger than one item, then you have an intersection
13:31 <Schuyler> voila
13:32 <Schuyler> I do not know what the memory footprint of this problem is
13:32 <Schuyler> you might be able to do it in memory using Osmium's V8 integration
13:33 <Schuyler> as a single threaded process
13:33 <aaronpk> Guess it's not that important how long the initial import takes as long as the diffs can be processed relatively quickly.
13:35 <Schuyler> true
13:36 <Schuyler> apparently we used HBase at SimpleGeo to store the nodes for the Hadoop run
13:37 <Schuyler> I'm asking Kim Vogt to write a blog post about what she did :-)
13:38 <Schuyler> but that was the basic gist. let me know if anything is unclear!
13:38 <aaronpk> oh good idea, haha
13:38 <aaronpk> that makes sense. Guess it's just a matter of figuring out how to churn through all that data.
13:38 <Schuyler> have a look at Osmium
13:39 <Schuyler> it provides really good primitives for processing OSM data in JavaScript
13:39 <aaronpk> will do
13:41 <aaronpk> MapQuest has been doing a lot of stuff with OSM data recently too. I might suggest to them adding this to their API.
13:46 <aaronpk> oh sad, I just realized my SimpleGeo sample code from my Web Directions talk isn't going to work anymore
14:02 <Schuyler> doh
14:16 kyledrake joined #geoloqi
14:43 caseorganic joined #geoloqi
17:12 kyledrake joined #geoloqi
18:14 <@mpanighetti> Off to see some live @MythBusters! See you there maybe! http://t.co/VHlHa5wo
21:25 patrickarlt joined #geoloqi
23:12 kyledrake joined #geoloqi