Edit: New Post PokemonGO – Mapzen Post (26.07.2016)
This time I have something a little bit different; OpenStreetMap in Unity3D!
If you don’t know what OpenStreetMap is (well I didn’t until last week); it’s a crowd sourced map api created and supported by what seems to be a quite big community. So you can think of it as an open source (and free) Google/Bing Maps. And it gets even better, there are many other smaller projects/services built upon OpenStreetMap data, specializing in certain fields like metropol maps, metro maps etc. All these data under ODb License. How awesome is that?
So that is pretty much how I found about Mapzen and their Vector Tile Service, which by their own description “provides a free & open vector tile service for OpenStreetMap base layer data, with worldwide coverage updated daily, available in GeoJSON, TopoJSON, and binary data formats”.
I won’t go into detail about their service, format and everything but you can read all about it in their wiki page. What I’ll do though, is a walkthrough of parsing and visualizing their data in Unity3D so when we’re done, it’ll look something like this;
On the left, you can see the original OpenStreetMap view and on the right, it’s the same data in triangles and debug lines… well yea I know I kinda suck at visualization.
Some bullets before we get started;
- Note that I’ll just go over some important parts of the chode here, not the whole thing. You can find the project files at the bottom of the page anyway.
- We’ll have to use three different coordinates systems and convert values back and forth to make this happen. First of all, we obviously have longitude & latitude, then we have OpenStreetMap tile coordinate system (origin at top left) and finally the 3D space coordinates in Unity3D, starting at bottom left (XZ plane). So you’ll see lots of conversion method calls and y axis sign chances through out the code. Becareful with those.
- You can use this nice tool to find the tile you need; PHPCalc for OpenStreetMap . It’s a must-have tool if you’re planning to use OpenStreetMap in your project.
Also TMS (Tile Map Service) might be a little bit confusing at the begin but it’s actually just a grid system on a mercator projection. Only thing is, this grid has different detail levels. At level 1, it divides whole thing into 4 sections, at level 2, it’s 16 etc. Try these and you’ll get the idea; (0,0,1), (0,1,1), (1,0,1), (1,1,1). Those are x coordinate, y coordinate and detail level respectively. As you can see, just those 4 tiles covers up the whole world map in level one but that number increases really fast as the detail level increases.
For this post, I’ll use detail level 14 and 9685/6207 coordinates (yeap, my neighbourhood); soΒ (9685,6207,14).Β But of course, that’s just an image of the data they provide, the real thing is actually here. That’s actually a readable link and goes like, “get me the building and road data for detail 14 at 9685×6207 coordinates”. I highly suggest using a JSON reformatter to be able to read it though (i.e. http://jsonviewer.stack.hu/).
So let’s get started;
public IEnumerator CreateTile(World w, Vector2 realPos, Vector2 worldCenter, int zoom)
{
var tilename = realPos.x + "_" + realPos.y;
var tileurl = realPos.x + "/" + realPos.y;
var url = "http://vector.mapzen.com/osm/water,earth,buildings,roads,landuse/" + zoom + "/";
JSONObject mapData;
if (File.Exists(tilename))
{
var r = new StreamReader(tilename, Encoding.Default);
mapData = new JSONObject(r.ReadToEnd());
}
else
{
var www = new WWW(url + tileurl + ".json");
yield return www;
var sr = File.CreateText(tilename);
sr.Write(www.text);
sr.Close();
mapData = new JSONObject(www.text);
}
Rect = GM.TileBounds(realPos, zoom);
CreateBuildings(mapData["buildings"]);
CreateRoads(mapData["roads"]);
}
This is the entry point for whole process. Let’s have a look at it; it returns IEnumerator because I’m using WWW object and calling this function in a coroutine. Then we have 4 parameters; World is my main script to handle environment related stuff, not really necessary here. realPos is the x,y coordinates of the tile in TMS system (so 9685,6207). worldCenter is where we’ll create this tile in Unity3D space (it’s Vector2 as I’m using Y=0 by default). And zoom is the detail level.
Rest of it should be quite straight forward, I’m downloading and caching the JSON data. I believe there is already a WWW object with caching but I did it myself. This way it’ll be easier to chance database (text, xml, sql) whenever I want. So if we already have the data for that tile, it’ll re-use the old data file. If not, it’ll just download the data from the mapzen server.
I also used a third party library to parse JSON in C# (see JSONObject above). It’s nothing fancy really, just parses the data and creates something like a nested dictionary for it, which simplifies searching stuff.
And finally we have two big function calls; CreateBuildings and CreateRoads, taking related JSON sections, mapData[“buildings”] and mapData[“roads”], respectively. You can find those sections in the map Json and again, for your own sanity, use a JSON formatter.
Now let’s have a look at the CreateBuilding function;
private void CreateBuildings(JSONObject mapData)
{
foreach (var geo in mapData["features"].list.Where(x => x["geometry"]["type"].str == "Polygon"))
{
var l = new List<Vector3>();
for (int i = 0; i < geo["geometry"]["coordinates"][0].list.Count - 1; i++)
{
var c = geo["geometry"]["coordinates"][0].list[i];
var bm = GM.LatLonToMeters(c[1].f, c[0].f);
var pm = new Vector2(bm.x - Rect.center.x, bm.y - Rect.center.y);
l.Add(pm.ToVector3xz());
}
try
{
var center = l.Aggregate((acc, cur) => acc + cur) / l.Count;
if (!BuildingDictionary.ContainsKey(center))
{
var bh = new BuildingHolder(center, l);
for (int i = 0; i < l.Count; i++)
{
l[i] = l[i] - bh.Center;
}
BuildingDictionary.Add(center, bh);
var m = bh.CreateModel();
m.name = "building";
m.transform.parent = this.transform;
m.transform.localPosition = center;
}
}
catch (Exception ex)
{
Debug.Log(ex);
}
}
}
This is rather straight forward actually. I read vertex data for each building from the json object and create a mesh using those vertices using BuildingHolder class. Vertex coordinates are in lat/long system originally so first I convert them to meters (GM.LatLonToMeters call), then I substract the tile center from that as I attach buildings to tiles as children. I keep buildings inside a dictionary using their center point as key which isn’t a good idea and might cause problems with complex polygons (when a big concave polygon surround a smaller one, their centers may overlap). It’s good enough for now but you should consider finding a better key for them.
Another thing to note is that building vertices are kept relative to their center as well. So vertice positions are relative to the building center and building position is relative to the tile center, it’s a nice and clear hierarchical system. And building meshes are created inside that BuildingHolder class which makes it extremely easy to switch visuals. i.e. if you want 3D polygons, all you would need is to change whatever is inside that class to add height and the rest of the code will work just the same.
private void CreateRoads(JSONObject mapData)
{
foreach (var geo in mapData["features"].list)
{
var l = new List<Vector3>();
for (int i = 0; i < geo["geometry"]["coordinates"].list.Count; i++)
{
var c = geo["geometry"]["coordinates"][i];
var bm = GM.LatLonToMeters(c[1].f, c[0].f);
var pm = new Vector2(bm.x - Rect.center.x, bm.y - Rect.center.y);
l.Add(pm.ToVector3xz());
}
var m = new GameObject("road").AddComponent<RoadPolygon>();
m.transform.parent = this.transform;
try
{
m.Initialize(geo["id"].str, this, l, geo["properties"]["kind"].str);
}
catch (Exception ex)
{
Debug.Log(ex);
}
}
}
CreateRoads is pretty much the same. Only different is taht I’m not generating polygons for the roads, just some debug lines so there is no mesh generation inside the RoadPolygon component. I’m just giving it the road vertices and it’ll draw lines between them in its update function.
You can find the project files below and there are a lot more than what’s covered here. Please do let me know if you have any questions or find any bugs/mistakes. I’m planning to write a few more blog posts on this, i.e. I already created a dynamic tiling system, which loads tiles as you move so you can travel all the way across usa or europe. Writing that was much easier than writing this blog post though, I’m having quite hard time writing in english so it may take some time π
Anyway, let me know what you think!
Cheers!
UnityOsm.unitypackage (19.27 kb)
New Post PokemonGO – Mapzen Post
New Updated version (25/07/2016)
I was trying the package but the main scene is not working, it just show a blue background whit a number..
this error apears:
NullReferenceException: Object reference not set to an instance of an object
Assets.Tile.CreateBuildings (.JSONObject mapData) (at Assets/Models/Tile.cs:54)
Assets.Tile+c__Iterator0.MoveNext () (at Assets/Models/Tile.cs:48)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
World:Start() (at Assets/World.cs:14)
I’m really sorry for the late reply but I’m terribly busy these days and couldn’t check this out yet, I’m sorry π
I bet it’s yet another small change in the data api xml.
Hey,
I just started with unity and this is a perfect example of what I wanted to archive!
Thanks alot for sharing your approach.
You can use this code in RoadPolygon.cs Initialize() method, to show the roads.
for (int i = 1; i < _verts.Count; i++)
{
GameObject roadPlane = GameObject.CreatePrimitive(PrimitiveType.Plane);
roadPlane.transform.position = (verts[i] + verts[i – 1]) / 2;
Vector3 scale = roadPlane.transform.localScale;
scale.z = Vector3.Distance(verts[i], verts[i – 1]) / 10;
roadPlane.transform.localScale = scale;
roadPlane.transform.LookAt(verts[i – 1]);
}
Hey there!
I haven’t tried this yet but yea it looks like a nice idea. There is soo much you can do with this project base really, shame I don’t have time for this anyΓΆpre. Create roads, generate buildings, traffic etc…
Thanks so much!
The Mapzen API has changed. You will need an access token and have to change the request to get this running. Nonetheless great work!
π
Hey!
Thanks for the notification, I’ll fix this asap (and hopefully complete and post the second part of this as well, it has been like… 1.5 years… wow -_-‘ )
Hi, could you explain in detail?
Thank you!
Gius
What are the steps to get this to work? Mine just shows a blue screen and the numbers 0, 0.
I attached the World script to an empty GameObject; is there more I need to do? I’ve added an API key.
Hey Jake,
I was planning to start woring on this again but things gone crazy in Turkey as you may know.
I’ll post a fix and probably a lot more asap.
Cheers,
Baran
Hello,
Thanks for the great short Tutorial. But could you please explain a bit more how you convert Long and Lat to Coordinates in Unity?
Admittedly I don’t know much about GIS systems at all so if I recall correctly, I just copy pasted that stuff from a website (or stackoverflow). I suggest reading into GIS stuff for that, cheers!
I understand you are crazy busy in Turkey, hope all is well!
Anyways, I am trying to dial in the detail to 16 with roads, but I am getting the following error.
NullReferenceException: Object reference not set to an instance of an object
Assets.Tile.CreateBuildings (.JSONObject mapData) (at Assets/Models/Tile.cs:54)
Assets.Tile+c__Iterator0.MoveNext () (at Assets/Models/Tile.cs:48)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
World:Start() (at Assets/World.cs:14)
Any ideas? (When you have time of course!)
Hey Ken,
Yes it’s been a crazy week here, thank you!
I don’t remember trying it with Level 16 but I’ll check it asap, looks like an easy fix.
I’m also working on a properly coded & structured version of this project, probably even a detailed blog post series but I’m not sure when/how I can find time for that. Still I’ll try to post somethings this week.
Cheers,
Baran
Awesome, thank you so much for your quick reply! No rush, I am a patient person and I am sure you are busy =)
Interestingly, level 17 works great, it’s just 16 that’s having an issue!
Hello,
Thank you for your work on this, this was exactly what I wanted to do for my project!
I just have a little problem, I don’t get as many roads as you do when I run your demo in Unity (here is a pic: http://imgur.com/1jXi1LJ)
Do you know why this is happening?
Hey Leo!
It’s strange, I’m not sure what’s going on really. But I’m planning to post a new fixed version of this project today, that should work better!
I’m impacient for the new post, has implemented the api token key? any idea how to implement the points of interest. shoops, bar, monuments…?
Hey Johny!
I’m working on it, was planning to post yesterday/today but can’t stop tweaking stuff. There are some bugs as well.
I’ll post it as soon as possible, possibly tonight or tomorrow.
And yea I’ve done token stuff. Not pois yet but it’ll be quite easy to implement now.
GeoConverter.CS has a line that reads:
private const double OriginShift = 2 * Math.PI * EarthRadius / 2;
Out of curiosity, why the multiplication and then division by 2. Might as well remove them.
Hey!
Well that’s not my code, I put a reference at the top of that file in the new package so you can check that link as well but it’s probably just for clarity.
New post is up!
Hi, BRNKHY!
Awesome tutorial, but I have one question. I’m triyng to generate buildings as 3d mesh, I understand that I just need to add one more dimension and build mesh with that data, but i’m stucked with that task 3-4 days already. Can you give me some tips?
p.s. Ssory, eng not my native lang.
Hey!
Well I can send you the unfinished script (3d buildings) tonite, roughly in 8-9 hours. I also posted a screenshot with 3D buildings on my twitter, so you can check that to see if that’s what you need (twitter: brnkhy).
Hi
if you have a Gameobject in the Scene. How can you get the Lat/long of this object?
Thanks for the answer.
I need additional help.
I want to paint a raster / matrix over the hole map. Every 0.005 lat/lng a quad mesh. Its hard for me to understand the relations between unity object koordinates and lat/lng. Maybe you or someone can help me π
Would be great!
Answered at http://barankahyaoglu.com/dev/pokemongo-clone-using-mapzen-part-4/
I have tried that. But didnt work very well i think. Maybe you see a misstake?
http://pastebin.com/hsE93T4U
Ok i found a solution. Thx
Nice! good job π
I cannot get “http://barankahyaoglu.com/dev/pokemongo-clone-using-mapzen-part-4/” to import into Unity 4. I rename the package as a “.unitypackage” is there something I’m missing? it crashes everytime. I can get your first two projects to import. Mapzen 1.1 imports, but there is nothing in the scene.Mapzen2 will not import, ends up wanting me to send a report to unity. Not sure why. THANKS!!
Yea I might not work with Unity4, I believe Unity5 scene files are not back compatible. But you can simply get source code from github page and use those scripts yourself. There isn’t much in the scene file anyway.
Thanks for the info. This deserves a Github entry:)
Hye Guney, glad you like it!
Very good your work, I’m using it to learn. I’m trying to add a polygon of a town or neighborhood, is this possible?
Hey!
I’m afraid I don’t get what you meant there. You mean like the bounding box around a neighborhood, like its borders?
Yes, or just different color on the ground. the OSM are the polygons of the neighborhoods, I’m trying to use this data
Hmm I’m not sure if OSM has neighborhood data. I know there is resitential areas under Landuse layer but that doesn’t cover neighborhoods either. I’ll look into this and let you know if I can find something. Oh wait there is boundaries layer right? That might have neighborhood data. I haven’t looked into that layer ever, so not sure at the moment, I’ll check it asap π
I added boundary layer last week, have you checked that? It has cities and municipalities borders. “place” layer has neighborhoods but doesn’t have borders for it, just a point
I saw the prints in git, but when testing on the map could not locate the borders on the map. I am in Brazil, using the coordinates lat -23.5528487 and long -46.630715. I’m also not finding the option to change the image of the ground, was removed? It is possible to change the color of the ground according to the area in the neighborhoods? I’m developing a native class GPS I hope to integrate it to move the character. You are doing an excellent job, congratulations!
It won’t let me reply your last post so I’m replying here π
About borders; are you sure you have boundary factory enabled and have settings for stuff like city/country borders?
And you can check MapImage plugin for base images. There has to be a qouta for image requests though, maybe you used it all up? Or sometimes they simply fail to respond (probably api issues).
as far as I know, mapzen doesnt have neighborhood polygons so you’ll have to do some extra work if you want to highlight ground depending on the neighborhood.
I have a database with the polygons of the neighborhoods, I could develop an api to return the polygons of the local player and neighboring polygons. in your project there is a plugin to add this information? as I do, convert the polygon coordinates and add point to point on the map?
hey how you deal with numerical value of x, y, z above 100 thousand result from mercator conversion
Hey!
If I remember correctly I was using relative positions to keep it in floating point range everywhere I can. But actually it’s not always possible so in Mapbox Unity SDK, we’re using double values occasionally to prevent losing precision.