Cloud visualization in Unity3D

A few months ago, I decided to give AerisWeather api a shot and added cloud forecast visualization on top of my scifi map project.

Live cloud forecast from @AerisWeather,
elevation and satellite imagery from @Mapbox,
visualized in @unity3d.#gamedev #madewithunity #indiedev #map #vfx #unity3d #datavisualization pic.twitter.com/nqHsGRBrZl

— Baran Kahyaoglu (@brnkhy) November 5, 2019

I got lots of questions about it since then so I thought it might be a good opportunity to break my blog post laziness!
In this post I’ll talk about converting 2d black and white cloud forecast textures provided by AerisWeather into somewhat realistic 3d particle based clouds in Unity. You can do this a few different ways really but I think this is the easiest, fastest (development time wise) and most straightforward one. On the negative side though, it’s a wasteful on particles (until/if I can find a way to improve that part).
We’ll use Unity’s VFX graph for this! VFX Graph makes it extremely easy to work with particles so we’ll be able to focus on (1) creating map url using AerisWeather Map Url Builder, (2) downloading and passing cloud data into VFX graph and (3) how to visualize/style particles.

Creating the cloud map in AerisWeather Map Url Builder

First of all, let’s create our cloud map. AerisWeather website has an amazing `Map Url Builder` tool to help you generate urls fast and very easily.

Map Url Builder is pretty easy to use. You add data by clicking buttons on the top menu and they appear in the layers menu on the left. For a readable weather forecast map, you can try flat base map, water mask and forecast clouds for example. This’ll let you see and identify locations and their cloud forecast. But for our project, we won’t need base map or water mask. So disabling them we’ll have our transparent background, cloud only texture and pretty url at the bottom of the page to use inside Unity.

I specifically choose Tile URL option there as I’m planning to decide which tiles to call inside unity, which will help me extend the project if necessary later on (i.e. rendering a whole region with multiple tiles instead of one static tile).

Downloading and passing cloud data into VFX graph

Now that we have our map, we can download it on Unity side and see what it looks like there. I used UnityWebRequestTexture to pull the texture, it’s async and returns a Texture2D just as I need. It doesn’t provide too much flexibility or anything but gets the basic job done very well.
So first I have bunch of fields for map url, my private api key, the vfx graph instance I will use and a Texture2D field to keep my cloud data.

[NonSerialized] private string _url = "https://maps.aerisapi.com/{3}/fsatellite/{0}/{1}/{2}/current.png";
[SerializeField] private string ApiKey;
[SerializeField] private VisualEffect _visualEffect;
[SerializeField] private Texture2D _texture;

If you’re not familiar with string formatting in C#, sections like {0} inside the map url means that I’ll use this string as a template but add additional data in place of {0} . You’ll how it works in a second. ApiKey is a serialized private field to make it easier to add and change through unity editor and scene. Same for VisualEffect actually. I’m planning to have one in my scene and use that one directly instead of instantiating in code. Just for ease of use really. And finally Texture2D is serialized so I’ll be able to see what it looks like in the inspector.

private IEnumerator DownloadImage(int z, int x, int y)
{
    var fullUrl = string.Format(_url, z, x, y, ApiKey);

    using (UnityWebRequest uwr = UnityWebRequestTexture.GetTexture(fullUrl))
    {
        yield return uwr.SendWebRequest();

        if (uwr.isNetworkError || uwr.isHttpError)
        {
            Debug.Log(uwr.error);
        }
        else
        {
          _texture = DownloadHandlerTexture.GetContent(uwr);
          _visualEffect.SetTexture("CloudTexture", _texture);
        }
    }
}

This DownloadImage method takes three parameters necessary to identify a map tile. Then it uses url template with these parameters and api key, creating a real url to use. Using block is pretty much how use use UnityWebRequestTexture. We make the call, start waiting for response. Then if it return an error we log it; if not, we create a texture and use SetTexture method to pass it into the visual effect object.
Finally I created a basic scene with just a few objects and scripts which looks like this;

There’s only one custom object named Clouds in the scene and it hosts both OverlayImagery script containing the functions above and Visual Effect script containing the vfx we’ll create in next section.

How to visualize cloud particles

I’ll try to explain everything but first of all, this is how the whole thing looks like;

Idea is rather simple really. I’m using a regular flipbook based cloud particle but since VFX graph doesn’t support spawning by a map (at least as far as I know), I’m doing a little trick by spawning particles everywhere but setting their scale and opacity by the alpha value of the given texture. Remember how the texture we created in AerisWeather Map Url Builder had transparent background? So if there’s no cloud, a value will be zero, particle scale and alpha will also be set zero, hence no clouds in that region.Of course this is a little wasteful as we’ll end up with lots of unused particles in empty spaces but it will work for now.
Now let’s have a closer look at stuff;

We’ll use two parameters;
(1) Size to determine how big the tile will be in Unity space. Texture we’ll get from AerisWeather will be 256×256 but most of the time you’ll want your tile to a different size in your application. So size will scale everything down/up to that value.
(2) CloudTexture to determine where the clouds will be. This is the cloud texture we get from AerisWeather and we set at the end of the code block above.

We create 1000 particles and randomly place them in a size x size AABox. We set their scale and alpha to zero to make them totally invisible initially. This random placement might feel little unreliable but since we have a big number of particles, it’s generally enough to fill up the tile and add some noise/irregularity to make it feel more organic. Of course you might need to increase particle count as the tile size increases though.

This is how we create a flipbook cloud. Flipbook player component will update the sprite and Face Camera Plane, as the name suggest, will turn it towards the camera. You can play with the Crop Factor if it doesn’t look smooth on edges.
If I remember correct, I used image sequences from this old Unity blog post; https://blogs.unity3d.com/2016/11/28/free-vfx-image-sequences-flipbooks/
You can try it with other image sequences as well but don’t forget to change Flip Book Size if necessary. 

This is the section where we read alpha value of corresponding pixel for each particle. We take the position of randomly created and placed particle, scale that position down to [0-1] range as we’ll use it as UV coordinates. Then query CloudTexture in that position and take the alpha value of the color.

And finally this is where we do the final tweaks on particle scale and opacity. We use original alpha value from cloud texture on a curve to have more flexibility on style and look. For example if you check the curve a little, it returns 0 for any value below 0.15 which helps hiding small clouds and have a little more contrast on the cloud edges. You can play with all this to match you app/game style and needs of course, this is the fun part!

And that’s it! I hope you like it and/or at least find it useful somehow. I’ll probably continue this with Mapbox Unity SDK integration post soon.

Cheers!