Volumetric Clouds - Part 1 - Covering the basics

The next big challenge I had to tackle was creating the clouds for our scene. I have never attempted to do anything like it, so I knew it would take quite a while to figure out. Similar to the rain shaders I've decided to break down the process into multiple parts, so that I can write them as I go before I forget everything.
The goal is to create some good looking rain clouds. Our reference looks like a combination of clouds and some fog.

 

1 - Options

 

Before starting any implementation I had a look at possible ways of approaching this problem and which one might be best suited to achieving, what I was going for.

1.1) Textured PlaneOption 1 was to just add a plane in the sky with some panning textures, however I quickly discarded that idea, as it looked very flat and not very realistic at all. It also lacked the depth. In the reference the cloud density increases with height, they overlap some taller buildings, vary in height, etc. and I knew pulling that off even with multiple planes might not be the way to go for this project.

1.2) Particles
There are some examples around the internet, where Particle systems are used to create local fog/ clouds, however in our scene it would require an immense amount of particles and they'd still present the problem of how to light them in the way clouds are lit. There is an article on Ryan Bruck's blog about Raymarching Height Maps, which might be useful: https://shaderbits.com/blog/ray-marched-heightmaps
I believe something like that, it doesn't have to be particles, might work for some smaller tufts of clouds/ fog.

1.3) Screenbased solution / PostProcess
Something else, that was suggested was using a Post Process Material for the fog and clouds. This might work fine for the fog, but I though trying to get nice cloud shapes would be pretty difficult and might cause the same problem as described under 1.1.

1.4) Volumetric Shader
I have been admiring the cloud effects of especially Horizon Zero Dawn and Red Dead Redemption 2 and both had presentations up on how they achieved those.
Guerilla Games: https://www.guerrilla-games.com/read/the-real-time-volumetric-cloudscapes-of-horizon-zero-dawn - The Real-Time Volumetric Clouds of Horizon Zero Dawn

Rockstar: http://advances.realtimerendering.com/s2019/index.htm - Creating the Atmospheric World of Red Dead Redemtion 2: A Complete and Integrated Solution
Of course I was well aware, that I wouldn't be able to create something even close to what these teams achieved in a significantly shorter time frame (I gave myself about 2 weeks), without having ever touched Volumetric Shaders before and having far less experience overall.
Using Volumetric Clouds have the advantage of solving the depth problem I'd have with just a plane if done properly look very good. On the other hand I'd have a lot of physics and techniques to dig through. I might be wasting my time, if I can't figure it out. However I have been keen to try out Volume Rendering for a while, so I thought this is the time to give it a go. Also a Volumetric Shader is more expensive and Volume Texture Resolutions need to be low, to avoid a huge negative impact on performance.

 

2 - Volume Textures 

 

2.1) Creating Volume Textures 
Fortunately Unreal has support for Volume textures build in, which made a lot of things a lot easier, however I needed to understand how to create them from 2D textures and how to convert 2D to 3D resolutions and vice-versa.
Guerilla Games' paper describes using the following volume textures:
1x 128^3 texture (4 channels) for defining the overall big shapes
1x 32^3 texture (3 channels) for adding smaller details

I created the 2D texture slices for those in Substance Designer. Ryan Brucks describes a way to do that in Unreal in his article about Authoring Pseudo-Volume Textures (https://shaderbits.com/blog/authoring-pseudo-volume-textures), which I initially gave a go. But once I found a way to do the same in Substance, it showed to be the far better way, because I can easily edit the textures in other softwares like Photoshop, which I can't do with the ones generated by Unreal.

Calculating the fitting 2D texture resolutions took me a while. Since Guerilla's paper says Volume Textures don't necessarily need to follow power of two resolutions I tried calculating it like this:

where f(x) is the side length of the 2D texture and x the side length of the 3D texture.

For a resolution of 128px this would result in a 2D resolution of roughly 1448px. However after some experimentation I found out, that 128px^3 = 2048px * 1024px, which keeps the power of two and still works for creating volume textures.
The other texture of 32px^3 therefore can be created from a 256px * 128px.
In some cases the square root is already a power of two, in other cases trying out the power of twos close to the square root usually works.

Example:
256^3 = 16.777.216
16.777.216‬‬^0.5 = 4.096

I then went on to creating the volume slices in Substance. On a square 2D Texture you can store x by x texture slices, e.g. 16 by 16, however if you do that on a non-square texture, you get really weird results in Unreal. I found out the hard way and spent a lot of time cursing, before figuring out what the problem was.

Broken Volume Texture

After having a closer look I figured out, that because my texture has a resolution of x by x/2 I can also only store x by x/2 slices, e.g. 16 by 8. Then suddenly the Volume Texture looked right.

Volume Slices and resulting Volume Texture

By the way, creating the Volume Slices in Substance Designer was only possible because they have 3D Worley and Perlin Noise. Otherwise I would have stuck with creating them in Unreal.
I made the UVWs for the slices by using Linear Gradients for the U and V Coordinates. For the W Coordinates I used a Tile Sampler, set the Color Parametrization Mode to Pattern Index and the Color Parametrization Multiplier to 1.

UVWs - W Coordinates

Finished UVWs

I could then feed those UVWs into the 3D Noise Nodes and get the Volume Slices. I adapted the method of creating the UVWs from Ryan Brucks' beforementioned article and translated the technique from Unreal to Substance. This setup allows me to quickly create the necessary textures.

UVW Setup in Substance Designer

When bringing the slice texture into Unreal and creating a Volume Texture out of it, it looks like this by default.

The Tile Size X and Tile Size Y needs to be set to the intended 3D Resolution, in my case 128.

2.2) Sampling Volume Textures  
Very early on I realized, that I wouldn't be able to rely purely on Unreal's Material Editor Nodes. Raymarching requires loops, which are not an option in Unreal, yet. That meant I had to utilize the Custom Node and write some HLSL, which I also have not done before.
Something, that helped a lot here is the fact, that Unreal is kind enough to show you the code for the Material in various languages, including HLSL.

View Shader Code

HLSL Shader Code

 When scrolling down, you can find the translations of the nodes you have plugged into the Material inputs. Unfortunately the variable names (local0, local1, ...) don't make it easier to figure out, what exactly is going on, but this means, if I do not know how something works in HLSL I can build it with nodes and then see what the HLSL solution is.

Node to HLSL Translations

That way and by using the HLSL documentation (https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl) I was able to figure out how to sample a Volume Texture. The general code for this is:

Texture3DSample(Tex, TexSampler, UVW));

Whilst trying to sample the Volume Texture I ran into a problem: It didn't tile. This wasn't mentioned anywhere, so I was confused for a bit.
In Unreal the Texture settings usually give you the option to set the tiling methods to Wrap or Clamp, Wrap being the default, but the option is missing in the volume texture. I was just assuming, that Volume Textures would behave the same as 2D Textures, which turned out to be wrong. So I had to tell the Sampler how to handle tiling.
Fortunately Unreal gives you the option on a sampler to set the sampler source, with the options 'From Texture Asset', 'Shared: Wrap' and 'Shared:Clamp'. The shared samplers use the same sampler for multiple textures, instead of creating one for each texture.

Sampler Sources

SamplerState GetMaterialSharedSampler(SamplerState TextureSampler, SamplerState SharedSampler)

Knowing that I could figure out how to make the tiling work and it worked perfectly fine.

Texture3DSample(Tex,GetMaterialSharedSampler(TexSampler,Material.Wrap_WorldGroupSettings), UVW);

Another solution, had I not been able to figure this out would have been to modify the UVWs to frac(UVW)  because the clamped sampler works in the 0-1 range and using frac would have remapped the UVW to chunks going from 0-1. Howere I do believe, that the other solution is a bit more elegant and not just a band aid.

In part 2 of the Volumetric Clouds I will go into writing the Raymarcher and how I dealt with trying to optimize it for our scene. But here is a small preview of what I have been able to come up with so far. It of course still needs loads of improvement, but I am positive, that I will get it to a good end result.

The scene has also evolved quite a bit. Litha has done a great job adding all the neon signs, improving the lighting and some of the materials. At least now I feel like things are moving in the right direction. It is alway incredible how environments take ages to be set up, but then suddenly things go a lot more quickly.