Wednesday 29 April 2020

Volumetric Clouds - Part 1 - Covering the basics


The next big challenge I had to tackle was creating the clouds for our scene. I have never attempted to do anything like it, so I knew it would take quite a while to figure out. Similar to the rain shaders I've decided to break down the process into multiple parts, so that I can write them as I go before I forget everything.
The goal is to create some good looking rain clouds. Our reference looks like a combination of clouds and some fog.


US storms: torrential rain in New York City and possible tornado ...

 

1 - Options

 

Before starting any implementation I had a look at possible ways of approaching this problem and which one might be best suited to achieving, what I was going for.

1.1) Textured PlaneOption 1 was to just add a plane in the sky with some panning textures, however I quickly discarded that idea, as it looked very flat and not very realistic at all. It also lacked the depth. In the reference the cloud density increases with height, they overlap some taller buildings, vary in height, etc. and I knew pulling that off even with multiple planes might not be the way to go for this project.

1.2) Particles
There are some examples around the internet, where Particle systems are used to create local fog/ clouds, however in our scene it would require an immense amount of particles and they'd still present the problem of how to light them in the way clouds are lit. There is an article on Ryan Bruck's blog about Raymarching Height Maps, which might be useful: https://shaderbits.com/blog/ray-marched-heightmaps
I believe something like that, it doesn't have to be particles, might work for some smaller tufts of clouds/ fog.

1.3) Screenbased solution / PostProcess
Something else, that was suggested was using a Post Process Material for the fog and clouds. This might work fine for the fog, but I though trying to get nice cloud shapes would be pretty difficult and might cause the same problem as described under 1.1.

1.4) Volumetric Shader
I have been admiring the cloud effects of especially Horizon Zero Dawn and Red Dead Redemption 2 and both had presentations up on how they achieved those.
Guerilla Games: https://www.guerrilla-games.com/read/the-real-time-volumetric-cloudscapes-of-horizon-zero-dawn - The Real-Time Volumetric Clouds of Horizon Zero Dawn

Rockstar: http://advances.realtimerendering.com/s2019/index.htm - Creating the Atmospheric World of Red Dead Redemtion 2: A Complete and Integrated Solution
Of course I was well aware, that I wouldn't be able to create something even close to what these teams achieved in a significantly shorter time frame (I gave myself about 2 weeks), without having ever touched Volumetric Shaders before and having far less experience overall.
Using Volumetric Clouds have the advantage of solving the depth problem I'd have with just a plane if done properly look very good. On the other hand I'd have a lot of physics and techniques to dig through. I might be wasting my time, if I can't figure it out. However I have been keen to try out Volume Rendering for a while, so I thought this is the time to give it a go. Also a Volumetric Shader is more expensive and Volume Texture Resolutions need to be low, to avoid a huge negative impact on performance.

 

2 - Volume Textures 

 

2.1) Creating Volume Textures 
Fortunately Unreal has support for Volume textures build in, which made a lot of things a lot easier, however I needed to understand how to create them from 2D textures and how to convert 2D to 3D resolutions and vice-versa.
Guerilla Games' paper describes using the following volume textures:
1x 128^3 texture (4 channels) for defining the overall big shapes
1x 32^3 texture (3 channels) for adding smaller details

I created the 2D texture slices for those in Substance Designer. Ryan Brucks describes a way to do that in Unreal in his article about Authoring Pseudo-Volume Textures (https://shaderbits.com/blog/authoring-pseudo-volume-textures), which I initially gave a go. But once I found a way to do the same in Substance, it showed to be the far better way, because I can easily edit the textures in other softwares like Photoshop, which I can't do with the ones generated by Unreal.

Calculating the fitting 2D texture resolutions took me a while. Since Guerilla's paper says Volume Textures don't necessarily need to follow power of two resolutions I tried calculating it like this:

where f(x) is the side length of the 2D texture and x the side length of the 3D texture.

For a resolution of 128px this would result in a 2D resolution of roughly 1448px. However after some experimentation I found out, that 128px^3 = 2048px * 1024px, which keeps the power of two and still works for creating volume textures.
The other texture of 32px^3 therefore can be created from a 256px * 128px.
In some cases the square root is already a power of two, in other cases trying out the power of twos close to the square root usually works.

Example:
256^3 = 16.777.216
16.777.216‬‬^0.5 = 4.096

I then went on to creating the volume slices in Substance. On a square 2D Texture you can store x by x texture slices, e.g. 16 by 16, however if you do that on a non-square texture, you get really weird results in Unreal. I found out the hard way and spent a lot of time cursing, before figuring out what the problem was.

Broken Volume Texture
After having a closer look I figured out, that because my texture has a resolution of x by x/2 I can also only store x by x/2 slices, e.g. 16 by 8. Then suddenly the Volume Texture looked right.

Volume Slices and resulting Volume Texture
By the way, creating the Volume Slices in Substance Designer was only possible because they have 3D Worley and Perlin Noise. Otherwise I would have stuck with creating them in Unreal.
I made the UVWs for the slices by using Linear Gradients for the U and V Coordinates. For the W Coordinates I used a Tile Sampler, set the Color Parametrization Mode to Pattern Index and the Color Parametrization Multiplier to 1.


UVWs - W Coordinates
Finished UVWs
I could then feed those UVWs into the 3D Noise Nodes and get the Volume Slices. I adapted the method of creating the UVWs from Ryan Brucks' beforementioned article and translated the technique from Unreal to Substance. This setup allows me to quickly create the necessary textures.

UVW Setup in Substance Designer
When bringing the slice texture into Unreal and creating a Volume Texture out of it, it looks like this by default.


The Tile Size X and Tile Size Y needs to be set to the intended 3D Resolution, in my case 128.




2.2) Sampling Volume Textures  
Very early on I realized, that I wouldn't be able to rely purely on Unreal's Material Editor Nodes. Raymarching requires loops, which are not an option in Unreal, yet. That meant I had to utilize the Custom Node and write some HLSL, which I also have not done before.
Something, that helped a lot here is the fact, that Unreal is kind enough to show you the code for the Material in various languages, including HLSL.

View Shader Code


HLSL Shader Code
 When scrolling down, you can find the translations of the nodes you have plugged into the Material inputs. Unfortunately the variable names (local0, local1, ...) don't make it easier to figure out, what exactly is going on, but this means, if I do not know how something works in HLSL I can build it with nodes and then see what the HLSL solution is.

Node to HLSL Translations

That way and by using the HLSL documentation (https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl) I was able to figure out how to sample a Volume Texture. The general code for this is:

Texture3DSample(Tex, TexSampler, UVW));

Whilst trying to sample the Volume Texture I ran into a problem: It didn't tile. This wasn't mentioned anywhere, so I was confused for a bit.
In Unreal the Texture settings usually give you the option to set the tiling methods to Wrap or Clamp, Wrap being the default, but the option is missing in the volume texture. I was just assuming, that Volume Textures would behave the same as 2D Textures, which turned out to be wrong. So I had to tell the Sampler how to handle tiling.
Fortunately Unreal gives you the option on a sampler to set the sampler source, with the options 'From Texture Asset', 'Shared: Wrap' and 'Shared:Clamp'. The shared samplers use the same sampler for multiple textures, instead of creating one for each texture.

Sampler Sources


SamplerState GetMaterialSharedSampler(SamplerState TextureSampler, SamplerState SharedSampler)

Knowing that I could figure out how to make the tiling work and it worked perfectly fine.

Texture3DSample(Tex,GetMaterialSharedSampler(TexSampler,Material.Wrap_WorldGroupSettings), UVW);

Another solution, had I not been able to figure this out would have been to modify the UVWs to frac(UVW)  because the clamped sampler works in the 0-1 range and using frac would have remapped the UVW to chunks going from 0-1. Howere I do believe, that the other solution is a bit more elegant and not just a band aid.

In part 2 of the Volumetric Clouds I will go into writing the Raymarcher and how I dealt with trying to optimize it for our scene. But here is a small preview of what I have been able to come up with so far. It of course still needs loads of improvement, but I am positive, that I will get it to a good end result.



The scene has also evolved quite a bit. Litha has done a great job adding all the neon signs, improving the lighting and some of the materials. At least now I feel like things are moving in the right direction. It is alway incredible how environments take ages to be set up, but then suddenly things go a lot more quickly.

Wednesday 15 April 2020

Rain Shader Part 2 - Moving Drops

This is the continuation of the procedural Rain Shader. After finishing the static drops I of course needed some that run down vertical surfaces. Since the initial setup is very similar I will skip explaining some of the first steps and jump straight into the more exciting stuff.

I looked for some reference video clips and found this one. There are some things, like merging raindrops or the trails, which I won't be able to recreate, but it helps to give me a general idea of what I am looking to achieve.


Before going into explaining further steps, my current state of the Material Function looks like this.
The UVs are not quadratic anymore, I set it to a different ratio, so I have more vertical space for the drops to run down. I have generated random offset values for. Since the XY-Position will be animated the method for offsetting is a bit more complicated.

Basic setup
The next step I wanted to work on was having the drops move down. The easiest way of achieving this is by just using a panner of offsetting the UVs over time. This however makes the drops all move at a constant rate, which looks very unnatural, since rain drops usually move with varying speed due to surface properties, etc.

Triplanar Mapping with Panner
Since I cannot change the position of the UVs for each individual drop I need to change the Center position of the drop by using another function. This can counteract the UV panning so that the drop sometimes seemingly moves faster or slower. The easiest solution, even though not the most elegant, since it is still a very uniform funtion, is using sine. After setting it up it looks something like this:


Not quite what I was looking for. I am going to dive a bit more into the math behind it to explain how it works.
Let's assume we have a time frame of 1 second. We know that the UVs travel a distance equivalent to the speed. In a function it can be described as distance = time * speed, where speed is a constant.

f(t) = t at speed 1
To simplify things it would be great to fit one phase of the sine function into the range of 0-1. Fortunately Unreal is kind enough to have that built in (I should have researched that first, I spent a good time scratching my head about it). Otherwise it would be fairly easy to do manually, since we know that 1 phase of sine is 2π, so the function would have to look like this: f(x) = sin(x*2π).


So this is up to the point, that the video shows. The problem is, that the sine function covers a far greater distance in 1 second, 4 units to be exact, however we want it to be equal to the distance the UVs cover. So whe need to divide it by 4/ multiply by 0.25 to fix that. Also since the drop center can only be in the UV range from -.5 to .5 I further multiply the function by (.5 - drop radius) to prevent it from leaving that space.
The last thing to do is offsetting the time input and there we go. The graph below illustrates the Y-Position change of the drop center over time.

Drop Y-Coordinate progression

Setup in UE4


This is getting there. It'd probably better to use a function that is not as uniform as the Sine-Function, maybe a combination of them, but I decided to keep it simple for now and really understand, what is going on mathematically, before trying something more ambitious.
This basically covers the Y-Axis movement.

In a next step I wanted to create a mask for the trails a raindrop would leave on a foggy surface. The main problem here is, that I won't be able to have the raindrops leave behind trails that stay. I will have to add some static ones to have that effect.

I ended up utilizing the UVs again to create 3 masks to then combine into the final mask.

Trail Masks

Trail Masks MF

 Combining them with the drops looks like this now. As I said the trails are not permanent, they fade out pretty quickly. I might add a control for how much trail appears behind the drop.


After finishing that part I wanted to go back to something easy again, so I added the same option of removing some of the drops as I did in the other Material Function.
This is how the Function looks at this point:

Overview 001
Obviously this is still very far from how I want it to look so in the next step I tackled giving the drops a kinda wiggly movement on the X-Axis. Since using sine-waves has worked before I decided to try and utilize them for this as well.
I tried out a few ways of combining sine waves at different frequencies.

f(x) = sin(2x) * sin(3x)

f(x) = sin(2x) + sin(3x)

I really liked the outcome of the second function, this method works well unless the difference in the frequencies is too high. With some small modifications, like changing the frequency for each drop by multiplying the frequency by the random value and remapping it to fit the X-Axis space of each cell I now had to somehow use it to make the drops wiggle and draw the trail.

What I did is using the green Y-Coordinates of the UVs (before they go into the panner node) and used that as the input for the sine-functions, which gave the following result:

X-Offset
The output of the marked multiply node goes into the x-position of the drop's center position, which makes the drop move, however the trail is still a straight line.
To do that I added the inverse(otherwise the trail and drop position are mirrored) of that to the panning UVs, masked out the red channel and getting the absolute values of that. I then use a smoothstep again to get the trails to the thickness of the rain drops.



And here is a clip of how all of that looks combined now.


This is starting to get closer to what I want this to look. I already know, that I will have to overlay a couple of those functions to create a more chaotic feeling.
Next I wanted to add some further distortion, utilising the same technique as with the other Rain Drop Material Function, using a normal map for that.

Distortion 


What I want to point out is that I will definitely still have to do some tweaking of the Parameters and figure out how to improve some bits, but now with less than a month left for finishing the FMP I will have to accept, that some parts might not turn out the way I want them to turn out.


The last thing I did was creating the normal map for the drops. It is not accurate, however it will help distorting the other textures to give the feeling of refraction.


Overview
This kinda wraps up all that went into the two Rain Material Functions, I hope it was somewhat understandable. I put a lot of time into constructing them and learned a lot of new things, especially manipulating UVs to create procedural shapes. It was certainly a lot of fun.
Since I still have to combine them and implement them into the other Shaders I will show the finished results in a different blog post.

Monday 13 April 2020

Week 14 - Sculpting and Windows

I spent the beginning of this week starting to redo the other Rain Material Function and made good progress, but again will explain it in a separate post. I also worked on and finished the Atlas Statue for the top of the Herald Building and made a Material for the upper floor windows of our buildings.



This is how our scene currently looks. We still have a long way to go in terms of lighting and atmosphere and time is running out. Might have to do an extra bit of pushing the next few weeks.

1 - Atlas Statue

Overall I spent about 2.5 days on this asset. I wish I could have polished it a bit more, but there are still so many tasks left to do I had to rush it a bit. Also, in all honesty, I have not really done any sculpting in a while, so I was a bit rusty.
It was quite difficult to make out any detail in the original concept.

Atlas Statue in the Concept
I did some research and found images of the Rockefeller Atlas statue, which fits our Art Deco theme so I took as the primary reference for my version. However we decided to keep the kneeling pose of the concept, since it made more sense composition wise.

Reference

ZBrush Sculpt

Retopo

Retopo

I feel like I could have saved a few more tris since the model is at about 7k tris, but I wanted to make sure that all the shapes are well defined.
For the textures I divided into 2 texture sheets, 1 for the body, 1 for the globe.

Globe Texture

Body Texture
I am quite happy with the result, considering the short amount of time I had to do it from start to finish. It certainly could have used a bit more polishing, but since you will only be able to see it from quite far away in the scene we'll hopefully get away with it. We are also still lacking the platform the atlas statue is standing on, that might be something I'll do next.

UPDATE: I made a quick placeholder platform and threw on some textures we already had. Litha might come up with a better version, we will see, but here is how the Atlas Statue looks in the level.

Now it is Atlas Square


2 - Interior Window Shader

Our Window Meshes are constructed in a way, that we have one plane for the glass and one for the interior, which I worked on this week as well.
Two functionalities which we agreed on from the beginning was, that we wanted the lights to turn on and off, as well as the blinds opening and closing in intervalls.
Fortunately Unreal has a Material Node called PerInstanceRandom, which can be utilized for that.

Interior Window Material Graph
The first thing I tackled was the opening and closing blinds. Litha made the textures for them. To move them up and down I just offset the UV Coordinates in the Y-Direction. However I needed to make sure, that the texture is not tiling, so I tried setting the Texture Tiling Mode to Clamp instead of Wrap, however that didn't work quite as intended, so I used a Custom Node to return a Mask of the UV 0 - 1 range. There are probably more elegant ways to solve this, for now this does the job.


I then multiplied the output of the custom node with the mask of the blinds to get the desired result.

The next part I had to worry about was the interior behind the blinds. Litha originally had suggested to just use emissive colors and see how that turns out.
I made a procedural vignette effect for that, however when just using that as the emissive it looked way to cartoony and flat, so I thought about alternative solutions. I also used the PerInstanceRandom Value to add slight color variation, lerping between the emissive colors and their negatives, in this case yellow and blue tones, which provided better results than using a HueShift instead.

Vignette
Originally we had hoped to ad parallax to the widows, but scrapped the idea, because we don't have enought time to make assets to populate the rooms. However it still seemed like the best option to me, so in order to test and simplify if I baked a cubemap of a cube interior. I made different versions for testing, two examples are below.



I went with a variant of the second one. I found out, that lower contrast and medium values worked better. This cubemap then could be used to create the sense of an interior as well as controlling the emissive strength. Of course it still looks a bit strange, so we might have to add some detail, but for now it still helps adding a lot to the overall scene.

Interior Cubemap in the Material Graph
I also used the Emissive Strength Parameter I added to lerp between a darker and lighter version of the cubemap, to increase the sense of lights turning on and of and not just saturation changes.

Next I moved on to animating the lights and blinds. I also wanted to be able to control which percentage of the windows would be animated and which static/turned off.

PerInstanceRandom Setup
I use the PerInstanceRandom value to the threshold/percentage value and return either 0 or the Random value. I then remap the values to the 0 - 1 range. This is very similar to what I did in the Rain Drop Function to even out the time offset.
Now I had to worry about how to transition between the on/off and open/closed states. The very obvious answer was to use a sine/cosine function, however I ideally want a function that isn't constantly changing, so I started looking for flattened sine/cosine functions and stumbled over this thread on StackExchange: https://math.stackexchange.com/questions/100655/cosine-esque-function-with-flat-peaks-and-valleys

I ended up using the following function, since it was simple and controllable in the way I wanted it to be.


I used this function to control the speed at which the state change happens and used different values for the lights and blinds, the blinds using a longer transition period. The Phase length is also controlable.
For the lights, if the PerInstanceRandom Value is below the treshold I return 0, so they are turned off by default, for the blinds I return the PerInstanceRandom Value.

All this combined looks like this in the level:

Material Cube


Now with the material added to the windows the scene is starting to feel slightly more alive (we will have to figure out some better settings), however we still have such a long way to go until it is even close to the concept in terms of atmosphere.
Next I will probably tackle the clouds/fog, finish the Rain Material Functions and start implementing the actual rain.