Snow Deformation Shader

Made by Niels van Eijk

Introduction

Snow in games has always been something that would catch my attention. Especially when you as a player were able to create footprints in it. When I found this in a game I played, it always made me stop for a moment and check it out. The more I started to learn about the technical and performance side of games over the last few years, the more I became interested in finding out how this was actually done and how this could be implemented in a game. So I decided to make an interactive snow shader for this GameLab research. Another reason I decided to make this was my interest in finding ways to link things that happen in a game to the visual side of the game.

Table of Contents

1 Requirements

In the final product, objects of different shapes and sizes should be able to make trails and imprints that match those different shapes and sizes. The objects should also be able to make these trails and imprints at different depths. Scalability is also an important requirement for this project. The snow deformation should seamlessly work between different snow meshes and it should be easy to expand the snow area. Finally the shader should be performant and run at 60+ FPS on a midrange pc, preferably with a large snow environment.

2 Deformation Methods

To do the deformation of the snow I found two main methods. In this part I will discuss both methods and explain some of the important things to keep in mind when using that method.

2.1 Raycasts

The first method is to deform the snow using raycasts. The raycasts basically ‘paint’ on a texture when they hit the snow mesh. After a bit more research I found out that if I were to use this method, it would be a big task every time an object with a new shape or size is added that can deform the snow. Every part of the object that can deform the snow, needs to do a raycast. This makes scaling very difficult and would be very time consuming to use when adding objects with different shapes and sizes. The main downside of this is that the tracks and prints will only work on one point in a predetermined shape and size, making this method not usable for this project.

2.2 Depth Render Texture

The other method I found is to use an orthographic camera under the snow mesh. This camera will look up and render the depth of all the objects in its frustum that should deform the snow. This way it does not matter what shape or size an object has and that makes it very easy to add new objects that can deform the snow without having to worry about adjusting anything to make the snow deform with the right shape or size. This method is also easy to scale and if I can make the depth camera move with the player, the texture size does not have to be too big. Possibly the biggest upside to this method is that a lot more work can be done by the GPU compared to the method that uses raycasts. The only downside to this is that the snow will ‘reset’ if the camera moves away from that area, although the camera frustum can be made relatively big without having much impact on performance. I went with this method, because it fit my requirements way better than the other one. Additionally it offers more to work on once my basic requirements are met. To further improve on this I could, for example, make the snow slowly regenerate, so it seems more realistic for the snow to be back to normal when the player walks far away and back.

3 Snow Deformation

To achieve the snow deformation effect that fits my requirements I first need a depth texture and then apply it to the right vertices of the snow plane. In this section I will go over the most important steps.

3.1 Camera Setup

To get a depth texture I used an orthographic camera with a frustum depth equal to the max snow height. This camera is set up so the full snow height fits in the camera frustum. The camera should then render to a render texture. At first this was the render texture I used to apply to the depth of the snow, but I ran into issues when trying to make it work when moving the depth camera. Because I wanted to make this easily scalable, I wanted to make a system where the camera could move along the snow mesh and only apply the depth to the area of the camera. For this to work, I had to offset the camera render texture to match the movement of the camera. I could not get this to work with the current setup, so I decided to manually get the depth from the camera render and offset that.

3.2 Depth Texture

To get the depth from the camera render I applied a post-processing shader in the camera’s built-in OnRenderImage() function using Graphics.Blit(). In the Blit() function, I pass a material with the shader that gets the color from the depth information of the render texture. This shader uses the _CameraDepthNormalsTexture global property and the DecodeDepthNormal() helper function in the fragment shader to convert the depth information from the render texture to colors. The global property is automatically set when using DepthTextureMode.DepthNormals as a depthTextureMode in the camera.

private void OnRenderImage(RenderTexture source, RenderTexture destination) {
    Graphics.Blit(source, destination, ConvertMat);
}

fixed4 frag (v2f i) : SV_Target
{
    fixed4 col = tex2D(_MainTex, i.uv);
 
    float4 NormalDepth;
 
    DecodeDepthNormal(tex2D(_CameraDepthNormalsTexture, i.uv), NormalDepth.w, NormalDepth.xyz);
 
    col.rgb = 1 - NormalDepth.w;
 
    return col;
}

(FM Coding, 2018)

This will create a grayscale depth texture, but to keep the texture files smaller I made it so the target texture only uses red values, because only one channel is necessary to keep track of the depth.

3.3 Persistent Texture

The depth texture with the setup so far, will be completely replaced every frame, so it only has the depth information of the current frame. To make sure the snow stays deformed even if the object leaves the depth camera frustum, I needed a persistent texture. The depth texture of the current frame has to be copied onto the persistent texture.

I do this in another post-processing shader by checking the highest value of the current depth texture and the persistent depth texture and then outputting that to the persistent texture.

fixed4 frag(v2f i) : SV_Target
{
    fixed4 mainCol = tex2D(_MainTex, i.uv);
 
    fixed4 persistentCol = tex2D(_PersistentTex, i.uv);
 
    fixed4 col = fixed4(max(mainCol.r, persistentCol.r), 0, 0, 0);
 
    return col;
}

This persistent texture can also be offset so it matches the camera movement. This is necessary when moving the camera, because if I don’t do this, the snow deformation will move with the camera, instead of it staying at the initial position in the snow. The camera offset is calculated outside the shader, so it only happens once per frame, by subtracting the current camera position from the previous frame camera position. Dividing it by the camera size times two (times two, because the camera frustum size is actually double the size you set in the inspector when using world coordinates) will give me the UV offset that I can use to offset the texture.

Shader.SetGlobalVector("RTCameraOffset", (prevPos - transform.position) / (RTCamera.orthographicSize * 2));

With the current method I did run into a problem when objects that are already in the snow enter the camera frustum when the camera moves towards them, The objects leave a trail, since the color at the edge of the render texture is not reset. To fix this I make sure the edge 1% of the render texture is always black using this if statement. It is probably not the best way to fix this, but I will discuss other options in the future references chapter.

if (i.uv.x >= 0.99 || i.uv.y >= 0.99 || i.uv.y <= 0.01 || i.uv.x <= 0.01) col = float4 (0, 0, 0, 0);
else col = tex2D(_MainTex, i.uv);

3.4 Applying the Depth Texture

To apply the depth texture and deform the snow, I made a vertex shader that uses the persistent texture from earlier as a height map for the snow. First it is important to calculate the UVs while taking into account the camera position to make sure the deformed snow area matches the depth texture. To calculate this I take the world position of the vertex and subtract the camera position. Then multiply by the camera size times two (for the same reason as before) and finally add 0.5, since the camera position is at the center of the texture. This UV position can then be used to read from the depth texture. After this, it is possible to move the camera while the deformation stays at the same position.
I wanted to make the snow a little smoother, so I decided to check some UV positions around the current one and took the average value to use as the actual height.

float3 vertexWorldPos = mul(unity_ObjectToWorld, v.vertex);
float3 uvPosition = 1 - ((vertexWorldPos - RTCameraPosition) / (RTCameraSize * 2) + 0.5);
 
float height;
for (int i = -1; i <= 1; i++) {
	for (int j = -1; j <= 1; j++) {
		height += tex2Dlod(_DispTex, float4(1 - uvPosition.x + i * _Rounding, uvPosition.z + j * _Rounding, 0, 0)).r;
	}
}
height /= 9;
 
v.vertex.y = _Displacement - (height * _Displacement);

As a final step in this vertex shader I pass the calculated uv positions and  height from earlier to the appdata struct as a standard color. Since I am not using this for anything else, I can also put this as a float4 in the Input of the surface shader, so I can use it there too. I wanted to do this because now I can use this information to lerp between two textures and give the snow a blend between these two textures based on the height of the snow at that point. This will make it easier to see how the snow is deformed. This method of passing information is not the most clean way of doing it, but I found out that in a surface shader there is a bug when trying to use the proper method, so I found this workaround.

v.color = float4(uvPosition.x, uvPosition.z, saturate(height), 0);

void surf(Input IN, inout SurfaceOutput o) {
	fixed4 c = lerp(tex2D(_SnowTex, IN.uv_SnowTex) * _SnowColor, tex2D(_GroundTex, IN.uv_SnowTex) * _GroundColor, IN.color.z);
	o.Albedo = c.rgb;

3.5 Tessellation

Tessellation divides a mesh into more vertices. My surface/tessellation shader that I use to deform the snow mesh is based on the distance-based tessellation shader from the unity manual. Tessellation is very important for this project, because the snow plane does not always need to have a lot of vertices, only when specified in the shader. In this case that is based on the distance from the camera. Using tessellation saves a lot on performance, especially since the biggest calculations for this project happen in the vertex shader.

4 Scaling

Scaling is one of the things that I kept in mind throughout this whole project. Earlier I mentioned a few choices I made where I kept scaling in mind. One of them being the moving depth camera. When using a moving camera the deformation only has to be applied to the snow area that is covered by the depth camera. This makes it very easy to add more snow planes or increase the size of existing ones, since all the deformation is done around the camera. To add another plane, the only thing you have to do is add the mesh and apply the snow material. It also works with flat meshes with a different shape. All together I think the ease of scaling this setup makes it fit my requirements very well.

5 Performance

Performance is another thing I kept in mind while working on this project, mainly while thinking about scaling. Running the game with one snow plane, the Unity project reaches around 75 FPS on a midrange pc. Adding around 35 snow planes drops the FPS to about 60. On a high-end pc I managed to reach 85 FPS with the 36 snow planes. This is performant enough to fit my requirements, but I do think there are many things to improve on, since I did not do any optimization yet.

6 Final Result

The final product meets all my predetermined requirements and even though I chose scalability over looks, it still looks pretty close to what I imagined when I started. Objects of different shapes and sizes can make different imprints in the snow. The scalability and performance are also good enough to fit my requirements. Overall I am happy with the final result of this project, although there are still quite a few things I would want to improve and add.

7 Future References

There are a lot of things I still have in mind to improve or add to this snow shader.

One of the main things I would still want to add is to make the shader work with an uneven terrain. Somewhere in the project I tried to make this and got pretty far, but I decided to give more priority to improving the scalability first.

I also would like to make the snow look a little better by blurring the depth texture, so the snow tracks are not as steep. Additionally I want the tessellation to work based on the depth texture. This way the parts of snow that are not affected by the deformation, are not tessellated. Doing this would also improve performance. To further improve on the performance, I want to combine the shaders. Right now some of them are separated, because I implemented them one by one and left the task to combine them as a polish task that I didn’t get to do.

Finally I would want to make the snow regenerate slowly and make the whole project ‘asset store-ready’ by making the values more easily adjustable and hide the debug variables.

8 Sources

Tran, T. (2018, July 3). Creating Snow Trails in Unreal Engine 4. [Article]. https://www.raywenderlich.com/5760-creating-snow-trails-in-unreal-engine-4

Amazing Snow Trails in Rise Of The Tomb Raider. (2016, February 8). [Screenshot]. Imgur. https://imgur.com/6XhTpse

Barre-Brisebois, C. (2014, March 17-21).Deformable Snow Rendering in Batman: Arkham Origins [Presentation]. https://www.gdcvault.com/play/1020177/Deformable-Snow-Rendering-in-Batman3

Mert Kirimgeri. (2020, March 28). Simple Snow Tracks Shader with Unity Shader Graph [Video]. YouTube. https://www.youtube.com/watch?v=ThlqTMBzyjI

GDC. (2017, May 6). Deformable Snow Rendering in Batman: Arkham Origins [Video]. YouTube. https://www.youtube.com/watch?v=87rg95XBalE

Unity Technologies. (2018, March 20). Unity – Manual: Surface Shaders with DX11 / OpenGL Core Tessellation. Unity Documentation. https://docs.unity3d.com/2020.1/Documentation/Manual/SL-SurfaceShaderTessellation.html

Unity Technologies. (2021, March 30). Unity – Scripting API: DepthTextureMode.DepthNormals. Unity Documentation. https://docs.unity3d.com/ScriptReference/DepthTextureMode.DepthNormals.html

Loïc Cayuela. (2018, December). RenderTexture data tutorial. [Report]. https://drive.google.com/file/d/1E_vjNknY1w4dnMadBQsZfh86-J4jd5O0/view

[FM Coding] How to Render Depth & World Space Normals on Game Screen? (Unity3D Beginner Tutorial). (2020, February 24). [Video]. YouTube. https://www.youtube.com/watch?v=5lsKlOoepw4