Jump to content

Shader Families

Josh

811 views

I started to implement quads for tessellation, and at that point the shader system reached the point of being unmanageable. Rendering an object to a shadow map and to a color buffer are two different processes that require two different shaders. Turbo introduces an early Z-pass which can use another shader, and if variance shadow maps are not in use this can be a different shader from the shadow shader. Rendering with tessellation requires another set of shaders, with one different set for each primitive type (isolines, triangles, and quads). And then each one of these needs a masked and opaque option, if alpha discard is enabled.

All in all, there are currently 48 different shaders a material could use based on what is currently being drawn. This is unmanageable.

To handle this I am introducing the concept of a "shader family". This is a JSON file that lists all possible permutations of a shader. Instead of setting lots of different shaders in a material, you just set the shader family one:

shaderFamily: "PBR.json"

Or in code:

material->SetShaderFamily(LoadShaderFamily("PBR.json"));

The shader family file is a big JSON structure that contains all the different shader modules for each different rendering configuration: Here are the partial contents of my PBR.json file:

{
	"turboShaderFamily" :
	{
		"OPAQUE":
		{
			"default":
			{
				"base":
				{
					"vertex": "Shaders/PBR.vert.spv",
					"fragment": "Shaders/PBR.frag.spv"
				},
				"depthPass":
				{
					"vertex": "Shaders/Depthpass.vert.spv"
				},
				"shadow":
				{
					"vertex": "Shaders/Shadow.vert.spv"
				}
			},
			"isolines":
			{
				"base":
				{
					"vertex": "Shaders/PBR_Tess.vert.spv",
					"tessellationControl": "Shaders/Isolines.tesc.spv",
					"tessellationEvaluation": "Shaders/Isolines.tese.spv",
					"fragment": "Shaders/PBR_Tess.frag.spv"
				},
				"shadow":
				{
					"vertex": "Shaders/DepthPass_Tess.vert.spv",
					"tessellationControl": "Shaders/DepthPass_Isolines.tesc.spv",
					"tessellationEvaluation": "Shaders/DepthPass_Isolines.tese.spv"
				},
				"depthPass":
				{
					"vertex": "Shaders/DepthPass_Tess.vert.spv",
					"tessellationControl": "DepthPass_Isolines.tesc.spv",
					"tessellationEvaluation": "DepthPass_Isolines.tese.spv"
				}
			},			
			"triangles":
			{
				"base":
				{
					"vertex": "Shaders/PBR_Tess.vert.spv",
					"tessellationControl": "Shaders/Triangles.tesc.spv",
					"tessellationEvaluation": "Shaders/Triangles.tese.spv",
					"fragment": "Shaders/PBR_Tess.frag.spv"
				},
				"shadow":
				{
					"vertex": "Shaders/DepthPass_Tess.vert.spv",
					"tessellationControl": "Shaders/DepthPass_Triangles.tesc.spv",
					"tessellationEvaluation": "Shaders/DepthPass_Triangles.tese.spv"
				},
				"depthPass":
				{
					"vertex": "Shaders/DepthPass_Tess.vert.spv",
					"tessellationControl": "DepthPass_Triangles.tesc.spv",
					"tessellationEvaluation": "DepthPass_Triangles.tese.spv"
				}
			},
			"quads":
			{
				"base":
				{
					"vertex": "Shaders/PBR_Tess.vert.spv",
					"tessellationControl": "Shaders/Quads.tesc.spv",
					"tessellationEvaluation": "Shaders/Quads.tese.spv",
					"fragment": "Shaders/PBR_Tess.frag.spv"
				},
				"shadow":
				{
					"vertex": "Shaders/DepthPass_Tess.vert.spv",
					"tessellationControl": "Shaders/DepthPass_Quads.tesc.spv",
					"tessellationEvaluation": "Shaders/DepthPass_Quads.tese.spv"
				},
				"depthPass":
				{
					"vertex": "Shaders/DepthPass_Tess.vert.spv",
					"tessellationControl": "DepthPass_Quads.tesc.spv",
					"tessellationEvaluation": "DepthPass_Quads.tese.spv"
				}
			}
		}
	}
}

A shader family file can indicate a root to inherit values from. The Blinn-Phong shader family pulls settings from the PBR file and just switches some of the fragment shader values.

{
	"turboShaderFamily" :
	{
		"root": "PBR.json",
		"OPAQUE":
		{
			"default":
			{
				"base":
				{
					"fragment": "Shaders/Blinn-Phong.frag.spv"
				}
			},
			"isolines":
			{
				"base":
				{
					"fragment": "Shaders/Blinn-Phong_Tess.frag.spv"
				}
			},
			"triangles":
			{
				"base":
				{
					"fragment": "Shaders/Blinn-Phong_Tess.frag.spv"
				}
			},
			"quads":
			{
				"base":
				{
					"fragment": "Shaders/Blinn-Phong_Tess.frag.spv"
				}
			}
		}
	}
}

If you want to implement a custom shader, this is more work because you have to define all your changes for each possible shader variation. But once that is done, you have a new shader that will work with all of these different settings, which in the end is easier. I considered making a more complex inheritance / cascading schema but I think eliminating ambiguity is the most important goal in this and that should override any concern about the verbosity of these files. After all, I only plan on providing a couple of these files and you aren't going to need any more unless you are doing a lot of custom shaders. And if you are, this is the best solution for you anyways.

Consequently, the baseShader, depthShader, etc. values in the material file definition are going away. Leadwerks .mat files will always use the Blinn-Phong shader family, and there is no way to change this without creating a material file in the new JSON material format.

The shader class is no longer derived from the Asset class because it doesn't correspond to a single file. Instead, it is just a dumb container. A ShaderModule class derived from the Asset class has been added, and this does correspond with a single .spv file. But you, the user, won't really need to deal with any of this.

The result of this is that one material will work with tessellation enabled or disabled, quad, triangle, or line meshes, and animated meshes. I also added an optional parameter in the CreatePlane(), CreateBox(), and CreateQuadSphere() commands that will create these primitives out of quads instead of triangles. The main reason for supporting quad meshes is that the tessellation is cleaner when quads are used. (Note that Vulkan still displays quads in wireframe mode as if they are triangles. I think the renderer probably converts them to normal triangles after the tessellation stage.)

tris.png.924dbf1f8023ab6142f6c6ad5fcfc2f2.png

quads.png.a7632c4712016a13b721ea264422a538.png

I also was able to implement PN Quads, which is a quad version of the Bezier curve that PN Triangles add to tessellation.

original.thumb.png.1e8450693214116ddf623a1d82d4de9b.png

tess.thumb.png.37d043cbdf82795bfa7ace7bed1f48f6.png

PNQuads.thumb.png.45be40c442274d05a4ab0d00c353d721.png

Basically all the complexity is being packed into the shader family file so that these decisions only have to be made once instead of thousands of times for each different material.

  • Like 2


4 Comments


Recommended Comments

On 12/12/2019 at 7:11 PM, Lethal Raptor Games said:

How do these handle geometry shaders?

A geometry shader can be added in the definition.

  • Like 1
  • Thanks 1

Share this comment


Link to comment

Is this the correct naming?

{
	"root": "PBR.json",
	"float":
	{
		"OPAQUE":
		{
			"default":
			{
				"base":
				{
					"vertex": "Landscape.vert.spv",
					"geometry": "Landscape.geom.spv",
					"fragment": "Landscape.frag.spv"
				}

 

All stages compile and the vert & frag work, but the geometry shader seems to be skipped.  Changes to it don't take effect.

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Blog Entries

    • By Josh in Josh's Dev Blog 2
      Documentation in Leadwerks 5 will start in the header files, where functions descriptions are being added directly like this:
      /// <summary> /// Sets the height of one terrain point. /// </summary> /// <param name="x">Horizontal position of the point to modify.</param> /// <param name="y">Vertical position of the point to modify.</param> /// <param name="height">Height to set, in the range -1.0 to +1.0.</param> virtual void SetHeight(const int x, const int y, const float height); This will make function descriptions appear automatically in Visual Studio, to help you write code faster and more easily:

      Visual Studio can also generate an XML file containing all of the project's function descriptions as part of the build process. The generated XML file will serve as the basis for the online documentation and Visual Studio Code extension for Lua. This is how I see it working:

      I am also moving all things private to private members. I found a cool trick that allows me to create read-only members. In the example below, you can access the "position" member to get an entity's local position, but you cannot modify it without using the SetPosition() method. This is important because modifying values often involves updating lots of things in the engine under the hood and syncing data with other threads. This also means that any method Visual Studio displays as you are typing is okay to use, and there won't be any undocumented / use-at-your-own risk types of commands like we had in Leadwerks 4.
      class Entity { private: Vec3 m_position; public: const Vec3& position; }; Entity::Entity() : position(m_position) {} It is even possible to make constructors private so that the programmer has to use the correct CreateTerrain() or whatever command, instead of trying to construct a new instance of the class, with unpredictable results. Interestingly, the constructor itself has to be added as a friend function for this to work.
      class Terrein { private: Terrain(); public: friend shared_ptr<World> CreateTerrain(shared_ptr<World>, int, int, int) }; The only difference is that inside the CreateTerrain function I have to do this:
      auto terrain = shared_ptr<Terrain>(new Terrain); instead of this, because make_shared() doesn't have access to the Terrain constructor. (If it did, you would be able to create a shared pointer to a new terrain, so we don't want that!)
      auto terrain = make_shared<Terrain>(); I have big expectations for Leadwerks 5, so it makes sense to pay a lot of attention to the coding experience you will have while using this. I hope you like it!
    • By Josh in Josh's Dev Blog 8
      An often-requested feature for terrain building commands in Leadwerks 5 is being implemented. Here is my script to create a terrain. This creates a 256 x 256 terrain with one terrain point every meter, and a maximum height of +/- 50 meters:
      --Create terrain local terrain = CreateTerrain(world,256,256) terrain:SetScale(256,100,256) Here is what it looks like:

      A single material layer is then added to the terrain.
      --Add a material layer local mtl = LoadMaterial("Materials/Dirt/dirt01.mat") local layerID = terrain:AddLayer(mtl) We don't have to do anything else to make the material appear because by default the entire terrain is set to use the first layer, if a material is available there:

      Next we will raise a few terrain points.
      --Modify terrain height for x=-5,5 do for y=-5,5 do h = (1 - (math.sqrt(x*x + y*y)) / 5) * 20 terrain:SetElevation(127 + x, 127 + y, h) end end And then we will update the normals for that whole section, all at once. Notice that we specify a larger grid for the normals update, because the terrain points next to the ones we modified will have their normals affected by the change in height of the neighboring pixel.
      --Update normals of modified and neighboring points terrain:UpdateNormals(127 - 6, 127 - 6, 13, 13) Now we have a small hill.

      Next let's add another layer and apply it to terrain points that are on the side of the hill we just created:
      --Add another layer mtl = LoadMaterial("Materials/Rough-rockface1.json") rockLayerID = terrain:AddLayer(mtl) --Apply layer to sides of hill for x=-5,5 do for y=-5,5 do slope = terrain:GetSlope(127 + x, 127 + y) alpha = math.min(slope / 15, 1.0) terrain:SetMaterial(rockLayerID, 127 + x, 127 + y, alpha) end end We could improve the appearance by giving it a more gradual change in the rock layer alpha, but it's okay for now.

      This gives you an idea of the basic terrain building API in Leadwerks 5, and it will serve as the foundation for more advanced terrain features. This will be included in the next beta.
    • By Josh in Josh's Dev Blog 0
      A new update is available for beta testers.
      Terrain
      The terrain building API is now available and you can begin working with it, This allows you to construct and modify terrains in pure code. Terrain supports up to 256 materials, each with its own albedo, normal, and displacement maps. Collision and raycasting are currently not supported.
      Fast C++ Builds
      Precompiled headers have been integrated into the example project. The Debug build will compile in about 20 seconds the first run, and compile in just 2-3 seconds thereafter. An example class is included which shows how to add files to your game project for optimum compile times. Even if you edit one of your header files, your game will still compile in just a few seconds in debug mode! Integrating precompiled headers into the engine actually brought the size of the static libraries down significantly, so the download is only about 350 MB now.
      Enums Everywhere
      Integer arguments have been replaced with enum values for window styles, entity bounds, and load flags. This is nice because the C++ compiler has some error checking so you don't do something like this:
      LoadTexture("grass.dds", WINDOW_FULLSCREEN); Operators have been added to allow combining enum values as bitwise flags.
      A new LOAD_DUMP_INFO LoadFlags value has been added which will print out information about loaded files (I need this to debug the GLTF loader!).
      Early Spring Cleaning
      Almost all the pre-processor macros have been removed from the Visual Studio project, with just a couple ones left. Overall the headers and project structure have been massively cleaned up.
×
×
  • Create New...