Jump to content


  • Content Count

  • Joined

  • Last visited

Everything posted by Josh

  1. Josh

    Shader Families

    I started to implement quads for tessellation, and at that point the shader system reached the point of being unmanageable. Rendering an object to a shadow map and to a color buffer are two different processes that require two different shaders. Turbo introduces an early Z-pass which can use another shader, and if variance shadow maps are not in use this can be a different shader from the shadow shader. Rendering with tessellation requires another set of shaders, with one different set for each primitive type (isolines, triangles, and quads). And then each one of these needs a masked and opaque option, if alpha discard is enabled. All in all, there are currently 48 different shaders a material could use based on what is currently being drawn. This is unmanageable. To handle this I am introducing the concept of a "shader family". This is a JSON file that lists all possible permutations of a shader. Instead of setting lots of different shaders in a material, you just set the shader family one: shaderFamily: "PBR.json" Or in code: material->SetShaderFamily(LoadShaderFamily("PBR.json")); The shader family file is a big JSON structure that contains all the different shader modules for each different rendering configuration: Here are the partial contents of my PBR.json file: { "turboShaderFamily" : { "OPAQUE": { "default": { "base": { "vertex": "Shaders/PBR.vert.spv", "fragment": "Shaders/PBR.frag.spv" }, "depthPass": { "vertex": "Shaders/Depthpass.vert.spv" }, "shadow": { "vertex": "Shaders/Shadow.vert.spv" } }, "isolines": { "base": { "vertex": "Shaders/PBR_Tess.vert.spv", "tessellationControl": "Shaders/Isolines.tesc.spv", "tessellationEvaluation": "Shaders/Isolines.tese.spv", "fragment": "Shaders/PBR_Tess.frag.spv" }, "shadow": { "vertex": "Shaders/DepthPass_Tess.vert.spv", "tessellationControl": "Shaders/DepthPass_Isolines.tesc.spv", "tessellationEvaluation": "Shaders/DepthPass_Isolines.tese.spv" }, "depthPass": { "vertex": "Shaders/DepthPass_Tess.vert.spv", "tessellationControl": "DepthPass_Isolines.tesc.spv", "tessellationEvaluation": "DepthPass_Isolines.tese.spv" } }, "triangles": { "base": { "vertex": "Shaders/PBR_Tess.vert.spv", "tessellationControl": "Shaders/Triangles.tesc.spv", "tessellationEvaluation": "Shaders/Triangles.tese.spv", "fragment": "Shaders/PBR_Tess.frag.spv" }, "shadow": { "vertex": "Shaders/DepthPass_Tess.vert.spv", "tessellationControl": "Shaders/DepthPass_Triangles.tesc.spv", "tessellationEvaluation": "Shaders/DepthPass_Triangles.tese.spv" }, "depthPass": { "vertex": "Shaders/DepthPass_Tess.vert.spv", "tessellationControl": "DepthPass_Triangles.tesc.spv", "tessellationEvaluation": "DepthPass_Triangles.tese.spv" } }, "quads": { "base": { "vertex": "Shaders/PBR_Tess.vert.spv", "tessellationControl": "Shaders/Quads.tesc.spv", "tessellationEvaluation": "Shaders/Quads.tese.spv", "fragment": "Shaders/PBR_Tess.frag.spv" }, "shadow": { "vertex": "Shaders/DepthPass_Tess.vert.spv", "tessellationControl": "Shaders/DepthPass_Quads.tesc.spv", "tessellationEvaluation": "Shaders/DepthPass_Quads.tese.spv" }, "depthPass": { "vertex": "Shaders/DepthPass_Tess.vert.spv", "tessellationControl": "DepthPass_Quads.tesc.spv", "tessellationEvaluation": "DepthPass_Quads.tese.spv" } } } } } A shader family file can indicate a root to inherit values from. The Blinn-Phong shader family pulls settings from the PBR file and just switches some of the fragment shader values. { "turboShaderFamily" : { "root": "PBR.json", "OPAQUE": { "default": { "base": { "fragment": "Shaders/Blinn-Phong.frag.spv" } }, "isolines": { "base": { "fragment": "Shaders/Blinn-Phong_Tess.frag.spv" } }, "triangles": { "base": { "fragment": "Shaders/Blinn-Phong_Tess.frag.spv" } }, "quads": { "base": { "fragment": "Shaders/Blinn-Phong_Tess.frag.spv" } } } } } If you want to implement a custom shader, this is more work because you have to define all your changes for each possible shader variation. But once that is done, you have a new shader that will work with all of these different settings, which in the end is easier. I considered making a more complex inheritance / cascading schema but I think eliminating ambiguity is the most important goal in this and that should override any concern about the verbosity of these files. After all, I only plan on providing a couple of these files and you aren't going to need any more unless you are doing a lot of custom shaders. And if you are, this is the best solution for you anyways. Consequently, the baseShader, depthShader, etc. values in the material file definition are going away. Leadwerks .mat files will always use the Blinn-Phong shader family, and there is no way to change this without creating a material file in the new JSON material format. The shader class is no longer derived from the Asset class because it doesn't correspond to a single file. Instead, it is just a dumb container. A ShaderModule class derived from the Asset class has been added, and this does correspond with a single .spv file. But you, the user, won't really need to deal with any of this. The result of this is that one material will work with tessellation enabled or disabled, quad, triangle, or line meshes, and animated meshes. I also added an optional parameter in the CreatePlane(), CreateBox(), and CreateQuadSphere() commands that will create these primitives out of quads instead of triangles. The main reason for supporting quad meshes is that the tessellation is cleaner when quads are used. (Note that Vulkan still displays quads in wireframe mode as if they are triangles. I think the renderer probably converts them to normal triangles after the tessellation stage.) I also was able to implement PN Quads, which is a quad version of the Bezier curve that PN Triangles add to tessellation. Basically all the complexity is being packed into the shader family file so that these decisions only have to be made once instead of thousands of times for each different material.
  2. With tessellation now fully implemented, I was very curious to see how it would perform when applied to arbitrary models. With tessellation, vertices act like control points for a Bezier mesh that is subdivided dynamically in screen space. Could tessellation be used to add new details to any low-poly model? Here is a low-res character model with a pointy head and obvious sharp edges all around his silhouette: When tessellation is enabled, the sharp edges go away and the mesh magically appears more detailed: Let's take a look at the "Doom Seeker" enemy model from Doom 3. The original model is 640 triangles: Tessellation gets rid of the blocky outline. And displacement adds new details to the model. What about other shapes? Can this technique be used to turn any low-poly model into a high-res version? Here is a tire model from "The Zone" DLC. Even with displacement set to zero on edge vertices, cracks still appear. So my conclusion on this is that tessellation is fantastic for closed organic shapes. In fact, I think it totally replaces separate LOD meshes for rocks, characters, and other organic shapes. However, it is not something you should use everywhere without any discretion.
  3. Previously I talked about the technical details of hardware tessellation and what it took to make it truly useful. In this article I will talk about some of the implications of this feature and the more advanced ramifications of baking tessellation into Turbo Game Engine as a first-class feature in the Although hardware tessellation has been around for a few years, we don't see it used in games that often. There are two big problems that need to be overcome. We need a way to prevent cracks from appearing along edges. We need to display a consistent density of triangles on the screen. Too many polygons is a big problem. I think these issues are the reason you don't really see much use of tessellation in games, even today. However, I think my research this week has created new technology that will allow us to make use of tessellation as an every-day feature in our new Vulkan renderer. Per-Vertex Displacement Scale Because tessellation displaces vertices, any discrepancy in the distance or direction of the displacement, or any difference in the way neighboring polygons are subdivided, will result in cracks appearing in the mesh. To prevent unwanted cracks in mesh geometry I added a per-vertex displacement scale value. I packed this value into the w component of the vertex position, which was not being used. When the displacement strength is set to zero along the edges the cracks disappear: Segmented Primitives With the ability to control displacement on a per-vertex level, I set about implementing more advanced model primitives. The basic idea is to split up faces so that the edge vertices can have their displacement scale set to zero to eliminate cracks. I started with a segmented plane. This is a patch of triangles with a user-defined size and resolution. The outer-most vertices have a displacement value of 0 and the inner vertices have a displacement of 1. When tessellation is applied to the plane the effect fades out as it reaches the edges of the primitive: I then used this formula to create a more advanced box primitive. Along the seam where the edges of each face meet, the displacement smoothly fades out to prevent cracks from appearing. The same idea was applied to make segmented cylinders and cones, with displacement disabled along the seams. Finally, a new QuadSphere primitive was created using the box formula, and then normalizing each vertex position. This warps the vertices into a round shape, creating a sphere without the texture warping that spherical mapping creates. It's amazing how tessellation and displacement can make these simple shapes look amazing. Here is the full list of available commands: shared_ptr<Model> CreateBox(shared_ptr<World> world, const float width = 1.0); shared_ptr<Model> CreateBox(shared_ptr<World> world, const float width, const float height, const float depth, const int xsegs = 1, const int ysegs = 1); shared_ptr<Model> CreateSphere(shared_ptr<World> world, const float radius = 0.5, const int segments = 16); shared_ptr<Model> CreateCone(shared_ptr<World> world, const float radius = 0.5, const float height = 1.0, const int segments = 16, const int heightsegs = 1, const int capsegs = 1); shared_ptr<Model> CreateCylinder(shared_ptr<World> world, const float radius = 0.5, const float height=1.0, const int sides = 16, const int heightsegs = 1, const int capsegs = 1); shared_ptr<Model> CreatePlane(shared_ptr<World> world, cnst float width=1, const float height=1, const int xsegs = 1, const int ysegs = 1); shared_ptr<Model> CreateQuadSphere(shared_ptr<World> world, const float radius = 0.5, const int segments = 8); Edge Normals I experimented a bit with edges and got some interesting results. If you round the corner by setting the vertex normal to point diagonally, a rounded edge appears. If you extend the displacement scale beyond 1.0 you can get a harder extended edge. This is something I will experiment with more. I think CSG brush smooth groups could be used to make some really nice level geometry. Screen-space Tessellation LOD I created an LOD calculation formula that attempts to segment polygons into a target size in screen space. This provides a more uniform distribution of tessellated polygons, regardless of the original geometry. Below are two cylinders created with different segmentation settings, with tessellation disabled: And now here are the same meshes with tessellation applied. Although the less-segmented cylinder has more stretched triangles, they both are made up of triangles about the same size. Because the calculation works with screen-space coordinates, objects will automatically adjust resolution with distance. Here are two identical cylinders at different distances. You can see they have roughly the same distribution of polygons, which is what we want. The same amount of detail will be used to show off displaced edges at any distance. We can even set a threshold for the minimum vertex displacement in screen space and use that to eliminate tessellation inside an object and only display extra triangles along the edges. This allows you to simply set a target polygon size in screen space without adjusting any per-mesh properties. This method could have prevented the problems Crysis 2 had with polygon density. This also solves the problem that prevented me from using tessellation for terrain. The per-mesh tessellation settings I worked on a couple days ago will be removed since it is not needed. Parallax Mapping Fallback Finally, I added a simple parallax mapping fallback that gets used when tessellation is disabled. This makes an inexpensive option for low-end machines that still conveys displacement. Next I am going to try processing some models that were not designed for tessellation and see if I can use tessellation to add geometric detail to low-poly models without any cracks or artifacts.
  4. @Rick Maybe because models have to be created without UV seams or cracks in displacement will appear. I am adding a displacement vertex value so that displacement can be pulled in along UV seams. I see tessellation being a common thing in two areas: terrain and brushes. With brush geometry I can add some extra vertices so that displacement gets tucked in along the edges and you never have to worry about it. It's common for PBR materials to include a displacement map so I think tessellation will be much more common in Turbo. Perhaps a script tool can also be created that detects UV seams and sets those vertices' displacement to zero.
  5. Josh

    Lighting Problem

    Very strange. Can you post the contents of smoke.mat?
  6. Unfortunately this feature does not work well when additional shader stages are added to the file: https://community.khronos.org/t/how-to-create-single-file-spir-v-shaders-for-vulkan/104078
  7. Along the way I discovered problems with single-file SPIR-V shaders when you start adding tessellation stages into the mix: https://community.khronos.org/t/how-to-create-single-file-spir-v-shaders-for-vulkan/104078 So I have reverted to having a separate SPV file for each stage.
  8. Now that I have all the Vulkan knowledge I need, and most work is being done with GLSL shader code, development is moving faster. Before starting voxel ray tracing, another hard problem, I decided to work one some *relatively* easier things for a few days. I want tessellation to be an every day feature in the new engine, so I decided to work out a useful implementation of it. While there are a ton of examples out there showing how to split a triangle up into smaller triangles, useful discussion and techniques of in-game tessellation is much more rare. I think this is because there are several problems to solve before this technical feature can really be made practical. Tessellation Level The first problem is deciding how much to tessellate an object. Tessellation should use a single detail level per set of primitives being drawn. The reason for this is that cracks will appear when you apply displacement if you try to use a different tessellation level for each polygon. I solved this with some per-mesh setting for tessellation parameters. Note: In Leadwerks Game Engine, a model is an entity with one or more surfaces. Each surface has a vertex array, indice array, and a material. In Turbo Game Engine, a model contains one of more LODs, and each LOD can have one or more meshes. A mesh object in Turbo is like a surface object in Leadwerks. We are not used to per-mesh settings. In fact, the material is the only parameter meshes contained other than vertex or indice data. But for tessellation parameters, that is exactly what we need, because the density of the mesh polygons gives us an idea of how detailed the tessellation should be. This command has been added: void Mesh::SetTessellation(const float detail, const float nearrange, const float farrange) Here are what the parameters do: detail: the higher the detail, the more the polygons are split up. Default is 16. nearrange: the distance below which tessellation stops increasing. Default is 1.0 meters. farrange: the distance below which tessellation starts increasing. Default is 20.0 meters. This command gives you the ability to set properties that will give a roughly equal distribution of polygons in screen space. For convenience, a similar command has been added to the model class, which will apply the settings to all meshes in the model. Surface Displacement Another problem is culling offscreen polygons so the GPU doesn't have to process millions of extra vertices. I solved this by testing if all control vertices lie outside one edge of the camera frustum. (This is not the same as testing if any control point is inside the camera frustum, as I have seen suggested elsewhere. The latter will cause problems because it is still possible for a triangle to be visible even if all its corners are outside the view.) To account for displacement, I also tested the same vertices with maximum displacement applied. To control the amount of displacement, a scale property has been added to the displacementTexture object scheme: "displacementTexture": { "file": "./harshbricks-height5-16.png", "scale": 0.05 } A Boolean value called "tessellation" has also been added to the JSON material scheme. This will tell the engine to choose a shader that uses tessellation, so you don't need to explicitly specify a shader file. Shadows will also be rendered with tessellation, unless you explicitly choose a different shadow shader. Here is the result: Surface displacement takes the object scale into account, so if you scale up an object the displacement will increase accordingly. Surface Curvature I also added an implementation of the PN Triangles technique. This treats a triangle as control points for a Bezier curve and projects tessellated curved surfaces outwards. You can see below the shot using the PN Triangles technique eliminates the pointy edges of the sphere. The effects is good, although it is more computationally expensive, and if a strong displacement map is in use, you can't really see a difference. Since vertex positions are being changed but texture coordinates are still using the same interpolation function, it can make texture coordinates appear distorted. To counter this, texture coordinates would need to be recalculated from the new vertex positions. EDIT: I found a better algorithm that doesn't produce the texcoord errors seen above. Finally, a global tessellation factor has been added that lets you scale the effect for different hardware levels: void SetTessellationDetail(const float detail) The default setting is 1.0, so you can use this to scale up or down the detail any way you like.
  9. Wow, that was interesting.
  10. I found another optimization that I think will pretty much eliminate slowdown when shadows are redrawn.

  11. Two and a half months later, we have shadow maps working:
  12. I’m not sure, I will have to experiment with it.
  13. @gamecreator If you are asking about shadows on vegetation, there is no vegetation system currently, so that problem is solved. Variance shadow maps don't work without all objects casting shadows so I think this will be in there.
  14. @awgsknite This is supported as of LE 4.6.
  15. I love watching these videos.
  16. Shadow maps are now supported in the new Vulkan renderer, for point, spot, and box lights! Our new forward renderer eliminates all the problems that deferred renderers have with transparency, so shadows and lighting works great with transparent surfaces. Transparent objects even receive lighting from their back face automatically! There is some shadow acne, which I am not going to leave alone right now because I want to try some ideas to eliminate it completely, so you never have to adjust offsets or other settings. I also want to take another look at variance shadow maps, as these can produce much better results than depthmap shadows. I also noticed some seams in the edges of point light shadows. Another interesting thing to note is that the new renderer handles light and shadows with orthographic projection really well. A new parameter has also been added to the JSON material loader. You can add a scale factor inside the normalTexture block to make normal maps appear bumpier. The default value is 1.0: "normalTexture": { "file": "./glass_dot3.tex", "scale": 2.0 } Also, the Context class has been renamed to "Framebuffer". Use CreateFramebuffer() instead of CreateContext().

    1. gamecreator


      Congrats on the progress!  Looking forward to some screenshots.

    2. Josh
  18. Okay, I now have all the Vulkan knowledge I need to finish the renderer.

  19. Josh

    Trials and Tribulations

    Width and height of the framebuffer during rendering were set to zero. I'm back on track.
  20. After about four days of trying to get render-to-texture working in Vulkan, I have everything working except...it doesn't work. No errors, no clue what is wrong, but the renderer is not even clearing the depth attachment, which is why the texture read shown here is flat red. There's not much else to say right now. I will keep trying to find the magic combination of cryptic obscure settings it takes to make Vulkan do what I want. This is very hard stuff, but once I have it working I think that gives me all the functionality I need to finish the Vulkan renderer. Post-processing effects are just another render-to-texture problem. Now, the Vulkan renderer is much more strict than OpenGL, and I think the way post-processing will work is with a an automatically created blur texture and a single post-process shader. It's very wasteful to render a bunch of different passes for each effect, so instead I think we will just have some different effects files and a shader that can include whatever effects you want. In fact, the post-process pass can probably be done as a third Vulkan subpass which would be very efficient.
  21. I would say this is almost definitely the problem. I used to use some apps like this, but it can be very dangerous to mess around with your Windows theme with unofficial stuff. Once I tried to install some unofficial WinXP theme and the whole thing turned every element in the UI pure black. My computer started and ran but was impossible to use at all because everything was black. XD I had to reformat.
  22. Reinstalling does not affect your projects but I doubt that would change anything. This sounds like a windows problem. You could delete your config file, but I think the problem lies with your windows updates or drivers.
  23. Woot, looks like I have render-to-texture working in Vulkan.

    1. Josh


      Nevermind, there is like 20 more steps!

  24. Happy American Independence Day. Most-Patriotic-Games-Duke-Nukem.jpg.opti

  25. Now that we have lights working in our clustered forward Vulkan renderer (same great technique the latest DOOM games are using) I am starting to implement shadow maps. The first issue that came up was managing render-to-texture when the texture might still be in use rendering the previous frame. At first I thought multiple shadowmaps would be needed per light, like a double-buffering system, but that would double the number of shadow textures and video memory. Instead, I created a simple object pool which stores spare shadowmaps that can be swapped around and used as they are needed. It turns out I already have pretty much all the code I need because Vulkan's swapchain creation works exactly the same way, by rendering to a series of images. I worked through the code slowly and came up with this by the end of the day, which runs successfully without throwing any validation errors: bool RenderBuffer::Initialize(shared_ptr<RenderContext> context, const int width, const int height, const int depth, const int colorcomponents, const bool depthcomponent, const int samples) { this->device = GameEngine::Get()->renderingthreadmanager->instance; int totalcomponents = colorcomponents; if (depthcomponent) totalcomponents++; std::fill(colortexture.begin(), colortexture.end(), nullptr); depthtexture = nullptr; //Create color images for (int i = 0; i < colorcomponents; ++i) { colortexture[i] = make_shared<RenderTexture>(); colortexture[i]->Initialize(VK_IMAGE_TYPE_2D, device->chaininfo.imageFormat, width, height, 1, 0, false, -1,-1,-1,-1,1, VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT); colorimages[i] = colortexture[i]->vkimage; } //Create depth image if (depthcomponent) { depthtexture = make_shared<RenderTexture>(); depthtexture->Initialize(VK_IMAGE_TYPE_2D, device->depthformat, width, height, 1, samples, false, -1, -1, -1, -1, 1, VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT); } //Create image views imageviews.resize(totalcomponents); VkImageViewCreateInfo createInfo = {}; createInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO; createInfo.viewType = VK_IMAGE_VIEW_TYPE_2D; createInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY; createInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY; createInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY; createInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY; createInfo.subresourceRange.baseMipLevel = 0; createInfo.subresourceRange.levelCount = 1; createInfo.subresourceRange.baseArrayLayer = 0; createInfo.subresourceRange.layerCount = 1; // Create color image views for (size_t i = 0; i < colorcomponents; i++) { createInfo.image = colorimages[i]; createInfo.format = device->chaininfo.imageFormat; createInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT; VkAssert(vkCreateImageView(device->device, &createInfo, nullptr, &imageviews[i])); } //Create depth image view if (depthcomponent) { createInfo.image = depthtexture->vkimage; createInfo.format = depthtexture->format; createInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_DEPTH_BIT; VkAssert(vkCreateImageView(device->device, &createInfo, nullptr, &imageviews[colorcomponents])); } //Create framebuffer VkFramebufferCreateInfo framebufferInfo = {}; framebufferInfo.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO; framebufferInfo.renderPass = device->renderpass->pass; framebufferInfo.attachmentCount = 2; framebufferInfo.pAttachments = imageviews.data(); framebufferInfo.width = width; framebufferInfo.height = height; framebufferInfo.layers = 1; VkAssert(vkCreateFramebuffer(device->device, &framebufferInfo, nullptr, &framebuffer)); return true; } The blog cover image isn't the Vulkan renderer, I just found a random image of Leadwerks shadows.
  • Create New...