Jump to content

Single-file SPIR-V shaders for Vulkan

Josh

808 views

It is now possible to compile shaders into a single self-contained file that can loaded by any Vulkan program, but it's not obvious how this is done. After poking around for a while I found all the pieces I needed to put this together.

Compiling

First, you need to compile each shader stage from a source code file into a precompiled SPIR-V file. There are several tools available to do this, but I prefer GLSlangValidator because it supports the Google #include extension. Put your vertex shader code in a text file named "shader.vert" and your pixel shader code in a file called "shader.frag". Create a .bat file in the same directory with the following contents:

glslangValidator.exe "shader.vert" -V -o "vert.spv"
glslangValidator.exe "shader.frag" -V -o "frag.spv"

Run the bat file and two .spv files will be saved.

Linking

Now we want to combine our two files representing different shader stages into a single file. This is done with the link tool from Khronos. Add the following lines to your .bat file to compile the two .spv files into one. It will also delete the existing files to clean things up a little:

spirv-link "vert.spv" "frag.spv" -o "shader.spv"
del "vert.spv"
del "frag.spv"

This will save a single file named "shader.spv" that you can load as one shader module and use for different stages in Vulkan.

Here are the required executables and a .bat file:
BuildShader.zip

Parsing

If you always use vertex and fragment stages then there is no problem, but what if the combined .spv file contains other stages, or is missing a fragment stage? We can easily account for this with a minimal SPIR-V file parser. We're not going to include any big bloated libraries to do this because we only need some basic information about what stages are contained in the shader. Fortunately, the SPIR-V specification is pretty simple and it doesn't take much code to extract the information we want:

std::string entrypointname[6];

auto stream = ReadFile(L"Shaders/shader.spv");

// Parse SPIR-V data
Assert(stream->ReadInt() == 0x07230203);
int version = stream->ReadInt();
int genmagnum = stream->ReadInt();
int bound = stream->ReadInt();
int reserved = stream->ReadInt();

bool stages[6] = {false,false,false,false,false,false};		

// Instruction stream
while (stream->Ended() == false)
{
	int pos = stream->GetPos();
	unsigned int bytes = stream->ReadUInt();
	int opcode = LOWORD(bytes);
	int wordcount = HIWORD(bytes);
	if (opcode == 15)
	{
		int executionmodel = stream->ReadInt();
		Assert(executionmodel >= 0);
		if (executionmodel < 6)
		{
			stream->ReadInt(); // entry point
			stages[executionmodel] = true;
			entrypointname[executionmodel] = stream->ReadString();
		}
	}
	stream->Seek(pos + wordcount * 4);
}

This code even retrieves the entry point name for each stage, so you can be sure you are loadng the shader correctly.

Here are the different shader stages from the SPIR-V specification:

  • 0: Vertex
  • 1: TessellationControl
  • 2: TessellationEvaluation
  • 3: Geometry
  • 4: Fragment
  • 5: GLCompute

That's it! We now have a standard single-file shader format for Vulkan programs. Your code for creating these will look something like this:

VkShaderModule shadermodule;

// Create shader module
VkShaderModuleCreateInfo shaderCreateInfo = {};
shaderCreateInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO;
shaderCreateInfo.codeSize = bank->GetSize();
shaderCreateInfo.pCode = reinterpret_cast<const uint32_t*>(bank->buf);
VkAssert(vkCreateShaderModule(device->device, &shaderCreateInfo, nullptr, &shadermodule));

// Create vertex stage info
VkPipelineShaderStageCreateInfo vertShaderStageInfo = {};
vertShaderStageInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO;
vertShaderStageInfo.stage = VK_SHADER_STAGE_VERTEX_BIT;
vertShaderStageInfo.module = shadermodule;
vertShaderStageInfo.pName = entrypointname[0].c_str();

VkPipelineShaderStageCreateInfo fragShaderStageInfo = {};
if (stages[4])
{
  // Create fragment stage info			
  fragShaderStageInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO;
  fragShaderStageInfo.stage = VK_SHADER_STAGE_FRAGMENT_BIT;
  fragShaderStageInfo.module = shadermodule;
  fragShaderStageInfo.pName = entrypointname[4].c_str();
}

// Create your graphics pipeline...

 



3 Comments


Recommended Comments

How will it work with custom shaders from the user? Suppose they will need to compile them.

Share this comment


Link to comment

I plan to add a tool for Visual Studio Code that will automatically call the required steps.

Then in the engine you can just call this:

auto shader = LoadShader("Shaders/myshader.spv");

 

  • Like 1

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Blog Entries

    • By Josh in Josh's Dev Blog 2
      The Leadwerks 5 beta will soon be updated with particle emitters and an example particle system plugin. Previously, I showed some impressive results with physically interactive particles that collide with and exert forces on the environment. I decided to use the plugin system for controlling particle behavior, as this offers the best performance and can be run on the physics thread. 
      A particle system plugin uses some predefined structures and functions to modify the behavior of particles when they are emitted or as they are updated. This allows for unlimited features to be added to the particle system, because anything you want can be added with a plugin. A system for sending settings to the plugin will be implemented in the future so you can adjust the plugin settings and see the results. The default particle settings and features will probably stay pretty barebones and I will just use the plugin system to add any advanced functionality since it is so flexible.
      void EmitParticle(ParticleModifier* mod, ParticleSystem* particlesystem, Particle* particle) { if (mod->emissionshape == EMISSION_SHAPE_BOX) { particle->position[0] = Random(-mod->area[0], mod->area[0]); particle->position[1] = Random(-mod->area[1], mod->area[1]); particle->position[2] = Random(-mod->area[2], mod->area[2]); } else if (mod->emissionshape == EMISSION_SHAPE_CYLINDER) { particle->position[0] = Random(-mod->area[0], mod->area[0]); particle->position[1] = Random(-mod->area[1], mod->area[1]); particle->position[2] = Random(-mod->area[2], mod->area[2]); auto l = sqrt(particle->position[0] * particle->position[0] + particle->position[1] * particle->position[1] + particle->position[2] * particle->position[2]); if (l > 0.0f) { particle->position[0] /= l; particle->position[1] /= l; particle->position[2] /= l; } } particle->position[0] += particlesystem->matrix[12]; particle->position[1] += particlesystem->matrix[13]; particle->position[2] += particlesystem->matrix[14]; } There are three other new Lua examples included. Coroutines.lua shows how a sequence of actions can be added to an entity before the game starts, and the actions will be executed in order:
      --Create model local model = CreateBox(world) --Add some behaviors to be executed in order model:AddCoroutine(MoveToPoint, Vec3(3,0,0), 2) model:AddCoroutine(MoveToPoint, Vec3(-3,0,0), 2) model:AddCoroutine(MoveToPoint, Vec3(0,0,0), 2) --Main loop while window:Closed() == false do world:Update() world:Render(framebuffer) end This is great for setting up cut scenes or other sequences of events.
      An example showing how to enable tessellation is also included. Tessellation is now a per-camera setting.
      camera:SetTessellation(10) The number you input is the size in pixels of the tessellated primitives. Use zero to disable tessellation. Tessellation is disabled by default on all cameras.
      Finally, an example showing how to use a texture loader plugin is included. All you have to do is load the plugin and after that textures can be loaded in VTF format:
      local vtfloader = LoadPlugin("Plugins/VTF.dll") local tex = LoadTexture("Materials/wall01.vtf")  
    • By Josh in Josh's Dev Blog 4
      I made some changes to the design of the particle system. I am less concerned with the exact behavior of particles as they move around and move interested right now in building a system with good performance and deep physics interactions. Although I want particle behavior to be customizable, I don't think scripts are the right tool for the job. C++ plugins are better suited for this for two reasons.
      C++ is much faster, and particles are a system that will make heavy use of that. Lua scripts can't be run on separate threads. In Leadwerks Engine 4 we have basic particle collisions, but I wanted something more interactive in the new system. I move the particle update code into the physics thread. I implemented collision as well as the ability for particles to exert forces on other objects. Here's what happens when some slow-moving smoke particles interact with a scene: The lower platform rotates freely while the upper platform is motorized.
      When the particle velocity is increase they start to behave like a stream of water:
      Best of all, the speed is surprisingly fast. 4000 particles with collision update in just 2 milliseconds. The code scales well across cores so if you have a lot of CPU cores simulations with 100,000 particles are possible.
      Right now particles are processed in the physics thread, and get sent to the rendering thread for display, but right now the main thread actually never sees the individual particles.
      This is fast enough I think particles will default to full physics. Instead of just being a dumb visual effect we are going to have fully interactive fluids and gases. Flamethrowers can fill a room with fire and it will creep around corners to fill a space.
    • By Josh in Josh's Dev Blog 7
      For finer control over what 2D elements appear on what camera, I have implemented a system of "Sprite Layers". Here's how it works:
      A sprite layer is created in a world. Sprites are created in a layer. Layers are attached to a camera (in the same world). The reason the sprite layer is linked to the world is because the render tweening operates on a per-world basis, and it works with the sprite system just like the entity system. In fact, the rendering thread uses the same RenderNode class for both.
      I have basic GUI functionality working now. A GUI can be created directly on a window and use the OS drawing commands, or it can be created on a sprite layer and rendered with 3D graphics. The first method is how I plan to make the new editor user interface, while the second is quite flexible. The most common usage will be to create a sprite layer, attach it to the main camera, and add a GUI to appear in-game. However, you can just as easily attach a sprite layer to a camera that has a texture render target, and make the GUI appear in-game on a panel in 3D. Because of these different usages, you must manually insert events like mouse movements into the GUI in order for it to process them:
      while true do local event = GetEvent() if event.id == EVENT_NONE then break end if event.id == EVENT_MOUSE_DOWN or event.id == EVENT_MOUSE_MOVE or event.id == EVENT_MOUSE_UP or event.id == EVENT_KEY_DOWN or event.id == EVENT_KEY_UP then gui:ProcessEvent(event) end end You could also input your own events from the mouse position to create interactive surfaces, like in games like DOOM and Soma. Or you can render the GUI to a texture and interact with it by feeding in input from VR controllers.

      Because the new 2D drawing system uses persistent objects instead of drawing commands the code to display elements has changed quite a lot. Here is my current button script. I implemented a system of abstract GUI "rectangles" the script can create and modify. If the GUI is attached to a sprite layer these get translated into sprites, and if it is attached directly to a window they get translated into system drawing commands. Note that the AddTextRect doesn't even allow you to access the widget text directly because the widget text is stored in a wstring, which supports Unicode characters but is not supported by Lua.
      --Default values widget.pushed=false widget.hovered=false widget.textindent=4 widget.checkboxsize=14 widget.checkboxindent=5 widget.radius=3 widget.textcolor = Vec4(1,1,1,1) widget.bordercolor = Vec4(0,0,0,0) widget.hoverbordercolor = Vec4(51/255,151/255,1) widget.backgroundcolor = Vec4(0.2,0.2,0.2,1) function widget:MouseEnter(x,y) self.hovered = true self:Redraw() end function widget:MouseLeave(x,y) self.hovered = false self:Redraw() end function widget:MouseDown(button,x,y) if button == MOUSE_LEFT then self.pushed=true self:Redraw() end end function widget:MouseUp(button,x,y) if button == MOUSE_LEFT then self.pushed = false if self.hovered then EmitEvent(EVENT_WIDGET_ACTION,self) end self:Redraw() end end function widget:OK() EmitEvent(EVENT_WIDGET_ACTION,self) end function widget:KeyDown(keycode) if keycode == KEY_ENTER then EmitEvent(EVENT_WIDGET_ACTION,self) self:Redraw() end end function widget:Start() --Background self:AddRect(self.position, self.size, self.backgroundcolor, false, self.radius) --Border if self.hovered == true then self:AddRect(self.position, self.size, self.hoverbordercolor, true, self.radius) else self:AddRect(self.position, self.size, self.bordercolor, true, self.radius) end --Text if self.pushed == true then self:AddTextRect(self.position + iVec2(1,1), self.size, self.textcolor, TEXT_CENTER + TEXT_MIDDLE) else self:AddTextRect(self.position, self.size, self.textcolor, TEXT_CENTER + TEXT_MIDDLE) end end function widget:Draw() --Update position and size self.primitives[1].position = self.position self.primitives[1].size = self.size self.primitives[2].position = self.position self.primitives[2].size = self.size self.primitives[3].size = self.size --Update the border color based on the current hover state if self.hovered == true then self.primitives[2].color = self.hoverbordercolor else self.primitives[2].color = self.bordercolor end --Offset the text when button is pressed if self.pushed == true then self.primitives[3].position = self.position + iVec2(1,1) else self.primitives[3].position = self.position end end This is arguably harder to use than the Leadwerks 4 system, but it gives you advanced capabilities and better performance that the previous design did not allow.
×
×
  • Create New...