Jump to content

Leadwerks 5 beta 2D Redux

Josh

453 views

Previously, we saw how the new renderer can combine multiple cameras and even multiple worlds in a single render to combine 3D and 2D graphics. During the process of implementing Z-sorting for multiple layers of transparency, I found that Vulkan does in fact respect rasterization order. That is, objects are in fact drawn in the same order you provide draw calls to a command buffer.

Quote

Within a subpass of a render pass instance, for a given (x,y,layer,sample) sample location, the following operations are guaranteed to execute in rasterization order, for each separate primitive that includes that sample location:

  1. Scissor test
  2. Sample mask generation
  3. Depth bounds test
  4. Stencil test, stencil op and stencil write
  5. Depth test and depth write
  6. Sample counting for occlusion queries
  7. coverage reduction
  8. Blending, logic operations, and color writes

Each of these operations is atomically executed for each primitive and sample location.

Execution of these operations for each primitive in a subpass occurs in primitive order.

Furthermore, individual primitives (polygons) are also rendered in the order they are stored in the indice buffer:

Quote

Primitives generated by drawing commands progress through the stages of the graphics pipeline in primitive order. Primitive order is initially determined in the following way:

  1. Submission order determines the initial ordering
  2. For indirect draw commands, the order in which accessed instances of the VkDrawIndirectCommand are stored in buffer, from lower indirect buffer addresses to higher addresses.
  3. If a draw command includes multiple instances, the order in which instances are executed, from lower numbered instances to higher.
  4. The order in which primitives are specified by a draw command:
    • For non-indexed draws, from vertices with a lower numbered vertexIndex to a higher numbered vertexIndex.
    • For indexed draws, vertices sourced from a lower index buffer addresses to higher addresses.

Within this order implementations further sort primitives:

  1. If tessellation shading is active, by an implementation-dependent order of new primitives generated by tessellation.
  2. If geometry shading is active, by the order new primitives are generated by geometry shading.
  3. If the polygon mode is not VK_POLYGON_MODE_FILL, by an implementation-dependent ordering of the new primitives generated within the original primitive.

Primitive order is later used to define rasterization order, which determines the order in which fragments output results to a framebuffer.

Now if you were making a 2D game with 1000 zombie sprites onscreen you would undoubtedly want to use 3D-in-2D rendering with an orthographic camera. Batching and depth discard would give you much faster performance when the number of objects goes up. However, the 2D aspect of most games is relatively simple, with only a dozen or so 2D sprites making up the user interface. Given that 2D graphics are not normally going to be much of a bottleneck, and that the biggest performance savings we have achieved was in making text a static object, I decided to rework the 2D rendering system into something that was a little simpler to use.

Sprites are no longer a 3D entity, but are a new type of pure 2D object. They act in a similar way as entities with position, rotation, and scale commands, but they only use 2D coordinates:

//Create a sprite
auto sprite = CreateSprite(world,100,100);

//Make blue
sprite->SetColor(0,0,1);

//Position in upper-left corner of screen
sprite->SetPosition(10,10)

Sprites have a handle you can set. By default this is in the upper-left corner of the sprite, but you can change it to recenter them. Sprites can also be rotated around the Z axis:

//Center the handle
sprite->SetHandle(0.5,0.5);

//Rotation around center
sprite->SetRotation(45);

SVG vector images are great for 2D drawing and GUIs because they can scale for different display resolutions. We support these as well, with an optional scale value the image can be rasterized at.

auto sprite = LoadSprite(world, "tiger.svg", 0, 2.0);

nanoSVGLogo-1.png.2efcb4bbfe35df145a11b3066a44a1b3.png

Text is now just another type of sprite:

auto text = CreateSprite(world, font, L"Hello, how are you today?\nI am fine.", 72, TEXT_LEFT);

These sprites are all displayed within the same world as the 3D rendering, so unlike what I previously wrote about...

  • You do not have to create extra cameras or worlds just to draw 2D graphics. (If you are doing something advanced then the multi-camera method I previously described is a good option, but you have to have very demanding needs for it to make a difference.)
  • Regular old screen coordinates you are used to will be used (coordinate [0,0] is top-left).

By default sprites will be drawn in the order they are created. However, I definitely see a need for additional control here and I am open to ideas. Should there be a sprite order value, a MoveToFront() method, or a system of different layers? I'm not sure yet.

I'm also not sure how per-camera sprites will be controlled. At this time sprites are stored in a per-world list, but we will want some 2D elements to only appear on some cameras. I am not sure yet how this will be controlled.

I am going to try to get an update out soon with these features so you can try them out yourself.

  • Like 4


3 Comments


Recommended Comments

Quote

Sprites are no longer a 3D entity, but are a new type of pure 2D object. They act in a similar way as entities with position, rotation, and scale commands, but they only use 2D coordinates:

So, what about the 3D sprites? What would they be called? Billboards?

Overall, this seems much more easier to understand. With SVGs, we can now have round HUD elements without fears of rasterization! :D

Share this comment


Link to comment
34 minutes ago, reepblue said:

So, what about the 3D sprites? What would they be called? Billboards?

Something like that. Yeah, I think this account for 99% of use cases.

  • Like 1

Share this comment


Link to comment
Quote

I'm also not sure how per-camera sprites will be controlled. At this time sprites are stored in a per-world list, but we will want some 2D elements to only appear on some cameras.

One possibility could be linking sprites to cameras?  If they're not linked to any camera, they are visible in all cameras (the default).

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Blog Entries

    • By Josh in Josh's Dev Blog 2
      Previously I talked about the technical details of hardware tessellation and what it took to make it truly useful. In this article I will talk about some of the implications of this feature and the more advanced ramifications of baking tessellation into Turbo Game Engine as a first-class feature in the 
      Although hardware tessellation has been around for a few years, we don't see it used in games that often. There are two big problems that need to be overcome.
      We need a way to prevent cracks from appearing along edges. We need to display a consistent density of triangles on the screen. Too many polygons is a big problem. I think these issues are the reason you don't really see much use of tessellation in games, even today. However, I think my research this week has created new technology that will allow us to make use of tessellation as an every-day feature in our new Vulkan renderer.
      Per-Vertex Displacement Scale
      Because tessellation displaces vertices, any discrepancy in the distance or direction of the displacement, or any difference in the way neighboring polygons are subdivided, will result in cracks appearing in the mesh.

      To prevent unwanted cracks in mesh geometry I added a per-vertex displacement scale value. I packed this value into the w component of the vertex position, which was not being used. When the displacement strength is set to zero along the edges the cracks disappear:

      Segmented Primitives
      With the ability to control displacement on a per-vertex level, I set about implementing more advanced model primitives. The basic idea is to split up faces so that the edge vertices can have their displacement scale set to zero to eliminate cracks. I started with a segmented plane. This is a patch of triangles with a user-defined size and resolution. The outer-most vertices have a displacement value of 0 and the inner vertices have a displacement of 1. When tessellation is applied to the plane the effect fades out as it reaches the edges of the primitive:

      I then used this formula to create a more advanced box primitive. Along the seam where the edges of each face meet, the displacement smoothly fades out to prevent cracks from appearing.

      The same idea was applied to make segmented cylinders and cones, with displacement disabled along the seams.


      Finally, a new QuadSphere primitive was created using the box formula, and then normalizing each vertex position. This warps the vertices into a round shape, creating a sphere without the texture warping that spherical mapping creates.

      It's amazing how tessellation and displacement can make these simple shapes look amazing. Here is the full list of available commands:
      shared_ptr<Model> CreateBox(shared_ptr<World> world, const float width = 1.0); shared_ptr<Model> CreateBox(shared_ptr<World> world, const float width, const float height, const float depth, const int xsegs = 1, const int ysegs = 1); shared_ptr<Model> CreateSphere(shared_ptr<World> world, const float radius = 0.5, const int segments = 16); shared_ptr<Model> CreateCone(shared_ptr<World> world, const float radius = 0.5, const float height = 1.0, const int segments = 16, const int heightsegs = 1, const int capsegs = 1); shared_ptr<Model> CreateCylinder(shared_ptr<World> world, const float radius = 0.5, const float height=1.0, const int sides = 16, const int heightsegs = 1, const int capsegs = 1); shared_ptr<Model> CreatePlane(shared_ptr<World> world, cnst float width=1, const float height=1, const int xsegs = 1, const int ysegs = 1); shared_ptr<Model> CreateQuadSphere(shared_ptr<World> world, const float radius = 0.5, const int segments = 8); Edge Normals
      I experimented a bit with edges and got some interesting results. If you round the corner by setting the vertex normal to point diagonally, a rounded edge appears.

      If you extend the displacement scale beyond 1.0 you can get a harder extended edge.

      This is something I will experiment with more. I think CSG brush smooth groups could be used to make some really nice level geometry.
      Screen-space Tessellation LOD
      I created an LOD calculation formula that attempts to segment polygons into a target size in screen space. This provides a more uniform distribution of tessellated polygons, regardless of the original geometry. Below are two cylinders created with different segmentation settings, with tessellation disabled:

      And now here are the same meshes with tessellation applied. Although the less-segmented cylinder has more stretched triangles, they both are made up of triangles about the same size.

      Because the calculation works with screen-space coordinates, objects will automatically adjust resolution with distance. Here are two identical cylinders at different distances.

      You can see they have roughly the same distribution of polygons, which is what we want. The same amount of detail will be used to show off displaced edges at any distance.

      We can even set a threshold for the minimum vertex displacement in screen space and use that to eliminate tessellation inside an object and only display extra triangles along the edges.

      This allows you to simply set a target polygon size in screen space without adjusting any per-mesh properties. This method could have prevented the problems Crysis 2 had with polygon density. This also solves the problem that prevented me from using tessellation for terrain. The per-mesh tessellation settings I worked on a couple days ago will be removed since it is not needed.
      Parallax Mapping Fallback
      Finally, I added a simple parallax mapping fallback that gets used when tessellation is disabled. This makes an inexpensive option for low-end machines that still conveys displacement.

      Next I am going to try processing some models that were not designed for tessellation and see if I can use tessellation to add geometric detail to low-poly models without any cracks or artifacts.
    • By Josh in Josh's Dev Blog 0
      For finer control over what 2D elements appear on what camera, I have implemented a system of "Sprite Layers". Here's how it works:
      A sprite layer is created in a world. Sprites are created in a layer. Layers are attached to a camera (in the same world). The reason the sprite layer is linked to the world is because the render tweening operates on a per-world basis, and it works with the sprite system just like the entity system. In fact, the rendering thread uses the same RenderNode class for both.
      I have basic GUI functionality working now. A GUI can be created directly on a window and use the OS drawing commands, or it can be created on a sprite layer and rendered with 3D graphics. The first method is how I plan to make the new editor user interface, while the second is quite flexible. The most common usage will be to create a sprite layer, attach it to the main camera, and add a GUI to appear in-game. However, you can just as easily attach a sprite layer to a camera that has a texture render target, and make the GUI appear in-game on a panel in 3D. Because of these different usages, you must manually insert events like mouse movements into the GUI in order for it to process them:
      while true do local event = GetEvent() if event.id == EVENT_NONE then break end if event.id == EVENT_MOUSE_DOWN or event.id == EVENT_MOUSE_MOVE or event.id == EVENT_MOUSE_UP or event.id == EVENT_KEY_DOWN or event.id == EVENT_KEY_UP then gui:ProcessEvent(event) end end You could also input your own events from the mouse position to create interactive surfaces, like in games like DOOM and Soma. Or you can render the GUI to a texture and interact with it by feeding in input from VR controllers.

      Because the new 2D drawing system uses persistent objects instead of drawing commands the code to display elements has changed quite a lot. Here is my current button script. I implemented a system of abstract GUI "rectangles" the script can create and modify. If the GUI is attached to a sprite layer these get translated into sprites, and if it is attached directly to a window they get translated into system drawing commands. Note that the AddTextRect doesn't even allow you to access the widget text directly because the widget text is stored in a wstring, which supports Unicode characters but is not supported by Lua.
      --Default values widget.pushed=false widget.hovered=false widget.textindent=4 widget.checkboxsize=14 widget.checkboxindent=5 widget.radius=3 widget.textcolor = Vec4(1,1,1,1) widget.bordercolor = Vec4(0,0,0,0) widget.hoverbordercolor = Vec4(51/255,151/255,1) widget.backgroundcolor = Vec4(0.2,0.2,0.2,1) function widget:MouseEnter(x,y) self.hovered = true self:Redraw() end function widget:MouseLeave(x,y) self.hovered = false self:Redraw() end function widget:MouseDown(button,x,y) if button == MOUSE_LEFT then self.pushed=true self:Redraw() end end function widget:MouseUp(button,x,y) if button == MOUSE_LEFT then self.pushed = false if self.hovered then EmitEvent(EVENT_WIDGET_ACTION,self) end self:Redraw() end end function widget:OK() EmitEvent(EVENT_WIDGET_ACTION,self) end function widget:KeyDown(keycode) if keycode == KEY_ENTER then EmitEvent(EVENT_WIDGET_ACTION,self) self:Redraw() end end function widget:Start() --Background self:AddRect(self.position, self.size, self.backgroundcolor, false, self.radius) --Border if self.hovered == true then self:AddRect(self.position, self.size, self.hoverbordercolor, true, self.radius) else self:AddRect(self.position, self.size, self.bordercolor, true, self.radius) end --Text if self.pushed == true then self:AddTextRect(self.position + iVec2(1,1), self.size, self.textcolor, TEXT_CENTER + TEXT_MIDDLE) else self:AddTextRect(self.position, self.size, self.textcolor, TEXT_CENTER + TEXT_MIDDLE) end end function widget:Draw() --Update position and size self.primitives[1].position = self.position self.primitives[1].size = self.size self.primitives[2].position = self.position self.primitives[2].size = self.size self.primitives[3].size = self.size --Update the border color based on the current hover state if self.hovered == true then self.primitives[2].color = self.hoverbordercolor else self.primitives[2].color = self.bordercolor end --Offset the text when button is pressed if self.pushed == true then self.primitives[3].position = self.position + iVec2(1,1) else self.primitives[3].position = self.position end end This is arguably harder to use than the Leadwerks 4 system, but it gives you advanced capabilities and better performance that the previous design did not allow.
    • By Roland in ZERO 5
      For this Zero project I have a need to call C++ classes from LUA. I have gone the same way as Josh has and used 'tolua++' for this. This works like this
       
      1 - Create PKG
      For each class you need to export you have to write a corresponding 'pkg' file which is a version of the C++ header file suitable for the 'tolua++' program.
       
      2 - Use 'tolua++'
      Then send the 'pkg' file to 'tolua++' which then will generate a source file with the LUA-export version of the class and a header file which defines the function to call in order to export the class.
       
      3 - Add & Compile
      The two generated files should be included in your C++ project and you have to call the function defined in the header at some time after LUA has been initialized by Leadwerks. After compilation and linking you should be able to use the C++ class in you LUA scripts
       
      Here is a simple example:
       
      C++ header file: CppTest.h

      #pragma once #include <string> class CppTest { int _value; std::string _string; public: CppTest(); virtual ~CppTest(); int get_value() const; void set_value( int value ); std::string get_string() const; void set_string( const std::string& value ); };
       
      C++ source file: CppTest.cpp

      #include "CppTest.h" CppTest::CppTest() { _value = 0; } CppTest::~CppTest() { } int CppTest::get_value() const { return _value; } void CppTest::set_value( int value ) { _value = value; } std::string CppTest::get_string() const { return _string; } void CppTest::set_string( const std::string& value ) { _string = value; }
       
      PKG file: CppTest.pkg

      $#include <string> $#include "CppTest.h" class CppTest : public Temp { CppTest(); virtual ~CppTest(); int get_value() const; void set_value( int value ); std::string get_string() const; void set_string( const std::string& value ); };
       
      After you written the CppTest.pkg file you have to compile it using 'tolua++' like this. Note that I use the same filename for the outputs but with an '_' added. That way its easy to keep track of things
       

      tolua++ -o CppTest_.cpp -n CppTest -H CppTest_.h CppTest.pkg

       
      Now tolua should have generated two files.
       
      Generated CppTest_.h

      /* ** Lua binding: CppTest ** Generated automatically by tolua++-1.0.92 on 08/18/16 11:36:39. */ /* Exported function */ TOLUA_API int tolua_CppTest_open (lua_State* tolua_S);
       
      Generated CppTest_.cpp (shortned down)

      /* ** Lua binding: CppTest ** Generated automatically by tolua++-1.0.92 on 08/18/16 11:36:39. */ #ifndef __cplusplus #include "stdlib.h" #endif #include "string.h" #include "tolua++.h" /* Exported function */ TOLUA_API int tolua_CppTest_open (lua_State* tolua_S); #include <string> #include "CppTest.h" /* function to release collected object via destructor */ #ifdef __cplusplus static int tolua_collect_CppTest (lua_State* tolua_S) { CppTest* self = (CppTest*) tolua_tousertype(tolua_S,1,0); delete self; return 0; } #endif --- snip --- snip --- snip --- --- snip --- snip --- snip --- --- snip --- snip --- snip --- /* Open function */ TOLUA_API int tolua_CppTest_open (lua_State* tolua_S) { tolua_open(tolua_S); tolua_reg_types(tolua_S); tolua_module(tolua_S,NULL,0); tolua_beginmodule(tolua_S,NULL); #ifdef __cplusplus tolua_cclass(tolua_S,"CppTest","CppTest","Temp",tolua_collect_CppTest); #else tolua_cclass(tolua_S,"CppTest","CppTest","Temp",NULL); #endif tolua_beginmodule(tolua_S,"CppTest"); tolua_function(tolua_S,"new",tolua_CppTest_CppTest_new00); tolua_function(tolua_S,"new_local",tolua_CppTest_CppTest_new00_local); tolua_function(tolua_S,".call",tolua_CppTest_CppTest_new00_local); tolua_function(tolua_S,"delete",tolua_CppTest_CppTest_delete00); tolua_function(tolua_S,"get_value",tolua_CppTest_CppTest_get_value00); tolua_function(tolua_S,"set_value",tolua_CppTest_CppTest_set_value00); tolua_function(tolua_S,"get_string",tolua_CppTest_CppTest_get_string00); tolua_function(tolua_S,"set_string",tolua_CppTest_CppTest_set_string00); tolua_endmodule(tolua_S); tolua_endmodule(tolua_S); return 1; } #if defined(LUA_VERSION_NUM) && LUA_VERSION_NUM >= 501 TOLUA_API int luaopen_CppTest (lua_State* tolua_S) { return tolua_CppTest_open(tolua_S); }; #endif
      You should include both of those in your project. You also have to call tolua_CppTest_open somewhere after Leadwerks has initialized LUA. I do it here in my App.cpp. Remember to #include "CppTest_.h" at the top of App.cpp.
       
      App.cpp (shortned down)

      #include "App.h" #include "CppTest_.h" using namespace Leadwerks; App::App() : window(NULL), context(NULL), world(NULL), camera(NULL) {} App::~App() { delete world; delete window; } bool App::Start() { int stacksize = Interpreter::GetStackSize(); //Get the global error handler function int errorfunctionindex = 0; #ifdef DEBUG Interpreter::GetGlobal("LuaErrorHandler"); errorfunctionindex = Interpreter::GetStackSize(); #endif //Create new table and assign it to the global variable "App" Interpreter::NewTable(); Interpreter::SetGlobal("App"); std::string scriptpath = "Scripts/Main.lua"; if (FileSystem::GetFileType("Scripts/App.Lua") == 1) scriptpath = "Scripts/App.Lua"; // ADDED to initialize the CppTest LUA implemenation tolua_CppTest_open(Interpreter::L); //Invoke the start script if (!Interpreter::ExecuteFile(scriptpath)) { System::Print("Error: Failed to execute script \"" + scriptpath + "\"."); return false; } --- snip --- snip --- snip --- --- snip --- snip --- snip --- --- snip --- snip --- snip ---
      LuaParser
      I read somewhere that Josh has a parser that automates this a bit by parsing lines in the header file that is commented with //lua and generates pkg-files. What a good idea. As I think programming is better that watching lousy programs on TV, I made my own version of this and called it "LuaParser". I have attached this program for those who like to use it. Here is what it does
       
      1. Parses all C++ header files (*.h) in the folder and subfolders for lines commented with //lua
      2. For such files it creates a pkg file suitable for tolua++ compilation
      3. It complies the generated pkg files into _.h and _.cpp files to be included into you project
       
      Here is same example from above declared for LuaParsing
       
      C++ header file: CppTest.h prepared for LuaParser
      #pragma once #include <string>//lua class CppTest { int _value; std::string _string; public: CppTest();//lua virtual ~CppTest();//lua int get_value() const;//lua void set_value( int value );//lua std::string get_string() const;//lua void set_string( const std::string& value );//lua };
       
      Extract the LuaParser.zip into you Source folder and open a command prompt there. The just type LuaParser and hit Enter. A number of files ending with '_' in the name will be generated. Include them in your project and call the function in each of the '_.h' files as mentioned.
       
      Windows version
      LuaParserWin-1.5.zip
       
      Linux version
      LuaParserLinux-1.0.tar.gz - support discontinued
       
      History
      1.0 Initial version
      1.1 Comment header in PKG files was inside class declaration instead of at top of file
      1.2 Didn't handle class inheritance
      1.3 - 1.5 Various minor fixes
       
      You can read more about tolua++ here
      tolua++ - Reference Manual
×
×
  • Create New...