Jump to content


  • Content Count

  • Joined

  • Last visited

Blog Entries posted by JMK

  1. JMK
    A new update is available to beta testers. This makes some pretty big changes so I wanted to release this before doing any additional work on the post-processing effects system.
    Terrain Fixed
    Terrain system is working again, with an example for Lua and C++.
    New Configuration Options
    New settings have been added in the "Config/settings.json" file:
    "MultipassCubemap": false, "MaxTextures": 512, "MaxCubemaps": 16, "MaxShadowmaps": 64, "MaxIntegerTextures": 32, "MaxUIntegerTextures": 32, "MaxCubeShadowmaps": 64, "MaxVolumeTextures": 16, "LuaErrorCommand": "code", "LuaErrorCommandArguments": "-g \"$(CurrentFile)\":$(LineNumber) \"$(AppDir)\"" The max texture values will allow you to reduce the array size the engine requires for textures. If you have gotten an error message about "not enough texture units" this setting can be used to bring your application down under the limit your hardware has.
    The Lua settings define the command that is run when a Lua error occurs. By default this will open Visual Studio code and display the file and line number an error occurs on. 
    String Classes
    I've implemented two string classes for better string handling. The String and WString class are derived from both the std::string / wstring AND the Object class, which means they can be used in a variable that accepts an object (like the Event.source member). 8-bit character strings will automatically convert to wide strings, but not the other way. All the Load commands used to have two overloads, one for narrow and one for wide strings. That has been replaced with a single command that accepts a WString, so you can call LoadTexture("brick,dds") without having to specify a wide string like this: L"brick.dds".
    The global string functions like Trim, Right, Mid, etc. have been added as methods on the two string classes, Eventually the global functions will be phased out.
    Lua Integration in Visual Studio Code
    Lua integration in Visual Studio Code is just about finished and it's amazing! Errors are displayed, debugging works great, and console output is displayed, just like any serious modern programming language. Developing with Lua in Leadwerks 5 is going to be a blast!

    Lua launch options are now available for Debug, Release, Debug 64f, and Release 64f.
    I feel the Lua support is good enough now that the .bat files are not needed. It's easier just to open VSCode and copy the example you want to run into Main.lua. These are currently located in "Scripts/Examples" but they will be moved into the documentation system in time.
    The black console window is going away and all executables are by default compiled as a windowed application, not a console app. The console output is still available in Visual Studio in the debug output, or it can be piped to a file with a .bat launcher.
    See the notes here on how to get started with VSCode and Lua.
  2. JMK
    A new update is available for beta testers.
    The dCustomJoints and dContainers DLLs are now optional if your game is not using any joints (even if you are using physics).
    The following methods have been added to the collider class. These let you perform low-level collision tests yourself:
    Collider::ClosestPoint Collider::Collide Collider::GetBounds Collider::IntersectsPoint Collider::Pick The PluginSDK now supports model saving and an OBJ save plugin is provided. It's very easy to convert models this way using the new Model::Save() method:
    auto plugin = LoadPlugin("Plugins/OBJ.dll"); auto model = LoadModel(world,"Models/Vehicles/car.mdl"); model->Save("car.obj"); Or create models from scratch and save them:
    auto box = CreateBox(world,10,2,10); box->Save("box.obj"); I have used this to recover some of my old models from Leadwerks 2 and convert them into GLTF format:

    There is additional documentation now on the details of the plugin system and all the features and options.
    Thread handling is improved so you can run a simple application that handles 3D objects and exits out without ever initializing graphics.
    Increased strictness of headers for private and public members and methods.
    Fixed a bug where directional lights couldn't be hidden. (Check out the example for the CreateLight command in the new docs.)
    All the Lua scripts in the "Scripts\Start" folder are now executed when the engine initializes, instead of when the first script is run. These will be executed for all programs automatically, so it is useful for automatically loading plugins or workflows. If you don't want to use Lua at all, you can delete the "Scripts" folder and the Lua DLL, but you will need to load any required plugins yourself with the LoadPlugin command.
    Shadow settings are simplified. In Leadwerks 4, entities could be set to static or dynamic shadows, and lights could use a combination of static, dynamic, and buffered modes. You can read the full explanation of this feature in the documentation here. In Leadwerks 5, I have distilled that down to two commands. Entity::SetShadows accepts a boolean, true to cast shadows and false not to. Additionally, there is a new Entity::MakeStatic method. Once this is called on an entity it cannot be moved or changed in any way until it is deleted. If MakeStatic() is called on a light, the light will store an intermediate cached shadowmap of all static objects. When a dynamic object moves and triggers a shadow redraw, the light will copy the static shadow buffer to the shadow map and then draw any dynamic objects in its range. For example, if a character walks across a room with a single point light, the character model has to be drawn six times but the static scene geometry doesn't have to be redrawn at all. This can result in an enormous reduction of rendered polygons. (This is something id Software's Doom engine does, although I implemented it first.)
    In the documentation example the shadow polygon count is 27000 until I hit the space key to make the light static. The light then renders the static scene (everything except the fan blade) into an image, there thereafter that cached image is coped to the shadow map before the dynamic scene objects are drawn. This results in the shadow polygons rendered to drop by a lot, since the whole scene does not have to be redrawn each frame.

    I've started using animated GIFs in some of the documentation pages and I really like it. For some reason GIFs feel so much more "solid" and stable. I always think of web videos as some iframe thing that loads separately, lags and doesn't work half the time, and is embedded "behind" the page, but a GIF feels like it is a natural part of the page.

    My plan is to put 100% of my effort into the documentation and make that as good as possible. Well, if there is an increased emphasis on one thing, that necessarily means a decreased emphasis on something else. What am I reducing? I am not going to create a bunch of web pages explaining what great features we have, because the documentation already does that. I also am not going to attempt to make "how to make a game" tutorials. I will leave that to third parties, or defer it into the future. My job is to make attractive and informative primary reference material for people who need real usable information, not to teach non-developers to be developers. That is my goal with the new docs.
  3. JMK
    A new update is available to beta testers.
    I updated the project to the latest Visual Studio 16.6.2 and adjusted some settings. Build speeds are massively improved. A full rebuild of your game in release mode will now take less than ten seconds. A normal debug build, where just your game code changes, will take about two seconds. (I found that "Whole program optimization" completely does not work in the latest VS and when I disabled it everything was much faster. Plus there's the precompiled header I added a while back.)
    Delayed DLL loading is enabled. This makes it so the engine only loads DLLs when they are needed. If they aren't used by your application, they don't have to be included. If you are not making a VR game, you do not need to include the OpenVR DLL. You can create a small utility application that requires no DLLs in as little as 500 kilobytes. It was also found that the dContainers lib from Newton Dynamics is not actually needed, although the corresponding DLLs are (if your application uses physics).
    A bug in Visual Studio was found that requires all release builds add the setting "/OPT:NOREF,NOICF,NOLBR" in the linker options:
    A new StringObject class derived from both the WString and Object classes is added. This allows the FileSystemWatcher to store the file path in the event source member when an event occurs. A file rename event will store the old file name in the event.extra member.
    The Entity::Pick syntax is changes slightly, removing the X and Y components for the vector projected in front of the entity. See the new documentation for details.
    The API is being finalized and the new docs system has a lot of finished C++ pages. There's a lot of new stuff documented in there like message dialogs, file and folder request dialogs, world statistics, etc. The Buffer class (which replaces the LE4 "Bank" class) is official and documented. The GUI class has been renamed to "Interface".
    Documentation has been organized by area of functionality instead of class hierarchy. It feels more intuitive to me this way.

    I've also made progress using SWIG to make a wrapper for the C# programming language, with the help of @klepto2 and @carlb. It's not ready to use yet, but the feature has gone from "unknown" to "okay, this can be done". (Although SWIG also supports Lua, I think Sol2 is better suited for this purpose.)
  4. JMK
    The Leadwerks 5 beta has been updated.
    A new FileSystemWatcher class has been added. This can be used to monitor a directory and emit events when a file is created, deleted, renamed, or overwritten. See the documentation for details and an example. Texture reloading now works correctly. I have only tested reloading textures, but other assets might work as well.
    CopyFile() will now work with URLs as the source file path, turning it into a download command.
    Undocumented class methods and members not meant for end users are now made private. The goal is for 100% of public methods and members to be documented so there is nothing that appears in intellisense that you aren't allowed to use.
    Tags, key bindings, and some other experimental features are removed. I want to develop a more cohesive design for this type of stuff, not just add random ways to do things differently.
    Other miscellaneous small fixes.
  5. JMK
    An often-requested feature for terrain building commands in Leadwerks 5 is being implemented. Here is my script to create a terrain. This creates a 256 x 256 terrain with one terrain point every meter, and a maximum height of +/- 50 meters:
    --Create terrain local terrain = CreateTerrain(world,256,256) terrain:SetScale(256,100,256) Here is what it looks like:

    A single material layer is then added to the terrain.
    --Add a material layer local mtl = LoadMaterial("Materials/Dirt/dirt01.mat") local layerID = terrain:AddLayer(mtl) We don't have to do anything else to make the material appear because by default the entire terrain is set to use the first layer, if a material is available there:

    Next we will raise a few terrain points.
    --Modify terrain height for x=-5,5 do for y=-5,5 do h = (1 - (math.sqrt(x*x + y*y)) / 5) * 20 terrain:SetElevation(127 + x, 127 + y, h) end end And then we will update the normals for that whole section, all at once. Notice that we specify a larger grid for the normals update, because the terrain points next to the ones we modified will have their normals affected by the change in height of the neighboring pixel.
    --Update normals of modified and neighboring points terrain:UpdateNormals(127 - 6, 127 - 6, 13, 13) Now we have a small hill.

    Next let's add another layer and apply it to terrain points that are on the side of the hill we just created:
    --Add another layer mtl = LoadMaterial("Materials/Rough-rockface1.json") rockLayerID = terrain:AddLayer(mtl) --Apply layer to sides of hill for x=-5,5 do for y=-5,5 do slope = terrain:GetSlope(127 + x, 127 + y) alpha = math.min(slope / 15, 1.0) terrain:SetMaterial(rockLayerID, 127 + x, 127 + y, alpha) end end We could improve the appearance by giving it a more gradual change in the rock layer alpha, but it's okay for now.

    This gives you an idea of the basic terrain building API in Leadwerks 5, and it will serve as the foundation for more advanced terrain features. This will be included in the next beta.
  6. JMK
    I've moved the GI calculation over to the GPU and our Vulkan renderer in Leadwerks Game Engine 5 beta now supports volume textures. After a lot of trial and error I believe I am closing in on our final techniques. Voxel GI always involves a degree of light leakage, but this can be mitigated by setting a range for the ambient GI. I also implemented a hard reflection which was pretty easy to do. It would not be much more difficult to store the triangles in a lookup table for each voxel in order to trace a finer polygon-based ray for results that would look the same as Nvidia RTX but perform much faster.
    The video below is only performing a single GI bounce at this time, and it is displaying the lighting on the scene voxels, not on the original polygons. I am pretty pleased with this progress and I think the final results will look great and run fast. In addition, the need for environment probes placed in the scene will soon forever be a thing of the past.

    2035564276_VoxelGI_raytracingprogress.mp4.a173c8eb756aa1403cccc972a3306d49.mp4 There is still a lot of work to do on this, but I would say that this feature just went from something I was very overwhelmed and intimidated by to something that is definitely under control and feasible.
    Also, we might not use cascaded shadow maps (for directional lights) at all but instead rely on a voxel raytrace for directional light shadows. If it works, that would be my preference because CSMs waste so much space and drawing a big outdoors scene 3-4 times can be taxing.
  7. JMK
    I am happy to show you a preview of the new documentation system I am working on:

    Let's take a look at what is going on here:
    It's dark, so you can stare lovingly at it for hours without going blind. You can switch between languages with the links in the header. Lots of internal cross-linking for easy access to relevant information. Extensive, all-inclusive documentation, including Enums, file formats, constructors, and public members. Data is fetched from a Github repository and allows user contributions. I am actually having a lot of fun creating this. It is very fulfilling to be able to go in and build something with total attention to detail.
  8. JMK
    All this month I have been working on a sort of academic paper for a conference I will be speaking at towards the end of the year. This paper covers details of my work for the last three years, and includes benchmarks that demonstrate the performance gains I was able to get as a result of the new design, based on an analysis of modern graphics hardware.
    I feel like my time spent has not been very efficient. I have not written any code in a while, but it's not like I was working that whole time. I had to just let the ideas settle for a bit.
    Activity doesn't always mean progress.
    Anyways, I am wrapping up now, and am very pleased with the results. It all turned out much much better than I was expecting.
  9. JMK
    TLDR: I made a long-term bet on VR and it's paying off. I haven't been able to talk much about the details until now.
    Here's what happened:
    Leadwerks 3.0 was released during GDC 2013. I gave a talk on graphics optimization and also had a booth at the expo. Something else significant happened that week.  After the expo closed I walked over to the Oculus booth and they let me try out the first Rift prototype.
    This was a pivotal time both for us and for the entire game industry. Mobile was on the downswing but there were new technologies emerging that I wanted to take advantage of. Our Kickstarter campaign for Linux support was very successful, reaching over 200% of its goal. This coincided with a successful Greenlight campaign to bring Leadwerks Game Engine to Steam in the newly-launched software section. The following month Valve announced the development of SteamOS, a Linux-based operating system for the Steam Machine game consoles. Because of our work in Linux and our placement in Steam, I was fortunate enough to be in close contact with much of the staff at Valve Software.
    The Early Days of VR
    It was during one of my visits to Valve HQ that I was able to try out a prototype of the technology that would go on to become the HTC Vive. In September of 2014 I bought an Oculus Rift DK2 and first started working with VR in Leadwerks. So VR has been something I have worked on in the background for a long time, but I was looking for the right opportunity to really put it to work. In 2016 I felt it was time for a technology refresh, so I wrote a blog about the general direction I wanted to take Leadwerks in. A lot of it centered around VR and performance. I didn't really know exactly how things would work out, but I knew I wanted to do a lot of work with VR.
    A month later I received a message on this forum that went something like this (as I recall):
    I thought "Okay, some stupid teenager, where is my ban button?", but when I started getting emails with nasa.gov return addresses I took notice.
    Now, Leadwerks Software has a long history of use in the defense and simulation industries, with orders for software from Northrop Grumman, Lockheed Martin, the British Royal Navy, and probably some others I don't know about. So NASA making an inquiry about software isn't too strange. What was strange was that they were very interested in meeting in person.
    Mr. Josh Goes to Washington
    I took my first trip to Goddard Space Center in January 2017 where I got a tour of the facility. I saw robots, giant satellites, rockets, and some crazy laser rooms that looked like a Half-Life level. It was my eleven year old self's dream come true. I was also shown some of the virtual reality work they are using Leadwerks Game Engine for. Basically, they were taking high-poly engineering models from CAD software and putting them into a real-time visualization in VR. There are some good reasons for this. VR gives you a stereoscopic view of objects that is far superior to a flat 2D screen. This makes a huge difference when you are viewing complex mechanical objects and planning robotic movements. You just can't see things on a flat screen the same way you can see them in VR. It's like the difference between looking at a photograph of an object versus holding it in your hands.

    What is even going on here???
    CAD models are procedural, meaning they have a precise mathematical formula that describes their shape. In order to render them in real-time, they have to be converted to polygonal models, but these objects can be tens of millions of polygons, with details down to threading on individual screws, and they were trying to view them in VR at 90 frames per second! Now with virtual reality, if there is a discrepancy between what your visual system and your vestibular system perceives, you will get sick to your stomach. That's why it's critical to maintain a steady 90 Hz frame rate. The engineers at NASA told me they first tried to use Unity3D but it was too slow, which is why they came to me. Leadwerks was giving them better performance, but it still was not fast enough for what they wanted to do next. I thought "these guys are crazy, it cannot be done".
    Then I remembered something else people said could never be done.

    So I started to think "if it were possible, how would I do it?" They had also expressed interest in an inverse kinematics simulation, so I put together this robotic arm control demo in a few days, just to show what could be easily be done with our physics system.
    A New Game Engine is Born
    With the extreme performance demands of VR and my experience writing optimized rendering systems, I saw an opportunity to focus our development on something people can't live without: speed. I started building a new renderer designed specifically around the way modern PC hardware works. At first I expected to see performance increases of 2-3x. Instead what we are seeing are 10-40x performance increases under heavy loads. During this time I stayed in contact with people at NASA and kept them up to date on the capabilities of the new technology.
    At this point there was still nothing concrete to show for my efforts. NASA purchased some licenses for the Enterprise edition of Leadwerks Game Engine, but the demos I made were free of charge and I was paying my own travel expenses. The cost of plane tickets and hotels adds up quickly, and there was no guarantee any of this would work out. I did not want to talk about what I was doing on this site because it would be embarrassing if I made a lot of big plans and nothing came of it. But I saw a need for the technology I created and I figured something would work out, so I kept working away at it.
    Call to Duty
    Today I am pleased to announce I have signed a contract to put our virtual reality expertise to work for NASA. As we speak, I am preparing to travel to Washington D.C. to begin the project. In the future I plan to provide support for aerospace, defense, manufacturing, and serious games, using our new technology to help users deliver VR simulations with performance and realism beyond anything that has been possible until now.
    My software company and relationship with my customers (you) is unaffected. Development of the new engine will continue, with a strong emphasis on hyper-realism and performance. I think this is a direction everyone here will be happy with. I am going to continue to invest in the development of groundbreaking new features that will help in the aerospace and defense industries (now you understand why I have been talking about 64-bit worlds) and I think a great many people will be happy to come along for the ride in this direction.
    Leadwerks is still a game company, but our core focus is on enabling and creating hyper-realistic VR simulations. Thank you for your support and all the suggestions and ideas you have provided over the years that have helped me create great software for you. Things are about to get very interesting. I can't wait to see what you all create with the new technology we are building.
  10. JMK
    After working out a thread manager class that stores a stack of C++ command buffers, I've got a pretty nice proof of concept working. I can call functions in the game thread and the appropriate actions are pushed onto a command buffer that is then passed to the rendering thread when World::Render is called. The rendering thread is where all the (currently) OpenGL code is executed. When you create a context or load a shader, all it does is create the appropriate structure and send a request over to the rendering thread to finish the job:

    Consequently, there is currently no way of detecting if OpenGL initialization fails(!) and in fact the game will still run along just fine without any graphics rendering! We obviously need a mechanism to detect this, but it is interesting that you can now load a map and run your game without ever creating a window or graphics context. The following code is perfectly legitimate in Leawerks 5:
    #include "Leadwerks.h" using namespace Leadwerks int main(int argc, const char *argv[]) { auto world = CreateWorld() auto map = LoadMap(world,"Maps/start.map"); while (true) { world->Update(); } return 0; } The rendering thread is able to run at its own frame rate independently from the game thread and I have tested under some pretty extreme circumstances to make sure the threads never lock up. By default, I think the game loop will probably self-limit its speed to a maximum of 30 updates per second, giving you a whopping 33 milliseconds for your game logic, but this frequency can be changed to any value, or completely removed by setting it to zero (not recommended, as this can easily lock up the rendering thread with an infinite command buffer stack!). No matter the game frequency, the rendering thread runs at its own speed which is either limited by the window refresh rate, an internal clock, or it can just be let free to run as fast as possible for those triple-digit frame rates.
    Shaders are now loaded from multiple files instead of being packed into a single .shader file. When you load a shader, the file extension will be stripped off (if it is present) and the engine will look for .vert, .frag, .geom, .eval, and .ctrl files for the different shader stages:
    auto shader = LoadShader("Shaders/Model/diffuse"); The asynchronous shader compiling in the engine could make our shader editor a little bit more difficult to handle, except that I don't plan on making any built-in shader editor in the new editor! Instead I plan to rely on Visual Studio Code as the official IDE, and maybe add a plugin that tests to see if shaders compile and link on your current hardware. I found that a pragma statement can be used to indicate include files (not implemented yet) and it won't trigger any errors in the VSCode intellisense:

    Although restructuring the engine to work in this manner is a big task, I am making good progress. Smart pointers make this system really easy to work with. When the owning object in the game thread goes out of scope, its associated rendering object is also collected...unless it is still stored in a command buffer or otherwise in use! The relationships I have worked out work perfectly and I have not run into any problems deciding what the ownership hierarchy should be. For example, a context has a shared pointer to the window it belongs to, but the window only has a weak pointer to the context. If the context handle is lost it is deleted, but if the window handle is lost the context prevents it from being deleted. The capabilities of modern C++ and modern hardware are making this development process a dream come true.
    Of course with forward rendering I am getting about 2000 FPS with a blank screen and Intel graphics, but the real test will be to see what happens when we start adding lots of lights into the scene. The only reason it might be possible to write a good forward renderer now is because graphics hardware has gotten a lot more flexible. Using a variable-length for loop and using the results of a texture lookup for the coordinates of another lookup  were a big no-no when we first switched to deferred rendering, but it looks like that situation has improved.
    The increased restrictions on the renderer and the total separation of internal and user-exposed classes are actually making it a lot easier to write efficient code. Here is my code for the indice array buffer object that lives in the rendering thread:
    #include "../../Leadwerks.h" namespace Leadwerks { OpenGLIndiceArray::OpenGLIndiceArray() : buffer(0), buffersize(0), lockmode(GL_STATIC_DRAW) {} OpenGLIndiceArray::~OpenGLIndiceArray() { if (buffer != 0) { #ifdef DEBUG Assert(glIsBuffer(buffer),"Invalid indice buffer."); #endif glDeleteBuffers(1, &buffer); buffer = 0; } } bool OpenGLIndiceArray::Modify(shared_ptr<Bank> data) { //Error checks if (data == nullptr) return false; if (data->GetSize() == 0) return false; //Generate buffer if (buffer == 0) glGenBuffers(1, &buffer); if (buffer == 0) return false; //shouldn't ever happen //Bind buffer glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, buffer); //Set data if (buffersize == data->GetSize() and lockmode == GL_DYNAMIC_DRAW) { glBufferSubData(GL_ELEMENT_ARRAY_BUFFER, 0, data->GetSize(), data->buf); } else { if (buffersize == data->GetSize()) lockmode = GL_DYNAMIC_DRAW; glBufferData(GL_ELEMENT_ARRAY_BUFFER, data->GetSize(), data->buf, lockmode); } buffersize = data->GetSize(); return true; } bool OpenGLIndiceArray::Enable() { if (buffer == 0) return false; if (buffersize == 0) return false; glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, buffer); return true; } void OpenGLIndiceArray::Disable() { glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0); } } From everything I have seen, my gut feeling tells me that the new engine is going to be ridiculously fast.
    If you would like to be notified when Leadwerks 5 becomes available, be sure to sign up for the mailing list here.
  11. JMK
    This is something I typed up for some colleagues and I thought it might be useful info for C++ programmers.
    To create an object:
    shared_ptr<TypeID> type = make_shared<TypeID>(constructor args…) This is pretty verbose, so I always do this:
    auto type = make_shared<TypeID>(constructor args…) When all references to the shared pointer are gone, the object is instantly deleted. There’s no garbage collection pauses, and deletion is always instant:
    auto thing = make_shared<Thing>(); auto second_ref = thing; thing = NULL; second_ref = NULL;//poof! Shared pointers are fast and thread-safe. (Don’t ask me how.)
    To get a shared pointer within an object’s method, you need to derive the class from “enable_shared_from_this<SharedObject>”. (You can inherit a class from multiple types, remember):
    class SharedObject : public enable_shared_from_this<SharedObject> And you can implement a Self() method like so, if you want:
    shared_ptr<SharedObject> SharedObject::Self() { return shared_from_this(); } Casting a type is done like this:
    auto bird = dynamic_pointer_cast<Bird>(animal); Dynamic pointer casts will return NULL if the animal is not a bird. Static pointer casts don’t have any checks and are a little faster I guess, but there’s no reason to ever use them.
    You cannot call shared_from_this() in the constructor, because the shared pointer does not exist yet, and you cannot call it in the destructor, because the shared pointer is already gone!
    Weak pointers can be used to store a value, but will not prevent the object from being deleted:
    auto thing = make_shared<Thing>(); weak_ptr<Thing> thingwptr = thing; shared_ptr<Thing> another_ref_to_thing = thingwptr.lock(); //creates a new shared pointer to “thing” auto thing = make_shared<Thing>(); weak_ptr<Thing> thingwptr = thing; thing = NULL; shared_ptr<Thing> another_ref_to_thing = thingwptr.lock(); //returns NULL! If you want to set a weak pointer’s value to NULL without the object going out of scope, just call reset():
    auto thing = make_shared<Thing>(); weak_ptr<Thing> thingwptr = thing; thingwptr.reset(); shared_ptr<Thing> another_ref_to_thing = thingwptr.lock(); //returns NULL! Because no garbage collection is used, circular references can occur, but they are rare:
    auto son = make_shared<Child>(); auto daughter = make_shared<Child>(); son->sister = daughter; daughter->brother = son; son = NULL; daughter = NULL;//nothing is deleted! The problem above can be solved by making the sister and brother members weak pointers instead of shared pointers, thus removing the circular references.
    That’s all you need to know!
  12. JMK
    I have been spending most of my time on something else this month in preparation for the release of the Leadwerks 5 SDK. However, I did add one small feature today that has very big implications for the way the engine works. You can load a file from a web URL:
    local tex = LoadTexture("https://www.github.com/Leadwerks/Documentation/raw/master/Assets/brickwall01.dds") Why is this a big deal? Well, it means you can post code snippets that can be copied and pasted without requiring download of any extra files. That means the documentation can include examples that use files that aren't required to be in the user's project directory:
    The documentation doesn't have to have any awkward zip files you are instructed to download like here, because any files that are needed in any examples can simply be linked directly to by the URL. So basically the default blank template can really be blank and doesn't need to include any "sample" files at all. If you have something like a model that has separate material and texture files, it should be possible to just link to the model file's URL and the rest of the associated files will be grabbed automatically.
  13. JMK
    The old-timers on this site will remember a time before Steam, when leadwerks.com had a native image gallery as part of our site. The gallery is gone but all the images were saved on Google Drive here. This was replaced with the Steam-based image gallery which has accrued a massive amount of content (about 1800 screenshots).
    With Steam flooding the market with new products there is pressure for developers to release more different products to occupy digital shelf space. For this reason, our new engine will be released with separate app IDs for different editions, rather than using the DLC system to differentiate versions. That means I will be making much less use of Steam-specific features in the future like screenshots, videos, and Workshop, since all that content would be split among several app IDs. Additionally, I plan on working with multiple software resellers and Steam is just one of them, so there is less reason to push all our content through that platform. 
    At the same time, advances in web technology have given us a vast amount of storage at a low cost. All of our user-generated files are hosted on Amazon S3 which costs almost nothing, so there is no longer the same pressure we used to have to keep our website data from getting too large.
    With this in mind, I am bringing back the native website gallery here. In time, all our old screenshots from both Google Drive and Steam will be migrated in. I will see that the Steam import system attempts to match Steam authors names with the forum name, but if no match is found the image will be attributed to "guest".
    Steam has been great for us, but looking back at our old site I do feel like something was lost when we offloaded a lot of content onto their system. I'm excited about bringing the native gallery back and the possibilities for the new engine. You can upload your images here:
    In the future, I am also interested in the idea of native video support, where we can upload a video or embed URL and have the website download the video, process it to a reliable format, generate thumbnails, and display it natively on our own site. In a world where everything on the web is simultaneously being monopolized and oversaturated, controlling your own data is critical. And it's not 2001, web space is cheap.
    I'm also really excited about the content that will be created with the new engine. It isn't going to be aimed at extreme beginners like LE4 was. It's easy to use, easier than LE4, in my opinion, but that's not the main selling point. Consequently I think we are going to see some very interesting work being done with the new engine and a lot of new energy on this site, more than we've ever seen before. So I am preparing for that.
    Get ready and dream big!
  14. JMK
    I implemented light bounces and can now run the GI routine as many times as I want. When I use 25 rays per voxel and run the GI routine three times, here is the result. (The dark area in the middle of the floor is actually correct. That area should be lit by the sky color, but I have not yet implemented that, so it appears darker.)

    It's sort of working but obviously these results aren't usable yet. Making matters more difficult is the fact that people love to show their best screenshots and love to hide the problems their code has, so it is hard to find something reliable to compare my results to.
    I also found that the GI pass, unlike all previous passes, is very slow. Each pass takes about 30 seconds in release mode! I could try to optimize the C++ code but something tells me that even optimized C++ code would not be fast enough. So it seems the GI passes will probably need to be performed in a shader. I am going to experiment a bit with some ideas I have first to provide better quality GI results first though.
  15. JMK
    The polygon voxelization process for our voxel GI system now takes vertex, material, and base texture colors into account. The voxel algorithm does not yet support a second color channel for emission, but I am building the whole system with that in mind. When I visualize the results of the voxel building the images are pretty remarkable! Of course the goal is to use this data for fast global illumination calculations but maybe they could be used to make a whole new style of game graphics.

    Direct lighting calculations on the CPU are fast enough that I am going to stick with this approach until I have to use the GPU. If several cascading voxel grids were created around the camera, and each updated asynchronously on its own thread, that might give us the speed we need to relieve the GPU from doing any extra work. The final volume textures could be compressed to DXT1 (12.5% their original size) and sent to the GPU.
    After direct lighting has been calculated, the next step is to downsample the voxel grid. I found the fastest way to do this is to iterate through just the solid voxels. This is how my previous algorithm worked:
    for (x=0; x < size / 2; ++x) { for (y=0; y < size / 2; ++y) { for (z=0; z < size / 2; ++z) { //Downsample this 2x2 block } } } A new faster approach works by "downsampling" the set of solid voxels by dividing each value by two. There are some duplicated values but that's fine:
    for (const iVec3& i : solidvoxels) { downsampledgrid->solidvoxels.insert(iVec3(i.x/2,i.y/2,i.z/2)) } for (const iVec3& i : downsampledgrid->solidvoxels) { //Downsample this 2x2 block } We can then iterate through just the solid voxels when performing the downsampling. A single call to memset will set all the voxel data to black / empty before the downsampling begins. This turns out to be much much faster than iterating through every voxel on all three axes.
    Here are the results of the downsampling process. What you don't see here is the alpha value of each voxel. The goblin in the center ends up bleeding out to fill very large voxels, because the rest of the volume around him is empty space, but the alpha value of those voxels will be adjusted to give them less influence in the GI calculation.

    For a 128x128x128 voxel grid, with voxel size of 0.125 meters, my numbers are now:
    Voxelization: 607 milliseconds Direct lighting (all six directions): 109 First downsample (to 64x64): 39 Second downsample (to 32x32): 7 Third downsample (to 16x16): 1 Total: 763 Note that voxelization, by far the slowest step here, does not have to be performed completely on all geometry each update. The direct lighting time elapsed is within a tolerable range, so we are in the running to make GI calculations entirely on the CPU, relieving the GPU of extra work and compressing our data before it is sent over the PCI bridge.
    Also note that a smaller voxel grids could be used, with more voxel grids spread across more CPU cores. If that were the case I would expect our processing time for each one to go down to 191 milliseconds total (39 milliseconds without the voxelization step), and the distance your GI covers would then be determined by your number of CPU cores.
    In fact there is a variety of ways this task could be divided between several CPU cores.
  16. JMK
    I have resumed work on voxel-based global illumination using voxel cone step tracing in Leadwerks Game Engine 5 beta with our Vulkan renderer. I previously put about three months of work into this with some promising results, but it is a very difficult system and I wanted to focus on Vulkan. Some of features we have gained since then like Pixmaps and DXT decompression make the voxel GI system easier to finish.
    I previously considered implementing Nvidia's raytracing techniques for Vulkan but the performance is terrible, even on the highest-end graphics cards. Voxel-based GI looks great and runs fast with basically no performance penalty.
    Below we have a section of the scene voxelized and lit with direct lighting. Loading the Sponza scene from GLTF format made it easy to display all materials and textures correctly.

    I found that the fastest way to manage voxel data was by storing the data in one big STL vector, and storing an STL set of occupied cells. (An STL set is like a map with only keys.) I found the fastest way to perform voxel raycasting was actually just to walk through the voxel data with optimized C++ code. This was much faster than my previous attempts to use octrees, and much simpler too! The above scene took about about 100 milliseconds to calculate direct lighting on a single CPU core, which is three times faster than my previous attempts. This definitely means that CPU-based GI lighting may be possible, which is my preferred approach. It's easier to implement, easy to parallelize, more flexible, more reliable, uses less video memory, transfers less data to the GPU, and doesn't draw any GPU power away from rendering the rest of the scene.
    The challenge will be in minimizing the delay between when an object moves, GI is recalculated, and when the data uploaded to the GPU and appears onscreen. I am guessing a delay somewhere around 200 milliseconds will be acceptable. It should also be considered that only an onscreen object will have a perceived delay if the reflection is slow to appear. An offscreen object will have no perceived delay because you can only see the reflection. Using screen-space reflections on pixels that can use it is one way to mitigate that problem, but if possible I would prefer to use one uniform system instead of mixing two rendering techniques.
    If this does not work then I will upload a DXT compressed texture containing the voxel data to the GPU. There are several stages at which the data can be handed off, so the question is which one works best?

    My design has changed a bit, but this is a pretty graphic.
    Using the pixmap class I will be able to load low-resolution versions of textures into system memory, decompress them to a readable format, and use that data to colorize the voxels according to the textures and UV coordinates of the vertices that are fed into the voxelization process.
  17. JMK
    I've had some more time to work with the Lua debugger in Leadwerks Game Engine 5 beta, and it's really amazing.  Adding the engine classes into the debug information has been pretty simple. All it takes is a class function that adds members into a table and returns it to Lua.
    sol::table Texture::debug(sol::this_state ts) const { auto t = Object::debug(ts); t["size"] = size; t["format"] = format; t["type"] = type; t["flags"] = flags; t["samples"] = samples; t["faces"] = faces; return t; } The base Object::debug function will add all the custom properties that you attach to the object:
    sol::table Object::debug(sol::this_state ts) const { sol::table t(ts, sol::create); for (auto& pair : entries) { if (pair.second.get_type() == sol::type::function) continue; if (Left(pair.first, 1) == "_") continue; t[pair.first] = pair.second; } return t; } This allows you to access both the built-in class members and your own values you attach to an object. You can view all these variables in the side panel while debugging, in alphabetical order:

    You can even hover over a variable to see its contents!

    The Lua debugger in Leadwerks 4 just sends a static stack of data to the IDE that is a few levels deep, but the new Lua debugger in VS Code will actually allow you to traverse the code and look all around your program. You can drill down as deep as you want, even viewing the positions of individual vertices in a model:

    This gives us a degree of power we've never had before with Lua game coding. Programming games with Lua will be easier than ever in our new game engine, and it's easy to add your own C++ classes to the environment.
  18. JMK
    Leadwerks Game Engine 5 Beta now supports debugging Lua in Visual Studio Code. To get started, install the Lua Debugger extension by DevCat.
    Open the project folder in VSCode and press F5. Choose the Lua debugger if you are prompted to select an environment.
    You can set breakpoints and step through Lua code, viewing variables and the callstack. All printed output from your game will be visible in the Debug Console within the VS Code interface.

    Having first-class support for Lua code in a professional IDE is a dream come true. This will make development with Lua in Leadwerks Game Engine 5 a great experience.
  19. JMK
    In Leadwerks Game Engine 4, bones are a type of entity. This is nice because all the regular entity commands work just the same on them, and there is not much to think about. However, for ultimate performance in Leadwerks 5 we treat bones differently. Each model can have a skeleton made up of bones. Skeletons can be unique for each model, or shared between models. Animation occurs on a skeleton, not a model. When a skeleton is animated, every model that uses that skeleton will display the same motion. Skeletons animations are performed on one or more separate threads, and the skeleton bones are able to skip a lot of overhead that would be required if they were entities, like updating the scene octree. This system allows for thousands of animated characters to be displayed with basically no impact on framerate:
    This is not some special case I created that isn't practical for real use. This is the default animation system, and you can always rely on it working this fast.
    There is one small issue that this design neglects, however, and that is the simple situation where we want to attach a weapon or other item to an animated character, that isn't built into that model. In Leadwerks 4 we would just parent the model to the bone, but we said bones are no longer entities so this won't work. So how do we do this? The answer is with bone attachments:
    auto gun = LoadModel(world, "Models/gun.gltf"); gun->Attach(player, "R_Hand"); Every time the model animation is updated, the gun orientation will be forced to match the bone we specified. If we want to adjust the orientation of the gun relative to the bone, then we can use a pivot:
    auto gun = LoadModel(world, "Models/gun.gltf"); auto gunholder = CreatePivot(world); gunholder->Attach(player, "R_Hand"); gun->SetParent(gunholder); gun->SetRotation(0,180,0); Attaching an entity to a bone does not take ownership of the entity, so you need to maintain a handle to it until you want it to be deleted. One way to do this is to just parent the entity to the model you are attaching to:
    auto gun = LoadModel(world, "Models/gun.gltf"); gun->Attach(player, "R_Hand"); gun->SetParent(player); gun = nullptr; //player model keeps the gun in scope In this example I created some spheres to indicate where the model bones are, and attached them to the model bones:
    Bone attachments involve three separate objects, the entity, the model, and the bone. This would be totally impossible to do without weak pointers in C++.
  20. JMK
    A new update is available for beta testers. This adds navmesh pathfinding, bone attachments, and the beginning of the Lua debugging capabilities.New commands for creating navigation meshes for AI pathfinding are included.
    NavMesh Pathfinding
    In Leadwerks Game Engine 5 you can create your own navmeshes and AI agents, with all your own parameters for player height, step height, walkable slope, etc.:
    shared_ptr<NavMesh> CreateNavMesh(shared_ptr<World> world, const float width, const float height, const float depth, const int tilesx, const int tilesz, const float agentradius = 0.4, const float agentheight = 1.8, const float agentstepheight = 0.501, const float maxslope = 45.01f); You can create AI agents yourself now:
    shared_ptr<NavAgent> CreateNavAgent(shared_ptr<NavMesh> navmesh, const float radius, const float height, const UpdateFlags updateflags) Here are some of the NavAgent methods you can use:
    void NavAgent::SetPosition(const float x, const float y, const float z); void NavAgent::SetRotation(const float angle); bool NavAgent::Navigate(const float x, const float y, const float z); New capabilities let you find a random point on the navmesh, or test to see if a point lies on a navmesh. As @reepblue pointed out, in addition to AI this feature could be used to test if a player is able to teleport to a position with VR locomotion:
    bool NavMesh::FindRandomPoint(Vec3& point) bool NavMesh::IntersectsPoint(const Vec3& point) You can call Entity::Attach(agent) to attach an entity to an agent so it follows it around.
    You can even create multiple navmeshes for different sized characters. In the video below, I created one navmesh for the little goblins, and another one with different parameters for the big guy. I created agents for each character on the appropriate sized navmesh, and then I created a big AI agent on both navmeshes. On the navmesh with big parameters, I use the regular navigation system, but the big agent on the little navmesh gets manually repositioned each frame. This results in an agent the little goblins walk around, and the end result is mixing of the two character sizes.
    The current implementation is static-only and will be built at the time of creation. You need to call Entity::SetNavigationMode(true) on any entities you want to contribute to the navmesh. Physics shape geometry will be used for navigation, not visible mesh geometry. I plan to add support for dynamic navmeshes that rebuild as the level changes next.
    Note that by default there is no interaction between physics and navigation. If you want AI agents to be collidable with the physics system, you need to create a physics object and position that as the agents move. This gives you complete control over the whole system.
    Bone Attachments
    To improve speed bones are a special type of object in Leadwerks Game Engine 5, and are an entity. Bone attachments allow you to "parent" an entity to a bone, so you can do things like place a sword in a character's hand. You can read more about bone attachments here:
    Lua Debugging in Visual Studio Code
    The beginnings of our Lua debugging system are taking shape. You can now launch your game directly from VSCode. To do this, open the project folder in VSCode. Install the "Lua Debugger" extension from DevCat. Press F5 and the game should start running. See the .vscode/launch.json file for more details.
  21. JMK
    An update is available for beta testers.
    What's new:
    GLTF animations now work! New example included. Any models from Sketchfab should work. Added Camera::SetGamma, GetGamma. Gamma is 1.0 by default, use 2.2 for dark scenes. Fixed bug that was creating extra bones. This is why the animation example was running slow in previous versions. Fixed bug where metalness was being read from wrong channel in metal-roughness map. Metal = R, roughness = G. Texture definitions in JSON materials are changed, but the old scheme is left in for compatibility. Textures are now an array:
    { "material": { "color": [ 1, 1, 1, 1 ], "emission": [ 0, 0, 0 ], "metallic": 0.75, "roughness": 0.5, "textures": [ { "slot": "BASE", "file": "./wall_01_D.tex" }, { "slot": "NORMAL", "file": "./wall_01_N.tex" } ] } } The slot value can be an integer from 0-31 or one of these strings:
    FPS example menu freezes. Close window to exit instead. Looping animations are not randomized so the animation example will show characters that appear identical even though they are separate skeletons animating independently. Unskinned GLTF animation is not yet supported (requires bone attachments feature)
  22. JMK
    A new beta update to Leadwerks Game Engine 5 is available now.
    New stuff:
    Streaming terrain CopyRect and Texture::SetSubPixels Texture saving Note that the "SharedObject" class has been renamed to "Object" and that math classes (Vec3, Vec4, Plane, Mat3, etc.) no longer derive from anything.
  23. JMK
    Last year Google and Binomial LLC partnered to release the Basic Universal library as open-source. This library is the successor to Crunch. Both these libraries are like OGG compression for textures. They compress data very well into small file sizes, but once loaded the data takes the same space in memory as it normally does. The benefit is that it can reduce the size of your game files. Crunch only supports DXT compression, but the newer Basis library supports modern compression formats like BC5, which gives artifact-free compressed normal maps, and BC7, which uses the same amount of space as DXT5 textures but pretty much eliminates blocky DXT artifacts on color images.
    I've added a Basis plugin to the Plugin SDK on Github so you can now load and save .basis texture files in the Leadwerks Game Engine 5 beta. Testing with a large 4096x2048 image in a variety of formats, here are my results.
    TGA (uncompressed): 24 MB PNG (lossless compression): 6.7 MB DDS (DXT1 compression): 5.33 MB Zipped DDS: 2.84 MB JPEG: 1.53 MB BASIS: 573 KB
    The zipped DXT option is the most like what you are using now in Leadwerks Game Engine 4. Since the overwhelming majority of your game size comes from texture files, we can extrapolate that Basis textures can reduce your game's data size to as little as 20% what it is now.
    With all the new image IO features, I wanted to write a script to convert texture files out of the Leadwerks 4 .tex file format and into something more transparent. If you drop the script below into the "Scripts/Start" folder it will automatically detect and convert .tex files into .dds:
    function BatchConvertTextures(path, in_ext, out_ext, recursive) local dir = LoadDir(path) for n = 1, #dir do local ftype = FileType(path.."/"..dir[n]) if ftype == 1 then if ExtractExt(dir[n]) == in_ext then local savefile = path.."/"..StripExt(dir[n]).."."..out_ext if FileTime(path.."/"..dir[n]) > FileTime(savefile) then local pixmap = LoadPixmap(path.."/"..dir[n]) if pixmap ~= nil then pixmap:Save(savefile) end end end elseif ftype == 2 and recursive == true then BatchConvertTextures(path.."/"..dir[n], in_ext, out_ext, true) end end end BatchConvertTextures(".", "tex", "dds", true) Here are my freightyard materials, out of the opaque .tex format and back into something I can easily view in Windows Explorer:

    These files are all using the original raw pixels as the .tex files, which internally are very similar to DDS. In order to convert them to .basis format, we need to get them into RGBA format, which means we need a DXT decompression routine. I found one and quickly integrated it, then revised my script to convert the loaded pixmaps to uncompressed RGBA format and save as PNG files.
    --Make sure FreeImage plugin is loaded if Plugin_FITextureLoader == nil then Plugin_FITextureLoader = LoadPlugin("Plugins/FITextureLoader.dll") end function BatchConvertTextures(path, in_ext, out_ext, recursive, format) local dir = LoadDir(path) for n = 1, #dir do local ftype = FileType(path.."/"..dir[n]) if ftype == 1 then if ExtractExt(dir[n]) == in_ext then local savefile = path.."/"..StripExt(dir[n]).."."..out_ext if FileTime(path.."/"..dir[n]) > FileTime(savefile) then --Load pixmap local pixmap = LoadPixmap(path.."/"..dir[n]) --Convert to desired format, if specified if format ~= nil then if pixmap ~= nil then if pixmap.format ~= format then pixmap = pixmap:Convert(format) end end end --Save if pixmap ~= nil then pixmap:Save(savefile) end end end elseif ftype == 2 and recursive == true then BatchConvertTextures(path.."/"..dir[n], in_ext, out_ext, true) end end end BatchConvertTextures("Materials/Freightyard", "tex", "png", true, TEXTURE_RGBA) Cool! Now I have all my .tex files back out as PNGs I can open and edit in any paint program. If you have .tex files that you have lost the source images for, there is now a way to convert them back.

    Now I am going to try converting this folder of .tex files into super-compressed .basis files. First I will add a line of code to make sure the Basis plugin has been loaded:
    --Make sure Basis plugin is loaded if Plugin_Basis == nil then Plugin_Basis = LoadPlugin("Plugins/Basis.dll") end And then I can simply change the save file extension to "basis" and it should work:
    BatchConvertTextures("Materials/Freightyard", "tex", "basis", true, TEXTURE_RGBA) And here it is! Notice the file size of the selected image.

    If you want to view basis textures in Windows explorer there is a handy-dandy thumbnail previewer available.
    Now let's see what the final file sizes are for this texture set.
    Uncompressed TEX files (DXT only): 35.2 MB Zipped TEX files: 25.8 MB BASIS: 7.87 MB 😲 When converted to Basis files the same textures take just 30% the size of the zipped package, and 22% the size of the extracted TEX files. That makes Basis Universal definitely worth using!
  24. JMK
    The terrain system in Leadwerks Game Engine 4 allows terrains up to 64 square kilometers in size. This is big enough for any game where you walk and most driving games, but is not sufficient for flight simulators or space simulations. For truly massive terrain, we need to be able to dynamically stream data in and out of memory, at multiple resolutions, so we can support terrains bigger than what would otherwise fit in memory all at once.
    The next update of Leadwerks Game Engine 5 beta supports streaming terrain, using the following command:
    shared_ptr<StreamingTerrain> CreateStreamingTerrain(shared_ptr<World> world, const int resolution, const int patchsize, const std::wstring& datapath, const int atlassize = 1024, void FetchPatchInfo(TerrainPatchInfo*) = NULL) Let's looks at the parameters:
    resolution: Number of terrain points along one edge of the terrain, should be power-of-two. patchsize: The number of tiles along one edge of a terrain piece, should be a power-of-two number, probably 64 or 128. datapath: By default this indicates a file path and name but can be customized. atlassize: Width and height of texture atlas texture data is copied into. 1024 is usually fine. FetchPatchInfo: Optional user-defined callback to override the default data handler. A new Lua sample is included that creates a streaming terrain:
    local terrain = CreateStreamingTerrain(world, 32768, 64, "Terrain/32768/32768") terrain:SetScale(1,1000,1) The default fetch patch function can be used to make your own data handler. Here is the default function, which is probably more complex than what you need for streaming GIS data. The key parts to note are:
    The TerrainPatchInfo structure contains the patch X and Y position and the level of detail. The member patch->heightmap should be set to a pixmap with format TEXTURE_R16. The member patch->normalmap should be set to a pixmap with format TEXTURE_RGBA (for now). You can generate this from the heightmap using MakeNormalMap(). The scale value input into the MakeNormalMap() should be the terrain vertical scale you intend to use, times two, divided by the mipmap level. This ensures normals are calculated correctly at each LOD. For height and normal data, which is all that is currently supported, you should use the dimensions patchsize + 1, because the a 64x64 patch, for example, uses 65x65 vertices. Don't forget to call patch->UpdateBounds() to calculate the AABB for this patch. The function must be thread-safe, as it will be called from many different threads, simultaneously. void StreamingTerrain::FetchPatchInfo(TerrainPatchInfo* patch) { //User-defined callback if (FP_FETCH_PATCH_INFO != nullptr) { FP_FETCH_PATCH_INFO(patch); return; } auto stream = this->stream[TEXTURE_DISPLACEMENT]; if (stream == nullptr) return; int countmips = 1; int mw = this->resolution.x; while (mw > this->patchsize) { countmips++; mw /= 2; } int miplevel = countmips - 1 - patch->level; Assert(miplevel >= 0); uint64_t mipmapsize = Round(this->resolution.x * 1.0f / pow(2.0f, miplevel)); auto pos = mipmappos[TEXTURE_DISPLACEMENT][miplevel]; uint64_t rowpos; patch->heightmap = CreatePixmap(patchsize + 1, patchsize + 1, TEXTURE_R16); uint64_t px = patch->x * patchsize; uint64_t py = patch->y * patchsize; int rowwidth = patchsize + 1; for (int ty = 0; ty < patchsize + 1; ++ty) { if (py + ty >= mipmapsize) { patch->heightmap->CopyRect(0, patch->heightmap->size.y - 2, patch->heightmap->size.x, 1, patch->heightmap, 0, patch->heightmap->size.y - 1); continue; } if (px + rowwidth > mipmapsize) rowwidth = mipmapsize - px; rowpos = pos + ((py + ty) * mipmapsize + px) * 2; streammutex[TEXTURE_DISPLACEMENT]->Lock(); stream->Seek(rowpos); stream->Read(patch->heightmap->pixels->data() + (ty * (patchsize + 1) * 2), rowwidth * 2); streammutex[TEXTURE_DISPLACEMENT]->Unlock(); if (rowwidth < patchsize + 1) { patch->heightmap->WritePixel(patch->heightmap->size.x - 1, ty, patch->heightmap->ReadPixel(patch->heightmap->size.x - 2, ty)); } } patch->UpdateBounds(); stream = this->stream[TEXTURE_NORMAL]; if (stream == nullptr) { patch->normalmap = patch->heightmap->MakeNormalMap(scale.y * 2.0f / float(1 + miplevel), TEXTURE_RGBA); } else { pos = mipmappos[TEXTURE_NORMAL][miplevel]; Assert(pos < stream->GetSize()); patch->normalmap = CreatePixmap(patchsize + 1, patchsize + 1, TEXTURE_RGBA); rowwidth = patchsize + 1; for (int ty = 0; ty < patchsize + 1; ++ty) { if (py + ty >= mipmapsize) { patch->normalmap->CopyRect(0, patch->normalmap->size.y - 2, patch->normalmap->size.x, 1, patch->normalmap, 0, patch->normalmap->size.y - 1); continue; } if (px + rowwidth > mipmapsize) rowwidth = mipmapsize - px; rowpos = pos + ((py + ty) * mipmapsize + px) * 4; Assert(rowpos < stream->GetSize()); streammutex[TEXTURE_NORMAL]->Lock(); stream->Seek(rowpos); stream->Read(patch->normalmap->pixels->data() + uint64_t(ty * (patchsize + 1) * 4), rowwidth * 4); streammutex[TEXTURE_NORMAL]->Unlock(); if (rowwidth < patchsize + 1) { patch->normalmap->WritePixel(patch->normalmap->size.x - 1, ty, patch->normalmap->ReadPixel(patch->normalmap->size.x - 2, ty)); } } } } There are some really nice behaviors that came about naturally as a consequence of the design.
    Because the culling algorithm works its way down the quad tree with only known patches of data, the lower-resolution sections of terrain will display first and then be replaced with higher-resolution patches as they are loaded in. If the cache gets filled up, low-resolution patches will be displayed until the cache clears up and more detailed patches are loaded in. If all the patches in one draw call have not yet been loaded, the previous draw call's contents will be rendered instead. As a result, the streaming terrain is turning out to be far more robust than I was initially expecting. I can fly close to the ground at 650 MPH (the speed of some fighter jets) with no problems at all.
    There are also some issues to note:
    Terrain still has cracks in it. Seams in the normal maps will appear along the edges, for now. Streaming terrain does not display material layers at all, just height and normal. But there is enough to start working with it and piping data into the system.
  25. JMK
    The new engine features advanced image and texture manipulation commands that allow a much deeper level of control than the mostly automated pipeline in Leadwerks Game Engine 4. This article is a deep dive into the new image and texture system, showing how to load, modify, and save textures in a variety of file formats and compression modes.
    Texture creation has been finalized. Here is the command:
    shared_ptr<Texture> CreateTexture(const TextureType type, const int width, const int height, const TextureFormat format = TEXTURE_RGBA, const std::vector<shared_ptr<Pixmap> > mipchain = {}, const int layers = 1, const TextureFlags = TEXTURE_DEFAULT, const int samples = 0); It seems like once you get to about 6-7 function parameters, it starts to make more sense to fill in a structure and pass that to a function the way Vulkan and DirectX do. Still, I don't want to go down that route unless I have to, and there is little reason to introduce that inconsistency into the API just for a handful of long function syntaxes.
    The type parameter can be TEXTURE_2D, TEXTURE_3D, or TEXTURE_CUBE. The mipchain parameter contains an array of images for all miplevels of the texture.
    We also have a new SaveTexture command which takes texture data and saves it into a file. The engine has built-in support for saving DDS files, and plugins can also provide support for additional file formats. (In Lua we provide a table of pixmaps rather than an STL vector.)
    bool SaveTexture(const std::wstring& filename, const TextureType type, const std::vector<shared_ptr<Pixmap> > mipchain, const int layers = 1, const SaveFlags = SAVE_DEFAULT); This allows us to load a pixmap from any supported file format and resave it as a texture, or as another image file, very easily:
    --Load image local pixmap = LoadPixmap("Materials/77684-blocks18c_1.jpg") --Convert to RGBA if not already if pixmap.format ~= TEXTURE_RGBA then pixmap = pixmap:Convert(TEXTURE_RGBA) end --Save pixmap to texture file SaveTexture("OUTPUT.dds", TEXTURE_2D, {pixmap}, 1, SAVE_DEFAULT) If we open the saved DDS file in Visual Studio 2019 we see it looks exactly like the original, and is still in uncompressed RGBA format. Notice there are no mipmaps shown for this texture because we only saved the top-level image.

    Adding Mipmaps
    To add mipmaps to the texture file, we can specify the SAVE_BUILD_MIPMAPS in the flags parameter of the SaveTexture function.
    SaveTexture("OUTPUT.dds", TEXTURE_2D, {pixmap}, 1, SAVE_BUILD_MIPMAPS) When we do this we can see the different mipmap levels displayed in Visual Studio, and we can verify that they look correct.

    Compressed Textures
    If we want to save a compressed texture, there is a problem. The SAVE_BUILD_MIPMAPS flags won't work with compressed texture formats, because we cannot perform a bilinear sample on compressed image data. (We would have to decompressed, interpolate, and then recompress each block, which could lead to slow processing times and visible artifacts.) To save a compressed DDS texture with mipmaps we will need to build the mipmap chain ourselves and compress each pixmap before saving.
    This script will load a JPEG image as a pixmap, generate mipmaps by resizing the image, convert each mipmap to BC1 compression format, and save the entire mip chain as a single DDS texture file:
    --Load image local pixmap = LoadPixmap("Materials/77684-blocks18c_1.jpg") local mipchain = {} table.insert(mipchain,pixmap) --Generate mipmaps local w = pixmap.size.x local h = pixmap.size.y local mipmap = pixmap while (w > 1 and h > 1) do w = math.max(1, w / 2) h = math.max(1, h / 2) mipmap = mipmap:Resize(w,h) table.insert(mipchain,mipmap) end --Convert each image to BC1 (DXT1) compression for n=1, #mipchain do mipchain[n] = mipchain[n]:Convert(TEXTURE_BC1) end --Save mipchain to texture file SaveTexture("OUTPUT.dds", TEXTURE_2D, mipchain, 1, SAVE_DEFAULT) If we open this file in Visual Studio 2019 we can inspect the individual mipmap levels and verify they are being saved into the file. Also note that the correct texture format is displayed.

    This system gives us fine control over every aspect of texture files. For example, if you wanted to write a mipmap filter that blurred the image a bit with each resize, with this system you could easily do that.
    Building Cubemaps
    We can also save cubemaps into a single DDS or Basis file by providing additional images. In this example we will load a skybox strip that consists of six images side-by-side, copy sections of the sky to different images, and then save them all as a single DDS file.
    Here is the image we will be loading. It's already laid out in the order +X, -X, +Y, -Y, +Z, -Z, which is what the DDS format uses internally.

    First we will load the image as a pixmap and check to make sure it is six times as wide as it is high:
    --Load skybox strip local pixmap = LoadPixmap("Materials/zonesunset.png") if pixmap.size.x ~= pixmap.size.y * 6 then Print("Error: Wrong image aspect.") return end Next we will create a series of six pixmaps and copy a section of the image to each one, using the CopyRect method.
    --Copy each face to a different pixmap local faces = {} for n = 1, 6 do faces[n] = CreatePixmap(pixmap.size.y,pixmap.size.y,pixmap.format) pixmap:CopyRect((n-1) * pixmap.size.y, 0, pixmap.size.y, pixmap.size.y, faces[n], 0, 0) end To save a cubemap you must set the type to TEXTURE_CUBE and the layers value to 6:
    --Save as cube map SaveTexture("CUBEMAP.dds", TEXTURE_CUBE, faces, 6, SAVE_DEFAULT) And just like that, you've got your very own cubemap packed into a single DDS file. You can switch through all the faces of the cubemap by changing the frames value on the right:

    For uncompressed cubemaps, we can just specify the SAVE_BUILD_MIPMAPS and mipmaps will automatically be created and saved in the file:
    --Save as cube map, with mipmaps SaveTexture("CUBEMAP+mipmaps.dds", TEXTURE_CUBE, faces, 6, SAVE_BUILD_MIPMAPS) Opening this DDS file in Visual Studio 2019, we can view all cubemap faces and verify that the mipmap levels are being generated and saved correctly:

    Now in Leadwerks Game Engine 4 we store skyboxes in large uncompressed images because DXT compression does not handle gradients very well and causes bad artifacts. The new BC7 compression mode, however, is good enough to handle skyboxes and takes the same space as DXT5 in memory. We already learned how to save compressed textures. The only difference with a skybox is that you store mipmaps for each cube face, in the following order:
    for f = 1, faces do for m = 1, miplevels do The easy way though is to just save as the RGBA image as a Basis file, with mipmaps. The Basis compressor will handle everything for us and give us a smaller file:
    --Save as cube map, with mipmaps SaveTexture("CUBEMAP+mipmaps.basis", TEXTURE_CUBE, faces, 6, SAVE_BUILD_MIPMAPS) The outputted .basis file works correctly when loaded in the engine. Here are the sizes of this image in different formats, with mipmaps included:
    Uncompressed RGBA: 32 MB BC7 compressed DDS: 8 MB Zipped uncompressed RGBA: 4.58 MB Zipped BC7 compressed DDS: 3.16 MB Basis file: 357 KB Note that the Basis file still takes the same amount of memory as the BC7 file, once it is loaded onto the GPU. Also note that a skybox in Leadwerks Game Engine 5 consumes less than 1% the hard drive space of a skybox in Leadwerks 4.
    Another option is to save the cubemap faces as individual images and then assemble them in a tool like ATI's CubemapGen. This was actually the first approach I tried while writing this article. I loaded the original .tex file and saved out a bunch of PNG images, as shown below.

    Modifying Images
    The CopyRect method allows us to copy sections of images from one to another as long as the images use the same format. We can also copy from one section of an image to another area on itself. This code will load a pixmap, copy a rectangle from one section of the image to another, and resave it as a simple uncompressed DDS file with no mipmaps:
    --Load image local pixmap = LoadPixmap("Materials/77684-blocks18c_1.jpg") --Convert to RGBA if not already if pixmap.format ~= TEXTURE_RGBA then pixmap = pixmap:Convert(TEXTURE_RGBA) end --Let's make some changes :D pixmap:CopyRect(0,0,256,256,pixmap,256,256) --Save uncompressed DDS file SaveTexture("MODIFIED.dds", TEXTURE_2D, {pixmap}, 1, SAVE_DEFAULT) When you open the DDS file you can see the copy operation worked:

    We can even modify compressed images that use any of the FourCC-type compression modes. This includes BC1-BC5.. The only difference that is instead of copying individual pixels, we are now working with 4x4 blocks of pixels. The easiest way to handle this is to just divide all your numbers by four:
    --Convert to compressed format pixmap = pixmap:Convert(TEXTURE_BC1) --We specify blocks, not pixels. Blocks are 4x4 squares. pixmap:CopyRect(0,0,64,64,pixmap,64,64) --Save compressed DDS file SaveTexture("COMPRESSED+MODIFIED.dds", TEXTURE_2D, {pixmap}, 1, SAVE_DEFAULT) When we open the image in Visual Studio the results appear identical, but note that the format is compressed BC1. This means we can perform modifications on compressed images without ever decompressing them:

    The Pixmap class has a GetSize() method which returns the image size in pixels, and it also has a GetBlocks() function. With uncompressed images, these two methods return the same iVec2 value. With compressed formats, the GetBlocks() value will be the image width and height in pixels, divided by four. This will help you make sure you are not drawing outside the bounds of the image. Note that this technique will work just fine with BC1-BC5 format, but will not work with BC6h and BC7, because these use more complex data formats.
    Texture SetSubpixels
    We have a new texture method that functions like CopyRect but works on textures that are already loaded on the GPU. As we saw with pixmaps, it is perfectly safe to modify even compressed data as long as we remember that we are working with blocks, not pixels. This is the command syntax:
    void SetSubPixels(shared_ptr<Pixmap> pixmap, int x, int y, int width, int height, int dstx, int dsty, const int miplevel = 0, const int layer = 0); To do this we will load a texture, and then load a pixmap. Remember, pixmaps are image data stored in system memory and textures are stored on the GPU. If the loaded pixmap's format does not match the texture pixel format, then we will convert the pixmap to match the texture:
    --Load a texture local texture = LoadTexture("Materials/77684-blocks18c_1.jpg") --Load the pixmap local stamp = LoadPixmap("Materials/stamp.png") if stamp.format ~= texture.format then stamp = stamp:Convert(texture.format) end To choose a position to apply the pixmap to, I used the camera pick function and used to picked texture coordinate to calculate an integer offset for the texture:
    local mpos = window:GetMousePosition() if window:MouseDown(MOUSE_LEFT) == true and mpos:DistanceToPoint(mouseposition) > 50 then mouseposition = mpos local mousepos = window:GetMousePosition() local pick = camera:Pick(framebuffer, mousepos.x, mousepos.y, 0, true, 0) if pick ~= nil then local texcoords = pick:GetTexCoords() texture:SetSubPixels(stamp, 0, 0, stamp.size.x, stamp.size.y, texcoords.x * texture.size.x - stamp.size.x / 2, texcoords.y * texture.size.y - stamp.size.x / 2, 0, 0) end end And here it is in action, in a Lua script example to be included in the next beta release: Note that this command does not perform any type of blending, it only sets raw texture data.

    What you see above is not a bunch of decals. It's just one single texture that has had it's pixels modified in a destructive way. The stamps we applied cannot be removed except by drawing over them with something else, or reloading the texture.
    The above textures are uncompressed images, but if we want to make this work with FourCC-type compressed images (everything except BC6h and BC7) we need to take into account the block size. This is easy because with uncompressed images the block size is 1 and with compressed images it is 4:
    texture:SetSubPixels(stamp, 0, 0, stamp.size.x / stamp.blocksize, stamp.size.y / stamp.blocksize, texcoords.x * (texture.size.x - stamp.size.x / 2) / stamp.blocksize, (texcoords.y * texture.size.y - stamp.size.x / 2) / stamp.blocksize, 0, 0) Now our routine will work with uncompressed or compressed images. I have it working in the image below, but I don't know if it will look much different from the uncompressed version.
    I'm not 100% sure on the block / pixel part yet. Maybe I will just make a rule that says "units must be divisible by four for compressed images".
    Anyways, there you have it. This goes much deeper than the texture commands in Leadwerks Game Engine 4 and will allow a variety of user-created tools and extensions to be written for the new level editor. It all starts with the code API, and this is a fantastic set of features, if I do say so myself. Which I do.
  • Create New...