Jump to content

Design Confusion

Josh

1,764 views

Sometimes I run into situations where I don't really know how to structure things. I don't mind this, because it usually results in some really elegant design once I figure out what to do. I just play with ideas and try not to force anything, and when the right idea arises, I will recognize it.

 

Explaining a problem to someone else can help facilitate that process. How many times have you solved a difficult problem right after you posted a description of it on a forum somewhere? The procedure of explaining it logically to someone else can help you think more clearly about it. And so, we have today's blog topic.

 

Below is a very rough start to the script editor. The syntax highlighting system was written about a year ago, and works beautifully, using the native text widget on both Windows and Mac.

blogentry-1-0-87950000-1329443513_thumb.jpg

 

In the Leadwerks3D source code, there is a base class called an "AssetEditor". From this class the material, model, shader, texture, font, and script editor classes are derived. Like the other asset editor windows, only one instance of the script editor window will be allowed open at any time. Unlike the other asset editor windows, which display only one asset at a time, the script editor will use tabs to display multiple files. Scripts aren't a typical asset like a material or a model, so it's fine for them to behave a little differently.

 

Any Leadwerks3D application can have its Lua state debugged. The engine uses networking commands to communicate with a debugger on a specified port. This means the engine can communicate with a debugger whether it's part of a C++ program, C# app, or standalone Lua interpreter. The debugger can display the Lua callstack and shows all variables and values in the Lua state, including full examination of C++ objects and members!

 

I do not intend for Leadwerks3D to "play a game" in the editor. We've tried that approach and there are a lot of problems. I want Leadwerks3D to be a very solid and stable level editor, with a visual interface to do everything you need. I also want better consistency between Lua and C++ programs. Therefore, Leadwerks3D will use a run game system more similar to 3D World Studio than the Leadwerks Engine editor. A dialog will allow you to choose the application to run, command-line parameters, and other settings. These will be saved between sessions, so you can hit a key to do a "Quick Launch" and run your game. It would be possible to hook the Lua debugger into any application as it is launched, which could be very helpful.

 

Let's go back to the script editor now. My inclination is to have F5 launch an interpreter and call the script for the currently selected tab. However, I don't think it's a good idea to use multiple game launch modes, I already described a uniform game launch mode for both Lua and C++ applications, but that seems sort of counter-intuitive if you are working in the script editor and just want to run something really quickly.

 

There;s also the question of whether we want to provide a standalone script editor and debugger outside of Leadwerks3D. Or should the debugger be a standalone application as well, since someone might want to use it with a C++ application? You see there are a lot of options and a lot of possible ways to set this up.

 

What about Lua compile errors? I can print that out in the engine log, but how will the editor display it? If a compile error occurs, should the program pause and display the line it occurred at? What if the user just doesn't care, and wants the program to keep going?

 

Alternatively, the user may want to just hit F5 in the script editor and check for basic syntax errors, which the command LuaL_LoadString() will detect. wacko.png

 

That's pretty much all my questions at this point. I don't expect anyone to come along and solve my problems, but the process of describing and discussing the issues will help me come to a resolution.



18 Comments


Recommended Comments

Make it so that F5 launches a .cmd file, which it creates from a list item field like in 3DWS. It should have also parameters, which Editor replaces with actual values. A F5 script could look like this:

{workvolume}:
cd {workdir}
cd source
mingw32-make
if errorlevel 1 goto errors
cd ..distro
game.exe {parameters} {scenefilename}
goto over
:errors
notepad++ errors.log
:over

 

And what F5 then compiled this into would be a launch.cmd file which looks like this:

d:
cd projectsgame1
cd source
mingw32-make
if errorlevel 1 goto errors
cd ..distro
game.exe 640 480 data/maps/level1.sbx
goto over
:errors
notepad++ errors.log
:over

 

There could be a set of favorite scripts which F5 can run, and also a default script which would be like:

luac {projectname} > errors.log
if errorlevel 1 goto errors
engine.exe {projectname} {parameters} {mapfilename}
goto over
:errors
notepad++ errors.log
:over

Share this comment


Link to comment

Getting the printed output of the program is not a problem. I can just create a process and read the output directly. I'm thinking more about the workflow and ease of use.

Share this comment


Link to comment

I have to agree. The number of times I've come up with a solution while explaining the problem to someone... and most of the time the someone I explain it too doesn't not understand programming but it still helps find a solution because you are pouring out your thoughts into words and for some reason that makes things make more sense.

Share this comment


Link to comment

Regarding the debug thing. Maybe a simple UDP socket on localhost would solve the problem of sending info from the program to the editor. Just a suggestion.

Share this comment


Link to comment

There's no technical problems. The problem is one of workflow. What is the best way to design this to work? (The UDP socket is actually exactly what I am doing. It allows the debugger to communicate both ways for code stepping.)

Share this comment


Link to comment

Isn't debugging done in Code::Blocks IDE? I don't quite understand why you need debugging in a CSG Modeller. 3DWS didn't have that either, and I never missed it.

Share this comment


Link to comment

3DWS didn't have that either, and I never missed it.

Maybe because you didn't have it :) And this is not the same thing. In 3DWS you had no scripts to debug, right-

Share this comment


Link to comment

But the programming should be consistent between all languages, including Lua. Most people probably prefer Code::Blocks for Lua projects also, because it has very sophisticated plug-ins for code clips management, and is thus more suitable for larger game projects than a simple text editor.

Share this comment


Link to comment

For the lua compile errors I would think following Visual Studio's model of having a window at the bottom of the editor that will show each error would work well. We can dbl click each error line to go to the error in the file.

 

I currently use Notepad++ for Lua programming in LE. Mainly because it has tabbed browsing of files which I think would be very handy in your script editor. Sometimes we have to bounce back and forth between scripts and without tabs it's very inefficient today.

 

Are you giving us access to an "int main" type of script like you do today or is that hidden from us and we work with these scripts that we attach to entities only?

 

I would have a separate button/key for syntax checking only. This is common in SQL editors where you have one option to check syntax, then another option to actually run it. It's very handy and with all these little scripts LE 3D will have I can see that being handy also.

 

So F6 to "compile" the script to check syntax errors and if errors show up a window at the bottom of the editor to allow us to go to the error line.

 

F5 to actually run the game. I wouldn't think you'd ever just run one script itself would you? I guess at first glance that wouldn't make much sense to me as it's part of the entire game and can have many connections/requirements to/from other scripts.

 

If a run-time error happens, stop the game on that line, and open the file that contains the error in another tab and highlight it yellow and show the error in a messagebox.

Share this comment


Link to comment

Since i am not a native english, i hope that i misunderstood the part of not being able to launch a script as a whole game, currently all my game is based on lua scripts which call each other and a launch (F5) button is all what i need.

I don't know 3DStudio but i hope we can do the same as now.

Sorry for my fear but i hope someone will chill me out

Share this comment


Link to comment

Don't worry, whatever Josh comes up with, you will be still able to do the following always:

1) Press Ctrl-S in Editor

2) Press F5 in ScriptEditor to compile (if changed) and run the game

because you have to be able to do that anyway for all other languages also:

1) Press Ctrl-S in Editor

2) Press F9 in Code::Blocks to compile (if changed) and run the game

 

Josh is only trying to integrate the Lua Editor into Leadwerks Editor, which is kinda useless in my opinion anyway. It could be just left out. Or he could find a way how to make the above 2 steps even easier, although it's OK how they are now too.

Share this comment


Link to comment

I don't see any point in setting up Leadwerks3D to launch command-line compilers for C++ or other languages (though you could). It gets very complicated for so many languages and platforms, and the feedback would be inferior to just using the real IDEs for that language. I just see it as launching either the script interpreter, or your own executable that you define.

 

I've been using C++ so long, that I am used to a big wall of text whenever I hit F5, but I think the feedback when Lua loads a file is just a single error line if anything goes wrong. Therefore, there is no need to print out a bunch of compile information. If you press F5 in the script editor, the Lua file can be evaluated, and if any syntax errors exist, a message box can pop up and the line in question can be highlighted. This is simply evaluating the syntax of the script, not running anything.

 

Now when we want to launch a scripted game, I can either launch it from the script editor, using the script in the selected tab, or I can do it through a "run game" dialog where you define whether it's a scripted game or your own executable. The advantage of the "run game" dialog is you can launch script and C++ games and debug their Lua states in one interface. However, I feel like it's more natural and easier to launch a script by hitting a hot key in the script editor, which is my main point of conflict in this design.

 

Should the debugger and game printed output dialogs be built into the script editor window? That seems logical, but what about C++ applications? Am I going to make a separate script debugger and output window for those? It's a bad idea to have two versions of the same interface.

Share this comment


Link to comment
but I think the feedback when Lua loads a file is just a single error line if anything goes wrong.

 

This could be inefficient if it's the case as you could have 20 syntax mistakes and it'll only show you the first one? Then you have to keep syntax checking over and over again? Sounds like a pain.

 

The advantage of the "run game" dialog is you can launch script and C++ games and debug their Lua states in one interface

 

People using C++ will most likely be launching their game from the C++ IDE so they can debug their C++ code. I know most C++ coders here were upset that Lua was "required" to get some editor information from models in a map. The C++ people seem to not really want anything to do with Lua. They (I) won't create an exe of their C++ program just to then launch it from the editor because you lose the ability to debug your C++ code which will have the bulk of logic in it. Any attempt to merge the editor with their IDE will most likely fail and not be a smart use of your time (imho).

 

I personally think your Lua design is mixed mode which makes things more difficult. You seem to still want to give an "int main" type Lua file and not fully commit to the Lua model script system for total interaction of the entire game logic for games using Lua. If you did, then your engine.exe program would be the only thing you need to launch when we press Play from the editor. You setup properties in the editor that engine.exe reads and uses for certain things like debug info. engine.exe should be a very generic game loop that reads settings from a file to determine specific things it should be doing. All your game logic should then be defined in the model Lua scripts. There is no need to then give us an "int main" Lua file. There would be no point in it.

 

Just so you are aware, this isn't me being radical. A good number of engines don't provide an "int main" to the developers and amazing games have been created with them. For LE Lua I think it'll help to take "int main" away as it starts to shape how people code with it. If there are too many possibilities it segregates the community.

 

On the C++ side:

Like I was saying I don't think the C++ people will be pressing Play from the editor. They will be running their game from their C++ IDE loading the map they are working on from the editor.

Share this comment


Link to comment

I agree with Rick, when I write a game in C++, I don't want it be infected with Lua at all, and wasting valuable memory for its GC and interpreter. I need all memory for my game to get the maximum content at maximum speed (everything loaded into memory makes things much faster).

 

But like I already said to diedir, if there would be a way to make the 2 keystrokes less (well actually 3 keystrokes, since there is Alt-Tab between them too).

 

Maybe F5 in Editor could be programmed to switch to the Code::Blocks IDE window, and then send a F9 keystroke. That would actually work! Then you can use your favorite language and your favorite IDE, and only the window class and F9 key would need to be configured in Editor. And you would have a debugger too for all languages.

Share this comment


Link to comment
This could be inefficient if it's the case as you could have 20 syntax mistakes and it'll only show you the first one? Then you have to keep syntax checking over and over again? Sounds like a pain.

That's just the way Lua works. It stops on the first error. This actually annoys me about C++ because one syntax errors causes a whole cascade of errors, and you end up with 40 error messages because you forgot one semicolon.

 

Maybe F5 in Editor could be programmed to switch to the Code::Blocks IDE window, and then send a F9 keystroke. That would actually work! Then you can use your favorite language and your favorite IDE, and only the window class and F9 key would need to be configured in Editor. And you would have a debugger too for all languages.

One of my design rules is not to rely on or try to integrate third-party tools I don't have complete control of. This is slightly different, but recall all the problems that arose when I relied on the third-party Newton serialized format.

 

I personally think your Lua design is mixed mode which makes things more difficult. You seem to still want to give an "int main" type Lua file and not fully commit to the Lua model script system for total interaction of the entire game logic for games using Lua.

That's correct. I think a global script is always needed for a Lua game and attempts to abstract that away end up being more confusing.

 

The ability to debug the Lua state of a C++ program is a new idea you have not been exposed to before, but I think it will end up being very useful, especially for professional studios. Think about when your game code is done and artists are producing maps for the game. They'll want to be able to easily launch the game with the current map, and they will want to be able to inspect the Lua state to see if something goes wrong.

Share this comment


Link to comment

I still think the window class + keystroke macro should be possible with F5, or then F9 in Editor, if you do some wierd stuff with Lua states in F5.

Or then you could add the possibility to write Lua or dll plug-ins for Editor, then the community can make that.

Share this comment


Link to comment

It'll be interesting to see how this works and if it will be different enough than how Lua works with LE today to provide real code sharing, but I'm not very optimistic about it because it still allows different design approaches which will reduce the amount of sharing. Not having access to "int main" doesn't limit a game in any way.

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Blog Entries

    • By Josh in Josh's Dev Blog 0
      I did a little experiment with FPS Creator Pack #75 by upsampling the images with Gigapixel, which uses deep learning to upsample images and infer details that don't appear in the original pixels. The AI neural network does a pretty impressive job, generating results that are look better than a simple sharpen filter: I doubled the size of the textures to 1024x1024. Then I generated normal maps from the high-res images using AMD's TGA to DOT3 tool, and saved the normal maps with BC5 DDS compression. The diffuse textures were saved with BC7 DDS compression. The images below are using a 4x magnification to demonstrate the difference.


      As you can see, the image that is upsampled with deep learning looks normal and the resized image looks like there is butter on the lens! It's hard to believe the above images came from a 256x128 section of an image.
      The workflow was pretty tedious, as I had to convert images to TGA, then to uncompressed or BC5 DDS, and then to BC7 in Visual Studio. Each BC7 texture took maybe 5-10 minutes to compress! So while this set represents the optimum quality for 2019 game tech, and the format for assets we want to use in LE5, the workflow has a lot of room for improvement.
      You can download the full package here:
      FPSCPack75TexturesHD.zip
    • By Josh in Josh's Dev Blog 2
      DPI scaling and the 2D drawing and GUI system were an issue I was a bit concerned about, but I think I have it worked out. This all goes back to the multi-monitor support that I designed back in September. Part of that system allows you to retrieve the DPI scale for each display. This gives you another piece of information in addition to the raw screen resolution. The display scale gives you a percentage value the user expects to see vector graphics at, with 100% being what you would expect with a regular HD monitor. If we scale our GUI elements and font sizes by the display scale we can adjust for screens with any pixel density.
      This shot shows 1920x1080 fullscreen with DPI scaling set to 100%:

      Here we see the same resolution, with scaling set to 125%:

      And this is with scaling set to 150%:

      The effect of this is that if the player is using a 4K, 8K, or any other type of monitor, your game can display finely detailed text at the correct size the user expects to see. It also means that user interfaces can be rendered at any resolution for VR.
      Rather than trying to automatically scale GUI elements I am giving you full control over the raw pixels. That means you have to decide how your widgets will be scaled yourself, and program it into the game interface, but there is nothing hidden from the developer. Here is my code I am working with now to create a simple game menu. Also notice there is no CreatePanel(), CreateButton(), etc. anymore, there is just one widget you create and set the script for. I might add an option for C++ actors as well, but since these are operating on the main logic thread there's not really a downside to running the code in Lua.
      local window = ActiveWindow() if window == nullptr then return end local framebuffer = window:GetFramebuffer() if framebuffer == nil then return end self.gui = CreateGUI(self.guispritelayer) --Main background panel self.mainpanel = CreateWidget(self.gui,"",0,0,framebuffer.size.x,framebuffer.size.y) self.mainpanel:SetScript("Scripts/GUI/Panel.lua", true) local scale = window.display.scale.y local w = 120 local h = 24 local sep = 36 local x = framebuffer.size.x / 6 local y = framebuffer.size.y / 2 - sep * 3 self.resumebutton = CreateWidget(self.mainpanel,"RESUME GAME",x,y,w,h) self.resumebutton:SetScript("Scripts/GUI/Hyperlink.lua", true) self.resumebutton:SetFontSize(14 * window.display.scale.y) y=y+sep*2 self.label2 = CreateWidget(self.mainpanel,"OPTIONS",x,y,w,h) self.label2:SetScript("Scripts/GUI/Hyperlink.lua", true) self.label2:SetFontSize(14 * window.display.scale.y) y=y+sep*2 self.quitbutton = CreateWidget(self.mainpanel,"QUIT", x,y, w,h) self.quitbutton:SetScript("Scripts/GUI/Hyperlink.lua", true) self.quitbutton:SetFontSize(14 * window.display.scale.y) w = 400 * scale h = 550 * scale self.optionspanel = CreateWidget(self.mainpanel,"QUIT", (framebuffer.size.x- w) * 0.5, (framebuffer.size.y - h) * 0.5, w, h) self.optionspanel:SetScript("Scripts/GUI/Panel.lua", true) self.optionspanel.color = Vec4(0.2,0.2,0.2,1) self.optionspanel.border = true self.optionspanel.radius = 8 * scale self.optionspanel.hidden = true  
    • By Josh in Josh's Dev Blog 2
      Previously I talked about the technical details of hardware tessellation and what it took to make it truly useful. In this article I will talk about some of the implications of this feature and the more advanced ramifications of baking tessellation into Turbo Game Engine as a first-class feature in the 
      Although hardware tessellation has been around for a few years, we don't see it used in games that often. There are two big problems that need to be overcome.
      We need a way to prevent cracks from appearing along edges. We need to display a consistent density of triangles on the screen. Too many polygons is a big problem. I think these issues are the reason you don't really see much use of tessellation in games, even today. However, I think my research this week has created new technology that will allow us to make use of tessellation as an every-day feature in our new Vulkan renderer.
      Per-Vertex Displacement Scale
      Because tessellation displaces vertices, any discrepancy in the distance or direction of the displacement, or any difference in the way neighboring polygons are subdivided, will result in cracks appearing in the mesh.

      To prevent unwanted cracks in mesh geometry I added a per-vertex displacement scale value. I packed this value into the w component of the vertex position, which was not being used. When the displacement strength is set to zero along the edges the cracks disappear:

      Segmented Primitives
      With the ability to control displacement on a per-vertex level, I set about implementing more advanced model primitives. The basic idea is to split up faces so that the edge vertices can have their displacement scale set to zero to eliminate cracks. I started with a segmented plane. This is a patch of triangles with a user-defined size and resolution. The outer-most vertices have a displacement value of 0 and the inner vertices have a displacement of 1. When tessellation is applied to the plane the effect fades out as it reaches the edges of the primitive:

      I then used this formula to create a more advanced box primitive. Along the seam where the edges of each face meet, the displacement smoothly fades out to prevent cracks from appearing.

      The same idea was applied to make segmented cylinders and cones, with displacement disabled along the seams.


      Finally, a new QuadSphere primitive was created using the box formula, and then normalizing each vertex position. This warps the vertices into a round shape, creating a sphere without the texture warping that spherical mapping creates.

      It's amazing how tessellation and displacement can make these simple shapes look amazing. Here is the full list of available commands:
      shared_ptr<Model> CreateBox(shared_ptr<World> world, const float width = 1.0); shared_ptr<Model> CreateBox(shared_ptr<World> world, const float width, const float height, const float depth, const int xsegs = 1, const int ysegs = 1); shared_ptr<Model> CreateSphere(shared_ptr<World> world, const float radius = 0.5, const int segments = 16); shared_ptr<Model> CreateCone(shared_ptr<World> world, const float radius = 0.5, const float height = 1.0, const int segments = 16, const int heightsegs = 1, const int capsegs = 1); shared_ptr<Model> CreateCylinder(shared_ptr<World> world, const float radius = 0.5, const float height=1.0, const int sides = 16, const int heightsegs = 1, const int capsegs = 1); shared_ptr<Model> CreatePlane(shared_ptr<World> world, cnst float width=1, const float height=1, const int xsegs = 1, const int ysegs = 1); shared_ptr<Model> CreateQuadSphere(shared_ptr<World> world, const float radius = 0.5, const int segments = 8); Edge Normals
      I experimented a bit with edges and got some interesting results. If you round the corner by setting the vertex normal to point diagonally, a rounded edge appears.

      If you extend the displacement scale beyond 1.0 you can get a harder extended edge.

      This is something I will experiment with more. I think CSG brush smooth groups could be used to make some really nice level geometry.
      Screen-space Tessellation LOD
      I created an LOD calculation formula that attempts to segment polygons into a target size in screen space. This provides a more uniform distribution of tessellated polygons, regardless of the original geometry. Below are two cylinders created with different segmentation settings, with tessellation disabled:

      And now here are the same meshes with tessellation applied. Although the less-segmented cylinder has more stretched triangles, they both are made up of triangles about the same size.

      Because the calculation works with screen-space coordinates, objects will automatically adjust resolution with distance. Here are two identical cylinders at different distances.

      You can see they have roughly the same distribution of polygons, which is what we want. The same amount of detail will be used to show off displaced edges at any distance.

      We can even set a threshold for the minimum vertex displacement in screen space and use that to eliminate tessellation inside an object and only display extra triangles along the edges.

      This allows you to simply set a target polygon size in screen space without adjusting any per-mesh properties. This method could have prevented the problems Crysis 2 had with polygon density. This also solves the problem that prevented me from using tessellation for terrain. The per-mesh tessellation settings I worked on a couple days ago will be removed since it is not needed.
      Parallax Mapping Fallback
      Finally, I added a simple parallax mapping fallback that gets used when tessellation is disabled. This makes an inexpensive option for low-end machines that still conveys displacement.

      Next I am going to try processing some models that were not designed for tessellation and see if I can use tessellation to add geometric detail to low-poly models without any cracks or artifacts.
×
×
  • Create New...