Jump to content

Design Confusion

Josh

1,805 views

Sometimes I run into situations where I don't really know how to structure things. I don't mind this, because it usually results in some really elegant design once I figure out what to do. I just play with ideas and try not to force anything, and when the right idea arises, I will recognize it.

 

Explaining a problem to someone else can help facilitate that process. How many times have you solved a difficult problem right after you posted a description of it on a forum somewhere? The procedure of explaining it logically to someone else can help you think more clearly about it. And so, we have today's blog topic.

 

Below is a very rough start to the script editor. The syntax highlighting system was written about a year ago, and works beautifully, using the native text widget on both Windows and Mac.

blogentry-1-0-87950000-1329443513_thumb.jpg

 

In the Leadwerks3D source code, there is a base class called an "AssetEditor". From this class the material, model, shader, texture, font, and script editor classes are derived. Like the other asset editor windows, only one instance of the script editor window will be allowed open at any time. Unlike the other asset editor windows, which display only one asset at a time, the script editor will use tabs to display multiple files. Scripts aren't a typical asset like a material or a model, so it's fine for them to behave a little differently.

 

Any Leadwerks3D application can have its Lua state debugged. The engine uses networking commands to communicate with a debugger on a specified port. This means the engine can communicate with a debugger whether it's part of a C++ program, C# app, or standalone Lua interpreter. The debugger can display the Lua callstack and shows all variables and values in the Lua state, including full examination of C++ objects and members!

 

I do not intend for Leadwerks3D to "play a game" in the editor. We've tried that approach and there are a lot of problems. I want Leadwerks3D to be a very solid and stable level editor, with a visual interface to do everything you need. I also want better consistency between Lua and C++ programs. Therefore, Leadwerks3D will use a run game system more similar to 3D World Studio than the Leadwerks Engine editor. A dialog will allow you to choose the application to run, command-line parameters, and other settings. These will be saved between sessions, so you can hit a key to do a "Quick Launch" and run your game. It would be possible to hook the Lua debugger into any application as it is launched, which could be very helpful.

 

Let's go back to the script editor now. My inclination is to have F5 launch an interpreter and call the script for the currently selected tab. However, I don't think it's a good idea to use multiple game launch modes, I already described a uniform game launch mode for both Lua and C++ applications, but that seems sort of counter-intuitive if you are working in the script editor and just want to run something really quickly.

 

There;s also the question of whether we want to provide a standalone script editor and debugger outside of Leadwerks3D. Or should the debugger be a standalone application as well, since someone might want to use it with a C++ application? You see there are a lot of options and a lot of possible ways to set this up.

 

What about Lua compile errors? I can print that out in the engine log, but how will the editor display it? If a compile error occurs, should the program pause and display the line it occurred at? What if the user just doesn't care, and wants the program to keep going?

 

Alternatively, the user may want to just hit F5 in the script editor and check for basic syntax errors, which the command LuaL_LoadString() will detect. wacko.png

 

That's pretty much all my questions at this point. I don't expect anyone to come along and solve my problems, but the process of describing and discussing the issues will help me come to a resolution.



18 Comments


Recommended Comments

Make it so that F5 launches a .cmd file, which it creates from a list item field like in 3DWS. It should have also parameters, which Editor replaces with actual values. A F5 script could look like this:

{workvolume}:
cd {workdir}
cd source
mingw32-make
if errorlevel 1 goto errors
cd ..distro
game.exe {parameters} {scenefilename}
goto over
:errors
notepad++ errors.log
:over

 

And what F5 then compiled this into would be a launch.cmd file which looks like this:

d:
cd projectsgame1
cd source
mingw32-make
if errorlevel 1 goto errors
cd ..distro
game.exe 640 480 data/maps/level1.sbx
goto over
:errors
notepad++ errors.log
:over

 

There could be a set of favorite scripts which F5 can run, and also a default script which would be like:

luac {projectname} > errors.log
if errorlevel 1 goto errors
engine.exe {projectname} {parameters} {mapfilename}
goto over
:errors
notepad++ errors.log
:over

Share this comment


Link to comment

Getting the printed output of the program is not a problem. I can just create a process and read the output directly. I'm thinking more about the workflow and ease of use.

Share this comment


Link to comment

I have to agree. The number of times I've come up with a solution while explaining the problem to someone... and most of the time the someone I explain it too doesn't not understand programming but it still helps find a solution because you are pouring out your thoughts into words and for some reason that makes things make more sense.

Share this comment


Link to comment

Regarding the debug thing. Maybe a simple UDP socket on localhost would solve the problem of sending info from the program to the editor. Just a suggestion.

Share this comment


Link to comment

There's no technical problems. The problem is one of workflow. What is the best way to design this to work? (The UDP socket is actually exactly what I am doing. It allows the debugger to communicate both ways for code stepping.)

Share this comment


Link to comment

Isn't debugging done in Code::Blocks IDE? I don't quite understand why you need debugging in a CSG Modeller. 3DWS didn't have that either, and I never missed it.

Share this comment


Link to comment

3DWS didn't have that either, and I never missed it.

Maybe because you didn't have it :) And this is not the same thing. In 3DWS you had no scripts to debug, right-

Share this comment


Link to comment

But the programming should be consistent between all languages, including Lua. Most people probably prefer Code::Blocks for Lua projects also, because it has very sophisticated plug-ins for code clips management, and is thus more suitable for larger game projects than a simple text editor.

Share this comment


Link to comment

For the lua compile errors I would think following Visual Studio's model of having a window at the bottom of the editor that will show each error would work well. We can dbl click each error line to go to the error in the file.

 

I currently use Notepad++ for Lua programming in LE. Mainly because it has tabbed browsing of files which I think would be very handy in your script editor. Sometimes we have to bounce back and forth between scripts and without tabs it's very inefficient today.

 

Are you giving us access to an "int main" type of script like you do today or is that hidden from us and we work with these scripts that we attach to entities only?

 

I would have a separate button/key for syntax checking only. This is common in SQL editors where you have one option to check syntax, then another option to actually run it. It's very handy and with all these little scripts LE 3D will have I can see that being handy also.

 

So F6 to "compile" the script to check syntax errors and if errors show up a window at the bottom of the editor to allow us to go to the error line.

 

F5 to actually run the game. I wouldn't think you'd ever just run one script itself would you? I guess at first glance that wouldn't make much sense to me as it's part of the entire game and can have many connections/requirements to/from other scripts.

 

If a run-time error happens, stop the game on that line, and open the file that contains the error in another tab and highlight it yellow and show the error in a messagebox.

Share this comment


Link to comment

Since i am not a native english, i hope that i misunderstood the part of not being able to launch a script as a whole game, currently all my game is based on lua scripts which call each other and a launch (F5) button is all what i need.

I don't know 3DStudio but i hope we can do the same as now.

Sorry for my fear but i hope someone will chill me out

Share this comment


Link to comment

Don't worry, whatever Josh comes up with, you will be still able to do the following always:

1) Press Ctrl-S in Editor

2) Press F5 in ScriptEditor to compile (if changed) and run the game

because you have to be able to do that anyway for all other languages also:

1) Press Ctrl-S in Editor

2) Press F9 in Code::Blocks to compile (if changed) and run the game

 

Josh is only trying to integrate the Lua Editor into Leadwerks Editor, which is kinda useless in my opinion anyway. It could be just left out. Or he could find a way how to make the above 2 steps even easier, although it's OK how they are now too.

Share this comment


Link to comment

I don't see any point in setting up Leadwerks3D to launch command-line compilers for C++ or other languages (though you could). It gets very complicated for so many languages and platforms, and the feedback would be inferior to just using the real IDEs for that language. I just see it as launching either the script interpreter, or your own executable that you define.

 

I've been using C++ so long, that I am used to a big wall of text whenever I hit F5, but I think the feedback when Lua loads a file is just a single error line if anything goes wrong. Therefore, there is no need to print out a bunch of compile information. If you press F5 in the script editor, the Lua file can be evaluated, and if any syntax errors exist, a message box can pop up and the line in question can be highlighted. This is simply evaluating the syntax of the script, not running anything.

 

Now when we want to launch a scripted game, I can either launch it from the script editor, using the script in the selected tab, or I can do it through a "run game" dialog where you define whether it's a scripted game or your own executable. The advantage of the "run game" dialog is you can launch script and C++ games and debug their Lua states in one interface. However, I feel like it's more natural and easier to launch a script by hitting a hot key in the script editor, which is my main point of conflict in this design.

 

Should the debugger and game printed output dialogs be built into the script editor window? That seems logical, but what about C++ applications? Am I going to make a separate script debugger and output window for those? It's a bad idea to have two versions of the same interface.

Share this comment


Link to comment
but I think the feedback when Lua loads a file is just a single error line if anything goes wrong.

 

This could be inefficient if it's the case as you could have 20 syntax mistakes and it'll only show you the first one? Then you have to keep syntax checking over and over again? Sounds like a pain.

 

The advantage of the "run game" dialog is you can launch script and C++ games and debug their Lua states in one interface

 

People using C++ will most likely be launching their game from the C++ IDE so they can debug their C++ code. I know most C++ coders here were upset that Lua was "required" to get some editor information from models in a map. The C++ people seem to not really want anything to do with Lua. They (I) won't create an exe of their C++ program just to then launch it from the editor because you lose the ability to debug your C++ code which will have the bulk of logic in it. Any attempt to merge the editor with their IDE will most likely fail and not be a smart use of your time (imho).

 

I personally think your Lua design is mixed mode which makes things more difficult. You seem to still want to give an "int main" type Lua file and not fully commit to the Lua model script system for total interaction of the entire game logic for games using Lua. If you did, then your engine.exe program would be the only thing you need to launch when we press Play from the editor. You setup properties in the editor that engine.exe reads and uses for certain things like debug info. engine.exe should be a very generic game loop that reads settings from a file to determine specific things it should be doing. All your game logic should then be defined in the model Lua scripts. There is no need to then give us an "int main" Lua file. There would be no point in it.

 

Just so you are aware, this isn't me being radical. A good number of engines don't provide an "int main" to the developers and amazing games have been created with them. For LE Lua I think it'll help to take "int main" away as it starts to shape how people code with it. If there are too many possibilities it segregates the community.

 

On the C++ side:

Like I was saying I don't think the C++ people will be pressing Play from the editor. They will be running their game from their C++ IDE loading the map they are working on from the editor.

Share this comment


Link to comment

I agree with Rick, when I write a game in C++, I don't want it be infected with Lua at all, and wasting valuable memory for its GC and interpreter. I need all memory for my game to get the maximum content at maximum speed (everything loaded into memory makes things much faster).

 

But like I already said to diedir, if there would be a way to make the 2 keystrokes less (well actually 3 keystrokes, since there is Alt-Tab between them too).

 

Maybe F5 in Editor could be programmed to switch to the Code::Blocks IDE window, and then send a F9 keystroke. That would actually work! Then you can use your favorite language and your favorite IDE, and only the window class and F9 key would need to be configured in Editor. And you would have a debugger too for all languages.

Share this comment


Link to comment
This could be inefficient if it's the case as you could have 20 syntax mistakes and it'll only show you the first one? Then you have to keep syntax checking over and over again? Sounds like a pain.

That's just the way Lua works. It stops on the first error. This actually annoys me about C++ because one syntax errors causes a whole cascade of errors, and you end up with 40 error messages because you forgot one semicolon.

 

Maybe F5 in Editor could be programmed to switch to the Code::Blocks IDE window, and then send a F9 keystroke. That would actually work! Then you can use your favorite language and your favorite IDE, and only the window class and F9 key would need to be configured in Editor. And you would have a debugger too for all languages.

One of my design rules is not to rely on or try to integrate third-party tools I don't have complete control of. This is slightly different, but recall all the problems that arose when I relied on the third-party Newton serialized format.

 

I personally think your Lua design is mixed mode which makes things more difficult. You seem to still want to give an "int main" type Lua file and not fully commit to the Lua model script system for total interaction of the entire game logic for games using Lua.

That's correct. I think a global script is always needed for a Lua game and attempts to abstract that away end up being more confusing.

 

The ability to debug the Lua state of a C++ program is a new idea you have not been exposed to before, but I think it will end up being very useful, especially for professional studios. Think about when your game code is done and artists are producing maps for the game. They'll want to be able to easily launch the game with the current map, and they will want to be able to inspect the Lua state to see if something goes wrong.

Share this comment


Link to comment

I still think the window class + keystroke macro should be possible with F5, or then F9 in Editor, if you do some wierd stuff with Lua states in F5.

Or then you could add the possibility to write Lua or dll plug-ins for Editor, then the community can make that.

Share this comment


Link to comment

It'll be interesting to see how this works and if it will be different enough than how Lua works with LE today to provide real code sharing, but I'm not very optimistic about it because it still allows different design approaches which will reduce the amount of sharing. Not having access to "int main" doesn't limit a game in any way.

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Blog Entries

    • By Josh in Josh's Dev Blog 4
      I made some changes to the design of the particle system. I am less concerned with the exact behavior of particles as they move around and move interested right now in building a system with good performance and deep physics interactions. Although I want particle behavior to be customizable, I don't think scripts are the right tool for the job. C++ plugins are better suited for this for two reasons.
      C++ is much faster, and particles are a system that will make heavy use of that. Lua scripts can't be run on separate threads. In Leadwerks Engine 4 we have basic particle collisions, but I wanted something more interactive in the new system. I move the particle update code into the physics thread. I implemented collision as well as the ability for particles to exert forces on other objects. Here's what happens when some slow-moving smoke particles interact with a scene: The lower platform rotates freely while the upper platform is motorized.
      When the particle velocity is increase they start to behave like a stream of water:
      Best of all, the speed is surprisingly fast. 4000 particles with collision update in just 2 milliseconds. The code scales well across cores so if you have a lot of CPU cores simulations with 100,000 particles are possible.
      Right now particles are processed in the physics thread, and get sent to the rendering thread for display, but right now the main thread actually never sees the individual particles.
      This is fast enough I think particles will default to full physics. Instead of just being a dumb visual effect we are going to have fully interactive fluids and gases. Flamethrowers can fill a room with fire and it will creep around corners to fill a space.
    • By Josh in Josh's Dev Blog 7
      For finer control over what 2D elements appear on what camera, I have implemented a system of "Sprite Layers". Here's how it works:
      A sprite layer is created in a world. Sprites are created in a layer. Layers are attached to a camera (in the same world). The reason the sprite layer is linked to the world is because the render tweening operates on a per-world basis, and it works with the sprite system just like the entity system. In fact, the rendering thread uses the same RenderNode class for both.
      I have basic GUI functionality working now. A GUI can be created directly on a window and use the OS drawing commands, or it can be created on a sprite layer and rendered with 3D graphics. The first method is how I plan to make the new editor user interface, while the second is quite flexible. The most common usage will be to create a sprite layer, attach it to the main camera, and add a GUI to appear in-game. However, you can just as easily attach a sprite layer to a camera that has a texture render target, and make the GUI appear in-game on a panel in 3D. Because of these different usages, you must manually insert events like mouse movements into the GUI in order for it to process them:
      while true do local event = GetEvent() if event.id == EVENT_NONE then break end if event.id == EVENT_MOUSE_DOWN or event.id == EVENT_MOUSE_MOVE or event.id == EVENT_MOUSE_UP or event.id == EVENT_KEY_DOWN or event.id == EVENT_KEY_UP then gui:ProcessEvent(event) end end You could also input your own events from the mouse position to create interactive surfaces, like in games like DOOM and Soma. Or you can render the GUI to a texture and interact with it by feeding in input from VR controllers.

      Because the new 2D drawing system uses persistent objects instead of drawing commands the code to display elements has changed quite a lot. Here is my current button script. I implemented a system of abstract GUI "rectangles" the script can create and modify. If the GUI is attached to a sprite layer these get translated into sprites, and if it is attached directly to a window they get translated into system drawing commands. Note that the AddTextRect doesn't even allow you to access the widget text directly because the widget text is stored in a wstring, which supports Unicode characters but is not supported by Lua.
      --Default values widget.pushed=false widget.hovered=false widget.textindent=4 widget.checkboxsize=14 widget.checkboxindent=5 widget.radius=3 widget.textcolor = Vec4(1,1,1,1) widget.bordercolor = Vec4(0,0,0,0) widget.hoverbordercolor = Vec4(51/255,151/255,1) widget.backgroundcolor = Vec4(0.2,0.2,0.2,1) function widget:MouseEnter(x,y) self.hovered = true self:Redraw() end function widget:MouseLeave(x,y) self.hovered = false self:Redraw() end function widget:MouseDown(button,x,y) if button == MOUSE_LEFT then self.pushed=true self:Redraw() end end function widget:MouseUp(button,x,y) if button == MOUSE_LEFT then self.pushed = false if self.hovered then EmitEvent(EVENT_WIDGET_ACTION,self) end self:Redraw() end end function widget:OK() EmitEvent(EVENT_WIDGET_ACTION,self) end function widget:KeyDown(keycode) if keycode == KEY_ENTER then EmitEvent(EVENT_WIDGET_ACTION,self) self:Redraw() end end function widget:Start() --Background self:AddRect(self.position, self.size, self.backgroundcolor, false, self.radius) --Border if self.hovered == true then self:AddRect(self.position, self.size, self.hoverbordercolor, true, self.radius) else self:AddRect(self.position, self.size, self.bordercolor, true, self.radius) end --Text if self.pushed == true then self:AddTextRect(self.position + iVec2(1,1), self.size, self.textcolor, TEXT_CENTER + TEXT_MIDDLE) else self:AddTextRect(self.position, self.size, self.textcolor, TEXT_CENTER + TEXT_MIDDLE) end end function widget:Draw() --Update position and size self.primitives[1].position = self.position self.primitives[1].size = self.size self.primitives[2].position = self.position self.primitives[2].size = self.size self.primitives[3].size = self.size --Update the border color based on the current hover state if self.hovered == true then self.primitives[2].color = self.hoverbordercolor else self.primitives[2].color = self.bordercolor end --Offset the text when button is pressed if self.pushed == true then self.primitives[3].position = self.position + iVec2(1,1) else self.primitives[3].position = self.position end end This is arguably harder to use than the Leadwerks 4 system, but it gives you advanced capabilities and better performance that the previous design did not allow.
    • By Josh in Josh's Dev Blog 3
      I wanted to work on something a bit different, and this sure is different. I've got a framework of a new particle system worked out. What's really special about this system is the amount of interactivity the particles will allow.
      Particle-world collisions. Particle-particle collisions (repulsion) Particle-particle cohesion (fluids with surface tension) Instead of just being a visual effect, I want our new particles to be fully interactive with physics so that particles can exert forces on objects. This will allow you to simulate fluids, smoke, and other effects in a realistic manner, not just dumb collision of particles bounding off walls. It should even be possible to simulate hydrophobic and hydrophillic liquids if you mix two together with different cohesion values.
      Basically what I want is something like Nvidia Flow on the CPU and exerting forces on the world. So if you had water falling on a water wheel the wheel would move because of the forces, or a blast of wind could knock objects over without any special force fields or other fake effects.
      I also have a technique worked out that will allow lighting of clouds and other masses of gas, with back-scattering.
      Emitters can be instanced so if you have one really high-quality torch effect, for example, you can instance it and use it as much as you like without any additional computational cost per instance.
      Particle emitters can be controlled with a Lua script or C++ actor. Two new functions are available, UpdateParticle() and EmitParticle(). Here is a script that controls particle behavior over time:
      entity.particleVelocity = Vec3(0,0,1) entity.particleAcceleration = Vec3(0,-1,0) entity.inverseSquareFalloff = true entity.particleRadiusBegin = 0.1 entity.particleRadiusEnd = 0.2 entity.particleColorBegin = Vec4(1,1,1,1) entity.particleColorEnd = Vec4(1,1,1,0) entity.particleMass = 1 entity.particleSpin = 5 function entity:Start() self.particleColorBeginHSL = HSL(self.particleColorBegin.rgb) self.particleColorEndHSL = HSL(self.particleColorEnd.rgb) local emitter = Emitter(self) if emitter == nil then return end local n for n = 1, #emitter.particles do emitter.particles[n].mass = self.particleMass emitter.particles[n].falloff = (n-1) / (#emitter.particles - 1) end end function entity:EmitParticle(index) local emitter = Emitter(self) if emitter == nil then return end emitter.particles[index].position = self:GetPosition(true) emitter.particles[index].velocity = TransformVector(self.particleVelocity,self,nil) emitter.particles[index].radius = self.particleRadiusBegin emitter.particles[index].color = self.particleColorBegin end function entity:UpdateParticle(index) local emitter = Emitter(self) if emitter == nil then return end emitter.particles[index].velocity = emitter.particles[index].velocity + self.particleAcceleration / 60 local falloff = emitter.particles[index].falloff if self.inverseSquareFalloff then falloff = falloff * falloff end emitter.particles[index].color.rgb = RGB(self.particleColorBeginHSL * (1 - falloff) + self.particleColorEndHSL * falloff) emitter.particles[index].color.a = self.particleColorBegin.a * (1 - falloff) + self.particleColorEnd.a * falloff emitter.particles[index].radius = self.particleRadiusBegin * (1 - falloff) + self.particleRadiusEnd * falloff emitter.particles[index].rotation = emitter.particles[index].rotation + self.particleSpin / 60 end A different script could be used to make particles emit from vertices of a model, to make the model appear to be on fire, or other effects. This will allow infinite customization to create any behavior you want.
      Particle physics will be calculated on the physics thread so I expect them to be very fast.
×
×
  • Create New...