ECS vs. Dynamic Typing for Lua

[Josh]

On 8/13/2025 at 10:56 PM, Alienhead said:

How is that well known engines such as Unity and others have worked around the issue you present and still allow multiple components??

People come to Leadwerks specifically because Unity is overly complicated.

[Josh]

7 minutes ago, Dreikblack said:

Less stuff in debug and for auto-complete

How would deriving all entities from a Pivot have any effect on autocompletion or debugging? Autocompletion shows all the public methods that are available. There are a lot of them. The debugger probably shows public and private members. There are a lot of those, too. The class does a lot of things.

[Dreikblack]

7 minutes ago, Josh said:

3. I have control over what properties are exposed by the Lua debugger.

Read again what i wrote then.... tldr: Entity and component would be in same place anyway if you will mix them and it's bad for new and old users for debugging, not better.

8 minutes ago, Josh said:

I just said C++ probably wouldn't gain anything by eliminating multiple components, and it's probably fine to leave that system as-is.

And again you somehow misread what i say. Whatever.

 

Just answer if the Editor interface related to components will be same or not?

[Josh]

On 8/15/2025 at 11:53 AM, beo6 said:

Why remove the option to create reusable components?
This sounds like you want to get rid of this to force people to always reimplement everything to fit their needs for no reason.

I think it worked perfectly fine for Leadwerks 4.

After some thought through standards where established, many components where made in a way to be compatible. Like the 1st person script.

Quite a few used my "Amnesia" - like notes script (i think). And it allowed some people to quickly put together some games.

Leadwerks 4 does not use multiple components, it uses a single script attached to an entity as a separate object. During the development of Leadwerks 4, I wanted to be able to add properties to the entity itself, as Lua is designed to do, but the binding library I was using at the time did not allow it. This is something we can actually do now with sol.

[Dreikblack]

13 minutes ago, Josh said:

People come to Leadwerks specifically because Unity is overly complicated.

Never heard people complaining about multi components in any engine (and all proper engines have it, i know only LW4 not having it). 

8 minutes ago, Josh said:

How would deriving all entities from a Pivot have any effect on autocompletion or debugging? Autocompletion shows all the public methods that are available.

And if pivot would not have methods and vars that does not make sense for it, autocomplete would not shows them. But they probably needed it to affect children of pivot anyway. Still annoying to have m_postion dublicated stuff.

[Josh]

Just now, Dreikblack said:

And if pivot would not have methods and vars that does not make sense for it, autocomplete would not shows them. But they probably needed it to affect children of pivot anyway.

A pivot has every method and member of the Entity class, since it is publically derived from Entity.

[Dreikblack]

1 minute ago, Josh said:

A pivot has every method and member of the Entity class, since it is publically derived from Entity.

I was saying about another hierarchy like pivot->entity-> Model | Light | Brush etc.

Anyway too late to change it

[Josh]

It looks like auto-completion filters out private members, but the debugger does not. I agree that would be a nice feature if they added it to Visual Studio.

[Dreikblack]

30 minutes ago, Josh said:

It looks like auto-completion filters out private members, but the debugger does not. I agree that would be a nice feature if they added it to Visual Studio.

Nice thought about filter. It does not exist, but i just found workaround - .natvis files https://learn.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects?view=vs-2022

It's white list only and every class member that meant to be seen need to be exposed in this file. It's gonna be tedious to make one, but results seems interesting, still can watch all vars in Raw View:

[Josh]

On 8/13/2025 at 10:18 PM, Pleca said:

Please, let’s keep both approaches. For example, in a dungeon filled with many torches, I’d use ECS for efficiency, while for a few unique objects, I’d prefer the OOP approach for simplicity. I’m with reepblue on this one.

A torch would just be a model with a particle emitter, so there would be no need for an Update function being called each frame. However, if you did have one for some reason, don't ever let anyone tell you that ECS is faster, except under extremely specific scenarios that only occur in very specific types of large games, maybe some kind of space simulation. Other aspects of this design can be debated, but performance absolutely is not a factor.

main.cpp

In this test, 2 milliseconds can be saved, in the debug build only, if 100,000 entities are being constantly updated. In release mode they run at the same speed (difference less than 1 millisecond). Not a single game I have ever played in my life comes close to those numbers, including large-scale zombie horde games.

Virtual methods also have zero measurable overhead.

We can argue about other benefits and drawbacks of such a design, but performance in C++ is definitely not a factor, one way or another, so don't ever worry about that.

[Josh]

7 minutes ago, Dreikblack said:

It's white list only and every class member that meant to be seen need to be exposed in this file. It's gonna be tedious to make one, but results seems interesting, still can watch all vars in Raw View

In Leadwerks/LSP/Lua/classes.json all the engine classes are listed in JSON tables with members in an array called "properties". That might be useful.

	"Entity":
	{
		"baseclass": "Object",
		"properties":
		[
			{
				"description": "user-defined object",
				"name": "extra",
				"type": "Object"
			},
			{
				"description": "read-only array of child entities",
				"name": "kids",
				"type": "Entity[]"
			},
			{
				"description": "read-only 4x4 matrix",
				"name": "matrix",
				"type": "Mat4"
			},
			{
				"description": "entity name",
				"name": "name",
				"type": "string"
			},
			{
				"description": "read-only angular velocity",
				"name": "omega",
				"type": "Vec3"
			},
			{
				"description": "entity parent",
				"name": "parent",
				"type": "Entity"
			},
			{
				"description": "read-only position",
				"name": "position",
				"type": "Vec3"
			},
			{
				"description": "user-defined properties",
				"name": "properties",
				"type": "table"
			},
			{
				"description": "read-only rotation",
				"name": "quaternion",
				"type": "Quat"
			},
			{
				"description": "read-only Euler rotation",
				"name": "rotation",
				"type": "Vec3"
			},
			{
				"description": "read-only scale",
				"name": "scale",
				"type": "Vec3"
			},
			{
				"description": "read-only list of managed speakers created by a call to [EmitSound](Entity_EmitSound.md)",
				"name": "speakers",
				"type": "Speaker[]"
			},
			{
				"description": "read-only container of tags",
				"name": "tags",
				"type": "string[]"
			},
			{
				"description": "read-only linear velocity",
				"name": "velocity",
				"type": "Vec3"
			}
		]
	},

 

[Josh]

@Dreikblack I went to deepai.com and had it generate a natvis file from the above. No idea if it works:

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="Entity">
    <BaseType>Object</BaseType>
    <DisplayString>{{name}}: {position=x, rotation=y, scale=z}</DisplayString>
    <Expand>
      <Item Name="Name">name</Item>
      <Item Name="Position">position</Item>
      <Item Name="Rotation">rotation</Item>
      <Item Name="Scale">scale</Item>
      <Item Name="Quaternion">quaternion</Item>
      <Item Name="Omega">omega</Item>
      <Item Name="Velocity">velocity</Item>
      <Item Name="Parent">parent</Item>
      <Item Name="Kids">kids</Item>
      <Item Name="Tags">tags</Item>
      <Item Name="Properties">properties</Item>
      <Item Name="Speakers">speakers</Item>
      <Item Name="Matrix">matrix</Item>
      <Item Name="Extra">extra</Item>
    </Expand>
    <Children>
      <!-- Optional: visualize child entities recursively -->
      <ArrayItems>
        <Size>kids.size()</Size>
        <ValuePointer>kids</ValuePointer>
      </ArrayItems>
    </Children>
  </Type>

  <!-- Visualize Vec3 -->
  <Type Name="Vec3">
    <DisplayString>{x={x}, y={y}, z={z}}</DisplayString>
  </Type>

  <!-- Visualize Mat4 -->
  <Type Name="Mat4">
    <DisplayString>Matrix</DisplayString>
    <!-- You can expand to visualize matrix elements if desired -->
  </Type>

  <!-- Visualize Quat -->
  <Type Name="Quat">
    <DisplayString>Quat({x}, {y}, {z}, {w})</DisplayString>
  </Type>

  <!-- Visualize Object -->
  <Type Name="Object">
    <!-- Customize as needed -->
    <DisplayString>Object at {Address}</DisplayString>
  </Type>

  <!-- Visualize string array (tags) -->
  <Type Name="string[]">
    <DisplayString>Tags ({size})</DisplayString>
    <Expand>
      <ArrayItems>
        <Size>size</Size>
        <ValuePointer>_Myfirst</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <!-- Visualize speakers array -->
  <Type Name="Speaker[]">
    <DisplayString>Speakers ({size})</DisplayString>
    <Expand>
      <ArrayItems>
        <Size>size</Size>
        <ValuePointer>_Myfirst</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <!-- Visualize table as needed -->
  <Type Name="table">
    <DisplayString>table</DisplayString>
    <!-- Expand properties if applicable -->
  </Type>
</AutoVisualizer>

 

[Dreikblack]

16 minutes ago, Josh said:

No idea if it works:

It's not  DisplayString format is wrong and BaseType with Children seems to be not existing tags. Fixed (with Vec3 in another file):

<?xml version="1.0" encoding="utf-8"?>
<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">
  <Type Name="Leadwerks::Entity">
    <DisplayString>
      name={name}
    </DisplayString>
    <Expand>
      <Item Name="Name">name</Item>
      <Item Name="Position">position</Item>
      <Item Name="Rotation">rotation</Item>
      <Item Name="Scale">scale</Item>
      <Item Name="Quaternion">quaternion</Item>
      <Item Name="Omega">omega</Item>
      <Item Name="Velocity">velocity</Item>
      <Item Name="Parent">parent</Item>
      <Item Name="Kids">kids</Item>
      <Item Name="Tags">tags</Item>
      <Item Name="Properties">properties</Item>
      <Item Name="Speakers">speakers</Item>
      <Item Name="Matrix">matrix</Item>
      <Item Name="Extra">extra</Item>
    </Expand>
  </Type>
</AutoVisualizer>

[wr3kage]

It feels like you're dumbing down the engine for the sake of onboarding. Catering to the lowest common denominator doesn't feel like an engine that will grow with you as a dev. I've been reconsidering my options, this feels like more of a risky engine to use now. 

[AnthonyPython]

2 hours ago, reepblue said:

component system are C++ users who seemed to be doing fine with the examples and documentation (or lack thereof) for the past few years.

my main is working in the lua in this case since I like being able to iterate faster. but this is less about fragmentation and more that there is a common notion of feeling betrayed if this is done.

 

3 hours ago, Josh said:

However, the Lua API can be greatly simplified if we store properties in the entity itself, instead of storing them in a series of sub-objects. Getting and setting properties, calling functions, and debugging are much more intuitive this way, especially for new programmers. This is how I wanted Leadwerks 4 to work, but technical limitations at the time prevented this.

I don't agree with this, this would actually start making it harder when you have so many properties and functions in the same table, how would a beginner even begin to know where to look and find something when it inevitably grows large. Even an experienced programmer has trouble finding stuff in such large files.

no one thinks unity is "too Complex" it's beginner friendly and the unity developer base would highly disagree with that.

[Josh]

The people need help learning to code games, including many people who are active on this forum. If this works out the way I am picturing, I will be able to show the users how to program gameplay mechanics, at a level they previously were not able to. I need to do some testing, see what such a design feels like, and see if it really works the way I am imagining.

If it works the way I am picturing, the tutorials will help everyone massively improve their skills. I just need to try it out and see what it feels like.

[AnthonyPython]

4 minutes ago, Josh said:

The people need help learning to code games, including many people who are active on this forum. If this works out the way I am picturing, I will be able to show the users how to program gameplay mechanics, at a level they previously were not able to.

I think you've got this wrong. people want examples sure, but they do not want to be hand held at every part to learn to code or to code a game or some specific mechanic always.

 

1 hour ago, Josh said:

If it works the way I am picturing, the tutorials will help everyone massively improve their skills. I just need to try it out and see what it feels like.

This creates tutorial hell experience ruts where people don't actually understand how to do it they don't actually improve, They just follow along with what ever some thing tells them to do. 

Then when it comes to actually making the thing they want they don't know how. except copy/pasta skills which doesn't help anyone in the slightest.
which also doesn't help you either because then you have to make every conceivable nook and cranny subject on doing X game or X mechanic, which is time you shouldn't be wasting to do because it is a never ending list and you could be using it in other area's. Am not saying to not make specialized game mechanic tutorial examples or such once in a while but I want you know what a slippery slope that is and you can be finding your self neglecting the engine and making these more than prioritizing in other area's.

Current Leadwerks 5 as a modern engine with CSG, and Level I/O with a multi-EC which easily attracts, older devs, and EC attracts modern workflow devs from a similar framework. source engine/kismet(UDK) entity I/O's for more beginner friendly people, basically no code mappers and people that like using level logic while letting more experienced or more familiar devs for more complex systems do what they wish using the EC either C++ or lua.
It is a extreme rarity of having a advanced engine which allows CSG and kismet like level logic (UDK) with the flexibility of using multiple components on top of a nice license which is what makes leadwerks shines to the public the most here. (and is what attracts us more advanced devs)

if you want to push it further and create all of these Entity based I/O's to allow that user base for entry level devs, I think that's fine, good even for that user base(I encourage it) and seems to be the area you should be focusing on more then instead of trying to dumb down the lua system. when you start messing with a devs ability to use a system they are already familiar with. Especially with experienced dev's that don't think anything is really wrong with the current system. (maybe just need to add in a couple helper functions here and there in lua) then you've got a problem. please consider keeping multi-component usage in lua for more experienced devs at the very least. we aren't asking for much here. we just don't want the rug pulled out underneath us here. which feels like what you are wanting to do. why alienate us? isn't your motto "My job is to make tools you love, with the features you want, and performance you can't live without." well we can't live with out the multi-component usage as more experience devs that either work C++ and or Lua.
Doing this would be very disrespectful to current devs that clearly show they use multi-components a lot actually and love doing it.

[Josh]

Here is a first look at what this might look like. I am compiling the experimental system using a preprocessor statement, so I can experiment with this without changing the code for the existing system.

I packed position, rotation, scale, etc. into a transform table, since you don't often need to look at these values. Script properties are displayed in a properties table, which I found was easier to view.

The relationships between entities seems more clear this way but I need to try it out in different situations and see what difference it makes. Note that in the debugger, "self" refers to the actual entity placed in the scene, in this case a Model object. This also means you don't have to type self.entity to access an entity command, because "self" is the entity.

If we open up the target value, the entity view by default is pretty compact:

And if I want to take a look at the properties on the target entity, I can open that up and verify that the target's health value is something that I am expecting it to be.

[AnthonyPython]

1 hour ago, Josh said:

The relationships between entities seems more clear this way but I need to try it out in different situations and see what difference it makes. Note that in the debugger, "self" refers to the actual entity placed in the scene, in this case a Model object. This also means you don't have to type self.entity to access an entity command, because "self" is the entity.

 

I do think doing this is a step in the right direction, wicked engine just has self as the entity it's self instead of having to go through an extra property,
but wicked also has function calls to get specific components and nothing else is in the lua entities table since it is also EC data orientated.

you should consider adding a GetComponent helper to the EC's currently if you are at least willing experiment with us here.

[Dreikblack]

9 hours ago, Josh said:

The relationships between entities seems more clear this way but I need to try it out in different situations and see what difference it makes. Note that in the debugger, "self" refers to the actual entity placed in the scene, in this case a Model object. This also means you don't have to type self.entity to access an entity command, because "self" is the entity.

Maybe just add properties table to entity than, self for entity and thats it? And people still would be able to use multi components in lua

 

And it looks like Entity already have properties table. So maybe just use it for examples and tutorials with single component? You can add GetSingleComponent() function with internal check if component table is not empty to return entity->component[0] and nil otherwise

[Josh]

14 hours ago, AnthonyPython said:

I do think doing this is a step in the right direction

The transform and properties table displayed don't actually exist in the API, that's just the way the debugger is displaying them, and it happens to display a table just because of how the debugger works. I could probably make that node have no icon, and remove the (Table) text and it would be more clear.

The entity.properties member is just a C++ table attached to the entity object. I'm not convinced it's even a good idea. You can attach properties directly to the entity object in Lua, and in C++ you have the userdata object to store your own data in.

[Josh]

Here's another example. I can tell my entity query code is not working correctly, because the returned list of entities contains the monster itself. I can easily see this just by seeing the memory address of players[1] and self are the same.

[Josh]

18 hours ago, AnthonyPython said:

you should consider adding a GetComponent helper to the EC's currently if you are at least willing experiment with us here.

If properties are added to the entity, what would GetComponent return? There is no component object.

It looks like in that engine, they pass the entity as the first argument (so it appears as self) but the component is a separate table, so the difference is just a matter of something like:

self.components[1]

Instead of:

self.entity

[Josh]

This is very interesting. I just created a box in code, set the health and team values, and the monster will start to attack the box.

local box = CreateBox(world)
box:SetPosition(0,1,0)
box:SetColor(1,0,0)
box.health = 100
box.team = 0

This tricks the monster into thinking the box is alive, and is on an opposing team, so he should attack it. I did not have to add the box into any lists or add a tag, or anything else. The monster just detects the entity because it meets the requirements of the query it uses when choosing a target:

local players = self.world:GetEntitiesInArea(p0, p1, "team","~=",self.team, "team","~=",nil, "health",">", 0)

This will filter through what could be a large number of entities and return only the ones we are interested in. Checking the team is not nil will ensure the monster does not start attacking a non-player object like a breakable crate, that could use a health value to make it break if the player shoots it.

I can dynamically add functions to the box entity, and after he finishes killing it, it disappears and he will come running after me:

function box:Damage(amount)
	self.health = self.health - amount
end

function box:Kill()
	self:SetHidden(true)
end

This is not even a component script, it's just code I stuck in main.lua before the main loop.

[Dreikblack]

Thats how it should be instead of removing components and putting everything in entities: