Jump to content

Lua binding in Leadwerks 5

Josh

1,968 views

The Leadwerks 5 API uses C++11 smart pointers for all complex objects the user interacts with. This design replaces the manual reference counting in Leadwerks 4 so that there is no Release() or AddRef() method anymore. To delete an object you just set all variables that reference that object to nullptr:

auto model = CreateBox();
model = nullptr; //poof!

In Lua this works the same way, with some caveats:

local window = CreateWindow()
local context = CreateContext(window)
local world = CreateWorld()

local camera = CreateCamera(world)
camera:SetPosition(0,0,-5)

local model = CreateBox()

while true do
	if window:KeyHit(KEY_SPACE) then
		model = nil
	end
	world:Render()
end

In the above example you would expect the box to disappear immediately, right? But it doesn't actually work that way. Lua uses garbage collection, and unless you are constantly calling the garbage collector each frame the model will not be immediately collected. One way to fix this is to manually call the garbage collector immediately after setting a variable to nil:

if window:KeyHit(KEY_SPACE) then
	model = nil
	collectgarbage()
end

However, this is not something I recommend doing. Instead, a change in the way we think about these things is needed. If we hide an entity and then set our variable to nil we can just defer the garbage collection until enough memory is accrued to trigger it:

if window:KeyHit(KEY_SPACE) then
	model:Hide()-- out of sight, out of mind
	model = nil
end

I am presently investigating the sol2 library for exposing the C++ API to Lua. Exposing a new class to Lua is pretty straightforward:

lua.new_usertype<World>("World", "Render", &World::Render, "Update", &World::Update);
lua.set_function("CreateWorld",CreateWorld);

However, there are some issues like downcasting shared pointers. Currently, this code will not work with sol2:

local a = CreateBox()
local b = CreateBox()
a:SetParent(b)-- Entity:SetParent() expects an Entity, not a Model, even though the Model class is derived from Entity

There is also no support for default argument values like the last argument has in this function:

Entity::SetPosition(const float x,const float y,const float z,const bool global=false)

This can be accomplished with overloads, but it would require A LOT of extra function definitions to mimic all the default arguments we use in Leadwerks.

I am talking to the developer now about these issues and we'll see what happens.

  • Like 1


54 Comments


Recommended Comments



Hiding and then setting to nil isn’t ideal from a usability standpoint. How about creating a base class method called Delete() which does this stuff for us in c++ as a more intuitive manner.

Share this comment


Link to comment
6 minutes ago, Rick said:

Hiding and then setting to nil isn’t ideal from a usability standpoint. How about creating a base class method called Delete() which does this stuff for us in c++ as a more intuitive manner.

All this would do is call Hide(). Maybe if we find other things this needs to do then it will be appropriate to have a Delete() method in the entity class.

Share this comment


Link to comment

The lua object is just a reference to C++ object right? Couldn't the C++ Delete() function delete itself? It's been awhile since I've used C++ but you could do:

class Entity{
  void Delete(Entity* e){
    e = nullptr;
  }
}

// whatever the smartpointer syntax is
Entity* e = new Entity();

e->Delete(e);

per https://stackoverflow.com/questions/1208961/can-an-object-instance-null-out-the-this-pointer-to-itself-safely

That way we have a standard interface of deleting objects between C++ and Lua.

Share this comment


Link to comment
6 minutes ago, Rick said:

The lua object is just a reference to C++ object right? Couldn't the C++ Delete() function delete itself? It's been awhile since I've used C++ but you could do:


class Entity{
  void Delete(Entity* e){
    e = nullptr;
  }
}

// whatever the smartpointer syntax is
Entity* e = new Entity();

e->Delete(e);

per https://stackoverflow.com/questions/1208961/can-an-object-instance-null-out-the-this-pointer-to-itself-safely

That way we have a standard interface of deleting objects between C++ and Lua.

Well actually the C++ class does not need anything to delete the object. Again, you just set the variable to nullptr:

auto model = CreateBox();
model = nullptr; //poof!

It's gone!

Share this comment


Link to comment
1 minute ago, Josh said:

Well actually the C++ class does not need anything to delete the object. Again, you just set the variable to nullptr:


auto model = CreateBox();
model = nullptr; //poof!

It's gone!

I get that's the case, but I'm trying to help find a consistent interface for this between the 2 languages because I think that's a better design and has value. You could make a global DeleteEntity(e) function perhaps that does that for both C++ objects and Lua objects so people just have to know 1 thing between the 2 languages and documentation can be shared to explain it between the 2 languages.

  • Like 1

Share this comment


Link to comment
1 hour ago, Rick said:

I get that's the case, but I'm trying to help find a consistent interface for this between the 2 languages because I think that's a better design and has value. You could make a global DeleteEntity(e) function perhaps that does that for both C++ objects and Lua objects so people just have to know 1 thing between the 2 languages and documentation can be shared to explain it between the 2 languages.

But then you're walking around with a variable that is no longer a valid entity, which is exactly what smart pointers seek to avoid:

auto box = CreateBox()
box:Delete()
box:SetPosition(1,2,3)

Moreover, one part of your code might be still using an entity when another part is done. This is why we don't have any explicit Delete(), Destroy(), Release() etc, function.

Share this comment


Link to comment
1 hour ago, Josh said:

But then you're walking around with a variable that is no longer a valid entity, which is exactly what smart pointers seek to avoid:


auto box = CreateBox()
box:Delete()
box:SetPosition(1,2,3)

Moreover, one part of your code might be still using an entity when another part is done. This is why we don't have any explicit Delete(), Destroy(), Release() etc, function.

I would assume you can make a global function that does the exact same thing as setting to nullptr. Delete(box); This way both lua and C++ can do the same thing. It's literally exactly the same thing just a more consistent interface between the 2 languages.

Share this comment


Link to comment
27 minutes ago, Rick said:

I would assume you can make a global function that does the exact same thing as setting to nullptr.

You can't. :) That's the beauty of this system. The scene Octree stores weak pointers to the entity and when the entity no longer exists, the weak pointers fail to convert into a shared_ptr, so they are skipped and removed, and the entity is no longer rendered. It removes a TON of error-prone cleanup of pointers I previously had to be very meticulous about.

It is also impossible to have an invalid pointer. :D

There is something weird to consider.

You don't want to call Hide() or Delete() or anything too aggressive because it is possible the entity might be in use somewhere else by some other script. But if you don't do this then the object will hang around until the garbage collector is called. So the correct way to handle this sometimes actually is to run the garbage collector after an entity is removed:

if window:KeyHit(KEY_SPACE) then
	model = nil
	collectgarbage()
end

This actually isn't so bad because in real life it is best to not be dynamically creating and deleting a lot of entities as the game is running.

I think in some situations you will simply set the variable to nil as if to say "I am done with this" while in other situations you will want to call Hide().

Share this comment


Link to comment

I could add a Destroy() method that removes all resources of any SharedObject (entities, materials, textures, etc.) but the object would still be valid and usable:

void Entity::Destroy()
{
	Hide();
	SetParent(nullptr);
	SetScript(nullptr);
	SetShape(nullptr);
	SetCollisionType(0);
	while (CountChildren())
	{
		GetChild(0)->Destroy();
	}
}

For example, the Lua code below would be perfectly legitimate:

local model = CreateBox()
model:Destroy()
model:Show()
local surf = model:AddSurface()
surf:AddVertex(0,1,1)
surf:AddVertex(0,-1,1)
surf:AddVertex(0,1,-1)
surf:AddTriangle(0,1,2)

And then you have a visible model again.
 

Share this comment


Link to comment
Quote

The scene Octree stores weak pointers to the entity and when the entity no longer exists, the weak pointers fail to convert into a shared_ptr, so they are skipped and removed

And a weak pointer fails to exist when the last variable that points to it sets it to nullptr?

Share this comment


Link to comment
19 minutes ago, Rick said:

And a weak pointer fails to exist when the last variable that points to it sets it to nullptr?

A weak pointer stops working when all shared pointers no longer exist. It's like a shared pointer that doesn't keep the object alive.

Share this comment


Link to comment

I think this is how you'd do it.

 

#include <memory>
#include <string>
#include <iostream>

using namespace std;

class Test{
private:
	string name;
public:
	Test(string _name){
		name = _name;
	}

	void Print(){
		cout << name;
	}
};


void Delete(shared_ptr<Test>& obj)
{
	obj.reset();
}

int main(){
	shared_ptr<Test> sptr(new Test("Test"));

	Delete(sptr);
	//sptr = nullptr;

	sptr.get()->Print();

	int input;

	cin >> input;

	return 0;
}

You can do an if(sptr) before trying to anything with it and that still works to make sure it's valid. If you do that on the C++ side for every command then in lua it'll never fail if we have multiple variables pointing to the one we deleted. The calls simply won't do anything, but at least they won't fail. You could make another function we can check perhaps like Exists(obj) that does the if statement on the shared_ptr and if not return false else return true? Just thinking high level on that one in case we want to know on the lua side if something was deleted.

Share this comment


Link to comment

What if there are other shared pointers pointing to the same object? I think reset will only invalidate that one variable:

int main(){
	shared_ptr<Test> sptr(new Test("Test"));
	auto a = sptr;
	Delete(sptr);
	//sptr = nullptr;
  	
	a->Print();

BTW you can just do sptr->Function() instead of sptr.get()->Function(). :)

Share this comment


Link to comment

From my testing reset() actually removes all references.

 

void Delete(shared_ptr<Test>& obj)
{
	long count = obj.use_count(); // will show 2

	obj.reset();

	count = obj.use_count(); // will show 0
}

int main(){
	shared_ptr<Test> sptr(new Test("Test"));

	shared_ptr<Test> ptr = sptr;

	Delete(sptr);
	//sptr = nullptr;

	sptr.get()->Print();

	int input;

	cin >> input;

	return 0;
}

Note the key is passing the shared_ptr by reference to the Delete() function. So you'd probably need 2 functions. The lua one we call that gets the shared pointer object we passed in and then another that takes the object by reference. This would be the case IF we can't pass by reference from lua. Not sure if that works with the bindings or not. If it does then great, only need the 1 delete.

Share this comment


Link to comment

If that were the case I would expect a to be NULL when it is printed here:

shared_ptr<std::string> ptr = make_shared <std::string>("Hello.");
auto a = ptr;
ptr.reset();
Print(*(a.get())); //print "Hello."

I think the object count is 0 because the shared_ptr no longer points to an object, not because all the other shared_ptrs were reset.

Share this comment


Link to comment

You may want to check that with any of the stuff you're currently doing as well. The following won't print because ptr doesn't pass the if check and count is 0 after the reset(). Perhaps some difference between objects and the string class? I mean you've seen it work with objects which you're creating in the engine, but this is a different usage with whatever string is doing behind the scenes. Or perhaps make_shared()?

	shared_ptr<std::string> ptr = make_shared <std::string>("Hello.");

	auto a = ptr;

	long count = ptr.use_count();

	ptr.reset();

	count = ptr.use_count();

	if(ptr)
		cout << *(a.get());

 

Share this comment


Link to comment

@Rick The behavior above makes perfect sense.

Smart pointers are like two people who know a language.

Let's say you and I are the only two people in the world who can speak German. If I forget German one day, the language does not die as long as one person knows it. I won't be able to tell anyone what Kartoffelsalat means, but you still know German and the language is not dead.

If we both forget to speak German then the language is dead like Latin.

Here's the equivalent code with manual reference counting:

std::string* ptr = new std::string("Hello.");
long count = 1;

auto a = ptr;
count++;

ptr = NULL;
count--;
	
if (a)
	cout << *(a);

delete a;
a = NULL;
count--;

 

Share this comment


Link to comment

But the above is saying if I asked you if you know German you're saying no I don't. The if statement check. Even though you really do, that check says you don't. However, as in our example with the Test class, the language is actually gone (and error happens when we try to access it's function because the memory doesn't seem to be there anymore). So why is the if check acting as it doesn't exist anymore which in the case of the Test() class shared_ptr it doesn't, but in this string case it does. You should always use the if check before trying to access a shared_ptr right? That's the benefit, that you can tell if it's valid or not anymore where straight pointers you can't.

Share this comment


Link to comment

Why are you basing your knowledge of one person's language skills on another person? :D

That's like saying if Shadmar's car is blue then you will drive Aggror's blue car.

Reseting a shared pointer is the equivalent to setting a pointer to NULL. You can't get any info on the object after that:

Entity* a = CreatePivot();
Entity* b = a;
a = NULL;
b->SetPosition(1,2,3);
delete b;
b = NULL;

Or a safer way in Leadwerks 4:

Entity* a = CreatePivot();

Entity* b = a;
b->AddRef();

a->Release()
a = NULL;

b->SetPosition(1,2,3);
b->Release();
b = NULL;

Now we simply do this:

shared_ptr<Entity> a = CreatePivot();
shared_ptr<Entity> b = a;
a = nullptr;
b->SetPosition(1,2,3);
b = nullptr;

If you want the removal of a to force b to be invalid, we use a weak pointer:

shared_ptr<Entity> a = CreatePivot();

weak_ptr<Entity> b = a;

a = nullptr;

shared_ptr<Entity> ptr = b.lock(); //creates a shared_ptr with a null value

if (ptr) ptr->SetPosition(1,2,3);

 

Share this comment


Link to comment

Correct, but when crossing language domains (C++/LUA) we can't set it to equal nullptr in lua directly like we can in C++. That's the entire point of my post. There is now a way to do that with the Delete() example I showed. That was why I showed it. Lua now has a way to be equal to C++ in the removal of the object vs hiding or even calling garbage collecting manually, both which are pretty hacky for when you want an object gone given you don't do that from C++ to remove the object.

Share this comment


Link to comment
2 hours ago, Rick said:

Correct, but when crossing language domains (C++/LUA) we can't set it to equal nullptr in lua directly like we can in C++. That's the entire point of my post. There is now a way to do that with the Delete() example I showed. That was why I showed it. Lua now has a way to be equal to C++ in the removal of the object vs hiding or even calling garbage collecting manually, both which are pretty hacky for when you want an object gone given you don't do that from C++ to remove the object.

Ah, I see what you mean. I will try it out and see if this works.

Share this comment


Link to comment

It sort of works. The object is deleted immediately, but Lua holds onto the invalid pointer. I am not sure how that is possible.

I think different copies of the Lua userdata actually share the same shared_ptr object, and it is only decremented after the GC collects all the userdata objects.

I will keep experimenting with this.

Share this comment


Link to comment

What do you mean by saying lua holds onto the invalid pointer? Do you mean that the lua variable doesn’t turn nil after Delete is called? I wouldn’t think it would given that lua variable is a shared pointer right? However if there was an Exists(obj) function that checked the lua shared ptr was valid and returns true or false then if we know a lua variable might be shared in our code like in the tower defense situation where 2 towers point to same obj then we use that Exists() to see if it’s actually valid on the c++ side

Share this comment


Link to comment
5 minutes ago, Rick said:

What do you mean by saying lua holds onto the invalid pointer? Do you mean that the lua variable doesn’t turn nil after Delete is called? I wouldn’t think it would given that lua variable is a shared pointer right? However if there was an Exists(obj) function that checked the lua shared ptr was valid and returns true or false then if we know a lua variable might be shared in our code like in the tower defense situation where 2 towers point to same obj then we use that Exists() to see if it’s actually valid on the c++ side

The object is deleted and its destructor is called but the shared_ptr somehow holds onto the pointer and keeps the same memory address. I have no idea how.

Normally your Exists() function is a check to see if something equals nil.

Share this comment


Link to comment
38 minutes ago, Josh said:

The object is deleted and its destructor is called but the shared_ptr somehow holds onto the pointer and keeps the same memory address. I have no idea how.

Normally your Exists() function is a check to see if something equals nil.

What's the negative aspect of the first part of what you said?

I would think because the C++ side is deleting things the Lua side isn't going to know about it so the lua variable won't be nil, which is why I was thinking the Exist() function is needed because it would do the if(shared_ptr) check in C++ which should correctly show it's no longer valid.

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...