Jump to content

Non-instancing copy command


Niosop
 Share

Recommended Posts

As I understand it the current CopyEntity command creates a new instance of an existing entity. So if I wanted to swap out the texture on just one instance of a model there's currently no way to do this because changing the texture on one would change it for all instances of that model and even another call to LoadModel would just create another instance of the original. I suppose you could pass all the possible textures to a custom shader and use some per instance value like color to choose between them, but that could get unwieldy very quickly. Could an argument be added to CopyEntity that would allow creation of a totally separate, non-instanced copy of an entity so that per model properties (like the material) could be changed without affecting the other instances?

 

Or if I'm missing some method for this that's already in place, let me know :)

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

You can do this by assigning a new material in object properties-appearence pannel into Leadwerks editor.

 

 

oops my bad :blink: it will change all instances, sorry :)

Omid Saadat

OD Arts Blog

 

AMD Phenom II X4 940 - Geforce 8800GTS - 4GB RAM - XP x86

AMD 6000+ - Geforce 9800 GT - 2GB RAM - XP x86 (Home pc)

Intel Core i7 - Geforce 310M - 4GB Ram - Win7 x64 (Laptop)

Link to comment
Share on other sites

and even another call to LoadModel would just create another instance of the original

You just need to copy (maybe also renaming would work) all the files and then load these, this will give you a new model which is no instance of any other entity.

Loading the same file does indeed always return just an instance.

 

I wrote these utility functions that do exactly this:

#include <string>
using namespace std;

TEntity LoadMeshUninstanced(str name, TEntity parent = 0)
{
// Prepare filename
static int loadedMeshes = 0; loadedMeshes++;
char buf[10];

string originalFilePath = AbstractPath(name);
originalFilePath.erase(originalFilePath.length()-4, 4);

string newFilePath = originalFilePath;
newFilePath += "_meshInstance";
newFilePath += itoa(loadedMeshes, buf, 10);

// # Copy all files
CopyFile((originalFilePath+".gmf").c_str(), (newFilePath+".gmf").c_str(), false);

// # Load model
TEntity mesh = LoadMesh(const_cast<str>((newFilePath+".gmf").c_str()), parent);

// # Delete temp files
DeleteFile((newFilePath+".gmf").c_str());

return mesh;
}

TEntity LoadModelUninstanced(str name, TEntity parent = 0)
{
// Prepare filename
static int loadedModels = 0; loadedModels++;
char buf[10];

string originalFilePath = AbstractPath(name);
originalFilePath.erase(originalFilePath.length()-4, 4);

string newFilePath = originalFilePath;
newFilePath += "_instance";
newFilePath += itoa(loadedModels, buf, 10);

// # Copy all files
// Models and lods
long hr = 1;
for(int lod = 0; hr != 0; lod++)
{
	string lodAdd = "";
	if(lod > 0)
	{
		lodAdd += "lod";
		lodAdd += itoa(lod, buf, 10);
	}

	hr = CopyFile((originalFilePath+lodAdd+".gmf").c_str(), (newFilePath+lodAdd+".gmf").c_str(), false);
}
// Other files
CopyFile((originalFilePath+".ini").c_str(), (newFilePath+".ini").c_str(), false);
CopyFile((originalFilePath+".lua").c_str(), (newFilePath+".lua").c_str(), false);
CopyFile((originalFilePath+".phy").c_str(), (newFilePath+".phy").c_str(), false);

// # Load model
TEntity model = LoadModel(const_cast<str>((newFilePath+".gmf").c_str()), parent);

// # Delete temp files
hr = 1;
for(int lod = 0; hr != 0; lod++)
{
	string lodAdd = "";
	if(lod > 0)
	{
		lodAdd += "lod";
		lodAdd += itoa(lod, buf, 10);
	}

	hr = DeleteFile((newFilePath+lodAdd+".gmf").c_str());
}
DeleteFile((newFilePath+".ini").c_str());
DeleteFile((newFilePath+".lua").c_str());
DeleteFile((newFilePath+".phy").c_str());

return model;
}

 

Usage example:

int main(void)
{
// Init
Initialize();
Graphics(640,480);
TEntity world = CreateWorld();
TEntity buffer = CreateBuffer(640,480,BUFFER_COLOR0|BUFFER_DEPTH|BUFFER_NORMAL|BUFFER_COLOR2);

// Cam
TEntity camera = CreateCamera();
MoveEntity(camera, Vec3(0,0,-2.5) );

// Light
TEntity light  = CreateSpotLight(15,0);
MoveEntity   (light,  Vec3(-1,5,-4) );
RotateEntity (light,  Vec3(45,0,0), 0);
SetShadowmapSize(light,512);
AmbientLight(Vec3(.05));

// Ground
TEntity plane  = CreateCube(0);
ScaleEntity  (plane,  Vec3(100,1,100) );
MoveEntity   (plane,  Vec3(0,-2.5,0) );

// Model
TEntity m1 = LoadModelUninstanced("abstract::oildrum.gmf");
PositionEntity   (m1,  Vec3(-1.5,0,0) );

TEntity m2 = LoadModelUninstanced("abstract::oildrum.gmf");
PositionEntity(m2,  Vec3(1.5,0,0) );
PaintEntity(GetChild(m2,1), LoadMaterial("abstract::oakbranch.mat"));

// Main Loop
while(!KeyHit(KEY_ESCAPE) && !AppTerminate())
{
	UpdateWorld();

	SetBuffer(buffer);
	RenderWorld();
	SetBuffer(BackBuffer());
	RenderLights(buffer);

	Flip(1);
}

// Terminate
return Terminate();
}

Notice both oil drums would have the "oakbranch.mat" applied if you used LoadModel.

Link to comment
Share on other sites

Do we have any way to make a copy of the model/mesh from memory instead of from file? That would be way cooler and cleaner as this "could" cause files to get left behind if something happens between making the file copy and deleting it. Although it's unlikely it would just be cleaner to be able to copy the memory of a loaded object already. I have to imagine there is a way to do this. I mean if you have a pointer to the loaded model in memory, we should be able to copy that memory to another location and get a pointer to it. I'm not that handy with low level memory stuff though, but it would be much cleaner that way.

 

Can't we do something with memcpy() in C/C++?

Link to comment
Share on other sites

Yes, I was thinking of taking this approach, the main problem with this is you can't use the zip file system without a whole bunch more hassle opening the zip, getting and renaming all objects, then recreating a new zip. So it might not be allowed by the Dexsoft license because you'd end up w/ unprotected files (unless the GMF format counts as protection). An in memory system would rock, or a way to change an instance of a model into an independent model. Maybe like Model:MakeUnique() or something.

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

An in memory system would rock

 

I have to imagine this is possible. I mean if we know the way it's stored in memory and we have a pointer to that memory, there must be a way to copy that memory to a new memory location. That would essentially create a new instance of that model and allow you to do whatever to it.

 

 

memcpy

 

function

<cstring>

 

void * memcpy ( void * destination, const void * source, size_t num );

 

Copy block of memory

Copies the values of num bytes from the location pointed by source directly to the memory block pointed by destination.

 

The underlying type of the objects pointed by both the source and destination pointers are irrelevant for this function; The result is a binary copy of the data.

 

So we need to get the size of a load model object in memory. I guess I just don't know how to get the engine to see this new model memory though. It would have to be able to be added to the world somehow. LoadModel() must do this. If we knew how it does that we could make this work I think.

 

Either way, this should 100% be added by Josh to the engine. It's such a basic thing to have.

Link to comment
Share on other sites

Can't just do a memcpy because the structure contains pointers to it's sub components. So even if you copy the parent structures data, it would point to the same child data. You'd need to do it recursively, and without more information on how stuff is laid out internally we can't do it from our side. Plus this might wreak havok w/ Blitzmax's garbage collection and stuff, I really don't know.

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

You'd need to do it recursively, and without more information on how stuff is laid out internally we can't do it from our side.

 

Where are the hackers at? I'm sure it can be figured out by some nice memory snooping tools. I wouldn't think that's a big deal to Josh.

Link to comment
Share on other sites

Would moving the material from per model to per instance cause a big performance hit? Some uniforms are passed per instance (like color, scale, etc), so is there something special about the textures or shaders that would break batching if they were per instance?

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

I think it would just mean you couldn't have a scene with a large amount of the same model because it would take up so much memory compared to the way it is now. I would say things like tree's but I think he made that neat trick with vegetation where you can have a ton of trees and it doesn't take a big hit.

Link to comment
Share on other sites

Some uniforms are passed per instance (like color, scale, etc), so is there something special about the textures or shaders that would break batching if they were per instance?

Nothing except the transformation matrix is passed per instance.

The colored is stored in the 4th row of it (which is actually a "hack").

Link to comment
Share on other sites

Yeah, if they can't share texture memory that would be a killer, but I would think that you could still share texture memory. What has to be the same for VBO to work? The vertex data, the normal data and the texture coordinates? I wouldn't think the actual texture data would have to be the same, but I'm just making wild guesses.

 

EDIT: Just saw Masterxilo's response. If the transformation matrix is the only thing that can change to still get the benefits of batching then we're back to only being able to do it by setting a material w/ the possible textures in texture slots and using something like the alpha channel info in the shader to choose among them.

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

Where are the hackers at? I'm sure it can be figured out by some nice memory snooping tools. I wouldn't think that's a big deal to Josh.

 

For Josh it wouldn't, since he has access to the actual BMax members. For us, it would be quite a lot of trouble.

Link to comment
Share on other sites

@Niosop:

 

The only thing that has to be the same in a VBO is vertex data, normal data, and texture coordinate data. If you are using primary/secondary colors or other indices for the array pointers in the VBO, those must also be the same, but since the textures are passed as samplers to the shader per-mesh, they could be different.

 

From what I remember, this instancing behavior is something Leadwerks implemented in its own way using some kind of reference system vs. instances.

52t__nvidia.png nVidia 530M cpu.gif Intel Core i7 - 2.3Ghz 114229_30245_16_hardware_memory_ram_icon.png 8GB DDR3 RAM Windows7_Start.gif Windows 7 Ultimate (64x)

-----

IconVisualStudio16.png Visual Studio 2010 Ultimate google-Chrome.png Google Chrome PhotoshopLinkIndicator.png Creative Suite 5 icon28.gif FL Studio 10 MicrosoftOfficeLive.png Office 15

-----

csharp.png Expert cpp.png Professional lua_icon.png Expert BMX Programmer

-----

i-windows-live-messenger-2009.pngskype-icon16.pngaim_online.pnggmail.pngicon_48x48_prism-facebook.pngtunein-web.pngyahoo.giftwitter16.png

Link to comment
Share on other sites

The reason instancing is used is because right now the engine does this:

 

-Make an array of mat4s for all the visible instances.

-Set the material/shader/textures.

-Set the vertex buffers

-Tell the GPU "take this array of matrices and render N copies of it". This is only one draw call that draws all instances of the object.

-Unset the vertex buffers

-Unset the material/textures/shader.

 

This makes it very very fast to draw lots of copies of an object.

My job is to make tools you love, with the features you want, and performance you can't live without.

Link to comment
Share on other sites

But it causes issues when we want to have 1 model with different textures, which is a very common approach to things. Couldn't we have both worlds where by default the above you have happens, but if we choose to we can make them handled as if they were different models completely so they can have different textures? Us having to copy model files on disk is a pretty bad hack and can lead to issues with copies of model files cluttering up the players hard drive if issues happen between making the model and copying it.

 

Scenery and such is perfect for the way you have it, but a character is a very common place for the same model used with different textures.

 

Also, in my character object I'd love to have settings for changing textures in the editor. How easy it would be to populate a scene with what looks like so many different characters if we had this ability all from 1 model on disk. Scenes could come to life, because right now I'm tired of looking at woods.

Link to comment
Share on other sites

You can actually do what you describe using a shader, without losing instancing. Put four different textures on one, and use the entity color to determine the texcoord offset for which texture is used. That's how I would do it.

My job is to make tools you love, with the features you want, and performance you can't live without.

Link to comment
Share on other sites

Why should we have to use a shader?

 

Why can't you just add support for non-instanced entities? Or atleast describe whether or not it is viable.

 

I am sick and tired of these hacky and backwards solutions to things that have a clear and concise way to be solved, it is just a bit irritating, and is a sign of ignorance and laziness in my honest opinion. If you want to know why people *cough* want source code, it is so they can fix things like this that you simply dodge in your response.

 

EDIT: Since I know people can't tell my tone properly from my posts, I just want to clarify this is in no way intended to be hostile. It is to be taken lightly, with a slight humorous undertone, but with a stern point about some general flaws that are causing many of the disagreements and quarrels around here.

  • Upvote 1
  • Downvote 1

52t__nvidia.png nVidia 530M cpu.gif Intel Core i7 - 2.3Ghz 114229_30245_16_hardware_memory_ram_icon.png 8GB DDR3 RAM Windows7_Start.gif Windows 7 Ultimate (64x)

-----

IconVisualStudio16.png Visual Studio 2010 Ultimate google-Chrome.png Google Chrome PhotoshopLinkIndicator.png Creative Suite 5 icon28.gif FL Studio 10 MicrosoftOfficeLive.png Office 15

-----

csharp.png Expert cpp.png Professional lua_icon.png Expert BMX Programmer

-----

i-windows-live-messenger-2009.pngskype-icon16.pngaim_online.pnggmail.pngicon_48x48_prism-facebook.pngtunein-web.pngyahoo.giftwitter16.png

Link to comment
Share on other sites

You can actually do what you describe using a shader, without losing instancing. Put four different textures on one, and use the entity color to determine the texcoord offset for which texture is used. That's how I would do it.

 

Cool, hadn't thought of using a texture offset. If we keep it to 4096x4096 texture sizes then we get 16 1k textures per texture sheet and could even use multiple textures. Red channel determines where on the texture sheet to look, green channel determines which texture sheet to use or something. Between the two of them that should be plenty. It's not as "clean" as non-instanced copies, but it should have better performance.

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

You can actually do what you describe using a shader, without losing instancing. Put four different textures on one, and use the entity color to determine the texcoord offset for which texture is used. That's how I would do it.

 

 

I have no idea how to do that. The problem is most models you buy come with 1 model and multiple textures and have coords setup already. Also, is there any limitations to your idea?

 

 

Tyler, I can't understand how anyone is supposed to read anything but anger in your post.

Link to comment
Share on other sites

I'm going to need to learn shaders programming at some point so I'll take a shot at this. I think I see how to do it, I'll just add a #MULTI-TEXTURE define to mesh.frag that will use the alpha channel of the color to map up to 16 textures in a single texture sheet. Should have something to show shortly.

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

It's not a hacky solution. It's efficient and works really well with the way the hardware works.

 

Non-instanced copies might someday be added, but you should be aware you are asking me to increase bugs and decrease performance.

My job is to make tools you love, with the features you want, and performance you can't live without.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
 Share

×
×
  • Create New...