Dynamic terrains

[VeTaL]

We really need dynamics terrain as fast as possible... Artists want so large scale of building, so we need dynamically loading of terrain and objects.

 

As i remember, somewhere in forums there were discussion about dynamically loading of models into a memory in another thread and then copying it to Leadwerks. We can write this loader, but we can do nothing with dynamic terrain loading.

Josh, can you help with it?

[VeTaL]

I checked size of terrain... black line in the left down corner is about 3 minutes of walk. So, player can walk through terrain for about 12 minutes from side to side... well, thats not enough.

[Josh]

The simplest thing you can do is set the meters per tile to 2. That will still give a good resolution, and you will quadruple your area, if it is set to 1.0 right now.

 

Making an infinite/paged system is perfectly possible, but the problem is it would severely restrict the engine for every game that wasn't designed to do that...which is the majority of games. It would involve a lot of constraints for the user, and there's not enough people who want to make a game like this enough to pay a lot extra for such technology. So I could devote all my time to designing a beautiful infinite system that would be really awesome, but I'd go broke doing it.

 

The terrain shown there is 1/4 the max terrain area, by the way.

[VeTaL]

//The simplest thing you can do is set the meters per tile to 2. That will still give a good resolution, and you will quadruple your area, if it is set to 1.0 right now.

We tested that, but we dont like resolution.

 

//The terrain shown there is 1/4 the max terrain area, by the way.

Yep, i know, but even 20 minutes is looking not enough...

 

I noted that now it is possible to attach only 1 terrain per map. I dont ask "beautiful infinite system", but just a possibility to use more terrain per level, so we could load others dynamically (maybe with the help of multithread-loader).

[Josh]

It's not that simple. Then I have to add support in the editor for multiple terrains. Each of the largest terrains requires about 50 mb of texture data on the GPU. Okay, now times 9, that exceeds the GPU memory of almost all cards. So a paging system would be needed, but that is no small task.

[VeTaL]

Maybe it would make a sense to work with not so large terrain? If we would use smaller one, for example 1000*1000

[VeTaL]

Or even smaller terrains?

If i can walk through side of 2K*2K terrain for 10 minutes, then i'd be able to walk through 512*512 for 2.5 minutes... so, 3 little (512*512) terrains would give 7.5 minutes of walking. It would use less memory and give me access to infinity terrains.

Then i would add LODs and so on, but i really need to use more than one terrain.

[wh1sp3r]

you can make them self It's not that big problem, but you need a own editor for that, that's could be a problem.

[VeTaL]

http://leadwerks.com/werkspace/index.php?/topic/1513-leadwerks-30/page__view__findpost__p__15546

There are some people, who want to see it

 

wh1sp3r, how can i do that? how can i load some terrains, when engine is designed to use only 1 terrain?

 

well, looking like i can make a special tool, which will separate terrain into different parts, for example:

i have a map

1 2 3

4 5 6

7 8 9

 

so tool would generate all combinations of maps into separate levels:

1 2

4 5

, next one

2 3

5 6

, next is

1 2

4 5

7 8

, and so on, but its not a good solution.

[Flexman]

You wouldn't be able to use a TTerrain object for that. Translate, Move or SetPosition doesn't work on it (so even if you could have multiple TTerrains, there's no way to position them for a treadmill system). And then the Vegetation system wouldn't work as that's tied to using a TTerrain object.

 

As soon as you get more than a few thousand units from world origin anyway, polygons begin to wobble. You need to periodically reset the world origin to fix that.

 

Please correct me if I'm wrong, I'd love to be able to do what you'd like to do. Sometimes you have to scale back our wants and work with what you can do. Sometimes less is more and it works out better.

 

Regards,

[VeTaL]

So, you want to say, that there is no way of loading dynamic terrain?

 

Reseting of the world origin looking like not a huge problem.

 

//Please correct me if I'm wrong, I'd love to be able to do what you'd like to do. Sometimes you have to scale back our wants and work with what you can do. Sometimes less is more and it works out better.

You're absolutely right, but as we are making RPG game, there is no sense to make in 2010 year a game without dynamic level loading.

[wh1sp3r]

i said, try it yourself make own terrain, don't use LE terrain.

[VeTaL]

Great

[Josh]

So few games would use this, and even fewer indie games would. STALKER and Crysis don't even need something like this. So my recommendation is to work within the constraints of the existing system to make your game. The only alternatives are to spend months writing an alternative terrain system that won't be supported and won't integrate well with our physics and vegetation, or pay me to spend 6 months working on this.

[VeTaL]

Well... looking like the only solution is to write a terrain loader, that would hold parts of terrain, generate new terrain from 9 parts, load them to memory in the second thread and then just copy it instead of the old one.

What do you thinking about it?

[wh1sp3r]

design a terrain parts in 3D program .. load them, write shader for heightmaps .. done.

 

problem is you can't copy object, only make instance ... = problem

[VeTaL]

Then there would be some problems with physics and trees...

 

But are there any variants, how to solve trouble with copying?

[Rick]

Each of the largest terrains requires about 50 mb of texture data on the GPU.

 

Is that normal or just the way you have it setup now? Just curious as to why it's so much?

[Masterxilo]

I'm wondering why he said it's only that few. I guess it's even much more vram per 4096^2 terrain:

(4096*4096) * (2+3+4)/(1024*1024) = 144 MB

 

The first part is obviously the resolution, the second part of the calculation is the amount of uncompressed raw data (!) per grid point: 2 bytes of height info, 3 bytes normal, 4 bytes color = terrain layer visibility map). There might be even more maps...

[Rick]

Interesting. Either way I don't think paging multiple terrains should really be part of the engine directly. I think the commands could be there, but leave it up to the developer to determine which terrains to have "on" or "off".

 

I've teamed up with a local indie group and working on a project that they picked unity, but with playing around with that it looks like you can piece terrains together in their editor. I haven't messed around enough with it, but it's just like any other object that you can run any commands on like load/unload.

 

I'd leave it up to the developer to determine what's on that terrain and to unload all the other objects as well. Of course I think you would need multithread asset loading then though so there is no noticeable slowdown.

[Josh]

Yeah, actually it is a lot more:

4096 * 4096 * 2 bytes = 33 mb (heightmap)

4096 * 4096 * 4 bytes = 66 mb (normal map)

4096 * 4096 * 4 bytes = 66 mb (alpha map)

 

165 mb for a 4096 x 4096 terrain. Two of them would break most GPUs.

 

Paging smaller terrains would involve a lot of problems. Your terrain disappears from memory, then any physics objects out of range fall through the terrain. So now it's not just terrain that has to be paged, it's all models. Then someone wonders why a target entity suddenly no longer exists when it gets released from memory... You get the idea.

 

Then there's issues with floating point precision. If you go much further from the terrain than what the largest terrain size allows, you will start seeing jittery motion due to floating point imprecision.

 

Then there's vegetation. It's fast to load, but it still takes some time to populate the data structure. And considering I just got it to work without bugs, I am not keen on complicating that system right now.

 

It's certainly possible, but I don't think right now is a time I should be making the internal workings of the engine more complicated.

 

Now maybe if the height data was left in system memory, and just the textures were paged, and the engine was switched over to use 64-bit floats, then we might have something that could be implemented right. So I will consider this for version 3.0. You'd still be limited by other factors like the number of objects you can store in memory, and the GPU memory their textures would take. So I suppose a paging system for static entities would be needed too, and a mechanism to specify a "paged" entity...

[Joh]

Also don't forget that if you create terrain mesh you don't even have clod so you will have a tons of poly on your screen.

If you really need to have more terrain you should write it by code, but as Josh said this will take many time, almost depend in how many time you have and also how good you are.. Doing a bad terrain won't help you tough.

You could create an rpg that wehn you reach the border you unload the map and load anotehr one, i remember that a final fantasy did this.

[VeTaL]

Well, if i'd use smaller one:

1024 * 1024 * 2 = 2 mb

1024 * 1024 * 4 = 4 mb

1024 * 1024 * 4 = 4 mb

 

total it would be only 10 mbs. Now, i'll write editor that would connect terrains by 9, so it would be 90 mbs for large terrain 3000*3000. Now, i can load 2 of them and it would take 180 mbs, wich is not too many as you use 165.

If this would be too much, i can make "parts" smaller to 512 and they would eat 0.5 + 1 + 1 = 2.5 mb.

 

About physics: i dont know if Julio added fucntions for terrain so i can remove from terrains physic some parts, so looking like i need to reload all new terrain.

What about

Your terrain disappears from memory, then any physics objects out of range fall through the terrain.

, i think freezing of physic would help: i freeze all physic for a while, load new terrain and unfreeze it again(but looking like a little lag would take a place).

 

About float precision: well, looking like this may be a large system, that would move all objects to new places, so the center of world would be moved to new place.

 

About vegetation: i didnt look deeper in it, so i can say nothing about it for now.

 

About tons of poly: i'm thinking about loading second terrain in another thread and then just copy it instead of first one.

[Josh]

You can have more than one terrain in a scene and modify the shader to handle the offset you want for the position. You can probably store the position in the entity color, or I can upload the terrain matrix. I didn't write it to support rotation because an extra mat4 multiplication for every terrain vertex seemed unnecessary and wasteful, and terrains have a heck of a lot of vertices.

[VeTaL]

How?

Its noted here ( http://www.leadwerks.com/wiki/index.php?title=Terrain ) that

5 layers are currently supported. There may be only one terrain atm.