Memory Increase on Map change

[Andy90]

Hello, when you change a map the usage memory increase but never decrease. This means there is something allive what should not be allive. You can reproduce it with this code. Just hit multiple times T and see how the memory increases every time.

 

#include "UltraEngine.h"
#include "ComponentSystem.h"
//#include "Steamworks/Steamworks.h"

using namespace UltraEngine;

int main(int argc, const char* argv[])
{
    
#ifdef STEAM_API_H
    if (not Steamworks::Initialize())
    {
        RuntimeError("Steamworks failed to initialize.");
        return 1;
    }
#endif

    RegisterComponents();

    auto cl = ParseCommandLine(argc, argv);
    
    //Load FreeImage plugin (optional)
    auto fiplugin = LoadPlugin("Plugins/FITextureLoader");

    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a world
    auto world = CreateWorld();

    //Load the map
    WString mapname = "Maps/start.ultra";
    if (cl["map"].is_string()) mapname = std::string(cl["map"]);
    auto scene = LoadMap(world, mapname);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        world->Update();
        world->Render(framebuffer);

#ifdef STEAM_API_H
        Steamworks::Update();
#endif

        if (window->KeyHit(KEY_T)) {
            scene = LoadMap(world, mapname);
        }

    }

#ifdef STEAM_API_H
    Steamworks::Shutdown();
#endif

    return 0;
}

 

[Josh]

I am guessing this is probably a circular reference in one of my components but will need to test more to find out...

[Josh]

bump

[Andy90]

Just want to push this again.

[Josh]

I am done for the day, but I did solve several memory leaks in another issue...will examine this one soon.

[Josh]

In my first test, this does produce a memleak. This is my testing code.

Next I will test simpler scenes and see what is the cause.

#include "Leadwerks.h"
#include "ComponentSystem.h"

using namespace UltraEngine;

int main(int argc, const char* argv[])
{
    RegisterComponents();

    auto cl = ParseCommandLine(argc, argv);

    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a world
    auto world = CreateWorld();

    //Load the map
    WString mapname = "Maps/memtest.map";
    if (cl["map"].is_string()) mapname = std::string(cl["map"]);
    auto scene = LoadMap(world, mapname);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        world->Update();
        world->Render(framebuffer);

        window->SetText(GetMemoryUsage(true));

        if (window->KeyHit(KEY_T)) {
            scene = LoadMap(world, mapname);
        }

        while (PeekEvent()) WaitEvent();// clear the events out
    }
    return 0;
}

 

[Josh]

Interesting...even an empty map with no entities in it causes the memory usage to permanently increase by about 10,000 bytes each load...

[Josh]

Okay, so when I set the sun light color to black / 0, which causes the scene to not contain any directional light, the memory usage becomes stable. So we know directional lights are one cause of a mem leak.

memtest.zip

[Andy90]

this is very intresting. So the leak could be related to lights ?

 

[Josh]

It would not surprise me.

A directional light isn't just a light. A directional light is a "virtual" light with no shadow, plus four extra box lights for the shadow stages, for each camera that uses lighting. So I can imagine something not getting cleaned up here very easily.

There could be additional causes, but I think I will focus on this aspect of it first. Working on this now.

 

[Josh]

Creating and deleting a directional light alone is enough to produce a memory leak.

#include "Leadwerks.h"

using namespace UltraEngine;

int main(int argc, const char* argv[])
{
    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a world
    auto world = CreateWorld();

    auto camera = CreateCamera(world);
    camera->SetClearColor(0.125);
    camera->Move(0, 4, -50);

    //Create light
    auto light = CreateDirectionalLight(world);
    light->SetRotation(45, 35, 0);
    light->SetColor(2);

    auto ground = CreateBox(world, 100, 0.25, 100);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        while (PeekEvent()) WaitEvent();

        window->SetText(GetMemoryUsage(true));

        if (window->KeyHit(KEY_SPACE))
        {
            if (light)
            {
                light = NULL;
            }
            else
            {
                light = CreateDirectionalLight(world);
                light->SetRotation(45, 35, 0);
                light->SetColor(2);
            }
        }

        world->Update();
        world->Render(framebuffer);
    }
    return 0;
}

 

[Josh]

Looks like just the shadow map with a spotlight is enough to cause a memory leak. With shadows disabled the memory usage stays flat.

#include "Leadwerks.h"

using namespace UltraEngine;

int main(int argc, const char* argv[])
{
    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a world
    auto world = CreateWorld();

    auto camera = CreateCamera(world);
    camera->SetClearColor(0.125);
    camera->Move(0, 4, -5);

    //Create light
    shared_ptr<Light> light;

    uint64_t tm = Millisecs();

    auto ground = CreateBox(world, 100, 0.25, 100);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        while (PeekEvent()) WaitEvent();

        window->SetText(GetMemoryUsage(true));

        auto now = Millisecs();
        if (window->KeyDown(KEY_SPACE) and now - tm > 100)
        {
            tm = now;
            if (light)
            {
                light = NULL;
            }
            else
            {
                light = CreateSpotLight(world);
                light->SetRotation(90, 0, 0);
                //light->SetShadows(false);
                light->SetColor(2);
                light->SetPosition(0, 2, 0);
            }
        }

        world->Update();
        world->Render(framebuffer);
    }
    return 0;
}

 

[Josh]

Okay, I found a major leak.

A while ago I implemented dynamically generated samplers based on the current texture settings. This is what allows textures to dynamically change their filter and clamp settings. However, there was a detail of texture cleanup I was missing, and that made it so textures could stay in memory even after everything appeared to be deleted.

I don't know if that is the total solution, but it is a significant find.

[Josh]

I'll have an update for everyone tomorrow morning. It's 5 PM here and I don't like to post updates towards the end of the day.