Jump to content

Basis Universal Texture Support

Josh

524 views

Last year Google and Binomial LLC partnered to release the Basic Universal library as open-source. This library is the successor to Crunch. Both these libraries are like OGG compression for textures. They compress data very well into small file sizes, but once loaded the data takes the same space in memory as it normally does. The benefit is that it can reduce the size of your game files. Crunch only supports DXT compression, but the newer Basis library supports modern compression formats like BC5, which gives artifact-free compressed normal maps, and BC7, which uses the same amount of space as DXT5 textures but pretty much eliminates blocky DXT artifacts on color images.

I've added a Basis plugin to the Plugin SDK on Github so you can now load and save .basis texture files in the Leadwerks Game Engine 5 beta. Testing with a large 4096x2048 image in a variety of formats, here are my results.

  • TGA (uncompressed): 24 MB
  • PNG (lossless compression): 6.7 MB
  • DDS (DXT1 compression): 5.33 MB
  • Zipped DDS: 2.84 MB
  • JPEG: 1.53 MB
  • BASIS: 573 KB

basis.jpg.1ed4e98077720193af1bf0f020bde8b0.jpg

The zipped DXT option is the most like what you are using now in Leadwerks Game Engine 4. Since the overwhelming majority of your game size comes from texture files, we can extrapolate that Basis textures can reduce your game's data size to as little as 20% what it is now.

With all the new image IO features, I wanted to write a script to convert texture files out of the Leadwerks 4 .tex file format and into something more transparent. If you drop the script below into the "Scripts/Start" folder it will automatically detect and convert .tex files into .dds:

function BatchConvertTextures(path, in_ext, out_ext, recursive)
    local dir = LoadDir(path)
    for n = 1, #dir do
        local ftype = FileType(path.."/"..dir[n])
        if ftype == 1 then
            if ExtractExt(dir[n]) == in_ext then
                local savefile = path.."/"..StripExt(dir[n]).."."..out_ext
                if FileTime(path.."/"..dir[n]) > FileTime(savefile) then
                    local pixmap = LoadPixmap(path.."/"..dir[n])
                    if pixmap ~= nil then
                        pixmap:Save(savefile)
                    end
                end
            end
        elseif ftype == 2 and recursive == true then
            BatchConvertTextures(path.."/"..dir[n], in_ext, out_ext, true)
        end
    end
end

BatchConvertTextures(".", "tex", "dds", true)

Here are my freightyard materials, out of the opaque .tex format and back into something I can easily view in Windows Explorer:

Untitled.jpg.be6bce9794a4a7355f9cb4a8aaddc8f1.jpg

These files are all using the original raw pixels as the .tex files, which internally are very similar to DDS. In order to convert them to .basis format, we need to get them into RGBA format, which means we need a DXT decompression routine. I found one and quickly integrated it, then revised my script to convert the loaded pixmaps to uncompressed RGBA format and save as PNG files.

--Make sure FreeImage plugin is loaded
if Plugin_FITextureLoader == nil then
    Plugin_FITextureLoader = LoadPlugin("Plugins/FITextureLoader.dll")
end

function BatchConvertTextures(path, in_ext, out_ext, recursive, format)
    local dir = LoadDir(path)
    for n = 1, #dir do
        local ftype = FileType(path.."/"..dir[n])
        if ftype == 1 then
            if ExtractExt(dir[n]) == in_ext then
                local savefile = path.."/"..StripExt(dir[n]).."."..out_ext
                if FileTime(path.."/"..dir[n]) > FileTime(savefile) then

                    --Load pixmap
                    local pixmap = LoadPixmap(path.."/"..dir[n])

                    --Convert to desired format, if specified
                    if format ~= nil then
                        if pixmap ~= nil then
                            if pixmap.format ~= format then
                                pixmap = pixmap:Convert(format)
                            end
                        end
                    end

                    --Save
                    if pixmap ~= nil then
                        pixmap:Save(savefile)
                    end

                end
            end
        elseif ftype == 2 and recursive == true then
            BatchConvertTextures(path.."/"..dir[n], in_ext, out_ext, true)
        end
    end
end

BatchConvertTextures("Materials/Freightyard", "tex", "png", true, TEXTURE_RGBA)

Cool! Now I have all my .tex files back out as PNGs I can open and edit in any paint program. If you have .tex files that you have lost the source images for, there is now a way to convert them back.

PNG.jpg.88df2fde3ed60e5b7d5785d25d7bd2c6.jpg

Now I am going to try converting this folder of .tex files into super-compressed .basis files. First I will add a line of code to make sure the Basis plugin has been loaded:

--Make sure Basis plugin is loaded
if Plugin_Basis == nil then
    Plugin_Basis = LoadPlugin("Plugins/Basis.dll")
end

And then I can simply change the save file extension to "basis" and it should work:

BatchConvertTextures("Materials/Freightyard", "tex", "basis", true, TEXTURE_RGBA)

And here it is! Notice the file size of the selected image.

basisu.jpg.d876352491ac01dab6e75b1735e0097f.jpg

If you want to view basis textures in Windows explorer there is a handy-dandy thumbnail previewer available.

Now let's see what the final file sizes are for this texture set.

  • Uncompressed TEX files (DXT only): 35.2 MB
  • Zipped TEX files: 25.8 MB
  • BASIS: 7.87 MB 😲

When converted to Basis files the same textures take just 30% the size of the zipped package, and 22% the size of the extracted TEX files. That makes Basis Universal definitely worth using!

  • Like 3


4 Comments


Recommended Comments

Neat.  Of course the next question is: how does the loading time of 100 textures of the old format compare to this one?

Share this comment


Link to comment

Decompression is fast, but I have not gathered any numbers. A fair test would be a zip compressed DDS vs. an unzipped Basis file. (Even if you store a Basis file in a zip package there is no point to adding zip compression because it won't make it any smaller.)

  • Like 1

Share this comment


Link to comment

The pink normal maps were bothering me so I made it so when the engine encounters a DXT5n image (these will only come from LE4 .tex files) it converts it to RGBA and swizzles the red/alpha channel. So you can say goodbye to pink normal maps forever, and I can remove the swizzle functionality in the Vulkan texture class. You should use BC5 for compressed normal maps.

dxt5n.thumb.jpg.a9e8ec0ea566ddc9f47359f2922dc79e.jpg

Share this comment


Link to comment

This is awesome. Please let us know if you encounter any problems with Basis Universal, and we will do whatever it takes to fix them.

Note that UASTC texture support has just been added to the BasisU repo. These files are much bigger than the ETC1S files you've been using, but they are *significantly* higher quality.

-Richard Geldreich

Binomial LLC

  • Thanks 1

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...