Jump to content

Upsampling Textures with AI

Josh

184 views

I did a little experiment with FPS Creator Pack #75 by upsampling the images with Gigapixel, which uses deep learning to upsample images and infer details that don't appear in the original pixels. The AI neural network does a pretty impressive job, generating results that are look better than a simple sharpen filter: I doubled the size of the textures to 1024x1024. Then I generated normal maps from the high-res images using AMD's TGA to DOT3 tool, and saved the normal maps with BC5 DDS compression. The diffuse textures were saved with BC7 DDS compression. The images below are using a 4x magnification to demonstrate the difference.

bicubic.thumb.png.7232bf7d8017ff6d12efcf8466ae5dd3.png

ai.thumb.png.89d32653a172cfd0ba353b2324e7b620.png

As you can see, the image that is upsampled with deep learning looks normal and the resized image looks like there is butter on the lens! It's hard to believe the above images came from a 256x128 section of an image.

The workflow was pretty tedious, as I had to convert images to TGA, then to uncompressed or BC5 DDS, and then to BC7 in Visual Studio. Each BC7 texture took maybe 5-10 minutes to compress! So while this set represents the optimum quality for 2019 game tech, and the format for assets we want to use in LE5, the workflow has a lot of room for improvement.

You can download the full package here:

FPSCPack75TexturesHD.zip

  • Like 1


6 Comments


Recommended Comments

It still has a long ways to go but it's definitely an improvement of just resizing up.  I would encourage you to get a true, crisp, 2048x2048 (or any dimension large image), downsize it, then use the downsized version to scale up.  Then compare your results to the original 2048x2048 texture.

Share this comment


Link to comment

Curious how it would do with this grass texture:

grass512.png.eec816a2579fc310912ea2628477b843.png

This is the original (from here)

grass2048.thumb.png.65a2fa19a17436b6c97580f8df3c33e5.png

Of course, the next natural question is how it does with upsizing seamless textures.  In other words, does it introduce seams?

Share this comment


Link to comment
3 hours ago, gamecreator said:

Curious how it would do with this grass texture:

It did not do very well with that one. Left to right, low-res, AI upsample, high-res:
1817611748_grasscompare.thumb.png.7f2caca59a751d4a7d5171f66918896a.png

  • Like 1

Share this comment


Link to comment

I think this is the situation with everything it does.  It does an OK job estimating (and much better than just upscaling) but it can't make up for all missing details.  The arch and your original image were blurry too; obviously the original images would be much more crisp and detailed.  That said, this might still be helpful if you already have a 1024x1024 image and you want to make it 2048x2048, or from there to 4092.  I suspect the more detail it already has to work with, the better it'll do.

Share this comment


Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Blog Entries

    • By Josh in Josh's Dev Blog 4
      Here are some screenshots showing more complex interface items scaled at different resolutions. First, here is the interface at 100% scaling:

      And here is the same interface at the same screen resolution, with the DPI scaling turned up to 150%:

      The code to control this is sort of complex, and I don't care. GUI resolution independence is a complicated thing, so the goal should be to create a system that does what it is supposed to do reliably, not to make complicated things simpler at the expense of functionality.
      function widget:Draw(x,y,width,height) local scale = self.gui:GetScale() self.primitives[1].size = iVec2(self.size.x, self.size.y - self.tabsize.y * scale) self.primitives[2].size = iVec2(self.size.x, self.size.y - self.tabsize.y * scale) --Tabs local n local tabpos = 0 for n = 1, #self.items do local tw = self:TabWidth(n) * scale if n * 3 > #self.primitives - 2 then self:AddRect(iVec2(tabpos,0), iVec2(tw, self.tabsize.y * scale), self.bordercolor, false, self.itemcornerradius * scale) self:AddRect(iVec2(tabpos+1,1), iVec2(tw, self.tabsize.y * scale) - iVec2(2 * scale,-1 * scale), self.backgroundcolor, false, self.itemcornerradius * scale) self:AddTextRect(self.items[n].text, iVec2(tabpos,0), iVec2(tw, self.tabsize.y*scale), self.textcolor, TEXT_CENTER + TEXT_MIDDLE) end if self:SelectedItem() == n then self.primitives[2 + (n - 1) * 3 + 1].position = iVec2(tabpos, 0) self.primitives[2 + (n - 1) * 3 + 1].size = iVec2(tw, self.tabsize.y * scale) + iVec2(0,2) self.primitives[2 + (n - 1) * 3 + 2].position = iVec2(tabpos + 1, 1) self.primitives[2 + (n - 1) * 3 + 2].color = self.selectedtabcolor self.primitives[2 + (n - 1) * 3 + 2].size = iVec2(tw, self.tabsize.y * scale) - iVec2(2,-1) self.primitives[2 + (n - 1) * 3 + 3].color = self.hoveredtextcolor self.primitives[2 + (n - 1) * 3 + 1].position = iVec2(tabpos,0) self.primitives[2 + (n - 1) * 3 + 2].position = iVec2(tabpos + 1, 1) self.primitives[2 + (n - 1) * 3 + 3].position = iVec2(tabpos,0) else self.primitives[2 + (n - 1) * 3 + 1].size = iVec2(tw, self.tabsize.y * scale) self.primitives[2 + (n - 1) * 3 + 2].color = self.tabcolor self.primitives[2 + (n - 1) * 3 + 2].size = iVec2(tw, self.tabsize.y * scale) - iVec2(2,2) if n == self.hovereditem then self.primitives[2 + (n - 1) * 3 + 3].color = self.hoveredtextcolor else self.primitives[2 + (n - 1) * 3 + 3].color = self.textcolor end self.primitives[2 + (n - 1) * 3 + 1].position = iVec2(tabpos,2) self.primitives[2 + (n - 1) * 3 + 2].position = iVec2(tabpos + 1, 3) self.primitives[2 + (n - 1) * 3 + 3].position = iVec2(tabpos,2) end self.primitives[2 + (n - 1) * 3 + 3].text = self.items[n].text tabpos = tabpos + tw - 2 end end  
    • By 💎Yue💎 in Dev Log 5
      The prototype of a four-wheeled vehicle is completed, where the third person player can get on and off the vehicle by pressing the E key.  To move the vehicle either forward or backward, is done with the keys W, and the key S, to brake with the space key.  And the principle is the same as when driving the character, a third person camera goes behind the car orbiting 360 degrees.

      I don't think the vehicle is that bad, but I'm absolutely sure it can be improved.  The idea is that this explorer works with batteries, which eventually run out during the night when there is no sunlight.
      Translated with www.DeepL.com/Translator
       
      Mechanics of the game.
      I'm going to focus on the mechanics of the game, establish starting point (Landing area), after the orbiter accident on Mars where all your companions died, now, to survive, you will have to repair your suit, oxygen runs out, good luck.  This involves replacing the oxygen condenser that is failing and the suit is stuck.

      On the ground and performance.
      The rocks, the terrain and the vehicle kill the SPF, but there is a solution, and everything is related to the chassis of the vehicle. That is to say that if I put a simple collision bucket for the vehicle, the yield recovers, something that does not happen if I put a collider of precise calculation for the car. This has the advantage of better performance but is not very accurate, especially when the car crashes with an object in front, because the horn of the car has no collision. And the solution to this, is to put a sliding joint, as was done with the area in which the player climbs the car and descends from it.


       
      On the rocks, I am trying to make them with the slightest polygons and the most distant from each other. 
      Obviously on Mars I can not create canyons, high mountains, is because the terrain does not produce shadows on itself, that's why the terrain tries to be as flat as possible, simulating a desert with dunes. 

      That's all for now.
       
    • By 💎Yue💎 in Dev Log 9
      The prototype is finished, and the mechanics of the game can be given way.  It has established a desert terrain in the form of dunes, this implies that there are no cannons or anything similar, because Leadwerks does not allow a terrain to cast shadows on that same terrain and this looks visually rare.
      So the terrain is like low-slope dunes. On the other hand, I think the texture of the terrain is already the definitive one, with the possibility of changes and suggestions on the part of those involved in this project.
      On the other hand we have taken the model of a habitat of the nasa, which certainly looks very nice. 
      The next steps, are to establish the starting point of the player, this must start near the capsule return to Mars somewhere on the map of 2024 x 2.
      And think about the first thing you should do, repair your suit? Seek a shelter? things like that.  


×
×
  • Create New...