Jump to content

Load Package not keeping folder hierarchy


SpiderPig
 Share

Recommended Posts

Hi guys,  I'm loading multiple packages from within sub folders and it although they are loading correctly and being password verified I cant load files from them.

The package loads from a folder in my game directory;

Loaded package!  C:/Users/Matt/Desktop/The Seventh World\GameData\Icons\Icons.zip Verified!

Then I load an icon which fails;

Error: Failed to load texture "C:/Users/Matt/Desktop/The Seventh World/GameData/Icons/Bark.tex"

But this works;

Loading texture "C:/Users/Matt/Desktop/The Seventh World/Bark.tex"

It seems loading a package doesn't register itself with it's own directory but to that of the game directory?  @Joshis this a bug?

Link to comment
Share on other sites

I'm sure I'm not explaining it right.  All my assets are in sub folders as per usual.  When I want to publish the game i package up a group of files in each of the folders so when steam users download the update it's not massive.  Loading each package works well, but I then cant load assets from the package because the file path to that asset has been changed.  Instead of being this;

C:/Users/Matt/Desktop/The Seventh World/GameData/Icons/Bark.tex

it needs to be this in order to work;

C:/Users/Matt/Desktop/The Seventh World/Bark.tex

The zip package is in;

GameData/Icons/ICONS.zip

"GameData/Icons/" has to be removed in order for the texture to be loaded.

 

Hope that's better.  Otherwise I'll put an example together.

Link to comment
Share on other sites

Oh, I think I see. The file system considers the zip file to be the same as the root game folder. Really I think all your zip packages should be in the base game directory.

My job is to make tools you love, with the features you want, and performance you can't live without.

Link to comment
Share on other sites

Yeah that's it.  I guess if it's not a bug, I could keep them in the base directory, or I could just change the path to them once the game is published.  It'll just be a pain to change all the file paths.  It's a shame really that many packages are needed for steam to keep the update sizes down.

Link to comment
Share on other sites

I haven't found the Steam-works docs yet but a google search showed this;

Quote

Each and every game on Steam is considered by Steam to be made up of 1MB blocks. When a developer prepares to upload a new version of a game to Steam, the Steam tools scan the game's files, and works out how to make that new version of the game out of the data blocks it already knows about, and uploads any new data blocks that it finds, along with a manifest which specifies how to build that game version out of the blocks.

When you download an update, what the Steam client does is downloads the manifest for the new version, then begins to build the new version of the game in a separate "downloading" folder in the "SteamApps" folder. It does this by comparing the new manifest with the old manifest for the version of the game you already have downloaded. It works out which data blocks it can copy across from the existing install (by using the old manifest to look up where in the existing files a particular block can be found), and which it has to download.

Once the new version of the game is fully built by copying and downloading, Steam deletes the old version of the game, and replaces it with the new version.

I think with encryption and/or compression, more than 1MB of data can change meaning the entire zip archive could be downloaded again for only a small file change.

Link to comment
Share on other sites

Here we go; https://partner.steamgames.com/doc/sdk/uploading#Building_Efficient_Depots

Quote

Building Efficient Depots for SteamPipe

The old Steam content system would patch updates on a file level, which meant that if a single byte in a file changed, the entire new file would be downloaded by all users. This was especially inefficient if the game used pack files, which are collections of game content files in a single big file. Pack files can easily exceed 1 GB, so updates often led to unnecessarily large downloads. A common way to avoid these large downloads was to add new pack files that overrode content of already shipped pack files. That worked for updates, but it hurt new users long-term, since they ended up downloading unused, already-patched content.

The new content system fixes this problem by splitting each file into roughly 1-MB chunks. Each chunk is then compressed and encrypted before being distributed by the Steam content system. If the game content has large redundant parts, these chunks are reused and the user only has to download each repeated chunk once. However, the real strength of this system is building efficient update patches. While the system is building a patch, the new content is scanned for already known chunks. If it finds them, it reuses them. This means if you change or inject a few bytes in a big file, the user only has to download the changes.

This works well in most cases, but there are still a few pitfalls that need to be avoided when designing the content layout of a game. Do not compress or encrypt your game data. This is already done for in-flight downloads and retail discs by the Steam content system. If you do it too, you will reduce the effectiveness of delta patching. If you package multiple data files in a single pack file, make sure that with each re-packaging, no unnecessary changes are made. One problematic practice is including the full name of the original source files on disk, because the names may change, depending on the build machine. Another bad pattern is including build time stamps for each file. If possible, always add new content to the end of your pack files and keep the order of existing files. Also, keep your pack file’s metadata (offset and sizes to individual files) in one place and don’t intersperse it with content files. Use a binary diff’ing tool like BeyondCompare to look at two builds of your pack files to make sure that hundreds of unwanted changes don’t show up.

If you follow these rules you will minimize patch sizes and only new content will need to be downloaded. Your customers will thank you for that and you will be able to increase the quality of your product by shipping more updates.

 

  • Upvote 1
Link to comment
Share on other sites

Hmm okay.  From what I'm thinking, all of my packages will be around the 10 to 30 MB range.  That way the updates should be small unless I add heaps.  I just didn't like the idea of adding 100+ packages to the main game directory.  I think what I'll do is just re-add the hierarchy into each zip.  That way I won't have to change file paths and can keep them in any sub folder.

Link to comment
Share on other sites

To be honest, the end user should decide how they want their games to be broken up. This way a hot fix doesn't mean the entire game to be redownloaded if the developer made a pack for scripts and another for materials and models and so on.

  • Upvote 1

Cyclone - Ultra Game System - Component PreprocessorTex2TGA - Darkness Awaits Template (Leadwerks)

If you like my work, consider supporting me on Patreon!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...