Jump to content

best way to transfer big files?


AnniXa
 Share

Recommended Posts

Hey, for my actual game i made a lobby that is syncing the folder with the stuff for the game session,

it syncing some .ini files, some .lua files and some very simple mesh files some shader.verts and a .mat file. it checks for MD5 sums and only syncs the neccesary files and stuff.

 

This works very well and fast at all, but now i wanted to make modders able to also change textures for a game mod, and i wanted to let it sync the textures also, but that .dds files are very big, and a 4mb file takes alot of time to transfer for any reason.

 

I use raknet for my network stuff and i write my own file transfer stuff since the raknet internal thingie was not working for me.

 

Actually i send a Reliable pack with file data to the client and it sends a reliable answer, every pack have a maximal packet size of ~1000, so this will be ~4000 packets that need to be sendet for a 4mb DDS file.

 

does anyone got experiences with such stuff?

Should i just spam all the packets without waiting for a reply from the client? but then there could be like 4000 packets in the buffer or something

 

~edit ahw when i start writing forum posts i mostly get the answers...

 

i had the update of the server sided synchronisation stuff setted to have a delay of 1000ms, so only 1 packet per second was send...

 

now it takes just seconds to transfer such files

 

but maybe someone knows some bether solutions to this?

Whuts wroong?

Link to comment
Share on other sites

For better performance you could use the zlib library to compress the textures and decompress them later.(The server has a kind of cache folder with the compressed new textures and the client only download them and decompress (decompressing with zlib is very fast.I had around 30MB/s)

Link to comment
Share on other sites

For better performance you could use the zlib library to compress the textures and decompress them later.(The server has a kind of cache folder with the compressed new textures and the client only download them and decompress (decompressing with zlib is very fast.I had around 30MB/s)

 

Actually i only compress the map chunk data, i thought dds would be compressed internally in any kind and so it would not bring alot?

 

But thats a good idea to compress all files before transfer, i will do it.

i wonder why i not thougt about that?^^

Whuts wroong?

Link to comment
Share on other sites

There are 2 raknet plugins which may interest your:

- http://www.jenkinssoftware.com/raknet/manual/directorydeltatransfer.html

- http://www.jenkinssoftware.com/raknet/manual/filelisttransfer.html

 

Never used them, so feedback if you do would be great! If you use C++ on Windows only I propose:

http://www.codeproject.com/Articles/7530/Zip-Utils-clean-elegant-simple-C-Win32

Link to comment
Share on other sites

I found today xz compressor, which beats even 7-zip ultra lzma2 mode, in both compression ratio and speed.

I found it when I was wondering with what those latest GNU coreutils (because I needed a SHA512 calculator) were packed with, as usually GNU things are packed with .tar.gz.

7-Zip can still read xz files, because they are still LZMA2 format, but just optimized very nicely.

Even in fast mode, xz packs very good, which is good for online networking.

I guess xz will get a standard for GNU things, because those guys only care about efficiency.

Ryzen 9 RX 6800M ■ 16GB XF8 Windows 11 ■
Ultra ■ LE 2.53DWS 5.6  Reaper ■ C/C++ C# ■ Fortran 2008 ■ Story ■
■ Homepage: https://canardia.com ■

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...