GGPK Defragmenter

We actually have a similar tool for this internally, but our one is an in-place defragment rather than input to output.

Old pack files can get quite fragmented after a lot of game patches. This is especially noticed internally where our developers are patching multiple times per day.

I should mention that the the patcher does reuse free space for deleted files, it's just that there are a lot of oddly sized small chunks left over after filling up available free chunks with new data.

This is exacerbated by the patcher trying to do atomic operations so that it can be closed at any time without causing corruption or loss of already downloaded assets. This means that if we add a file to a directory, for example, we have to make a new directory entry with the file added, commit that to disk, then swap the parent directory entry pointer over to the new one, then commit that to disk.

As files are added or removed it can involve directory entries moving around a lot throughout the pack file, adding to the fragmentation.

There are some improvements I'd like to do one day to prevent quite so much fragmentation, especially of the metadata, but in the mean time, feel free to use this tool.

EDIT: By the way, that code is very clean. Great job!
Path of Exile II - Game Director
Last edited by Jonathan on Apr 15, 2012, 7:11:18 PM
"
maximilious wrote:
it looks for the stored password and hacks your account.


It is always a good idea to be suspicious of random executables you download from the internet, so I can't really blame you. In this case though, Zoxc has been kind enough to give us the code for his tool.

If you are worried about using the pre-compiled executable version, I advise that you run it from source code directly (which is what I did).

You can do this by installing a ruby interpretor and installing the "bindata" package with "gem install bindata".
Path of Exile II - Game Director
"
tpapp157 wrote:
"
Zoxc wrote:
What does GGPK stand for? :)

My guess is something like Grinding Gears PacKage file.


Good Game, Player Killer.
Lead Developer. Follow us on: Twitter | YouTube | Facebook | Contact Support if you need help!
"
tpapp157 wrote:

My guess is something like Grinding Gears PacKage file.


Correct.

"

The hashes are murmurhash2 on a lowercase'd copy of the entry's name, i think


The hashes for each filename entry in a directory is a lowercase Murmur2 hash. This is used to do fast traversal when looking up a file by path.

File content hashes are SHA-256, and directory content hashes are SHA-256 of the concatenated SHA-256 hashes of the children.

As I hinted above, the FREE nodes are left over when old structures are deleted (most commonly when they are resized or recreated when adding to the pack). Each free node contains the offset to the next free node in a free node chain.
Path of Exile II - Game Director
Last edited by Jonathan on Apr 16, 2012, 12:49:37 AM
"
Zoxc wrote:
Uploaded a new version with larger FREE nodes. You can run it again to possibly prevent problems with the client.


The client should recover from that. It should be fairly resilient. I ran it on a local dev copy and then patched straight after and it worked just fine.
Path of Exile II - Game Director
"
Bipolartuna wrote:
do I delete the old file? Just did this and my folder went from 7gb to 10gb =/

Followed this instructions

"
radiatoren wrote:
1. You need to run cmd.exe

2. Go to the location of your client.exe in the command prompt by using cd.. and cd <foldername> as commands.

3. Write "ggpk_defragment.exe content.ggpk content2.ggpk"

The program will take several minutes to finish. After it has finished it will tell you.

4. Go to the folder of your client.exe.

5. Change name of content.ggpk to contentold.ggpk

6. Change name of content2.ggpk to content.ggpk

Then you have done it right! :)

ps. I assume you have moved ggpk_defragment.exe into the path of exile folder, I assume that Client.exe and content.ggpk is in the same folder and that folder is the path of exile folder - anything else would seem illogical.


If you look in your game folder, you should have two content files, one with 'old' in them etc.
After you rename the files, test the game to make sure its working, then delete or move the other/backup/old file to a different directory/hdd.
"
briant023 wrote:
Sirs,

i have a problem that checking resources stack. Is this alaso a solution to the problem?


Do you mean Stuck?

If so, ensure you're running the game with Administrator privileges and that you don't have any security software blocking the patch, as it needs write access to your HDD.

This tool is more for people wanting to save space and improve load time performance.
"
Can we delete the old content file?


I would only recommend deleting it after you have confirmed your new/compressed copy is working correctly.

But yes, it can be deleted afterwards.
(some people may want to keep it as a backup in a different directory/hdd, in case they run into issues in the future etc)
"
ftw wrote:
should you not use this defrag on an ssd?



This tool defrags the content file itself, which is an archive and can become fragmented after patches, so yes, defragging it on a SSD is ok/safe and will offer slight space savings along with slight improvements to load times.

"
briant023 wrote:
"
Henry_GGG wrote:
"
briant023 wrote:
Sirs,

i have a problem that checking resources stack. Is this alaso a solution to the problem?


Do you mean Stuck?

If so, ensure you're running the game with Administrator privileges and that you don't have any security software blocking the patch, as it needs write access to your HDD.

This tool is more for people wanting to save space and improve load time performance.




already disable my Av and firewall sir. Still 30mins waiting on checking resources


Any change?
If not, you may want to restart your PC.
"
ignarsoll wrote:
I can't see a reason why would anybody want to run it multiple times without checking first if the game works fine. There shouldn't be a problem if the user complies with the instructions.


I think I just heard the collective gasp of all QA testers around the world.

Report Forum Post

Report Account:

Report Type

Additional Info