Allocating Space slow under Linux: SOLVED

Great work tracking all that down!

Hopefully it will lead to some actionable items for the devs.

I'm not sure if the devs need to REMOVE the FlushViewOfFile call for wine, but I can certainly see an argument for reducing the number of times it is called (for both Windows and Wine... makes the code easier to manage).
It's definitely a Wine bug. I'm trying to get it fixed.
"
It's definitely a Wine bug. I'm trying to get it fixed.


Oh.. you think that the way it's calling msync() is a bug? I thought it was just possibly because of the frequency of the calls, and not specifically a wine BUG.. just a usage issue.

But if you think it is a wine bug, and they can fix it, that's even better. Good luck and keep us posted!
Well, it certainly doesn't take ten hours to complete under Windows! :)

The filesystems are different, but not different enough to justify a factor-of-1000 slowdown. That very strongly indicates a Wine bug (especially considering it runs quickly with MS_ASYNC). I've submitted a patch http://www.winehq.org/pipermail/wine-patches/2013-March/122718.html that makes msync asynchronous, just gotta wait and see if they'll accept it. In the meantime, libeatmydata is a passable workaround.

[edit] Probably not a typo, just a play-it-safe decision that was never revisited.
Last edited by FeepingCreature on Mar 8, 2013, 12:17:39 PM
Such a small typo with such a huge performance cost.

"
+#ifndef MS_ASYNC
+#define MS_ASYNC 1
+#endif

- if (msync( addr, *size_ptr, MS_SYNC )) status = STATUS_NOT_MAPPED_DATA;
+ if (msync( addr, *size_ptr, MS_ASYNC )) status = STATUS_NOT_MAPPED_DATA;


Well done.
Thanks! Let's hope they merge it. I don't know if it's worth it to patch it in PoE before that - on the one hand, it's the project that GGG can control (as opposed to Wine) and it's a really easy, small, contained fix, on the other hand, if the patch gets accepted we should have it fixed in the next Wine version, so it might still not be worth the effort. Up to them, I guess.
Last edited by FeepingCreature on Mar 8, 2013, 4:09:53 PM
Excellent work there! Thanks for finishing the job.

I tried overloading just fsync() myself but as it didn't help, I kind of gave up :)
I can't get this working in Arch yet. It basivally stops at two bars.

Downloaded from Here.

I run "LD_PRELOAD=/usr/lib/libeatmydata.so wine Client.exe", but I get this error:

"
ERROR: ld.so: object '/usr/lib/eatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.


100% the right directory.

I am editing the source code to mirror the patch above as we speak. I will be back with results.

edit: It didn't let me run ./configure because of some issue with wine being 32 bit so used ./configure --enable-win64 instead. It's still super slow. It's been like 4 hours and I've made it 7 bars. Teh fudge...

edit2: Here I am ~14 hours later and I have one bar to go. So that'll probably be another 45 minutes. -_-'

I wasn't able to try it but if someone has the same issue, and the fix didn't speed things up, try installing a 32 bit arch install in your 64 bit OS. Follow this tut:

https://wiki.archlinux.org/index.php/Install_bundled_32-bit_system_in_Arch64

And then try the LD_PRELOAD thing. If that doesn't work. Put the changes in the source code.

Just get the code from the wine GIT repository. The link that was posted earlier shows the difference between the patch he did, and the original code. The "+" means that line of code was added, and the "-" means it was removed. The line numbers and file name/location are also shown in the diff posted above, it's like four lines that need to be added so it's a pretty easy change. Goto the ROOT directory and follow these instruction on this page: www.winehq.org/docs/wineusr-guide/installing-wine-source

I didn't try this, so it's possible you might do all of this stuff, and it still might not fix anything. Hahahaha. If you have a 64-bit OS ./configure will give an error that can be fixed by appending --enable-win64. I say try installing 32 bit arch because I did all the above stuff with a 64-bit OS and it was the same speed.

I hope I didn't ramble too much and someone can decipher what I said. I haven't gone to sleep yet. x.x
Last edited by RedSavage on Mar 23, 2013, 7:19:29 PM
"
RedSavage wrote:
I can't get this working in Arch yet.


I was having a bit similar problems with Gentoo as it is a source based distro as well. I needed to get a 32-bit version of the library and couldn't get that by default at all.

Therefore I made my own. It's simple enough to provide here. Save it as nosync.c.
"

#include <unistd.h>
#include <stdio.h>
#include <sys/mman.h>
#include <dlfcn.h>

static int (*libc_msync)(void *, size_t, int) = 0;

int msync(void *addr, size_t length, int flags)
{
if (libc_msync)
return libc_msync(addr, length, MS_ASYNC); // Convert to MS_ASYNC call always
else
return 0; // Skip call completely if we don't have a valid pointer
}

void __attribute__((constructor)) nosync_init(void)
{
printf("Initializing nosync library.\n");
void *dlptr = dlopen("libc.so.6", RTLD_LAZY | RTLD_LOCAL | RTLD_DEEPBIND);

if (dlptr == 0)
{
printf("Unable to load libc symbols: %s\n", dlerror());
return;
}

// Get msync() pointer
libc_msync = dlsym(dlptr, "msync");

printf("Replacement msync() located at %p\n", msync);
printf("Original msync() located at %p\n", libc_msync);
}


It needs to be compiled into a 32-bit library:
"

gcc -m32 -fPIC -shared -Wl,-soname,libnosync.so -o libnosync.so nosync.c -ldl


That creates a libnosync.so file that you need to preload into Wine. Because of how Wine works (launcher scripts, part of it is 64-bit, some of it is 32), it can be a bit tricky. I found the following script to work. You need to change the paths to correct ones (including PoE one) first and feel free to adjust wine parameters:

"

#!/bin/sh
cd ~/.wine/drive_c/Ohjelmatiedostot/Grinding\ Gear\ Games/Path\ of\ Exile/
wineserver -k
export LD_PRELOAD=${HOME}/path/to/libnosync.so
WINENOPULSE=1 wine Client.exe --softwareaudio $@


There could be simpler solutions. Feel free to improve on it. This works with older Wine 1.4.1 at least. It does complain about ignoring the library for 64-bit parts but printf() calls are there to tell you that it does work for 32-bit parts that are actually important.
Last edited by JukiJulma on Mar 28, 2013, 3:42:08 AM
If your distribution supports multi-arch, you just need to add i386 as one of the architectures, and get it through whatever method is required by your distribution.

In my case on debian, it is in a package called "eatmydata". When I apt-get install eatmydata by default it wants the 64-bit version since that is the version of my OS. Because I added i386 as an architecture, I just have to apt-get install eatmydata:i386 and it installs the 386 version instead.

Report Forum Post

Report Account:

Report Type

Additional Info