1.) What for?
Usually, porting my favorite manga ripper [HakuNeko] would involve slightly more exotic target platforms like [FreeBSD UNIX]. This changed with version 1.4.2 however, as this version – the most current at the time of writing – would no longer compile on Windows machines due to some issues with its build toolchain. And that’s the most common platform for the tool!
This is what the lead developer had to say about the issue while even suggesting the use of [FMD] instead of HakuNeko on Windows:
Normally I wouldn’t mind that much and keep using 1.4.1 for now, but unfortunately this is not an option. Quite a few Manga websites have changed by now, breaking compatibility with the previous version. As this is breaking most of HakuNekos’ functionality for some important sites, it became quite unusable on Windows, leaving Linux as the only officially supported platform.
As using virtual machines or remote VNC/X11 servers for HakuNeko proved to be too tedious for me, I thought I’d try to build this by myself. As the MSYS2/MinGW Windows toolchain seemed to be broken for 1.4.2, I tried – for the very first time – to cross-compile on Linux, choosing CentOS 7.3 x86_64 and MinGW32 4.9.3 for the task. This was quite the challenge, given that HakuNeko comes completely unprepared for cross-compiling.
2.) First, the files
Took me many hours / days to get it done – 100% statically linked too – and it’s finished now! What I won’t provide is an installer (don’t care about that), but here are my v1.4.2 builds for Windows:
- [HakuNeko 1.4.2 for Win32] (MinGW32 4.9.3, statically linked against wxWigets 3.0.2, cURL 7.47.0 and OpenSSL 1.0.2h)
- [HakuNeko 1.4.2 for Win32] (portable version, compiler & linking identical to above)
As of today, those versions have been tested successfully on the following operating systems:
- Windows XP Professional SP3 / POSReady2009
- Windows XP Professional x64 Edition SP2 w. Server 2003 updates
- Windows Server 2003 R2 x64 SP2
- Windows Vista Enterprise x64 SP2
- Windows 7 Professional x64 SP1
- Windows 10 Professional x64 build #1607
Please be aware that not all of the functionality has been tested by me, just a few downloads that wouldn’t have worked with 1.4.1, a few more random downloads here and there, plus chapter-wise .cbz packaging. It’s looking pretty good I think. Here are some sample screen shots as “proof” of it working (click to enlarge):
3.) What has been done to make this work?
3a.) Initial work:
First of all, cross-compiling is a bottomless, hellish pit, a horrible place that you never want to enter unless a.) The build toolchain of the software you wanna compile is very well prepared for it or b.) you really have/want to get your hands on that build or c.) you hate yourself so much you have to hurt yourself or d.) you actually enjoy c.).
The reasons for choosing cross-compiling were that Ronny Wegener had said, that the MSYS2/MinGW32 build would fail on Windows, plus it would require GCC version 5.3 to link with the bundled, pre-built static libraries (OpenSSL, cURL, wxWidgets).
So I thought it would be more likely to work if I were to run my own MinGW setup on Linux, not relying on the bundled stuff but linking against the libraries that come with MinGW32 4.9.3 on my platform of choice – CentOS 7.3 Linux.
One exception was the GUI library wxWidgets 3.0.2 that I had to cross-compile and statically link by myself as well, but luckily, that’s easy despite its size. wxWidgets is one piece of software that does come well-prepared for cross-compiling! In my case, that made it as simple as this (parallel compile with 6 CPUs):
$ ./configure --prefix=/usr/local/i686-w64-mingw32 --host=i686-w64-mingw32 --build=x86_64-linux \ --enable-unicode --with-expat --with-regex --with-opengl --with-libpng --with-libjpeg --with-libtiff \ --with-zlib --with-msw --enable-ole --enable-uxtheme --disable-shared $ make -j6 # make install
3b.) HakuNeko build toolchain / Makefile modifications for cross-compiling:
HakuNeko is much harder, and I don’t even remember half of what I did, but most of it was manually editing the ./Makefile after
$ ./configure --config-mingw32 would have produced something rather broken.
Let’s get to it, file paths are relative to the source root. First, edit the following parts of the ./Makefile (you need to look for them in different places of the file). First, the
PREFIX, should be in the bottom half of the file:
PREFIX = /usr/local/i686-w64-mingw32/
CC and the
CC = i686-w64-mingw32-g++ CFLAGS = -c -Wall -O2 -std=c++11 \ -I/usr/local/i686-w64-mingw32/lib/wx/include/i686-w64-mingw32-msw-unicode-static-3.0 \ -I/usr/local/i686-w64-mingw32/include/wx-3.0 -D_FILE_OFFSET_BITS=64 -D__WXMSW__ -mthreads \ -DCURL_STATICLIB -I/usr/i686-w64-mingw32/sys-root/mingw/include
-DPORTABLE, if you want to build the portable version of HakuNeko.
Then, the Windows resource compiler, controlled by
RC = /usr/bin/i686-w64-mingw32-windres RCFLAGS = -J rc -O coff -F pe-i386 -I/usr/i686-w64-mingw32/sys-root/mingw/include \ -I/usr/local/i686-w64-mingw32/include
And finally, the static linking part, which is the hardest stuff to get done right,
LD = i686-w64-mingw32-g++ LDFLAGS = -s -static -static-libgcc -static-libstdc++ -mwindows -DCURL_STATICLIB LDLIBS = -L/usr/local/i686-w64-mingw32/lib -Wl,--subsystem,windows -mwindows \ -lwx_mswu_xrc-3.0-i686-w64-mingw32 -lwx_mswu_webview-3.0-i686-w64-mingw32 \ -lwx_mswu_qa-3.0-i686-w64-mingw32 -lwx_baseu_net-3.0-i686-w64-mingw32 \ -lwx_mswu_html-3.0-i686-w64-mingw32 -lwx_mswu_adv-3.0-i686-w64-mingw32 \ -lwx_mswu_core-3.0-i686-w64-mingw32 -lwx_baseu_xml-3.0-i686-w64-mingw32 \ -lwx_baseu-3.0-i686-w64-mingw32 -L/usr/i686-w64-mingw32/sys-root/mingw/lib -lcurl -lidn -liconv \ -lssh2 -lssl -lcrypto -lpng -ljpeg -ltiff -lexpat -lwxregexu-3.0-i686-w64-mingw32 -lz -lrpcrt4 \ -lwldap32 -loleaut32 -lole32 -luuid -lws2_32 -lwinspool -lwinmm -lshell32 -lcomctl32 -lcomdlg32 \ -ladvapi32 -lwsock32 -lgdi32
Took a while to find the libraries (and static library order!) necessary to satisfy all the dependencies properly.
If you need it, here is the modified Makefile I’ve used to cross-compile:
- [HakuNeko Makefile] for cross-compiling HakuNeko 1.4.2 for Windows on CentOS 7.3 x86_64 Linux (needs statically linked & installed wxWidgets first).
3c.) Source code modifications:
However, something will still not be quite right, because some of the crypto libraries will provide the MD5 functions
MD5_Update() as well as
MD5_Final(), and those are already defined by HakuNeko itself. This will break the static linking, as redundant definitions won’t work. We’ll rely on the libraries (libcrypto, libssl), and comment the built-in stuff out in src/v7/v7.c:
void MD5_Init(MD5_CTX *c);
void MD5_Update(MD5_CTX *c, const unsigned char *data, size_t len);
void MD5_Final(unsigned char *md, MD5_CTX *c);
/* void MD5_Init(MD5_CTX *c);
* void MD5_Update(MD5_CTX *c, const unsigned char *data, size_t len);
* void MD5_Final(unsigned char *md, MD5_CTX *c);
On top of that, the configure system may have generated src/main.cpp as well as src/main.h. Those are bogus files, turning the entire tool into nothing but one large “Hello World” program. Plus, that’s hard to debug, as the binary won’t even output “Hello World” on a Windows terminal when it’s built as a GUI tool. I only found the issue when testing it with Wine on Linux.
Please delete src/main.cpp and src/main.h before continuing.
Now, if you’re really lucky, you should be able to run something like
$ make -j6 and watch everything work out nicely. Or watch it crash and burn, which is the much, much more likely scenario, given I’ve likely only given you half of what I did to the build tools.
Well, in any case, no need to run
$ make install of course, just grab the binary build/msw/bin/hakuneko.exe and copy it off to some Windows machine, then try to run it. If you’ve built the portable version, you may wish to rename the file to hakuneko-portable.exe, just like the official developers do.
4.) The future
Let’s just hope that the developers of HakuNeko can get this fixed for versions >=1.4.3, because I really, really don’t want to keep doing this. It’s extremely painful, as cross-compiling is exactly the kind of living hell I heard a lot of people saying it is! I think it’s a miracle I managed to compile and run it at all, and it was so frustrating and tedious for somebody like me (who isn’t a developer).
The statement that this took “hours / days” wasn’t an exaggeration. I think it was something like 10-12 man hours of pure frustration to get it done. I mean, it does feel pretty nice when it finally works, but I wouldn’t bet on myself being able to do this again for future versions…
So please, make it work on Windows again, if possible, and keep HakuNeko cross-platform! It’s still my favorite tool for the task! Thanks!