Jul 072017
 

Nekopara Vol.3 logo1. Introduction

Of course I would never play something like Nekopara *cough*, so this is just a post describing a technical solution to a compatibility problem! Ok?! Good.

Yeah, it’s another one of those “something broke on XP / XP x64, so let’s fix it” articles. I’ve already been pla…  eh.. investigating Nekopara Volumes 0, 1 and 2, and while the developer claims it needs Windows Vista or higher, those titles worked just fine on XP and XP x64. The final Volume 3 however broke.

I wondered why, given they’re all pretty similar, so I started unpacking the .exe files, looking for information. What I found in the meta data was that Vol.0-2 have been using the TVP(Kirikiri) or maybe the forked [Kirikiri Z] game scripting engine, whereas Vol.3 swapped that for the [Ares CatSystem2] engine, for whatever reason. My assumption would be, that the CatSystem2 thingy was actually built for Vista+ for real, thus breaking XP compatibility. Plus, some other minor components are broken as well (some installers, patches, etc., just like the older volumes).

Now, I’ve already been talking to a guy called UncleVasya / Oleg Ovcharenko, who built a [stub DLL solution] for games based on the Clausewitz Engine (Europa Universalis 4, Hearts of Iron 4, Crusader Kings 2 and finally Stellaris), making it work on XP. It’s pretty similar to the XCOM hacks[1][2]. So I asked him about this one as well, and with quite some work and some additional (important) hints from him regarding the Steam version, I managed to make it run!

So, first things first: Thanks Oleg, you’re doing great work! :)

I will now show you how to make this visual novel / game work on XP x64 and XP, both for the slightly trickier Steam version (whether you choose to play the censored or the uncensored version doesn’t matter, the corresponding patch will be discussed as well), as well as the normal version.

Note: All screenshots in this post are 8-bit (256 color) PNG files. They may look a bit bad at times, but better than JPEG in the case of those specific images. Reason for not using truecolor PNG: 8-bit saves a ton of bandwidth.

2. How to make the non-Steam version work on XP / XP x64

Software required:

  1. [Nekopara Vol.3]
  2. [7-zip] archiver
  3. NTCore [CFF Explorer] (optional; only needed for patches)
  4. Olegs’ [patcher]

2a. The main game

First, buy the game and download it. Do not pirate it! You suck if you do (I actually fooled around with a pirated version as well, but only after buying the game). When running the installer, you’ll notice that it already breaks early on after invoking the launcher:

Nekopara Vol.3s' installer already breaks

Nekopara Vol.3s’ installer already fails to execute on XP

As you can see, it calls InitializeCriticalSectionEx(), which is a newer, Vista+ version of InitializeCriticalSection(), see the MSDN[1][2] for details. Since the new version works differently, you can’t just hex edit your way out of this one.

First, unpack Olegs’ patcher to some subdirectory of your choice. Then, unpack the Nekopara Vol.3 installer (the .exe file) into a subfolder using 7-zip, and look for a file called INSTALL.exe. Copy that file into the directory where Olegs patcher resides, so where files like xp_EU4_1.21.cmd and xp_Stellaris_1.6.cmd can be found.

Since the scripts from Oleg aren’t made for hacking our files, we’ll write a new one for this, let’s call it xp_installer.cmd. Edit that with a text editor, and add the following lines:

@ECHO OFF
rundll32.exe zernel32.dll,PatchFile INSTALL.exe

Make sure xp_installer.cmd, zernel32.dll and the INSTALL.exe from Nekopara are in the same directory, then execute xp_installer.cmd. either by just double clicking it, or by opening a cmd terminal and by running it from there. Like this (you don’t need to run the extra commands, they’re just there to show you more information):

Olegs' patch doing its magic on INSTALL.exe

Olegs’ patch doing its magic on INSTALL.exe!

After that, rename your original INSTALL.exe in the directory where you unpacked the Nekopara Vol.3 installer, creating a backup file. Copy the following files from the patcher directory back to the installer directory: INSTALL.exe, zernel32.dll, z3d9.dll, zs2_32.dll and normaliz.dll. The “z” files are now implementing the missing functions, while redirecting all the others to the real Windows libraries like kernel32.dll, d3d9.dll, ws2_32.dll etc.

You don’t need to repack anything, just run INSTALL.exe directly, and you’ll no longer be greeted with an error message, but with this:

The installer works now

The installer works now, great

Install the game to a directory of your choice. Now, if you click the NEKOPARAvol3.exe in the directory where the game was installed, the same launcher comes up again, but now it allows you to configure and play the actual game instead of installing it…

Nekopara Vol.3s' launcher after installation

Nekopara Vol.3s’ launcher after installation

…or does it? Well, the “System settings” part’ll work, yes, but when clicking that alluring “Start” button, you’ll run into yet another wall:

Nekopara still won't execute due to GetTickCount64()

What now? GetTickCount64(), that’s what.

Guess which function call doesn’t exist on XP? See the MSDN[1][2] again. GetTickCount64() really is an improvement over GetTickCount(), but still, XP simply doesn’t have this either. As you can see from the title bar, the offending binary is cs2.exe, which is the actual game. We can get rid of the issue by using Olegs’ patcher again, so it’s the same process as with INSTALL.exe, just use this script instead, call it xp_cs2.cmd or something:

@ECHO OFF
rundll32.exe zernel32.dll,PatchFile cs2.exe

Again, in case something goes wrong, rename your original cs2.exe before copying back the patched version with its .dll files. After copying back, you can run the game either by invoking cs2.exe directly, or by launching it from the NEKOPARAvol3.exe launcher:

Running the non-Steam version of Nekopara Vol.3 on XP x64

Running the non-Steam version of Nekopara Vol.3 on XP x64 (click to enlarge)

2b. Making patches work as well

Patches are essentially also just self-extracting archives that execute a launcher after unpacking. We’ll discuss the patch 11 in this case. Running it will produce a different kind of error (people who know the content restoration patches for the Steam version may have seen this error as well):

Nekopara Vol.3 patch failure

Nekopara Vol.3 patch failure, due to it not being “a valid Win32 application”.

This error means that the header of the binary is asking for a more modern platform. This may make sense, if the program really calls modern functions, but you know, there are modern applications that don’t ask for it and then fail with calls to things like GetTickCount64(), and there are programs which ask for a modern platform without ever having an actual need for it. The patchers are in the latter category of programs.

Unpack the patcher nekopara3_v11_update.exe using 7-zip, and look for a file called updater.exe. Create a backup copy of it, then open this file in NTCores’ CFF Explorer, and click on the “Optional Header” part. You’ll see something like this, I’ve marked the relevant lines with some red blocks for you:

updater.exe in CFF Explorer

updater.exe in CFF Explorer (click to enlarge)

The marked fields show values like 0006 and 0000, as you can see. The significant number is the last or rightmost, so 6 and 0. This corresponds to the platform target Windows NT 6.0, or in other words: Windows Vista. Just rewrite that to show the following numbers, then save the file:

Patch the header to NT 5.1

Patch the header to NT 5.1 (click to enlarge)

NT 5.1 (0005, 0001) equals Windows XP. Note that the kernel versions 5.0 mean Windows 2000, 5.2 means Server 2003 or XP x64 (slightly more modern). Again, no need to repack anything, just save the file after the modifications have been made and execute updater.exe afterwards, you should be getting this:

Nekopara Vol.3 non-Steam patcher working on XP

And here we have a working patcher (click to enlarge)

Yay! And now, for the Steam version of Nekopara Vol.3…

3. How to make the Steam version work on XP / XP x64

Software required:

  1. [Nekopara Vol.3] on Steam (a censored version)
  2. [Content restoration patch] (optional; only required if you have to do perverted things to the cat girls)
  3. at0ms’ [Steamless]
  4. A Windows Vista or newer machine (needed to run Steamless, can be a virtual machine)
  5. [7-zip] archiver
  6. NTCore [CFF Explorer] (optional; only needed for the content restoration patch)
  7. Olegs’ [patcher]

3a. The main game

First, buy the game on Steam and download it. If you really need the uncensored version (you probably do, heh?), buy the content restoration patch at Denpasoft and download that as well. Of course, running the game as-is won’t work, otherwise we wouldn’t need this article in the first place:

The Steam version of Nekopara Vol.3 breaks on XP as well of course

The Steam version of Nekopara Vol.3 breaks on XP as well of course, due to GetTickCount64() call, a newer and better version of GetTickCount(), see MSDN[1][2].

Now, what I didn’t get at first was that patching the Steam versions’ NEKOPARAvol3.exe can never work out of the box. The reason is, that the offending function calls aren’t plainly there for us to see – the actual game binary cs2.exe is encrypted and packed into a SteamStub binary as its payload data. This is a part of the Steamworks DRM system wrapping our program up.

To be able to patch it, we (unfortunately) need to crack its cryptographic DRM protection system first. Now, let me say this again: I do not condone piracy. Don’t fucking crack and distribute this game. You’re an ass if you do. Removing the DRM part is only being done so we can fix the game on XP, keep that in mind!

Well, let’s start; First, boot up a Vista or newer Windows, and install Steamless on it. I actually tried to compile Steamless for XP, but this is .Net 4.5.2 stuff. To make it work on .Net 4.0 would require modifications of its build files / source code, which is a bit over my head right now. So we’re stuck with needing a modern Windows OS to do this. Copy the problematic NEKOPARAvol3.exe from your Steam game installation directory over to that machine, or just install Steam and the game on the modern Windows OS as well (which is what I actually did).

Launch Steamless, open that .exe and decrypt / unpack it, Steamless will leave your binary alone, and create a new, fixed one, so you don’t need to create a manual backup copy:

Steamless cracking NEKOPARAvol3.exe

Steamless cracking NEKOPARAvol3.exe (click to enlarge)

Copy the fixed file back to XP, and rename it back to NEKOPARAvol3.exe. Create a backup of the original .exe in your Steam game installation directory, while you’re at it.

Unpack Olegs’ patcher in a directory of your choice, and move the NEKOPARAvol3.exe there as well, that’s where files akin to xp_EU4_1.21.cmd and xp_Stellaris_1.6.cmd can be found. Since those patcher scripts aren’t targeted at Nekopara Vol.3, we’ll write our own, call it xp_neko_3.cmd or something, open it in a text editor and enter the following lines:

@ECHO OFF
rundll32.exe zernel32.dll,PatchFile NEKOPARAvol3.exe

Make sure that NEKOPARAvol3.exe, zernel32.dll and xp_neko_3.cmd are together in the same folder, then execute xp_neko_3.cmd either by double-clicking it, or by opening a cmd terminal and executing it from there. Like this:

Olegs' patcher handling the decrypted NEKOPARAvol3.exe

Olegs’ patcher handling the now-decrypted Steam version of NEKOPARAvol3.exe

Copy the fully fixed .exe back into the Steam game installation directory, together with the patchers’ stub libraries zernel32.dll, z3d9.dll, zs2_32.dll and normaliz.dll, which will handle the functions usually missing on XP.

Now, run the game either by executing NEKOPARAvol3.exe, or by launching it from within Steam, and you should be greeted with something like this:

Nekopara Vol.3 running on XP x64 in its Steam version

Nekopara Vol.3 running on XP x64 in its Steam version (click to enlarge)

Great (or something)!

Please be aware that if the binary is ever overwritten by Steam because of some update or whatever, you have to re-do the procedure, meaning the Steamless unpacking plus applying Olegs’ patch. If the game terminates without any error when launched from within Steam, try to run NEKOPARAvol3.exe directly instead, and you’ll see the error messages – Steam tends to suppress them.

3b. The content restoration patch (this also applies to the patches for Nekopara Vol.1 and Vol.2)

So you want to lewd the cat girls? Perverted! Plus, Windows XP / XP x64 won’t let you, because the patch is asking for a newer platform (despite not actually requiring it though):

Nekopara Vol.3 content restoration patch failure on XP

Nekopara Vol.3 content restoration patch failure on XP, due to the patch not being “a valid Win32 application”.

But if you absolutely have to, here’s how. Unpack the nekopara_vol3_Steam_R18DLC.exe you bought and downloaded from Denpasoft using 7-zip. Look for the file SteamPatch.exe, and open it in CFF Explorer:

The Nekopara Vol.3 Content restoration patchs' SteamPatch.exe in CFF Explorer

The Nekopara Vol.3 Content restoration patchs’ SteamPatch.exe in CFF Explorer

Now, this is similar to the procedure described for updater.exe for a non-Steam versions’ patch. The significant (rightmost) numbers in the fields where it days 0006 and 0000 represent Windows NT 6.0, or in other words Windows Vista. Since the patcher doesn’t really need any Vista-specific functions, we’ll just fix the header that is currently asking for a NT 6.0 platform as follows:

The Nekopara Vol.3 Content restoration patchs' SteamPatch.exe in CFF Explorer, fixed for XP

Change the fields to 5.1 (0005 and 0001 respectively) to have it check for XP+ instead, and it’s fixed!

Save the file after modifying it. Just like for the non-Steam version patches, there is no need to repack anything. Just run updater.exe directly, and you’ll now get this:

The Nekopara Vol.3 content restoration patch for the Steam version working on XP x64

The Nekopara Vol.3 content restoration patch for the Steam version working on XP x64. Because you’re in it for the Hentai.

There you go, pervert! You now have the fully restored version of Nekopara Vol.3 on Steam, running on XP or XP x64.

And last but not least: Thanks again, Oleg! I made you touch some weird shit, but you still fixed it and gave me the right ideas about the Steam version as well, yay! ;)

4. Bonus feature: How to make Mechwarrior Online work on XP / XP x64 after their launcher upgrade

While entirely unrelated to the weird Japanese shit above, I’ll just mention this here as well, because it doesn’t deserve its own post, given the simplicity of the “solution”; Piranha Games decided to give Mechwarrior Online (MWO) a new game launcher called “MWO Portal”, that is now built with .Net 4.5.2, just like Steamless, breaking it on XP. Mind you, the game itself would still work just fine, even the 64-bit version on XP x64.

The new MWOPortal launcher

Windows XP / XP x64 users will likely never see this launcher work on their OS (Unless ExtendedXP really takes off, it’s pretty good already, but yeah).

Since hacking .Net 4.5 stuff to run on .Net 4 / .Net 4 CP is not something I can do yet, MWO would be gone from all XP machines. There is an easy fix for this though:

Get the game on Steam! The Steam version doesn’t include the launcher, as Steam itself is handling both the execution and the updates of MWO. Without the launcher, MWO still works just fine! :)

May 312017
 

HakuNeko logo1.) What for?

Usually, porting my favorite manga ripper [HakuNeko] would involve slightly more exotic target platforms like [FreeBSD UNIX]. This changed with version 1.4.2 however, as this version – the most current at the time of writing – would no longer compile on Windows machines due to some issues with its build toolchain. And that’s the most common platform for the tool!

This is what the lead developer had to say about the issue while even suggesting the use of [FMD] instead of HakuNeko on Windows:

“The latest release does not compile under windows due to some header include contradictions of sockets […]”
    -in a [comment] to HakuNeko ticket #142 by [Ronny Wegener], HakuNeko project leader

Normally I wouldn’t mind that much and keep using 1.4.1 for now, but unfortunately this is not an option. Quite a few Manga websites have changed by now, breaking compatibility with the previous version. As this is breaking most of HakuNekos’ functionality for some important sites, it became quite unusable on Windows, leaving Linux as the only officially supported platform.

As using virtual machines or remote VNC/X11 servers for HakuNeko proved to be too tedious for me, I thought I’d try to build this by myself. As the MSYS2/MinGW Windows toolchain seemed to be broken for 1.4.2, I tried – for the very first time – to cross-compile on Linux, choosing CentOS 7.3 x86_64 and MinGW32 4.9.3 for the task. This was quite the challenge, given that HakuNeko comes completely unprepared for cross-compiling.

2.) First, the files

Took me many hours / days to get it done – 100% statically linked too – and it’s finished now! What I won’t provide is an installer (don’t care about that), but here are my v1.4.2 builds for Windows:

As of today, those versions have been tested successfully on the following operating systems:

  • Windows XP Professional SP3 / POSReady2009
  • Windows XP Professional x64 Edition SP2 w. Server 2003 updates
  • Windows Server 2003 R2 x64 SP2
  • Windows Vista Enterprise x64 SP2
  • Windows 7 Professional x64 SP1
  • Windows 10 Professional x64 build #1607

Please be aware that not all of the functionality has been tested by me, just a few downloads that wouldn’t have worked with 1.4.1, a few more random downloads here and there, plus chapter-wise .cbz packaging. It’s looking pretty good I think. Here are some sample screen shots as “proof” of it working (click to enlarge):

HakuNeko 1.4.2 downloading "Kobayashi-san Chi no Maid Dragon" on XP x64

HakuNeko 1.4.2 downloading “Kobayashi-san Chi no Maid Dragon” on XP x64 (Note: I own that Manga in paper form)

3.) What has been done to make this work?

3a.) Initial work:

First of all, cross-compiling is a bottomless, hellish pit, a horrible place that you never want to enter unless a.) The build toolchain of the software you wanna compile is very well prepared for it or b.) you really have/want to get your hands on that build or c.) you hate yourself so much you have to hurt yourself or d.) you actually enjoy c.).

The reasons for choosing cross-compiling were that Ronny Wegener had said, that the MSYS2/MinGW32 build would fail on Windows, plus it would require GCC version 5.3 to link with the bundled, pre-built static libraries (OpenSSL, cURL, wxWidgets).

So I thought it would be more likely to work if I were to run my own MinGW setup on Linux, not relying on the bundled stuff but linking against the libraries that come with MinGW32 4.9.3 on my platform of choice – CentOS 7.3 Linux.

One exception was the GUI library wxWidgets 3.0.2 that I had to cross-compile and statically link by myself as well, but luckily, that’s easy despite its size. wxWidgets is one piece of software that does come well-prepared for cross-compiling! In my case, that made it as simple as this (parallel compile with 6 CPUs):

$ ./configure --prefix=/usr/local/i686-w64-mingw32 --host=i686-w64-mingw32 --build=x86_64-linux \
 --enable-unicode --with-expat --with-regex --with-opengl --with-libpng --with-libjpeg --with-libtiff \
 --with-zlib --with-msw --enable-ole --enable-uxtheme --disable-shared
$ make -j6
# make install

3b.) HakuNeko build toolchain / Makefile modifications for cross-compiling:

HakuNeko is much harder, and I don’t even remember half of what I did, but most of it was manually editing the ./Makefile after $ ./configure --config-mingw32 would have produced something rather broken.

Let’s get to it, file paths are relative to the source root. First, edit the following parts of the ./Makefile (you need to look for them in different places of the file). First, the PREFIX, should be in the bottom half of the file:

PREFIX = /usr/local/i686-w64-mingw32/

CC and the CFLAGS:

CC = i686-w64-mingw32-g++
CFLAGS = -c -Wall -O2 -std=c++11 \
 -I/usr/local/i686-w64-mingw32/lib/wx/include/i686-w64-mingw32-msw-unicode-static-3.0 \
 -I/usr/local/i686-w64-mingw32/include/wx-3.0 -D_FILE_OFFSET_BITS=64 -D__WXMSW__ -mthreads \
 -DCURL_STATICLIB -I/usr/i686-w64-mingw32/sys-root/mingw/include

Add -DPORTABLE, if you want to build the portable version of HakuNeko.

Then, the Windows resource compiler, controlled by RC and RCFLAGS:

RC = /usr/bin/i686-w64-mingw32-windres
RCFLAGS = -J rc -O coff -F pe-i386 -I/usr/i686-w64-mingw32/sys-root/mingw/include \
 -I/usr/local/i686-w64-mingw32/include

And finally, the static linking part, which is the hardest stuff to get done right, LD, LDFLAGS and LDLIBS:

LD = i686-w64-mingw32-g++
LDFLAGS = -s -static -static-libgcc -static-libstdc++ -mwindows -DCURL_STATICLIB
LDLIBS = -L/usr/local/i686-w64-mingw32/lib   -Wl,--subsystem,windows -mwindows \
 -lwx_mswu_xrc-3.0-i686-w64-mingw32 -lwx_mswu_webview-3.0-i686-w64-mingw32 \
 -lwx_mswu_qa-3.0-i686-w64-mingw32 -lwx_baseu_net-3.0-i686-w64-mingw32 \
 -lwx_mswu_html-3.0-i686-w64-mingw32 -lwx_mswu_adv-3.0-i686-w64-mingw32 \
 -lwx_mswu_core-3.0-i686-w64-mingw32 -lwx_baseu_xml-3.0-i686-w64-mingw32 \
 -lwx_baseu-3.0-i686-w64-mingw32 -L/usr/i686-w64-mingw32/sys-root/mingw/lib -lcurl -lidn -liconv \
 -lssh2 -lssl -lcrypto -lpng -ljpeg -ltiff -lexpat -lwxregexu-3.0-i686-w64-mingw32 -lz -lrpcrt4 \
 -lwldap32 -loleaut32 -lole32 -luuid -lws2_32 -lwinspool -lwinmm -lshell32 -lcomctl32 -lcomdlg32 \
 -ladvapi32 -lwsock32 -lgdi32

Took a while to find the libraries (and static library order!) necessary to satisfy all the dependencies properly.

If you need it, here is the modified Makefile I’ve used to cross-compile:

  • [HakuNeko Makefile] for cross-compiling HakuNeko 1.4.2 for Windows on CentOS 7.3 x86_64 Linux (needs statically linked & installed wxWidgets first).

3c.) Source code modifications:

However, something will still not be quite right, because some of the crypto libraries will provide the MD5 functions MD5_Init(), MD5_Update() as well as MD5_Final(), and those are already defined by HakuNeko itself. This will break the static linking, as redundant definitions won’t work. We’ll rely on the libraries (libcrypto, libssl), and comment the built-in stuff out in src/v7/v7.c:

  1. void MD5_Init(MD5_CTX *c);
  2. void MD5_Update(MD5_CTX *c, const unsigned char *data, size_t len);
  3. void MD5_Final(unsigned char *md, MD5_CTX *c);

…becomes:

  1. /* void MD5_Init(MD5_CTX *c);
  2.  * void MD5_Update(MD5_CTX *c, const unsigned char *data, size_t len);
  3.  * void MD5_Final(unsigned char *md, MD5_CTX *c);
  4.  */

On top of that, the configure system may have generated src/main.cpp as well as src/main.h. Those are bogus files, turning the entire tool into nothing but one large “Hello World” program. Plus, that’s hard to debug, as the binary won’t even output “Hello World” on a Windows terminal when it’s built as a GUI tool. I only found the issue when testing it with Wine on Linux. ;)

Please delete src/main.cpp and src/main.h before continuing.

Now, if you’re really lucky, you should be able to run something like $ make -j6 and watch everything work out nicely. Or watch it crash and burn, which is the much, much more likely scenario, given I’ve likely only given you half of what I did to the build tools.

Well, in any case, no need to run $ make install of course, just grab the binary build/msw/bin/hakuneko.exe and copy it off to some Windows machine, then try to run it. If you’ve built the portable version, you may wish to rename the file to hakuneko-portable.exe, just like the official developers do.

4.) The future

Let’s just hope that the developers of HakuNeko can get this fixed for versions >=1.4.3, because I really, really don’t want to keep doing this. It’s extremely painful, as cross-compiling is exactly the kind of living hell I heard a lot of people saying it is! I think it’s a miracle I managed to compile and run it at all, and it was so frustrating and tedious for somebody like me (who isn’t a developer).

The statement that this took “hours / days” wasn’t an exaggeration. I think it was something like 10-12 man hours of pure frustration to get it done. I mean, it does feel pretty nice when it finally works, but I wouldn’t bet on myself being able to do this again for future versions…

So please, make it work on Windows again, if possible, and keep HakuNeko cross-platform! It’s still my favorite tool for the task! Thanks! :)

Feb 272017
 

Nekopara Chocola logo…he was just too lazy to take pictures, especially given my rather stupid photography setup. Hah, but it’s not like I stopped buying weird stuff and the figures I’m gonna show you today are just a fraction of what I got, so here we go!

1.) The Nekos / Catgirls from Nekopara

The first (as you may be able to guess from the logo) are the Nekopara mini figures, from the extremely successful and totally not steamy[1] visual novel series of the same name. Actually, the project will now even get an Anime OVA due to an equally successful [Kickstarter project] that I totally did not pledge for (I guess I still need to do something about my name showing up in the credits :roll: ).

I had planned to ask people to pledge for the extension to that – their [slacker backer] campaign – but it has already overshot the last stretch goal, so not much of a point unless to really want to get your hands on the OVA and/or the merchandise. At least we get a Nyanko bonus OVA now as well. Anyway, here are some pictures – unfortunately I didn’t get a hold of the limited preorder bonus anymore, as usually, click to enlarge:

Nekopara Nyankos

The chibi figures of our kitties, f.l.t.r.: Azuki, Cinnamon, Coconut, Vanilla, Chocola, Maple, Shigure (who’s actually human) and Miruku/Milk. Cute as fuck!

Ah yes, sorry for the licensing stamps. I was too lazy to create the usual two versions, one for here and one for off-site use. If somebody really wants a clean version of some of the photos, let me know in the comments – the license won’t change though.  Let’s look at the individual kittens, pair-wise:

 

Those two are the main catgirls, featuring a touching backstory of having been abandoned when very young and then saved by the main character whose role the player takes in the visual novel. Said backstory will be featured in a mini VN as well as a mini Anime OVA thanks to the Anime crowdfunding campaigns breaking through the $800.000 and $1.000.000 stretch goal milestones. They’re the youngest amongst the cats, and of unknown breed. Also, they’re twins, despite not looking the part.

Next:

 

Here we have the local tsundere loli Azuki (I do like her attitude!), a Munchkin cat, who is the oldest of the pack and the pretty perverted Cinnamon, whom we will see more of in the final VN that’ll come out some time this year – I forgot the actual release date. She’s supposed to be a Scottish Fold.

More:

 

Besides being the largest of the pack (in several ways), Coconuts’ most distinct feature are probably her dichromatic eyes. She’s the second-youngest and pretty naïve as well. The other one here is Maple, an American Curl who tends to get into fights with Coconut on a regular basis. Maple will also be shown more of in the final VN. And two more to go:

 

As said, Shigure is no actual catgirl, but the main characters sister, and a huge brocon as well (although there is no corresponding siscon going on here). She raised all of the first six catgirls shown above, and she supposedly did a great job with that. To the right we see Milk (Hepburn pronunciation “Mi-ru-ku”), who is a side character, not a member of the main cast. She’s a very young kitten helping her owner at a [Takoyaki] stand, and she’s cute as hell.

Yay for genetically engineered catgirls! (We seriously need a Kickstarter campaign for that!)

[1] The games feature explicit content only if you buy the original versions. If you get the games on Steam, they will be censored. There are content restoration patches for those versions as well, but in essence, you can get both uncensored as well as a slightly cheaper, censored versions. The Anime OVAs will not contain any explicit content.

2.) Illyasviel von Einzbern

I had actually sworn to myself that I would never ever watch Magical Girl stuff again (I did it once, in the late 90s’). But then, there came the Yuri, and when there is Girls Love, there’s just no way I’m gonna hold back, no matter the setting. So yes, I did watch all of Fate/Kaleid Liner Prisma☆Illya, and it’s… pretty questionable stuff, at least in season 2 & 3 if I remember correctly. This is Loli Yuri, you have been warned! In any case, let’s go all pink, sigh…

 

Yeah, I know. Just don’t say anything. I feel conflicted and embarrassed enough posting her here! Let’s take a closer look:

Illya, angle shot

Angle shot – I mean, yeah, she’s beautifully crafted, the detail level is high, and it’s a very dynamic pose. The problem with this clearly lies elsewhere :roll:

Let’s take a look at her face:

Illyas' face

Yep, Phat Companys’ sculptors and color producers did a pretty much perfect job here…

It gets much worse though…

Cardholder Analysis 1

This is wrong for too many reasons. Please do not watch the Anime, for I fear you might learn all of them.

You know, I’d love to tell you that my one complaint with this figure is the lack of detail on her card holder, and that I took this photo to show you said lack of detail… And of course, the card holder being totally out of focus here has nothing to do with anything! And that there is an actually correctly taken shot of this doesn’t mean anything either! Ah yeah, here:

Cardholder Analysis 2

The card holder *does* indeed lack some detail however… The FuRyu figure of her has the detailed pattern on it for instance.

Now if we can agree that none of this ever happened, we’re all good! But then again, I could’ve just not posted her in the first place, guess I feel like a criminal who does want to be caught after all.

Well, whatever, so what’s left? There will be two more scale figures in the future, of which one is particularly beautiful – look forward to a nice Kimono for once! And there are also some Nendoroids who’re missing for now, and depending on when a certain Madoka★Magica 2-figure set is really coming out, there will be another two to add to the list.

Guess I still have a lot of pictures to take. Could be that this blog will once more shift from technical stuff to weird Anime merchandise in next few weeks. :roll:

Jan 252017
 

H.265/HEVC logo1.) Introduction

After doing a [somewhat proper comparison between x264 and x265] a while back, I thought I’d do another one at extremely low bitrates. It reminded me of the time I’ve been using ISDN at 64kbit/s (my provider didn’t let me use CAPI channel aggregation for 128kbit/s), which was the first true flat rate in my country. ‘Cause I’ve been thinking this:

“Can H.265/HEVC enable an ISDN user to stream 1080p content in any useful form?” and “What would H.264/AVC look like in that case?”

Let me say this first: It reaaally depends on how you define “useful”. :roll:

Pretty much nobody uses ISDN these days, and V.9x 56kbaud modems are dying out in the 1st world as well, so this article doesn’t make a lot of sense. To be fair, I didn’t even pick encoding settings fit for low-latency streaming either, nor are my settings fit for live encoding. So it’s just for the lulz, but still! I wanted to see whether it could be done at all, in theory.

To make it happen, I had to choose extremely low bitrates not just for video, but audio as well. There are even subtitles included in my example, which are present in Matroska-style zlib-compressed [.ass] format, so compressed text essentially.

For the audio part, I chose the Fraunhofer FhG-AAC encoder to encode to the lowest possible constant bitrate, which is 8kbit/s HEv2-AAC. That’s a VoIP-focused version of the codec targeted at preserving human speech as well as possible at conditions as bad as they get. And yes, it sounds terrible. But it still gets across just enough to be able to understand what people are saying and what type of sounds are occurring in a scene. Music and most environmental sounds are terrible in quality, but they are still discernible.

For video, I picked a 2-pass ABR mode with a 50kbit/s target bitrate, which is insanely low even for the Anime content I picked (my apologies, Mr. “[Anime is not what everyone watches]”, but yes, I picked Anime again). Note that 2D animated content is pretty easy on the encoders in this case, so the results would’ve likely been a lot worse with 1080p live action content. As for the encoder settings, you can find those [down below] and as for how I’m taking the screenshots, I’ll spare you those details, they’re pretty similar to the stuff shown in the link at the top.

Before we start with the actual quality comparison, I should mention that my test results actually overshot their target, so they’re really unsuitable for live streaming even in the ISDN case. I just didn’t care enough for trying to push the bitrate down any further. Regular streaming would still be possible with my result files, but not without prebuffering. See here:

$ ls -hs *.mkv
2.6M Piaceː Watashi no Italian - Episode 02-H.264+HEv2AAC-V50kbit-A8kbit.mkv
2.0M Piaceː Watashi no Italian - Episode 02-H.265+HEv2AAC-V50kbit-A8kbit.mkv
 76M Piaceː Watashi no Italian - Episode 02.mkv
$ for i in {'Piaceː Watashi no Italian - Episode 02.mkv','Piaceː Watashi no Italian - \
Episode 02-H.264+HEv2AAC-V50kbit-A8kbit.mkv','Piaceː Watashi no Italian - \
Episode 02-H.265+HEv2AAC-V50kbit-A8kbit.mkv'}; do mediainfo "$i" | grep -i "overall bit rate"; done
Overall bit rate                         : 2 640 kb/s
Overall bit rate                         : 88.6 kb/s
Overall bit rate                         : 69.2 kb/s

The first one is the source (note: From [Crunchyroll], legal to watch and record in my country at this time), the second my x264 and the third my x265 versions. Let’s show you the bitrate overshoot of just the video streams in my versions:

$ for i in {'Piaceː Watashi no Italian - Episode 02-H.264+HEv2AAC-V50kbit-A8kbit.mkv','Piaceː \
Watashi no Italian - Episode 02-H.265+HEv2AAC-V50kbit-A8kbit.mkv'}; do mediainfo \
--Output="Video;%BitRate/String%" "$i"; done
71.1 kb/s
51.6 kb/s

So as you can see, x264 messed up pretty big, overshooting by 21.1kbit/s (42.2%), whereas x265 almost landed on target, overshooting by a mere 1.6kbit/s (3.2%) overall. And still… well… Let’s give you an overview first (as usual, click to enlarge images):

2.) Quality comparisons

Note that the color shown in those thumbnails is not representative of the real images, this has been transformed to 256 color .png to make it easier to download (again, if your browser supports it, .webp will be loaded instead transparently). This is just to show you some basic differences between what x264 and x265 are able to preserve, and what they are not. Also, keep in mind, that “~50kbit/s nominal bitrate” means 71.1kbit/s for x264 and 51.6kbit/s for x265!

Overall, x264 fucks up big time. There are frames with partial macroblock drops and completely blank frames even! Also, a lot of frames lose their color either partially or completely as well, making them B/W. And that’s given that x264 even invested 42.2% more bitrate than what I aimed for!

x265 has no such severe issues, all frames are completely there in full color, and that at a bitrate reasonable close to the target. Let’s look at a few interesting cases side by side:

Scene 1 (left: x264, middle: x265, right: source file):

There are some indications of use of larger CTUs (coding tree unit, H.265s’ replacement for macroblocks) in x265s’ case, which is supposed to be one of its strong points, especially for very large resolution encoding (think: 4K/UHD, 8K). While larger blocks can mean loss of detail in that area, it’s ok for larger areas of uniform color, which this Anime has a ton of. H.264/AVC can’t do that so well, because the upper limit for a macroblocks’ size is rather low with 16×16 pixels. You can see the macroblock size pretty clearly in the blocky frame to the left. You need to look a bit more carefully in x265s’ case, but there are a few spots where I believe it can be seen as well. In my case the CTU size for x265 was 32×32px.

Hm, maybe --ctu 64 would’ve been better for this specific case, but whatever.

Lets look at two more mostly color-related comparisons:

Scenes 2 & 3 (left: x264, middle: x265, right: source file):

 

In the first case it seems as if x264 is trying to preserve shades of green more than anything, but in the second case, something terrible happens. There is a lot of red in the scene before this one, and there is quite some red on those can labels as well. It seems x264 doesn’t know where to put the color anymore, and the reds bleed almost all over the frame. And it stays like that for the entire scene as well, which means for several seconds. The greens and browns are lost. Block artifacts are excessive as well, but at least x264 managed to give us whole frames here, with some color even.

Well, the color kinda went everywhere, but uhm, yeah…

Two more:

Scenes 4 & 5 (left: x264, middle: x265, right: source file):

 

I really don’t know what’s with x264 and the reds. Shouldn’t green have priority? I mean, not just in the chroma subsampling, but in encoding as well? But red seems what x264 drops last, and it happens more than once. Given the detail and movements in that last part, even x265 fails though. Yes, it does preserve more color, but it doesn’t come remotely close to the source at this bitrate.

And that other frame with the cuteness overload? There are a lot like those, where x264 just kinda panics, drops everything it has and then frantically tries to (re?)construct the current frame, sometimes only partially until the next I-frame arrives or so.

So that’s it for my quick & dirty “ultra low bitrate” comparison between x264 and x265, at pretty taxing encoding settings once again.

3.) Additional information

x264 encoding settings:

$ mediainfo Piaceː\ Watashi\ no\ Italian\ -\ Episode\ 02-H.264+HEv2AAC-V50kbit-A8kbit.mkv | grep -i \
"encoding settings"
Encoding settings                        : cabac=1 / ref=16 / deblock=1:-2:0 / analyse=0x3:0x133 / \
me=umh / subme=10 / psy=1 / psy_rd=0.40:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 \
/ 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=18 / \
lookahead_threads=4 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / \
constrained_intra=0 / bframes=16 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / \
open_gop=1 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / \
rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=50 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=81 \
/ qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:0.60

x265 encoding settings (note: 10 bits per color channel were chosen, same as for x264):

$ mediainfo Piaceː\ Watashi\ no\ Italian\ -\ Episode\ 02-H.265+HEv2AAC-V50kbit-A8kbit.mkv | grep -i \
"encoding settings"
Encoding settings                        : cpuid=1049087 / frame-threads=3 / wpp / pmode / pme / \
no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1920x1080 / interlace=0 / total-frames=0 \
/ level-idc=0 / high-tier=1 / uhd-bd=0 / ref=6 / no-allow-non-conformance / no-repeat-headers / \
annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / \
keyint=250 / bframes=16 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=40 / \
lookahead-slices=0 / scenecut=40 / no-intra-refresh / ctu=32 / min-cu-size=8 / rect / amp / \
max-tu-size=32 / tu-inter-depth=4 / tu-intra-depth=4 / limit-tu=0 / rdoq-level=1 / signhide / \
no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / no-strong-intra-smoothing / max-merge=5 \
/ limit-refs=1 / limit-modes / me=3 / subme=4 / merange=57 / temporal-mvp / weightp / weightb / \
no-analyze-src-pics / deblock=0:0 / no-sao / no-sao-non-deblock / rd=6 / no-early-skip / rskip / \
no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / rdpenalty=0 / psy-rd=1.60 / \
psy-rdoq=5.00 / no-rd-refine / analysis-mode=0 / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / \
bitrate=50 / qcomp=0.75 / qpstep=4 / stats-write=0 / stats-read=2 / stats-file=265/v.stats / \
cplxblur=20.0 / qblur=0.5 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / \
zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / sar=1 / overscan=0 / \
videoformat=5 / range=1 / colorprim=2 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 \
/ max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / \
slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05

x264 version:

$ x264 --version
x264 0.148.x
(libswscale 3.0.0)
(libavformat 56.1.0)
built on Sep  6 2016, gcc: 6.2.0
x264 configuration: --bit-depth=10 --chroma-format=all
libx264 configuration: --bit-depth=10 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat license: nonfree and unredistributable
WARNING: This binary is unredistributable!

x265 version:

$ x265 --version
x265 [info]: HEVC encoder version 2.2+23-58dddcf01b7d
x265 [info]: build info [Linux][GCC 6.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2

Encoding & testing platform:

$ uname -sr
Linux 2.6.32-573.8.1.el6.x86_64
$ cat /etc/redhat-release 
CentOS release 6.8 (Final)
$ cat /proc/cpuinfo | grep "model name" | uniq
model name	: Intel(R) Core(TM) i7 CPU       X 980  @ 3.33GHz

4.) Answers

Q: “Can H.265/HEVC enable an ISDN user to stream 1080p content in any useful form?

A: It can probably stream something that at least resembles the original source in a recognizable fashion, but… whether you can call that “useful” or not is another thing entirely…

Q: “What would H.264/AVC look like in that case?”

A: Like shit! :roll:

Apr 222016
 

Sakura Trick - Haruka×Yuu logoBefore I say anything else or show you any photos; I would like to thank a certain Mr. Isohata, who happens to run [this eBay shop] called “Toys Glory” right here. He might not always provide the cheapest offers, but let me tell you this: That man delivers! Not only did he manage to surprise me by being able to get me a really rare, exclusive (and in the Western World: pretty much unobtainable) [Tokisaki Kurumi] version, no, he also managed to get me todays’ two little gems, which I couldn’t manage to find anywhere else anymore. And it didn’t take him longer than 48 hours to get his hands on them even – in a sealed state! Hell, I don’t know how he does that, but if you’re sitting in the Western world and you’re looking for something rare with no other options left… Just drop him a message, chances are that if anyone can get that rarity for you, it’s probably him! Oh, and his English is pretty good, best I’ve read corresponding with Japanese people, so no worries there!

Now, that I’m a fan of Yuri Anime (=”girls’ love”, sometimes also inappropriately called “shoujo ai”[1] in the West) is probably not exactly a secret anymore, but from my point of view, there just aren’t many good series in that department. Or let’s say… unobstructedly enjoyable ones. I should really prepare my top list for the genre, but it’ll take a while longer I guess. But to cut to the chase: If you’re looking for a really fluffy, kinda innocent/naïve slice-of-life Yuri Anime, look no further: There is only one. And that’s Sakura Trick / 桜Trick.

Sakura Kiss

You can probably guess where this is going…

I was always kinda disappointed by different figures from different Yuri Anime, because essentially, they’re never posable. And what good are they, if you can’t… uhm… combine them.

Also, it seems Anime released by the infamous [Studio Deen] aren’t seeing a lot of love from figure sculptors (because most of Deens’ series are really low quality), but recently they’ve been improving quite some. And Sakura Trick? Hell, it’s freaking beautiful. Especially if you own a pair of Yuri goggles. :roll:

But there ARE two figures from Sakura Trick after all – the two main characters – which fit the bill. Unfortunately they’re from Waves’ Beach Queens series (“Premium” branch in this case though), which surely aren’t my favorites. Reason? Well, dressing up any and every girl from random Anime in Bikinis just to show them off with as little fabric on their bodies as possible isn’t exactly my thing. But ok, in this case, it’s forgiven and forgotten! We’re talking about Sonoda Yuu × Takayama Haruka after all. ;)

Wave went through the trouble of making the heads on these exchangeable for posing purposes. That means they had to integrate plastic sockets into the heads to be able to connect the ball joint, because the two are made of quite hard polystone, not PVC. The material’s a kind of artificial stone made from polyurethane resin and stone aggregates to give it a porcelain-like feel to the touch. It doesn’t smell like PVC does, it’s supposed to allow for more refined detail in sculpting and you need to be careful – because it can shatter easily.

The scale on these is 1/10, not 1/8, so they’re rather tiny. Let’s have a look, now shall we?

As said, the sockets are pretty heavy and feel well made and all, but… they’re in the way as well. Because if you want to pose the two figures, eh, properly, the sockets won’t let you because they put too much distance between the two. But first things first, let’s change those heads:

Theeere we go. Yuu sure sports some nice sculpting around her belly! Well, there is a small flaw on her hand, but to the naked eye this is not too noticeable, so I’m fine with that. And now, as for the actual posing:

Aw, adorable.

Now, as for the detail level, it may not seem overly great, but consider the scale! 1/10 is quite a bit smaller than 1/8 after all. Ah, just see for yourself:

Sakura Trick - Haruka×Yuu size comparison

This is how a 1/10 scale figure compares to a Nendoroid and a 1/8 scale, both of which are Hirasawa Yui from K-On! Not counting the baseplates, the heights are (from left to right): 15.2cm / 5.98″, 9.8cm / 3.86″ and 18.5cm / 7.28″.

Well, in the end, those two were pretty expensive. They weren’t cheap at launch already, probably because they’re pretty niche after all, and I had to pay roughly twice the original price. It wasn’t breaking the bank or anything, but no matter the quality – rarity costs money. If you’re willing to go (and pay for) the extra mile when it comes to harder-to-get stuff, see paragraph 1. ;) If you’re actually in Japan, this should be much easier I guess, but from Europe or the US, options are limited after all.

Now, let me schedule that third rewatch of Sakura Trick… Because there’s no Yuri like this one…

[1] Shoujo/Shōjo (jap. 少女) mostly means “little girl” or “young girl”. The term “shoujo ai” was meant to just mean “girls love” in the western world, but this is easily mistakable, as the term is being understood as “pedophilia” in Japan. My personal assumption being that it just means something like “love for really young girls”. Saying that you like “shoujo ai” to the wrong person is one mistake you don’t wanna make, ever!

Apr 192016
 

Banpresto: Hanayamata Kyun-charas - logoAnd while I’m waiting for a real rarity, here we are with more Anime Chibi figures in the meantime! You want more, right? Yeah, chances are pretty high you don’t, but here they are anyway. Amongst Moeblob Anime series (“cute girls doing cute things” as they say in the US), there are a few with either no or less high-profile merchandise available, and the Anime Hanayamata is one example of them. Not the worst of all (think Gochuumon), but yeah.

Not that the series should be that good actually, it’s extremely formulaic in nature and is comprised of characters made from 100% standard templates. Some of those shows are really uninteresting, but some I still like a lot for no apparent reason other than… Moe infection? Or maybe it’s really just well made somehow.

Well, whatever, I wanted some small chibis of the main cast, but alas, good old Goodsmile Company hasn’t made any Nendoroids of them, so what now?

Luckily those ‘roids have been so ridiculously successful, that several companies started to try and come up with competitive products – one of them being Banpresto with their “Kyun-Chara” series. Besides wanting the Hanayamata characters in PVC form, it’s also an opportunity to compare them with the current top dog in the market. Let’s see (CTRL+click to enlarge, as always)…

Banpresto: Hanayamata Kyun-charas - whole group

The main cast, from left to right: Machi, Naru, Hana, Yaya and Tami – The names alone already radiate Moe.

So what’s this even about? Aside from the typical coming-of-age and all-girls friendship/bonding stuff, it’s focused on some kind of modern freestyle dance called [Yosakoi], which combines classical Japanese dance movements with modern music, mostly JPop stuff I guess. I found that intriguing as I’d never heard of it before, and it wasn’t the typical school band or Idol stuff. Being artistically interesting (animation, drawing style etc.) I got kind of hooked. “Kind of” probably being an understatement… :roll:

Let’s look at the individual characters before the quick Kyun-Chara vs. Nendoroid comparison:

Cutesy, flowery stuff it is. Hana’s one of the two main characters, and by far the more lively one. Also: The classic “Western, Aryan exchange student” (yeah, I used that word just now, you know why, Japan!). I’m not going to comment on my impressions regarding quality before the Nendoroid comparison, but let’s just say, there’s good and bad stuff. Let’s continue for now:

And there we have our wallflower, Sekiya Naru. If you’re vaguely familiar with this type of show, I won’t need to say much more than that, just one thing: Her name is also a part of “Naruko”, the wooden clappers they’re holding. Originally used to chase away birds from fields, it’s now a musical accessory. Next girl:

Now if there ever was a run-of-the-mill Tsundere (Search for it on the web, if you don’t know the lingo, and if you really want to know), here she is. The rose theme probably fits with her being the jealous type as well, even if the color doesn’t match perfectly. But we already got a girl with a yellow theme, she’ll come last… First comes Miss Princess:

Despite the Lily (=Yuri) sometimes being a symbol for love between girls, this is not the case here. Nishimikado Tami is just the groups’ typical well-mannered, calm and introverted rich girl. Like all five of them, she’ll have to overcome one “drama” in the course of the series. That’s, if you can even call it that. It’s an 98% lighthearted show after all. And now, for the last one:

Tokiwa Machis’ the most stiff and stern of the five, and she’s late to the game as well, so not much time for her to soften up. She doesn’t all turn fluffy even at the end, which is probably a good thing. While she’s another Tsundere, her character is slightly less formulaic than Yaya.

How Hanayamata managed to take all this run-of-the-mill stuff that has been done to death for hundreds of times and still make it unbelievably enjoyable is still beyond me. Maybe it’s really the artwork. Or the music. Or the writing (nah, can’t be, lol). Whatever, it’s absolutely nice feel-good stuff. If you can take several kilograms of pink sugar per episode that is. ;)

Ah, I wanted to compare the little figures to the #1 in the field, GSCs’ Nendoroids, right? Let’s pick two pairs and have a quick look:

First of all, I thought the Kyun-Charas would be smaller. 7cm they said, but it’s more like 8.5cm (3.35″). They’re more petite however, having slimmer legs, slimmer waists etc, so in essence the are deformed more strongly. Also, they’re not posable like Nendoroids, so what you see here is all you get. Legs can be removed, but they don’t all have the same joints, so exchanging anything but heads is impossible. They’re not made for that.

Now, let’s talk quality: There are two things where Banpresto is still clearly behind. First, the deburring of the plastic. Parts of the Kyun-Charas will lack proper deburring and thus have slightly frayed edges. That just looks a bit cheap, so well-rounded, smooth edges are a must to achieve a flawless look.

The second are the color gradients when it comes to hair. Naru doesn’t seem to even have any. Machi does, but as you can see it’s pretty inferior to the Nendoroids, even the early ones.

The faces are ok however, definitely up to the level of GSC. And the clothes even surpass the Nendoroids I own, that’s a really good level of detail. So… it’s a mixed bag. Some parts are top-notch, while others are still a bit lacking, even if we don’t consider the non-posable character of Kyun-Charas.

So, if there were ‘roids, I’d probably have bought those instead. But the Kyun-Charas aren’t too shabby either. Just don’t expect them to use up less space than Nendoroids like I did, they’re almost equal in that department.

Aaand the next ones to appear on stage are some really hard-to-get specimens, think “Yuri”. If you like that, you may wish to check back…

Mar 232016
 

Logo for the Akaza Akari and Hirasawa Yui Nendoroids as well as the Strike Witches Perrine Clostermann and Francesca LucchiniYeah, more PVC incoming; It’s getting a bit cramped in my display cabinet, as it wasn’t really made for displaying 1/8 scale figures anyway, but there is still a little bit of room, so here are some figures I’ve had sitting on my wishlist for quite a few months. And in best tradition, some of them are a bit “borderline acceptable”. Or all of them, depending on your stance. Ah, and my apologies for the license stamps on the images. I forgot to keep processed copies without the stamps for the weblog… Meh.

The first two are Nendoroids, once more some Hirasawa Yui from K-On! (’cause there is no such thing as enough Yui! But this one is going to be modified for a special purpose.), and Akaza Akari from Yuru Yuri, and then two figures from Strike Witches – needless to say my two favorites – Perrine Clostermann and Francesca Lucchini, both made by Alter. Here are the boxes:

1.) The Nendoroids

Now as I said, Yui is supposed to become more special, as I had an idea to get myself a bobblehead for my car based on her quite long ago. Thing is, bobbleheads seem to be something pretty US American, so there simply are none based on Anime characters, despite Nendoroids having the perfect dimensions for it. Thus, I’d need to build my own:

My first attempt was to just drive some nails into Yuis’ feet and legs, but while that works for a static application, it’s nowhere near stable enough for putting her in an actual car, so I’m probably going to switch to screws. The legs can be twisted a bit so she won’t bend over due to inertia, but the thing is that the legs are partially hollow, reducing the amount of material the screw can safely sit in. I’ll need to get longer ones that go all the way up and into the hip joints as well to stand a chance of this being stable enough. Otherwise some hole/bump in the road or some violent breaking maneuver might send her flying off the baseplate.

The bobbleheads’ neck itself will be solved by replacing the plastic joint with either a tension spring or a pressure spring with optional stiffening material. I have briefly tested this already with another roid and it works well enough. All I need is a solid enough mount.

Further progress will be reported on this blog, but I can’t say when, as that depends on the intensity of my typically pretty severe laziness. ;)

Now, Akari:

Strangely, I got the limited preorder version with the additional parquet-style baseplate, as you can see on the photos. I’m not sure whether this was just luck, or whether it wasn’t limited after all. Maybe they just had some in stock still, although that’d be weird, given the age of the figure. The “Akariiin!~” plate is also included, but I forgot to take a picture of it. Ehm, probably better that way anyhow.

2.) The Alter 1/8 scale figures

Alter’s known to produce some high quality stuff, and I got to see that for myself with the [K-On! complete set], and hell, that’s some pretty decent stuff! So how about the Strike Witches? While the series is basically just about cute girls fighting aliens (or whatever the “Neroi” really are) with magically empowered guns and rocket launchers while presenting the curious viewer a distinct lack of apparel, the series’ military advisers seem to have ensured a certain amount of details regarding the Witches’ array of weaponry. Not to the extent of Girls und Panzer maybe, but still. Let’s look at the cool and well-mannered Tsundere character first, this is Perrine Clostermann:

Not exceptional it seems, besides a pretty spacious setup for a 1/8 figure. Surely not bad either though. Let’s zoom in a bit:

The Bren gun doesn’t boast too much detail, but it looks good enough. Same goes for her propeller striker units (there is a “in motion disc version” of the rotors as well, but I put the still ones on). The nicest part is probably her face, featuring metal wire frame glasses, which do look pretty good, surely better than some thicker and more flimsy plastic would. The downside is, that there is no actual glass, just the frame. Also, her front bangs could’ve been done a bit better!

Now, let’s look at the Strike Witches’ resident Loli character, Francesca Lucchini:

Now, believe it or not, the most important part for me about her was the gun, an American Browning M1919A6 MG. Now I’m not really that much of a WW2 infantry weapons nerd, but I got the impression that Alter got that piece done really well, so let’s take a closer look:

Now that’s really Alter-level quality right there. It’s a bit of a shame that the ammo box isn’t opened up a bit wider so we can catch a better glimpse of the belt-feed, but it’s still awesome. The shoulder belt looks like she’s truly in motion as well, and while it’s not easy to put the weapon in her hands properly, it looks perfect when done right.

And now, since this series is totally not about panty shots and won’t have the girls flash them at you like several times per minute or anything….

Francesca Lucchinis pantsu

You’re totally not gonna see this like all the time in the series!!!!111 Actually, this is probably barely legal by European standards, given her Loli status, but ya know, Japan… Click to enlarge if you really have to!

In my defense: It’s pretty hard to take any kind of picture of hers while not having her flash her panties at least at some angle. :roll: But there you go, serving the classic “blue white stripes”pantsu fetish that seems to exist amongst certain viewers in Japan. Not that I’m complaining or anything. ;)

And now, there is barely any room left to put any more scale figures… Hmmm…

Feb 202016
 

H.265/HEVC logoRecently, after [successfully compiling] the next generation x265 H.265/HEVC video encoder on Windows, Linux and FreeBSD, I decided to ask for guidance when it comes to compressing Anime (live action will follow at a later time) in the Doom9 forums, [see here]. Thing is, I didn’t understand all of the knobs x265 has to offer, and some of the convenient presets of x264 didn’t exist here (like --tune film and --tune animation). So for a newbie it can be quite hard to make x265 perform well without sacrificing far too much CPU power, as x265 is significantly more taxing on the processor than x264.

Thanks to [Asmodian] and [MeteorRain]/[LittlePox] I got rid of x265s’ blurring issues, and I took their settings and turned them up to achieve more quality while staying within sane encoding times. My goal was to be able to encode 1080p ~24fps videos on an Intel Xeon X5690 hexcore @ 3.6GHz all-core boost clock at >=1fps for a target bitrate of 2.5Mbit.

In this post, I’d like to compare 7 scenes from the highly opulent Anime [The Garden of Words] (言の葉の庭) by [Makoto Shinkai] (新海 誠) at three different average bitrates, 1Mbit, 2.5Mbit (my current x264 default) and 5Mbit. The Blu-Ray source material is H.264/AVC at roughly 28Mbit on average. Also, both encoders are running in 10-bit color depth mode instead of the common 8-bit, meaning that the internal arithmetic precision is boosted from 8- to 16-bit integers as well. While somewhat “non-standard” for H.264/AVC, this is officially supported by H.265/HEVC for Blu-Ray 4K. The mode of operation is 2-pass to aim for comparable file sizes and bitrates. The encoding speed penalty for switching from x264 to x265 at the given settings is around a factor of 8. Somewhat.

The screenshots below are losslessly compressed 1920×1080 images. Since this is all about compression, I chose to serve the large versions of the images in WebP format to all browsers which support it (Opera 11+, Chromium-based Browsers like Chrome, Iron, Vivaldi, the Android Browser or Pale Moon as the only Gecko browser). This is done, because at maximum level, WebP does lossless compression much more efficiently, so the pictures are smaller. This helps, because my server only has 8Mbit/s upstream. If your browser doesn’t support WebP (like Firefox, IE, Edge, Safari), it’ll be fed lossless PNG instead. All of this happens automatically, you don’t need to do anything!

Now, let’s start with the specs.

Specifications:

Here are the source material encoding settings according to the video stream header:

cabac=1 / ref=4 / deblock=1:1:1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=0.40:0.00 /
mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 /
fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=1 / sliced_threads=0 / slices=4 /
nr=0 / decimate=1 / interlaced=0 / bluray_compat=1 / constrained_intra=0 / bframes=3 / b_pyramid=1 /
b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=1 / weightp=1 / keyint=24 / keyint_min=1 /
scenecut=40 / intra_refresh=0 / rc_lookahead=24 / rc=2pass / mbtree=1 / bitrate=28229 /
ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 /
vbv_maxrate=31600 / vbv_bufsize=30000 / nal_hrd=vbr / filler=0 / ip_ratio=1.40 / aq=1:0.60

x264 10-bit encoding settings (pass 1 & pass 2), 2.5Mbit example:

--fps 24000/1001 --preset veryslow --tune animation --open-gop --b-adapt 2 --b-pyramid normal -f -2:0
--bitrate 2500 --aq-mode 1 -p 1 --slow-firstpass --stats v.stats -t 2 --no-fast-pskip --cqm flat
--non-deterministic

--fps 24000/1001 --preset veryslow --tune animation --open-gop --b-adapt 2 --b-pyramid normal -f -2:0
--bitrate 2500 --aq-mode 1 -p 2 --stats v.stats -t 2 --no-fast-pskip --cqm flat --non-deterministic

x265 10-bit encoding settings (pass 1 & pass 2), 2.5Mbit example:

--y4m -D 10 --fps 24000/1001 -p veryslow --open-gop --bframes 16 --b-pyramid --bitrate 2500 --rect
--amp --aq-mode 3 --no-sao --qcomp 0.75 --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0
--rdoq-level 1 --tu-inter-depth 4 --tu-intra-depth 4 --ctu 32 --max-tu-size 16 --pass 1
--slow-firstpass --stats v.stats --sar 1 --range full

--y4m -D 10 --fps 24000/1001 -p veryslow --open-gop --bframes 16 --b-pyramid --bitrate 2500 --rect
--amp --aq-mode 3 --no-sao --qcomp 0.75 --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0
--rdoq-level 1 --tu-inter-depth 4 --tu-intra-depth 4 --ctu 32 --max-tu-size 16 --pass 2
--stats v.stats --sar 1 --range full

Since x265 can only read raw YUV and Y4M, the source video is being fed to it via [libavs’] avconv tool, piping it into x265. The avconv commandline for that looks as follows:

$ avconv -r 24000/1001 -i input.h264 -f yuv4mpegpipe -pix_fmt yuv420p -r 24000/1001 - 2>/dev/null

If you want to do something similar, but you don’t like avconv, you can use [ffmpeg] as a replacement, the options are completely the same. Note that you should always specify the correct frame rates (-r) for input and output, or the bitrate setting of the encoder will be applied wrongly!

x264 on the other hand was linked against libav directly, using its decoding capabilities without any workarounds.

Let’s compare:

“The Garden of Words” has a lot of rain. This is a central story element of the 46 minute movie, and it’s hard on any encoder, because a lot of stuff is moving on screen all the time. Let’s take a look at such a scene for our first side-by-side comparison. Each comparison is done in two rows: H.264/AVC first (including the source material), and below that H.265/HEVC, also including the source.

Let’s go:

Scene 1, H.264/AVC encoded by x264 0.148.x:

Scene 1, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

It has been said that x265 performs specifically well at two things: Very high resolutions (which we don’t have here) and low bitrates. And yep, it shows. When comparing the 1Mbit shots, it becomes clear pretty quickly that x265 manages to preserve more detail for the parts with lots of motion. x264 on the other hand starts to wash out the scene pretty severely, smearing out some raindrops, spray water and parts of the foliage. Also, it’s pretty bad around the outlines as well, but that’s true for both encoders. You can spot that easily with all the aliasing artifacts on the raindrops.

Moving up a notch, it becomes very hard to distinguish between the two. When zooming in you can still spot a few minor differences (note the kids umbrella, even if it’s not marked), but it’s quite negligible. Here I’m already starting to think x265 might not be worth it in all cases. There are still differences between the two 2.5Mbit shots and the original however, see the red areas of the umbrella and the most low-contrast, dark parts of the foliage.

At 5Mbit, I really can’t see any difference anymore. Maybe the colors are a little off or something, but when seen in motion, distinguishing between the two and the original becomes virtually impossible. Given that we just threw a really difficult scene at x264 and x265, this should be a trend to continue throughout the whole test.

Now, even more rain:

Scene 2, H.264/AVC encoded by x264 0.148.x:

Scene 2, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

Now this is extreme at 1Mbit! Looking at H.264, the spray water on top of the cable can’t even be told apart from the cloud in the background anymore. Detail loss all over the scene is catastrophic in comparison to the original. Tons of raindrops are simply gone entirely, and the texture details on the tower and the angled brick wall of the house to the left? Almost completely washed out and smeared.

Now, let’s look at H.265 @ 1Mbit. The spray water is also pretty bad, but it’s amazing how much more detail was preserved overall. Sure, there are still parts of the raindrops missing, but it’s much, much closer to the original. We can now see details on the walls as well, even the steep angle one on the left. The only serious issue is the red light bleeding at the tower. There is very little red there in the original, so I’m not sure what happened there. x264 does this as well, but x265 is a bit worse.

At the next level, the differences are less pronounced again, but there is still a significant enough improvement when going from x264 to x265 at 2.5Mbit: The spray water on the cable becomes more well-defined, and more rain is being preserved. Also, the textures on the walls are a tiny little bit more detailed and crisp. Once again though, x265 is bleeding too much red light at the tower.

Since it’s noticeably not fully on the level of the source still, let’s look at 5Mbit briefly. x265 is able to preserve a tiny little bit more rain detail here, coming extremely close to the original. In motion, you can’t really see the difference however.

Now, let’s get steamy:

Scene 3, H.264/AVC encoded by x264 0.148.x:

Scene 3, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

1Mbit first again: Let me just say: It’s ugly. x264 pretty much messes up the steam coming from the iron. We get lots of block artifacts now. Some of the low-contrast patterns on the ironing board are being smeared out at a pretty terrible level. Also, the bokeh background partly shows block artifacts and banding. x265 produces quite a lot of banding here itself, but no blocks. Also, outlines and sharp contrasts are more well-defined, and the low contrast part is done noticeably better.

At 2.5Mbit, the patterns repeat themselves now. The steam is only slightly better with x265, outlines are slightly more well-defined, and the low-contrast patterns are slightly more visible. For some of the blurred parts, x265 seems to be a tiny little bit to prone to banding though, in a very few spots, x264 might be just that 1% better. Overall however, x265 wins this, and even if it’s just for the better outlines.

At 5Mbit, you really need to zoom in and analyze very small details, e.g. around the outer border of the steam. Yes, x265 does better again. But you’d not really be able to notice this when watching.

How about we go cry a little bit:

Scene 4, H.264/AVC encoded by x264 0.148.x:

Scene 4, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

Cutting onions would be a classic fun part in a slice-of-life anime. Here, it’s just kitchen work. And quite the bad looking one for H.264 at 1Mbit. The letters on the knife are partly lost completely, becoming unreadable. The onion parts that fly off are visibly worse than when encoded with x265 at the same bitrate. Also, x264 produced block artifacts in the blurred bokeh areas again, that simply aren’t there with x265.

On the next level, the two come much closer to each other. However, x265 simply does the outlines better. Less artifacts and sharper, just like with the writing on the knifes’ blade as well. The issues with the bokeh are nearly gone. What’s left is a negligible amount of blocking for x264 and banding for x265. Not really noticeable however.

Finally, at 5Mbit, x265 shows those ever so slightly more well-done outlines. But that’s about it, the rest looks nice for both, and pretty much identical to the source.

Now, please, dig in:

Scene 5, H.264/AVC encoded by x264 0.148.x:

Scene 5, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

Let’s keep this short: x264 does blocking, and bad transitions/outlines. x265 does it better. Plain and simple.

At 2.5Mbit, x265 nearly reaches quality transparency when compared to the original, something x264 falls short of, just a bit. While x265 does the outlines and the steam part quite like in the original frame, x264 rips the outlines apart a bit too much, and slight block artifacts can again be seen for the steam part.

At 5Mbit, x264 still shows some blocking artifacts in a part that most lossy image/video compression algorithms traditionally suck at: The reds. While not true for all human beings, most eyes perceive much finer gradients for greens, then blues, and do worst with reds. Meaning, our eyes have an unequal sensitivity distribution when it comes to colors. So image and video codecs try to save bitrate in the reds first, because supposedly it’d be less noticeable. To me subjectively, x265 achieves transparency here, meaning it looks just like the original. x264 doesn’t manage entirely. Close, but not not entirely.

Next:

Scene 6, H.264/AVC encoded by x264 0.148.x:

Scene 6, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

This is a highly static scene, with only few moving parts, so there is some rain again, and some shadow cast by raindrops as well. Now, for the static parts, incremental B frames really work wonders here. Most detail is being preserved by both encoders. What’s supposed to be happening is happening: The encoders save bit rate where the human eye can’t easily tell: In the parts where stuff is moving around very quickly. That’s how we lose a lot of raindrop shadows and some drops as well. x264 seems to have trouble separating the scene into even smaller macro blocks though? Not sure if that’s the reason, but a lot of mesh detail for the basket on the balcony on the top right is lost – x265 does better there! This is maybe because x264 couldn’t distinct the moving drops from the static background so well anymore?

At 2.5Mbit, the scenes become almost indistinguishable. The more static content we have, the easier it gets of course, so the transparency threshold becomes lower. And if you ask me, both of them reach perfect quality at 5Mbit.

Let’s throw another hard one at them for the last round:

Scene 7, H.264/AVC encoded by x264 0.148.x:

Scene 7, H.265/HEVC encoded by x265 1.9+15-425b583f25db:

Enough rain already? Pffh! Here we have a lot of foliage and low contrast added to the mix. And it gets smeared a lot by x264, rain detail lost, fine details of the bushes turning into green mud, that’s how it goes. x265 also loses too much detail here (I mean, 1Mbit is really NOT much), but again, it fares quite a bit better.

At 2.5Mbit, both encoders do very well. Somehow, this scene doesn’t seem to penalize x264 that much at the medium level. You’d really need your magnifying glass to find the spots where x265 still does better, which surprises me a bit for this scene. And finally, at 5Mbit – if you ask me – visual transparency is reached for both x264 and x265.

Final thoughts:

Clearly it’s true what a lot of people have been saying. x265 rocks at low bitrates, if configured correctly. But that isn’t gonna give me perfect quality or anything. Yeah, it sucks less – much less – than x264 in that department, but at a higher 2.5Mbit, where both start looking quite decent, x265 having just a slight edge… it becomes hardly justifiable to use it, simply because it’s that much slower to run it at decent settings.

Also, you need to take device compatibility into account. Sure, a powerful PC can always play the stuff. No matter if it’s some UNIX, Linux, MacOS X or Windows. But how about video game consoles? Older TVs? That kind of thing. Most of those can only play H.264/AVC. Or course, if you’re only using your PC and you have a lot of time and electricity to burn, then hey – why not?

But I’ll have to think really long and really hard about whether I want to replace x264 with x265 at this given point in time. Overall, it might not be practical enough on my current hardware yet. Maybe I’d need an AVX/AVX2-capable processor, as x265 has tons of optimizations for those instruction set extensions. But I’m gonna stay on my Xeon X5690 for quite a while, so SSE 4.2 is the latest I have.

I’d say, if you can stand some quality degradation, then x265 might be the way to go, as it can give you much smaller file sizes at lower bitrates with slight degradation.

If you’re aiming for high bitrates and quality, it might not be worth it right now, at least for 1080p. It’s been said the tables are turning once more when going up to 4K and UHD, but I haven’t tested that yet, as all my content – both Anime and live action movies – are “low resolution” *cough* 1080p or 720p.

Addendum:

Screenshots were taken using the latest stable mplayer 1.3.0 on Linux. Thank god they’re bundling it with ffmpeg now, making things much easier. This choice was made because mplayer can easily grab screenshots from specific spots in a video in an automated fashion. I used framesteps for this, like this:

$ mplayer ./TEST-H.265/HEVC-1mbit.mkv -nosound -vf framestep=24 \
 -vo png:z=9:outdir=./screenshots/1mbit/H.265/HEVC/:prefix=H.265/HEVC-1mbit-

This will decode every 24th frame of the video file TEST-H.265/HEVC-1mbit.mkv, and grab it into a .png file with maximum lossless compression as supported by mplayer. The .png files will be prefixed with a user-defined string and numbered, like H.265/HEVC-1mbit-00000001.pngH.265/HEVC-1mbit-00000002.png and so on, until the end of file is reached.

To encode the full size screenshots to WebP, the most recent [libwebp-0.5.0], or rather one of its companion tools – cwebp – was used as follows:

$ cwebp -z 9 -lossless -q 100 -noalpha -mt ./input.png -o ./output.webp

Now… somebody wanna grant me remote access to some quad socket Haswell-EX Xeon box for free?

No?

Meh… :roll:

Jan 272016
 

HakuNeko logoJust yesterday I’ve showed you how to modify and compile the [HakuNeko] Manga Ripper so it can work on FreeBSD 10 UNIX, [see here] for some background info. I also mentioned that I couldn’t get it to build on CentOS 6 Linux, something I chose to investigate today. After flying blind for a while trying to fix include paths and other things, I finally got to the core of the problem, which lies in HakuNekos’ src/CurlRequest.cpp, where the code calls CURLOPT_ACCEPT_ENCODING from cURLs typecheck-gcc.h. This is critical stuff, as [cURL] is the software library needed for actually connecting to websites and downloading the image files of Manga/Comics.

It turns out that CURLOPT_ACCEPT_ENCODING wasn’t always called that. With cURL version 7.21.6, it was renamed to that from the former CURLOPT_ENCODING as you can read [here]. And well, CentOS 6 ships with cURL 7.19.7…

When running $ ./configure && make from the unpacked HakuNeko source tree without fixing anything, you’ll run into this problem:

g++ -c -Wall -O2 -I/usr/lib64/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread -o obj/CurlRequest.cpp.o src/CurlRequest.cpp
src/CurlRequest.cpp: In member function ‘void CurlRequest::SetCompression(wxString)’:
src/CurlRequest.cpp:122: error: ‘CURLOPT_ACCEPT_ENCODING’ was not declared in this scope
make: *** [obj/CurlRequest.cpp.o] Error 1

So you’ll have to fix the call in src/CurlRequest.cpp! Look for this part:

  1. void CurlRequest::SetCompression(wxString Compression)
  2. {
  3.     if(curl)
  4.     {
  5.         curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, (const char*)Compression.mb_str(wxConvUTF8));
  6.         //curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, (const char*)memcpy(new wxByte[Compression.Len()], Compression.mb_str(wxConvUTF8).data(), Compression.Len()));
  7.     }
  8. }

Change CURLOPT_ACCEPT_ENCODING to CURLOPT_ENCODING. The rest can stay the same, as the name is all that has really changed here. It’s functionally identical as far as I can tell. So it should look like this:

  1. void CurlRequest::SetCompression(wxString Compression)
  2. {
  3.     if(curl)
  4.     {
  5.         curl_easy_setopt(curl, CURLOPT_ENCODING, (const char*)Compression.mb_str(wxConvUTF8));
  6.         //curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, (const char*)memcpy(new wxByte[Compression.Len()], Compression.mb_str(wxConvUTF8).data(), Compression.Len()));
  7.     }
  8. }

Save the file, go back to the main source tree and you can do:

  • $ ./configure && make
  • # make install

And done! Works like a charm:

HakuNeko fetching Haiyore! Nyaruko-san on CentOS 6.7 Linux

HakuNeko fetching Haiyore! Nyaruko-san on CentOS 6.7 Linux!

And now, for your convenience I fixed up the Makefile and rpm/SPECS/specfile.spec a little bit to build proper rpm packages as well. I can provide them for CentOS 6.x Linux in both 32-bit as well as 64-bit x86 flavors:

You need to unzip these first, because I was too lazy to allow the rpm file type in my blogging software.

The naked rpms have also been submitted to the HakuNeko developers as a comment to their [More Linux Packages] support ticket which you’re supposed to use for that purpose, so you can get them from there as well. Not sure if the developers will add the files to the projects’ official downloads.

This build of HakuNeko has been linked against the wxWidgets 2.8.12 GUI libraries, which come from the official CentOS 6.7 package repositories. So you’ll need to install wxGTK to be able to use the white kitty:

  • # yum install wxGTK

After that you can install the .rpm package of your choice. For a 64-bit system for instance, enter the folder where the hakuneko_1.3.12_el6_x86_64.rpm file is, run # yum localinstall ./hakuneko_1.3.12_el6_x86_64.rpm and confirm the installation.

Now it’s time to have fun using HakoNeko on your Enterprise Linux system! Totally what RedHat intended you to use it for! ;) :roll:

Jan 262016
 

HakuNeko logoSince I’ve started using FreeBSD as a Linux and Windows replacement, I’ve naturally always been looking at porting my “known good” software over to the UNIX OS, or at replacing it by something that gets the job done without getting on my nerves too much at the same time. For most parts other than TrueCrypt, that was quite achievable, even though I had to endure varying degrees of pain getting there. Now, my favorite Manga / Comic ripper on Windows, [HakuNeko] was the next piece of software on the list. It’s basically just a more advanced Manga website parser and downloader based on stuff like [cURL], [OpenSSL] or the [wxWidgets] GUI libraries.

I didn’t even know this until recently (shame on me for never looking closely), but HakuNeko is actually free software licensed under the MIT license. Unfortunately, the source code and build system are quite Linux- and Windows-centric, and there exist neither packages nor ports of it for FreeBSD UNIX. Actually, the code doesn’t even build on my CentOS 6.7 Linux right now (I have yet to figure out the exact problem), but I managed to fix it up so it can compile and work on FreeBSD! And here’s how, step by step:

1.) Prerequisites

Please note that from here on, terminal commands are shown in this form: $ command or # command. Commands starting with a $ are to be executed as a regular user, and those starting with # have to be executed as the superuser root.

Ok, this has been done on FreeBSD 10.2 x86_32 using HakuNeko 1.3.12, both are current at the time of writing. I guess it might work on older and future releases of FreeBSD with different releases of HakuNeko as well, but hey, who knows?! That having been said, you’ll need the following software on top of FreeBSD for the build system to work (I may have missed something here, if so, just install the missing stuff like shown below):

  • cURL
  • GNU sed
  • GNU find
  • bash
  • OpenSSL
  • wxWidgets 2.8.x

Stuff that’s not on your machine can be fetched and installed as root from the official package repository, Like e.g.: # pkg install gsed findutils bash wx28-gtk2 wx28-gtk2-common wx28-gtk2-contrib wx28-gtk2-contrib-common

Of course you’ll need the HakuNeko source code as well. You can get it from official sources (see the link in first paragraph) or download it directly from here in the version that I’ve used successfully. If you take my version, you need 7zip for FreeBSD as well: # pkg install p7zip.

Unpack it:

  • $ 7z x hakuneko_1.3.12_src.7z (My version)
  • $ tar -xzvf hakuneko_1.3.12_src.tar.gz (Official version)

The insides of my archive are just vanilla as well however, so you’ll still need to do all the modifications by yourself.

2.) Replace the shebang lines in all scripts which require it

Enter the unpacked source directory of HakuNeko and open the following scripts in your favorite text editor, then replace the leading shebang lines #!/bin/bash with #!/usr/local/bin/bash:

  • ./configure
  • ./config_clang.sh
  • ./config_default.sh
  • ./res/parsers/kissanime.sh

It’s always the first line in each of those scripts, see config_clang.sh for example:

  1. #!/bin/bash
  2.  
  3. # import setings from config-default
  4. . ./config_default.sh
  5.  
  6. # overwrite settings from config-default
  7.  
  8. CC="clang++"
  9. LD="clang++"

This would have to turn into the following (I also fixed that comment typo while I was at it):

  1. #!/usr/local/bin/bash
  2.  
  3. # import settings from config-default
  4. . ./config_default.sh
  5.  
  6. # overwrite settings from config-default
  7.  
  8. CC="clang++"
  9. LD="clang++"

3.) Replace all sed invocations with gsed invocations in all scripts which call sed

This is needed because FreeBSDs sed and Linux’ GNU sed aren’t exactly that compatible in how they’re being called, different options and all.

In the text editor vi, the expression :%s/sed /gsed /g can do this globally over an entire file (mind the whitespaces, don’t omit them!). Or just use a convenient graphical text editor like gedit or leafpad for searching and replacing all occasions. The following files need sed replaced with gsed:

  • ./configure
  • ./res/parsers/kissanime.sh

4.) Replace all find invocations with gfind invocations in ./configure

Same situation as above with GNU find, like :%s/find /gfind /g or so, but only in one file:

  • ./configure

5.) Fix the make check

This is rather cosmetic in nature as $ ./configure won’t die if this test fails, but you may still wish to fix this. Just replace the string make --version with gmake --version (there is only one occurrence) in:

  • ./configure

6.) Fix the DIST variables’ content

I don’t think that this is really necessary either, but while we’re at it… Change the DIST=linux default to DIST=FreeBSD in:

  • ./configure

Again, only one occurrence.

7.) Run ./configure to create the Makefile

Enough with that, let’s run the first part of the build tools:

  • $ ./configure --config-clang

Notice the --config-clang option? We could use GCC as well, but since clang is FreeBSDs new and default platform compiler, you should stick with that whenever feasible. It works for HakuNeko, so we’re gonna use the default compiler, which means you don’t need to install the entire GCC for just this.

There will be error messages looking quite intimidating, like the basic linker test failing, but you can safely ignore those. Has something to do with different function name prefixes in FreeBSDs libc (or whatever, I don’t really get it), but it doesn’t matter.

However, there is one detail that the script will get wrong, and that’s a part of our include path. So let’s handle that:

8.) Fix the includes in the CFLAGS in the Makefile

Find the line containing the string CFLAGS = -c -Wall -O2 -I/usr/lib64/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread or similar in the newly created ./Makefile. After the option -O2 add the following: -I/usr/local/include. So it looks like this: CFLAGS = -c -Wall -O2 -I/usr/local/include -I/usr/lib64/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread. That’s it for the Makefile.

9.) Fix the Linux-specific conditionals across the C++ source code

And now the real work starts, because we need to fix up portions of the C++ code itself as well. While the code would build and run fine on FreeBSD, those relevant parts are hidden behind some C++ preprocessor macros/conditionals looking for Linux instead. Thus, important parts of the code can’t even compile on FreeBSD, because the code only knows Linux and Windows. Fixing that isn’t extremely hard though, just a bit of copy, paste and/or modify. First of all, the following files need to be fixed:

  • ./src/MangaConnector.cpp
  • ./src/Logger.cpp
  • ./src/MangaDownloaderMain.cpp
  • ./src/MangaDownloaderConfiguration.cpp

Now, what you should look for are all conditional blocks which look like #ifdef __LINUX__. Each will end with an #endif line. Naturally, there are also #ifdef __WINDOWS__ blocks, but those don’t concern us, as we’re going to use the “Linux-specific” code, if you can call it that. Let me give you an example right out of MangaConnector.cpp, starting at line #20:

  1. #ifdef __LINUX__
  2. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  3. #endif

Now given that the Linux code builds just fine on FreeBSD, the most elegant and easier version would be to just alter all those #ifdef conditionals to inclusive #if defined ORs, so that they trigger for both Linux and FreeBSD. If you do this, the block from above would need to change to this:

  1. #if defined __LINUX__ || __FreeBSD__
  2. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  3. #endif

Should you ever want to create different code paths for Linux and FreeBSD, you can also just duplicate it. That way you could later make changes for just Linux or just FreeBSD separately:

  1. #ifdef __LINUX__
  2. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  3. #endif
  4. #ifdef __FreeBSD__
  5. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  6. #endif

Whichever way you choose, you’ll need to find and update every single one of those conditional blocks. There are three in Logger.cpp, three in MangaConnector.cpp, two in MangaDownloaderConfiguration.cpp and again three in MangaDownloaderMain.cpp. Some are more than 10 lines long, so make sure to not make any mistakes if duplicating them.

Note that you can maybe extend compatibility even further with additional directives like __OpenBSD__ or __NetBSD__ for additional BSDs or __unix__ for a wide range of UNIX systems like AIX or HP-UX. None of which has been tested by me of course.

When all of that is done, it’s compile and install time:

10.) Compile and install

You can compile as a regular user, but the installation needs root by default. I’ll assume you’ll want to install HakuNeko system-wide, so, we’ll leave the installation target directories on their defaults below /usr/local/. While sitting in the unpacked source directory, run:

  • $ gmake
  • # gmake install

If nothing starts to crash and burn, this should compile and install the code. clang will show some warnings during compilation, but you can safely ignore that.

11.) Start up the white kitty

The installation procedure will also conveniently update your window manager as well, if you’re using panels/menus. Here it’s Xfce4:

HakuNeko is showing up as an "Internet" tool

HakuNeko (“White Cat”) is showing up as an “Internet” tool. Makes sense.

With the modifications done properly it should fire up just fine after initializing its Manga connectors:

HakuNeko with the awesomeness that is "Gakkou Gurashi!" being selected from the HTTP source "MangaReader"

HakuNeko with the awesomeness that is “Gakkou Gurashi!” being selected from the HTTP source [MangaReader].

Recently the developers have also added [DynastyScans] as a source, which provides access to partially “rather juicy” Yuri Dōjinshi (self-published amateur and sometimes semi-professional works) of well-known Manga/Anime, if you’re into that. Yuri, that is (“girls love”). Mind you, not all, but a lot of the stuff on DynastyScans can be considered NSFW and likely 18+, just as a word of warning:

HakuNeko fetching a Yuru Yuri Dōjinshi from DynastyScans, bypassing their download limits by not fetching packaged ZIPs - it works perfectly!

HakuNeko fetching a Yuru Yuri Dōjinshi called “Secret Flowers” from DynastyScans, bypassing their download limits by not fetching packaged ZIPs – it works perfectly!

Together with a good comic book reader that can read both plain JPEG-filled folders and stuff like packaged .cbz files, HakuNeko makes FreeBSD a capable comic book / Manga reading system. My personal choice for a reader to accompany HakuNeko would be [QComicBook], which you can easily get on FreeBSD. There are others you can fetch from the official package repository as well though.

Final result:

HakuNeko and QComicBook make a good team on FreeBSD UNIX

HakuNeko and QComicBook make a good team on FreeBSD UNIX – I like the reader even more than ComicRack on Windows.

And, at the very end, one more thing, even though you’re likely going to be aware of this already: Just like Anime fansubs, fan-translated Manga or even Dōjinshi are sitting in a legal grey zone, as long as the book in question hasn’t been licensed in your country. It’s being tolerated, but if it does get licensed, ownership of a fan-translated version will likely become illegal, which means you should actually buy the stuff at that point in time.

Just wanted to have that said as well.

Should you have trouble building HakuNeko on FreeBSD 10 UNIX (maybe because I missed something), please let me know in the comments!