Mar 132017

puTTY logoAnyone who has logged in to a UNIX or Linux machine remotely coming from a Windows box probably knows puTTY, which is practically “the” SSH and telnet client for Windows. In conjunction with a small X11 server like Xming you can even do Remote X. To my surprise, a new version has been released just last month, as a colleague told me! So there is version 0.68 now, and it comes in both a 32-bit and a 64-bit flavor.

Of course I had to try the 64-bit version on XP x64, and it did fail to execute:

64-bit puTTY failure on XP x64

A classic: 64-bit puTTY failure on XP x64, because it’s “not a valid Win32 application”

Out of curiosity, I fetched the puTTY source code from [here], because I thought I could just compile & link it myself. Building for 32-bit proved to be relatively easy; Just load the corresponding solution file into Microsoft Visual Studio 2010, build the whole project, done. But I wanted 64-bit! So I created a x64 build target and gave it a shot, but it couldn’t find the _addcarry_u64 intrinsic.

After a bit of searching on the web, it became clear that the intrinsics header of Visual Studio didn’t provide it. It’s too old, you need Visual Studio 2013 or newer for that. Funny part is, puTTY only comes with project files for 2010 and 2012, how are they building their x64 version? No idea. Maybe they’re linking against a different library version or something.

One attempt that I was going to make (build it with VS2013, linking against an older platform SDK) isn’t done yet, because I need to prepare my Windows 7 VM for it. I did manage to compile and run puTTY as 64-bit code on XP x64 by hacking up their program though! In the unpacked source tree, open sshbn.h and take a look at line #70:

expand/collapse source code
  1. #elif defined _MSC_VER && defined _M_AMD64
  3.   /*
  4.    * 64-bit BignumInt, using Visual Studio x86-64 compiler intrinsics.
  5.    *
  6.    * 64-bit Visual Studio doesn't provide very much in the way of help
  7.    * here: there's no int128 type, and also no inline assembler giving
  8.    * us direct access to the x86-64 MUL or ADC instructions. However,
  9.    * there are compiler intrinsics giving us that access, so we can
  10.    * use those - though it turns out we have to be a little careful,
  11.    * since they seem to generate wrong code if their pointer-typed
  12.    * output parameters alias their inputs. Hence all the internal temp
  13.    * variables inside the macros.
  14.    */
  16.   #include 
  17.   typedef unsigned char BignumCarry; /* the type _addcarry_u64 likes to use */
  18.   typedef unsigned __int64 BignumInt;
  19.   #define BIGNUM_INT_BITS 64
  20.   #define BignumADC(ret, retc, a, b, c) do                \
  21.       {                                                   \
  22.           BignumInt ADC_tmp;                              \
  23.           (retc) = _addcarry_u64(c, a, b, &ADC_tmp);      \
  24.           (ret) = ADC_tmp;                                \
  25.       } while (0)
  26.   #define BignumMUL(rh, rl, a, b) do              \
  27.       {                                           \
  28.           BignumInt MULADD_hi;                    \
  29.           (rl) = _umul128(a, b, &MULADD_hi);      \
  30.           (rh) = MULADD_hi;                       \
  31.       } while (0)
  32.   #define BignumMULADD(rh, rl, a, b, addend) do                           \
  33.       {                                                                   \
  34.           BignumInt MULADD_lo, MULADD_hi;                                 \
  35.           MULADD_lo = _umul128(a, b, &MULADD_hi);                         \
  36.           MULADD_hi += _addcarry_u64(0, MULADD_lo, (addend), &(rl));     \
  37.           (rh) = MULADD_hi;                                               \
  38.       } while (0)
  39.   #define BignumMULADD2(rh, rl, a, b, addend1, addend2) do                \
  40.       {                                                                   \
  41.           BignumInt MULADD_lo1, MULADD_lo2, MULADD_hi;                    \
  42.           MULADD_lo1 = _umul128(a, b, &MULADD_hi);                        \
  43.           MULADD_hi += _addcarry_u64(0, MULADD_lo1, (addend1), &MULADD_lo2); \
  44.           MULADD_hi += _addcarry_u64(0, MULADD_lo2, (addend2), &(rl));    \
  45.           (rh) = MULADD_hi;                                               \
  46.       } while (0)

I just commented out the entire codeblock using that modern _addcarry_u64 intrinsic, and replaced it with the code being used for the 32-bit version:

  1. #elif defined _MSC_VER && defined _M_AMD64
  3.   /* 32-bit BignumInt, using Visual Studio __int64 as BignumDblInt 
  4.    * This is compatible with VS2010 & VS2012 for building a x86_64
  5.    * version of puTTY (no __int128 with those compilers).
  6.    */
  8.   typedef unsigned int BignumInt;
  9.   #define BIGNUM_INT_BITS  32
  10.   #define DEFINE_BIGNUMDBLINT typedef unsigned __int64 BignumDblInt

I built that and it works, even though I keep thinking I should be using a wide 128-bit data type here (just like the original x64 code), but then we don’t have __int128 in MSVC before 2013, and I’m on 2010. And I don’t know how to use SSE registers in that context with things like __m128, which is why I left it alone. Looking good anyway:

puTTY 64-bit XP x64 version logged in to Windows 2000

puTTY 64-bit XP x64 version logged in to Windows 2000 using a modern SSH server (click to enlarge)


puTTY 64-bit XP x64 version logged in to FreeBSD 10.3 UNIX

puTTY 64-bit XP x64 version logged in to FreeBSD 10.3 UNIX (click to enlarge)

In any case, here is a complete 64-bit build of puTTY that works on NT5.2 operating systems like Windows XP Professional x64 Edition or Windows Server 2003 x64:

  • [puTTY 0.68][1] (x64 version for NT5.2, portable without installer)

Maybe I’ll try to build a version with VS2013 on Windows 7 for the same platform target, we’ll see. But at least this works!

Oh and… No, I don’t really think anyone actually needs a 64-bit version of puTTY ;). Plus the 32-bit one works just fine on XP x64 / Server 2003 out-of-the-box anyway. But hey… You know… :roll:

[1] puTTY is © 1997-2017 Simon Tatham and is licensed under the MIT license

Mar 012017

Notepadqq @ CentOS 6 Linux logoIt’s rather rare for me to look for a replacement of some good Windows software for Linux/UNIX instead of the other way around, but the source code editor [Notepad++] is one example of such software. The program gedit on the Gnome 2 desktop environment of my old CentOS 6 enterprise Linux isn’t bad, but it isn’t exactly good either. The thing I was missing most was a search & replace engine capable of regular expressions.

Of course, vi can do it, but at times, vi can be a bit hard to use, so I kinda looked for a Notepad++ replacement. What I found was [Notepadqq], which is basically a clone using the Qt5 UI. However, this editor is clearly made for more modern systems, but I still looked for a way to get it to compile and run on my CentOS 6.8 x86_64 Linux system. And I found one. Here are the most important prerequisites:

  • A new enough GCC (I used 6.2.0), because the v4.4.7 platform compiler won’t work with the modern C++ stuff
  • Qt5 libraries from the [EPEL] repository
  • git

First, you’ll want a new compiler. That part is surprisingly easy, but a bit time consuming. First, download a fresh GCC tarball from a server on the [mirrors list], those are in the releases/ subdirectory, so a file like gcc-6.3.0.tar.bz2 (My version is still 6.2.0). It seems Notepadqq only needs some GCC 5, but since our platform compiler won’t cut it anyway, why not just use the latest?

Now, once more, this will take time, could well be hours, so you might wanna do the compilation step over night, the last step needs root privileges:

$ tar -xzvf ./gcc-6.3.0.tar.bz2
$ cd ./gcc-6.3.0/
$ ./configure --program-suffix="-6.3.0"
$ make
# make install

And when you do this, please never forget to add a --program-suffix for the configuration step!  You might seriously fuck things up if you miss that! So double-check it!

When that’s finally done, let’s handle Qt5 next. I’ll be using a binary distribution to make things easy, and yeah, I didn’t just install the necessary packages, I got the whole Qt5 blob instead, too lazy to do the cherry picking. Oh, and if you don’t have it, add git as well:

# yum install
# yum install qt5* git

I assume # yum install qt5-qtwebkit qt5-qtwebkit-devel qt5-qtsvg qt5-qtsvg-devel qt5-qttools qt5-qttools-devel should also be enough according to the requirements, but I didn’t try that. Now, enter a free directory or one you generally use for source code and fetch the latest Notepadqq version (this will create a subfolder we’ll cd to):

$ git clone
$ cd ./notepadqq

After that, we need to make sure that we’ll be using the correct compiler and that we’re linking against the correct libraries that came with it (like*). To do that, set the following environment variables, assuming you’re using the bash as your shell (use lib/ instead of lib64/ folders if you’re on 32-bit x86):

$ export CC="gcc-6.3.0"
$ export CXX="g++-6.3.0"
$ export CPP="cpp-6.3.0"
$ export CFLAGS="-I/usr/local/include/ -L/usr/local/lib64/"
$ export CXXFLAGS="-I/usr/local/include/ -L/usr/local/lib64/"
$ export LDFLAGS="-L/usr/local/lib64/"

The C-related settings are probably not necessary as Qt5 stuff should be pure C++, but you’ll never know, so let’s play it safe.

With that we’re including and linking against the correct libraries and we’ll be using our modern compiler as well. Time to actually compile Notepadqq. To do that, we’ll still need to tell it where to find the Qt5 versions of the qmake and lrelease binaries, but luckily, we can solve that with some simple configuration options. So, let’s do this, the last step requires root privileges again, from within the notepadqq/ directory that git clone created for us:

$ ./configure --qmake /usr/bin/qmake-qt5 --lrelease /usr/bin/lrelease-qt5
$ make
# make install

Now, there are some weird linking issues that I never got fixed on CentOS (some developer please tell me how, I have the same crap when building x265!). Because of that we still can’t launch Notepadqq as-is, we need to give it an LD_LIBRARY_PATH to find the proper libraries at runtime. Let’s just create an executable launcher script /usr/local/sbin/ for that. Edit it and enter the following code:

LD_LIBRARY_PATH="/usr/local/lib64" /usr/local/bin/notepadqq "$@"

Use this as your launcher script for Notepadqq and you’re good to go with your Notepad++ replacement on good old CentOS 6.x:

Running Notepadqq on CentOS 6 Linux

Running the latest Notepadqq on CentOS 6 Linux with Qt5 version 5.6.1, state 2017-03-01

Now, let’s see whether it’s even that good actually… :roll:

Aug 232016

UnrealIRCd logoOne of the services I’ve been running on for years now has been the IRC server UnrealIRCd. It’s available for Linux, UNIX and also Windows, so it’s a pretty neat choice I think. A few days ago however, a user had notified me, that his client couldn’t connect when using SSL/TLS encryption after an update of the software. I’m pretty sure this was due to the OpenSSL developers disabling the SSL v3 protocol by default. So his client only had TLS and my old UnrealIRCd 3.x only had SSL v3 => handshake failure:

error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

So what now? Just shoving a newer SSL library under my IRC server wouldn’t work in a stable fashion. So far, the only software I have ever seen which can be “magically” upgraded to modern protocols and ciphers this way was the Gene6 FTP server. All the way from OpenSSL 0.9.6 to 1.0.2. No idea how they did it.

Two options: Have users recompile their libraries and clients to enable SSL v3 (yeah, as if…), or try and backport a current (=2016-07-28) UnrealIRCd 4 to my server. One that supports both modern TLS v2 with modern ciphers as well as good old SSL v3, so legacy clients may connect in an encrypted fashion as well.

Why backport? Because it’s freaking Windows 2000 (and no, newer versions do *not* work), and UnrealIRCd dropped support for that, so I absolutely needed to recompile the server and several libraries it depends on. Now that was one wild ride for a user like me, I’m telling you.

Ah yes, this isn’t exactly a good step-by-step guide or anything, so in case you just wanna grab the files, scroll all the way down! If you want to know a few of the details… I don’t even remember all the things I did, but let’s see…


Here’s what you need:

  1. The Microsoft [Visual C++ 2008 runtime SP1 redistributable package] (only on the system where the server is supposed to run, not on the build system)
  2. Microsoft VisualStudio 2008 (I guess 2010 also works, as long as you have the v90 toolset available)
  3. Perl. I used [Strawberry Perl 5.24].
  4. The latest UnrealIRCd [dev package]. It’s for UnrealIRCd v3.4, but that doesn’t matter.
  5. The UnrealIRCd [source code]. I used the current/bugfixed version 4.0.5 for this build.
  6. A precompiled version of pcre2 supporting Windows 2000, I only found one eligible one [here]. (I failed to recompile/relink pcre2 properly, even with the version from the dev package :( )
  7. The stock [tre 0.8.0 library] source code, because it supports VS2008. The version shipped with the dev package doesn’t.
  8. The latest [OpenSSL library] source code, it’ll serve as a replacement for the older one shipped with the dev package.

If you cannot obtain Visual Studio 2008 via any (legal!) means, that’d probably mean you’re out of luck though. Luckily, I got all versions from Microsofts MSDNAA / DreamSpark program, but if you’re stuck on something like VS2012, 2013 or 2015, I cannot help you. Maybe this can still work out, but you’ll still need the 2008 version to get the v90 toolset (I guess, not an expert here…)


There are quite a few, but here are the ones that I still remember:

1.) Additional headers are required to link some of the software, there are free ones available. You can grab them [here]. Put them into the VC\include\ subdirectory of your Visual Studio 2008 installation folder. On top of those two, inttypes.h and stdint.h you’ll also need unistd.h, but that one’s easy: Just make a copy of io.h in that same folder and rename that copy to unistd.h and you’re done.

2.) First, cURL-SSL was built with the nmake options ENABLE_IPV6=no and ENABLE_IDN=no set. IPv6 support on Windows 2000 does exist by using an [experimental update], but it’s function calls are different than with Microsofts’ final version, so it’s unusable by most software. Also, IDN support is only available [for Windows XP and later], so internationalized domain names using non-ASCII characters don’t work. UnrealIRCd is to be linked against this version.

3.) tre replaced with latest stock tre 0.8.0 and recompiled, UnrealIRCd is to be linked against this build.

4.) Before building OpenSSL, it may need modifications to its makefile ms\ntdll.mak, which is generated by the ms\do_nasm step described in OpenSSLs INSTALL.W32, depending on your requirements. It is here where you can enable older, weaker ciphers and the older SSL v3/v2 protocols. Enable these deprecated version only if you absolutely need them!

Look for line 21 (Note, that the ^ line breaks aren’t in the file originally, it’s all in one line. I just added them here for readability purposes):

  1. CFLAG= /MD /Ox /O2 /Ob2 -DOPENSSL_THREADS  -DDSO_WIN32 -W3 -Gs0 -GF -Gy -nologo ^

You could replace this with the following, allowing weak ciphers and SSL v3, but not SSL v2 for example:

  1. #CFLAG= /MD /Ox /O2 /Ob2 -DOPENSSL_THREADS  -DDSO_WIN32 -W3 -Gs0 -GF -Gy -nologo ^
  8. CFLAG= /MD /Ox /O2 /Ob2 -DOPENSSL_THREADS  -DDSO_WIN32 -W3 -Gs0 -GF -Gy -nologo ^

Compile as shown in the documentation, and install somewhere.

5.) Before UnrealIRCd can use the new version of OpenSSL it may need modifications to match the ones patched into the OpenSSL makefile. By default, it will also block stuff like SSL v3. Enter its source tree and open ssl\ssl.c, then locate lines 245 and 321, which will look like this:

  1. SSL_CTX_set_options(ctx_server, SSL_OP_NO_SSLv3);

Just comment that out:

  1. /** SSL_CTX_set_options(ctx_server, SSL_OP_NO_SSLv3); **/

If you enabled SSLv2 as well and want the IRC server to be able to use it, do the same for lines 244 and 320, look for this…

  1. SSL_CTX_set_options(ctx_client, SSL_OP_NO_SSLv2);

…and comment it out again:

  1. /** SSL_CTX_set_options(ctx_client, SSL_OP_NO_SSLv2); **/

Now compile and link as shown in the UnrealIRCd documentation. Like the developers I’d recomment assembling a proper command line for this, as editing the makefile all the time can be cumbersome, especially if you’re running into trouble along the way.

What else?

Some of the VS project files may be preconfigured for platform toolsets you don’t have (like v100, v110, etc.) or may be set to produce a Debug build by default. Make sure you’re using only the v90 toolset and produce only Release builds. To learn how, check out the Visual Studio documentation online. It’s not that hard for the stuff you need to build with the GUI.

And here is the file:

Note that I may have done something horribly wrong along the way with this, because it really works only on Windows 2000. This is not how it should be. But launching it on a newer operating system yields something like this:

UnrealIRCd runtime error on anything greater than or equal to Windows XP

Yeah… umm… riiight…

And after pressing OK, this:

UnrealIRCd runtime error on anything greater than or equal to Windows XP #2


I searched for those errors on the web for a little, but couldn’t find anything that would’ve told me why it breaks like this on “modern” operating systems, yet still works on Windows 2000. Oh, the build system was XP x64 by the way. Well, it doesn’t really matter, the standard build of the developers works on XP+ anyway, and this works only on Windows 2000. Mission accomplished in any case.

In this incarnation, the server can support SSL v3 as well as TLS v1.2 protocols and supports the following ciphers:


The necessary tools for creating an SSL/TLS certificate and for installing a Windows service for the server are also included (openssl.exe, unrealsvc.exe).


UnrealIRCd and the software it was linked against in this case is released under the following licenses:

Any modifications to any of the software packages above as posted on this page are hereby licensed under the same license as the original software before modifications were applied. When downloading any unmodified source code, you’ll have to patch it yourself before building for a Windows 2000 platform target.

And what now?

Well, I guess my server supports IRC+TLS for all modern clients now, so yay! ;) URLs are the same as before: [irc+ssl://] with SSL v3/TLS v1.2 or [irc://] if you wish to connect without any encryption enabled, all plain text.

Jan 262016

HakuNeko logoSince I’ve started using FreeBSD as a Linux and Windows replacement, I’ve naturally always been looking at porting my “known good” software over to the UNIX OS, or at replacing it by something that gets the job done without getting on my nerves too much at the same time. For most parts other than TrueCrypt, that was quite achievable, even though I had to endure varying degrees of pain getting there. Now, my favorite Manga / Comic ripper on Windows, [HakuNeko] was the next piece of software on the list. It’s basically just a more advanced Manga website parser and downloader based on stuff like [cURL], [OpenSSL] or the [wxWidgets] GUI libraries.

I didn’t even know this until recently (shame on me for never looking closely), but HakuNeko is actually free software licensed under the MIT license. Unfortunately, the source code and build system are quite Linux- and Windows-centric, and there exist neither packages nor ports of it for FreeBSD UNIX. Actually, the code doesn’t even build on my CentOS 6.7 Linux right now (I have yet to figure out the exact problem), but I managed to fix it up so it can compile and work on FreeBSD! And here’s how, step by step:

1.) Prerequisites

Please note that from here on, terminal commands are shown in this form: $ command or # command. Commands starting with a $ are to be executed as a regular user, and those starting with # have to be executed as the superuser root.

Ok, this has been done on FreeBSD 10.2 x86_32 using HakuNeko 1.3.12, both are current at the time of writing. I guess it might work on older and future releases of FreeBSD with different releases of HakuNeko as well, but hey, who knows?! That having been said, you’ll need the following software on top of FreeBSD for the build system to work (I may have missed something here, if so, just install the missing stuff like shown below):

  • cURL
  • GNU sed
  • GNU find
  • bash
  • OpenSSL
  • wxWidgets 2.8.x

Stuff that’s not on your machine can be fetched and installed as root from the official package repository, Like e.g.: # pkg install gsed findutils bash wx28-gtk2 wx28-gtk2-common wx28-gtk2-contrib wx28-gtk2-contrib-common

Of course you’ll need the HakuNeko source code as well. You can get it from official sources (see the link in first paragraph) or download it directly from here in the version that I’ve used successfully. If you take my version, you need 7zip for FreeBSD as well: # pkg install p7zip.

Unpack it:

  • $ 7z x hakuneko_1.3.12_src.7z (My version)
  • $ tar -xzvf hakuneko_1.3.12_src.tar.gz (Official version)

The insides of my archive are just vanilla as well however, so you’ll still need to do all the modifications by yourself.

2.) Replace the shebang lines in all scripts which require it

Enter the unpacked source directory of HakuNeko and open the following scripts in your favorite text editor, then replace the leading shebang lines #!/bin/bash with #!/usr/local/bin/bash:

  • ./configure
  • ./
  • ./
  • ./res/parsers/

It’s always the first line in each of those scripts, see for example:

  1. #!/bin/bash
  3. # import setings from config-default
  4. . ./
  6. # overwrite settings from config-default
  8. CC="clang++"
  9. LD="clang++"

This would have to turn into the following (I also fixed that comment typo while I was at it):

  1. #!/usr/local/bin/bash
  3. # import settings from config-default
  4. . ./
  6. # overwrite settings from config-default
  8. CC="clang++"
  9. LD="clang++"

3.) Replace all sed invocations with gsed invocations in all scripts which call sed

This is needed because FreeBSDs sed and Linux’ GNU sed aren’t exactly that compatible in how they’re being called, different options and all.

In the text editor vi, the expression :%s/sed /gsed /g can do this globally over an entire file (mind the whitespaces, don’t omit them!). Or just use a convenient graphical text editor like gedit or leafpad for searching and replacing all occasions. The following files need sed replaced with gsed:

  • ./configure
  • ./res/parsers/

4.) Replace all find invocations with gfind invocations in ./configure

Same situation as above with GNU find, like :%s/find /gfind /g or so, but only in one file:

  • ./configure

5.) Fix the make check

This is rather cosmetic in nature as $ ./configure won’t die if this test fails, but you may still wish to fix this. Just replace the string make --version with gmake --version (there is only one occurrence) in:

  • ./configure

6.) Fix the DIST variables’ content

I don’t think that this is really necessary either, but while we’re at it… Change the DIST=linux default to DIST=FreeBSD in:

  • ./configure

Again, only one occurrence.

7.) Run ./configure to create the Makefile

Enough with that, let’s run the first part of the build tools:

  • $ ./configure --config-clang

Notice the --config-clang option? We could use GCC as well, but since clang is FreeBSDs new and default platform compiler, you should stick with that whenever feasible. It works for HakuNeko, so we’re gonna use the default compiler, which means you don’t need to install the entire GCC for just this.

There will be error messages looking quite intimidating, like the basic linker test failing, but you can safely ignore those. Has something to do with different function name prefixes in FreeBSDs libc (or whatever, I don’t really get it), but it doesn’t matter.

However, there is one detail that the script will get wrong, and that’s a part of our include path. So let’s handle that:

8.) Fix the includes in the CFLAGS in the Makefile

Find the line containing the string CFLAGS = -c -Wall -O2 -I/usr/lib64/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread or similar in the newly created ./Makefile. After the option -O2 add the following: -I/usr/local/include. So it looks like this: CFLAGS = -c -Wall -O2 -I/usr/local/include -I/usr/lib64/wx/include/gtk2-unicode-release-2.8 -I/usr/include/wx-2.8 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -D__WXGTK__ -pthread. That’s it for the Makefile.

9.) Fix the Linux-specific conditionals across the C++ source code

And now the real work starts, because we need to fix up portions of the C++ code itself as well. While the code would build and run fine on FreeBSD, those relevant parts are hidden behind some C++ preprocessor macros/conditionals looking for Linux instead. Thus, important parts of the code can’t even compile on FreeBSD, because the code only knows Linux and Windows. Fixing that isn’t extremely hard though, just a bit of copy, paste and/or modify. First of all, the following files need to be fixed:

  • ./src/MangaConnector.cpp
  • ./src/Logger.cpp
  • ./src/MangaDownloaderMain.cpp
  • ./src/MangaDownloaderConfiguration.cpp

Now, what you should look for are all conditional blocks which look like #ifdef __LINUX__. Each will end with an #endif line. Naturally, there are also #ifdef __WINDOWS__ blocks, but those don’t concern us, as we’re going to use the “Linux-specific” code, if you can call it that. Let me give you an example right out of MangaConnector.cpp, starting at line #20:

  1. #ifdef __LINUX__
  2. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  3. #endif

Now given that the Linux code builds just fine on FreeBSD, the most elegant and easier version would be to just alter all those #ifdef conditionals to inclusive #if defined ORs, so that they trigger for both Linux and FreeBSD. If you do this, the block from above would need to change to this:

  1. #if defined __LINUX__ || __FreeBSD__
  2. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  3. #endif

Should you ever want to create different code paths for Linux and FreeBSD, you can also just duplicate it. That way you could later make changes for just Linux or just FreeBSD separately:

  1. #ifdef __LINUX__
  2. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  3. #endif
  4. #ifdef __FreeBSD__
  5. wxString MCEntry::invalidFileCharacters = wxT("/\r\n\t");
  6. #endif

Whichever way you choose, you’ll need to find and update every single one of those conditional blocks. There are three in Logger.cpp, three in MangaConnector.cpp, two in MangaDownloaderConfiguration.cpp and again three in MangaDownloaderMain.cpp. Some are more than 10 lines long, so make sure to not make any mistakes if duplicating them.

Note that you can maybe extend compatibility even further with additional directives like __OpenBSD__ or __NetBSD__ for additional BSDs or __unix__ for a wide range of UNIX systems like AIX or HP-UX. None of which has been tested by me of course.

When all of that is done, it’s compile and install time:

10.) Compile and install

You can compile as a regular user, but the installation needs root by default. I’ll assume you’ll want to install HakuNeko system-wide, so, we’ll leave the installation target directories on their defaults below /usr/local/. While sitting in the unpacked source directory, run:

  • $ gmake
  • # gmake install

If nothing starts to crash and burn, this should compile and install the code. clang will show some warnings during compilation, but you can safely ignore that.

11.) Start up the white kitty

The installation procedure will also conveniently update your window manager as well, if you’re using panels/menus. Here it’s Xfce4:

HakuNeko is showing up as an "Internet" tool

HakuNeko (“White Cat”) is showing up as an “Internet” tool. Makes sense.

With the modifications done properly it should fire up just fine after initializing its Manga connectors:

HakuNeko with the awesomeness that is "Gakkou Gurashi!" being selected from the HTTP source "MangaReader"

HakuNeko with the awesomeness that is “Gakkou Gurashi!” being selected from the HTTP source [MangaReader].

Recently the developers have also added [DynastyScans] as a source, which provides access to partially “rather juicy” Yuri Dōjinshi (self-published amateur and sometimes semi-professional works) of well-known Manga/Anime, if you’re into that. Yuri, that is (“girls love”). Mind you, not all, but a lot of the stuff on DynastyScans can be considered NSFW and likely 18+, just as a word of warning:

HakuNeko fetching a Yuru Yuri Dōjinshi from DynastyScans, bypassing their download limits by not fetching packaged ZIPs - it works perfectly!

HakuNeko fetching a Yuru Yuri Dōjinshi called “Secret Flowers” from DynastyScans, bypassing their download limits by not fetching packaged ZIPs – it works perfectly!

Together with a good comic book reader that can read both plain JPEG-filled folders and stuff like packaged .cbz files, HakuNeko makes FreeBSD a capable comic book / Manga reading system. My personal choice for a reader to accompany HakuNeko would be [QComicBook], which you can easily get on FreeBSD. There are others you can fetch from the official package repository as well though.

Final result:

HakuNeko and QComicBook make a good team on FreeBSD UNIX

HakuNeko and QComicBook make a good team on FreeBSD UNIX – I like the reader even more than ComicRack on Windows.

And, at the very end, one more thing, even though you’re likely going to be aware of this already: Just like Anime fansubs, fan-translated Manga or even Dōjinshi are sitting in a legal grey zone, as long as the book in question hasn’t been licensed in your country. It’s being tolerated, but if it does get licensed, ownership of a fan-translated version will likely become illegal, which means you should actually buy the stuff at that point in time.

Just wanted to have that said as well.

Should you have trouble building HakuNeko on FreeBSD 10 UNIX (maybe because I missed something), please let me know in the comments!

Sep 022015

colorecho logoRecently, I am doing a lot of audio/video transcoding, and for that I’m using tools like mkvextract, mkvmerge, x264, avconv and fghaacenc on the command line. Mostly this is to get rid of old files that are either very inefficiently encoded – like DivX3 or MPEG-2 – or are just too large/high-bitrate for what the content’s worth. Since I do have large batches of relatively similar files, I wrote myself a few nice loops that would walk through an entire stack of videos and process all of it automatically. Because I want to see at first glance how far through a given video stack a script is, I wanted to output some colored notifications on the shell after each major processing step, telling me the current video number. That way, I could easily see how far the job had progressed. We’re talking processes here that take multiple days, so it makes sense.

Turns out that this was harder than expected. In the old DOS days, you would just load ANSI.SYS for that, and get ANSI color codes to work with. The same codes still work in modern UNIX terminals. But not within the Windows cmd for whatever reason. Now there is an interface for that kind of stuff, but seemingly no tools to use it. Like e.g. the echo command simply can’t make use of it.

Now there is project porting ANSI codes back to the cmd, and it’s called [ANSICON]. Basically just a DLL hooked into the cmd. Turns out I can’t use that, because I’m already using [clink] to make my cmd more UNIX-like and actually usable, which also is additional code hooked into the cmd, and the two confict with each other. More information about that [here].

So how the hell can I do this then? Turns out there is a batch script that can do it, but it’s a very outlandish hack that doesn’t seem to want to play nice with my shells if I do @echo off instead of echo off as well. I guess it’s clinks work breaking stuff here, but I’m not sure. Still, here is the code, as found by [Umlüx]:

expand/collapse source code
  2. SETLOCAL EnableDelayedExpansion
  4. FOR /F "tokens=1,2 delims=#" %%A IN ('"PROMPT #$H#$E# & ECHO ON & FOR %%b IN (1) DO REM"') DO (
  5.   SET "DEL=%%A"
  6. )
  8. ECHO say the name of the colors, dont read
  10. CALL :ColorText 0a "blue"
  11. CALL :ColorText 0C "green"
  12. CALL :ColorText 0b "red"
  13. ECHO.
  14. CALL :ColorText 19 "yellow"
  15. CALL :ColorText 2F "black"
  16. CALL :ColorText 4e "white"
  18. GOTO :eof
  20. :ColorText
  22. ECHO OFF
  24.  "%~2"
  25. findstr /v /a:%1 /R "^$" "%~2" NUL
  26. DEL "%~2" > NUL 2>&1
  28. GOTO :eof

So since those two solutions didn’t work for me, what else could I try?

Then I had that crazy idea: Since the interface is there, why not write a little C program (actually ended up being C++ instead) that uses it? Since I was on Linux at the time, I tried to write it there and attempt my first cross-compile for Windows using a pre-built [MinGW-w64] compiler, that luckily just played along nicely on my CentOS 6.6 Linux. You can get pre-built binaries for 32-Bit Windows targets [here], and for 64-bit Windows targets [here]. Thing is, I know zero C++, so that took some time and it’s quite crappy, but here’s the code:

expand/collapse source code
  1. #include <iostream>
  2. #include "windows.h"
  4. // Set NT commandline color:
  5. void SetColor(int value) {
  6.     SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), value);
  7. }
  9. int main(int argc, char *argv[]) {
  10.   SetColor(atoi(argv[2]));
  11.   std::cout << "\r\n" << argv[1] << "\r\n";
  12.   SetColor(7);
  13.   return 0;
  14. }

Update 2016-06-22: Since I wanted to work with wide character strings in recent days (so, non-ANSI Unicode characters), my colorecho command failed, because it couldn’t handle them at all. There was no wide character support. I decided to update the program to also work with that using Windows-typical UCS-2le/UTF-16, here’s the code:

expand/collapse source code
  1. #include <iostream>
  2. #include <io.h>
  3. #include <fcntl.h>
  4. #include "windows.h"
  6. // Set NT commandline color:
  7. void SetColor(int value) {
  8.     SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), value);
  9. }
  11. // Unicode main
  12. int wmain(int argc, wchar_t *argv[]) {
  13.   SetColor(_wtoi(argv[2]));
  14.   _setmode(_fileno(stdout), _O_U16TEXT);      // Set stdout to UCS-2le/UTF-16
  15.   std::wcout << "\r\n" << argv[1] << "\r\n";  // Write unicode
  16.   SetColor(7);                                // Set color back to default
  17.   return 0;
  18. }

I also updated the download section and the screenshots further below. End of update.

This builds nicely using MinGW on Linux for 64-bit Windows targets. Like so: While sitting in the MinGW directory (I put my code there as echo.cpp as well, for simplicities’ sake), run: ./bin/x86_64-w64-mingw32-g++ echo.cpp -o colorecho-x86_64.exe. If you want to target 32-bit Windows instead, you’d need to use the proper MinGW for that (it comes in a separate archive): ./bin/i686-w64-mingw32-g++ echo.cpp -o colorecho-x86_64.exe

When building like that, you’d also need to deploy and libgcc_s_seh-1.dll along with the EXE files though. The DLLs can be obtained from your local MinGW installation directory, subfolder ./x86_64-w64-mingw32/lib/ for the 64-bit or ./i686-w64-mingw32/lib/ for the 32-bit compiler. If you don’t want to do that and rather rely on Microsofts own C++ redistributables, you can also compile it with Microsoft VisualStudio Express, using its pre-prepared command line. You can find that here, if you have Microsofts VisualStudio installed – version 2010 in my case:

VC2010 command lines

The command lines of Visual Studio (click to enlarge)

Here, the “Visual Studio Command Prompt” is for a 32-bit build target, and “Visual Studio x64 Win64 Command Prompt” is for building 64-bit command line programs. Choose the appropriate one, then change into a directory where you have echo.cpp and run the following command: cl /EHsc /W4 /nologo /Fecolorecho.exe echo.cpp, giving you the executable colorecho.exe.

Alternatively, you can just download my pre-compiled versions here:

Update 2016-06-22: And here are the Unicode versions:

End of update.

Note that this tool does exactly what I need it to do, but it’ll likely not do exactly what you’d need it to do. Like e.g. the line breaks it adds before and after its output. That’s actually a job for the shells echo command to do, not some command line tool. But I just don’t care. So that’s why it’s basically a piece of crap for general use. The syntax is as follows, as shown for the 64-bit VC10 build:

colorecho-x86_64-vc10.exe "I am yellow!" 14

When run, it looks like this:

Colorecho running

colorecho invoked on the Windows cmd

So the first parameter is the string to be echoed, the second one is the color number. That number is 2-digit and can affect both the foreground and the background. Here the first 16 of them, which are foreground only:

  • 0: Black
  • 1: Dark blue
  • 2: Dark green
  • 3: Dark cyan
  • 4: Dark red
  • 5: Dark purple
  • 6: Dark yellow
  • 7: Light grey
  • 8: Dark grey
  • 9: Light blue
  • 10: Light green
  • 11: Light cyan
  • 12: Light red
  • 13: Light purple
  • 14: Light yellow
  • 15: White

If you go higher than that, you’ll also start changing the background colors and you’ll get different combinations of foreground and background colorization.

The background colors actually follow the same order, black, dark blue, dark green, etc. Numbers from 0..15 are on black background, numbers from 16..31 are on a dark blue background and so on. This makes up pretty much the same list:

  • 0..15: 16 foreground colors as listed above on the default black background, and
  • 16..31: on a dark blue background
  • 32..47: on a dark green background
  • 48..63: on a dark cyan background
  • 64..79: on a dark red background
  • 80..95: ona dark purple background
  • 96..111: on a dark yellow background
  • 112..127: on a light grey background
  • 128..143: on a dark grey background
  • 144..159: on a light blue background
  • 160..175: on a light green background
  • 176..191: on a light cyan background
  • 192..207: on a light red background
  • 208..223: on a light purple background
  • 224..239: on a light yellow background
  • 240..255: on a white background

Going over 255 will simply result in an overflow causing the system to start from the beginning, so 256 is equal to 0, 257 to 1, 260 to 4 and so on.

Update 2016-06-22: With the Unicode version, you can now even do stupid stuff like this:

colorecho unicode with DejaVu Sans Mono font on Windows cmd

It could be useful for some people? Maybe?

Note that this has been rendered using the DejaVu Sans Mono font, you can get it from [here]. Also, you need to tell the Windows cmd which fonts to allow (only monospaced TTFs work), you can read how to do that [here], it’s a relatively simple registry hack. I have yet to find a font that would work and also render CJK characters though, but with the fonts I myself have available at the moment, asian stuff won’t work, you’ll just get some boxes instead:

colorecho unicode CJK fail

Only characters which have proper glyphs for all desired codepoints in the font in use can be rendered, at least on Windows XP.

Note that this is not a limitation of colorecho, it does output the characters correctly, “ですね” in this case. It’s just that DejaVu Sans Mono has no font glyphs to actually draw them on screen. If you ever come across a neat cmd-terminal-compatible font that can render most symbols and CJK, please tell me in the comments, I’d be quite happy about that! End of update.

If somebody really wants to use this (which I doubt, but hey), but wishes it to do things in a more sane way, just request it in the comments. Or, if you can, just take the code, change and recompile it by yourself. You can get Microsofts VisualStudio Express – in the meantime called [VisualStudio Community] – free of charge anyway, and MinGW is free software to begin with.