Mar 132017
puTTY logo

Anyone who has logged in to a UNIX or Linux machine remotely coming from a Windows box probably knows puTTY, which is practically “the” SSH and telnet client for Windows. In conjunction with a small X11 server like Xming you can even do Remote X. To my surprise, a new version has been released just last month, as a colleague told me! So there is version 0.68 now, and it comes in both a 32-bit and a 64-bit flavor.

Of course I had to try the 64-bit version on XP x64, and it did fail to execute:

64-bit puTTY failure on XP x64

A classic: 64-bit puTTY failure on XP x64, because it’s “not a valid Win32 application”

Out of curiosity, I fetched the puTTY source code from [here], because I thought I could just compile & link it myself. Building for 32-bit proved to be relatively easy; Just load the corresponding solution file into Microsoft Visual Studio 2010, build the whole project, done. But I wanted 64-bit! So I created a x64 build target and gave it a shot, but it couldn’t find the _addcarry_u64 intrinsic.

After a bit of searching on the web, it became clear that the intrinsics header of Visual Studio didn’t provide it. It’s too old, you need Visual Studio 2013 or newer for that. Funny part is, puTTY only comes with project files for 2010 and 2012, how are they building their x64 version? No idea. Maybe they’re linking against a different library version or something.

One attempt that I was going to make (build it with VS2013, linking against an older platform SDK) isn’t done yet, because I need to prepare my Windows 7 VM for it. I did manage to compile and run puTTY as 64-bit code on XP x64 by hacking up their program though! In the unpacked source tree, open sshbn.h and take a look at line #70:

expand/collapse source code
  1. #elif defined _MSC_VER && defined _M_AMD64
  3.   /*
  4.    * 64-bit BignumInt, using Visual Studio x86-64 compiler intrinsics.
  5.    *
  6.    * 64-bit Visual Studio doesn't provide very much in the way of help
  7.    * here: there's no int128 type, and also no inline assembler giving
  8.    * us direct access to the x86-64 MUL or ADC instructions. However,
  9.    * there are compiler intrinsics giving us that access, so we can
  10.    * use those - though it turns out we have to be a little careful,
  11.    * since they seem to generate wrong code if their pointer-typed
  12.    * output parameters alias their inputs. Hence all the internal temp
  13.    * variables inside the macros.
  14.    */
  16.   #include 
  17.   typedef unsigned char BignumCarry; /* the type _addcarry_u64 likes to use */
  18.   typedef unsigned __int64 BignumInt;
  19.   #define BIGNUM_INT_BITS 64
  20.   #define BignumADC(ret, retc, a, b, c) do                \
  21.       {                                                   \
  22.           BignumInt ADC_tmp;                              \
  23.           (retc) = _addcarry_u64(c, a, b, &ADC_tmp);      \
  24.           (ret) = ADC_tmp;                                \
  25.       } while (0)
  26.   #define BignumMUL(rh, rl, a, b) do              \
  27.       {                                           \
  28.           BignumInt MULADD_hi;                    \
  29.           (rl) = _umul128(a, b, &MULADD_hi);      \
  30.           (rh) = MULADD_hi;                       \
  31.       } while (0)
  32.   #define BignumMULADD(rh, rl, a, b, addend) do                           \
  33.       {                                                                   \
  34.           BignumInt MULADD_lo, MULADD_hi;                                 \
  35.           MULADD_lo = _umul128(a, b, &MULADD_hi);                         \
  36.           MULADD_hi += _addcarry_u64(0, MULADD_lo, (addend), &(rl));     \
  37.           (rh) = MULADD_hi;                                               \
  38.       } while (0)
  39.   #define BignumMULADD2(rh, rl, a, b, addend1, addend2) do                \
  40.       {                                                                   \
  41.           BignumInt MULADD_lo1, MULADD_lo2, MULADD_hi;                    \
  42.           MULADD_lo1 = _umul128(a, b, &MULADD_hi);                        \
  43.           MULADD_hi += _addcarry_u64(0, MULADD_lo1, (addend1), &MULADD_lo2); \
  44.           MULADD_hi += _addcarry_u64(0, MULADD_lo2, (addend2), &(rl));    \
  45.           (rh) = MULADD_hi;                                               \
  46.       } while (0)

I just commented out the entire codeblock using that modern _addcarry_u64 intrinsic, and replaced it with the code being used for the 32-bit version:

  1. #elif defined _MSC_VER && defined _M_AMD64
  3.   /* 32-bit BignumInt, using Visual Studio __int64 as BignumDblInt 
  4.    * This is compatible with VS2010 & VS2012 for building a x86_64
  5.    * version of puTTY (no __int128 with those compilers).
  6.    */
  8.   typedef unsigned int BignumInt;
  9.   #define BIGNUM_INT_BITS  32
  10.   #define DEFINE_BIGNUMDBLINT typedef unsigned __int64 BignumDblInt

I built that and it works, even though I keep thinking I should be using a wide 128-bit data type here (just like the original x64 code), but then we don’t have __int128 in MSVC before 2013, and I’m on 2010. And I don’t know how to use SSE registers in that context with things like __m128, which is why I left it alone. Looking good anyway:

puTTY 64-bit XP x64 version logged in to Windows 2000

puTTY 64-bit XP x64 version logged in to Windows 2000 using a modern SSH server (click to enlarge)


puTTY 64-bit XP x64 version logged in to FreeBSD 10.3 UNIX

puTTY 64-bit XP x64 version logged in to FreeBSD 10.3 UNIX (click to enlarge)

In any case, here is a complete 64-bit build of puTTY that works on NT5.2 operating systems like Windows XP Professional x64 Edition or Windows Server 2003 x64:

  • [puTTY 0.68][1] (x64 version for NT5.2, portable without installer)

Maybe I’ll try to build a version with VS2013 on Windows 7 for the same platform target, we’ll see. But at least this works!

Oh and… No, I don’t really think anyone actually needs a 64-bit version of puTTY ;). Plus the 32-bit one works just fine on XP x64 / Server 2003 out-of-the-box anyway. But hey… You know… :roll:

[1] puTTY is © 1997-2017 Simon Tatham and is licensed under the MIT license

Apr 082016

H.265/HEVC logoPreviously, I have shown you [how to compile x265 on Windows] using Microsoft Visual Studio 2010 in a way that results in binaries compatible with Windows NT 5.1/5.2, or in other words: Windows XP, XP x64 and Windows Server 2003. And while that works for most purposes, today I’d like to show you how to build an actual multilib binary, that can handle all three color bit depths supported by x265, the standardized 8- and 10-bit (MAIN and MAIN10 profiles) as well as 12-bit (MAIN12 profile). With that, it’s all in one exe instead of three. As before though, multilib x265 is only supported on 64-Bit Windows. But first, once again…

1.) Giving you the binaries

There were a lot of improvements since the last version I published back in February of course, also performance-wise. So here’s the current version from Multicoreware for both 32-bit and 64-bit Windows, compiled with MSVC 2010 SP1 and yasm 1.3.0. This requires the Microsoft Visual C++ 2010 runtime to work, see previous article:

This time around, the binaries have been tested as well! On regular 32-bit Windows XP, only fundamental binary compatibility was tested. However, all versions, so the 32-Bit one and the 64-Bit multilib ones have been ran through a 2-pass ABR encoding test with output verification for 8-bit color depth (32- & 64-bit) as well as 10- and 12-bit color depths (64-bit only) on Windows XP Professional x64 Edition using the following command line (see previous post for details):

avconv -r 24000/1001 -i input.h264 -f yuv4mpegpipe -pix_fmt yuv420p -r 24000/1001 - 2>NUL^ 
 | .\x265.exe - --y4m -D 10 --fps 24000/1001 -p veryslow^
 --open-gop --ref 6 --bframes 16 --b-pyramid --bitrate 2500 --rect --amp --aq-mode 3^
 --no-sao --qcomp 0.75 --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0^
 --rdoq-level 1 --tu-inter-depth 4 --tu-intra-depth 4 --ctu 32 --max-tu-size 16 --pass 1^
 --slow-firstpass --stats v.stats --sar 1 --range full -o pass1.h265 & avconv^
 -r 24000/1001 -i input.h264 -f yuv4mpegpipe -pix_fmt yuv420p -r 24000/1001 - 2>NUL^
 | .\x265.exe - --y4m -D 10 --fps 24000/1001 -p veryslow --open-gop --ref 6^
 --bframes 16 --b-pyramid --bitrate 2500 --rect --amp --aq-mode 3 --no-sao --qcomp 0.75^
 --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0 --rdoq-level 1^
 --tu-inter-depth 4 --tu-intra-depth 4 --ctu 32 --max-tu-size 16 --pass 2^
 --stats v.stats --sar 1 --range full -o pass2.h265

Needless to say, they should work fine on Windows Vista/7/8/8.1/10/Server 2008/Server 2012/HS 2007/HS 2011 as well.

From time to time, I’ll release new binaries, so you might wanna check back every few months or so, if you’re interested. You can also request a build in the comments if you’re growing impatient and need a specific version more quickly because of some bugfix / feature improvement in x265.

2.) Compiling an XP/2003-compatible x265 multilib binary yourself

First, please look at the previous article I linked to in the beginning, point 2. You need the software prerequisites listed in 2a and you might still wish to read through 2b to understand some of the stuff better. You don’t need to actually run any of the commands shown there though.

Now, the multilib build is done a bit differently from the rest, as everything is scripted, so this is 100% command line work, no graphical cmake, no running the full Visual Studio IDE. Usually, with all software in place, sitting in the root directory of the x265 source tree, all you need to do is to go to build\vc10-x86_64\ and run ./multilib.bat. This won’t give us an XP/2003-compatible binary however, and the reason lies within the build script multilib.bat, here is the stock version:

expand/collapse source code (multilib.bat)
  1. @echo off
  2. if "%VS100COMNTOOLS%" == "" (
  3.   msg "%username%" "Visual Studio 10 not detected"
  4.   exit 1
  5. )
  7. call "%VS100COMNTOOLS%\..\..\VC\vcvarsall.bat"
  9. @mkdir 12bit
  10. @mkdir 10bit
  11. @mkdir 8bit
  13. @cd 12bit
  14. cmake -G "Visual Studio 10 Win64" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON
  15. if exist x265.sln (
  16.   MSBuild /property:Configuration="Release" x265.sln
  17.   copy/y Release\x265-static.lib ..\8bit\x265-static-main12.lib
  18. )
  20. @cd ..\10bit
  21. cmake -G "Visual Studio 10 Win64" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF
  22. if exist x265.sln (
  23.   MSBuild /property:Configuration="Release" x265.sln
  24.   copy/y Release\x265-static.lib ..\8bit\x265-static-main10.lib
  25. )
  27. @cd ..\8bit
  28. if not exist x265-static-main10.lib (
  29.   msg "%username%" "10bit build failed"
  30.   exit 1
  31. )
  32. if not exist x265-static-main12.lib (
  33.   msg "%username%" "12bit build failed"
  34.   exit 1
  35. )
  36. cmake -G "Visual Studio 10 Win64" ../../../source -DEXTRA_LIB="x265-static-main10.lib;x265-static-main12.lib" -DLINKED_10BIT=ON -DLINKED_12BIT=ON
  37. if exist x265.sln (
  38.   MSBuild /property:Configuration="Release" x265.sln
  39.   :: combine static libraries (ignore warnings caused by winxp.cpp hacks)
  40.   move Release\x265-static.lib x265-static-main.lib
  41.   LIB.EXE /ignore:4006 /ignore:4221 /OUT:Release\x265-static.lib x265-static-main.lib x265-static-main10.lib x265-static-main12.lib
  42. )
  44. pause

So I took all the options from the files generated by the original cmake when doing the normal build, and added them to the script to ensure our output binaries would be XP-compatible. This is the fixed build script:

expand/collapse source code (multilib.bat, patched for XP/2003)
  1. @echo off
  2. if "%VS100COMNTOOLS%" == "" (
  3.   msg "%username%" "Visual Studio 10 not detected"
  4.   exit 1
  5. )
  7. call "%VS100COMNTOOLS%\..\..\VC\vcvarsall.bat"
  9. @mkdir 12bit
  10. @mkdir 10bit
  11. @mkdir 8bit
  13. @cd 12bit
  15. if exist x265.sln (
  16.   MSBuild /property:Configuration="Release" x265.sln
  17.   copy/y Release\x265-static.lib ..\8bit\x265-static-main12.lib
  18. )
  20. @cd ..\10bit
  22. if exist x265.sln (
  23.   MSBuild /property:Configuration="Release" x265.sln
  24.   copy/y Release\x265-static.lib ..\8bit\x265-static-main10.lib
  25. )
  27. @cd ..\8bit
  28. if not exist x265-static-main10.lib (
  29.   msg "%username%" "10bit build failed"
  30.   exit 1
  31. )
  32. if not exist x265-static-main12.lib (
  33.   msg "%username%" "12bit build failed"
  34.   exit 1
  35. )
  36. cmake -DCMAKE_BUILD_TYPE="Release" -DCMAKE_CONFIGURATION_TYPES="Release" -G "Visual Studio 10 Win64" ../../../source -DWINXP_SUPPORT=ON -DEXTRA_LIB="x265-static-main10.lib;x265-static-main12.lib" -DLINKED_10BIT=ON -DLINKED_12BIT=ON
  37. if exist x265.sln (
  38.   MSBuild /property:Configuration="Release" x265.sln
  39.   :: combine static libraries (ignore warnings caused by winxp.cpp hacks)
  40.   move Release\x265-static.lib x265-static-main.lib
  41.   LIB.EXE /ignore:4006 /ignore:4221 /OUT:Release\x265-static.lib x265-static-main.lib x265-static-main10.lib x265-static-main12.lib
  42. )
  44. pause

You can just rename your original script for backup and put the fixed code in its place, build\vc10-x86_64\multilib.bat, then run it on the command line. If all the required tools are present, it will compile a 12-bit library, then a 10-bit library (both static) and finally an 8-bit binary that will have the other two libraries statically linked in. The final x265.exe can then be found in build\vc10-x86_64\8bit\Release\. To check whether it’s the real thing, look for the bitness by running .\x265.exe --version while sitting in that folder on the command line. You should see something like this:

x265 multilib binary

A x265 multilib binary shows that it’s “8-bit+10-bit+12-bit”

Per-color-channel bitness can be defined with x265s’ command line option -D. So that’d be -D 8, -D 10 or -D 12. Note that only 8- and 10-bit are part of the official Blu-Ray UHD/4k specification however.

3.) A side note

In case you’re new to this, you might not get why “8-bit” and “10-bit” etc. Aren’t color spaces supposed to be 16-bit, 24-bit, 32-bit etc.? Well, it seems that in the world of video processing, people don’t refer to whole color space bitness, but rather individual color channel bitness. So with three channels (red, green, blue for instance), you’d have 8/10/12 bits per channel, so that’s 24-, 30- and 36-bit total, or 16.7 million, 1 billion and 64 billion colors.

The more important part – and the reason why nobody encodes to 12-bit – is the internal arithmetic precision of x265 though (same applies to x264). At 8-bit color depth, arithmetic precision is also at 8-bits. When you hop over to 10-bit, you can’t use 8-bit operations and data types any longer, so everything is done at 16-bit precision. This makes the code slower, but also more efficient in preserving color gradients. Since 10-bit H.265/HEVC is officially a part of Blu-Ray UHD/4k, this would be the sweet spot, unless you’re dealing with devices too slow to play it.

Going to 12-bit won’t boost the precision further, it just gives you more colors, that most of today’s displays won’t be able to show anyway. Not much benefit.

So that’s that.

Have fun! :)

Update: All x265 releases have now been consolidated on [this page]! All future XP- and XP x64-compatible releases of x265 plus a relatively recent version of avconv to act as a decoder for feeding x265 with any kind of input video streams will be posted there as well.

Jan 052015

DirectX logo[1] Quite some time ago – I was probably playing around with some DirectShow audio and video codec packs on my Windows system – I hit a wall in my favorite media player, which is [Mediaplayer Classic Home Cinema], in its 64-Bit version to be precise. I love the player, because it has its own built-in splitters and decoders, its small, light-weight, and it can actually use DXVA1 video acceleration on Windows XP / XP x64 with AMD/ATi and nVidia graphics cards. So yeah, Blu-Ray playback with very little CPU load is possible as long as you deal with the ACSS encryption layer properly. Or decode files from your hard disk instead. But I’m wandering from the subject here.

One day, I launched my 64-Bit MPC-HC, tried to decode a new Blu-Ray movie I got, and all of a sudden, this:

MPC-HC "Failed to query the needed interfaces for playback"

MPC-HC failing to render any file – whether video or audio, showing only a cryptic error message

I tried to get to the bottom of this for weeks. Months later I tried again, but I just couldn’t solve it. A lack of usable debug modes and log files didn’t help either. Also, I failed to properly understand the error message “Failed to query the needed interfaces for playback”. Main reason for my failure was that I thought MPC-HC had it all built in – container splitters, A/V decoders, etc. But still, the 64-Bit version failed. Interestingly, the 32-Bit version still worked fine, on XP x64 in this specific case. Today, while trying to help another guy on the web who had issues with his A/V decoding using the K-Lite codec pack, I launched Microsofts excellent [GraphEdit] tool to build a filter graph to show him how to debug codec problems with Microsofts DirectShow system. You can download the tool easily [here]. It can visualize the entire stack of system-wide DirectShow splitters and decoders on Windows, and can thus help you understand how this shit really works. And debug it.

Naturally, I launched the 32-Bit version, as I’ve been working with 32-Bit A/V tools exclusively since that little incident above – minus the x264 encoder maybe, which has its own 64-Bit libav built in. Out of curiosity, I started the 64-Bit version of GraphEdit, and was greeted with this:

GraphEdit x64 failure due to broken DirectShow core

GraphEdit x64 failure due to broken DirectShow core

“DirectShow core components failed to initialize.”, eh? Now this piqued my interest. Immediately the MPC-HC problem from a few years ago came to my mind, and I am still using the very same system today. So I had an additional piece of information now, which I used to search the web for solutions with. Interestingly, I found that this is linked to the entire DirectShow subsystem being de-registered and thus disabled on the system. I had mostly found people who had this problem for the 32-Bit DirectShow core on 64-Bit Windows 7. Also, I learned the name of the DirectShow core library.


On any 64-Bit system, the 32-Bit version of this library would sit in %WINDIR%\SysWOW64\ with its 64-Bit sibling residing in %WINDIR%\system32\. I thought: What if I just tried to register the core and see what happens? So, with my 64-Bit DirectShow core broken, I just opened a shell with an administrative account, went to %WINDIR%\system32\ and ran regsvr32.exe quartz.dll. And indeed, the libary wasn’t registered/loaded. See here:

Re-registering quartz.dll

Re-registering the 64-Bit version of quartz.dll (click to enlarge)

Fascinating, I thought. Now I don’t know what kind of shit software would disable my entire 64-Bit DirectShow subsystem. Maybe one of those smart little codec packs that usually bring more problems that solutions with them? Maybe it was something else I did to my system? I wouldn’t know what, but it’s not like I can remember everything I did to my systems’ DLLs. Now, let’s try to launch the 64-Bit version of GraphEdit again, with a 64-Bit version of [LAVfilters] installed. That’s basically [libav] on Windows with a DirectShow layer wrapped around and a nice installer shipped with it. YEAH, it’s a codec pack alright. But in my opinion, libav and ffmpeg are the ones to be trusted, just like on Linux and UNIX too. And Android. And iOS. And OSX. Blah. Here we go:

64-Bit GraphEdit being fed the Open Source movie "Big Buck Bunny"

64-Bit GraphEdit being fed the Open Source movie “Big Buck Bunny” (click to enlarge)

And all of a sudden, GraphEdit launches just fine, and presents us a properly working filter graph after having loaded the movie [Bick Buck Bunny] in its 4k/UHD version. We can see the container being split, video and audio streams being picked up by the pins of the respective libav decoders, which feed the streams to a video renderer and a DirectSound output device – which happens to be an X-Fi card in this case. All pins are connected just fine, so this looks good. Now what does it look like in MPC-HC now? Like this (warning – large image, 5MB+, this will take time to load from my server!):

64-Bit MPC-HC playing the 4k/UHD version of "Big Buck Bunny"

64-Bit MPC-HC playing the 4k/UHD version of “Big Buck Bunny” (click to enlarge)

So there we go. It seems MPC-HC does rely on DirectShow after all, at least in that it tries to initialize the subsystem. It can after all also use external filters, or in other words system-wide DirectShow codecs too, where its internal codec suite wouldn’t suffice, if that’s ever the case. So MPC-HC seems to want to talk to DirectShow at play time in any case, even if you haven’t even allowed it to use any external filters/codecs. Maybe its internal codec pack even is DirectShow-based too? And if DirectShow is simply not there, it won’t play anything. At all. And I failed to solve this for years. And today I launch one program in a different bitness than usual, and 5 minutes later, everything works again. For all it takes is something like this:

regsvr32.exe %WINDIR%\system32\quartz.dll

Things can be so easy sometimes, if only you know what the fuck really happened…

The importance and significance of error handling and reporting to the user can never be understated!

But hey! Where there is a shell, there is a way, right? Even on Windows. ;)

[1] © Siegel, K. “DirectX 9 logo design“. Kamal Siegel’s Artwork.

Apr 082014

Windows XP dark age logoSince I love exaggerated drama, I’d like to tell you a little story today. One that repeats itself every now and then, but sometimes it just stirs up shit more than usually. Ok? Here goes: One day, in a relatively nice world (or at least some inhabitants would’ve called it that), there was a young maiden growing up. She was not alone though, but a member of a large family with lots of older relatives, and her name was Windows XP. Having Windows Me and Windows 2000 as her parents (although some voices claim the real father was a guy named “Neptune”, but nobody can prove anything) made her a nice young lass. Since she was the youngest of the family, everybody loved her! Ok, not everybody liked her modern looks, but luckily, mommy 2000’s GDI clothes fit her soon enough, so she was able to dress up more seriously when requested. While their world wasn’t the only one even back then, the family chose to largely ignore or shut out other worlds around them, and Windows XP couldn’t care less. Growing massively in popularity she was soon the hottest thing in her own world, much to her satisfaction of course.

Windows XP on the rise

Windows XP on the rise

Childhood inevitably coming to an end slowly, she grew up to become a 64-Bit operating system even, and being the only one of the family with that feature at the time made her mighty proud. She went a bit international even, and worked on both the x86 and IA-64 Itanium continents, albeit the latter sunk to the ground of the ocean a few years ago, but oh well. At least she could now offer some serious new things like GPT or StorPort drivers, having matured from NT 5.1 to NT 5.2.

XP grows up, all x64 now

XP grows up, all x64 now

When her first sister was born, a young fugly wench named Vista, all that Windows XP had to spare for her was a smirk. With Vista not able to reach XPs and XP x64s glory, she thought herself invincible and walked through her world head high, all peaches and cream! Those other worlds’ people, like Linux, BSD UNIX or even MacOS X she just belittled rather than anything else, so great was her popularity at home by now. By that time Windows XP had grown relatively mature though, and little would she know that the true danger did not come from other worlds really, but from within her own freaking house!

The then-new brat

The then-new brat

Her over-confident smile froze to ice quickly after the birth of her then-youngest sister, named 7. Windows 7 was prettier, had new stuff XP didn’t and nobody really cared about that IA-64 Itanium continent having been flooded anyway. All you hear about that is cynical laughter, as only fools like Hewlett Packard had built their expensive houses there to begin with. Sure, Windows 7 having an even more perfect body than XP she couldn’t wear those nice old GDI clothes that well anymore (they just somehow don’t fit quite so well), but who was to be in opposition to that these days? All older relatives and parents were long gone, and Windows XP stood alone, and it was then when she realized that the end was coming for her, far sooner than she expected, having ignored all the signs…

By all means, she just couldn’t do those fancy new Direct3D 10/11 tricks that 7 could seemingly just pull out of the hat without much effort while even looking good doing so. As fast as the rise came, as long as her reign was, from that point on, there was only one way. Down. Indeed, the grim reaper was coming for her. And well…

For most citizens of the Microsoft world, he paid her the one and final visit today.

Or at least, so they would believe… and indeed, the dusk *is* here, night *is* falling, and for her, there will be no real sunrise ever again. And so, as she stumbles across the graveyard of her ancestors, in tears of despair, suddenly they are calling her down into the icy depths with a wild roar and a cold grip!

The icy depths are calling her

Come down here, where you belong, bitch! Die, just like all of us died before you! (True story guys!!!11 This happened today, on the 8th of April in 2014, no shit! Oh and: Original drawing by [Ian Strandberg].)

So, will she escape from that perilous graveyard? And if so, what for? To live on confused, in a world she can no longer see clearly, whose inhabitants she partially can’t even talk to anymore, a world that had abandoned her? Some legends say, that certain operating systems may carry on forever, despite all odds. Surely she heard those weird stories of ancients from other worlds such as Xenix (a very, very distant relative that shared next to nothing with her) or even OpenVMS. Maybe she too can trick death one last time as she did before already, and find her a place that does not despise her. Could be that that place is right here on this very machine I am typing this on, who knows? ;) And there might be others, who do not fear her being the outcast that she now is.

As she crawls over the mossy walls of that cemetery gravely wounded, suddenly, a bitter grin flits across her face as an evil thought found its way into her mind. She hears herself laugh out load, hysterically even, at the joy of that very thought!

For just recently, a new sister was born. And her name was 8. And some say, she be pretty damn good looking too. Maybe one day, her old rival Windows 7 will join her in her eternal exile, sharing the same bitterness, as she will have seen her own new rival force her out of her own house. Now if that day ever comes, that grin’d go from ear to ear, that much’s for sure!

Two old ladies, detached from the modern world, smiling bitterly at their fate. Somehow, a comforting thought she pondered.

PS.: I have started to write this and do that graveyard picture with a fever at nice 38°C/100°F. As I have now finished this weirdo crap about good old Windows XP, the thermometer is sitting at 38.5°C/102°F and it’s getting seriously hard to concentrate and maintain focus. So parts of this “article” might be unusually strange (ahem) or contain errors. If so, my apologies. I think I can no longer focus on it now, I’ll have to go to bed. I just had to finish this trash first somehow, as the 8th of April 2014, the end of all Microsoft support for Windows XP and XP x64 only comes once! ;) Long live XP x64, against all odds (or just emigrate to Linux or BSD UNIX, probably better anyway)!

Apr 022013

Frog ASPIToday I would like to discuss a few useful tricks for Windows XP x64. Yeah. Again. But this time around, it’s at least partially about stuff that you can’t easily find on the Internet anymore, whether it’s the information or the actual software part that’s missing. There are several software components that are sometimes hard to use or find for XP x64. Some are even hard to set up for regular 32-bit Windows XP. The following solutions will be discussed:

  • 1.) The ASPI Layer (digital media access)
  • 2.) UDF 2.5 (for Blu-Ray and HD-DVD access)
  • 3.) exFAT (some modern cameras etc. need this file system)
  • 4.) EXT2/3/4 (for Linux and UNIX compatibility)
  • 5.) Universal SSD TRIM (keep your SSD clean, tidy and fast, like on modern systems)

So, let’s start with #1:

1.) The ASPI layer:

Edit: Note that in the meantime, the FrogASPI solution described here is no longer the only option. Another one that works even for 64-bit applications accessing the ASPI layer has been found, see further below!

1.a) FrogASPI:

One of those things that have been abandoned by Microsoft is the ASPI layer, the Advanced SCSI Programming Interface. Meant as a way for digital storage media access it was most prominently used to read from optical drives digitally (also on ATA, not just SCSI), so you didn’t need to rip audio CDs via MSCDEX on Win98 or via the crappy analog link from your drive to the sound card. ASPI can also be used to access other types of storage devices, but this is the most important part. Some older software, like my beloved Xing AudioCatalyst (an ancient CD ripper including the Fraunhofer mp3 encoder) might still need ASPI to access audio CDs.

However, Adaptec stopped shipping ASPI drivers for Microsoft Windows after Microsoft had abandoned the API and introduced its own replacement called SPTI, the SCSI PassThrough Interface. As a matter of fact, you can still install Adaptecs ASPI layer on 32-Bit Windows XP, as it includes a working 32-Bit kernel driver. So for 32-Bit XP, it’s still fine. However, there is no such driver for XP x64 (and also not for 32/64-Bit Vista/7/8). So, no ASPI at all?

For a loong time I indeed had to live without it, until I found that french guy named Millenod (That’s his nick name, I will not disclose his real name) who had written a 100% userspace ASPI layer, that would work on any newer Windows operating system on x86, no matter the bitness. This is called FrogASPI, and unfortunately, Millenods website for it is down by now. In its core, it is actually just an SPTI wrapper. Back in those days, I even wrote the guy an email, thanking him for his work. Here is a part of his reply:

“FrogAspi is effectively an SPTI wrapper. I decided to work in “user” mode, instead of kernel ones, for many reasons.. It was for me the fastest way to implement a generic ASPI layer, which is not OS specific as drivers.”

-Millenod, Developer of FrogASPI

After renaming the FrogAspi.dll to the proper name wnaspi32.dll and putting it into %WINDIR%\SysWOW64\ for 64-Bit Windows, it can be used by any ASPI-enabled application. For 32-Bit Windows, please use %WINDIR%\system32! See, what Adaptecs own aspichk.exe has to say about what we just did:

Adaptec ASPI Check

You’ll see that some files are reported as missing. You do not have to care about that though, ASPI32.SYS would’ve been the 32-Bit kernel driver, WOWPOST.EXE is a 16-Bit Windows ASPI helper tool, WINASPI.DLL the corresponding 16-Bit Windows driver. None of those are needed. Now, that FrogASPI is mapping all ASPI calls to SPTI, we can begin to actively use it even on 64-Bit Windows. See AudioCatalyst for instance, with ASPI being available:

AudioCatalyst using FrogASPI

AudioCatalyst reading the Postal Original Soundtrack CD using FrogASPI

As you can see, everything works just fine. Oh, and besides, in case you want AudioCatalysts CDDB feature back (as shown in this screenshot), just add the following lines to your AudioCatalyst.ini, sitting in the programs installation folder, this’ll switch from the now-broken CDDB to FreeDB:

  2. CDDBHTTPPath=/~freedb/cddb.cgi
  3. CDDBLocation=Random FreeDB Site
  4. CDDBTCPIPPort=8880
  5. CDDBConnectVia=TCP/IP

You can download FrogASPI right here:

Unfortunately I cannot give you a download link for Xing AudioCatalyst, as it’s commercial software even today.

1.b) StarBurn ASPI (An update from 2016-12-05):

In the meantime, user Steve Sybesma has [commented] about a different solution provided by [StarBurn Software]. With their recording software comes a native ASPI layer for both 32-bit and 64-bit systems with full compatibility for even 64-bit programs that want to use ASPI. I decided to fetch the necessary DLLs from their SDK and release them in the form of a standalone InnoSetup installer for Windows 2000 and newer. The installer will auto-detect your operating systems’ bitness and will install the 32-bit ASPI layer on 32-bit systems as well as both the 32-bit as well as the 64-bit one on 64-bit systems.

Here it is:

It has been successfully tested on Windows 2000 Pro SP4, Windows XP Pro SP3, Windows XP Pro x64 Edition SP2 as well as Windows 10 Pro x64. If you don’t want to trust my installer, that’s fine as well of course. You can just install the StarBurn software from its original source, it should give you the same level of ASPI support, just with more additional stuff being installed!

Now, to #2, the UDF 2.5 file system for HD-DVDs and Blu-Rays:

2.) UDF 2.5:

Now this one was a real bitch. It took me months to finally get it nailed! UDF 2.5 is a file system. And it’s used for basically two type of media: HD-DVD discs and Blu-Ray discs, including the most modern 3D Blu-Rays. It just bugged me like hell, that I couldn’t access the discs as I normally would in Windows Explorer or on the command line. Windows XP simply does not have the proper file system kernel driver. And while it’s relatively easy to find one for 32-Bit WinXP, it’s immeasurably harder to find the single one existing driver for Windows XP x64. I’m not even sure if it still exists on the web..

One day I came across a person whose name I forgot, but that guy had searched the web for months just like me, and he found the driver just before it went offline. So he re-uploaded it, with his single file sharing link in some post on some unknown website being the only thing that saved my bacon. Maybe one day, somebody will find the XP x64 UDF 2.5 driver here on my site after desperate searching? Who knows.

So, the only existing driver has been developed by Panasonic/Matsushita, a company that also builds optical drives. It works on Windows XP x64 as well as Server 2003 x64. And here it is, together with the Toshiba-made UDF 2.5 driver for regular 32-Bit Windows XP just for completeness:

Again, I would like to emphasize that it took me months to find that freaking 64-Bit XP driver, I’m not even sure what Panasonic/Matsushita developed it for, but here it is for your enjoyment. After installation and reboot, you can browse a HD-DVD or Blu-Ray just like any CD or DVD disc, see here:

UDF 2.5 at work in XP x64

The Blu-Ray folder structure of the german uncut version of “The Boondock Saints”, viewed using the Panasonic/Matsushita UDF 2.5 driver for XP x64

And now, #3, the exFAT file system:

3.) exFAT:

exFAT is a successor to the older FAT32 file system that uses a 64-bit address field instead, allowing for larger than 4GB files, the most notorious limitation of regular FAT. exFAT is now being used on newer cameras and other handheld devices for memory card formatting. Since the comparably heavy-weight NTFS is typically not used on such devices, exFAT is the replacement we get. Also, exFAT is the official file system for SDXC and Memory Stick XC cards, so they may very well come exFAT-preformatted! The only sour part is, that exFAT is kind of proprietary (Microsoft), like NTFS. That’s a bit crappy, but well.

I have however already written an article about exFAT on XP x64, so I will just link to it. exFAT is also very easy to get for both 32-Bit and 64-Bit flavors of Windows XP:

And now, finally, some free file systems from the Linux world:

4.) EXT2/3/4:

EXT is one of the most widely if not the most widely used file system in the entire world of Linux. It has also been ported to other systems like BSD, Solaris or ReactOS. And also Windows actually, although the implementation is not the most seamless you can dream up. But still, you can get read/write access to all currently used EXT versions under Windows, even the fast new EXT4.

The software that I found most useful for this is called [ext2fsd], short for EXT2 file system driver. It comes with the actual driver plus a helper tool for mounting and for enabling write access. It would be nice to just be able to mount EXT filesystems seamlessly from the start, without having to use that mounting tool, but we’re not that far yet it seems. However, drives can be configured to auto-mount later, so you need the tool only once for each drive. Write access has to be enabled by hand.

Currently, ext2fsd supports the following systems: NT 5.0 (Windows 2000), NT 5.1 (Windows XP 32-Bit), NT 5.2 (Windows XP x64, Server 2003), NT 6.0 (Windows Vista), NT 6.1 (Windows 7, Server 2008). Where not explicitly mentioned, it supports both 32-Bit and 64-Bit versions of said operating systems. And here is what it looks like in action:

EXT4 mounted on XP x64

A Linux-created EXT4 filesystem mounted on XP x64 for read/write using ext2fsd

The only strange thing is, that ext2fsd calls the EXT4 file system “EXT3”. It shouldn’t. But other than that, it works just fine. I haven’t tested a lot of r/w, but so far it worked fine, without any corruption or anything. The helper tool also gives you a function for flushing the write cache, and a nice statistics window for monitoring the file system usage while mounted. It also allows the user to manually unmount of course. Even mkfs is there, so you can format drives as EXT under Windows, the only useful tool missing would be fsck for checking & repairing such filesystems. But you can’t have everything I guess.

So, usage is not as seamless as it would be with native NTFS/FAT32/exFAT drivers, but it’s ok and it greatly boosts the Linux interoperability of XP x64 and other Windows systems. Also, EXT filesystems can easily be used in other UNIX-style systems like Solaris, BSD or MacOS X, so this allows a lot of different systems to exchange data easily using removable sticks or harddrives. Since it of course supports large files (>4GB) and it’s more easy to handle across a wide spectrum of operating systems, unlike exFAT it might one day become the #1 choice for data exchange, replacing FAT32 and NTFS in that field. However, today, NTFS on Linux is probably a lot more widespread than this EXT driver on Windows, so there is still a long way to go.

Of course, there is a downside to this too: All the POSIX permissions like ownership, read/write/execute bits etc. just do not work on Windows. You can neither read nor write any such meta data using ext2fsd. I have neither checked the umask that ext2fsd uses yet, nor what ownership it sets, but I would guess it’s root:root with permissions 777 or rwxrwxrwx. But whatever it is, you will most likely need to take care of that when mounting on your UNIX-style system.

I hope this was helpful, if you have any comments about these helpful add-ons, just post away!

5.) Universal SSD TRIM:

This is a 2014-11-03 edit of the original article.

Usually, I would tell users who would like to use an SSD on Windows XP or XP x64 to re-align their partitions with a [gparted ISO] after installation to get proper alignment, and more importantly to use either Intel, Corsair or Samsung SSDs so that they can TRIM their SSDs properly using their respective manufacturers’ XP-compatible software tools, maintaining full write speed over the entire life time of the SSD drive.

It seems that I can now finally confirm, that the latter criterion no longer needs to be met. Basically, I found an easy way to TRIM any SSD on Windows XP or Windows XP x64, as long as the following logical criteria are met:

  • The SSD itself needs to support TRIM. Well, that was obvious.
  • The controller driver needs to support TRIM. Now I can’t speak for just any driver, but I have tested this on Intels ICH10-R and an ancient ICH7-M, both worked just fine with AHCI drivers installed. Usually, AHCI is what you want. Avoid IDE/legacy PATA modes. You may need to configure this properly in your systems BIOS and you may need to retrofit AHCI drivers in XP if your system was installed using IDE mode to avoid bluescreens on boot. If you need help doing that, please ask away in the comments.

So, what’s the solution? It’s called the ADATA SSD Toolbox! Basically, ADATA developers seem to have forgot or were actually never instructed to install an artificial vendor obstruction inside their software. So what they did was to write an SSD toolbox that just complies to the SATA standard, implementing its TRIM command for any compatible SSD, no-holds-barred! I have tested this on an Intel 320 SSD, as well as on a Crucial m500 SSD now, despite no official XP support being present:

So there you go! Now pretty much any TRIM capable SSD can be made to accept TRIM on Windows XP and XP x64! Together with gparted for partition alignment, we have a full-featured SSD solution for XP, leaving nothing to be desired!

Download the ADATA SSD Toolbox here: [ADATA SSD Toolbox v2.0.1].