Jul 072017
 

Nekopara Vol.3 logo1. Introduction

Of course I would never play something like Nekopara *cough*, so this is just a post describing a technical solution to a compatibility problem! Ok?! Good.

Yeah, it’s another one of those “something broke on XP / XP x64, so let’s fix it” articles. I’ve already been pla…  eh.. investigating Nekopara Volumes 0, 1 and 2, and while the developer claims it needs Windows Vista or higher, those titles worked just fine on XP and XP x64. The final Volume 3 however broke.

I wondered why, given they’re all pretty similar, so I started unpacking the .exe files, looking for information. What I found in the meta data was that Vol.0-2 have been using the TVP(Kirikiri) or maybe the forked [Kirikiri Z] game scripting engine, whereas Vol.3 swapped that for the [Ares CatSystem2] engine, for whatever reason. My assumption would be, that the CatSystem2 thingy was actually built for Vista+ for real, thus breaking XP compatibility. Plus, some other minor components are broken as well (some installers, patches, etc., just like the older volumes).

Now, I’ve already been talking to a guy called UncleVasya / Oleg Ovcharenko, who built a [stub DLL solution] for games based on the Clausewitz Engine (Europa Universalis 4, Hearts of Iron 4, Crusader Kings 2 and finally Stellaris), making it work on XP. It’s pretty similar to the XCOM hacks[1][2]. So I asked him about this one as well, and with quite some work and some additional (important) hints from him regarding the Steam version, I managed to make it run!

So, first things first: Thanks Oleg, you’re doing great work! :)

I will now show you how to make this visual novel / game work on XP x64 and XP, both for the slightly trickier Steam version (whether you choose to play the censored or the uncensored version doesn’t matter, the corresponding patch will be discussed as well), as well as the normal version.

Note: All screenshots in this post are 8-bit (256 color) PNG files. They may look a bit bad at times, but better than JPEG in the case of those specific images. Reason for not using truecolor PNG: 8-bit saves a ton of bandwidth.

2. How to make the non-Steam version work on XP / XP x64

Software required:

  1. [Nekopara Vol.3]
  2. [7-zip] archiver
  3. NTCore [CFF Explorer] (optional; only needed for patches)
  4. Olegs’ [patcher]

2a. The main game

First, buy the game and download it. Do not pirate it! You suck if you do (I actually fooled around with a pirated version as well, but only after buying the game). When running the installer, you’ll notice that it already breaks early on after invoking the launcher:

Nekopara Vol.3s' installer already breaks

Nekopara Vol.3s’ installer already fails to execute on XP

As you can see, it calls InitializeCriticalSectionEx(), which is a newer, Vista+ version of InitializeCriticalSection(), see the MSDN[1][2] for details. Since the new version works differently, you can’t just hex edit your way out of this one.

First, unpack Olegs’ patcher to some subdirectory of your choice. Then, unpack the Nekopara Vol.3 installer (the .exe file) into a subfolder using 7-zip, and look for a file called INSTALL.exe. Copy that file into the directory where Olegs patcher resides, so where files like xp_EU4_1.21.cmd and xp_Stellaris_1.6.cmd can be found.

Since the scripts from Oleg aren’t made for hacking our files, we’ll write a new one for this, let’s call it xp_installer.cmd. Edit that with a text editor, and add the following lines:

@ECHO OFF
rundll32.exe zernel32.dll,PatchFile INSTALL.exe

Make sure xp_installer.cmd, zernel32.dll and the INSTALL.exe from Nekopara are in the same directory, then execute xp_installer.cmd. either by just double clicking it, or by opening a cmd terminal and by running it from there. Like this (you don’t need to run the extra commands, they’re just there to show you more information):

Olegs' patch doing its magic on INSTALL.exe

Olegs’ patch doing its magic on INSTALL.exe!

After that, rename your original INSTALL.exe in the directory where you unpacked the Nekopara Vol.3 installer, creating a backup file. Copy the following files from the patcher directory back to the installer directory: INSTALL.exe, zernel32.dll, z3d9.dll, zs2_32.dll and normaliz.dll. The “z” files are now implementing the missing functions, while redirecting all the others to the real Windows libraries like kernel32.dll, d3d9.dll, ws2_32.dll etc.

You don’t need to repack anything, just run INSTALL.exe directly, and you’ll no longer be greeted with an error message, but with this:

The installer works now

The installer works now, great

Install the game to a directory of your choice. Now, if you click the NEKOPARAvol3.exe in the directory where the game was installed, the same launcher comes up again, but now it allows you to configure and play the actual game instead of installing it…

Nekopara Vol.3s' launcher after installation

Nekopara Vol.3s’ launcher after installation

…or does it? Well, the “System settings” part’ll work, yes, but when clicking that alluring “Start” button, you’ll run into yet another wall:

Nekopara still won't execute due to GetTickCount64()

What now? GetTickCount64(), that’s what.

Guess which function call doesn’t exist on XP? See the MSDN[1][2] again. GetTickCount64() really is an improvement over GetTickCount(), but still, XP simply doesn’t have this either. As you can see from the title bar, the offending binary is cs2.exe, which is the actual game. We can get rid of the issue by using Olegs’ patcher again, so it’s the same process as with INSTALL.exe, just use this script instead, call it xp_cs2.cmd or something:

@ECHO OFF
rundll32.exe zernel32.dll,PatchFile cs2.exe

Again, in case something goes wrong, rename your original cs2.exe before copying back the patched version with its .dll files. After copying back, you can run the game either by invoking cs2.exe directly, or by launching it from the NEKOPARAvol3.exe launcher:

Running the non-Steam version of Nekopara Vol.3 on XP x64

Running the non-Steam version of Nekopara Vol.3 on XP x64 (click to enlarge)

2b. Making patches work as well

Patches are essentially also just self-extracting archives that execute a launcher after unpacking. We’ll discuss the patch 11 in this case. Running it will produce a different kind of error (people who know the content restoration patches for the Steam version may have seen this error as well):

Nekopara Vol.3 patch failure

Nekopara Vol.3 patch failure, due to it not being “a valid Win32 application”.

This error means that the header of the binary is asking for a more modern platform. This may make sense, if the program really calls modern functions, but you know, there are modern applications that don’t ask for it and then fail with calls to things like GetTickCount64(), and there are programs which ask for a modern platform without ever having an actual need for it. The patchers are in the latter category of programs.

Unpack the patcher nekopara3_v11_update.exe using 7-zip, and look for a file called updater.exe. Create a backup copy of it, then open this file in NTCores’ CFF Explorer, and click on the “Optional Header” part. You’ll see something like this, I’ve marked the relevant lines with some red blocks for you:

updater.exe in CFF Explorer

updater.exe in CFF Explorer (click to enlarge)

The marked fields show values like 0006 and 0000, as you can see. The significant number is the last or rightmost, so 6 and 0. This corresponds to the platform target Windows NT 6.0, or in other words: Windows Vista. Just rewrite that to show the following numbers, then save the file:

Patch the header to NT 5.1

Patch the header to NT 5.1 (click to enlarge)

NT 5.1 (0005, 0001) equals Windows XP. Note that the kernel versions 5.0 mean Windows 2000, 5.2 means Server 2003 or XP x64 (slightly more modern). Again, no need to repack anything, just save the file after the modifications have been made and execute updater.exe afterwards, you should be getting this:

Nekopara Vol.3 non-Steam patcher working on XP

And here we have a working patcher (click to enlarge)

Yay! And now, for the Steam version of Nekopara Vol.3…

3. How to make the Steam version work on XP / XP x64

Software required:

  1. [Nekopara Vol.3] on Steam (a censored version)
  2. [Content restoration patch] (optional; only required if you have to do perverted things to the cat girls)
  3. at0ms’ [Steamless]
  4. A Windows Vista or newer machine (needed to run Steamless, can be a virtual machine)
  5. [7-zip] archiver
  6. NTCore [CFF Explorer] (optional; only needed for the content restoration patch)
  7. Olegs’ [patcher]

3a. The main game

First, buy the game on Steam and download it. If you really need the uncensored version (you probably do, heh?), buy the content restoration patch at Denpasoft and download that as well. Of course, running the game as-is won’t work, otherwise we wouldn’t need this article in the first place:

The Steam version of Nekopara Vol.3 breaks on XP as well of course

The Steam version of Nekopara Vol.3 breaks on XP as well of course, due to GetTickCount64() call, a newer and better version of GetTickCount(), see MSDN[1][2].

Now, what I didn’t get at first was that patching the Steam versions’ NEKOPARAvol3.exe can never work out of the box. The reason is, that the offending function calls aren’t plainly there for us to see – the actual game binary cs2.exe is encrypted and packed into a SteamStub binary as its payload data. This is a part of the Steamworks DRM system wrapping our program up.

To be able to patch it, we (unfortunately) need to crack its cryptographic DRM protection system first. Now, let me say this again: I do not condone piracy. Don’t fucking crack and distribute this game. You’re an ass if you do. Removing the DRM part is only being done so we can fix the game on XP, keep that in mind!

Well, let’s start; First, boot up a Vista or newer Windows, and install Steamless on it. I actually tried to compile Steamless for XP, but this is .Net 4.5.2 stuff. To make it work on .Net 4.0 would require modifications of its build files / source code, which is a bit over my head right now. So we’re stuck with needing a modern Windows OS to do this. Copy the problematic NEKOPARAvol3.exe from your Steam game installation directory over to that machine, or just install Steam and the game on the modern Windows OS as well (which is what I actually did).

Launch Steamless, open that .exe and decrypt / unpack it, Steamless will leave your binary alone, and create a new, fixed one, so you don’t need to create a manual backup copy:

Steamless cracking NEKOPARAvol3.exe

Steamless cracking NEKOPARAvol3.exe (click to enlarge)

Copy the fixed file back to XP, and rename it back to NEKOPARAvol3.exe. Create a backup of the original .exe in your Steam game installation directory, while you’re at it.

Unpack Olegs’ patcher in a directory of your choice, and move the NEKOPARAvol3.exe there as well, that’s where files akin to xp_EU4_1.21.cmd and xp_Stellaris_1.6.cmd can be found. Since those patcher scripts aren’t targeted at Nekopara Vol.3, we’ll write our own, call it xp_neko_3.cmd or something, open it in a text editor and enter the following lines:

@ECHO OFF
rundll32.exe zernel32.dll,PatchFile NEKOPARAvol3.exe

Make sure that NEKOPARAvol3.exe, zernel32.dll and xp_neko_3.cmd are together in the same folder, then execute xp_neko_3.cmd either by double-clicking it, or by opening a cmd terminal and executing it from there. Like this:

Olegs' patcher handling the decrypted NEKOPARAvol3.exe

Olegs’ patcher handling the now-decrypted Steam version of NEKOPARAvol3.exe

Copy the fully fixed .exe back into the Steam game installation directory, together with the patchers’ stub libraries zernel32.dll, z3d9.dll, zs2_32.dll and normaliz.dll, which will handle the functions usually missing on XP.

Now, run the game either by executing NEKOPARAvol3.exe, or by launching it from within Steam, and you should be greeted with something like this:

Nekopara Vol.3 running on XP x64 in its Steam version

Nekopara Vol.3 running on XP x64 in its Steam version (click to enlarge)

Great (or something)!

Please be aware that if the binary is ever overwritten by Steam because of some update or whatever, you have to re-do the procedure, meaning the Steamless unpacking plus applying Olegs’ patch. If the game terminates without any error when launched from within Steam, try to run NEKOPARAvol3.exe directly instead, and you’ll see the error messages – Steam tends to suppress them.

3b. The content restoration patch (this also applies to the patches for Nekopara Vol.1 and Vol.2)

So you want to lewd the cat girls? Perverted! Plus, Windows XP / XP x64 won’t let you, because the patch is asking for a newer platform (despite not actually requiring it though):

Nekopara Vol.3 content restoration patch failure on XP

Nekopara Vol.3 content restoration patch failure on XP, due to the patch not being “a valid Win32 application”.

But if you absolutely have to, here’s how. Unpack the nekopara_vol3_Steam_R18DLC.exe you bought and downloaded from Denpasoft using 7-zip. Look for the file SteamPatch.exe, and open it in CFF Explorer:

The Nekopara Vol.3 Content restoration patchs' SteamPatch.exe in CFF Explorer

The Nekopara Vol.3 Content restoration patchs’ SteamPatch.exe in CFF Explorer

Now, this is similar to the procedure described for updater.exe for a non-Steam versions’ patch. The significant (rightmost) numbers in the fields where it days 0006 and 0000 represent Windows NT 6.0, or in other words Windows Vista. Since the patcher doesn’t really need any Vista-specific functions, we’ll just fix the header that is currently asking for a NT 6.0 platform as follows:

The Nekopara Vol.3 Content restoration patchs' SteamPatch.exe in CFF Explorer, fixed for XP

Change the fields to 5.1 (0005 and 0001 respectively) to have it check for XP+ instead, and it’s fixed!

Save the file after modifying it. Just like for the non-Steam version patches, there is no need to repack anything. Just run updater.exe directly, and you’ll now get this:

The Nekopara Vol.3 content restoration patch for the Steam version working on XP x64

The Nekopara Vol.3 content restoration patch for the Steam version working on XP x64. Because you’re in it for the Hentai.

There you go, pervert! You now have the fully restored version of Nekopara Vol.3 on Steam, running on XP or XP x64.

And last but not least: Thanks again, Oleg! I made you touch some weird shit, but you still fixed it and gave me the right ideas about the Steam version as well, yay! ;)

4. Bonus feature: How to make Mechwarrior Online work on XP / XP x64 after their launcher upgrade

While entirely unrelated to the weird Japanese shit above, I’ll just mention this here as well, because it doesn’t deserve its own post, given the simplicity of the “solution”; Piranha Games decided to give Mechwarrior Online (MWO) a new game launcher called “MWO Portal”, that is now built with .Net 4.5.2, just like Steamless, breaking it on XP. Mind you, the game itself would still work just fine, even the 64-bit version on XP x64.

The new MWOPortal launcher

Windows XP / XP x64 users will likely never see this launcher work on their OS (Unless ExtendedXP really takes off, it’s pretty good already, but yeah).

Since hacking .Net 4.5 stuff to run on .Net 4 / .Net 4 CP is not something I can do yet, MWO would be gone from all XP machines. There is an easy fix for this though:

Get the game on Steam! The Steam version doesn’t include the launcher, as Steam itself is handling both the execution and the updates of MWO. Without the launcher, MWO still works just fine! :)

Feb 242017
 

Firefox + HTML5 + XP logo1.) Introduction & Explanation

This is one thing that has brought to me by two users ([SK1] on [Voodooalert]German flag and [liquidLD] who talked to me about this on IRC), and because I got a bit pissed off by it myself, I decided to look into the matter. Basically, HTML5 video on Windows XP / XP x64. But not just with webm (VP8/VP9), but also with H.264/AVC. Let’s face it, a lot of videos on the web rely on H.264 and sometimes you simply can’t watch certain videos or you won’t get all the available resolutions. Of course you could just rely on Adobe Flash, but since Google basically took over with their Pepperflash plugin and their Chrome browser no longer supports XP, it’s not the best move either. Especially when you think about Adobes’ history with critical security loopholes in Flash. HTML5 is just much, much safer, and free as well, and Firefox still supports XP.

Note that this guide is thus based on Firefox exclusively. Anything starting with version 47 should work, official support came in 49, and I’ll be using the current version, 51.0.1 at the time of writing.

So, why doesn’t it “just work” in the first place? It did a few years back, right? Because H.264 playback relies on a DRM plugin, on Linux it would be the Google Widevine plugin, on Windows it’s the Adobe Primetime plugin. So yes, Firefox does support DRM out of the box. But even if content isn’t signed and encrypted, the browser still relies on those plugins to play H.264. And the problem is, that Adobe found some problems with that plugin on XP, so they disabled support on the platform. Their version 17 plugin is still being rolled out with the browser however, and it is binary-compatible with XP, so let’s show you how to re-enable it!

2.) Making it work

On Windows XP and XP x64, the plugin should reside in the folder:

%USERPROFILE%\Application Data\Mozilla\Firefox\Profiles\<your profile folder>\gmp-eme-adobe\17\

That folder should contain the files eme-adobe.dll, eme-adobe.info and eme-adobe.voucher. If it doesn’t (maybe because you have a DRM-free version of Firefox), just create the folder structure yourself, get the necessary files from [here] and place them in that folder.

Having the files present won’t enable Adobe Primetime for you however as you can see on about:plugins (Note: The Cisco stuff you can see there is just for WebRTC, so it’s unusable for HTML5 <video>), we still need to tweak a few things on the about:config page of Firefox. Look for the following properties and set them to the values shown below. If a property doesn’t exist yet – media.gmp-eme-adobe.forceSupported most likely won’t – just create them yourself, all of them are boolean properties and all of them need to be set to true:

media.gmp-eme-adobe.enabled		true
media.gmp-eme-adobe.forceSupported	true
media.gmp-eme-adobe.visible		true
media.gmp.decoder.enabled		true
media.eme.enabled			true
media.mediasource.mp4.enabled		true
media.mp4.enabled			true

After making those changes, you’ll need to restart Firefox. Now you might already be good to go, but on some configurations, about:plugins might show something like this:

HTML5+H.264 on Firefox not yet working

Adobe Primetime seems enabled, but there is no file information? So it’s not actually loading the eme-adobe.dll yet (click to enlarge)

If that happens, open your preferences menu on the top right, click on “Add-ons”, then “Plugins” or just go to about:addons. What you should be seeing is this:

 

However, if Adobe Primetime shows a notice saying that it’s going to be “installed shortly”, forget it. Just do it manually on the plugins’ options page you can see on the right image. To do so, click “Check for Updates”. The warning should be gone momentarily. After that, re-check about:plugins, and you should be getting this:

Adobe Primetime fully enabled

Adobe Primetime fully enabled (click to enlarge)

3.) Testing

Now you can do a quick check on the [Youtube HTML5 page], and it should confirm that everything’s working:

Youtube confirming full HTML5 video support

Youtube confirming full HTML5 video support including H.264 and Media Source Extensions (click to enlarge)

With MSE, even Javascript players (like the Flowworks player) bytestreaming H.264 to Firefox should work! Of course, that’s not very thorough. What you’d want is a real playback test, since you can never be sure what you’re getting on Youtube without a bit of extra work. Decent playback tests are currently available on [Quirksmode], and it should look like this:

Firefox playing HTML5 H.264/AVC video in Firefox on Windows XP x64

Firefox playing HTML5 H.264/AVC video on Windows XP x64 (click to enlarge)

With this, even stuff like Netflix works, because you’re getting not just H.264 playback, but also DRM support. Now, whether DRM support is a good thing or not… You’ll have to decide that for yourself. I’m not supportive of DRM content on the web, but if you want to view or listen to such content, you can!

Just one last word of warning though: Adobe has ended their support for XP with a reason, as the Primetime content decryption plugin has shown problems and instabilities on XP! I’ve been using this for about a week now, and I’ve had one case of a video getting stuck, which is a typical symptom of Primetime throwing up on you. Don’t worry though, Firefox won’t crash. Just move the video slider a bit or restart the video, and it’ll work again! You don’t even need to restart the browser, and such occurrences seem to be quite rare, so I’m fine with it.

There you go!

4.) Thanks

Big thanks fly out to [the guys at MSFN] who came up with all of this. I basically got 100% of my information from them, so thank you! You rock! :)

Update: If you update your version of Firefox to the latest and final 52.0 ESR (extended support release), the last version which will be officially supported until 09-2017 for XP, you might notice that Adobe Primetime just disappeared after the update. That’s because the installer may delete the property media.gmp-eme-adobe.visible from your prefs.js. To reenable it, you’ll have to manually recreate the boolean property and set it to true:

media.gmp-eme-adobe.visible		true

Restart Firefox after the change, and the plugin should reappear on about:plugins and about:addons!

Update 2: It seems there is an issue with fresh installations of Adobe Primetime when using Firefox 52.x ESR instead of Firefox 51.x ESR, as reported by [Newb the Newbd]. The solution is to open the URL about:config in Firefox once more and look for media.gmp-manager.url. Its value should look something like this: https://aus5.mozilla.org/update/3/GMP/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml. Edit that configuration option and replace the variable %VERSION% with the string 51.0 to fake an older Firefox version. Now check for updates for Adobe Primetime in the plugins section of about:addons. It should install and start working properly now!

Aug 052016
 

VirtualDimension logoShort story: It’s [VirtualDimension].

Long story… It’s most definitely not what Microsoft added in Windows 10. Besides it being limited to Windows 10, it just sucks for a multitude of reasons. And there I was, having hopes for it as well… If you’ve ever used multiple desktops on a graphical, X11-based Linux or UNIX window manager / desktop environment, you’d know what I’m talking about. Usually, what you’d get on those systems, whether KDE, Gnome, Xfce4, LXDE, or whatever is just one small, configurable panel which allows you to control multiple virtual desktops. On my Gnome 2 on CentOS 6.8 Linux, it looks like this (others are very similar):

Virtual desktops on Gnome 2

Virtual desktops on Gnome 2

The leftmost desktop is my usual “Internet” environment, here I chat, read emails, browse the web for anything work-related and so on. The second one is a Linux distribution development desktop. Here I’m building a derivative of Klaus Knoppers’ [Knoppix] distro. Then comes the testing environment for said distribution on desktop #3. Usually some shells and one VMware Player instance. Next to that are two more VM desktops for software testing and for writing user guides for software installation on different operating systems. At the moment that’s MacOS X and a Windows XP x64 software build VM. Usually there’s also a Windows 7 one. One is empty (for arbitrary stuff), then comes the server administration desktop with 9 open shells, one for each server. And the last one is my private desktop with yet another web browser, and some shells for spawning screen sessions for running software compilations, encoding runs and the likes.

Now, I have a 30″ screen both at home and at work, resolution is 2560×1600. But it’s just never enough screen real estate. So I wanted well-integrated virtual desktops for Windows as well, but last time I tried out some software, I couldn’t find a good one. Recently, I tried again for some reason, like “let’s give this one last shot”. And I tried a lot of programs!

Among the software tested were [Dexpot], [Finestra], [VirtuaWin], [WindowsPager], Xilisoft [Multiple Desktops] and the Windows PowerToy predecessor of [Desktops 2.0] written by Windows Hacker Mark Russinovich and Bryce Cogswell. And finally, [VirtualDimension]. Some of those are free and open source software, others are not.

One of my primary requirements was compatibility to Windows XP x64. Of course it’d be nice if it worked on Windows 10 as well. But most of the above had important features missing or were severely misbehaving on XP. Some were just very, very sluggish when switching desktops. Others had missing features to begin with, like previews on the desktop tiles. A blank desktop tile doesn’t help at all, as I need to see roughly what’s running where at a glance.

I’m not gonna make this a lengthy top list or anything, I’m just gonna show you what the software of my choice – VirtualDimension – could do for me, let’s look at the tiles first:

VirtualDimension on XP x64

VirtualDimension on XP x64

We’ll start with my good old XP x64 first. Here you can see my system tray, and Miranda being open. VirtualDimension cannot be embedded into the taskbar properly (damn), but it has an “always on top” feature. Since the contact list in my docked and always-launched Miranda doesn’t go all the way down, there is free and unused space there. Perfect for VirtualDimension! And since it’s always on top, it doesn’t disappear when clicking on Miranda for chatting.

Given the source code is definitely coming from a UNIX or Linux user (given he built it with GCC/Mingw), some features immediately ring a bell. Like “mouse warp”, where you switch desktops by moving your mouse to the border of the screen. I disabled that, don’t like it. But yeah, it’s there.

Important: While it doesn’t give you live window geometry previews, it does give you iconized previews, so you can always identify any desktop quickly by seeing what’s running there. The desktops can also be named, and there is an OSD that you can have pop up on you when switching, like so:

VirtualDimension OSD

OSD showing right after a desktop switch

In this case I had just switched to desktop #2, which is for A/V processing exclusively. This is just the top left part of the screen, where one of my eight transcoding shells was running a x265 benchmark prototype test. Color, display duration, transparency to mouse clicks on the OSD part, font and size are configurable.

Also, you can freely define keyboard shortcuts for switching desktops as well. I chose CTRL+Shift+Right as well as CTRL+Shift+Left for switching desktops and Alt+Right / Alt+Left for pushing a window to the next/previous desktop as those don’t conflict with other shortcuts I’m using.

What else can it do? Let’s right click on one of the icons in the preview tiles:

VirtualDimension iconized window right click

Clicking on a program icon in VirtualDimensions’ preview tiles gives you this menu

The first five options from the top are global ones. However, the ones below are specific to the icon you right-clicked. With “Activate”, you’d switch to the target desktop and put focus on that programs’ window. The others are pretty self-explanatory as well I guess. We also get a graceful “Close” option, and a brutal “Kill” option that’s equivalent to murdering the process in task manager. Maybe useful since it’s faster that way.

And if we click on the free area?

VirtualDimension, right click on the free area of a preview tile

Right-clicking on the free area of a preview tile gives you a list of all programs on that desktop.

Ok, not sure how useful that is, but at least it may help with identifying the windows on a desktop in more detail, as you get the window titles here. For my encoding shells I could get very quick glance at the progress, but not exactly in great detail. So the helpfulness of this is limited.

What else?

Well, it’s extremely fast! That’s one major plus for VirtualDimension, as several of the other solutions (open source ones as well) were abysmally slow, at least on XP x64. But damn, VirtualDimension just flies! And its memory footprint is minimal. I saw less than 12MiB of consumption here. Even if you add a truckload of Desktops (there seems to be no upper limit), it just won’t slow down unless you spawn like 50 CPU intensive processes all over the place killing your CPU or maxing out your RAM. But that wouldn’t have been VirtualDimensions fault then. Its memory footprint will linearly grow by spawning more desktops, so with eight you may see around 20MiB. Still neat.

And what’s bad about it?

Well, sometimes, if you have a lot of windows on one desktop, the icons are’t cut off in the right spot at the bottom of the preview tile, so they overflow just a little bit. Just a cosmetic issue. Also, you should maybe deactivate the shell integration. With this, VirtualDimension hooks itself into all windows (such a DLL hook means entering another processes’ memory area). With that, you can get its functions via right clicking on a windows’ title bar, like on UNIX.

Nice, but dangerous! This can trigger anti-cheat systems in online games, because they really don’t like you stepping into their processes’ memory areas! That’s what cheating tools do to modify a games’ parameters on the fly as well. You don’t wanna be banned because of such a thing!

In my case, I managed to lock myself out of Mechwarrior Online because of this. I wasn’t banned, but the login process wouldn’t even let me launch its window. Disabling the feature, launching MWO, then re-enabling it and trying to log in caused a pretty abnormal process termination:

Mechwarrior Online really doesn't like VirtualDimensions' shell integration feature

Mechwarrior Online really doesn’t like VirtualDimensions’ shell integration feature! And no, there was no “update available”… (click to enlarge)

There is an exception list for this feature, and I added all of MWOs’ .exe files to it, to no avail. Better to stay away from this one.

Now, well, this otherwise beautiful piece of software was dropped by its developer around 2005. About the time my XP x64 came out. Latest alpha build is from some time in 2006. So this is ancient! It even supports Windows 98 and NT 4.0, I mean… So, how about Windows 10 then? I mean, Windows 10 doesn’t even have a GDI UI anymore, this is like one completely different world. Since I do have a Windows 10 machine (yeah, ew), let’s check it out:

VirtualDimension on Windows 10

VirtualDimension on Windows 10 – hey, it really works!?

Miranda seemingly can’t dock properly on Windows 10 anymore. It kinda… floats near the desktop border when docked. It’s strange and it wastes space, but well, I don’t know how to fix that yet. But anyway, I embedded VirtualDimension into Miranda (by just moving the window there, removing its title bar and resizing it properly again). And guess what?

It just works™!

I launched some Metro / Modern UI apps in a window as well, and while those aren’t shown in the preview tiles, they can be controlled with keyboard shortcuts, just like regular windows. Also, it’s just as blazing fast as it is on XP. Ah, and… yeah, it actually does work on all 64-bit x86 Windows versions it seems! It’s amazing, but an ancient piece of 32-bit software that does alter a Windows systems’ usage pattern quite fundamentally still works fine on Windows 10 64-bit. I gotta say, I’m pretty relieved, because Windows 10s’ own solution just sucks – where is my live preview? – and I don’t want to change my usage paradigm too much when switching operating systems (even from Linux/UNIX to Windows and back).

Some of the other solutions like Dexpot or Finestra may be faster on Windows 10 than they are on a just half-supported XP x64, but nah, with this tool, I don’t need them anymore, no matter the operating system.

VirtualDimension is as perfect as it gets, despite its age! Or maybe because of it?

Still, anyone interested in picking up that [VirtualDimension project on SourceForge] and in continuing its development? ;) I guess I can’t touch that code, would probably just mess everything up. Ah, it’s C++ by the way…

A few things could use some fixing, like the icon overflowing issue, Modern UI window detection, certain, rare windows being sticky on all Desktops even if nobody told them to do so (Miranda, X-Chat DCC windows) and that exclusion list for the shell integration, which doesn’t hook into all windows properly when active either anyway.

Would be so nice if somebody could continue working this! :)

Until that happens (I know, it never will), I’ll just continue using v0.95 alpha. ;)

Apr 082016
 

H.265/HEVC logoPreviously, I have shown you [how to compile x265 on Windows] using Microsoft Visual Studio 2010 in a way that results in binaries compatible with Windows NT 5.1/5.2, or in other words: Windows XP, XP x64 and Windows Server 2003. And while that works for most purposes, today I’d like to show you how to build an actual multilib binary, that can handle all three color bit depths supported by x265, the standardized 8- and 10-bit (MAIN and MAIN10 profiles) as well as 12-bit (MAIN12 profile). With that, it’s all in one exe instead of three. As before though, multilib x265 is only supported on 64-Bit Windows. But first, once again…

1.) Giving you the binaries

There were a lot of improvements since the last version I published back in February of course, also performance-wise. So here’s the current version from Multicoreware for both 32-bit and 64-bit Windows, compiled with MSVC 2010 SP1 and yasm 1.3.0. This requires the Microsoft Visual C++ 2010 runtime to work, see previous article:

This time around, the binaries have been tested as well! On regular 32-bit Windows XP, only fundamental binary compatibility was tested. However, all versions, so the 32-Bit one and the 64-Bit multilib ones have been ran through a 2-pass ABR encoding test with output verification for 8-bit color depth (32- & 64-bit) as well as 10- and 12-bit color depths (64-bit only) on Windows XP Professional x64 Edition using the following command line (see previous post for details):

avconv -r 24000/1001 -i input.h264 -f yuv4mpegpipe -pix_fmt yuv420p -r 24000/1001 - 2>NUL^ 
 | .\x265.exe - --y4m -D 10 --fps 24000/1001 -p veryslow^
 --open-gop --ref 6 --bframes 16 --b-pyramid --bitrate 2500 --rect --amp --aq-mode 3^
 --no-sao --qcomp 0.75 --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0^
 --rdoq-level 1 --tu-inter-depth 4 --tu-intra-depth 4 --ctu 32 --max-tu-size 16 --pass 1^
 --slow-firstpass --stats v.stats --sar 1 --range full -o pass1.h265 &amp; avconv^
 -r 24000/1001 -i input.h264 -f yuv4mpegpipe -pix_fmt yuv420p -r 24000/1001 - 2>NUL^
 | .\x265.exe - --y4m -D 10 --fps 24000/1001 -p veryslow --open-gop --ref 6^
 --bframes 16 --b-pyramid --bitrate 2500 --rect --amp --aq-mode 3 --no-sao --qcomp 0.75^
 --no-strong-intra-smoothing --psy-rd 1.6 --psy-rdoq 5.0 --rdoq-level 1^
 --tu-inter-depth 4 --tu-intra-depth 4 --ctu 32 --max-tu-size 16 --pass 2^
 --stats v.stats --sar 1 --range full -o pass2.h265

Needless to say, they should work fine on Windows Vista/7/8/8.1/10/Server 2008/Server 2012/HS 2007/HS 2011 as well.

From time to time, I’ll release new binaries, so you might wanna check back every few months or so, if you’re interested. You can also request a build in the comments if you’re growing impatient and need a specific version more quickly because of some bugfix / feature improvement in x265.

2.) Compiling an XP/2003-compatible x265 multilib binary yourself

First, please look at the previous article I linked to in the beginning, point 2. You need the software prerequisites listed in 2a and you might still wish to read through 2b to understand some of the stuff better. You don’t need to actually run any of the commands shown there though.

Now, the multilib build is done a bit differently from the rest, as everything is scripted, so this is 100% command line work, no graphical cmake, no running the full Visual Studio IDE. Usually, with all software in place, sitting in the root directory of the x265 source tree, all you need to do is to go to build\vc10-x86_64\ and run ./multilib.bat. This won’t give us an XP/2003-compatible binary however, and the reason lies within the build script multilib.bat, here is the stock version:

expand/collapse source code (multilib.bat)
  1. @echo off
  2. if "%VS100COMNTOOLS%" == "" (
  3.   msg "%username%" "Visual Studio 10 not detected"
  4.   exit 1
  5. )
  6.  
  7. call "%VS100COMNTOOLS%\..\..\VC\vcvarsall.bat"
  8.  
  9. @mkdir 12bit
  10. @mkdir 10bit
  11. @mkdir 8bit
  12.  
  13. @cd 12bit
  14. cmake -G "Visual Studio 10 Win64" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON
  15. if exist x265.sln (
  16.   MSBuild /property:Configuration="Release" x265.sln
  17.   copy/y Release\x265-static.lib ..\8bit\x265-static-main12.lib
  18. )
  19.  
  20. @cd ..\10bit
  21. cmake -G "Visual Studio 10 Win64" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF
  22. if exist x265.sln (
  23.   MSBuild /property:Configuration="Release" x265.sln
  24.   copy/y Release\x265-static.lib ..\8bit\x265-static-main10.lib
  25. )
  26.  
  27. @cd ..\8bit
  28. if not exist x265-static-main10.lib (
  29.   msg "%username%" "10bit build failed"
  30.   exit 1
  31. )
  32. if not exist x265-static-main12.lib (
  33.   msg "%username%" "12bit build failed"
  34.   exit 1
  35. )
  36. cmake -G "Visual Studio 10 Win64" ../../../source -DEXTRA_LIB="x265-static-main10.lib;x265-static-main12.lib" -DLINKED_10BIT=ON -DLINKED_12BIT=ON
  37. if exist x265.sln (
  38.   MSBuild /property:Configuration="Release" x265.sln
  39.   :: combine static libraries (ignore warnings caused by winxp.cpp hacks)
  40.   move Release\x265-static.lib x265-static-main.lib
  41.   LIB.EXE /ignore:4006 /ignore:4221 /OUT:Release\x265-static.lib x265-static-main.lib x265-static-main10.lib x265-static-main12.lib
  42. )
  43.  
  44. pause

So I took all the options from the files generated by the original cmake when doing the normal build, and added them to the script to ensure our output binaries would be XP-compatible. This is the fixed build script:

expand/collapse source code (multilib.bat, patched for XP/2003)
  1. @echo off
  2. if "%VS100COMNTOOLS%" == "" (
  3.   msg "%username%" "Visual Studio 10 not detected"
  4.   exit 1
  5. )
  6.  
  7. call "%VS100COMNTOOLS%\..\..\VC\vcvarsall.bat"
  8.  
  9. @mkdir 12bit
  10. @mkdir 10bit
  11. @mkdir 8bit
  12.  
  13. @cd 12bit
  14. cmake -DCMAKE_BUILD_TYPE="Release" -DCMAKE_CONFIGURATION_TYPES="Release" -G "Visual Studio 10 Win64" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DWINXP_SUPPORT=ON -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON
  15. if exist x265.sln (
  16.   MSBuild /property:Configuration="Release" x265.sln
  17.   copy/y Release\x265-static.lib ..\8bit\x265-static-main12.lib
  18. )
  19.  
  20. @cd ..\10bit
  21. cmake -DCMAKE_BUILD_TYPE="Release" -DCMAKE_CONFIGURATION_TYPES="Release" -G "Visual Studio 10 Win64" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DWINXP_SUPPORT=ON -DENABLE_SHARED=OFF -DENABLE_CLI=OFF
  22. if exist x265.sln (
  23.   MSBuild /property:Configuration="Release" x265.sln
  24.   copy/y Release\x265-static.lib ..\8bit\x265-static-main10.lib
  25. )
  26.  
  27. @cd ..\8bit
  28. if not exist x265-static-main10.lib (
  29.   msg "%username%" "10bit build failed"
  30.   exit 1
  31. )
  32. if not exist x265-static-main12.lib (
  33.   msg "%username%" "12bit build failed"
  34.   exit 1
  35. )
  36. cmake -DCMAKE_BUILD_TYPE="Release" -DCMAKE_CONFIGURATION_TYPES="Release" -G "Visual Studio 10 Win64" ../../../source -DWINXP_SUPPORT=ON -DEXTRA_LIB="x265-static-main10.lib;x265-static-main12.lib" -DLINKED_10BIT=ON -DLINKED_12BIT=ON
  37. if exist x265.sln (
  38.   MSBuild /property:Configuration="Release" x265.sln
  39.   :: combine static libraries (ignore warnings caused by winxp.cpp hacks)
  40.   move Release\x265-static.lib x265-static-main.lib
  41.   LIB.EXE /ignore:4006 /ignore:4221 /OUT:Release\x265-static.lib x265-static-main.lib x265-static-main10.lib x265-static-main12.lib
  42. )
  43.  
  44. pause

You can just rename your original script for backup and put the fixed code in its place, build\vc10-x86_64\multilib.bat, then run it on the command line. If all the required tools are present, it will compile a 12-bit library, then a 10-bit library (both static) and finally an 8-bit binary that will have the other two libraries statically linked in. The final x265.exe can then be found in build\vc10-x86_64\8bit\Release\. To check whether it’s the real thing, look for the bitness by running .\x265.exe --version while sitting in that folder on the command line. You should see something like this:

x265 multilib binary

A x265 multilib binary shows that it’s “8-bit+10-bit+12-bit”

Per-color-channel bitness can be defined with x265s’ command line option -D. So that’d be -D 8, -D 10 or -D 12. Note that only 8- and 10-bit are part of the official Blu-Ray UHD/4k specification however.

3.) A side note

In case you’re new to this, you might not get why “8-bit” and “10-bit” etc. Aren’t color spaces supposed to be 16-bit, 24-bit, 32-bit etc.? Well, it seems that in the world of video processing, people don’t refer to whole color space bitness, but rather individual color channel bitness. So with three channels (red, green, blue for instance), you’d have 8/10/12 bits per channel, so that’s 24-, 30- and 36-bit total, or 16.7 million, 1 billion and 64 billion colors.

The more important part – and the reason why nobody encodes to 12-bit – is the internal arithmetic precision of x265 though (same applies to x264). At 8-bit color depth, arithmetic precision is also at 8-bits. When you hop over to 10-bit, you can’t use 8-bit operations and data types any longer, so everything is done at 16-bit precision. This makes the code slower, but also more efficient in preserving color gradients. Since 10-bit H.265/HEVC is officially a part of Blu-Ray UHD/4k, this would be the sweet spot, unless you’re dealing with devices too slow to play it.

Going to 12-bit won’t boost the precision further, it just gives you more colors, that most of today’s displays won’t be able to show anyway. Not much benefit.

So that’s that.

Have fun! :)

Update: All x265 releases have now been consolidated on [this page]! All future XP- and XP x64-compatible releases of x265 plus a relatively recent version of avconv to act as a decoder for feeding x265 with any kind of input video streams will be posted there as well.

Jan 152015
 

4Kn logoWhile I’ve been planning to build myself a new RAID-6 array for some time (more space, more speed), I got interested in the latest and greatest of hard drive innovations, which is the 4Kn Advanced Format. Now you may now classic hard drives with 512 byte sectors and the regular Advanced Format also known as 512e, which uses 4kiB physical sector sizes, but emulates 512 byte sectors for compatibility reasons. Interestingly, [Microsoft themselves state], that “real” 4Kn harddrives, which expose their sector size to the operating system with no glue layers in between are only supported in Windows 8 and above. So even Windows 7 has no official support.

On top of that, Intel [has stated], that their SATA controller drivers do not support 4Kn, so hooking one such drive up to your Intel chipsets’ I/O controller hub (ICH) or platform controller hub (PCH) will not work. Quote:

“Intel® Rapid Storage Technology (Intel® RST) version 9.6 and newer supports 4k sector disks if the device supports 512 byte emulation (512e). Intel® RST does not support 4k native sector size devices.”

For clarity, to make 4Kn work in a clean fashion, it must be supported on three levels, from lowest to highest:

  1. The firmware: For mainboards, this means your system BIOS/UEFI. For dedicated storage controllers, the controller BIOS itself.
  2. The kernel driver of the storage controller, so that’s your SATA AHCI/RAID drivers or SAS drivers.
  3. Any applications above it performing raw disk access, whether kernel or user space. File system drivers, disk cloning software, low level benchmarks, etc.

Granted, 4Kn drives are extremely new and still very rare. There is basically only the 6TB Seagate enterprise drives available ([see here]) and then some Toshiba drives, also enterprise class. But, to protect my future investments in that RAID-6, I got myself a [Toshiba MG04ACA300A] 3TB drive, which was the only barely affordable 4Kn disk I could get, basically also the only one available right now besides the super expensive 6TB Seagates. That way I can check for 4Kn compatibility relatively cheaply (click to enlarge images):

If you look closely, you can spot the nice 4Kn logo right there. In case you ask yourselves “Why 4Kn?”, well, mostly cost and efficiency. 4kiB sectors are 8 times as large as classic 512 byte ones. Thus, for the same data payload you need 8 times less sector gaps, 8 times less synchronization markers and 8 times less address markers. Also, a stronger checksum can be used for data integrity. See this picture from [Wikipedia]:

Sectors

Sector size comparison (Image is © Dougolsen under the CC-BY 3.0 unported license)

Now this efficiency is already there with 512e drives. 512e Advanced Format was supposedly invented, because more than half the programs working with raw disks out there can’t handle variable sector sizes and are hardcoded for 512n. That also includes system firmwares, so your mainboards’ BIOS/UEFI. To solve those issues, they used 4kiB sectors, then let a fast ARM processor translate them into 512 byte sectors on the fly to give legacy software something it could understand.

4Kn on the other hand is the purist, “right” approach. No more emulation, no more cheating. No more 1GHz ARM dual core processor in your hard drive just to be able to serve data fast enough.

Now we already know that Intel controllers won’t work. For fun, I hooked it up to my ASUS P6T Deluxes’ secondary SATA controller though, a Marvell 88SE6120. Plus, I gave the controller the latest possible driver, the quite hard-to-get version 1.2.0.8400. You can download that [here] for x86 and x64.  To forestall the result: It doesn’t work. At all. This is what the systems’ log has to say about it (click to enlarge):

So that’s a complete failure right there. Even after the “plugged out” message, the timeouts would still continue to happen roughly every 30 seconds, accompanied by the whole operating system freezing for 1-2 seconds every time. I cannot say for any other controllers like the Marvell 9128 or Silicon Image chips and others, but I do get the feeling that none of them will be able to handle 4Kn.

Luckily, I do already have the controller for my future RAID-6 right here, an Areca ARC-1883ix-12, the latest and greatest tech with PCIe x8 3.0 and SAS/12Gbit ports with SATA/6Gbit encapsulation. Its firmware and driver supports 4Kn fully as you can see in Arecas [specifications]. The controller features an out-of-band management system via its own ethernet port and integrated web server for browser-based administration, even if the system doesn’t even have any OS booted up. All that needs to be installed on the OS then is a very tiny driver (click to enlarge):

Plus, Areca gives us one small driver for many Windows operating systems. Only for the Windows XP 32-Bit NT5.1 kernel you’ll get a SCSI Miniport driver exclusively, while all newer systems (WinXP x64, Windows Vista, 7, 8) get a more efficient StorPort driver. So, plugged the controller in, installed the driver, hooked up the disk, and it seems we’re good to go:

The 4Kn drive is being recognized

The 4Kn drive is being recognized (click to enlarge)

Now, any legacy master boot record (MBR) partition table has a 32-bit address field. That means, it can address 232 elements. With each element being 512 bytes large, you reach 2TiB. So that’s where the 2TiB limit comes from. With 4Kn however, the smallest addressable atom is now eight times as large: 4096 bytes! So we should be able to reach 16TiB due to the larger sector size. Supposedly, some USB hard drive manufacturers have used this trick (by emulating 4Kn) to make their larger drives work easily on Windows XP. When trying to partition the Toshiba drive however, I hit a wall, as it seems Windows disk management is about as stupid as was the FAT32 formatter on Windows 98:

MBR initialization failed

MBR initialization failed (click to enlarge)

That gets me thinking. On XP x64, I can still just switch from MBR to the GPT partitioning scheme to be able to partition huge block devices. But what about Windows XP 32-bit? I don’t know how the USB drive manufacturers do it, so I can only presume they ship the drives pre-partitioned if its one of those that don’t come with a special mapping tool for XP. In my case, I just switch to GPT and carry on (click to enlarge):

Now I guess I am the first person in the world to be able to look at this, and potentially the last too:

fsutil.exe showing a 4Kn drive on XP x64

fsutil.exe showing a native SATA 4Kn drive on XP x64, encapsulated in SAS. Windows 7 would show the physical and logical sector size separately due to its official 512e support. Windows XP always reports the logical sector size (click to enlarge)

So far so good. The very first and most simple test? Just copy a file onto the newly formatted file system. I picked the 4k (no pun intended) version of the movie “Big Buck Bunny”:

Copying a first file onto the 4Kn disks NTFS file system

Copying a first file onto the 4Kn disks NTFS file system

Hidden files and folders are shown here, but Windows doesn’t seem to want to create a System Volume Information\ folder for whatever reason. Other than that it’s very fast and seems to work just nicely. Since the speed is affected by the RAID controllers write back cache, I thought I’d try HD Tune 2.55 for a quick sequential benchmark. Or in other words: “Let’s hit our second legacy software wall” (click to enlarge):

Yeah, so… HD Tune never detects anything above 2TiB, but this? At first glance, 375GB might sound quite strange for a 3TB drive. But consider this: 375 × 8 = 3000. What happened here is that HD Tune got the correct sector count of the drive, but misinterpreted each sectors’ size as 512 bytes. Thus, it reports the devices’ size as eight times as small. Reportedly, this is also the exact way how Intels RST drivers fail when trying to address a 4Kn drive. HD Tune 2.55 is thus clearly hardcoded for 512n. There is no way to make this work. Let’s try the paid version of the tool which is usually quite ahead of its free and legacy counterpart (click to enlarge):

Indeed, HD Tune Pro 5.00 works just as it should when accessing the raw drive. Users who don’t want to pay are left dead in the water here. Next, I tried HDTach, also an older tool. HDTach however reads from a formatted file system instead of from a raw block device. The file system abstracts the device to a higher level, so HDTach doesn’t know and doesn’t need to know anything about sectors. As a result, it also just works:

HD Tach benchmarking NTFS on a 4Kn drive

HD Tach benchmarking NTFS on a 4Kn drive (click to enlarge)

Next, let’s try an ancient benchmark, that again accesses drives on the sector level: The ATTO disk benchmark. It is here where we learn that 4Kn, or generally variable sector sizes aren’t space magic. This tool was written well before the times of 512e or 4Kn, and look at that (click to enlarge):

Now what does that tell us? It tells us, that hardware developers feared the chaotic ecosystem of tools and software that accesses disks at low levels. Some might be cleanly programmed, where most may not. That doesn’t just include operating systems’ built-in toolsets, but also 3rd party software, independently from the operating system itself. Maybe it also affects disk cloning software like from Acronis? Volume shadow copies? Bitlocker? Who knows. Thing is, to be sure, you need to test that stuff. And I presume that to go as far as hard drive manufacturers did with 512e, they likely found one abhorrent hell of crappy software during their tests. Nothing else will justify ARM processors at high clock rates on hard drives just to translate sector sizes plus all the massive work that went into defining the 512e Advanced Format standard before 4Kn Advanced Format.

Windows 8 might now fully support 4Kn, but that doesn’t say anything about the 3rd party software you’re going to run on that OS either. So we still live in a Windows world where a lot of fail is waiting for us. Naturally, Linux and certain UNIX systems have adapted much earlier or have never even made the mistake of hardcoding sector sizes into their kernels and tools.

But now, to the final piece of my preliminary tests: Truecrypt. A disk encryption software I still trust despite the project having been shut down. Still being audited without any terrible security hole discoveries so far, it’s my main choice for cross-platform disk encryption, working cleanly on at least Windows, MacOS X and Linux.

Now, 4Kn is disabled for MacOS X in Truecrypts source code, but seemingly, this [can be fixed]. I also discovered that TC will refuse to use anything other than 512n on Linux if Linux kernel crypto is unavailable or disabled by the user in TC, see this part of Truecrypts’ CoreUnix.cpp:

#if defined (TC_LINUX)
if (volume-&gt;GetSectorSize() != TC_SECTOR_SIZE_LEGACY)
{
  if (options.Protection == VolumeProtection::HiddenVolumeReadOnly)
    throw UnsupportedSectorSizeHiddenVolumeProtection();
 
  if (options.NoKernelCrypto)
    throw UnsupportedSectorSizeNoKernelCrypto();
}
#endif

Given that TC_SECTOR_SIZE_LEGACY equals 512, it becomes clear that hidden volumes are unavailable as a whole with 4Kn on Linux, and encryption is completely unavailable altogether if kernel crypto isn’t there. So I checked out the Windows specific parts of the code, but couldn’t find anything suspicious in the source for data volume encryption. It seems 4Kn is not allowed for bootable system volumes (lots of “512’s” there), but for data volumes it seems TC is fully capable of working with variable sector sizes.

Now this code has probably never been run before on an actual SATA 4Kn drive, so let’s just give it a shot (click to enlarge):

Amazingly, Truecrypt, another software written and even abandoned by its original developers before the advent of 4Kn works just fine. This time, Windows does create the System Volume Information\ folder on the file system within the Truecrypt container, and fsutil.exe once again reports a sector size of 4096 bytes. This shows clearly that TC understands 4Kn and passes the sector size on to any layers above itself in the kernel I/O stack flawlessly (The layer beneath it should be either the NT I/O scheduler or maybe the storage controller driver directly and the layer above it the NTFS file system driver, if my assumptions are correct).

Two final tests for data integrities’ sake:

Both a binary diff and SHA512 checksums prove, that the data copied from a 512n medium to the 4Kn one is still intact

Both a binary diff and SHA512 checksums prove, that the data copied from a 512n medium to the 4Kn one is still intact

So, my final conclusion? Anything that needs to work with a raw block device on a sector-by-sector level needs to be checked out before investing serious money in such hard drives and storage arrays. It might be cleanly programmed, with some foresight. It also might not.

Anything that sits above the file system layer though (anything that reads and writes folders and files instead of raw sectors) will always work nicely, as such software does not need to know anything about sectors.

Given the possibly enormous amount of software with hardcoded routines for 512 byte sectors, my assumption would be that the migration to 4Kn will be quite a sluggish one. We can see that the enterprise sector is adapting first, clearly because Linux and UNIX systems adapt much faster. The consumer market however might not see 4Kn drives anytime soon, given 512 byte sectors have been around for about 60 years (!) now.

Update 2014-01-16 (Linux): I just couldn’t let it go, so I took the Toshiba 4Kn drive to work with me, and hot plugged it into an Intel ICH10R. So that’s the same chipset as the one I ran the Windows tests on, an Intel X58. Only difference is, that now we’re on CentOS 6.6 Linux running the 2.6.32-504.1.3.el6.x86_64 kernel.  This is what dmesg had to say about my hotplugging:

ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: ATA-8: TOSHIBA MG04ACA300A, FP2A, max UDMA/100
ata3.00: 732566646 sectors, multi 2: LBA48 NCQ (depth 31/32), AA
ata3.00: configured for UDMA/100
ata3: EH complete
scsi 2:0:0:0: Direct-Access     ATA      TOSHIBA MG04ACA3 FP2A PQ: 0 ANSI: 5
sd 2:0:0:0: Attached scsi generic sg7 type 0
sd 2:0:0:0: [sdf] 732566646 4096-byte logical blocks: (3.00 TB/2.72 TiB)
sd 2:0:0:0: [sdf] Write Protect is off
sd 2:0:0:0: [sdf] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 2:0:0:0: [sdf] 732566646 4096-byte logical blocks: (3.00 TB/2.72 TiB)
 sdf:
sd 2:0:0:0: [sdf] 732566646 4096-byte logical blocks: (3.00 TB/2.72 TiB)
sd 2:0:0:0: [sdf] Attached SCSI disk

Looking good so far, also the Linux kernel typically cares rather less about the systems BIOS, bypassing whatever crap it’s trying to tell the kernel. Which is usually a good thing. Let’s verify with fdisk:

Note: sector size is 4096 (not 512)
 
WARNING: The size of this disk is 3.0 TB (3000592982016 bytes).
DOS partition table format can not be used on drives for volumes
larger than (17592186040320 bytes) for 4096-byte sectors. Use parted(1) and GUID 
partition table format (GPT).

Now that’s more like it! fdisk is warning me, that it will be limited to addressing 16TiB on this disk. A regular 512n or 512e drive would be limited to 2TiB as we know. Awesome. So, I created a classic MBR style partition on it, formatted it using the EXT4 file system, and mounted it. And what we get is this:

Filesystem            Size  Used Avail Use% Mounted on
/dev/sdf1             2.7T   73M  2.6T   1% /mnt/sdf1

And Intel is telling us that they don’t manage to give us any Windows drivers that can do 4Kn? Marvell doesn’t even comment on their inabilities? Well, suck this: Linux’ free driver for an Intel ICH10R south bridge (or any other that has a driver coming with the Linux kernel for that matter) seems to have no issues with that whatsoever. I bet it’s the same with BSD. Just weak, Intel. And Marvell. And all you guys who had so much time to prepare and yet did nothing!

Update 2014-01-20 (Windows XP 32-Bit): So what about regular 32-Bit Windows XP? There are stories going around that some USB drives with 3-4TB capacity would use a 4Kn emulation (or real 4Kn, bypassing the 512e layer by telling the drive firmware to do so?), specifically to enable XP compatibility without having to resort to special mapping tools.

Today, I had the time to install XP SP3 on a spare AMD machine (FX9590, 990FX), which is pretty fast thanks to a small, unused testing SSD I still had lying around. Before that I wiped all GPT partition tables from the 4Kn drive, both the one at the start as well as the backup copy at the end of the drive using dd. Again, for this test, the Areca ARC-1883ix-12 was used, now with its SCSI miniport driver, since XP 32-Bit does not support StorPort.

Please note, that this is a German installation of Windows XP SP3. I hope the screenshots are still understandable enough for English speakers.

Recognition and MBR initialization seems to work just fine this time, unlike on XP x64:

The 4Kn Toshiba as detected by Windows XP Pro 32-Bit SP3, again on an Areca ARC-1883ix-12

The 4Kn Toshiba as detected by Windows XP Pro 32-Bit SP3, again on an Areca ARC-1883ix-12 (click to enlarge)

Let’s try to partition it:

Partitioning the drive once more, MBR style

Partitioning the drive once more, MBR style

Sure looks good! And then, we get this:

A Master Boot Record, Windows XP and 4Kn: It does work after all

A Master Boot Record, Windows XP and 4Kn: It does work after all (click to enlarge)

So why does XP x64 not allow for initialization and partitioning of a 4Kn drive using MBR? Maybe because it’s got GPT for that? So in any case, it’s usable on both systems, the older NT 5.1 (XP 32-Bit) as well as the newer NT 5.2 (XP x64, Server 2003). Again, fsutil.exe confirms proper recognition of our 4Kn drive:

fsutil.exe reporting a 4kiB sector size, just like on XP x64

fsutil.exe reporting a 4kiB sector size, just like on XP x64

So all you need – just like on XP x64 – is a proper controller with proper firmware and drivers!

There is one hard limit here though that XP 32-Bit users absolutely need to keep in mind; Huge RAID volumes using LUN carving/splitting and software JBOD/disk spanning using Microsofts Dynamic Volumes are no longer possible when using 4Kn drives. Previously, you could tell certain RAID controllers to just serve huge arrays to the OS in 2TiB LUN slices (e.g. best practice for 3ware controllers on XP 32-Bit). Then, in Windows, you’d just make those slices Dynamic Volumes and span a single NTFS file system over all of them, thus pseudo-breaking the 2TiB barrier.

This can no longer be done, as Dynamic Volumes seemingly do not work with 4Kn drives on Microsoft operating systems before Windows 8, or at least not on XP 32-Bit. The option for converting the volume from MBR to GPT is simply greyed out in Windows disk management.

That means that the absolute maximum volume size using 4Kn disks on 32-Bit Windows XP is 16TiB! On XP x64 – thanks to GPT – it’s just a few blocks short of 256TiB, a limit imposed on us by the NTFS file systems’ 32-bit address field and 64kiB clusters, as 232 * 64KiB × 1024 × 1024 × 1024 = 256TiB.

And that concludes my tests, unless I have time and an actual machine to try FreeBSD or OpenBSD UNIX. Or maybe Windows 7. The likelihood for that is not too high at the moment though.

May 292013
 

Metro Last Light logoAnd here’s another one. There were games that did not support Windows XP at all, like [Dishonored] or [XCOM]. Some were just without all official support, but working flawlessly nonetheless, and others just carelessly compiled and linked, which could be fixed by a [rather simple function call redirection] to a proper backport DLL that Microsoft actually offers officially. So in many cases, XP and XP x64 support is not impossible at all. So why does it disappear? Simply because of the one factor that rules all and everything: Money.

Supporting an operating system may seem simple, just see whether your game runs and be done with it. It never is that simple though. So my assumption is, that official support is simply vanishing because of cost. Cost of testing, quality assurance and ongoing after sales support. That stuff is probably more expensive than I thought, and I cannot think of any other reason why this would be the case also for 4A games’ latest release: [Metro Last Light]. If the cost of testing, QA and support is greater than return of investment expectations (=sales on XP platforms), then they won’t go there.

So, Metro Last Light is kind of an in-between thing. If you look at its [Steam page], you’ll see that it doesn’t completely rule out XP, but it limits support to the regular 32-Bit NT 5.1 version. The reasons are simple. While XP x64 may seem like a better option for modern games due to its greater RAM support and 4GB userspace windows for 32-Bit applications, it is simple not as wide-spread as good old 32-Bit XP. So from a businessmans perspective, it may seem like a good idea to throw out XP x64 support and keep XP x86 in, as about 22x as many people are using XP x86 when compared to XP x64 according to the [latest Steam survey] from April 2013 (If you click on this link later, you’ll probably get different figures as the link’ll point to a more recent survey page).

So, 0.38% of all users are on XP x64. Actually, a +0.01% rise since last month, pffh. And in comparison, 8.25% of all Steam users are on XP x86, or 32-Bit. Since Metro Last Light still has that Direct3D 9 renderer that Metro 2033 also featured, they might have thought “we can still squeeze some cash out of XP, but not XP x64”.

Still, the good thing is, that business decisions not always have to affect technical details in every single aspect. Sure, there ain’t no support whatsoever, but who cares, if you can get this nonetheless, on Windows XP x64 Edition:

Metro Last Light on XP x64

So yeah, it just works. No hiccups, no nothing. It’s still sad that developer 4A games would state upon inquiry that they do not support XP x64. It would be nice if they could just say something like “We cannot officially support this, if you’re running into issues, you’re on your own, but yes, it will usually run, but don’t take it for granted.”. That would be cool. Because in this case I had to buy the game (there is no demo!) to run the test. I will admit I actually tried to evaluate the situation using cracked copies, but it seems the cracks are actually less compatible to XP x64 than the actual game. ‘Cause the cracked binaries all crashed. The legit game does not! It was risky to buy without knowing, but at least now I can share this with you!

The configuration of the successful test:

  • ASUS P6T Deluxe mainboard, X58 chipset
  • Intel Core i7 980X @ 4.0GHz
  • 48GB of DDR-III/1456 CL10 memory
  • 2 x EVGA GeForce GTX 580 3GB Classified Ultras (running Metro Last Light in Single card mode, need to update the driver for SLI)
  • 314.07 GeForce driver w.o. Metro Last Light support (the nVidia game ready driver for the game is version [320.18])

So there you go! Another desperate win for the dying breed! ;)

Apr 022013
 

Frog ASPIToday I would like to discuss a few useful tricks for Windows XP x64. Yeah. Again. But this time around, it’s at least partially about stuff that you can’t easily find on the Internet anymore, whether it’s the information or the actual software part that’s missing. There are several software components that are sometimes hard to use or find for XP x64. Some are even hard to set up for regular 32-bit Windows XP. The following solutions will be discussed:

  • 1.) The ASPI Layer (digital media access)
  • 2.) UDF 2.5 (for Blu-Ray and HD-DVD access)
  • 3.) exFAT (some modern cameras etc. need this file system)
  • 4.) EXT2/3/4 (for Linux and UNIX compatibility)
  • 5.) Universal SSD TRIM (keep your SSD clean, tidy and fast, like on modern systems)

So, let’s start with #1:

1.) The ASPI layer:

Edit: Note that in the meantime, the FrogASPI solution described here is no longer the only option. Another one that works even for 64-bit applications accessing the ASPI layer has been found, see further below!

1.a) FrogASPI:

One of those things that have been abandoned by Microsoft is the ASPI layer, the Advanced SCSI Programming Interface. Meant as a way for digital storage media access it was most prominently used to read from optical drives digitally (also on ATA, not just SCSI), so you didn’t need to rip audio CDs via MSCDEX on Win98 or via the crappy analog link from your drive to the sound card. ASPI can also be used to access other types of storage devices, but this is the most important part. Some older software, like my beloved Xing AudioCatalyst (an ancient CD ripper including the Fraunhofer mp3 encoder) might still need ASPI to access audio CDs.

However, Adaptec stopped shipping ASPI drivers for Microsoft Windows after Microsoft had abandoned the API and introduced its own replacement called SPTI, the SCSI PassThrough Interface. As a matter of fact, you can still install Adaptecs ASPI layer on 32-Bit Windows XP, as it includes a working 32-Bit kernel driver. So for 32-Bit XP, it’s still fine. However, there is no such driver for XP x64 (and also not for 32/64-Bit Vista/7/8). So, no ASPI at all?

For a loong time I indeed had to live without it, until I found that french guy named Millenod (That’s his nick name, I will not disclose his real name) who had written a 100% userspace ASPI layer, that would work on any newer Windows operating system on x86, no matter the bitness. This is called FrogASPI, and unfortunately, Millenods website for it is down by now. In its core, it is actually just an SPTI wrapper. Back in those days, I even wrote the guy an email, thanking him for his work. Here is a part of his reply:

“FrogAspi is effectively an SPTI wrapper. I decided to work in “user” mode, instead of kernel ones, for many reasons.. It was for me the fastest way to implement a generic ASPI layer, which is not OS specific as drivers.”

-Millenod, Developer of FrogASPI

After renaming the FrogAspi.dll to the proper name wnaspi32.dll and putting it into %WINDIR%\SysWOW64\ for 64-Bit Windows, it can be used by any ASPI-enabled application. For 32-Bit Windows, please use %WINDIR%\system32! See, what Adaptecs own aspichk.exe has to say about what we just did:

Adaptec ASPI Check

You’ll see that some files are reported as missing. You do not have to care about that though, ASPI32.SYS would’ve been the 32-Bit kernel driver, WOWPOST.EXE is a 16-Bit Windows ASPI helper tool, WINASPI.DLL the corresponding 16-Bit Windows driver. None of those are needed. Now, that FrogASPI is mapping all ASPI calls to SPTI, we can begin to actively use it even on 64-Bit Windows. See AudioCatalyst for instance, with ASPI being available:

AudioCatalyst using FrogASPI

AudioCatalyst reading the Postal Original Soundtrack CD using FrogASPI

As you can see, everything works just fine. Oh, and besides, in case you want AudioCatalysts CDDB feature back (as shown in this screenshot), just add the following lines to your AudioCatalyst.ini, sitting in the programs installation folder, this’ll switch from the now-broken CDDB to FreeDB:

  1. CDDBServer=freedb.org
  2. CDDBHTTPPath=/~freedb/cddb.cgi
  3. CDDBLocation=Random FreeDB Site
  4. CDDBTCPIPPort=8880
  5. CDDBConnectVia=TCP/IP

You can download FrogASPI right here:

Unfortunately I cannot give you a download link for Xing AudioCatalyst, as it’s commercial software even today.

1.b) StarBurn ASPI (An update from 2016-12-05):

In the meantime, user Steve Sybesma has [commented] about a different solution provided by [StarBurn Software]. With their recording software comes a native ASPI layer for both 32-bit and 64-bit systems with full compatibility for even 64-bit programs that want to use ASPI. I decided to fetch the necessary DLLs from their SDK and release them in the form of a standalone InnoSetup installer for Windows 2000 and newer. The installer will auto-detect your operating systems’ bitness and will install the 32-bit ASPI layer on 32-bit systems as well as both the 32-bit as well as the 64-bit one on 64-bit systems.

Here it is:

It has been successfully tested on Windows 2000 Pro SP4, Windows XP Pro SP3, Windows XP Pro x64 Edition SP2 as well as Windows 10 Pro x64. If you don’t want to trust my installer, that’s fine as well of course. You can just install the StarBurn software from its original source, it should give you the same level of ASPI support, just with more additional stuff being installed!

Now, to #2, the UDF 2.5 file system for HD-DVDs and Blu-Rays:

2.) UDF 2.5:

Now this one was a real bitch. It took me months to finally get it nailed! UDF 2.5 is a file system. And it’s used for basically two type of media: HD-DVD discs and Blu-Ray discs, including the most modern 3D Blu-Rays. It just bugged me like hell, that I couldn’t access the discs as I normally would in Windows Explorer or on the command line. Windows XP simply does not have the proper file system kernel driver. And while it’s relatively easy to find one for 32-Bit WinXP, it’s immeasurably harder to find the single one existing driver for Windows XP x64. I’m not even sure if it still exists on the web..

One day I came across a person whose name I forgot, but that guy had searched the web for months just like me, and he found the driver just before it went offline. So he re-uploaded it, with his single file sharing link in some post on some unknown website being the only thing that saved my bacon. Maybe one day, somebody will find the XP x64 UDF 2.5 driver here on my site after desperate searching? Who knows.

So, the only existing driver has been developed by Panasonic/Matsushita, a company that also builds optical drives. It works on Windows XP x64 as well as Server 2003 x64. And here it is, together with the Toshiba-made UDF 2.5 driver for regular 32-Bit Windows XP just for completeness:

Again, I would like to emphasize that it took me months to find that freaking 64-Bit XP driver, I’m not even sure what Panasonic/Matsushita developed it for, but here it is for your enjoyment. After installation and reboot, you can browse a HD-DVD or Blu-Ray just like any CD or DVD disc, see here:

UDF 2.5 at work in XP x64

The Blu-Ray folder structure of the german uncut version of “The Boondock Saints”, viewed using the Panasonic/Matsushita UDF 2.5 driver for XP x64

And now, #3, the exFAT file system:

3.) exFAT:

exFAT is a successor to the older FAT32 file system that uses a 64-bit address field instead, allowing for larger than 4GB files, the most notorious limitation of regular FAT. exFAT is now being used on newer cameras and other handheld devices for memory card formatting. Since the comparably heavy-weight NTFS is typically not used on such devices, exFAT is the replacement we get. Also, exFAT is the official file system for SDXC and Memory Stick XC cards, so they may very well come exFAT-preformatted! The only sour part is, that exFAT is kind of proprietary (Microsoft), like NTFS. That’s a bit crappy, but well.

I have however already written an article about exFAT on XP x64, so I will just link to it. exFAT is also very easy to get for both 32-Bit and 64-Bit flavors of Windows XP:

And now, finally, some free file systems from the Linux world:

4.) EXT2/3/4:

EXT is one of the most widely if not the most widely used file system in the entire world of Linux. It has also been ported to other systems like BSD, Solaris or ReactOS. And also Windows actually, although the implementation is not the most seamless you can dream up. But still, you can get read/write access to all currently used EXT versions under Windows, even the fast new EXT4.

The software that I found most useful for this is called [ext2fsd], short for EXT2 file system driver. It comes with the actual driver plus a helper tool for mounting and for enabling write access. It would be nice to just be able to mount EXT filesystems seamlessly from the start, without having to use that mounting tool, but we’re not that far yet it seems. However, drives can be configured to auto-mount later, so you need the tool only once for each drive. Write access has to be enabled by hand.

Currently, ext2fsd supports the following systems: NT 5.0 (Windows 2000), NT 5.1 (Windows XP 32-Bit), NT 5.2 (Windows XP x64, Server 2003), NT 6.0 (Windows Vista), NT 6.1 (Windows 7, Server 2008). Where not explicitly mentioned, it supports both 32-Bit and 64-Bit versions of said operating systems. And here is what it looks like in action:

EXT4 mounted on XP x64

A Linux-created EXT4 filesystem mounted on XP x64 for read/write using ext2fsd

The only strange thing is, that ext2fsd calls the EXT4 file system “EXT3”. It shouldn’t. But other than that, it works just fine. I haven’t tested a lot of r/w, but so far it worked fine, without any corruption or anything. The helper tool also gives you a function for flushing the write cache, and a nice statistics window for monitoring the file system usage while mounted. It also allows the user to manually unmount of course. Even mkfs is there, so you can format drives as EXT under Windows, the only useful tool missing would be fsck for checking & repairing such filesystems. But you can’t have everything I guess.

So, usage is not as seamless as it would be with native NTFS/FAT32/exFAT drivers, but it’s ok and it greatly boosts the Linux interoperability of XP x64 and other Windows systems. Also, EXT filesystems can easily be used in other UNIX-style systems like Solaris, BSD or MacOS X, so this allows a lot of different systems to exchange data easily using removable sticks or harddrives. Since it of course supports large files (>4GB) and it’s more easy to handle across a wide spectrum of operating systems, unlike exFAT it might one day become the #1 choice for data exchange, replacing FAT32 and NTFS in that field. However, today, NTFS on Linux is probably a lot more widespread than this EXT driver on Windows, so there is still a long way to go.

Of course, there is a downside to this too: All the POSIX permissions like ownership, read/write/execute bits etc. just do not work on Windows. You can neither read nor write any such meta data using ext2fsd. I have neither checked the umask that ext2fsd uses yet, nor what ownership it sets, but I would guess it’s root:root with permissions 777 or rwxrwxrwx. But whatever it is, you will most likely need to take care of that when mounting on your UNIX-style system.

I hope this was helpful, if you have any comments about these helpful add-ons, just post away!

5.) Universal SSD TRIM:

This is a 2014-11-03 edit of the original article.

Usually, I would tell users who would like to use an SSD on Windows XP or XP x64 to re-align their partitions with a [gparted ISO] after installation to get proper alignment, and more importantly to use either Intel, Corsair or Samsung SSDs so that they can TRIM their SSDs properly using their respective manufacturers’ XP-compatible software tools, maintaining full write speed over the entire life time of the SSD drive.

It seems that I can now finally confirm, that the latter criterion no longer needs to be met. Basically, I found an easy way to TRIM any SSD on Windows XP or Windows XP x64, as long as the following logical criteria are met:

  • The SSD itself needs to support TRIM. Well, that was obvious.
  • The controller driver needs to support TRIM. Now I can’t speak for just any driver, but I have tested this on Intels ICH10-R and an ancient ICH7-M, both worked just fine with AHCI drivers installed. Usually, AHCI is what you want. Avoid IDE/legacy PATA modes. You may need to configure this properly in your systems BIOS and you may need to retrofit AHCI drivers in XP if your system was installed using IDE mode to avoid bluescreens on boot. If you need help doing that, please ask away in the comments.

So, what’s the solution? It’s called the ADATA SSD Toolbox! Basically, ADATA developers seem to have forgot or were actually never instructed to install an artificial vendor obstruction inside their software. So what they did was to write an SSD toolbox that just complies to the SATA standard, implementing its TRIM command for any compatible SSD, no-holds-barred! I have tested this on an Intel 320 SSD, as well as on a Crucial m500 SSD now, despite no official XP support being present:

So there you go! Now pretty much any TRIM capable SSD can be made to accept TRIM on Windows XP and XP x64! Together with gparted for partition alignment, we have a full-featured SSD solution for XP, leaving nothing to be desired!

Download the ADATA SSD Toolbox here: [ADATA SSD Toolbox v2.0.1].

Feb 282013
 

Windows 7 LogoYou may or may not know that I am the materialized GUI nazi, both on UNIX and Windows systems. For Windows, it’s the Windows 2000 UI that I consider superior above all others. So today I’m looking at Windows 7 and how I tried to make it look more like Windows 2000. Actually, it’s not that much about “looks” really, but function.

What I hated most about 7 was how it wastes screen real estate by using super-fat bars and window borders as well as partially huge icons and lots of unnecessary information that clogs the UI, like “libraries” or “favorites” or that “folder band” in Windows Explorer. Of course you may find them useful.

I however do not.

I do find some new features useful though, like the start menu search field. I intended to keep all that was good in Windows 2000 and XP x64 using the 2000 UI and integrate useful new features into it. I am not going to explain what I did in detail (lots of registry hacking, DLL hacking and using the [Classic Shell] for the rest). So instead I am just going to provide you a few links:

In the end, you might see something like this, quite classic with a little bit of useful modernness (this is a german Test-Windows 7):

Windows 7 classic

So, as you can see, this is not really ready for use, and not really fully tidied up yet. It’s just a test anyway. This is already saving a lot more space than the original Windows 7 Basic or Aero setup. Window borders are far slimmer, all active bars are classic, all thick ones that cannot be properly resized have been removed. Looking somewhat ok for now, but there are a few downsides.

Microsoft did remove lots of desktop composition features from the classic theme, that were still active back in the days of Windows XP and 2000. So while it looks classic, not all features of the original classic UI are there. Like you won’t see transparent representations of icons when you move them around on the desktop. The icon will just disappear and reappear as you release the mouse button on its designated target position. Lame! But that may be unfixable.

Rumours are saying that this composition functionality is still present in Windows Vista classic. I’m gonna check that out for sure! For now: Partial success.

Update: The desktop composition is indeed more complete in Windows Vistas classic theme than it is on Windows 7. On Vista, the transparent icons will be shown while dragging them to a new position. Means that finally, Vista is doing something better than 7. Sigh…

Nov 112012
 

Dishonored LogoDeveloped by Arkane Studios and published by Bethesda, “Dishonored” is supposed to be a really cool new stealth / assassination style game with a deep story and a tense atmosphere amplified by really nice artwork. If you want to get an impression of this game, please watch the [debut trailer] and the [gameplay trailer]. Meanwhile, the cool song from the gameplay trailer has gotten so much attention, that it has been released as an [mp3] including its own [remix contest].

Basically, the game is set in a steampunkish world with swords, muzzle-loader pistols but also some Tesla high tech stuff. And in the dark corners of the plague-ridden, rotting city there is even magic. So, everything seems cool, and even though this is a console port, it seems to be one well-done at least. Yeah, the textures aren’t fly, even if you choose to fully [tune the engine], but the watercolor style is still more than nice enough to look at in my opinion.

Now, the one question that is interesting to me of course: Why does Bethesda insist this game needs Windows Vista/7 as a minimum? The renderer is DirectX 9.0c, sitting in an Unreal Engine 3, like many modern games. So why? Maybe some special function calls like in XCOM that need to be hacked to work? Maybe some modern API calls that do not exist on XP? Or something else? No. None of those. It was simply laziness in the software testing process I assume. Or maybe Bethesda just didn’t want to care to have to offer official XP support. Whatever the reasons are, it works just fine anyway, see here (click to enlarge):

Dishonored running on Windows XP Professional x64 Edition

So there we go! Another Vista/7 only game, that works just fine on Windows XP. Make sure though that you’re running your XP with SP3, and your XP x64 with SP2. You should really run the latest patchlevels anyway. So if you have everything up to date, you should be just fine. No modifications or hacks required for this one!

Oct 312012
 

XCOM Enemy unknown LogoWhen it was announced that Sid Meiers company Firaxis would do a sequel for the UFO/XCOM series in a classic style, fans rejoiced! UFO: Enemy unknown and XCOM: Terror from the Deep were two of the best strategy & tactics games ever. Plain and simple. Now, XCOM: Enemy unknown was released recently, based on the Unreal Engine 3. What some people didn’t understand, since it was DirectX 9.0c only: Why did it require NT6 aka Windows Vista/7/8?

Well, Firaxis decided to use some of NT6 kernels features to open files. Sounds very simple. It is. But the functions that Firaxis chose to use are NT6 only. Actually, Microsoft provides a DLL called filext.dll that brings those functions to XP and XP x64, but Firaxis chose not to use it. Or maybe they just didn’t care enough. So they’re calling a very few NT6-exclusive functions in kernel32.dll, which are simply not there on XP and XP x64.

A user on the steam forums called [KawaiiSara] now debugged that problem and wrote a stub DLL called zernel32.dll that intercepts all kernel32.dll calls, then redirects the NT6 specific ones to the filext.dll that Microsoft provides and passes through the rest to the real kernel32.dll. To make that work, all kernel32.dll calls in the games EXE file have to be replaced with zernel32.dll (or whatever you wanna call your own stub DLL) calls using a hex editor.

Then, user [ScavengerSpb] developed a similar stub DLL based on KawaiiSaras code for Windows XP x64 Edition. While KawaiiSara has posted both the binary DLLs as well as the source code, ScavengerSpb has as of yet only published the binary versions, so use this at your own risk!

Links:

This is confirmed to work both on the demo as well as on the final version of the game. Of course, there is no official support for this hack, so use this at your own risk. It might break in the future due to patches or whatever. Also, when Steam decides to patch the games EXE, the hex edit needs to be reapplied, or it won’t be calling zernel32.dll anymore.

Still, KawaiiSara, ScavengerSpb, you’re my heroes of the day! If ScavengerSpb decides to fulfill my request to publish his XP x64 source code, I will update this post. The code can easily be built using Microsoft Visual Studio Express (2008/2010), which is free.

For your enjoyment, see the following screenshots of the XCOM: Enemy unknown demo version running on Windows XP x64 Edition thanks to the work of KawaiiSara and ScavengerSpb (click to enlarge to 2560×1600):

XCOM Enemy unknown ingame screenshot 1

XCOM Enemy unknown ingame screenshot 2

Thank god that there are some good Hackers out in this world! What would I do without them?

Update: ScavengerSpbs source code for XP x64 added to the links!