Mar 062015
 

F6 logoThis is a problem I’ve wanted to solve for quite some time now: How to install older Windows on systems with an AHCI SATA or maybe SCSI or SAS controller, when there is no floppy drive and Windows setup won’t recognize any hard drive? Usually, you’ll have to press F6 during the early setup sequence, and then provide the drivers with either a real floppy, or a USB floppy drive that’s on the hardware compatibility list (not all USB floppies work). While adding non-supported USB floppies to any installation disc is possible when using [nLite] (see farther below), sometimes you may not want to build your own image or maybe you don’t have any floppies available, USB included.

What I always wanted to do was to find a way to just use some USB pendrive for this, like it’s possible with newer Windows systems. And yesterday, I finally found the solution. My original attempts to modify the Windows CD itself were all unsuccessful, and today I believe that it is hardly possible to just talk setup into accepting USB pendrives as floppies, unless your systems BIOS can actively mask the pendrive as such. The solution is much more weird than any such modification actually.

But this time around, I won’t dive into the technical details up front, but simply give you the solution first. If you’re interested in how this really works, just read the lower part 3.)!

1.) How to use any USB drive as a F6 floppy medium for storage driver loading during legacy Windows setup

1.a.) Let’s go step by step. You’ll need to be on Microsoft Windows for this. First we need nu2’s bfi.exe[1] plus a few additional files. I have packaged them all together for you in this file: [buildf6]. Please unpack the archive, and enter the newly created folder buildf6\. In that folder you’ll see a subfolder f6\. Put the storage drivers you wish to use to support your target system in that folder. Please keep in mind, that it has to be F6 capable drivers. Usually these are small and consist of a txtsetup.oem file plus some *.inf and *.sys, maybe *.dll files. When you’ve placed all necessary drivers in there, run the floppy image builder script: buildf6.bat. This will create a 1.44MiB image file called f6.img.

In case you put too many drivers in the f6\ folder and it’s growing too large to build the image file, you can also alter the script to have it build a larger 2.88MiB floppy image instead of the 1.44MiB one. Just open buildf6.bat with your favorite text editor, it’ll show this:

  1. @echo off
  2. cls
  3. .\bfi\bfi.exe -v -t=144 -b=boot\boot_sector.bin -f=.\f6.img -o=BCDL.BIN -o=BCDL-IDE.SYS -o=VIDE-CDD.SYS .\boot\chainload_cd .\f6
  4. pause

Simply change the -t option to -t=288 to get a larger floppy image which can hold more drivers.

1.b.) Next thing we need is [Syslinux] to prepare ourselves a bootable and extremely minimalistic Linux system. Actually, it’s not even Linux at all, it’s just a bootloader. The latest version 6.xx can be obtained directly from [here] at kernel.org. Unpack it, launch cmd.exe and enter the following subdirectory of Syslinux on the shell: bios\win32\ or maybe bios\win64\ in case you’re on a 64-bit Windows machine. Plug your desired USB key into the machine. I’ll assume drive letter F: here. Run the following command to prepare your USB drive, and beware – all data stored on it will be lost:

syslinux.exe --mbr --active --install F:

Please copy your previously created image buildf6\f6.img over to F:\, and also copy Syslinux’ bios\memdisk\memdisk file directly to F:\. Don’t create any subfolders, all files belong in the root directory. Now, create a new text file F:\syslinux.cfg, open it with an editor and enter the following:

default 1
 
label 1
kernel memdisk
append initrd=f6.img floppy

Save the file. The contents of the USB drive should now look like this:

Our F6 USB sticks content

Our bootable F6 USB sticks’ content

2.) Run it

Now, you need to put the Windows 2000/XP/2003 CD into the target machines optical drive, plug the USB drive in, and set the systems BIOS to boot from USB. If you’re on a UEFI-based machine, make sure to boot in legacy/BIOS mode. The machine will boot from the USB drive, and then go on and continue booting from the CD, as usual. When the prompt for pressing F6 comes up, please do so. When Windows setup asks you to provide a floppy for the intended drivers, confirm that you’ve already put it in, and setup will magically pick it up. See the following screenshots:

XP x64 accessing the virtual F6 floppy

XP x64 accessing the virtual F6 floppy

XP x64 will now load the Areca SAS controller driver - no real floppies involved

XP x64 will now load the Areca SAS controller driver – no real floppies involved

As said, this even works with Windows 2000. Possibly even with Windows NT 4.0 and older systems capable of booting from a CD, but I haven’t tested systems that old. I tried it with Windows 2000 Professional though, and it just works:

Windows 2000 accessing the virtual floppy drive

Windows 2000 accessing the virtual floppy drive (German)

Windows 2000 will load a Mylex AcceleRAID 352 driver for installation

Windows 2000 will load a Mylex AcceleRAID 352 driver for installation (German)

Just keep booting from the USB stick until setup is complete, so that the virtual floppy drive is present throughout all stages of the installation process. As soon as Windows is completely installed, shut down the machine, remove USB drive and CD, and you’re good to go!

Now, for the addendum…

3.) What’s really going on here

I mentioned that I believe that Windows can’t be tricked into accepting any arbitrary USB drive as a F6 floppy. If possible at all, it’d require a low-level binary hack, as Windows setup is calling the hardware interrupt 13hex to access the floppy drive, which is about as low-level as it gets. And that’s hardcoded… somewhere. For the USB floppy part, well, you can widen the field of supported drives (see even [further below]), but according to my attempts you can’t tell Windows to treat a USB device of class 08, subclass 06 and interface 80 – like a USB pendrive – just like a class 08, subclass 04 and interface 00 device – like a real USB floppy. For more information please refer to the [USB mass storage specifications].

So what we’re doing instead is something much more freakish. The exact process that this solution goes through looks like this:

  1. The system BIOS executes the boot code within the boot sector of the USB drive.
  2. The boot code boots Syslinux’ ldlinux.sys loader.
  3. Syslinux continues to higher stages by loading the Syslinux core module, ldlinux.c32.
  4. Syslinux now pulls our floppy image f6.img into a ramdisk, and executes it as if it was an operating system kernel as specified in syslinux.cfg, effectively chainloading it (It also redirects INT13hex to the virtual floppy, so when any code calls the interrupt, it’ll get to the Ramdisk instead of any real floppy!).
  5. This works, because the floppy image itself has a boot sector that the x86 host processor can execute directly as it’s x86 machine code!
  6. Instead of a real operating system, the virtual floppies’ boot sector now executes BCDL.BIN, a pure machine code executable, which was embedded into the image by buildf6.bat from buildf6\boot\chainload_cd\ together with its CD drivers BCDL-IDE.SYS and VIDE-CDD.SYS which support real IDE as well as SATA optical drives.
  7. BCDL.BIN loads its drivers and chainloads the CD by employing standard El Torito optical drive booting.
  8. The Ramdisk stays where it is, posing as a real A: INT13hex floppy drive until reboot.
  9. Windows setup boots from CD.
  10. Besides BCDL, we also have our required storage drivers on that floppy image. Windows 2000/XP/2003 may now load them by locating and reading txtsetup.oem just as it always did.

It may sound complicated, but once you know what to do and how, it’s a pretty easy and fast process. With nothing but your USB key, the drivers and the installation CD you can set up ancient Windows pretty much anywhere, as long as you still have an optical drive at least. I’d imagine this to be specifically useful for notebooks.

4.) The USB floppy thing – How to add non-supported drives

This is just for completeness’ sake. Should you wish to just keep using a USB floppy, but unfortunately have a newer unsupported one, you can add it to your setup CDs HCL. This requires copying the entire CD off to another Windows machine, modifying a small file and rebuilding a new ISO using [nLite] though.

You can find the file on your installation CD in \I386\ for 32-bit Windows (2000/XP/2003) and in \AMD64\ for 64-bit Windows (XP x64/2003 x64), it’s called TXTSETUP.SIF. Open it with your favorite text editor and locate the following part in the Section [HardwareIdsDatabase]:

txtsetup.sif, USB part of the [HardwareIdsDatabase] section

TXTSETUP.SIF, USB part of the [HardwareIdsDatabase] section

Depending on your operating system, this may be found on totally different line numbers. Now you need to determine the USB vendor ID and device ID of your USB floppy drive. I usually do this on Linux or BSD UNIX by running lsusb (FreeBSD needs the usbutils package for that, # pkg install usbutils). Here is an example with some USB device, not a floppy, but just so you know what to look for, it’s the two 4-letter hexadecimal codes separated by a colon, the first is the vendor ID, the second the device ID:

lsusb showing USB devices on FreeBSD

lsusb showing USB devices on FreeBSD (click to enlarge)

You can also get it on Windows though, just go to “Start\Settings\Control Panel”, and then “System\Hardware\Device Manager”. Locate your USB drive by exploding the USB part of the device tree on the very bottom, right click the correct device and choose “Properties”, then “Details”. Et voilà:

Getting a USB devices vendor ID (VID) and device ID (PID) on Windows

Getting a USB devices vendor ID (VID_****) and device ID (PID_****) on Windows

Patching it in is a simple process of adding an according line to TXTSETUP.SIF, like this:

Adding an unsupported USB floppy device in TXTSETUP.SIF (shown with an invalid non-floppy device here, but it works like that with any unsupported real USB floppy drive).

Adding an unsupported USB floppy device in TXTSETUP.SIF (shown with an invalid non-floppy device here, but it works exactly like that with any unsupported real USB floppy drive).

As said, you’ll have to build yourself a new ISO from that using nLite and burn it to CD. But with that, you can now use any previously unsupported USB floppy drive for F6 driver installations.

5.) Profit

So that’s how it can be achieved after all, and I always thought it to be impossible. It’s quite the obscure method, and its gain in usefulness over nLite driver embedding is maybe debatable, but I believe this can be faster and more flexible. That may be especially true if you have to deal with lots of different machines with different SATA AHCI driver requirements or other storage controllers. Building new ISOs using nLite over and over again can be time consuming after all.

[1] Lagerweij, B. BFI is © 2000-2015 Nu2 productions. All rights reserved.

May 282014
 

YaCy logoJust recently I have published my [vision for our networked society], claiming that freedom, self-determination and independence can be reached through decentralization, putting control over our services and data back into the hand of the users. My idea was to use distributed hash tables on a lower level to power search engines or social networks, distributed across a wide field of user-operated home servers. I thought my idea was pure utopia. Something we’d need to work hard for years to accomplish.

After I published it, users approached me via various channels, pointing out already existing software that deals with the centralization problems and dangers of today, like for instance the decentralized social network [Diaspora*] or more significantly even, [YaCy], which is a DHT-based search engine just like I envisioned it.

Let me show you the simple way the YaCy developers chose to show us, what they’re doing exactly. If you’ve read my article about decentralization linked at the beginning, you’ll immediately recognize what’s going on here (Images taken from YaCy):

So you can see, where this is going? In the right direction is where! And how is it implemented? Basically a Java server built on top of the [Apache Solr] / [Lucene] full text search engine well known in certain enterprises with a web interface on top. The web interface can be used for administration and as a simple web search, like we know it already. The Java code works with both Oracle Java 1.7 (Windows, MacOS X) as well as OpenJDK 1.7, which is preferred on Linux. I haven’t tested it, but I presume it might also work on BSD UNIX, as some BSD systems do support OpenJDK 1.7 too. Could also work on OpenSolaris I guess, and it can run with user privileges.

If you want to go the Oracle route on Linux, this also seems to work, at least for me, despite the YaCy developers asking for OpenJDK. But then again, if you wanna stay clear of any even remotely fishy software licenses, just go with OpenJDK!

In case you haven’t noticed yet, I have already joined the YaCy DHT network as a node, and the search using my node as an entry point into the DHT superstructure is embedded in this weblog already, look at the top right and you’ll find it! Mind you, it ain’t the fastest thing on the track, and the quality of its results won’t yet match Google or anything, but we’re getting there! I may also choose to embed it at [http://www.xin.at], not just here. But we’ll see about that.

Also, the web interface has a few nice monitoring gadgets, let’s have a look at those for my own fresh node, too:

Now, YaCy doesn’t provide data all by itself. Like in my original idea, the network needs to pull in data from outside the superstructure, from the regular Internet and make it searchable. For that, YaCys administration web features a simple crawler that you can send off to index everything on a certain domain via HTTP/HTTPS, like “http://wp.xin.at/”, or from a Samba/Windows share server, or from local files, or FTP etc. There is also a more complex, extremely configurable and powerful crawler, but I’ve only used the simple one so far. And it also visualizes what it does, look here:

So the web interface is pretty cool, and it actually works, too! The Crawler also has parsers for tons of file types, like PDF, Office documents (Microsoft and Open-/LibreOffice), media files, ZIP files etc., so it can index the contents and/or meta data of such files too, full text!

While I do not need it, you may actually also disconnect YaCy from the “freeworld” network entirely and use it as an intranet search engine. There is even professional support for that if you’d like to use it in that context within your company.

So there we go, a free, decentralized search engine, that lies not in the hand of some opaque megacorporation, but in our very own hands. How could I’ve missed this?! I have no idea. But it seems even I have walked the world in blindness too, for the three pillars of my vision are more real than I’d have thought; Independence, Self-determination, Freedom. It’s all right there!

And you don’t even need a home server for that. You can just run it on your desktop or laptop too. Not perfect, but this works due to the massively fail-proof nature of the DHT network, as described in my earlier publication.

Seriously, this is pretty damn awesome! We’re far closer to my “stage II” than I would’ve believed just 2 weeks ago. All we need to do now is to solve the social problem and make people actually migrate to freedom! Yeah, it’s the hardest of problems, but at least we have the tech sitting there, ready to be used.

So now it’s your turn (and mine actually): Let’s inform people, lets educate whomever we can, and help ourselves lose the chains!!

Jan 222014
 

clink logoTypically, any avid Linux/UNIX user would sneer at the default Windows command line shell, cmd.exe. I do use it from time to time though, for tools like [eac3to] or [x264]. By Bash and other UNIX shells’ standards, cmd.exe is simply weak, powerless and inefficient though. On top of that, its scripting language – simply called Batch – is quite awkward and nothing compared to say Bash. You can go the classic “do one thing well” [1] route and extend the Batch functionality with specialized tools like grep, awk and sed for Windows, or even by alternative scripting languages like MS VBScript or Perl, which I sometimes do. But oh well, we’re not gonna fix that.

But using the awkward interactive shell? We can fix that! Say hello to [clink]!

Welcome to cmd+clink

Welcome to cmd+clink

So what’s clink? A way of enhancing cmd.exe (and partly even the newer [Windows PowerShell]) with the [GNU readline] libraries’ functionality and with other upgrades. “Aha” you might think, “so how’s that gonna help?”. Well, it’s going to help by adding tons of stuff you would normally only see on a UNIX-style shell, I’ll give you a few examples:

  • A better auto-completion that is not limited to auto-completing local file names:
    • cmd can still auto-complete file and folder names in your current directory by pressing <TAB>. It will now do so incrementally though!
    • cmd can now also auto-complete all executable program names in your %PATH%! You wanna launch notepad? Type “not”, press <TAB>, <ENTER>, done. If there are ambiguous results, double-<TAB> will show you a list of eligible file names or other auto-completable objects!
    • It also works for environment variables. You want to go to your users home directory in %USERPROFILE%? Type “cd %u”, press <TAB>, it will auto-complete to “cd %user”, as there are three variables called %USERPROFILE%, %USERDOMAIN% and %USERNAME%. Extend that by adding a p: “cd %userp”, press <TAB> again and you got the whole variable, then <ENTER>. All that can greatly speed up your shell usage!
    • If all that’s not enough for you, you can extend and modify the auto-completion behavior using the Lua scripting language.
  • The command history is no longer “Doskey” style. It will stay persistent across sessions, and is now incrementally searchable (<CTRL>+<R> / <CTRL>+<S>)!
  • The clipboard integration is now far better. Simply paste stuff from the clipboard to the shell using <CTRL>+<V> like everywhere else, or use the right mouse button.
  • Powerful line editing features like delete last word with <CTRL>+<W>, blank screen (<CTRL>+<L>), undo (<CTRL>+<Z>)  or other stuff like quit the shell (<CTRL>+<D>), scroll around without having to use the mouse (<SHIFT>+<PgUp> and <SHIFT>+<PgDn>) and much, much more.

A few sample shots of the auto-completion:

Auto-completing a program in the search path

Auto-completing a program name in the search path

Auto-completing an environment variable

Auto-completing an environment variable

One feature I didn’t get to work yet was environment variable expansion using <CTRL>+<ALT>+<E>, which works on bash in UNIX. Maybe I did something wrong there, but in case I didn’t, let’s see how fast [new bug reports] (link removed due to project migration to github, the issue described remains unfixed, but exists only on XP, XP x64 & Server 2003 systems) are attended to in the clink project.

But even without that, it’s pretty powerful already, and I love it, so much more convenient. And there I was, just looking for a way to quit the shell with <CTRL>+<D> and got so much more instead! :)

[1] Arnold Robbins, Nelson H. F. Beebe. Classic Shell Scripting, page 4. O’Reilly Media, Inc., 2005.

Jan 202014
 

ZFS logoI have been using PC-BSD UNIX for testing purposes since I think October last year. Since I only created a 30GB virtual disk image for that system in VMware Player, I started running out of space recently. As PC-BSD uses the ZFS as its root file system – originally developed by Sun – and everybody seems to think that it’s the end of all means, I thought lets try and grow my file system online!

First of all, this thing has storage pools, or so-called zpools. A zpool can have entire disks or partitions (called “slices” on BSD, as partitions are something slightly different there) as members. Then, it can make RAID arrays out of that, or add a fast SSD as a caching or log device etc.

When you’re done with your zpool, you need the actual ZFS on top as far as I understand this (I’m still very new to ZFS). There you can have extremely nice features like block-level data deduplication or transparent compression like on NTFS.

So, how to grow this thing? Since I didn’t dare to resize the slice with the BSD disklabel and BSD partition structures inside, I went ahead and just added another slice that I would then just plug into my root storage pool called “tank”. BSD doesn’t use conventional structures like other operating systems. It can use a “slice” (meaning a regular MBR/GPT partition), but it will create its own “BSD disklabel” inside, which is like another partition table, and then it will create its BSD style partitions using that disklabel. Typically, other operating systems cannot handle BSD disk structures. But they can handle “slices”, so I did add another such “slice” or partition with a bootable gparted ISO, but I believe you can reach the same goal by running this as root given the disk /dev/da0, filling the remaining disk space up completely with that new slice:

gpart add -t freebsd da0

Ok, this is the current status before modifying anything else, as you can see, I have only 3.2GB of space left on my ZFS:

[thrawn@unixbox ~]$ zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
tank                      24.7G   3.2G    31K  legacy
tank/ROOT                 11.5G   3.2G    31K  legacy
tank/ROOT/default         11.5G   3.2G  11.5G  /mnt
tank/tmp                  99.5K   3.2G  99.5K  /tmp
tank/usr                  13.0G   3.2G    31K  /mnt/usr
tank/usr/home             4.26G   3.2G    32K  /usr/home
tank/usr/home/thrawn      4.26G   3.2G  4.26G  /usr/home/thrawn
tank/usr/jails              31K   3.2G    31K  /usr/jails
tank/usr/obj                31K   3.2G    31K  /usr/obj
tank/usr/pbi              8.03G   3.2G  8.03G  /usr/pbi
tank/usr/ports             778M   3.2G   536M  /usr/ports
tank/usr/ports/distfiles   241M   3.2G   241M  /usr/ports/distfiles
tank/usr/src                31K   3.2G    31K  /usr/src
tank/var                  6.82M   3.2G    31K  /mnt/var
tank/var/audit              33K   3.2G    33K  /var/audit
tank/var/log               442K   3.2G   442K  /var/log
tank/var/tmp              6.33M   3.2G  6.33M  /var/tmp

Now, to make sure our zpool plus its ZFS file system will grow automatically when adding new devices, run the following, activating automatic expansion for our zpool:

[thrawn@unixbox] ~# zpool set autoexpand=on tank

Time to add stuff to the zpool now. Growing ZFS itself shouldn’t be necessary, as it scales with the zpool, at least that’s what I think it does. What I tried to do then was to add the new slice to the pool “tank”, but that failed initially. See what the zpool command had to say about what I attempted to do:

[thrawn@unixbox] ~# zpool add tank /dev/da0s2
cannot add to 'tank': root pool can not have multiple vdevs or separate logs

Ok so… What the hell? I googled around a bit and found out, that this seems to be a leftover limitation coming from SunOS/Solaris, where ZFS originally comes from. This problem means, we cannot add data, log or cache devices to this pool, because its the one we’re booting from. There is a dirty little trick that will make zpool believe it’s just a data pool though! Just clear the bootfs string, then retry and add bootfs after that again, as we wouldn’t have a boot volume otherwise. Like that:

[thrawn@unixbox] ~# zpool set bootfs= tank
[thrawn@unixbox] ~# zpool add tank /dev/da0s2
[thrawn@unixbox] ~# zpool set bootfs=tank tank

The first line clears the flag by setting it to an empty string for the zpool “tank”, then we’re adding the new slice to it, and finally, we’re making “tank” our bootfs again. That’s a bit scary, so you should probably make sure you have a full backup system image before trying this. For me, it did work, and immediately afterwards I got my +30GB from the added slice:

[thrawn@unixbox ~]$ zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
tank                      24.7G  33.2G    31K  legacy
tank/ROOT                 11.5G  33.2G    31K  legacy
tank/ROOT/default         11.5G  33.2G  11.5G  /mnt
tank/tmp                  99.5K  33.2G  99.5K  /tmp
tank/usr                  13.0G  33.2G    31K  /mnt/usr
tank/usr/home             4.26G  33.2G    32K  /usr/home
tank/usr/home/thrawn      4.26G  33.2G  4.26G  /usr/home/thrawn
tank/usr/jails              31K  33.2G    31K  /usr/jails
tank/usr/obj                31K  33.2G    31K  /usr/obj
tank/usr/pbi              8.03G  33.2G  8.03G  /usr/pbi
tank/usr/ports             778M  33.2G   536M  /usr/ports
tank/usr/ports/distfiles   241M  33.2G   241M  /usr/ports/distfiles
tank/usr/src                31K  33.2G    31K  /usr/src
tank/var                  6.82M  33.2G    31K  /mnt/var
tank/var/audit              33K  33.2G    33K  /var/audit
tank/var/log               440K  33.2G   440K  /var/log
tank/var/tmp              6.33M  33.2G  6.33M  /var/tmp

Took a deep breath, rebooted, and it worked just fine. And it works instantly and online! There is no lengthy resizing process involved for ZFS, it seems to just grow on demand, writing data to the new underlying devices as needed. It IS pretty awesome actually.

But I still have a lot to learn there, as all these tools like zpool and zfs are pretty weird, and I’m not used to BSD disk structures or the ZFS structures. For somebody like me, who has not really handled anything but MBR/GPT style primary partitions before (minus some Windows dynamic disks), it’s actually quite confusing. But I’ll get used to it I guess.

With all those other super powerful features that ZFS has, I think this was just a glimpse of what can be achieved!

Oh, and before I forget to mention it, some initial help for my hopeless self was provided by [kmoore134] on the [PC-BSD forums].

Sep 192013
 

Knoppix logoSince 2 or 3 years I have been building a Knoppix-based Linux-on-CD distribution for teaching (exercises and exams) at our University. A colleague of mine has written the proper server/client tools for source code submission and evaluation while I have adapted and hardened the system for our needs, based on tons of modifications on the CD and also on the server side, from where a lot of local behavior is controlled by server-side scripts which can dis-/enable the network or USB ports on clients, but which can also shut clients down or verify the media to effectively prevent CD forgery (Its pretty cool to see them students try, so far they never came even remotely close to breaking it).

Now I am supposed to ensure that the system can use sound. It seems some Java-based exercises may contain sound creation and playback in the future. To make sure the system can support new sound chips (which it currently does not) and potentially also new GPUs I tried to hack a new Linux kernel into the system to replace the good old 2.6.32.6, which had the ALSA sound system built in, so I couldn’t update it as a kernel module. Also, updating the whole 6.2.1 thing to Knoppix 7 is just not an option, as it would be far too much work. The kernel update it had to be.

And so the adventure began…

Continue reading »

Aug 272013
 

Miranda logoOk, a quick one, just so that I can’t forget that piece of information: For my beloved old ICQ 2003a #3800, received files could be stored in folders that were named after the senders user name. Quite handy, as I also linked that against the same users with the same nick names on other services. However, file transfers with good old ICQ 2003a don’t work anymore these days, so I switched over to Miranda on Windows (and licq on Linux) quite some time ago.

Typically, Miranda stores received files in a folder named %miranda_path%\Received Files\%userid%, where %miranda_path% is some sub folder in the users profile directory, and %userid% is the ICQ UIN of the user, a simple numeric value. What I wanted was to get the old ICQ behavior back, I wanted to have Miranda store the files in a folder named after the senders nick, not the UIN. That piece of information is hard to come by, if you don’t want to read the source code. Luckily, I found the information on the Miranda forums.

Just use %nick% instead of %userid%. And don’t try stuff like %username%, as this is a Windows-internal variable that Miranda will also interpret on top of the in-core Miranda variables. Unfortunately there is no public list of all internal Miranda variables that you can use… But yeah. That one I now know at least.

Jul 192013
 

Paperless office logo 1I am not sure whether you guys are suffering from the same pain as I did, but have you ever tried to sell something, and then the buyer wants the original receipt, so that he can use it in case he or she needs the manufacturer warranty? Yeah. If you have a mess like I do with your receipts, then you’ll have to search for a long, long time before you get to that paper. And every single time you go through that mighty stack, it gets less and less well-sorted. Same for school certificates that I needed recently, or medical statements etc.

paperless office logo 3

So, to avoid that mess, and finally get order into that crazy document chaos, what are you gonna do? Well, you can properly sort the stuff in folders with index cards, but for me, that just doesn’t work out. I simply can’t avoid the chaos permanently. I might have a very detailed and flawless order on my file system, but as strict as it is there, as chaotic it is in real life. So it seems there is an idea forming already: If the digital realm is in order, and the real life one is in chaos, why not transfer all data from one into the other? No more crawling through stacks of paper.

Sounds nice, right? Well, I’ve been looking into options for weeks now, and it only took that long, because said options are extremely expensive. What I was looking at? Document scanners. Fast and small little desktop devices that would scan all your documents, perform optical character recognition within the scanning workflow to make all documents full-text searchable, and then batch-save them to disk. Of course, in my case it also had to be Windows XP x64 compatible. Luckily, a lot of the slightly more professional devices (instead of the SoHo ones) are XP x64, sometimes even Win2000 compatible. After looking at HP, Kodak, Canon, Brother and Fujitsu, I finally decided that it was going to be the slightly older (read: cheaper) Fujitsu fi-6110.

A colleague of mine had bought the ScanSnap ix500 previously, and I was quite impressed with the results, he was using it to digitize tons of schooling material. Also Fujitsu, but the SoHo variant. Well, I went one notch up, so here is the 500€ Fujitsu fi-6110 document scanner! Oh, yeah, the photos are abysmally bad, I still suck at using the Nikon V1:

This thing is really tiny, but very heavy. It seems there is a lot of solid metal in that thing. According to the [manufacturers specifications], it’s supposed to be able to handle roughly 2000 sheets a day. Not too shabby I’d say. Well, so far I have generated over 2200 PDF files in less than a week, which probably means something in between 3000 or 4000 sheets of paper already. It’s not slow.

The crazy thing is: Even torn paper, or extremely crumpled paper is usually fed in without any trouble. Also, the scanner can take mixed formats, say Letter, A4, A5 and even arbitrary sized sales slips, all in one go. When it generates the according PDF files, the page size will be adapted automatically, as will paper fed in in an inclined position. With thousands of partially really massively challenging sheets already, I had only 3 or 4 jams and a few double feeds that the scanner detects with an ultrasonic sound sensor.

Also, there are lots of other nice options like blind color dropout/remapping (if you can’t see “red”, you can let the software fix that), blank page skipping, blah! Also, you get TWAIN and ISIS drivers with this, which is not available on the SoHo series. So if you just wanna scan some picture, you can just do that and import it to say Photoshop or Corel PhotoPaint via TWAIN. Sure, that ScanalldPro software looks like it has collected a lot of dust over the years, but it’s not bad, reasonably powerful and low on resource usage.

So, how does it work in the end? Together with my little [Windows Search 4.0 on XP x64 hack], it works pretty damn well!

I can now just type a product name, a schools name or a doctors name in that little search field in my task bar, and WDS 4.0 will immediately pop up the documents I might be looking for, and fast too, thanks to the index. Both the WDS 4.0 stock PDF filter and Adobes own [XP x64 iFilter] work just fine. Now I’m definitely going to digitize every single document, contract, receipt, whatever, and make it searchable. It is really that smooth! Sure it was expensive, maybe even too much so. But it does make my life a lot easier I must say. Not a bad piece of equipment. Also, besides hard drives it’s the first piece of Fujitsu hardware I have ever owned. Makes a damn good impression so far!

Jul 182013
 

Buffalo logoSince a colleague of mine has [rooted] our Buffalo Terastation III NAS (TS-XLC62) at work a while back, we changed the remote shell from Telnet to SSH and did a few other nice hacks. But there is one problem: The TS-XLC62 does not monitor the hard drives’ health by SMART, even though parts of the required smartmontools are installed on the tiny embedded Linux system. They’re just stitting there unused, just like the sshd before.

Today I’m going to show you how you can make this stuff work and how to enable SMART email notifications on this system, which has no standard Linux mail command, but a Buffalo-specific tmail command instead. We will enable the background smartd service, and configure it properly for this specific Terastation model. All of the steps shown here are done on a rooted TS-XLC62, so make sure you’re always root here:

Buffalo Terastation IIIThe smartmontools on the box are actually almost complete. Only the drive database and init scripts are missing, and for some reason, running update-smart-drivedb on the system would fail. So we need to get the database from another Linux/UNIX or even Windows machine running smartmontools. Usually, on Linux you can find the file here: /usr/local/share/smartmontools/drivedb.h“. Copy it onto the Terastation using scp from another *nix box: scp /usr/local/share/smartmontools/drivedb.h root@<terastation-host-or-ip>:/usr/local/share/smartmontools/“. You can use [FileZilla] or [puTTY] to copy stuff over from a Windows machine instead.

Note that this only makes sense if you have smartmontools 5.40 or newer (smartctl -V tells the version). Older releases cannot have their drive databases updated seperately, but it will most likely still work fine.

Now, log in to your Terastation using Telnet or SSH, and you can test whether it’s working by running a quick info check on one of the hard drives. We will need to specify the controller type as marvell, as the SATA controller of the Marvell Feroceon  MV78XX0 SoC in the box cannot be addressed by regular ATA/SCSI commands. Run:

smartctl -d marvell -i /dev/sda

In my case I get this, as I have already replaced the first failing Seagate hard drive with an even crappier WD one already (yeah, yeah, I know, but it was the only one available), it’s also not yet known by the smartmontools database:

smartctl version 5.37 [arm-none-linux-gnueabi] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model:     WDC WD20EARX-00PASB0
Serial Number:    WD-WCAZAL555899
Firmware Version: 51.0AB51
User Capacity:    2,000,398,934,016 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Thu Jul 18 09:54:53 2013 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Now that that’s done we should make sure that smartmontools’ daemon called smartd will be running in the background doing regular checks on the drives. But since we will need to configure email notifications for that, we need to make sure that smartd can send emails first. The Terastation has no mail command however, only some Buffalo tmail command, that is no valid drop-in replacement for mail as the syntax is different.

So we need to write some glue-code, that will then later be invoked by smartd. I call this mailto.sh, and I’ll place it in /usr/local/sbin/. It’s based on [this article], that gave me a non-working solution on my Terastation for several reasons, but that’s easily fixed up, so it shall look somewhat like this (you’ll need to fill in several of the variables with your own data of course), oh, and as always, don’t forget to do chmod 550 on it when it’s done:

expand/collapse source code
  1. #! /bin/bash
  2. ##############################################################
  3. # Written as glue code, so that smartmontools/smartd can use #
  4. # Buffalos own "tmail", as we don't have "mail" installed    #
  5. # on the Terastation.                                        #
  6. ##############################################################
  7.  
  8. # User-specific declarations:
  9.  
  10. TMP_FILE=/tmp/Smartctl.error.txt
  11. SMTP=&lt;PUT SMTP HOST NAME HERE&gt;
  12. SMTP_PORT=25
  13. FROM=&lt;PUT "FROM" EMAIL ADDRESS HERE&gt;
  14. SUBJECT="SMART Error"
  15. FROM_NAME=&lt;PUT "FROM" NAME HERE. LIKE "TERASTATION MAILER"&gt;
  16. ENCODING="UTF-8"
  17. BYTE=8
  18.  
  19. # Code:
  20.  
  21. # Write email metadata to the temp file (smartd gives us this):
  22. echo To:  $SMARTD_ADDRESS &gt; $TMP_FILE 
  23. echo Subject:  "$SMARTD_SUBJECT" &gt;&gt; $TMP_FILE 
  24. echo &gt;&gt; $TMP_FILE 
  25. echo &gt;&gt; $TMP_FILE 
  26.  
  27. # Save the email message (STDIN) to the temp file:
  28. cat &gt;&gt; $TMP_FILE 
  29.  
  30. # Append the output of smartctl -a to the message:
  31. smartctl -a -d $SMARTD_DEVICETYPE $SMARTD_DEVICE &gt;&gt; $TMP_FILE 
  32.  
  33. # Now email the message to the user using Buffalos mailer:
  34. tmail -s $SMTP -t $SMARTD_ADDRESS -f $FROM -sub $SUBJECT \
  35. -h $FROM_NAME -c $ENCODING -b $BYTE -s_port $SMTP_PORT &lt; $TMP_FILE 
  36.  
  37. # Delete temporary file
  38. rm -f $TMP_FILE

So this is our mailer script wrapping the stuff coming from smartd's invocation of mail around Buffalos own tmail. Now how do we make smartd call this? Let’s edit /usr/local/etc/smartd.conf to make it happen, fill in your email address where it says here, like you changed all the variables in mailto.sh before:

  1. # Monitor all four harddrives in the Buffalo Terastation with self-tests running
  2. # on Sunday 01:00AM for disk 1, 02:00AM for disk 2, 03:00AM for disk 3 and 04:00AM
  3. # for disk 4:
  4.  
  5. /dev/sda -d marvell -a -s L/../../7/01 -m &lt;EMAIL&gt; -M exec /usr/local/sbin/mailto.sh
  6. /dev/sdb -d marvell -a -s L/../../7/02 -m &lt;EMAIL&gt; -M exec /usr/local/sbin/mailto.sh
  7. /dev/sdc -d marvell -a -s L/../../7/03 -m &lt;EMAIL&gt; -M exec /usr/local/sbin/mailto.sh
  8. /dev/sdd -d marvell -a -s L/../../7/04 -m &lt;EMAIL&gt; -M exec /usr/local/sbin/mailto.sh

Now if you want to test the functionality of the mailer beforehand, you can use this instead:

/dev/sda -d marvell -a -s L/../../7/01 -m &lt;EMAIL&gt; -M exec /usr/local/sbin/mailto.sh -M test

To test it, just run smartd -d on the shell. This will give you debugging output including some warnings caused by a bit of unexpected output that tmail will pass to smartd. This is non-critical though, it should look similar to this:

smartd version 5.37 [arm-none-linux-gnueabi]
Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Opened configuration file /usr/local/etc/smartd.conf
Configuration file /usr/local/etc/smartd.conf parsed.
Device: /dev/sda, opened
Device: /dev/sda, is SMART capable. Adding to "monitor" list.
Monitoring 1 ATA and 0 SCSI devices
Executing test of /usr/local/sbin/mailto.sh to <EMAIL> ...
Test of /usr/local/sbin/mailto.sh to <EMAIL> produced unexpected 
output (50 bytes) to STDOUT/STDERR: 
smtp_port 25
Get smtp portnum 25
pop3_port (null)

Test of /usr/local/sbin/mailto.sh to <EMAIL>: successful

Now you can kill smartd on a secondary shell by running the following command. We will be re-using this in an init script later too, as the Terastation init functions are leaving quite a lot to be desired, so I’ll go into the details a bit:

kill `ps | grep smartd | grep -v grep | cut -f1 -d"r"`

This command will get the process id of smartd and feed it to the kill command. The delimiter “r” is used for the cut command, because whitespace won’t work in some cases where the leading character of the ps output is also a whitespace, so it’ll match the first letter of the user running smartd, which has to be root.

To understand this better, just run ps | grep smartd | grep -v grep while smartd is running. If the PID is 5-digit, the leading character will be a number from the PID, but if it is 4-digit, the leading character is a whitespace instead, which would make cut -f1 -d " " report an empty string in our case, hence cut -f1 -d"r"… Very dirty, I know… Don’t care though. ;) You may remove the -M test directive from /usr/local/etc/smartd.conf now, if you’ve played around with that, so the smart spam will stop. :roll:

Finally, to make our monitoring run as a smooth auto-starting daemon in the background, we will need to write ourselves that init script. The default smartmontools one won’t work out of the box, as a few functions like killproc or daemon are missing on the Terastations embedded Linux. Yeah, I was too lazy to port them over. So a few adaptions will make it happen in a simplified fashion. See this reduced and adapted init script called smartd sitting in /etc/init.d/:

expand/collapse source code
  1. #! /bin/sh
  2. SMARTD_BIN=/usr/local/sbin/smartd
  3.  
  4. RETVAL=0
  5. prog=smartd
  6. pidfile=/var/lock/subsys/smartd
  7. config=/usr/local/etc/smartd.conf
  8.  
  9. start()
  10. {
  11.         [ $UID -eq 0 ] || exit 4
  12.         [ -x $SMARTD_BIN ] || exit 5
  13.         [ -f $config ] || exit 6
  14.         echo -n $"Starting $prog: "
  15.         $SMARTD_BIN $smartd_opts
  16.         RETVAL=$?
  17.         echo
  18.         [ $RETVAL = 0 ] &amp;&amp; touch $pidfile
  19.         return $RETVAL
  20. }
  21.  
  22. stop()
  23. {
  24.         [ $UID -eq 0 ] || exit 4
  25.         echo -n $"Shutting down $prog: "
  26.         kill `ps | grep smartd | grep -v grep | cut -f1 -d"r"`
  27.         RETVAL=$?
  28.         echo
  29.         rm -f $pidfile
  30.         return $RETVAL
  31. }
  32.  
  33. *)
  34.         echo $"Usage: $0 {start|stop}"
  35.         RETVAL=2
  36.         [ "$1" = 'usage' ] &amp;&amp; RETVAL=0
  37.  
  38. esac
  39.  
  40. exit $RETVAL

So yeah, instead of killproc we’re making due with kill and most of the service functions have been removed, limiting the script to start and stop. Plus, it will not check for multiple start invocations in this version, so it’s possible to start multiple smartd daemons and stop will only work for one running process at a time, so you’ll need to pay attention. Could be fixed easily, but I think it’s good enough that way. To make smartd start on boot, link it properly, somewhat like that, I guess S90 should be fine:

ln -s /etc/init.d/smartd /etc/rc.d/sysinit.d/S90smartd

Also, you can start and stop smartd from the shell more conveniently now without having to run smartd in the foreground and kill it from a secondary shell as it doesn’t have CTRL+C kill it. You can now just do these two things instead, like on any other SysVinit system, only with the limitations described above:

root@TS-XLC62:~# /etc/init.d/smartd stop
Shutting down smartd: Terminated
root@TS-XLC62:~# /etc/init.d/smartd start
Starting smartd: 
root@TS-XLC62:~#

Better, eh? Now, welcome your SMART monitoring-enabled Buffalo Terastation with email notifications being sent on any upcoming hard drive problems detected by courtesy of smartmontools! :cool:

Edit: And here is a slighty more sophisticated init script, that will detect whether smartd is already running or not on start, so that multiple starts can no longer happen. It will also detect if smartd has been killed from outside the scope of the init scripts (like when it crashed or something) by looking at the PID file:

expand/collapse source code
  1. #! /bin/sh
  2. SMARTD_BIN=/usr/local/sbin/smartd
  3. RETVAL=0
  4. prog=smartd
  5. pidfile=/var/lock/subsys/smartd
  6. config=/usr/local/etc/smartd.conf
  7.  
  8. start()
  9. {
  10.   [ $UID -eq 0 ] || exit 4
  11.   [ -x $SMARTD_BIN ] || exit 5
  12.   [ -f $config ] || exit 6
  13.   if [ -f $pidfile ]; then
  14.     echo "PID file $pidfile found! Will not start,"
  15.     echo "smartd probably already running!"
  16.     PID=`ps | grep smartd | grep -v grep | grep -v "smartd start" | cut -f1 -d"r"`
  17.     if [ ${#PID} -gt 0 ]; then
  18.       echo "Trying to determine smartd PID: $PID"
  19.     elif [ ${#PID} -eq 0 ]; then
  20.       echo "No running smartd process found. You may want to"
  21.       echo "delete $pidfile and then try again."
  22.     fi
  23.     exit 6
  24.   elif [ ! -f $pidfile ]; then
  25.     echo -n $"Starting $prog: "
  26.     $SMARTD_BIN $smartd_opts
  27.     RETVAL=$?
  28.     echo
  29.     [ $RETVAL = 0 ] &amp;&amp; touch $pidfile
  30.     return $RETVAL
  31.   fi
  32. }
  33.  
  34. stop()
  35. {
  36.   [ $UID -eq 0 ] || exit 4
  37.   PID=`ps | grep smartd | grep -v grep | grep -v "smartd stop" | cut -f1 -d"r"`
  38.   if [ ${#PID} -eq 0 ]; then
  39.     echo "Error: No running smartd process detected!"
  40.     echo "Cleaning up..."
  41.     echo -n "Removing $pidfile if there is one... "
  42.     rm -f $pidfile
  43.     echo "Done."
  44.     exit 6
  45.   elif [ ${#PID} -gt 0 ]; then
  46.     echo -n $"Shutting down $prog: "
  47.     kill `ps | grep smartd | grep -v grep | grep -v "smartd stop" | cut -f1 -d"r"`
  48.     RETVAL=$?
  49.     echo
  50.     rm -f $pidfile
  51.     return $RETVAL
  52.   fi
  53. }
  54.  
  55. case "$1" in
  56.   start)
  57.     start
  58.     ;;
  59.   stop)
  60.     stop
  61.     ;;
  62.   restart)
  63.     stop
  64.     start
  65.     ;;
  66.   status)
  67.     ps | grep smartd | grep -v grep | grep -v status
  68.     RETVAL=$?
  69.     ;;
  70.   *)
  71.     echo $"Usage: $0 {start|stop|restart|status}"
  72.     RETVAL=2
  73.     [ "$1" = 'usage' ] &amp;&amp; RETVAL=0
  74. esac
  75.  
  76. exit $RETVAL

May 042013
 

Corsair logoCorsair, originally known only for memory modules, has since expanded into many different markets, like power supplies, enclosures, watercooling solutions, input devices and last but not least solid state drives. Corsairs offerings of the latter kind are typically based on a variety of SSD controller chips from LAMD (like the LM87800) and Sandforce (SF-2100, SF-2200) to the now widespread and well-received Marvell controllers (88SS9174).

Now, there are currently two competitors in the market which offer users a software tool to manage their solid state drives, one is Intel with its SSD toolbox, and the other is Samsung with its SSD magician. Both fully support Windows XP and Windows XP x64 Edition so you can get TRIM support for those older operating systems that cannot issue the ATA command on their own, like say Windows 7 does. The result is that there will be very little write performance degradation. None at all, as long as you don’t fully fill the SSD up.

So, TRIM always makes sure that there is some space on the SSD marked as free, so write commands can be issued instead of expensive read-modify-writes. And now, Corsair enters the fray, providing its own SSD toolbox, that also supports Windows XP operating systems and features several configuration options. See the following screenshots:

So, as you can see, Corsairs SSD toolbox supports not only TRIM, but also a reconfiguration of their drives overprovisioning proportions. That means you can take away some usable space and add it to the spare area that is being used to reallocate damaged regions of the usable area to itself. Also, you can shrink the spare area, increasing the amount of usable space. Neat.

Unfortunately, I do not currently own a Corsair SSD, so I can’t really test the functionality, as it doesn’t work with my Intel SSD of course. But I think since the Corsair SSD toolbox is not “always on” like Samsungs SSD magician, its TRIM scheduling option will most likely work by using the internal Windows task scheduler, like the Intel solution does. Preferrable if you ask me, because that way, the toolbox doesn’t always need to run in the background.

So now it’s three companies officially supporting TRIM on Windows XP: Intel, Samsung and Corsair!

Apr 022013
 

Frog ASPIToday I would like to discuss a few useful tricks for Windows XP x64. Yeah. Again. But this time around, it’s at least partially about stuff that you can’t easily find on the Internet anymore, whether it’s the information or the actual software part that’s missing. There are several software components that are sometimes hard to use or find for XP x64. Some are even hard to set up for regular 32-bit Windows XP. The following solutions will be discussed:

  • 1.) The ASPI Layer (digital media access)
  • 2.) UDF 2.5 (for Blu-Ray and HD-DVD access)
  • 3.) exFAT (some modern cameras etc. need this file system)
  • 4.) EXT2/3/4 (for Linux and UNIX compatibility)
  • 5.) Universal SSD TRIM (keep your SSD clean, tidy and fast, like on modern systems)

So, let’s start with #1:

1.) The ASPI layer:

Edit: Note that in the meantime, the FrogASPI solution described here is no longer the only option. Another one that works even for 64-bit applications accessing the ASPI layer has been found, see further below!

1.a) FrogASPI:

One of those things that have been abandoned by Microsoft is the ASPI layer, the Advanced SCSI Programming Interface. Meant as a way for digital storage media access it was most prominently used to read from optical drives digitally (also on ATA, not just SCSI), so you didn’t need to rip audio CDs via MSCDEX on Win98 or via the crappy analog link from your drive to the sound card. ASPI can also be used to access other types of storage devices, but this is the most important part. Some older software, like my beloved Xing AudioCatalyst (an ancient CD ripper including the Fraunhofer mp3 encoder) might still need ASPI to access audio CDs.

However, Adaptec stopped shipping ASPI drivers for Microsoft Windows after Microsoft had abandoned the API and introduced its own replacement called SPTI, the SCSI PassThrough Interface. As a matter of fact, you can still install Adaptecs ASPI layer on 32-Bit Windows XP, as it includes a working 32-Bit kernel driver. So for 32-Bit XP, it’s still fine. However, there is no such driver for XP x64 (and also not for 32/64-Bit Vista/7/8). So, no ASPI at all?

For a loong time I indeed had to live without it, until I found that french guy named Millenod (That’s his nick name, I will not disclose his real name) who had written a 100% userspace ASPI layer, that would work on any newer Windows operating system on x86, no matter the bitness. This is called FrogASPI, and unfortunately, Millenods website for it is down by now. In its core, it is actually just an SPTI wrapper. Back in those days, I even wrote the guy an email, thanking him for his work. Here is a part of his reply:

“FrogAspi is effectively an SPTI wrapper. I decided to work in “user” mode, instead of kernel ones, for many reasons.. It was for me the fastest way to implement a generic ASPI layer, which is not OS specific as drivers.”

-Millenod, Developer of FrogASPI

After renaming the FrogAspi.dll to the proper name wnaspi32.dll and putting it into %WINDIR%\SysWOW64\ for 64-Bit Windows, it can be used by any ASPI-enabled application. For 32-Bit Windows, please use %WINDIR%\system32! See, what Adaptecs own aspichk.exe has to say about what we just did:

Adaptec ASPI Check

You’ll see that some files are reported as missing. You do not have to care about that though, ASPI32.SYS would’ve been the 32-Bit kernel driver, WOWPOST.EXE is a 16-Bit Windows ASPI helper tool, WINASPI.DLL the corresponding 16-Bit Windows driver. None of those are needed. Now, that FrogASPI is mapping all ASPI calls to SPTI, we can begin to actively use it even on 64-Bit Windows. See AudioCatalyst for instance, with ASPI being available:

AudioCatalyst using FrogASPI

AudioCatalyst reading the Postal Original Soundtrack CD using FrogASPI

As you can see, everything works just fine. Oh, and besides, in case you want AudioCatalysts CDDB feature back (as shown in this screenshot), just add the following lines to your AudioCatalyst.ini, sitting in the programs installation folder, this’ll switch from the now-broken CDDB to FreeDB:

  1. CDDBServer=freedb.org
  2. CDDBHTTPPath=/~freedb/cddb.cgi
  3. CDDBLocation=Random FreeDB Site
  4. CDDBTCPIPPort=8880
  5. CDDBConnectVia=TCP/IP

You can download FrogASPI right here:

Unfortunately I cannot give you a download link for Xing AudioCatalyst, as it’s commercial software even today.

1.b) StarBurn ASPI (An update from 2016-12-05):

In the meantime, user Steve Sybesma has [commented] about a different solution provided by [StarBurn Software]. With their recording software comes a native ASPI layer for both 32-bit and 64-bit systems with full compatibility for even 64-bit programs that want to use ASPI. I decided to fetch the necessary DLLs from their SDK and release them in the form of a standalone InnoSetup installer for Windows 2000 and newer. The installer will auto-detect your operating systems’ bitness and will install the 32-bit ASPI layer on 32-bit systems as well as both the 32-bit as well as the 64-bit one on 64-bit systems.

Here it is:

It has been successfully tested on Windows 2000 Pro SP4, Windows XP Pro SP3, Windows XP Pro x64 Edition SP2 as well as Windows 10 Pro x64. If you don’t want to trust my installer, that’s fine as well of course. You can just install the StarBurn software from its original source, it should give you the same level of ASPI support, just with more additional stuff being installed!

Now, to #2, the UDF 2.5 file system for HD-DVDs and Blu-Rays:

2.) UDF 2.5:

Now this one was a real bitch. It took me months to finally get it nailed! UDF 2.5 is a file system. And it’s used for basically two type of media: HD-DVD discs and Blu-Ray discs, including the most modern 3D Blu-Rays. It just bugged me like hell, that I couldn’t access the discs as I normally would in Windows Explorer or on the command line. Windows XP simply does not have the proper file system kernel driver. And while it’s relatively easy to find one for 32-Bit WinXP, it’s immeasurably harder to find the single one existing driver for Windows XP x64. I’m not even sure if it still exists on the web..

One day I came across a person whose name I forgot, but that guy had searched the web for months just like me, and he found the driver just before it went offline. So he re-uploaded it, with his single file sharing link in some post on some unknown website being the only thing that saved my bacon. Maybe one day, somebody will find the XP x64 UDF 2.5 driver here on my site after desperate searching? Who knows.

So, the only existing driver has been developed by Panasonic/Matsushita, a company that also builds optical drives. It works on Windows XP x64 as well as Server 2003 x64. And here it is, together with the Toshiba-made UDF 2.5 driver for regular 32-Bit Windows XP just for completeness:

Again, I would like to emphasize that it took me months to find that freaking 64-Bit XP driver, I’m not even sure what Panasonic/Matsushita developed it for, but here it is for your enjoyment. After installation and reboot, you can browse a HD-DVD or Blu-Ray just like any CD or DVD disc, see here:

UDF 2.5 at work in XP x64

The Blu-Ray folder structure of the german uncut version of “The Boondock Saints”, viewed using the Panasonic/Matsushita UDF 2.5 driver for XP x64

And now, #3, the exFAT file system:

3.) exFAT:

exFAT is a successor to the older FAT32 file system that uses a 64-bit address field instead, allowing for larger than 4GB files, the most notorious limitation of regular FAT. exFAT is now being used on newer cameras and other handheld devices for memory card formatting. Since the comparably heavy-weight NTFS is typically not used on such devices, exFAT is the replacement we get. Also, exFAT is the official file system for SDXC and Memory Stick XC cards, so they may very well come exFAT-preformatted! The only sour part is, that exFAT is kind of proprietary (Microsoft), like NTFS. That’s a bit crappy, but well.

I have however already written an article about exFAT on XP x64, so I will just link to it. exFAT is also very easy to get for both 32-Bit and 64-Bit flavors of Windows XP:

And now, finally, some free file systems from the Linux world:

4.) EXT2/3/4:

EXT is one of the most widely if not the most widely used file system in the entire world of Linux. It has also been ported to other systems like BSD, Solaris or ReactOS. And also Windows actually, although the implementation is not the most seamless you can dream up. But still, you can get read/write access to all currently used EXT versions under Windows, even the fast new EXT4.

The software that I found most useful for this is called [ext2fsd], short for EXT2 file system driver. It comes with the actual driver plus a helper tool for mounting and for enabling write access. It would be nice to just be able to mount EXT filesystems seamlessly from the start, without having to use that mounting tool, but we’re not that far yet it seems. However, drives can be configured to auto-mount later, so you need the tool only once for each drive. Write access has to be enabled by hand.

Currently, ext2fsd supports the following systems: NT 5.0 (Windows 2000), NT 5.1 (Windows XP 32-Bit), NT 5.2 (Windows XP x64, Server 2003), NT 6.0 (Windows Vista), NT 6.1 (Windows 7, Server 2008). Where not explicitly mentioned, it supports both 32-Bit and 64-Bit versions of said operating systems. And here is what it looks like in action:

EXT4 mounted on XP x64

A Linux-created EXT4 filesystem mounted on XP x64 for read/write using ext2fsd

The only strange thing is, that ext2fsd calls the EXT4 file system “EXT3”. It shouldn’t. But other than that, it works just fine. I haven’t tested a lot of r/w, but so far it worked fine, without any corruption or anything. The helper tool also gives you a function for flushing the write cache, and a nice statistics window for monitoring the file system usage while mounted. It also allows the user to manually unmount of course. Even mkfs is there, so you can format drives as EXT under Windows, the only useful tool missing would be fsck for checking & repairing such filesystems. But you can’t have everything I guess.

So, usage is not as seamless as it would be with native NTFS/FAT32/exFAT drivers, but it’s ok and it greatly boosts the Linux interoperability of XP x64 and other Windows systems. Also, EXT filesystems can easily be used in other UNIX-style systems like Solaris, BSD or MacOS X, so this allows a lot of different systems to exchange data easily using removable sticks or harddrives. Since it of course supports large files (>4GB) and it’s more easy to handle across a wide spectrum of operating systems, unlike exFAT it might one day become the #1 choice for data exchange, replacing FAT32 and NTFS in that field. However, today, NTFS on Linux is probably a lot more widespread than this EXT driver on Windows, so there is still a long way to go.

Of course, there is a downside to this too: All the POSIX permissions like ownership, read/write/execute bits etc. just do not work on Windows. You can neither read nor write any such meta data using ext2fsd. I have neither checked the umask that ext2fsd uses yet, nor what ownership it sets, but I would guess it’s root:root with permissions 777 or rwxrwxrwx. But whatever it is, you will most likely need to take care of that when mounting on your UNIX-style system.

I hope this was helpful, if you have any comments about these helpful add-ons, just post away!

5.) Universal SSD TRIM:

This is a 2014-11-03 edit of the original article.

Usually, I would tell users who would like to use an SSD on Windows XP or XP x64 to re-align their partitions with a [gparted ISO] after installation to get proper alignment, and more importantly to use either Intel, Corsair or Samsung SSDs so that they can TRIM their SSDs properly using their respective manufacturers’ XP-compatible software tools, maintaining full write speed over the entire life time of the SSD drive.

It seems that I can now finally confirm, that the latter criterion no longer needs to be met. Basically, I found an easy way to TRIM any SSD on Windows XP or Windows XP x64, as long as the following logical criteria are met:

  • The SSD itself needs to support TRIM. Well, that was obvious.
  • The controller driver needs to support TRIM. Now I can’t speak for just any driver, but I have tested this on Intels ICH10-R and an ancient ICH7-M, both worked just fine with AHCI drivers installed. Usually, AHCI is what you want. Avoid IDE/legacy PATA modes. You may need to configure this properly in your systems BIOS and you may need to retrofit AHCI drivers in XP if your system was installed using IDE mode to avoid bluescreens on boot. If you need help doing that, please ask away in the comments.

So, what’s the solution? It’s called the ADATA SSD Toolbox! Basically, ADATA developers seem to have forgot or were actually never instructed to install an artificial vendor obstruction inside their software. So what they did was to write an SSD toolbox that just complies to the SATA standard, implementing its TRIM command for any compatible SSD, no-holds-barred! I have tested this on an Intel 320 SSD, as well as on a Crucial m500 SSD now, despite no official XP support being present:

So there you go! Now pretty much any TRIM capable SSD can be made to accept TRIM on Windows XP and XP x64! Together with gparted for partition alignment, we have a full-featured SSD solution for XP, leaving nothing to be desired!

Download the ADATA SSD Toolbox here: [ADATA SSD Toolbox v2.0.1].