It seems that two months after the last maintenance in May another one needs to be done on 2015-07-08 around 00:00 AM – 06:00 AM UTC+1. Again this means that all XIN.at services may see some downtime during this period, this weblog included. So no eMail server, no web server, no anything. By trend these maintenances tend to put me offline for short periods of time only, but who knows what UPCs gonna do. Just so you know.
NFC – or “near-field communication” technology is a now-booming system for sending and receiving small chunks of data over very short distances. You may have heard about modern cellphones supporting the system to read information from small tags – in essence chips you stick onto something to provide local, small pieces of information. This can serve augmented reality purposes for instance, in a sense at least, providing metadata about objects anywhere in the world.
It’s nowadays also being used for payment though, both in conjunction with smartphones and their active NFC chips as well as debit/credit banking cards and their integrated, passive NFC circuitry.
- NFC basics
- NFC-capable banking cards
- Using a modern Android phone to fetch data from a banking card
- The theft issue
- Modern cards may be more close-lipped
- Killing NFC for good
1.) NFC basics
So there are connections between active chips (say: phone to phone) as well as active-passive ones, in which case the active side (a phone, an electronic cashier) will talk to the passive one. In the latter case, the active chip will generate an electromagnetic field which reaches a copper coil embedded in the passive device or tag, creating enough inductive voltage to power that passive NFC chip.
According to information that can be found on the web and in some specifications, the range should be about 20cm with data transfer rates of 106kbit/s, 212kbit/s or 424kbit/s, and in some non-standard cases 848kbit/s. That’d be 13.25kiB/s, 26.5kiB/s, 53kiB/s or 106kiB/s respectively. The time to build up a connection is around one tenth of a second. There are NFC range extenders [like this one] for active chips however, which can boost the range up to almost 1 meter! And that’s were the alarms start ringing in my head.
Now, why is any of that dangerous to begin with? Because it’s being used for payments and because there may be a significant information leaking issue with some of those banking cards.
2.) NFC-capable banking cards
First of all, I’d like to thank two of my colleagues, which shall remain anonymous, for providing a.) a fully affected debit card and b.) a NFC-capable Android smartphone.
Let’s take a look at our affected card (click to enlarge images, as usual):
Now this is not my own card, so I didn’t have unlimited access to it. Since my own cards – both debit and credit – were not NFC-capable yet, I simply ordered a new one from my bank. There are other people on the web who used CT/X-Ray like [here] or [here] to visualize the internals of such cards, but I wanted a cheap solution that every layman can copy easily. As a matter of fact, any bright light (even a cellphones LED flash, when used as a torch) is sufficient, see here:
For more clarity, see the next image:
Now this coil has two functions: First – as mentioned above – it provides inductive voltage and with it up to 15mA of power to run the NFC chip and potentially some flash memory. Second, it also is the NFC chips’ antenna to properly receive the signal on NFCs 13.56MHz radio frequency spectrum. So, how about we talk to that chip a little ourselves, now shall we?
3.) Using a modern Android phone to fetch data from a Banking card
A Frenchman named [Julien Millau] luckily has developed an Android app called “Banking card reader NFC (EMV)”, which you can find on [Google Play] for free, including the source code as it’s licensed under the [Apache License, v2.0]. There are other apps too, tailored towards cards with local features (I’ll get to those later), but this is a good, generic one.
So what you’ll need is an NFC-capable Android smartphone, that app, and some banking card with NFC enabled. If you’ve got a chatty one on top of things, you can do this:
The basic card info might not look like much, as it’s supposed to show only the cards serial number. Some cards – like this one here – however give you the bank account number instead! Nice one. So this is our information leak #1.
As you can see on the other two images, the card also features some flash memory, holding a very interesting transaction log. By sending hexadecimal commands of the form
00 B2 NN 5C 00 to the card, where
NN equals the log entry number, we can get a nice transaction log including amounts paid. So
00 B2 01 5C 00 would get log entry #1,
00 B2 08 5C 00 gets #8,
00 B2 0E 5C 00 gets #14 and so on. After decoding, you get the date and amount of money spent for each transaction, and that includes both NFC transactions and normal full-contact transactions, where you put your card into a real chipreader and enter your pin.
So no matter how you pay, it will be logged on such cards. And that log can be read. Given that NFC is completely pinless, we can just fetch such data without any authentication or encryption holding us back! That’s leak #2. Again, keep in mind that there are those range boosters for active NFC chips! If I put a powered NFC patch kit on my Android phone, in a worst-case scenario I could just walk by you and potentially fetch your transaction logs and bank account number!
Now that did raise a few eyebrows, which is why some banks have reacted to the issue, like my own bank too. But first, to another problem:
4.) The theft issue
Besides leaking information, there is another problem: As said, NFC access is pinless. It’s used for 25€ micropayments mostly, limiting the damage somewhat. Typically, you’ll get 3-5 payments before you have to plug the card back into an ATM or electronic cashier and re-authenticate it using the pin, after which you’ll get another 3-5 contactless payments activated. So with 5 usable payments, you can lose 125€, should your card be stolen. But it doesn’t end there.
In my own country, Austria, we also have an offline cash replacement technology called [Quick]. With that, you can basically charge your banking card and carry the charge around like real cash. It’s being used for machines where online connections are economically unfeasible, like cigarette vending machines or pay and display machines, where you buy tickets for car parking. The maximum charge for Quick amounts to 400€ total.
Thing is, should you ever choose to charge the full amount, this triggers an activation of Quick-over-NFC! This is actually intentional, so that’s what you have to do to get to that feature, contactless offline payments. The real problem is, that with Quick-over-NFC, all limits are gone, which is confirmed [here]. So a thief could just waste the entire charge of the card at his hearts’ content, upping the potential worst case loss to a full 525€! Holy hell, that does actually hurt already! Even if you call your bank and get the card locked due to theft, that money is still gone due to the offline nature of Quick. Just like real cash. So better hold on to your card, if you’ve already got that feature activated and money charged onto it!
But let’s get back to the data leak issue again:
5.) Modern cards may be more close-lipped
Banks aren’t entirely ignorant to the problem and related critizisms received, so some of them actually did try to improve the situation. When trying to read my brand-new card from Bank Austria for instance, what we get is this:
First of all, this newer card doesn’t give away my bank account number, but really just the serial number. That takes care of leak #1 to at least some degree. Secondly, the card doesn’t seem to have a transaction log anymore. At least it doesn’t hand one out using known commands. It can of course still be used for NFC payments using [PayWave] or, as it is in my case, [PayPass] and Quick, if activated. But yes, this is more secure, at least when considering the info leak.
But what if I just want to lock it down for good, once and for all?
We can never be sure that there really is no transaction log after all. Maybe we just don’t know the necessary commands. Plus, there still is the micropayment issue.
Now, some banks give you the option to deactivate the feature at your local branch bank, sometimes for free. Volksbank here does this for instance. Not sure how this works and whether it’s really final though. Others may give you the option to send you a NFC-free card, as my bank does. That is if you do know about it and proactively order one for 14€… By default they’d just send you a fully NFC-capable one before the old one expires.
Some banks do neither of the two. Which is why you may want to handle things yourself.
6. Killing NFC for good
Remember that poor mans’ X-Ray from above? All we need to do is to cut the copper coil to fully disable all NFC functionality. I used a microdrill for this, which may be slightly dangerous for the chip due to fast static charge buildup, but it worked fine in my case. You can also use a manual drill or even melt your way through with a soldering iron. Just make sure to not pick a spot that sits within the cards magnetic strip! In any case, we mark the spot first:
A few seconds later, my cards’ NFC feature has effectively been dealt with. Tests with both Android phones and actual electronic cashiers have shown that yes, it’s truly gone. All the other full-contact functions like cash withdrawal and payments have also been tested and still work absolutely fine!
So that’s it, no more contactless payments, no more reading information out of the card wirelessly, no more Quick-over-NFC (which only concerns Austrian people anyway, but yeah). Just make sure that the edge of the hole is properly deflashed, so your card won’t get stuck in any ATMs or whatever.
So, all of the good things are still there, and all of what I consider to be the bad things are now gone! Finally, I can put my tin foil hat off again.
Ah yes, tin foil! Before I forget it, another colleague of mine also tried to shield his card using tin foil instead. And indeed, that seems to be sufficient too, in case you don’t wanna physically modify your card. You can even buy readily-made shielded card sleeves to protect you from unauthorized NFC accesses, like [this one here].
I do prefer the final solution instead, but it’s up to you, the option to do it temporarily instead is there also.
So, stay safe!
Some days in June you’d find me in the castle of Oberkapfenberg, attending the [Medieval Festival] there. Lots of Jesters, Shows, medieval music, swordplay, archery, old foods & drinks, the castle falconrys’ birds of prey show and craftsmen can be found there, plus a ton of cosplay of course. Now while I can’t provide any good photographs of that, I’d like to show you something I got from [Ulf fum Oachberg], a medieval blacksmith who attends the festivities every year I’ve been there so far. Well actually it’s more like Umlüx talked me into writing this for pressuring him to write something about that [Böker Cronidur-30 knife] he got back then.
I always wanted to buy something from the guy, so we approached the smithy with all its swords and polearms and other metal goods, and talked to him for what was probably half an hour or so. He’ll take the time to explain his forging techniques, his equipment and the steels used in his blades. You can also watch him work right on-site, as he has fire, bellows, hammer & anvil all with him there. One knife however immediately caught my eye, and after a bit of bartering (which I suck at, but yeah), I got it for 164€, all hand-forged and hand-carved (click to enlarge the pictures):
As you may recognize from the blades’ banding, this is not monosteel, but a damascene-style compound steel. It’s made from multiple folded steels which do contain carbon too, making this blade not stainless, but giving it a high resistance to shattering and a supposedly very resilient edge. The handle is made from the horn of a roebuck, featuring a hand-carved lizard ornament. A piece of what I think is some harder leather (I forgot to ask about it) sits on top of the handle, where the blade begins to show itself.
Let’s take a closer look:
Now, Ulf said that while this knife is expensive, I’ll likely never need another one for the rest of my life. I was a bit worried about the carbon in the steel, so I asked him about whether the blade would need any special kind of treatment. I did remember that Japanese carbon steel blades were usually treated with clove oil for some coating on the metal. Ulf however said, that cutting a few slices off a chunk of bacon or some dried sausage every now and then is sufficient. Cleaning the knife with a piece of dry cloth afterwards shall still leave enough fat on the metal to form a nice, protective layer.
And, as grandmother always says (and Ulf too), if you do need to wash it, never let hot water touch a sharp blade, and dry it immediately after washing. Well that much’s common sense I’d say.
Still, can I judge its resilience and longevity? No, of course not. I know not nearly enough about knives and steels to do so. Let’s try to look at what I can; Believing Ulf the blacksmith, I’d like to think that this is some high quality Damascus steel right there. But what about the blades’ angle? Extremely sharp knives tend to have angles as low as 15-10° or even less, some even sharper knives might have hollow ground blades. Those tend to wear out really fast though. Let’s have a look:
The blade can be measured at an almost exact 30° angle (yeah ok, it’s not a perfectly perspectively correct measurement, but well). Now it does get a bit narrower at the edge due to the sharpening, but not by much. My guess would be around 26-28° at the very edge. That should put it into a category of blades with very high durability and slightly less sharpness. Which is not bad in my book. Cutting dried meat almost feels like going through butter still, so there’s nothing to complain here:
What’s left now is a proper sheath. Since I don’t want to just buy some run-of-the mill one for this knife, I’ll likely go to our local shoemaker and tanner (Hell, we actually still got one in my town! Can you believe that?!) and get a custom-made one for it, which should be much more fitting than some mass-produced stuff.
I really wonder how this knife will handle 30 years down the road from now. I guess I just have to find out, it’s just going to take a bit of time.
This is just a minor update after [part 2], but anyway. My old workstation (the one I’m migrating away from) just broke down a few days ago, so I had to do something, and quickly. Since I still don’t have any disks for my new RAID-6, I had to pull the existing RAID array from my old box and attach it to my new workstation in a hurry. It does look quite ugly too, with the RAID lying around on the table beside an open Lian Li PC-A79B. This is not how it was supposed to be, but well… In the meantime I found out that it was my Tagan Piperock 1300W power supply which broke down (Built by Topower by the way). Sad, because I liked it for its sturdy metal screw-on modular plugs, but well. So the machine now sits in its final location, it just doesn’t look too good at the moment:
In any case, I wanted to play around with that new Corsair “Professional Series Platinum AX1200i” of mine, which is a fully digital power supply unit featuring an I²C port. With that, you can hook it up to Corsairs Link [Commander], or you can use the dongle provided with the unit and hook it up to an internal USB header on your mainboard. Here’s a crop of a photo previously shown, this is the dongle:
Now what this actually is, is a [Silicon Labs] – or Silabs in short – I²C to [USBXPress] bridge chip. So it’s not using the same USB HID device class of the Link Commander, but a completely different protocol, which is also why we’re tied to using the Corsair Link software. The free software project [CorsairLinkPlusPlus] won’t work with it at all as it supports only the Link Commander itself.
Having said that, I can’t use the Corsair Link software – which uses .Net 4.5 – on XP x64, it just won’t work on the old OS. The drivers that come with the device though are from Silabs and DO support XP and XP x64. The USB vendor ID was changed from Silabs to Corsair though, so it’s not
1B1C:1C00, making it impossible to install original Silabs drivers. But that’s ok, what Corsair’s shipping with the power supply works just as well.
You may not wish to install the whole Corsair Link software on XP just to get the drivers though. So I have isolated the drivers from the package for you to install them separately. The Hydro water cooler driver is also provided, but you don’t need it if it’s just for a power supply like in my case:
- [Corsair Link / Silabs USBXpress dongle driver] for XP/XP x64.
- [Corsair Link /Silabs hydro series driver] for XP/XP x64.
But, while you can install the dongle, you can’t talk to it, lacking the userland software for that. Now when I said “how to run Corsair Link on XP x64″ in the title, I have to admit I was lying a bit. Because what I did was to virtualize the dongle using Oracle VirtualBox 4.2.26 and then run the Corsair Link software on a Windows 7 x64 virtual machine. Now, before launching that, the XP x64 host systems device manager will show this:
Just so it’s handled automatically for every boot of my Windows 7 VM, I created a USB device filter in the virtual machines’ settings:
Now when you start up the VM, VirtualBox will grab the device and replace it with a device called “VirtualBox USB”, thus making it unavailable on the host machine. Just install Corsair Link in the VM, and everything will work nicely:
Many have described the software as buggy and crappy, but for me it gets the job done. All I wanted was to change the behavior of the unit, disabling its passive mode at low loads. While a nice feature, the internal thermal probe reports temperatures of up to 60°C at 300W load with the fan sitting still, and I don’t quite like that. I don’t see why it is needed to artificially accelerate the aging process of the PSUs electrolytic capacitors like that, so I set the fan speed to 40%, resulting in slightly short of 800rpm. Very silent, and good enough even for high loads. I now get down to 28-35°C depending on ambient temperatures without perceiving any additional noise. It may reach 40°C on really hot days I guess, but that’s a lot better than 60°C.
Just sad that we can’t define a complete custom fan curve for this unit, based on load or temperature readings. It’s possible with system fans when working with the Link Commander, but not for this one.
Naturally, virtualization also works if you’re on Linux or BSD UNIX or Solaris or whatever. It’s cumbersome, yes, but if you need it only to tell the PSUs firmware to keep the fan spinning, it’s ok. You don’t need to keep the software running, that’s the sweet part. The settings will be stored in the power supplys’ firmware directly.
Only downside is: You need a Windows Vista/7 or newer license for that of course. But maybe we’ll see some free software in the future, people are working on it, that much’s for sure!
Now let’s hope part 4 of this log will be my new hard disks, because I’m really starting to run low on disk space already…
So after the release of that crazy crowdfunded (and free of charge) movie [Kung Fury], there is also a game! Now that was fast. Made by the Swedish game developers of [Hello There], the game is basically a clone of [One Finger Death Punch], as many gamers have already pointed out. Not that anybody seems to mind that – me included. It’s a superficially very simple 2-button street fighting game, where one button means “punch/kick/whatever to the left” and the other “punch/kick/whatever to the right”. Don’t let the seeming simplicity fool you though. There is more skill involved than you might think…
So let’s have a look at the intro of the game, which strongly resembles an 80s arcade machine style:
So with the use of some Direct3D 9.0c shaders, the game simulates the look of an old CRT monitor, just like the arcade machines of old had! At the press of a button or after waiting for a bit we’re greeted with this:
Another button press and we can hear our virtual player throwing a coin into the machine, which gives us three lives (after being hit three times, we’d go down for good). And then, whenever any enemy approaches us from either side, you just press left or right to punch, kick, shoot or electrify the guy. It’s ok, they’re all Nazis anyway. We do this with our pals Barbarianna, Triceracop and Hackerman standing around in the background – all three as seen in the movie of course, just like all the enemies we’re beating up:
That screenshot is from the very beginning of the game, where we can only see our lowest-end Nazi foes. There are some Swedish Aryans too, which can take two hits, then that clone chick with Kung Fury essence infused into her, which needs a more advanced left/right combo to put down, and more. Like the kicking machine and the mysterious Ninja, all as seen in the movie. As long as you don’t miss too much (you have limited range) or get hit, you’ll build up a score bonus too. Not sure if there are more enemies than that, I haven’t really gotten that far yet.
Actually, I did reach a new High Score while doing those screenshots accidentally, leaving both chicks behind me, pretty neat:
Now Thor might still be doable, but Hackerman will be one tough nut to crack. I don’t think I’ll ever make it to the top though, the game is pretty damn hard. As it progresses, it starts speeding up more and more, and it’ll also throw more of the harder enemies at you, which will require quick reaction and sharp perception to get the combos right. “Just mash two buttons” may sound easy as said, but don’t underestimate it! Like with “One Finger Death Punch”, only the most skillful players will have a chance to reach the top!
When you’ve got enough, just press <Esc> (on the PC), and you’re asked whether you really want to quit. In an interesting way:
If you confirm to quit the game, you’ll get another shader-based CRT effect thrown in:
I haven’t really managed to play this for more than 5 minutes in a row, which sounds like very little, but this game is extremely fast-paced, so I can’t take much more in one go. It’s quite a lot of fun though, and while not as sophisticated as “One Finger Death Punch”, it’s awesome in its own right, given the Kung Fury cheesiness, the CRT look and the chiptune-like soundtrack of the game.
The game is available in both paid and partially also free editions on several platforms now, and while I’ve read that the free versions do have ads, the paid ones definitely don’t, as I can vouch for on the PC platform at least. So here are the links:
- [PC version] @ Steam for 1.99€ / $2.50. Supports >=Windows XP, >=MacOS X 10.6 and SteamOS plus regular Linux on x86_32/x86_64.
- [PC+ARM version] @ Windows Store for $2.29. Supports Windows >=8.0 on x86_32/x86_64 and ARM architectures.
- [Android version] @ Google Play for free or for 2.46€ with ads removed. Also available as a [separate APK file]. Supports Android >=2.0.1.
- [iOS version] @ iTunes for free or for $1.99 with ads removed. Supports iOS >=6.0 on the iPhone 5/6, iPad and iPod Touch.
Now if you’ll excuse me, I’ll take another shot at number 3!
After [part 1] we now get to the second part of the Taranis RAID-6 array and its host machine. This time we’ll focus on the Areca controller fan modification or as I say “Noctuafication”, the real power supply instead of the dead mockup shown before and a modification of it (not the electronics, I won’t touch PSU electronics!) plus the new CPU cooler, which has been designed by a company which sits in my home country, Austria. It’s Noctuas most massive CPU cooler produced to this date, the NH-D15. Also, we’ll see some new filters applied to the side part of the case, and we’ll take a look at the cable management, which was a job much more nasty than it looks.
Now, let’s get to it and have a look at what was done to the Areca controller:
So as you can see above, the stock heatsink & fan unit was removed. Reason being that it emits a very high-pitched, loud noise, which just doesn’t fit into the new machine which creates more like a low-pitched “wind” sound. In my old box, which features a total of 19 40×40mm fans you wouldn’t hear the card, but now it’s becoming a disturbance.
Note that when doing this, the Arecas fan alarm needs to be disabled. What the controller does due to lack of a rpm signal cable is to measure the fan’s “speed” by measuring its power consumption. Now the original fan is a 12V DC 0.09A unit, whereas the Noctua only needs 0.06A, thus triggering the controllers audible alarm. In my case not so troublesome. Even if it would fail – which is highly unlikely for a Noctua in its first 10 years or so – there are still the two 120mm side fans.
Cooling efficiency is slightly lower now, with the temperature of the dual-core 1.2GHz PowerPC 476FP CPU going from ~60°C to ~65°C, but that’s still very much ok. The noise? Pretty much gone!
Now, to the continued build:
So there it is, although not yet with final hardware all around. In any case, even with all that storage goodness sitting in there, the massive Noctua NH-D15 simply steals the show here. That Xeon X5690 will most definitely never encounter any thermal issues! And while the NH-D15 doesn’t come with any S1366 mounting kit, Noctua will send you one NM-I3 for free, if you email them your mainboard or CPU receipt as well as the NH-D15 receipt to prove you own the hardware. Pretty nice!
In total we can see that cooler, the ASUS P6T Deluxe mainboard, the 6GB RAM that are just there for testing, the Areca ARC-1883ix-12, a Creative Soundblaster X-Fi XtremeMusic, and one of my old EVGA GTX580 3GB Classified cards. On the top right of the first shot you can also spot the slightly misaligned Areca flash backup module again.
While all my previous machines were in absolute chaos, I wanted to have this ONE clean build in my life, so there it is. For what’s inside in terms of cables, very little can be seen really. Considering 12 SAS lanes, 4 SATA cables, tons of power cables and extensions, USB+FW cables, fan cables, an FDD cable, 12 LED cathode traces bundled into 4 cables for the RAID status/error LEDs and I don’t know what else. Also, all the internal headers are used up. 4 × USB for the front panel, one for the combo drives’ card reader and one for the Corsair Link USB dongle of the power supply, plus an additional mini-Firewire connector at the rear.
Talking about the cabling, I found it nearly impossible to even close the rear lid of the tower, because the Great Cthulhu was literally sitting back there. It may not look like it, but it took me many hours to get it under some control:
Now it can be closed without much force at least! Lots of self-adhesive cable clips and some pads were used here, but that’s just necessary to tie everything down, otherwise it just won’t work at all. Two fan cables and resistors are sitting there unused, as the fans were re-routed to the mainboard headers instead, but everything else you can see here is actually necessary and in use.
Now, let’s talk about the power supply. You may have noticed it already in the pictures above, but this Corsair AX1200i doesn’t look like it should. Indeed, as said, I modified it with an unneeded fan grill I took out of the top of the Lian Li case. Reason is, that this way you can’t accidentally drop any screws into the PSU when working on the machine, and that can happen very quickly. If you miss just one, you’re in for one nasty surprise when turning the machine on! Thanks fly out to [CryptonNite], who gave me that idea. Of course you could just turn the PSU around and let it suck in air from the floor (The Lian Li PC-A79B supports this), but I don’t want to have to tend to the bottom dust filter all the time. So here’s what it looks like:
With 150W of power at +5V, this unit should also be good enough for driving all that HDD drive electronics. Many powerful PSUs ignore that part largely and only deliver a lot at +12V for CPUs, graphics cards etc. Fact is, for hard drives you still need a considerable amount of 5V power! Looking at Seagates detailed specifications for some of the newer enterprise drives, you can see a peak current of 1.45A @ 5V in a random write scenario, which means 1.45A × 5V = 7.25W per disk, or 12 × 7.25W = 87W total for 12 drives. That, plus USB requiring +5V and some other stuff. So with 150W I should be good. Exactly the power that my beloved old Tagan PipeRock 1300W PSU also provided on that rail.
Now, as for the side panels:
And one more, an idea I got from an old friend of mine, [Umlüx]. Since I might end up with a low pressure case with more air being blown out rather than sucked in, dust may also enter through every other unobstructed hole, and I can’t have that! So we shut everything out using duct tape and paper inlets (a part of which you have maybe seen on the power supply closeup already):
That’s it for now, and probably for a longer time. The next thing is really going to be the disks, and since I’m going for 6TB 4Kn enterprise disks, it’s going to be terribly expensive. And that alone is not the only problem.
First we got the weak Euro now, which seems to be starting to hurt disk drive imports, and then there is this crazy storage “tax” (A literal translation would be “blank media compensation”) that we’re getting in October after years of debate about it in my country. The tax is basically supposed to cover the monetary loss of artists due to legal private recordings from radio or TV stations to storage media. The tax will affect every device that features any kind of storage device, whether mechanical/magnetic, optical or flash. That means individual disks, SSDs, blank DVDs/BDs, USB pendrives, laptops, desktop machines, cellphones and tablets, pretty much everything. Including enterprise class SAS drives.
Yeah, talk about some crazy and stupid “punish everybody across the board for what only a few do”! Thanks fly out to the Austro Mechana (“AUME”, something like “GEMA” in Germany) and their fat-ass friends for that. Collecting societies… legal, institutionalized, large-scale crime if you ask me.
But that means that I’m in between a rock and a hard place. What I need to do is to find the sweet spot between the idiot tax and the Euros currency rate, taking natural price decline into account as well. So it’s going to be very hard to pick the right time to buy those drives. And as I am unwilling to step down to much cheaper 512e consumer – or god forbid shingled magnetic recording – drives with read error rates as high as <1 in 1014 bits, we’re talking ~6000€ here at current prices. Since it’s 12 drives, even a small price drop will already have great effect.
We’ll see whether I’ll manage to make a good decision on that front. Also, space on my current array is getting less and less by the week, which is yet another thing I need to keep my eyes on.
Edit: [Part 3 is now ready]!
Well, I’m not gonna say too much, and yeah, I’m a day late, but I’m just gonna say: Kung Fury is out! If you’re reading this, you’ll probably already know it and you’ve probably already seen this over the top crazy 80s trash anyway, but just in case you haven’t, you can [watch it directly on YouTube], where the guys have released the 30 minute official version for free, or right here, embedded for your viewing pleasure!
I do recommend having at least one beer before starting with this though! Here you go:
The official 30min Kung Fury release!
Todays post shall be about storage. My new storage array actually. I wanted to make this post episodic, with multiple small posts that make sort of a build log, but since I’m so damn lazy, I never did that. So by now, I have quite some material piled up, which you’re all getting in one shot here. This is still not finished however, so don’t expect any benchmarks or even disks – yet! Some parts will be published in the near future, in the episodic manner I had actually intended to go for. So…
I’ve been into parity RAID (redundant array of independent/inexpensive disks) since the days of PATA/IDE with the Promise Supertrak SX6000, which I got in the beginning of 2003. At first with six 120GB Western Digital disks in RAID-5 (~558GiB of usable capacity), then upgraded to six 300GB Maxtor MaxLine II disks (~1.4TiB, the first to break the TiB barrier for me). It was very stable, but so horribly slow and fragmented at the end, that playback of larger video files – think HDTV, Blu-Rays were hitting the market around that time – became impossible, and the space was once again filled up at the end of 2005 anyway.
2006, that was when I got the controller I’m still using today, the 3ware 9650SE-8LPML. Typically, I’d say that each upgrade has to give me double capacity at the very least. Below that I wouldn’t even bother with replacing either disks or a whole subsystem, given the significant costs. The gain has to be large enough to make it worthwhile.
The 3ware had its disks upgraded once too, going from a RAID-6 array consisting of 8×1TB Hitachi Deskstars (~5.45TiB usable) to 8×2TB Hitachi Ultrastars (~10.91TiB usable), which is where I’m sitting at right now. All of this – my whole workstation – is installed in an ancient EYE-2020 server tower from the 90s, which so far has housed everything starting from my old Pentium II 300MHz with a Voodoo² SLI setup all the way up to my current Core i7 980X hexcore with a nVidia SLI subsystem. Talk about some long-lasting hardware right there. So here’s what the “Helios” RAID-6 array and that ugly piece of steel look like today, and please forgive me for not providing any pictures of the actual RAID controller or its battery backup unit, I don’t have any nice photos of them, so I have to point you to some web search regarding the 3ware 9650SE-8LPML, as always, please CTRL+click to enlarge:
As you can see, that makes 16 × 40mm fans. It’s not like server-class super noisy, but it for sure ain’t silent either. It’s quite amazing that the Y.S. Tech fans in there have survived running 24/7 from 2003 to 2015, that’s a whopping 12 years! They are noisier now, and every few weeks one of the bearings would go to saw-blade mode for a brief moment, but what can you expect. None have died so far, so that’s a win in my book for any consumer hardware (which the HDCS was).
Thing is, I have two of those 3ware RAID controllers now, but each one has issues. One wouldn’t properly synchronize on the PCIe bus, negotiating only a single PCIe lane, and that thing is PCIe v1.1 even, which means a 250MiB/s limit in that crippled mode. The second one syncs properly, but has a more pressing issue; Whenever there are sharp environmental temperature changes (opening the window for 5 minutes when it’s cool outside is enough), the controller randomly starts dropping drives from the array. It took me a LONG time to figure that out, as you probably can imagine. Must be some bad soldering spots on the board or something, but I couldn’t really identify any.
Plus, capacity is running out again. Now, the latest 3ware firmware would enable me to upgrade this to at least 8 × 6TB, but with 4K video coming up and with my desire to build something very long-lasting, I decided to retire “Helios”. Ah, yes. The name…
Consider me as being childish here, but naming is something very important for me, when it comes to machines and disks or arrays. I had decided to name each array once per controller. For disk upgrades, it simply gets a new number. So there was the IDE one, “Polaris”. Then “Polaris 2″, then “Helios” and “Helios 2″.
Supposedly, a famous Taranis pilot once said this:
“The taranis is a ship for angry men or people who prefer to deal in absolutes. None of that cissy boy, ‘we danced around a bit, shot some ammo then ran away LOL’, or, ‘I couldn’t break his tank so I left’, crap. It goes like this:
You fly Taranis. A fight starts. Someone dies.”
I flew on the wing of a Taranis pilot for only one single time. A lot of people died that night, including our entire wing!
In any case, I wanted to 1up this a bit. From certain enterprise storage solutions I of course knew the concept of hot-swapping and more importantly error reporting LEDs on the front of a storage enclosure. Since that’s extremely useful, I wanted both for my new array in a DIY way. I also wanted to get rid of the Antec HDCS, which had served me for 12 years now, and ultimately also semi-retire my case, after understanding that it was just too cramped for this. A case that had served me for 17 years, 24/7.
Holy shit. That’s a long time!
So I had to come up with a good solution. The first part was: I needed hot-swap bays that could do error reporting in a way supported by at least some RAID controllers. I found only ONE aftermarket bay that would fully satisfy my requirements. The controller could come later, I would just pick it from a pool of controllers supporting the error LEDs of the cages.
It was the Chieftec SST-2131SAS ([link 1], [link 2]), the oldest of Chieftecs SAS/SATA bays. It had to be the old one, because the newer TLB and CBP series no longer have any hard disk error reporting capability built in for whatever reason, and on top of that, the older SST series shows much less plastic and just steel and what I think is magnesium alloy, feels awesome:
So there is no fancy digital I²C bus for error reporting on the bays, just some plain LED connectors that do require the whole system to have a common electrical ground to work for closing the circuit, as we only got cathode pins. I got myself four such bays, which makes for a total of 12 possible drives. As you may already be guessing, I’m going for more than just twice the capacity on this one.
For a fast, well-maintainable controller, I went for the Areca [ARC-1883ix-12], which was released just at the end of 2014. It supports both I²C as well as the old “just an error LED” solution my bays have, pretty nice!
Areca (and I can confirm this first-hand) is very well known for their excellent support, which means a lot of points have to go to them for that. Sure the Taiwanese Areca guys don’t speak perfect English, but given their technical competence, I can easily overlook that. And then they support a ton of operating systems, including XP x64, even after it’s [supposed] demise (The system shall run with a mirror of my current XP x64 setup at first, and either some Linux or FreeBSD UNIX later). This thing comes with a dual-core ROC (RAID-on-Chip) running at 1.2GHz, +20% when compared to its predecessor. Plus, you get 2GiB of cache, which is Reg. ECC DDR-III/1866. Let’s just show you a few pictures before going into detail:
So there are several things to notice here:
- It’s got an always-full-power fan and a big cooler, so it’s not going to run cool. Like, ever.
- It requires PCIe power! Why? Because all non-PEG devices sucking more than 35W have to, by PCIe specification. This one eats up to 37.2W (PEG meaning the “PCI Express Graphics” device class, graphics cards get 75W from the slot itself).
- It has Ethernet. Why? Because you need no management software. The management software runs completely *ON* the card itself!
The really interesting part of course is the Ethernet plug. In essence, the card runs a complete embedded operating system, including a web server to enable the administrator to manage it in an out-of-band way.
That means that a.) it can be managed on all operating systems even without a driver and b.) it can even be managed, when the host operating system has crashed fatally, or when the machine sits in the system BIOS or in DOS. Awesome!
Ok, but then, there is heat. The system mockup build I’m going to show you farther below was still built with the “lets plug it in the top PCIe x4 slot” idea in mind. That would include my EVGA GeForce GTX580 3GB Classified Ultra SLI system still being there, meaning that the controller would have to sit right above an extremely hot GPU.
By now, I’ve abandoned this idea for a thermally more viable solution, replacing the SLI with a GeForce GTX Titan Black I got for an acceptable price. In the former setup, the controllers many thermal probes have reported temperatures reaching 90°C during testing, and that’s without the GPUs even doing much, so yeah.
But before we get to the mockup system build, there is one more thing, and that’s the write cache backup for the RAID controller for cases of power failures. Typically, Lithium-Ion batteries are used for that, but I’m already a bit fed up with my 3ware batteries having gone belly-up every 2 years. So I wanted to ditch that. There are such battery backup units (“BBUs”) for the Areca, but it may also be combined with a so-called flash backup module (“FBM”). Typically, a BBU would keep the DRAM and its write cache alive on the controller during power outages for like maybe 24-48 hours, waiting for the main AC power to return. Then, the controller would flush the cached data to the disks to retain a consistent state.
An FBM does it differently: It uses capacitors instead, plus a small on-board SSD. It would keep the memory alive for just seconds, just enough to copy the data off the DRAM and onto its local SSD. Then it would power off entirely. The data gets fetched back after any arbitrary amount of downtime upon power-up of the system, and flushed out to the RAID disks. The hope here is, that the supercapacitors being used by such modules can survive for much longer than the LiOn batteries.
There is one additional issue though: Capacity (both in terms of electrical capacitance and SSD capacity) is limited by price and physical dimensions. So the FBM can only cover 2GiB of cache, but not the larger sizes of 4GiB or 8GiB.
That’s where Areca support came into play, readily helping you with any pre-purchase question. I talked to a guy there, and described my workload profile to him, which boils down to highly sequential I/O with relatively few parallel streams (~40% read + ~60% write), and very little random R/W. He told me that based on that use case, more cache doesn’t make sense, as that’d be useful only for highly random I/O profiles with a very high workload and high parallelism. Think busy web servers or mail servers. But for me, 4GiB or the maximum of 8GiB of cache wouldn’t do more than what the stock 2GiB does.
As such, I forgot about the cache upgrade idea and went with the flash backup module instead of a conventional BBU. That FBM is called the ARC-1883-CAP:
So, let’s put all we have for now together, and look at some build pictures:
Let me tell you one thing; Yes, the Lian Li PC-A79B is nice, because it’s so manageable. The floors in the HDD cages can be removed even, so that any HDD bay can fit, with no metal noses in the way in the wrong places. Its deep, long and generally reasonably spacious.
But – there is always a but – when you’re coming from an ancient steel monster like I did, the aluminium just feels like thin paper or maybe tin foil. The EYE-2020 can could the weight of a whole man standing on top of it. But with an aluminium tower you’ll have to be careful not to bend anything when just pulling out the mainboard tray. The HDD cage feels as if you could very easily rip it out entirely with just one hand.
Aluminium is really soft and weak for a case material, so that’s a big minus. But I can have a ton of drives, a much better cooling concept and a much, much, MUCH cleaner setup, hiding a lot of cables from the viewer and leaving room for air to move around. Because that part was already quite terrible in my old EYE.
Please note that the above pictures do not show the actual system as it’s supposed to look like in the end though. The RAID controller already moved one slot downwards, away from the 4 PCIe lanes coming from the ICH10R (“southbridge”), which in turn is connected to the IOH (“northbridge”) only via a 2GiB/s DMI v1 bus. So it went down one slot, onto the PCIe/PEG x16 slot which is connected to the X58 chipsets IOH directly. This should take care of any potential bandwidth problems, given that the ICH10R also has to route all my USB 2.0 ports, the LAN ports, all Intel SATA ports including my system SSD and the BD drives, one Marvell eSATA controller and one Marvell SAS Controller to the IOH and with it ultimately to the CPU & RAM, all via a bus that might’ve gotten a bit overcrowded when using a lot of those subsystems at once.
Also, this tiny Intel cooler isn’t gonna stay there, it just came “for free” with the second ASUS P6T Deluxe I bought, together with a Core i7 930. Well, as a matter of fact, that board… umm… let’s just say it had a little accident and had to be replaced *again*, but that’s a story for the next episode. A Noctua NH-D15 monster and the free S1366 mounting kit that Noctua sends you if you need one, plus a proper power supply all have already arrived, so there might be a new post soon enough, with even more Noctuafication also being on the way! Well, as soon as I get out of my chair to actually get something done at least.
And for those asking the obvious question “what drives are you gonna buy for this?”, the answer to that (or at least the current plan) is either the 6TB Seagate Enterprise Capacity 3.5 in their 4Kn version, the [ST6000NM0014], or the 6TB Hitachi Ultrastar 7K6000, also in their 4Kn version, that’d be the [HUS726060AL4210]. Given that I want drives with a read error rate of <1 error in 1015 bits read instead of <1 in 1014, like it is for consumer drives, those would be my primary drives of choice. Seagates cheap [SMR] (shingled magnetic recording) disks are completely unacceptable for me anyway, and from what I’ve heard so far, I can’t really trust Hitachis helium technology with being reliable either, so it all boils down to 6TB enterprise class drives with conventional air filling for now. That’s if there aren’t any dramatic changes in the next few months of course.
Those disks are all non-encrypting drives by the way, as encryption will likely be handled by the Areca controllers own AES256 ASIC and/or Truecrypt or Veracrypt.
Ah, I almost forgot, I’m not even done here yet. As I may get a low-air-pressure system in the end, with less air intake than exhaust, potentially sucking dust in everywhere, I’m going to filter or block dust wherever I possibly can. And the one big minus for the Chieftec bays is that they have no dust filters. And the machine sits in an environment with quite a lot of dust, so every hole has to be filtered or blocked, especially those that air gets sucked through directly, like the HDD bays.
For that I got myself some large 1×1 meter stainless steel filter roll off eBay. This filter has a tiny 0.2mm mesh aperture size and 0.12mm wire diameter, so it’s very, very fine. I think it was originally meant to filter water rather than air, but that doesn’t mean it can’t do the job. With that, I could get those bays properly modified. I don’t want them to become dust containers eventually after all.
I went for steel to have something easy enough to work with, yet still stable. Now, it took me an entire week to get this done properly, and that’s because it’s some really nasty work. First, let’s look at one of the trays that need filtering, so you can see why it’s troublesome:
So as you can see, I had to cut out many tiny pieces, that would then be glued into the tray front from the inside, for function as well as neat looks. This took more than ten man-hours for all 4 bays (12 trays), believe it or not. This is what it looks like:
Now that still leaves the other hexagonal holes in the bay frame, that air may get sucked through and into the bays inside. Naturally, we’ll have to handle them as well:
And here is our final product, I gotta say, it looks reaaal nice! And all you’d have to do every now and then is to go over the front with your vacuum cleaner, and you’re done:
So yeah, that’s it for now, more to follow, including the new power supply, more dust filtering and blocking measures, all bays installed in the tower and so on and so forth…
Edit: [Part 2 is now ready]!
Just so everybody knows, UPC – my Internet provider – will be doing routine maintenance work on Wednesday, the 13th of May in 2015, because they have to do… yeah, “stuff”. That’s how specific they are, technically. In any case, the time window for this maintenance will be 01:00am – 06:00am UTC+1 DST, so if you cannot reach any XIN.at service during that time frame, you’ll know why. Naturally, that’ll affect this weblog just as well as all other services I’m offering, like eMail, FTP+ES, IRC, etc. Please note that given my past experience it may very well happen that this maintenance window will be… uhm… let’s say “accidentally extended” by UPC due to reasons nobody will talk about. So if you still can’t reach any of my services at 07:00am UTC+1 DST, then yeah… chances are it’s not me to blame, at least not this time around.
So let’s just hope it’ll just be only a few minutes of blackness this time, eh?
Think about David Hasselhoff what you will, but one thing’s fore sure: He’s definitely an icon of the 80s of the past millennium, there’s no denying that. And now he’s really pulled one off, as he did the official intro song for the crowd funded movie “Kung Fury“, that set sail to collect all the 80s charm and weave it into a crazy action comedy of yet unseen proportions. Let’s give you a few buzz words right there: Cops, Kung Fu, hacking time itself, Kung Führer (yes, Hitler), ancient female viking warriors with guns, dinosaurs, evil robots. That’s enough for now I’m guessing.
The thing here is, that it’s not just the Hoff making some PR for Kung Fury – though it’ll most definitely work quite well in that regard – the song is actually catchy and surprisingly good, capturing the very soul of the 80s perfectly!
Now, before I show you the music video, you should probably take a look at the official Kung Fury web site or maybe [here] first, to read about what Kung Fury is supposed to be, and to watch the official trailer. Whet that’s done, take a look at the madness The Hoff has created here, together with a lot of scenes from the actual movie:
As of 2015-04-18, 11:00 UTC+1 DST, this video has had about 2.1 million views on YouTube, having been online for roughly 2 days. Never doubt The Hoff!
Now if you want fun, and over-the-top 80s cheesiness, you’ll love this piece of trash (I just went ahead and bought the song, and I rarely do that), and you should probably go watch Kung Fury in any way you can when it’s being released next month! It will even come as a free download, heh, talk about awesome!