Nov 242016
 

Broken Windows logo[1] I know what I should do if a system service on Microsoft Windows starts crashing of course; Fixing it is the way to go! But sometimes you simply can’t, because the component causing a certain instability can’t be swapped out or updated. Now Windows services do have a mechanism for monitoring and restarting a service upon failure, but it seems that only works if the system gets an actual error code back from the service upon termination. But it doesn’t seem to work (at least for me) if the service just dies abnormally. Windows recognizes the service has stopped somehow of course, but the restart procedure just doesn’t kick in.

So I thought I’d do it myself, programmatically. And it’s actually pretty easy. I solved this with VBScript, Windows Batch and Mark Russinovichs’ pslist plus grep. So the prerequisites are:

  • Microsoft Windows (well, huh..)
  • MS Windows Script(ing) Host / VBScript, Windows should come with this preinstalled since Windows 2000.
  • [pslist]
  • [grep][src] (grep is optional, I used GNU grep 2.5.4 in this case, licensed under the [GPLv3+])

Make sure the pstools and grep are within your %PATH%, so Windows can find those .exe files. If you don’t want to use grep, you can also use Microsofts’ own find command, if your version of Windows has it.

I divided this into two small scripts. Since the main part is Batch, it might be problematic if you run it at very short intervals, checking for the services’ status, because you get a command window popping up on the desktop. Since most users wouldn’t want that, another script acts as a launcher, hiding the cmd.exe window so it’s run fully in the background without disturbing any potential users or administrators. The launcher looks like this, in my case it’s meant to watch over an Apache web server:

  1. Set WshShell = CreateObject("WScript.Shell")
  2. WshShell.Run chr(34) & "C:\Server\Scripts\monitor-httpd.bat" & Chr(34), 0
  3. Set WshShell = Nothing

And that script C:\Server\Scripts\monitor-httpd.bat we’re launching looks like this:

  1. @ECHO OFF
  2. FOR /F "tokens=* delims= usebackq" %%I IN (`pslist ^| grep httpd`) DO SET HTTPDSTATUS=%%I
  3. IF NOT DEFINED HTTPDSTATUS (net start "Apache2.2") ELSE (SET HTTPDSTATUS=)

A version relying on Microsoft find instead of GNU grep could look like this:

  1. @ECHO OFF
  2. FOR /F "tokens=* delims= usebackq" %%I IN (`pslist ^| find /I "httpd"`) DO SET HTTPDSTATUS=%%I
  3. IF NOT DEFINED HTTPDSTATUS (net start "Apache2.2") ELSE (SET HTTPDSTATUS=)

To get a services’ exact name, just launch services.msc from Start \ Run or run the command net start on a cmd terminal.

As you can see, this greps “httpd” from the process list and pushes its output into %%I and finally into %HTTPDSTATUS%. We have to use a FOR /F for that, as Windows has no way of pushing command outputs from subshells into shell variables like UNIX has (like e.g. var=`command` or var=$(command)). Then we check for the status of that variable. If it’s not defined, then the process http.exe was nowhere to be found! In that case we restart the associated system service (needs proper permissions!). If the variable is defined, we do nothing but unsetting it, since we can assume the service is operating normally. Or at the very least it’s running. ;)

You can automate that by using the Windows task scheduler:

Scheduling an Apache web server "watchdog"

Scheduling an Apache web server “watchdog” (German Windows)

Create a Schedule to your liking and you’re done! If you can afford the affected service to be down for 5 minutes and no longer, just run it every 4 minutes or so.

The solution shown above can easily be adapted to monitor and restart any Windows service you have, as long as the service isn’t fundamentally broken so that it wouldn’t even start up anymore. Also, you can do a lot more, like sending notification eMails with a command line mailer like [blat] when crashes do occur. Of course, this is only useful for services that crash rarely. If it dies every few minutes, you should reaaally fix it instead of just pushing the restart button all the time… ;)

And that’s that!

[1] © Mar.0007. Original Version for desktopwallpapers4.me.

May 232013
 

MemoryTen logoThere is this one story that I wanted to share for quite some time now, but for some reason I always forgot about it. 2 years back I was playing around with this chinese-developed RISC processor, a Loongson 2F (“龙芯”), which was built into a Lemote Yeeloong 8089B netbook. So I was [setting up and patching Debian Linux Wheezy/sid for this very weird machine] back then, and at some point I got the idea of upgrading the 1GB of memory to 2GB, which – according to the memory controller specifications – was the absolute maximum. The RAM required was the tricky part though, the memory controller required DDR-II/667, but in a dense single-rank configuration, which is extremely rare.

On my quest for a 2GB of single-rank DDR-II/667 SODIMM I came across a german company that would do a build-to-order, but they required me to order several thousand modules, which was unacceptable, me only needing a single one. Later I found that huge US memory store called [MemoryTen], and they had (and still have!) [one fitting module] on their website, but it was completely out of stock back then. So I sent them an eMail, asking whether they would get more in the future.

A day later a man called Sal Scuderi replied and asked if I wanted Kingston instead, because it was cheaper etc. But I insisted on single-rank RAM, after which the man asked me to wait a bit as he would see what he could do.

Actually, a few days later he said, that his boss (actually the goddamn company president, how’s that even possible?!) would inspect their factory in Irvine, California a few days later, and that if I really wanted that module, they could order a single one built just in time, which the company president could then pick up from the manufacturing line after inspection and take back to their HQ on his flight back! I was stunned by this. Was this guy serious? I mean, this is one damn huge company, would they still have the sort of flexibility to do something like that?

So, overjoyed by this, I confirmed the order. It didn’t even cost extra! How could that be? Mister Scuderi wrote me again, as his big boss had returned, the requested module in his carry-on luggage. They sent it to me, and 5 days later I had that package from the US on my doorstep:

I checked that sticker that showed the manufacturing date, and indeed back in 2011, the module was absolutely brand new, just a few days old. So this was truly built-to-order? Hard to believe, but obviously true. Now, to my disappointment the module worked, but not in the actual target machine. While the memory controller supported the module, I didn’t take the PMON2000 firmware into account. And as some contacts in China confirmed after I did some research, the 1GB memory limit was hardcoded into the firmware. Bad luck, heh. In the end, I did not dare to try and build my own modified PMON2000, that was just far too hard and risky. So I left it as it was.

Still, that doesn’t belittle what Sal Scuderi and his boss from MemoryTen did to get me that RAM built! Amazing indeed. And I thought service and support like that (for just $50 total!) didn’t exist anymore. But it seems there are some good people in some good companies left. And I’d say, Sal and MemoryTen are some of them! See my final correspondence with Sal, you’ll see how taken aback I actually was: ;)

> Hello, Sal!

> Uhm. Are you crazy? My apologies for saying that, but you have to be, if you
> just let your factory build me a single module on my request? I know no
> company in the world (as of yet) which would do a build-to-order without
> the customer buying at least a few thousand DIMMs?!?

> But if you really just did that for a single customer buying a single
> specific module, you’ve earned my respect..

> Regards,
> Michael Lackner 

Hi Michael,

It’s a done deal, and yes we are crazy, but it’s a good crazy, we aim
to please, and I’m sure you will be a good sales person besides being
a good customer,

Sal

Not sure what he meant by “sales person” anymore, maybe I should reread the whole conversation, but yeah, you get the idea. :) Awesome stuff!