Oct 222014

Webchat logoXIN.at has been running an IRC chat server for some time now, but the problem always lies with people needing some client software to use it, like X-Chat or Nettalk or whatever.

People usually just don’t want to install yet another chat client software, no matter how old and well-established IRC itself may be. Alternatively, they can use some other untrusted web interface to connect to either the plain text [irc://www.xin.at:6666] or the encrypted [irc+ssl://www.xin.at:6697] server via a browser, but this isn’t optimal either. Since JavaScript cannot open TCP sockets on its own, and hence cannot connect to an IRC server directly, there are only two kinds of solutions:

  • Purely client-based as a Java Applet or Adobe Flash Applet, neither of wich are very good options.
  • JavaScript client + server backend for handling the actual communication with the IRC server.
    • Server backends exist in JavaScript/Node.js, Perl, Python, PHP etc.

Since I cannot run [Node.js] and [cgi:irc] is unportable due to its reliance on UNIX sockets, only Python and PHP remained. Since PHP was easier for me, I tried the old [WebChat2] software developed by Chris Chabot for this. To achieve connection-oriented encryption security, I wrapped SSL/TLS around the otherwise unencrypting PHP socket server of WebChat2. You can achieve this with cross-platform software like [stunnel], which can essentially wrap SSL around almost every servers connection (minus the complex FTP protocol maybe). While WebChat2’s back end is based on PHP, the front end uses JavaScript/Comet. This is what it looks like:

So that should do away with the “I don’t wanna install some chat client software” problem, especially when considering that most people these days don’t even know what Internet Relay Chat is anymore. ;) It also allows anonymous visitors on this web log to contact me directly, while allowing for a more tap-proof conversation when compared with what typical commercial solutions would give you (think WhatsApp, Skype and the likes). Well, it’s actually not more tap-proof considering the server operator can still read all communication at will, but I would like to believe that I am a more trustworthy server operator than certain big corporations. ;)

Oh, and if you finally do find it in yourself to use some good client software, check out [XChat] on Linux/UNIX and its fork [HexChat] on Windows, or [LimeChat] on MacOS X. There are mobile clients too, like for Android ([AndroIRC], [AndChat]), iOS ([SIRCL], [TurboIRC]), Windows Phone 8 ([IRC Free], [IRC Chat]), Symbian 9.x S60 ([mIRGGI]) and others.

So, all made easy now, whether client software or just web browser! Ah and before I forget it, here’s the link of course:

Edit: Currently, only the following browsers are known to work with the chat (older version may sometimes work, but are untested):

  • Mozilla FireFox 31+
  • Chromium (incl. Chrome/SRWare Iron) 30+
  • Opera 25+
  • Apple Safari 5.1.7+
  • KDE Konqueror 4.3.4+

The following browsers are known to either completely break or to make the interface practically unusable:

  • Internet Explorer <=11
  • Opera <=12.17
Jul 112014

TK-IP101 logoAttention please: This article contains some pretty negative connotations about the software shipped with the TRENDnet TK-IP101 KVM-over-IP product. While I will not remove what I have written, I have to say that TRENDnet went lengths to support me in getting things done, including allowing me to decompile and re-release their Java software in a fixed form under the free GNU General Public license. Please [read this] to learn more. This is extremely nice, and so it shall be stated before you read anything bad about this product, so you can see things in perspective! And no, TRENDnet has not asked me to post this paragraph, those are my own words entirely.

I thought that being able to manage my server out-of-band would be a good idea. It does sound good, right? Being able to remotely control it even if the kernel has crashed and being able to remotely access everything down to the BIOS level. A job for a KVM-over-IP switch. So I got this slightly old [TK-IP101] from TRENDnet. Turns out that wasn’t the smartest move, and it’s actually a 400€ piece of hardware. The box itself seems pretty ok at first, connecting to your KVM switch fabric or a single server via PS/2, USB and VGA. Plus, you can hook up a local PS/2 keyboard and mouse too. Offering what was supposed to be highly secure SSL PKI autentication via server+client certificates, so that only clients with the proper certificate may connect plus a web interface, this sounded really good!



It all breaks down when it comes to the software though. First of all, the guide for certificate creation that is supposed to be found on the CD that comes with the box is just not there. Also, the XCA software TRENDnet suggests one should use was also missing. Not good. Luckily, the software is open source and can be downloaded from the [XCA SourceForge project]. It’s basically a graphical OpenSSL front end. Create a PEM-encoded root certificate, PEM-encoded server certificate and a PKCS#12 client certificate, the latter signed by the root cert. So much for that. Oh, and I uploaded that TRENDnet XCA guide for you in case it’s missing on your CD too, its a bit different for the newer version of XCA, but just keep in mind to create keys beforehand and to use certificate requests instead of certificates. You then need to sign the requests with the root certificate. With that information plus the guide you should be able to manage certificate creation:

But it doesn’t end there. First I tried the Windows based viewer utility that comes with its own certificate import tool. Import works, but the tool will not do client+server authentication. What it WILL do before terminating itself is this:

TK-IP101 IPViewer.exe bug

TK-IP101 IPViewer.exe bug

I really tried to fix this. I even ran it on Linux with Wine just to do an strace on it, looking for failing open() calls. Nothing. So I thought… Why not try the second option, the Java Viewer that goes by the name of KViewer.jar? Usually I don’t install Java, but why not try it out with Oracles Java 1.7u60, eh? Well:

So yeah. What the hell happened there? It took me days to determine the exact cause, but I’ll cut to the chase: With Java 1.6u29, Oracle introduced multiple changes in the way SSL/TLS worked, also due to the disclosure of the BEAST vulnerability. When testing, I found that the software would work fine when run with JRE 1.6u27, but not with later versions. Since Java code is pretty easily decompiled (thanks fly out to Martin A. for pointing that out) and the Viewer just came as a JAR file, I thought I’d embark on the adventure of decompiling Java code using the [Java Decompiler]:

Java Decompiler decompiling KViewer.jar's Classes

Java Decompiler decompiling KViewer.jar’s Classes

This results in surprisingly readable code. That is, if you’re into Java. Which I am not. But yeah. The Java Decompiler is pretty convenient as it allows you to decompile all classes within a JAR and to extract all other resources along with the generated *.java files. And those I imported into a Java development environment I knew, Eclipse Luna.

Eclipse Luna

Eclipse Luna

Eclipse Luna (using a JDK 7u60) immediately complained about 15 or 16 errors and about 60 warnings. Mostly that was missing primitive declarations and other smaller things that even I managed to fix, got rid even of the warnings. But the SSL bug persisted in my Java 7 build just as it did before. See the following two traces, tracing SSL and handshaking errors, one working ok on JRE 1.6u27, and one broken on JRE 1.7u60:

So first I got some ideas from stuff [posted here at Oracle], and added the following two system properties in varying combinations directly in the Main class of KViewer.java:

public static void main(String[] paramArrayOfString)
  /* Added by the GAT from http://wp.xin.at                        */
  /* This enables insecure TLS renegotiation as per CVE-2009-3555  */
  /* in interoperable mode.                                        */
  java.lang.System.setProperty("sun.security.ssl.allowUnsafeRenegotiation", "false");
  java.lang.System.setProperty("sun.security.ssl.allowLegacyHelloMessages", "true");
  /* ------------------------------------------------------------- */
  KViewer localKViewer = new KViewer();
  localKViewer.mainArgs = paramArrayOfString;

This didn’t really do any good though, especially since “interoperable” mode should work anyway and is being set as the default. But today I found [this information on an IBM site]!

It seems that Oracle fixed the BEAST vulnerability in Java 1.6u29 amongst other things. They seem to have done this by disallowing renegotiations for affected implementations of CBCs (Cipher-Block Chaining). Now, this KVM switch can negotiate only a single cipher: SSL_RSA_WITH_3DES_EDE_CBC_SHA. See that “CBC” in there? Yeah, right. And it got blocked, because the implementation in that aged KVM box is no longer considered safe. Since you can’t just switch to a stream-based RC4 cipher, Java has no other choice but to drop the connection! Unless…  you do this:

public static void main(String[] paramArrayOfString)
  /* Added by the GAT from http://wp.xin.at                             */
  /* This disables CBC protection, thus re-opening the connections'     */
  /* BEAST vulnerability. No way around this due to a highly restricted */
  /* KLE ciphersuite. Without this fix, TLS connections with client     */
  /* certificates and PKI authentication will fail!                     */
  java.lang.System.setProperty("jsse.enableCBCProtection", "false");
  /* ------------------------------------------------------------------ */
  /* Added by the GAT from http://wp.xin.at                        */
  /* This enables insecure TLS renegotiation as per CVE-2009-3555  */
  /* in interoperable mode.                                        */
  java.lang.System.setProperty("sun.security.ssl.allowUnsafeRenegotiation", "false");
  java.lang.System.setProperty("sun.security.ssl.allowLegacyHelloMessages", "true");
  /* ------------------------------------------------------------- */
  KViewer localKViewer = new KViewer();
  localKViewer.mainArgs = paramArrayOfString;

Setting the jsse.enableCBCProtection property to false before the negotiation / handshake will make your code tolerate CBC ciphers vulnerable to BEAST attacks. Recompiling KViewer with all the code fixes including this one make it work fine with 2-way PKI authentication using a client certificate on both Java 1.7u60 and even Java 1.8u5. I have tested this using the 64-Bit x86 VMs on CentOS 6.5 Linux as well as on Windows XP Professional x64 Edition and Windows 7 Professional SP1 x64.

De-/Recompiled & "fixed" KViewer connecting to a machine much older even than its own crappy code

De-/Recompiled & “fixed” KViewer.jar connecting to a machine much older even than its own crappy code

I fear I cannot give you the modified source code, as TRENDnet would probably hunt me down, but I’ll give you the compiled byte code at least, the JAR file, so you can use it yourself. If you wanna check out the code, you could just decompile it yourself, losing only my added comments: [KViewer.jar]. (ZIPped, fixed / modified to work on Java 1.7+)

Both the modified code and the byte code “binary” JAR have been returned to TRENDnet in the context of my open support ticket there. I hope they’ll welcome it with open arms instead of suing me for decompiling their Java viewer.

In reality, even this solution is nowhere near perfect. While it does at least allow you to run modern Java runtime environments instead of highly insecure older ones plus using pretty secure PKI auth, it still doesn’t fix the man-in-the-middle-attack issues at hand. TRENDnet should fix their KVM firmware, enable it to run the TLSv1.2 protocol with AES256 Galois-Counter-Mode ciphers (GCM) and fix the many, many problems in their viewer clients. The TK-IP101 being an end-of-life product means that this is likely never gonna happen though.

It does say a lot when the consumer has to hack up the software of a supposedly high-security 400€ networking hardware piece by himself just to make it work properly.

I do still hope that TRENDnet will react positively to this, as they do not offer a modern replacement product to supersede the TK-IP101.

Sep 102013

NSA logoWith all that talk about the [National Security Agency] stealing our stuff (especially our most basic freedoms), it was time to look at a few things that Mr. Snowden and others before him have found out about how the NSA actually attempts to break certain encryption ciphers that are present in OpenSSLs and GnuTLSs cipher suites. Now that it has been clearly determined that a NSA listening post has been established in Vienna, Austria (protestors are on the scene), it may seem a good thing to look over a few details here. Especially now that the vulnerabilities are widely known and potentially exploitable by other perpetrators.

I am no cryptologist, so I won’t try to convince you that I understand this stuff. But from what I do understand, there is a side-channel attack vulnerability in certain block ciphers like for instance AES256-CBC-SHA or RSA-DES-CBC-SHA. I don’t know what it is exactly that’s vulnerable, but whoever may listen closely on one of the endpoints (client or server) of such a connection may determine crucial information by looking at the connections timing information, which is the side channel. Plus, there is another vulnerability concerning the Deflate protocol compression in TLS, which you shouldn’t confuse with stuff like mod_deflate in Apache, as this “Deflate” exists within the TLS protocol itself.

As most client systems – especially mobile operating systems like Android, iOS or Blackberry OS – are compromised and backdoored, it is quite possible that somebody is listening. I’m not saying “likely”, but possible. By hardening the server, the possibility of negotiating a vulnerable encrypted connection becomes zero – hopefully at least. :roll:

Ok, I’m not going to say “this is going to protect you from the NSA completely”, as nobody can truly know what they’re capable of. But it will make you more secure, as some vulnerable connections will no longer be allowed, and compromised/vulnerable clients are secure as long as they connect to a properly configured server. Of course you may also lock down the client by updating your browser for instance, as Firefox and Chrome have been known to be affected. But for now, the server-side.

I am going to discuss this for the Apache web server specifically, but it’s equally valid for other servers, as long as they’re appropriately configurable.Big Apache web server logoFirst, make sure your Apache is compatible with the SSL/TLS compression option SSLCompression [on|off]. Apache web servers starting from 2.2.24 or 2.4.3 should have this directive. Also, you should use [OpenSSL >=1.0] (link goes to the Win32 version, for *nix check your distributions package sources) to be able to use SSLCompression and also more modern TLSv1.1 and TLSv1.2 versions. If your server is new enough and properly SSL-enabled, please check your SSL configuration either in httpd.conf or in a separate ssl.conf included in httpd.conf, which is what some installers use as a default. You will need to change the SSLCipherSuite directive to not allow any vulnerable block ciphers, disable SSL/TLS protocol compression, and a few things more. Also make sure NOT to load mod_deflate, as this opens up similar loopholes as the default for the SSL/TLS protocols themselves do!

Edit: Please note that mixing Win32 versions of OpenSSL >=1.0 with the standard Apache version from www.apache.org will cause trouble, so a drop-in replacement is not recommended for several reasons, two being that that Apache version is linked against OpenSSL 0.9.8* (breaking TLS v1.1/1.2) and also built with a VC6 compiler, where OpenSSL >=1.0 is built with at least a VC9 compiler. Trying to run all VC9 binaries (Apache+PHP+SSL) only works on NT 5.1+ (Windows XP/2003 or newer), so if you’re on Win2000 you’ll be stuck with older binaries or you’ll need to accept stability and performance issues.

Edit 2: I now found out that the latest version of OpenSSL 0.9.8, namely 0.9.8y also supports switching off SSL/TLS deflate compression. That means you can somewhat safely use 0.9.8y which is bundled with the latest Apache 2.2 release too. It won’t give you TLS v1.1/1.2, but leaves you with a few safe ciphers at least!

See here:

SSLEngine On
SSLCertificateFile <path to your certificate>
SSLCertificateKeyFile <path to your private key>
ServerName <your server name:ssl port>
SSLCompression off
SSLHonorCipherOrder on
SSLProtocol All -SSLv2

This could even make you eligible for a VISA/Mastercard PCI certification if need be. This disables all known vulnerable block ciphers and said compression. On top of that, make sure that you comment out the loading of mod_deflate if not already done:

# LoadModule mod_deflate modules/mod_deflate.so

Now restart your webserver and enjoy!

The same thing can of course be done for mail servers, FTP servers, IRC servers and so on. All that is required is a proper configurability and compatibility with secure libraries like OpenSSL >=1.0 or at least 0.9.8y. If your server can do that, it can also be secured against these modern side channel attacks!

If you wish to verify the safety specifically against BEAST/CRIME attack vectors, you may want to check out [this tool right here]. It’s available as a Java program, .Net/C# program and source code. For the Java version, just run it like this:

java -jar TestSSLServer.jar <server host name> <server port>

This will tell you whether your server supports deflate, which cipher suites it supports and whether it’s BEAST or CRIME vulnerable. A nice point to start! For the client side, a similar cipher suite configuration may be possible to ensure the client won’t allow the negotiation of a vulnerable connection. Just updating your software may be an easier way in certain situations of course. A good looking output of that tool might appear somewhat like this:

Supported versions: SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Deflate compression: no
Supported cipher suites (ORDER IS NOT SIGNIFICANT):
  (TLSv1.0: idem)
  (TLSv1.1: idem)
Server certificate(s):
  2a2bf5d7cdd54df648e074343450e2942770ab6ff0: EMAILADDRESS=me@myserver.com, CN=www.myserver.com, OU=MYSERVER, O=MYSERVER.com, L=My City, ST=My County, C=COM
Minimal encryption strength:     strong encryption (96-bit or more)
Achievable encryption strength:  strong encryption (96-bit or more)
BEAST status: protected
CRIME status: protected

Plus, as always: Using open source software may give you an advantage here, as you can at least reduce the chances of inviting a backdoor eavesdropping on your connections onto your system. As for smartphones: Better downgrade to Symbian or just throw them away altogether, just like your tablets (yeah, that’s not the most useful piece of advice, I know…).

Update: And here a little something for your SSL-enabled UnrealIRCD IRC server.

UnrealIRCD logoThis IRC server has a directive called server-cipher-list in the context set::ssl, so it’s set::ssl::server-cipher-list. Here an example configuration, all the non-SSL specific stuff has been removed:

set {
  ssl { 
    trusted-ca-file "your-ca-cert.crt";
    certificate "your-server-cert.pem";
    key "your-server-key.pem";
    renegotiate-bytes "64m";
    renegotiate-time "10h";

Update 2: And some more from the Gene6 FTP server, which is not open source, but still extremely configurable. Just drop in OpenSSL >=1.0 (libeay32.dll, ssleay32.dll, libssl32.dll) as a replacement, and add the following line to your settings.ini files for SSL-enabled FTP domains, you can find the files in the Accounts\yourdomainname subfolders of your G6 FTP installation:

Gene6 FTP server logo


With that and those OpenSSL >=1.0 libraries, your G6 FTP server is now fully TLSv1.2 compliant and will use only safe ciphers!

Finally: As I am not the most competent user in the field of connection-oriented encryption, please just post a comment if you find some incorrect or missing information, thank you!

Sep 052013

OpenLDAP logoWhen we migrated from CentOS Linux 5 to 6 at work, I also wanted to make our OpenLDAP clients communicate with the OpenLDAP server – named slapd – in an encrypted fashion, so nobody can grab user names or home directory locations off the wire when clients and server communicate. So yes, we are using OpenLDAP to store all user names, password hashes, IDs, home directory locations and NFS exports in a central database. Kind of like a Microsoft domain system. Only more free.

To do that, I created my own certificate authority and a server certificate with a Perl script called CA.pl. What I missed is to set the validity period of both the CA certificate and server certificate to something really high. So it was the default – 1 year. Back then I thought it would be going to be interesting when that runs out, and so it was!

All of a sudden, some users couldn’t log in anywhere anymore, other users still worked. On some systems, a user would be recognized, but no home directory mount would occur. A courtesy of two different cache levels in mixed states on different clients (nscd cache and sssd cache) with the LDAP server itself already being non-functional.

So, the log files on clients and server didn’t tell anything. ANYthing. A mystery, even at higher log levels. So after fooling around for like two hours I decided to fire up Wireshark and analyze the network traffic directly. And what I found was a “certificate invalid” message on the TLSv1/TCP protocol layer. I crafted something similar here for you to look at, as I don’t have a screenshot of the original problem:

Wireshark detecting TLS problems

This is a problem slightly different from the actual one. It would say “Certificate invalid” instead of “Unknown CA”. In this case I just used a wrong CA to kind of replay the issue and show you what it might look like. IP and Mac addresses have been removed from this screenshot.

Kind of crazy that I had to use Wireshark to debug the problem. So clearly, a new certificate authority (CA) and server certificate were required. I tried to do that again with CA.pl, but it strangely failed this time in the certificate creation process. One that I tried to create manually with OpenSSL as I usually do with HTTPS certificates didn’t work as the clients complained that there was no ciphers available. I guess I did something wrong there.

There is a nice Perl+GTK tool that you can use to build CA certs as well as go through the process of doing signing requests and then signing certs with CA certs and exporting the resulting files in different formats however. That tool is called [TinyCA2] and is based on OpenSSL, as usual. See here:

The process is all the same as it is with pure command line OpenSSL or CA.pl: First, create your own certificate authority, then a server certificate with its corresponding request to the CA. Then sign that request with the CA certificate and you’re all done and you can now export the CA cert, signed server cert and private key to *.pem files or other types of encodings if necessary. And here you can make sure, that both your CA cert and server cert have very long validity periods.

Please note, that the CA cert should have a slightly longer validity period, as the server cert is created and signed afterwards, and having the server certificate being valid for just 1 minute longer than the signing CAs own certificate is unsupported and can result in serious problems with the software. In my case, I just gave the CA certificate 10 years + 1 day and the server certificate 10 years. Easy enough.

After that, put the CA cert as well as server cert + key on your OpenLDAP server as specified in /etc/openldap/slapd.conf and the CA cert on your clients as specified by /etc/sssd/sssd.conf or if you’re not using RedHats sssd, then it’d be /etc/openldap/ldap.conf and /etc/nslcd.conf, which are however deprecated and not recommended anymore. Also make sure your LDAP server can access the files (maybe chown ldap:ldap or chown root:ldap) and furthermore please make sure nobody can steal the private key!!

Then, restart your LDAP server as well as sssd on all clients, or nslcd on all clients if you’re using the old stuff. If an NFS server is also part of your infrastructure, you will also need to restart the rpcidmapd service on all systems accessing that NFS or you’ll get a huge ownership mess when all user ids are being wrongly mapped to the unprivileged user nobody, which may happen with NFS when your idmapd is in an inconsistent state after fooling around with sssd or nslcd.

That’s of course not meant to be guide on how to set up OpenLDAP + TLS as a whole, just for replacing the certificates. So if you ever get “user does not exist” upon calling id user or su – user and people can’t log in anymore, that might be your issue if you’re using OpenLDAP with TLS or SSL. Just never ever delete root from your /etc/passwd. Otherwise even you will be locked out as soon as slapd – your LDAP server – goes down! And then you’ll need Knoppix or something similar – nobody wants to actually stand up from ones chair and to need to physically access the servers. ;)

Jul 022012

Apache HTTP Server logoFor quite some time now, my Apache webserver (WAMPS – Windows+Apache+MySQL+PHP+SSL) was working just fine. I had weird bandwidth problems once, but they’re all miraculously gone. I wasn’t thinking anything bad, when all of a sudden my forced HTTP->HTTPS URL rewrite was failing. At first I suspected the htdigest password authentication that was to follow, but that was not it. After a little bit of testing – including some telnet to port 443 – it was clear, that the HTTP server was sending plain HTTP over the HTTPS port. A quick check of all my SSL related settings didn’t turn up anything. Well, of course not, I didn’t change anything, so why should the settings be a problem? Now let me point you at a strange error log entry that I couldn’t identify correctly at first:

[Fri Jun 29 22:19:00 2012] [error] [client] Invalid method in request \x16\x03\x01
[Fri Jun 29 22:19:01 2012] [error] [client] Invalid method in request \x16\x03

Now if you ever see those popping up in your Apache error log, you might be experiencing something like me. It was clear that SSL was broken after testing with telnet, but why? On the web, there were many explanations but never the one that finally nailed it: It was the certificates lifetime that had expired! Strangely Apache hadn’t complained about that when loading the certificate on startup:

[Fri Jun 29 22:47:19 2012] [info] Loading certificate & private key of SSL-aware server
[Fri Jun 29 22:47:19 2012] [info] Init: Generating temporary RSA private keys (512/1024 bits)
[Fri Jun 29 22:47:24 2012] [info] Init: Generating temporary DH parameters (512/1024 bits)
[Fri Jun 29 22:47:24 2012] [info] Init: Initializing (virtual) servers for SSL
[Fri Jun 29 22:47:24 2012] [info] Configuring server for SSL protocol
[Fri Jun 29 22:47:24 2012] [debug] ssl_engine_init.c(465): Creating new SSL context (protocols: SSLv2, SSLv3, TLSv1)
[Fri Jun 29 22:47:24 2012] [debug] ssl_engine_init.c(420): Configuring TLS extension handling
[Fri Jun 29 22:47:24 2012] [debug] ssl_engine_init.c(795): Configuring RSA server certificate
[Fri Jun 29 22:47:24 2012] [debug] ssl_engine_init.c(834): Configuring RSA server private key
[Fri Jun 29 22:47:24 2012] [info] mod_ssl/2.2.21 compiled against Server: Apache/2.2.21, Library: OpenSSL/0.9.8r

Looks perfectly fine? Well, it ain’t. Checking the certificate on the server reveals this (checking on the client is impossible as the server won’t send the certificate anymore, unless you got cached certs). So, well, I created a new self-signed one using OpenSSL, and that’s that. It’s strange that Apache doesn’t log the expired certificate, I think it most definitely should. But well, there you go. If your SSL suddenly stops working, it might pay off to check the certificates lifetime first, either on the server, or by checking a local cached version.