Sep 022015
 

colorecho logoRecently, I am doing a lot of audio/video transcoding, and for that I’m using tools like mkvextract, mkvmerge, x264, avconv and fghaacenc on the command line. Mostly this is to get rid of old files that are either very inefficiently encoded – like DivX3 or MPEG-2 – or are just too large/high-bitrate for what the content’s worth. Since I do have large batches of relatively similar files, I wrote myself a few nice loops that would walk through an entire stack of videos and process all of it automatically. Because I want to see at first glance how far through a given video stack a script is, I wanted to output some colored notifications on the shell after each major processing step, telling me the current video number. That way, I could easily see how far the job had progressed. We’re talking processes here that take multiple days, so it makes sense.

Turns out that this was harder than expected. In the old DOS days, you would just load ANSI.SYS for that, and get ANSI color codes to work with. The same codes still work in modern UNIX terminals. But not within the Windows cmd for whatever reason. Now there is an interface for that kind of stuff, but seemingly no tools to use it. Like e.g. the echo command simply can’t make use of it.

Now there is project porting ANSI codes back to the cmd, and it’s called [ANSICON]. Basically just a DLL hooked into the cmd. Turns out I can’t use that, because I’m already using [clink] to make my cmd more UNIX-like and actually usable, which also is additional code hooked into the cmd, and the two confict with each other. More information about that [here].

So how the hell can I do this then? Turns out there is a batch script that can do it, but it’s a very outlandish hack that doesn’t seem to want to play nice with my shells if I do @echo off instead of echo off as well. I guess it’s clinks work breaking stuff here, but I’m not sure. Still, here is the code, as found by [Umlüx]:

expand/collapse source code
  1. ECHO OFF
  2. SETLOCAL EnableDelayedExpansion
  3.  
  4. FOR /F "tokens=1,2 delims=#" %%A IN ('"PROMPT #$H#$E# & ECHO ON & FOR %%b IN (1) DO REM"') DO (
  5.   SET "DEL=%%A"
  6. )
  7.  
  8. ECHO say the name of the colors, dont read
  9.  
  10. CALL :ColorText 0a "blue"
  11. CALL :ColorText 0C "green"
  12. CALL :ColorText 0b "red"
  13. ECHO.
  14. CALL :ColorText 19 "yellow"
  15. CALL :ColorText 2F "black"
  16. CALL :ColorText 4e "white"
  17.  
  18. GOTO :eof
  19.  
  20. :ColorText
  21.  
  22. ECHO OFF
  23.  
  24.  "%~2"
  25. findstr /v /a:%1 /R "^$" "%~2" NUL
  26. DEL "%~2" > NUL 2>&1
  27.  
  28. GOTO :eof

So since those two solutions didn’t work for me, what else could I try?

Then I had that crazy idea: Since the interface is there, why not write a little C program (actually ended up being C++ instead) that uses it? Since I was on Linux at the time, I tried to write it there and attempt my first cross-compile for Windows using a pre-built [MinGW-w64] compiler, that luckily just played along nicely on my CentOS 6.6 Linux. You can get pre-built binaries for 32-Bit Windows targets [here], and for 64-bit Windows targets [here]. Thing is, I know zero C++, so that took some time and it’s quite crappy, but here’s the code:

expand/collapse source code
  1. #include <iostream>
  2. #include "windows.h"
  3.  
  4. // Set NT commandline color:
  5. void SetColor(int value) {
  6.     SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), value);
  7. }
  8.  
  9. int main(int argc, char *argv[]) {
  10.   SetColor(atoi(argv[2]));
  11.   std::cout << "\r\n" << argv[1] << "\r\n";
  12.   SetColor(7);
  13.   return 0;
  14. }

Update 2016-06-22: Since I wanted to work with wide character strings in recent days (so, non-ANSI Unicode characters), my colorecho command failed, because it couldn’t handle them at all. There was no wide character support. I decided to update the program to also work with that using Windows-typical UCS-2le/UTF-16, here’s the code:

expand/collapse source code
  1. #include <iostream>
  2. #include <io.h>
  3. #include <fcntl.h>
  4. #include "windows.h"
  5.  
  6. // Set NT commandline color:
  7. void SetColor(int value) {
  8.     SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), value);
  9. }
  10.  
  11. // Unicode main
  12. int wmain(int argc, wchar_t *argv[]) {
  13.   SetColor(_wtoi(argv[2]));
  14.   _setmode(_fileno(stdout), _O_U16TEXT);      // Set stdout to UCS-2le/UTF-16
  15.   std::wcout << "\r\n" << argv[1] << "\r\n";  // Write unicode
  16.   SetColor(7);                                // Set color back to default
  17.   return 0;
  18. }

I also updated the download section and the screenshots further below. End of update.

This builds nicely using MinGW on Linux for 64-bit Windows targets. Like so: While sitting in the MinGW directory (I put my code there as echo.cpp as well, for simplicities’ sake), run: ./bin/x86_64-w64-mingw32-g++ echo.cpp -o colorecho-x86_64.exe. If you want to target 32-bit Windows instead, you’d need to use the proper MinGW for that (it comes in a separate archive): ./bin/i686-w64-mingw32-g++ echo.cpp -o colorecho-x86_64.exe

When building like that, you’d also need to deploy libstdc++-6.so and libgcc_s_seh-1.dll along with the EXE files though. The DLLs can be obtained from your local MinGW installation directory, subfolder ./x86_64-w64-mingw32/lib/ for the 64-bit or ./i686-w64-mingw32/lib/ for the 32-bit compiler. If you don’t want to do that and rather rely on Microsofts own C++ redistributables, you can also compile it with Microsoft VisualStudio Express, using its pre-prepared command line. You can find that here, if you have Microsofts VisualStudio installed – version 2010 in my case:

VC2010 command lines

The command lines of Visual Studio (click to enlarge)

Here, the “Visual Studio Command Prompt” is for a 32-bit build target, and “Visual Studio x64 Win64 Command Prompt” is for building 64-bit command line programs. Choose the appropriate one, then change into a directory where you have echo.cpp and run the following command: cl /EHsc /W4 /nologo /Fecolorecho.exe echo.cpp, giving you the executable colorecho.exe.

Alternatively, you can just download my pre-compiled versions here:

Update 2016-06-22: And here are the Unicode versions:

End of update.

Note that this tool does exactly what I need it to do, but it’ll likely not do exactly what you’d need it to do. Like e.g. the line breaks it adds before and after its output. That’s actually a job for the shells echo command to do, not some command line tool. But I just don’t care. So that’s why it’s basically a piece of crap for general use. The syntax is as follows, as shown for the 64-bit VC10 build:

colorecho-x86_64-vc10.exe "I am yellow!" 14

When run, it looks like this:

Colorecho running

colorecho invoked on the Windows cmd

So the first parameter is the string to be echoed, the second one is the color number. That number is 2-digit and can affect both the foreground and the background. Here the first 16 of them, which are foreground only:

  • 0: Black
  • 1: Dark blue
  • 2: Dark green
  • 3: Dark cyan
  • 4: Dark red
  • 5: Dark purple
  • 6: Dark yellow
  • 7: Light grey
  • 8: Dark grey
  • 9: Light blue
  • 10: Light green
  • 11: Light cyan
  • 12: Light red
  • 13: Light purple
  • 14: Light yellow
  • 15: White

If you go higher than that, you’ll also start changing the background colors and you’ll get different combinations of foreground and background colorization.

The background colors actually follow the same order, black, dark blue, dark green, etc. Numbers from 0..15 are on black background, numbers from 16..31 are on a dark blue background and so on. This makes up pretty much the same list:

  • 0..15: 16 foreground colors as listed above on the default black background, and
  • 16..31: on a dark blue background
  • 32..47: on a dark green background
  • 48..63: on a dark cyan background
  • 64..79: on a dark red background
  • 80..95: ona dark purple background
  • 96..111: on a dark yellow background
  • 112..127: on a light grey background
  • 128..143: on a dark grey background
  • 144..159: on a light blue background
  • 160..175: on a light green background
  • 176..191: on a light cyan background
  • 192..207: on a light red background
  • 208..223: on a light purple background
  • 224..239: on a light yellow background
  • 240..255: on a white background

Going over 255 will simply result in an overflow causing the system to start from the beginning, so 256 is equal to 0, 257 to 1, 260 to 4 and so on.

Update 2016-06-22: With the Unicode version, you can now even do stupid stuff like this:

colorecho unicode with DejaVu Sans Mono font on Windows cmd

It could be useful for some people? Maybe?

Note that this has been rendered using the DejaVu Sans Mono font, you can get it from [here]. Also, you need to tell the Windows cmd which fonts to allow (only monospaced TTFs work), you can read how to do that [here], it’s a relatively simple registry hack. I have yet to find a font that would work and also render CJK characters though, but with the fonts I myself have available at the moment, asian stuff won’t work, you’ll just get some boxes instead:

colorecho unicode CJK fail

Only characters which have proper glyphs for all desired codepoints in the font in use can be rendered, at least on Windows XP.

Note that this is not a limitation of colorecho, it does output the characters correctly, “ですね” in this case. It’s just that DejaVu Sans Mono has no font glyphs to actually draw them on screen. If you ever come across a neat cmd-terminal-compatible font that can render most symbols and CJK, please tell me in the comments, I’d be quite happy about that! End of update.

If somebody really wants to use this (which I doubt, but hey), but wishes it to do things in a more sane way, just request it in the comments. Or, if you can, just take the code, change and recompile it by yourself. You can get Microsofts VisualStudio Express – in the meantime called [VisualStudio Community] – free of charge anyway, and MinGW is free software to begin with.

Jan 222014
 

clink logoTypically, any avid Linux/UNIX user would sneer at the default Windows command line shell, cmd.exe. I do use it from time to time though, for tools like [eac3to] or [x264]. By Bash and other UNIX shells’ standards, cmd.exe is simply weak, powerless and inefficient though. On top of that, its scripting language – simply called Batch – is quite awkward and nothing compared to say Bash. You can go the classic “do one thing well” [1] route and extend the Batch functionality with specialized tools like grep, awk and sed for Windows, or even by alternative scripting languages like MS VBScript or Perl, which I sometimes do. But oh well, we’re not gonna fix that.

But using the awkward interactive shell? We can fix that! Say hello to [clink]!

Welcome to cmd+clink

Welcome to cmd+clink

So what’s clink? A way of enhancing cmd.exe (and partly even the newer [Windows PowerShell]) with the [GNU readline] libraries’ functionality and with other upgrades. “Aha” you might think, “so how’s that gonna help?”. Well, it’s going to help by adding tons of stuff you would normally only see on a UNIX-style shell, I’ll give you a few examples:

  • A better auto-completion that is not limited to auto-completing local file names:
    • cmd can still auto-complete file and folder names in your current directory by pressing <TAB>. It will now do so incrementally though!
    • cmd can now also auto-complete all executable program names in your %PATH%! You wanna launch notepad? Type “not”, press <TAB>, <ENTER>, done. If there are ambiguous results, double-<TAB> will show you a list of eligible file names or other auto-completable objects!
    • It also works for environment variables. You want to go to your users home directory in %USERPROFILE%? Type “cd %u”, press <TAB>, it will auto-complete to “cd %user”, as there are three variables called %USERPROFILE%, %USERDOMAIN% and %USERNAME%. Extend that by adding a p: “cd %userp”, press <TAB> again and you got the whole variable, then <ENTER>. All that can greatly speed up your shell usage!
    • If all that’s not enough for you, you can extend and modify the auto-completion behavior using the Lua scripting language.
  • The command history is no longer “Doskey” style. It will stay persistent across sessions, and is now incrementally searchable (<CTRL>+<R> / <CTRL>+<S>)!
  • The clipboard integration is now far better. Simply paste stuff from the clipboard to the shell using <CTRL>+<V> like everywhere else, or use the right mouse button.
  • Powerful line editing features like delete last word with <CTRL>+<W>, blank screen (<CTRL>+<L>), undo (<CTRL>+<Z>)  or other stuff like quit the shell (<CTRL>+<D>), scroll around without having to use the mouse (<SHIFT>+<PgUp> and <SHIFT>+<PgDn>) and much, much more.

A few sample shots of the auto-completion:

Auto-completing a program in the search path

Auto-completing a program name in the search path

Auto-completing an environment variable

Auto-completing an environment variable

One feature I didn’t get to work yet was environment variable expansion using <CTRL>+<ALT>+<E>, which works on bash in UNIX. Maybe I did something wrong there, but in case I didn’t, let’s see how fast [new bug reports] (link removed due to project migration to github, the issue described remains unfixed, but exists only on XP, XP x64 & Server 2003 systems) are attended to in the clink project.

But even without that, it’s pretty powerful already, and I love it, so much more convenient. And there I was, just looking for a way to quit the shell with <CTRL>+<D> and got so much more instead! :)

[1] Arnold Robbins, Nelson H. F. Beebe. Classic Shell Scripting, page 4. O’Reilly Media, Inc., 2005.