Now if you decide to read this post, you’ll be going through a lot of technical crap again, so to make it easier for you, click on the posts logo to the left. There you get an interesting picture to look at if you get bored by me talking, and if you apply the cross-eye technique (although you’ll have to cross your eyes up to a level where it might be a bit of a strain, just try it until you get a “lock” on the “middle” picture), you’ll see the image in sterescopic 3D. Yay, boobies in stereoscopic 3D, not real ones though, very unfortunate. Better than nothing though, eh?
Now, where was I? Ah yes, stereoscopy. These days it seems to be a big thing, not just in movie theatres, but also for the home user with 3D Blu-Rays and 3D HDTVs. I’m quite into transcoding Blu-Rays using the x264 encoder as well as a bunch of other helper tools, and I’m archiving all my discs to nice little MKV files, that I prefer to having to actually insert the BD and deal with all the DRM crap like ACSS encryption and BD+. Ew.
But, with 3D, there is an entirely new challenge to master. On a Blu-Ray, 3D information is stored as a so-called MVC stream, which is an extension to H.264/AVC, which also means, there is no Microsoft Codec involved anymore, so there is no VC-1 3D. Only H.264. This extension is for the right eye and works like a diff stream or like DTS MLP data roughly, so while the full 2D left eye stream may be 10GB in size, the right eye one (as it only contains parts that are different from the left eye stream) will be maybe around 5GB. Usually, stereoscopic 3D works by giving you one image for your left and one for your right eye, while filtering the left eye stream for the right eye (so the right eye never sees the left eye stream) and vice-versa. There are slight differences in the streams, which trick your brain into interpreting the information presented to it as 3D depth.
A cheap version of “filtering” is the cross-eye technique, that works best for plain images, like the logo above, which is a so called side-by-side 3D image. Side-by-side is cheap, easy, and for movies it’s very compatible, because pretty much any 3D screen supports it. Also, there are free tools available to work with SBS 3D. But, before SBS 3D can be created, the method of the Blu-Ray (which is not SBS but “frame packing”, see my description of MVC above) needs to be decoded first. Since the MVC right eye stream is a diff, and not a complete one, the complete SBS right eye stream needs to be constructed from the MVC and the left-eye H.264/AVC first. That can be done using certain plugins in the AviSynth frameserver under Windows. One of those plugins is H264StereoSource, which offers the function
Using this, i successfully transcoded Prometheus 3D. However, that was the only success ever. I tried it again as I got new 3D Blu-Rays, but all of a sudden, this (click to enlarge):
As you can see in the shell window, there is lots of weird crap instead of regular x264 output. At some point, rather sooner than later, x264 would just crash. Taking whatever output it has generated by that time, you get something that looks like this (click to enlarge):
Clearly, this is bad. As an SBS, the right frame should look awfully familiar, almost mirroring the left one. As you can see, this looks awfully not very much like anything. Fragments of the shapes of the left eye stream can be recognized in the right eye one when the movie plays, but it’s totally garbled and a bit too.. well… green. Whatever.
H264StereoSource() fucked up big time, so I had to look into another solution. Luckily, [Sharc], a [Doom9] user pointed me in the right direction by mentioning [BD3D2MK3D].
This is yet another GUI with some of the tools I already use in the background plus some additional ones, most significantly another AviSynth plugin called SSIFSource2.dll, offering the function
ssifSource2(). That one reads an SSIF container (which is H.264/AVC and MVC linked together) directly, and can not only decode it, but also generate proper left/right eye output from it directly.
Now first, this is what my old AviSynth script looked like, the now broken one:
LoadPlugin("C:\Program Files (x86)\VFX\AviSynth 2.5\plugins\DGAVCDecode.dll")
LoadPlugin("C:\Program Files (x86)\VFX\BDtoAVCHD\H264StereoSource\H264StereoSource.dll")
left = AVCSource("D:\left.dga")
right = H264StereoSource("D:\crash.cfg", 132566).AssumeFPS(24000,1001)
return StackHorizontal(HorizontalReduceBy2(left), HorizontalReduceBy2(right)).Trim(0, 132516)
This loads the H.264/AVC decoder, the stereo plugin, reads the left-eye stream index file, a dga created by DGAVCIndex, and loads a configuration file that defines both streams as they were demuxed by using eac3to. Then it returns a shrinked version of the output (compressing the 3840×1080 full SBS stream to 1920×1080 half SBS, that most TVs can handle), trimming the last 50 frames away to work around a frame alignment bug between decoder and encoder. The cfg file looks somewhat like this:
InputFile = "D:\left.h264"
InputFile2 = "D:\right.h264"
FileFormat = 0
POCScale = 1
DisplayDecParams = 1
ConcealMode = 0
RefPOCGap = 2
POCGap = 2
IntraProfileDeblocking = 1
DecFrmNum = 0
By playing around with it I found out that it was truly the right-eye MVC decoding that did no longer work,
expand/collapse source code
H264StereoSource() was just broken, so
ssifSource2() then! BD3D2MK3D will allow you to use its GUI to analyze a decrypted 3D Blu-Ray folder structure, and generate an AviSynth script for you. There was only one part that did not work for me, and that was the determination of the frame count. But there’s an easy fix. The AviSynth script that BD3D2MK3D will generate will look somewhat like this:
# Avisynth script generated Wed Mar 06 18:07:23 CET 2013 by BD3D2MK3D v0.13
# to convert "D:\0_ADVDWORKDIR\My Movie\3D\MYMOVIE\BDMV\STREAM\SSIF\00001.ssif"
# (referenced by playlist 00001.mpls)
# to Half Side by Side, Left first.
# Movie title: My Movie 3D
# Source MPLS information:
# MPLS file: 00001.mpls
# 1: Chapters, 16 chapters
# 2: h264/AVC (left eye), 1080p, fps: 24 /1.001 (16:9)
# 3: h264/AVC (right eye), 1080p, fps: 24 /1.001 (16:9)
# 4: DTS Master Audio, English, 7.1 channels, 16 bits, 48kHz
# (core: DTS, 5.1 channels, 16 bits, 1509kbps, 48kHz)
# 5: AC3, English, 5.1 channels, 640kbps, 48kHz, dialnorm: 28dB
# 6: DTS Master Audio, German, 5.1 channels, 16 bits, 48kHz
# (core: DTS, 5.1 channels, 16 bits, 1509kbps, 48kHz)
# 7: AC3, Turkish, 5.1 channels, 640kbps, 48kHz
# 8: Subtitle (PGS), English
# 9: Subtitle (PGS), German
# 10: Subtitle (PGS), Turkish
#LoadPlugin("C:\Program Files (x86)\VFX\BD3D2MK3D\toolset\stereoplayer.exe\DirectShowMVCSource.dll")
## Alt method using ssifSource2 (for SBS or T&B only):
LoadPlugin("C:\Program Files (x86)\VFX\BD3D2MK3D\toolset\stereoplayer.exe\ssifSource2.dll")
SIDEBYSIDE_L = 14 # Half Side by Side, Left first
SIDEBYSIDE_R = 13 # Half Side by Side, Right first
OVERUNDER_L = 16 # Half Top/Bottom, Left first
OVERUNDER_R = 15 # Half Top/Bottom, Right first
SIDEBYSIDE_L = 14 # Full Side by Side, Left first
SIDEBYSIDE_R = 13 # Full Side by Side, Right first
OVERUNDER_L = 16 # Full Top/Bottom, Left first
OVERUNDER_R = 15 # Full Top/Bottom, Right first
ROWINTERLEAVED_L = 18 # Row interleaved, Left first
ROWINTERLEAVED_R = 17 # Row interleaved, Right first
COLINTERLEAVED_L = 20 # Column interleaved, Left first
COLINTERLEAVED_R = 19 # Column interleaved, Right first
OPTANA_RC = 45 # Anaglyph: Optimised, Red - Cyan
OPTANA_CR = 46 # Anaglyph: Optimised, Cyan - Red
OPTANA_YB = 47 # Anaglyph: Optimised, Yellow - Blue
OPTANA_BY = 48 # Anaglyph: Optimised, Blue - Yellow
OPTANA_GM = 49 # Anaglyph: Optimised, Green - Magenta
OPTANA_MG = 50 # Anaglyph: Optimised, Magenta - Green
COLORANA_RC = 39 # Anaglyph: Colour, Red - Cyan
COLORANA_CR = 40 # Anaglyph: Colour, Cyan - Red
COLORANA_YB = 41 # Anaglyph: Colour, Yellow - Blue
COLORANA_BY = 42 # Anaglyph: Colour, Blue - Yellow
COLORANA_GM = 43 # Anaglyph: Colour, Green - Magenta
COLORANA_MG = 44 # Anaglyph: Colour, Magenta - Green
SHADEANA_RC = 33 # Anaglyph: Half-colour, Red - Cyan
SHADEANA_CR = 34 # Anaglyph: Half-colour, Cyan - Red
SHADEANA_YB = 35 # Anaglyph: Half-colour, Yellow - Blue
SHADEANA_BY = 36 # Anaglyph: Half-colour, Blue - Yellow
SHADEANA_GM = 37 # Anaglyph: Half-colour, Green - Magenta
SHADEANA_MG = 38 # Anaglyph: Half-colour, Magenta - Green
GREYANA_RC = 27 # Anaglyph: Grey, Red - Cyan
GREYANA_CR = 28 # Anaglyph: Grey, Cyan - Red
GREYANA_YB = 29 # Anaglyph: Grey, Yellow - Blue
GREYANA_BY = 30 # Anaglyph: Grey, Blue - Yellow
GREYANA_GM = 31 # Anaglyph: Grey, Green - Magenta
GREYANA_MG = 32 # Anaglyph: Grey, Magenta - Green
PUREANA_RB = 23 # Anaglyph: Pure, Red - Blue
PUREANA_BR = 24 # Anaglyph: Pure, Blue - Red
PUREANA_RG = 25 # Anaglyph: Pure, Red - Green
PUREANA_GR = 26 # Anaglyph: Pure, Green - Red
MONOSCOPIC_L = 1 # Monoscopic 2D: Left only
MONOSCOPIC_R = 2 # Monoscopic 2D: Right only
# Load main video (0 frames)
#DirectShowMVCSource("D:\0_ADVDWORKDIR\My Movie\3D\MYMOVIE\BDMV\STREAM\SSIF\00001.ssif", seek=false, seekzero=true, stf=SIDEBYSIDE_L)
## Alt method using ssifSource2
ssifSource2("D:\0_ADVDWORKDIR\My Movie\3D\MYMOVIE\BDMV\STREAM\SSIF\00001.ssif", 153734, left_view = true, right_view = true, horizontal_stack = true)
# Resize is necessary only for Half-SBS or Half-Top/Bottom,
# or when the option to Resize to 720p is on.
# Anaglyph and Interleaved modes are in RGB32, and must be converted to YV12.
So here, when
ssifSource2() is being called, there is a number “153734”, where initially there will only be “0”. I got the run time and frame rate of the movie from the file eac3to_demux.log, that BD3D2MK3D generates. Using that, you can easily calculate your total frame number to enter there, like I have already done in this example. Also enter the framerate in the
AssumeFPS() function. Here, 24000,1001 means 24000/1001p or in other words roughly 23.976fps, which is the NTSC film frame rate. Now, the tricky part: For this function to operate, the libraries and binaries of BD3D2MK3D must be in the systems search path and our x264 encoder must sit in the same directory as the libs, also you will want to make sure you’re using a 32-Bit version of x264. In our example, the installation path of BD3D2MK3D is C:\Program Files (x86)\VFX\BD3D2MK3D\, and the tools and libs are in C:\Program Files (x86)\VFX\BD3D2MK3D\toolset\stereoplayer.exe\. So, open a cmd shell, copy your preferred 32-bit x264.exe to C:\Program Files (x86)\VFX\BD3D2MK3D\toolset\stereoplayer.exe\ and then add the necessary paths to the search path:
set %PATH%=C:\Program Files (x86)\VFX\BD3D2MK3D\toolset\stereoplayer.exe;C:\Program Files (x86)\VFX\BD3D2MK3D\toolset;%PATH%
Of course, you’ll need to adjust the paths to reflect your local BD3D2MK3D installation folder. Now, you can call x264 using the generated AviSynth script as its input (usually called _ENCODE_3D_MOVIE.avs), and it should work! Just make sure you call x264.exe from that folder C:\Program Files (x86)\VFX\BD3D2MK3D\toolset\stereoplayer.exe\, and not from somewhere else on the system, or you’ll get this:
avs [error]: Can't create the graph!
For me, x264 using
ssifSource2() sometimes talks about adding tons of duplicate frames while transcoding. I am not sure what this means and how it may affect things, but for now output seems to work just fine regardless. When it does happen, it looks like this:
Nonetheless, it creates half-SBS output, and using MKVtoolnix you can multiplex that into an MKV/MK3D container setting the proper stereoscopy flags (usually half SBS, left eye first) and everything will work just fine.
My method using half-SBS will cost half the horizontal resolution unfortunately, but few TVs support full SBS, plus it’s costly because files would become far larger at similar quality settings. So far I’ve been pretty surprised how good, sharp and detailed it still looks, despite the halved resolution. So far it’s the only real possibility anyway, as full-SBS (using two 1920×1080 frames besides each other) is not very wide-spread, large and bandwidth-hungry, and for the frame packed mode that Blu-Rays use, there are simply no free tools available so far, so direct MVC transcoding is not an option.
But, at least it works now, thanks to the [help from the Doom9 forums]!
Update: It seems the frame duplicates are a really weird bug, that makes the output very jittery and no longer as smooth as you would expect. Even stranger is the fact, that it is not deterministic. Sometimes it happens a lot, adding tens of thousands of frame duplicates. Then, abort, restart the process and the problem is magically gone. Sometimes you might have to restart 10-20 times and all of a sudden it works. How is this even possible? I do not know. Luckily, BD3D2MK3D also comes with a second Avisynth plugin that you can use instead of ssifSource2.dll, this one is called DirectShowMVCSource.dll and gives you the corresponding function
DirectShowMVCSource() you will also no longer need to determine a frame count, it will be able to do that all by itself. Also, BD3D2MK3D will automatically add it to the Avisynth script that it generates, it’s just commented out. So, uncomment it, comment the ssifSource2 lines instead, and you’re good to go. So far, the error rate seems to be far lower with
DirectShowMVCSource(), although I had a problem once, with a very long movie / high frame count, here a 2nd pass encode would just stall at 0% CPU usage.
But other than that it seems to be more solid, even if it has to suboptimally filter stuff through DirectShow. There is another more significant drawback though:
DirectShowMVCSource() is a lot slower than
ssifSource2(). I would say, it’s a factor of 3-4 even. But at least it’s giving us another option!