Aug 222012
 

x264 LogoSince Linux results from my [x264 benchmark] are becoming more numerous as of late, I decided to write a little guide that can show people how to compile and install the libav suite as well as x264 itself on Linux, and additionally also MacOS X to get the benchmark to work there. In its current state, there might still be some errors, or maybe missing things here and there in the guides, so if you spot anything that’s not right, or that doesn’t work as intended, feel free to comment here! The guides are available in two languages, english and german, you can switch the language on top of the guide.

Please do also comment on it here, in case you have any ideas for improvement! Here are the according links:

Update: MacOS X on PPC added! Thanks fly out to Blacktron and Viper for researching ways of doing this on MacOS X on Intel, and also many thanks to FS03 for giving me remote access to his MacOS X on PowerPC machine to figure out ways to build the code there. Great work!

Update 2: After a lot of hard work, i finally added a guide for the BeOS-based open source operating system Haiku! It’s not a trivial installation/configuration, but when done right, it works just as beautifully as it does on Linux and MacOS X. :)

Update 3: That was even more freaky than on Haiku. Library calls resulting in segmentation faults because of certain compiler flags, linking problems, processor instruction misdetection, a lot of tools that are named like GNU tools, but just aren’t and don’t behave like them – you name it, OpenSolaris has it all! If you’re looking for a Unix system to break your configure/make style source tree, get OpenSolaris, it’ll do it, no-holds-barred! But in the end, I defeated Suns system with one hell of a lot of fixes and workarounds, the guides are now linked in the list above! :)

Update 4: Added guides for the Windows-compatible NT5-style open source operating system ReactOS. ReactOS is an attempt to implement the entire Windows kernel and userland APIs in a free and open source system. However, because of lots of kernel bugs, you need to use a specific trunk build of the system to be able to run the benchmark without bluescreen. Also, a small change needs to be applied to the benchmark script. You can, however, use the regular Windows version on the benchmark if you stick to the guide!

Update 5: Added a guide for NetBSD 6.0 RC2. While I failed to create proper builds on FreeBSD and OpenBSD due to a weirdly broken GNU portable assembler, it succeeded on NetBSD 6.0 RC2, so the corresponding guides have been added to the ever-growing list of tested operating systems! :)

Update 6: Another newcomer, a guide for DragonFly BSD 3.0.2 has been added. That means, that 50% of the real BSD distributions have now been successfully covered! The hardness of getting it to work on DragonFly BSD was comparable to NetBSD, so harder than Linux, but far easier than Solaris or Haiku.

Update 7: And yet another one joins the pack, PC-BSD is now part of the family. Required a different set of tricks, but it worked again!

Update 8: Upon a users request, an installation guide for regular Windows machines has now been added.

Update 9: Closing in on covering the entirety of all BSD UNIX systems, FreeBSD 9.2 has now also been conquered and links have been added!

Update 10: And here comes another obscure BSD UNIX called MidnightBSD, currently in version 0.4. The system had been forked from FreeBSD 6, and has since then developed in its own direction, adding portions from OpenBSD and NetBSD on the way. Also, the developer has named the system after his black cat, Midnight. As usual, guides in English and German have been added, this was yet another hairy ride.

Update 11: Well, there was this gas (GNU assembler) problem with it being too old on OpenBSD. But now I finally managed to compile my own GNU binutils on OpenBSD (at least partially) and with that working assembler and some other little tricks I got libav and x264 to work on OpenBSD 5.4 perfectly fine! According to certain posts on the OpenBSD misc mailing list, the next version 5.5’ll come with the Clang compiler backend and a newer, modern assembler. So in the future this might be a great deal easier. But for now I have fought for months against OpenBSD 5.3 and now 5.4, and finally, another success! Respective guides have been added, as usual. And since there were also some system limits confusing me a lot (failed malloc() system call due to a per-user memory allocation limit), thanks got to fly out to Mr. Vrincianu, misc mailing list owner at OpenBSD.org! His help was the final piece I needed to solve this once and for all! :)

Update 12: Since OpenSolaris is no more, and it was merged into Oracle Solaris, there is now a new Oracle Solaris 11.2 guide, and it got a bit simpler too when compared to OpenSolaris and older versions of libav/x264. On top of that, if you choose to run the brand new OpenBSD 5.6 UNIX, you can probably just follow the regular Linux guide, as compatibility between libav and x264 with OpenBSD is now much higher since version 5.6 when using modern versions of libav+x264 too!

Update 13: And with GNU/Hurd, another free platform joins the ranks. The GNU operating system with its Mach kernel makes it surprisingly easy to build libav and x264, as long as you stick with the Debian GNU/Hurd distribution.

Update 14: Thanks to the [hard work of RayeR], we now also have an amazing x264 version for DOS! There are a few limitations though: Naturally, there is only single CPU support, and all assembly optimizations had to be switched off because of a problem with either the compiler or linker of DJGPP. The new guide focuses on FreeDOS 1.1, which is also free software.

CC BY-NC-SA 4.0 x264 benchmark on Linux, GNU/Hurd, MacOS X, (Open)Solaris, NetBSD, DragonFly BSD, PC-BSD and Haiku (hey, even ReactOS!): A guide by The GAT at XIN.at is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

  12 Responses to “x264 benchmark on Linux, GNU/Hurd, MacOS X, (Open)Solaris, NetBSD, DragonFly BSD, PC-BSD and Haiku (hey, even ReactOS!): A guide”

  1. Hey, wanted to share my results.

    01:19:38.347 | wbloop | 1/4/8 | Intel i7-4790K 4.00GHz @ 4.20GHz | 8GB DDR-III/1333 CL9 | AsRock Fatal1ty Z97 Killer | Windows 10 Pro x64

    • Good morning wbloop,

      Thanks for your contribution, your result has been [added]! :) Hmm, strange, where are you guys coming from all of a sudden? Has this post been linked somewhere?! Previously, nobody had shown any interest in submitting here.

      It’s not really that big a benchmark project anyway.

  2. Hello,
    just wanted to share my results

    0000:55:55.510 | Mazzle | 1/6/12 | Intel i7-6800k 3.40GHz @ 4.00GHz | 32GB DDR-IV/2666 CL16 | ASRock X99 Taichi | Windows 10 Educational x64

    • Short update:
      Did a second run this time with the Official r2705 Build

      0000:29:51.423 | Mazzle | 1/6/12 | Intel i7-6800k 3.40GHz @ 4.00GHz | 32GB DDR-IV/2666 CL16 | ASRock X99 Taichi | Windows 10 Educational x64 (Official r2705 Build)

  3. Hey,

    i wanted to share my result.

    01:01:47.604 | TheDwoon | 1/6/12 | Intel i7-6800k 3.4GHz | 32 GB DDR4 2400MHz CL15 | Asus X99 AII | Windows 7 Pro

    (CPU at stock speed)

    Greetings,

    TheDwoon

    • Hello Daniel,

      Nice to hear from you again! :) I’ve added your result to the [list]!

      Ah, and I have to apologize in case the list (and this blog) go offline, it’s because spambots have been flooding my server occasionally in the past year, and sometimes this would result in a (likely unintentional) DDoS.

      My ancient server just can’t keep up with a few dozen simultaneous attacks on the blog. ;)

  4. Hi, I wanted to share my result. Cheers

    0002:12:50.387 | corrado113 | 1/4/8 | Intel Core i7-3637QM 2.20GHz | 8GB DDR-III/1600 CL9 | HP Pavilion G6-2333sl | Windows 10 Home x64

    • Hello Corrado,

      Thank you for your contribution! :) I’ve added your result to the [list]! Interesting to see that people from the “outside” would be interested in contributing here and there, since I built this for just one forum community. It’s still rare to see people from somewhere else try this benchmark out, but it’s definitely welcome!

      Ah, you were the first with an i7-3637QM as well.

      Well, have a nice evening! :)

  5. Hey,
    I am following your guide on Scientific Linux 6.x and it looks like libav compiles fine. But when I try to compile the x264 part, I get a lot of undefined references. Some of them are in avresample. If I disable avresample, then I get errors of undefined reference to compress/uncompress.

    I have uninstalled all versions of x264-libs and ffmpeg-libs as well.

    Any help would be appreciated.

    • Hello Narayan,

      I have never tried to compile on Scientific Linux 6, but it being a derivative of RHEL 6 just as much as CentOS (with wich I have lots of experience), it should be possible to get it done. The most important part is that I can look at your config.log file and get the exact error message with which your build fails during make. This may help a lot in get it solved.

      You may also consider trying older versions of the software. I have encountered problems linking x264 against libav several times, because they do not always play well together. Typically, the libav developers change some stuff, then the x264 link fails. After some time, x264 will catch up, and it’s fine again. Reverting to older versions *might* solve the issue too. You should just pay attention to pick versions from rougly the same timeframe, or failure is assured.

      All libav releases can be obtained [here], x264 releases [here]. Try a few combinations (going back 1 year or 2), maybe it’ll solve the whole issue right away. Otherwise, please provide as much information as possible so we can figure out what exactly is going wrong. That would be config.log in any case and the error output of make in case in fails in the compilation stage and not during configuration. If you provide any such information, please do not post it directly if it’s extensive, as this might be too much of a “Wall of Text” here. If it’s more than say 50 lines, Please put the logs in a file, and just link it off-site. Thanks!

      Monday morning around 08:00AM UTC+1 I should be able to try a build of the current versions of libav+x264 on CentOS 6.6 to see whether configuring, compiling or linking fails there too.

      If we can’t get it done that way, I’ll fetch myself a Scientific Linux 6 ISO and install it into a virtual machine to replicate your issue. That’d be the last resort though.

    • Hello Narayan,

      Sorry for the delay, but I got caught up in other work.

      Not sure if you’ll still read this, but I can now confirm that x264 does currently not link against libav 11.2, again because of the [opus audio codec], I’ve seen it before. See here:

      gcc -o x264  x264.o input/input.o input/timecode.o input/raw.o input/y4m.o output/raw.o output/matroska.o output/matroska_ebml.o output/flv.o output/flv_bytestream.o filters/filters.o filters/video/video.o filters/video/source.o filters/video/internal.o filters/video/resize.o filters/video/cache.o filters/video/fix_vfr_pts.o filters/video/select_every.o filters/video/crop.o filters/video/depth.o input/avs.o input/thread.o input/lavf.o input/ffms.o libx264.a -ldl -lffms2 -L. -lavformat -lavcodec -lswscale -lavutil -lm -lz -lbz2 -lpthread -lswscale -lavutil  -m64  -lm -lpthread -ldl
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_subpacket':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:377: undefined reference to `avresample_is_open'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_frame':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:221: undefined reference to `avresample_is_open'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_init_resample':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:163: undefined reference to `avresample_open'
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:169: undefined reference to `avresample_convert'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_frame':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:236: undefined reference to `avresample_convert'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_flush_resample':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:118: undefined reference to `avresample_convert'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_subpacket':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:409: undefined reference to `avresample_close'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_flush':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:564: undefined reference to `avresample_close'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_close':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:586: undefined reference to `avresample_free'
      /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.8.5/../../../libavcodec.a(opusdec.o): In function `opus_decode_init':
      /home/myuser/x264src/libav-11.2/libavcodec/opusdec.c:636: undefined reference to `avresample_alloc_context'
      collect2: error: ld returned 1 exit status
      make: *** [x264] Error 1

      Now this was with my custom compiled GCC 4.8.5, but CentOS 6.6s stock GCC 4.4.7 throws just the same error, so it doesn’t really matter and shouldn’t for Scientific Linux either.

      This can be fixed, but you must not disable the entirety of avresample. You can get rid of opus specifically! libav’s configure script has options for listing all encoders, decoders, (de)muxers, etc. Like –list-decoders or –list-muxers. Using that, I’ll just run stuff like $ ./configure --list-decoders | grep -i opus to find the switches. In the current case its two decoders, one encoder and one parser. Let’s rip ’em out, compile and install:

      $ ./configure --disable-decoder=opus,libopus --disable-encoder=opus --disable-parser=opus
      $ make
      $ su
      # make install

      After that, please retry to configure and make x264, it should work now. The resulting libav build will of course lack opus support, but for the x264 binary this is completely irrelevant.

      If you run into a missing libffms2.so.2 file when invoking x264 after that, yeah, it’s another linker issue, that makes x264 not look for the library in /usr/local/lib/ as it should. You can just tell x264 to use libav instead of ffmpegsource to work around the issue until it gets fixed by passing the option --demuxer lavf to x264 on every call, that’s the easiest way.

      Should you want to run the x264 benchmark, just alter the launcher script accordingly, and you’re good to go. It should read something like this then:

      #!/bin/sh
      # Pass 1:
      x264 --preset veryslow --tune film --b-adapt 2 --b-pyramid normal \ 
      -r 3 -f -2:0 --bitrate 10000 --aq-mode 1 -p 1 --slow-firstpass \
      --stats benchmark.stats -t 2 --no-fast-pskip --cqm flat \
      --demuxer lavf elephantsdream_source.264 -o benchmark_1stpass.264
      
      # Pass 2:
      x264 --preset veryslow --tune film --b-adapt 2 --b-pyramid normal \
      -r 3 -f -2:0 --bitrate 10000 --aq-mode 1 -p 2 --stats benchmark.stats \
      -t 2 --no-fast-pskip --cqm flat --demuxer lavf \
      elephantsdream_source.264 -o benchmark_2ndpass.264

      All of that if you really want to run the latest versions together… I guess you could still fetch some source from 2012-2014 or something.

      Hope this helps!

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre lang="" line="" escaped="" cssfile="">

(required)

(required)