Thread bump time!

There really doesn't seem to be much else on the net comparing real-world performance of Handbrake encoding. The few sites that use handbrake often seem to get some really weird results out, with very poor scaling even from i3 to hex i7 for instance, as though they're using really low resolution videos or strange settings. Plus they of course don't disclose their testing methodology or the videos they use, so the results are largely meaningless.

Anandtech did some handbrake tests on their Kaveri review and TBH they seem to largely mirror the 'synthetic' ones (although it still uses a real video and the x264 codec) they normally run, but as expected the very low res chart shows poor core scaling.

The 0.9.5 charts were got were IMO a really useful and interesting comparison, and it would be great to see the same again for 0.9.9 and newer CPUs as well as existing ones for comparison.

I've just been thinking (and typing, but changed my mind) about perhaps modifying some settings from the default presets as there are some options I change on both presets before doing any of my own encoding, but then this is a speed comparison benchmark I guess so I'm not sure how worthwhile that would be, especially since modifying settings could make the benchmarking tedious, and it would be hard to know if they were set correctly. The Normal + High Profile presets we were using probably cover enough combined, as one option I had in mind was the Very Fast x264 preset, used in Normal profile in Handbrake, which performs somewhat differently to the rest of the slower presets, but it's not used in the High Profile Handbrake preset. The decomb filter is on by default on the HP preset OTOH, but again I'm not sure what sort of impact that would have on benchmarks - as long as it's equal between CPUs, it's not really an issue.

However, what about testing scaling performance, e.g. 1080p down to 720p?

I think it's important to keep the number of runs down so the process doesn't look overwhelming though, maybe have one or two 'important' runs, plus other optional ones - from the 0.9.5 chars the three videos seemed to scale between CPUs fairly similarly, so even one filled column would be useful IMHO.

We'd also have to think how to handle QuickSync and OpenCL/UVD. QS results are interesting but in no way comparable to software encoding of course. Off on a tangent, a few places who have compared IVB to Haswell QS using computed PSNR/SSIM comparisons claim Haswell offers better quality, but Anandtech and others who subjectively compare real image quality suggest Haswell falls behind IVB. Maybe Haswell has been unfortunately tuned for PSNR/SSIM numbers rather than PSY quality? The x264 describes the problem with doing this here: http://x264dev.multimedia.cx/archives/458
And the Anandtech article: http://www.anandtech.com/show/7007/i...-perspective/8

As for OpenCL/UVD, the options still don't seem to show up in 0.9.9 for me? However these two options AFAIK don't really impact quality, so are probably more comparable to the pure software encoding.

One suggestion I'd make though - maybe highlight the overclocked results in the charts somehow so it's easier to compare stock CPUs?