Codec Wiki Blog https://av1.wiki/blog Codec Wiki Blog Wed, 02 Jul 2025 00:00:00 GMT https://validator.w3.org/feed/docs/rss2.html https://github.com/jpmonette/feed en <![CDATA[Deep Dive into SVT-AV1's Evolution (Part 2): Encoder Parameters Revisited]]> https://av1.wiki/blog/svt-av1-fourth-deep-dive-p2 https://av1.wiki/blog/svt-av1-fourth-deep-dive-p2 Wed, 02 Jul 2025 00:00:00 GMT Welcome to the second part of my SVT-AV1 testing analysis!

I received lots of kind words, including constructive feedback to improve on my methodology, I sincerely appreciate it! While I couldn’t incorporate those suggestions for this follow-up, stay tuned, because you’ll likely hear more from me before summer ends.

Before we continue, if you haven’t seen Part 1 yet, I recommend giving it a quick look! It covers the test methodology, sample clips, and base encoder settings in detail. The only change to this Part is my decision to remove the full graphs in an attempt to cut down on bloat. If you really want to, you can still access them on the repository where we upload pictures for the blog posts here.

Now, without further ado, let’s dive right back in where we left off!

SVT-AV1 v3.0.x Parameters Revisited​

Today, we are looking at 8 encoder features present in SVT-AV1 v3.0.x! Although we won't be revisiting every encoder parameter like we did back in the first deep dive on v1.8.0, we are going to concentrate on a few important ones, some of them coming straight from the SVT-AV1-PSY project!

Let's start with the feature you are likely most exciting about: varboost!

Variance Boost​

The author of variance boost made a highly visual explanation of the feature in the official SVT-AV1 documentation you can find here. This overview will probably do a better job than me in describing what it does, but in a few words, varboost allocates more bits to low-contrast areas in a frame.

You can enable varboost via the --enable-variance-boost parameter and control its behavior by changing its --variance-boost-strength or its --variance-octile setting. Basically, the strength controls how much areas are to be boosted, while octile controls how much of the area needs to be deemed low-contrast before being boosted.

In every graph of this section, you will notice vbs_o_ in the legend. The vb stands for varboost, the s stands for strength and lastly the o stands for octile. In the last subsection, you will even see vbs_o_c_ with the c standing for curve. To take one example, vbs1o4c2 means varboost at strength 1 and octile 4 on the 2nd curve.

Varboost Strength​

We're going to test all 4 available strengths values, with octile as the variable...

...starting with --variance-boost-strength 1!​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

At HQ, on almost all metrics (excepted VMAF which is more inconsistent here), all octile values are more efficient than varboost disabled. With one notable exception in octile 8 which often scores the same or worse than no varboost depending on the clip. The lower octiles seem to score increasingly better here.

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

At LQ, the results tend to be the same, however there are more outliers and overall the octile value seem to be more negligible. No varboost often scores better than many octile values on VMAF, which is an interesting behavior to observe.

---> Speed graphs:

Preview

In terms of speed, we won't be surprised to learn that the lower octile you go, the slower the encoding instance will be, as the output filesize is bigger and that tends to slow down SVT-AV1. By bitrate normalizing, we would get closer performance between all options.

---> Interpretation:

1 is the most conservative strength value, so we expect reasonable results from varboost here. All octile values consistently provide efficiency improvements except for --variance-octile 8. It is counter-intuitive to see lower octiles score better in efficiency so far, but we'll see if that changes as we increase the strength.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)
Let's continue with --variance-boost-strength 2, the default strength value in every iteration of SVT-AV1:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

The picture is consistent with what we've previously seen, so I won't repeat myself. However, we must keep in mind that the higher strength and the lower octile we go, the more the curve is shifted to the top right, which may not paint the most fair picture of the situation and might skew our interpretation in favor of the one end of the graph or another. That being said, except for VMAF which behaves a bit differently with varboost, we see the default of --variance-boost-strength 2 --variance-octile 6 always increases efficiency over no varboost across basically all clips and metrics, so we can confidently say it does live up to its promise.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)
Now for --variance-boost-strength 3:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

It's more of the same, varboost at strength 3 tends to perform better in efficiency at higher quality targets though there can still be gains at lower qualities depending on the content at hand.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)
And lastly, --variance-boost-strength 4, which will print you a warning that this may be too aggressive in some usecases:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

So, what does this unusual varboost strength have in store for us? Well, nothing as dangerous as we're led to believe. In fact, by in large, we observe the same things as the other three strengths. You may have noticed this yourself, as we've increased the strength, Butteraugli has tended to highlight lower octiles more and more at high quality and no varboost more and more at low quality. Still, varboost on often provides great gains and the optimal octile depends a lot on the sample. The default octile 6 is a fine, safe default.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Varboost Octile​

Now, let's do the contrary and test all 8 octile values while solely adjusting the strength...

...starting with --variance-octile 1!​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

For an octile value that supposedly should boost too many blocks in frames, 1 scores pretty consistently better than no varboost, especially at high quality. What's more, even though it can be argued all varboost curves are extremely close to one another, higher strengths are ever so slightly better performing than lower ones at high quality, while the contrary is more often observed at low quality though not always! Fallout seems to benefit less from varboost, heck it can be harmful at low quality according to some metrics. You will be unsurprised to hear that the higher the strength the slower encoding tends to be. Again, speed is affected by the resulting output filesize.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
Let's look at --variance-octile 2:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Octile 2's results are similar to 1, that is to say close curves between all strengths, with a tendency for higher strengths to be better at high quality, for lower strengths to be at times preferable at low quality, and for varboost to overall be beneficial over it disabled, except according to VMAF.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
Let's test --variance-octile 3:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

It can be summarized the same as octile 1 and 2.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
And --variance-octile 4:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

I swear it's not just me getting lazy, there's not much more to say than as already been said!

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
And --variance-octile 5!​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Cf. previous interpretations.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
What about --variance-octile 6, the default octile value in every iteration of SVT-AV1:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

The default octile value tends, again, to perform better on higher strengths at high quality, and sometimes lower strengths at low quality. We notice it is getting easier to distinct between strengths as we've increased octile, which can be good depending on how you see things. Something we haven't discussed until now is how higher strengths tend to be more inconsistent according to SSIMU2 as you get closer to the CRF40-50 range, which could be problematic depending on your usecase, and thus something to keep in mind.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
Following with --variance-octile 7 tests:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Cf. previous interpretations.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)
And lastly --variance-octile 8 testing:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Well, here we are, octile 8 barely provides any gains and can be (quite) harmful depending on the clip and metric, especially at low qualities, so this can hardly ever be recommended.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
The Mandalorian (HQ)The Mandalorian (LQ)

Varboost Curve​

We will be comparing all 3 available varboost curves...

...with the default settings first:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

We observe that all three curves tend to be blurred together, with the notable exception of curve 2 scoring non-negligibly better on a few clips, consistently across all metrics. Speed-wise, I would argue it's a tie as all three curves are sometimes situationally slightly slower or faster.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)
I also conducted the tests with a different combination of varboost settings I have used on occasion:​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

The winner is less clear here, this time we can say it's a tie. Such results are why testing with a different combination of settings can be educative.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Varboost Conclusion​

We have seen that, in SVT-AV1 v3.0.x, varboost on more often than not provides consistent improvements whatever the combination of settings, though we see better gains at high qualities and/or with higher strengths and lower octiles, less gains at low qualities and/or with lower strengths and higher octiles.

It is best to stay on the default --variance-boost-strength 2 --variance-octile 6 unless you want to hyper-tune for your content, or you want to encode at lower than CRF20 where I guess a combination like --variance-boost-strength 3 --variance-octile 3 can provide more consistent efficiency gains.

--variance-boost-strength 4 is still clearly better than no varboost and sometimes lower strengths too, but it often performs worse than 3. --variance-octile 1 & 2 often came out on top, but they may be too aggressive or inflate filesizes too much, so a more conservative value is advised.

My last recommendation would be to remember to adjust CRF in consequence, as enabling varboost, increasing the strength and decreasing octile can drastically boost filesizes. If you understand how these graphs work, then you know you aren't compromising anything by increasing CRF to compensate for the filesize increase.

--tune​

The original SVT-AV1 implementation offers three tunes to choose from:

  • --tune 0 (VQ - Visual Quality): Favors sharper decisions, reducing blur but potentially introducing artifacts.
  • --tune 1 (PSNR - Peak Signal-to-Noise Ratio): The default in mainline SVT-AV1.
  • --tune 2 (SSIM - Structural Similarity Index Measure): The previous efficiency champion.

Has the dynamics between tunes changed since we last tested them on SSIMULACRA2 almost a year and a half ago? Let's see!

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

In most cases, on most metrics, tune 1 comes out on top in efficiency, with tune 2 sometimes trading blows and even winning. Tune 0 is usually quite a bit behind, except on XPSNR. In terms of speed, all tunes seem to perform closely from one another, without a clear winner.

For most users, sticking with the default (tune 1) is recommended, as it provides the best balance of everything. However, tune 0 can be worth experimenting with if you prefer sharper outputs. Just be aware of the potential trade-offs in artifacting.

Specialized Tunes in SVT-AV1 Forks​

The SVT-AV1-PSY based forks introduced new tunes to catter to new usecases. We won't be testing these forks today but I'm mentioning them for reference purposes. Note that the following two --tune 3 modes are entirely different depending on the encoder variant!

  • --tune 3 (SVT-AV1-PSY(EX) exclusive): A psychovisual enhancement of --tune 2, borrowing some features from --tune 0 and other tweaks. A general-purpose psychovisual tune for a wide range of content.
  • --tune 3 (SVT-AV1-HDR): Acts as a grain-optimized mode, disabling CDEF, restoration, and temporal filtering while applying aggressive psychovisual adjustments. Best suited for noisy live-action content.

We may want to confirm our findings with a visual analysis on the encoded samples:

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Lastly, my own opinion regarding the tunes is that it probably does not matter as much as you think. Again, unless you're willing to hyper-tune, which implies testing parameters each time you encode a new source, no one can guess in advance what's going to be best for that content because it depends on an almost infinite amount of variables. The thing with psychovisual approaches is that metrics may be saying something and your eyes something else. Plus it is wildly subjective: my eyes may disagree with yours, or anyone else's. Don't fret too much over the tune.

Let's continue with the newly introduced parameters of v3.0.0, starting out with an important feature in the context of AV1: luma bias!

--luminance-qp-bias​

Anyone familiar with AV1 for long enough is aware that its encoders have struggled on dark scenes forever. As the encoder implementations mature, performance in such scenes naturally improve, however in many cases they persist in allocating insufficient bits to these darker scenes. This setting changes (almost) everything! It effectively applies a dumb qp offset to frames of lower overall brightness. The higher the value, the stronger the effect is. This implementation has one advantage and one weakness: it gives the user control over the bitrate balancing between bright and dark frames, however if only parts of the frame are dark and the rest is fairly bright, it may not fix cases of localized detail loss or blurring.

So let's see the effect it can have on efficiency and visuals!

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

On the HQ graphs, we can see no luma-bias tends to score last or at least lower than conservative luma-bias values. On the LQ graphs, we usually observe the opposite, that is to say that no luma-bias tends to barely come out on top, with increasing luma-bias values decreasing efficiency ever so slightly.

If we look at performance, the impact of luma-bias is negligible at lower CRF values, but gradually increases with higher CRF values for the simple reason that the frames whose QP will be reduced will take a bit more time to encode, as is the expected behavior of any encoder. The effect on QP, and thus performance, is going to be more important the higher base CRF you start with.

Let's look at some BD-rate numbers directly, to try and clarify the graphs:

Luma-Bias EffectLB0LB10LB20LB30LB40LB50LB60LB70LB80LB90LB100
Avatar0%+0.18%+0.59%+0.74%+0.89%+1.11%+1.28%+1.45%+1.63%+1.73%+1.94%
Ducks0%+1.33%+2.34%+3.04%+3.76%+4.43%+5.02%+5.74%+6.29%+6.82%+7.43%
Fallout0%-0.31%+0.79%+1.29%+1.84%+2.56%+2.72%+3.04%+3.40%+3.73%+4.06%
Minecraft0%+0.69%+1.43%+1.99%+2.64%+3.10%+3.47%+3.87%+4.26%+4.59%+5.06%
Sol Levante0%-0.09%-0.12%-0.02%+0.21%+0.29%+0.44%+0.42%+0.53%+0.60%+0.75%
Suzume0%-0.45%-0.43%-0.34%-0.31%-0.30%-0.21%-0.11%-0.07%+0.02%+0.08%
Mandalorian0%+0.57%+0.91%+1.29%+1.56%+1.80%+2.24%+2.30%+2.53%+2.78%+3.03%

Unfortunately, I couldn't easily separate the BD-rate numbers at low quality and the ones at high quality, so we can't exactly reproduce what I interpreted from the graphs earlier. Still, this gives insightful data, which showcase that the BD-rate across the entire quality range is improved the most on the anime clips and tends to be harmful on live action and gaming clips, again across the entire quality range, not when isolating a smaller range where we could make luma-bias look way more appealing.

Luma-bias is no magic, it simply offsets a frame QP depending on its average luminance. So unless rate control is badly tuned, the feature is not expected in theory to bring any efficiency improvements.

My advice for choosing a luma-bias value is to simply isolate a test sample in your source and test out a few values (like 10, 30, 50...), look at the bitrate balance between dark and bright frames by checking the impact on visuals and filesizes, and finally decide what you prefer. On that note, you cannot go wrong by using a conservative value below 50 in my opinion.

Beware that the author of the feature warned me it isn't suited for the PQ transfer used in most UHD blu-rays! Use luma-bias exclusively on SDR and HDR HLG videos.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

--sharpness​

Sharpness is a straighforward parameter, though it may not do exactly what you'd assume it to do. It does not affect the encodes' clarity but rather impacts the deblocking filter sharpness, which can lead to increased fidelity.

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Like luma-bias, this feature can only be useful if the deblocking filter isn't properly tuned for all usecases to begin with. On surface, it looks like there is not much room for improvements. It does depend on the metric and clip, but in many of these, all sharpness values perform extremely closely from each another, apart from 3 (a bit lower or higher) and 4+ (usually noticeably lower).

Speed wise, negative sharpness values perform closely to the default of 0, and increasing positive values become slower and slower, but as we can see on the x axis, it's rather negligible.

Again, there are so many data points that a BD-rate table will help visualize stuff differently:

Sharpness Effect-7-6-5-4-3-2-10+1+2+3+4+5+6+7
Avatar0%-0.04%-0.05%-0.04%-0.16%-0.04%-0.03%-0.02%-0.03%-0.01%+1.07%+3.44%+6.05%+8.28%+9.65%
Ducks0%-0.05%-0.09%-0.10%-0.14%-0.10%-0.09%-0.06%+0.04%+0.06%+2.14%+8.94%+14.24%+18.19%+20.74%
Fallout0%-0.05%-0.07%-0.13%-0.16%-0.15%-0.13%-0.13%+0.01%+0.01%+0.99%+5.62%+10.23%+13.68%+16.00%
Minecraft0%-0.04%-0.08%-0.12%-0.17%-0.11%-0.08%-0.01%+0.21%+0.26%-1.14%+0.88%+3.88%+5.60%+6.83%
Sol Levante0%-0.03%-0.07%-0.10%-0.02%-0.01%-0.17%-0.06%-0.05%+0.04%-3.61%-3.61%-2.65%-1.89%-1.53%
Suzume0%-0.03%-0.07%-0.15%-0.13%-0.17%-0.20%-0.13%-0.75%-0.77%-0.46%+2.73%+5.87%+7.89%+8.91%
Mandalorian0%-0.02%-0.04%-0.01%-0.17%-0.13%-0.06%-0.20%-0.95%-1.07%-1.86%-0.49%+1.09%+2.16%+2.65%

The same disclaimer is applicable: BD-rate numbers across such a wide quality range are bound to not be representative of smaller ranges, and doing such average across all 4 metrics could reduce the impact of outliers and make the situation look better than it actually is. Which is why this information is important to pair with the individual graphs, and the visual comparisons.

Anyway, on average we can see in the table above that negative sharpness values (-7 to -1) generally show minor BD-rate improvements or neutral effects, while positive sharpness values (+1 to +7) can show decent, consistent BD-rate improvements or neutral effects with 1 & 2, and either bigger gains or significant degradations with 3 and above.

We notice that "Sol Levante" loves sharpness (any value really), whereas the gaming clips and "ducks take off" rather dislike it.

So, what effect does sharpness have on visuals:

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

--tf-strength​

Temporal filtering in SVT-AV1 combines information from multiple nearby video frames to create cleaner reference pictures with reduced noise, which helps improve compression quality especially for noisy source material.

The feature was often considered too strong and often created unavoidable blocking on keyframes, so we historically disabled temporal filtering. Thankfully, a strength parameter has been introduced which has allowed to tame its effects.

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

On this corpus of clips, we instantly notice tf-strength 4 performs very poorly and should probably never be used. Lower strengths than the default 3 tend to score increasingly better, though we seem to hit a ceiling below 2. No temporal filtering is competitive with low tf-strengths, though it still gets beaten slightly at times. There is an interesting outlier in Minecraft where low strengths and especially no tf perform noticeably worse for some reason. The difference in speed between all options is overall negligible.

From this, it is recommended at a minimum to reduce --tf-strength from its default 3 to 1, or below, to at least completely eliminate the tf blocking issue.

Additional Parameter in SVT-AV1 Forks​

The SVT-AV1-PSY based forks include an additional --kf-tf-strength parameter which decouples tf strength on keyframes, and allows the user to concurrently fix the blocking issue and use a stronger tf strength on all other frames if they want. In mainline SVT-AV1, tf-strength is the same between keyframes and other frames, unless you use tune 0 in which case tf-strength on keyframes will be 1 value lower than on other frames (so for example 1 on keyframes if --tune 0 --tf-strength 2 is set, the strength being capped at a minimum of 0 in any case).

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

--lossless​

SVT-AV1 finally added a lossless mode in v3.0.0! Until then, aomenc was the only AV1 software encoder capable of doing lossless encoding, as both rav1e and SVT-AV1 wouldn't allow you to set Q0/CRF0. It probably shouldn't be expected to see better lossless compression out of SVT-AV1, rather, the encoder aims to achieve feature parity with aomenc, but can it do it fast?

I initially intended this section to compare aomenc to SVT-AV1, but complications quickly arose from my testing as you'll soon be able to see. Obviously, there is no question of efficiency graphs here as the metrics are expected to always be maxed out with lossless enabled. Anyway, no graphs will be needed here.

In fact, SVT-AV1 lossless mode isn't actually mathematically lossless. Rather, the feature was designed to reach a PSNR of 100 more often than not (not inf!), as we can see in the Merge Request that introduced the feature.

Running --preset 4 --lossless 1 proves it on my test samples:

"Lossless" TestPSNR (average)PSNR (min)PSNR (max)
Avatar96.61600686.680008inf
Ducksinfinfinf
Fallout104.37840292.930594inf
Minecraft96.26305287.993916inf
Sol Levante119.890723106.868980inf
Suzume116.226350102.759654inf
Mandalorian102.54689391.381363inf

XPSNR and W-VMAF are not included here as the numbers were nonsensical due to the chroma weighting at play. As for Butteraugli's different intensity multiplier and SSIMULACRA2's harmonic scoring, they are not expected to affect the score of a lossless frame, however the GPU implementation itself isn't 100% score accurate to their original's so they can't be considered reliable for this specific usecase. Therefore, regular ffmpeg's PSNR filter was used here.

Sure, the scores are high, but not lossless-like! Funnily enough, the anime clips' average score a bit higher. That makes SVT-AV1 more lossless on anime than on live action! Is this confirmation that SVT-AV1 is a weeb encoder? You tell me. Anyway, for some reason, "ducks take off" is the only clip that is properly lossless out of SVT-AV1 here.

What's even more concerning is that even though the outputs are not mathematically lossless, the filesizes are often bigger than libx264 -preset veryfast -qp 0, the latter of which is properly lossless:

"Lossless" Testx264 FilesizeSVT-AV1 Filesize
Avatar282,198,409 o280,441,784 o (-0.6%)
Ducks244,077,992 o234,884,891 o (-3.8%)
Fallout272,838,887 o278,457,794 o (+2.1%)
Minecraft382,935,638 o473,884,851 o (+23.8%)
Sol Levante438,207,769 o448,937,567 o (+2.4%)
Suzume372,844,589 o384,223,861 o (+3.1%)
Mandalorian182,526,455 o185,680,771 o (+1.7%)

Needless to say the AV1 encodes took at least 10x longer to encode as well!

Considering that "Ducks" is mathematically lossless and a few percents smaller, it shows that the format is in fact capable of compression gains in this department. Simply, the current iteration is either broken or misleading in its true intentions.

While the encodes are not mathematically lossless, I doubt anyone can see any difference with our bare eyes, but I still made comps so you can see for yourself.

---> Visual Comparisons:

--fast-decode​

SVT-AV1 ships with its own built-in method for reducing decoding bottlenecks by smartly tuning down or disabling specific internal tools that trade off some efficiency for decoding performance.

The encoder offers two --fast-decode levels, with 2 being more aggressive. The default is 0.

Three presets have been tested with the feature to quantify what the effects would be for different usecases. Due to some hardware-related issues, all the encodes in this section were re-run, so the speed numbers of disabled fast-decode encodes are different from before, however this ensured there would be no impact on my subsequent analysis.

Preset 2​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

The efficiency impact of fast-decode is clearly visible, be it at low or high quality targets. Beware of the much different scales on the y-axix between the HQ and LQ graphs, which could mislead you into thinking the effect at low quality is lesser, even though that's not the case.

We observe that fast-decode can non-negligibly influence the speed of your encoding instances, with 1 appearing slower than 0 and 2 being faster than both. There's an exception in "Sol Levante" where 0 and 1 perform the same.

Let's look at the effects on decoding speed with data I nicely aggregated into tables!

Decoding speeds were collected using ffmpeg 7.1.0 from Arch's official repository using: ffmpeg -hide_banner -benchmark -i "" -f null - > /dev/null. The tests were repeated 5 times and the performance numbers you'll see are the average of these 5 runs. I can say with confidence the speed deviation was largely negligible, but better safe than sorry!

Preset 2 Decoding Test (CRF12)FD0FD1FD2
Avatar197fps204fps (+3.6%)212fps (+7.6%)
Ducks118fps126fps (+6.8%)125fps (+5.9%)
Fallout65fps67fps (+3.1%)68fps (+4.6%)
Minecraft82fps85fps (+3.7%)91fps (+11.0%)
Sol Levante98fps105fps (+7.1%)107fps (+9.2%)
Suzume360fps401fps (+11.4%)397fps (+10.3%)
Mandalorian326fps348fps (+6.7%)340fps (+4.3%)

In our high quality target, --fast-decode 1 provides a +6% decoding performance increase on average, while --fast-decode 2 provides +7.5%.

Preset 2 Decoding Test (CRF33)FD0FD1FD2
Avatar429fps465fps (+8.4%)482fps (+12.4%)
Ducks341fps366fps (+7.3%)342fps (+0.3%)
Fallout131fps140fps (+6.9%)140fps (+6.9%)
Minecraft182fps211fps (+15.9%)217fps (+19.2%)
Sol Levante194fps211fps (+8.8%)210fps (+8.2%)
Suzume684fps766fps (+12.0%)770fps (+12.6%)
Mandalorian885fps912fps (+3.1%)946fps (+6.9%)

At CRF33, --fast-decode 1's decoding speeds were faster of about +8.9% and --fast-decode 2 of about +9.5%.

Sure, --fast-decode 2 doesn't provide substancial decoding benefits over 1, heck even over 0 at times, however it also acts as a "fast-encode" parameter, so it can be argued it has its purpose.

Before we continue with the visual comparisons, I will add that we are not on a low powered device, so the importance of such gains may not be immediately visible nor perfectly accurate to be honest.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Preset 4​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

It can be observed the gaming clips's efficiency is less affected by the fast-decode feature, if at all. Even then, --fast-decode 2 continue to provide encoding performance benefits.

As we move to faster presets, the resulting streams are expected to be less complex and thus easier to decode. Is that assumption correct? And if so, does it imply that fast-decode will have a smaller impact when using preset 4?

Preset 4 Decoding Test (CRF12)FD0FD1FD2
Avatar185fps202fps (+9.2%)203fps (+9.7%)
Ducks117fps126fps (+7.7%)121fps (+3.4%)
Fallout67fps71fps (+6.0%)70fps (+4.5%)
Minecraft87fps95fps (+9.2%)94fps (+8.0%)
Sol Levante91fps100fps (+9.9%)99fps (+8.8%)
Suzume324fps372fps (+14.8%)360fps (+11.1%)
Mandalorian312fps336fps (+7.7%)326fps (+4.5%)

Looking at the FD0 column of the below and above table, compared to the preset 2 ones, my claim seems to barely apply and only consistently at low quality levels at that. We'll see how it goes for preset 6 in the next sub-section.

Anyway, the average decoding speed impact at HQ is as follow:

  • +9.2% for --fast-decode 1,
  • +7.1% for --fast-decode 2.
Preset 4 Decoding Test (CRF33)FD0FD1FD2
Avatar432fps494fps (+14.4%)499fps (+15.5%)
Ducks354fps394fps (+11.3%)375fps (+5.9%)
Fallout143fps156fps (+9.1%)146fps (+2.1%)
Minecraft197fps258fps (+31.0%)254fps (+28.9%)
Sol Levante200fps224fps (+12.0%)217fps (+8.5%)
Suzume696fps811fps (+16.5%)763fps (+9.6%)
Mandalorian902fps981fps (+8.8%)999fps (+10.8%)

--fast-decode 1 gives a hefty +14.7% increase in decoding performance at lower quality targets, as certain clips like "Minecraft" greatly benefit from it. --fast-decode 2 brings a more modest +11.6% improvement.

The trade-offs are quite a bit different than at preset 2 which is quite fascinating, but --preset 4 seems to draw more benefit from fast-decode!

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Preset 6​

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

The efficiency situation here is the same as on preset 4, however we notice --fast-decode 1 is no longer slower than disabled. Instead, it places itself in-between 0 and 2 in terms of performance, offering another kind of trade-off again!

What influence on decoding speeds can we expect at --preset 6?

Preset 6 Decoding Test (CRF12)FD0FD1FD2
Avatar218fps216fps (-0.9%)228fps (+4.6%)
Ducks128fps127fps (-0.8%)130fps (+1.6%)
Fallout76fps74fps (-2.6%)77fps (+1.3%)
Minecraft110fps106fps (-3.6%)112fps (+1.8%)
Sol Levante100fps104fps (+4.0%)107fps (+7.0%)
Suzume382fps381fps (-0.3%)394fps (+3.1%)
Mandalorian343fps343fps (0.0%)357fps (+4.1%)

This time, be it at high or low quality, the base decoding speed without fast-decode is almost always faster.

At CRF12, --fast-decode 1 and --fast-decode 2 respectively deliver a -0.6% decoding speed regression and a +3.4% speed improvement.

Preset 6 Decoding Test (CRF33)FD0FD1FD2
Avatar468fps480fps (+2.6%)508fps (+8.5%)
Ducks358fps364fps (+1.7%)382fps (+6.7%)
Fallout169fps168fps (-0.6%)177fps (+4.7%)
Minecraft278fps303fps (+9.0%)335fps (+20.5%)
Sol Levante209fps215fps (+2.9%)221fps (+5.7%)
Suzume667fps694fps (+4.0%)715fps (+7.2%)
Mandalorian878fps912fps (+3.9%)950fps (+8.2%)

The gains at low qualities are more appealing, at +3.4% for --fast-decode 1 and +8.8% for --fast-decode 2.

--preset 6 benefits less from fast-decode, especially mode 1, probably due to a shift in the decoding bottlenecks past this point.

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Fast Decode Conclusion​

In performance constrained scenarios, like low power ARM devices, fast-decode could come in handy to help smooth out your playback experience, at the possible cost of some efficiency.

Keep in mind it has been observed that the output of fast-decode modes can be more prone to macro-blocking depending on source characteristics, so proceed with caution.

Tiles​

AV1 tiles are a straightforward method of splitting the video frame into independent tiles of equal size to hopefully increase encoding and decoding thread-ability. In SVT-AV1, tiles don't increase encoding speeds but they can help devices (especially low-powered ones) to software decode AV1 more easily. We are going to challenge these claims.

In the following graphs and tables, c refers to --tile-columns and r refers to --tile-rows. For instance, c2r1 can be read as --tile-columns 2 --tile-rows 1.

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Except on "Suzume", and to some extent on "Mandalorian" at low qualities, where we see the higher number of tiles start to impact efficiency, the effect of tiles on efficiency is by in large negligible. It does seem like encoding speeds aren't particularly impacted by tiles in this encoder.

Let's also look at decoding speeds, as we did with fast-decode!

What follows are the relative decoding speed improvements relative to c0r0, starting with tests at CRF12:

Tiles Decoding Test (CRF12)c0r0c1r0c1r1c2r0c2r1
Avatar185fps335fps (+81%)446fps (+141%)477fps (+158%)487fps (+163%)
Ducks117fps224fps (+91%)330fps (+182%)348fps (+197%)410fps (+250%)
Fallout67fps125fps (+87%)186fps (+178%)202fps (+201%)223fps (+233%)
Minecraft87fps164fps (+89%)223fps (+156%)256fps (+194%)267fps (+207%)
Sol Levante91fps162fps (+78%)205fps (+125%)219fps (+141%)217fps (+138%)
Suzume324fps544fps (+68%)679fps (+110%)732fps (+126%)729fps (+125%)
Mandalorian312fps520fps (+67%)631fps (+102%)681fps (+118%)688fps (+121%)

The same tests were conducted at CRF33 to quantify the differences in impact between two very distinctive quality targets:

Tiles Decoding Test (CRF33)c0r0c1r0c1r1c2r0c2r1
Avatar432fps698fps (+62%)780fps (+81%)816fps (+89%)758fps (+75%)
Ducks354fps650fps (+84%)845fps (+139%)860fps (+143%)920fps (+160%)
Fallout143fps259fps (+81%)355fps (+148%)375fps (+162%)386fps (+170%)
Minecraft197fps346fps (+76%)432fps (+119%)489fps (+148%)481fps (+144%)
Sol Levante200fps295fps (+48%)285fps (+43%)312fps (+56%)292fps (+46%)
Suzume696fps991fps (+42%)1020fps (+47%)1082fps (+55%)1008fps (+45%)
Mandalorian902fps1183fps (+31%)1164fps (+29%)1207fps (+34%)1128fps (+25%)

No, you aren't dreaming! We immediately realize that the impact of tiles is significantly greater than either of the fast-decode modes, an especially appealing outcome given their relatively minor effect on compression efficiency. Decoding speeds improve even further at higher quality settings, with the smallest gain reaching +67% at CRF12. That kind of difference can turn an unplayable file into one that runs smoothly. A single additional tile can have important consequences.

Tile-rows on average offer less benefit than tile-columns, and in some cases even cause slight regressions at lower qualities. Still, seeing up to +250% improvements in decoding speed is nothing to scoff at. Dav1d’s performance is genuinely impressive, now exceeding 1000fps on certain clips!

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

If I had to give recommendations based on what we saw, for a good balance between losses in efficiency and increased decoding performance, consider the following tile settings:

  • --tile-columns 1 --tile-rows 0: for 1080p and above
  • --tile-columns 2 --tile-rows 0: for 4K and above

Of course, if decoding speed isn’t a concern at all, you can stick with the default --tile-columns 0 --tile-rows 0. But even then, I believe enabling tiles is worth considering for future-proofing purposes.

One can combine fast-decode and tiles to decrease decoding complexity further.

Statistics​

Who doesn't love a bunch of random and useless stats? Well, if you recognize yourself in this, I got you. I compiled a few, though I only included stuff that ended up being used in these two parts. Yes, test encodes, test graphs or test comps figures are not included here.

Here's this deep dive in numbers:

  • 10577 total AV1 encodes...
  • ...which account for a total size of 294GB...
  • ...which makes for an average of about 28MB per file
  • 3928 total graphs
  • 2030 total decoding runs
  • 483 slow.pics comps...
  • ...for a grand total of about 12000 distinct screenshots uploaded!

Even I was flabbergasted when I discovered the actual scope of this benchmarking session!

Conclusion​

In this second part, we took a deep dive into several key SVT-AV1 encoding parameters, re-evaluating them in today's context. The goal was mainly to explore how these settings impact efficiency and encoding speeds, but at times also decoding performance or practical usability. While I could have revisited even more parameters, the sheer time investment required means I had to draw the line somewhere. I made sure to go really in-depth with the ones that mattered most in my opinion.

As always, your mileage may vary regarding any kind of speed figures. The key is to test things yourself, with your own clips, workflows, and goals in mind. What works for one setup might not suit another, and that’s what makes this kind of testing both challenging... and somewhat rewarding too.

This wraps up Part 2, but there’s more to come. I'm always thinking about what to explore next, and your feedback helps shape that direction. So if you have suggestions, requests, or thoughts to share, I’d love to hear them, in the usual places.

Thanks again for reading, and I hope you found this deep dive insightful!

Future​

First of all, I do not consider this testing complete. I have been told some comps had issues, and while re-generating them, I ended up rate-limited by slow.pics again, though only temporarily fortunately. As you can imagine, even with scripts, preparing and uploading these takes quite a while already, but I'm taking even more precautions to avoid getting banned again. I decided not to delay this Part further, so you wouldn't have to wait more, but I'll ask you to be patient a bit longer for me to add the fixed comps for Part 1 and the varboost octiles comps for this Part. I expect to be able to update both articles before the end of the week.

EDIT (July 4): These concerns have been addressed. Thanks for your patience.

As I alluded to in the last section, I have ideas for future blog posts. First of all, SVT-AV1 v3.1.0 is right around the corner, and it is touted to be quite an update especially for VBR and realtime usecases! It could be the occasion for me to do the long awaited target bitrate tests and compare efficiency with CRF on our test samples...

Second, most of the ideas I proposed in past articles remain valid possibilities and I still want to test other AV1 encoders in this format. AV2 is approaching too so it would be interesting to make some early comparisons of AV1 with AVM, if there's enough time for that.

Third, there's still so much that could be improved in my methodology, to make it more robust and precise, so I will continue experimenting as usual to find a better overall formula!

Lastly, I’ve also been working on a few AV1-related projects behind the scenes, including one that involves SVT-AV1 directly... I’m looking forward to sharing more about it in the future, so stay tuned!

I want to extend my thanks a second time to the people who contributed, directly or indirectly, to the making of this article, including Gianni Rosato (gb82), Line (Lumen), Soda, Emre, Bolu, Julio Barba, the people behind slow.pics for hosting thousands of screenshots each time I make these articles, the SVT-AV1 dev team for the work on this amazing encoder, and probably others I'm forgetting...

Consider supporting me by making a donation on my Ko-Fi page.

]]> video compression benchmarks <![CDATA[Deep Dive into SVT-AV1's Evolution (Part 1): Presets Analysis from v2.0 to v3.0]]> https://av1.wiki/blog/svt-av1-fourth-deep-dive-p1 https://av1.wiki/blog/svt-av1-fourth-deep-dive-p1 Fri, 20 Jun 2025 00:00:00 GMT It's been almost a year since SVT-AV1 v2.0.0 dropped in March 2024, and we finally got v3.0.0 in late February of this year. Minor versions v3.0.1 and v3.0.2 came along afterward with some bug fixes and ARM SIMD improvements, but they didn't meaningfully alter encoding results.

So what's actually different between these versions? I've been wanting to run tests across all the major releases from v2.0.0 to v3.0.x to see how the speed vs quality trade-offs have evolved this past year. Using SSIMULACRA2, Butteraugli, XPSNR, and VMAF (plus some methodology tweaks I'll get into), I'll break down what each version brought to the table and in a second part, we'll also deep dive a few specific options that appeared in the encoder since my first blog post release so you can figure out what you may want to use for your projects. That includes variance boost, fast decode, temporal filtering strength and a few others...

Feedback​

Although the reception was warm, I got less feedback than usual this time around, but honestly, that hasn't prevented me from wanting more. And better! I'm keeping the stuff that seemed well-received: the diverse test clips, visual comparisons, and my ongoing attempt to stay as objective as possible.

In my last post's conclusion, I mentioned some frustrations: XPSNR was acting up, metrics were taking forever to calculate, and there was just too much data to make sense of. Good news is I've tackled all of these. I fixed the XPSNR issue (it expected mod-8 inputs), started using the new Vship to speed up metrics calculation, and found a cleaner way to present all the data points at just two quality levels, like I used to do.

Also, it needs to be pointed out I've been way too optimistic about how often I can get these posts out. Especially when I revamp the methodology each time. I'm sorry about that, again. Though, my ambition may also be at fault: the size of this article got so out of hand I eventually decided to split it into two parts for convenience's sake.

Methodology​

You'll find both graphs and visual comparisons in this analysis. The graphs give you objective data on encoder efficiency across different settings, using various metrics. The image comparisons show the actual encoded samples so you can judge quality for yourself, adding that subjective element that numbers alone can't capture.

This time, I'm using a new tool called Metrics from the self-proclaimed Psychovisual Experts group, which provides scripts for measuring and comparing video codecs. I've heavily modified these scripts for my specific needs, but if you want to run your own tests locally, definitely check out the original Metrics toolkit!

Here's how the testing works: I use relatively short video samples covering a wide range of content types, all converted to uncompressed y4m format (if they weren't already!) for consistency. These lossless files go straight into SvtAv1EncApp, so we're measuring single-instance encoder performance here. Keep in mind that serious AV1 encoding pipelines should probably use chunked encoding (with a tool like Av1an), especially on higher core count systems.

Once encoding is done, we run multiple full-reference metrics comparing the encodes against the original source. Using several different metrics helps compensate for each one's weaknesses and gives a more complete picture of the actual visual differences between encodes. I try to stray away from the industry standard metrics which tend to have poor correlation to Mean Opinion Scores (MOS), and instead focus more on psychovisual metrics that better represent actual visual quality. I calculate SSIMULACRA2 and Butteraugli scores using Vship, an accurate GPU-accelerated port of the Zig implementation I used before that's also much faster. XPSNR and VMAF scores come from their respective ffmpeg filters, but with a twist!

If you remember, last time I started making use of Harmonic Mean for SSIMU2 to better account for inconsistent scoring behaviors within a video. We'll be doing it again today.

This time sees the addition of the (in)famous VMAF metric, but in a radically different form than you've probably ever seen until now. Indeed, it is well known VMAF is rather unreliable, but with a few modifications to the scoring method, we can try to make it better. I'm computing scores across all three color planes (instead of just luma as VMAF isn't chroma aware), then weighting them with this formula: ((4.0 * vmaf_y) + vmaf_u + vmaf_v) / 6.0. This approach is inspired by the better-vmaf mod. I also chose to use VMAF's neg model and to disable the motion component (motion.motion_force_zero=true) since it's notorious in our niche encoding communities for inflating scores during motion and producing nonsensical results (bad looking frames having near perfect scores...). I'll refer to this metric as W-VMAF in the rest of this post.

XPSNR gets similar treatment: motion component disabled (by commenting out this line) and the same weighting formula: ((4.0 * xpsnr_mse_y) + xpsnr_mse_u + xpsnr_mse_v) / 6.0. As you may glimpse from said formula, XPSNR's dB values are converted back to linear MSE, we then calculate the weighted average and convert back to dB. I'll refer to this metric as W-XPSNR in the rest of this post.

Butteraugli runs in stock configuration except for the intensity multiplier set to 203 nits instead of the currently default 80 (based on an industry reference, used by MPV for instance).

I believe these modifications produce more meaningful results than stock metrics, but proving that is beyond this post's scope. You'll have to trust the methodology or test it yourself. All this data gets aggregated to create the benchmark graphs.

The "speed graphs" plot Constant Rate Factor (CRF) against encoding time to show speed efficiency at different quality targets. For compression efficiency, I plot metric scores against output file size. Since Metrics doesn't use bits per pixel (BPP) and I had received feedback a few blog posts back that this metric could be confusing, I decided to drop it this time around.

To achieve more accurate efficiency curves with fewer probes, it is more effective to prioritize probes in the higher-quality (lower CRF) range of the quality spectrum, since bitrate will increase quicker. I used the following formula, courtesy of better-vmaf's author, to determine ten CRF values from 10 to 50 to use for the testing: crfs = [min_q + (max_q - min_q) * ((step / (q_steps - 1)) ** scaling_factor) for step in range(q_steps)], then equally split into High Quality and Low Quality graphs and visual comparisons.

How to read the graphs? For the compression efficiency graphs, higher and further left is better, except for Butteraugli, which is a distance metric where lower scores mean better quality, so you want bottom-left instead of top-left. For the encoding speed graphs, further left means faster.

One important caveat about speed measurements: while the efficiency results should be reproducible regardless of your hardware, measuring encoding speed is trickier. The performance numbers you see here will likely differ on your setup depending on your specific hardware configuration. Please take them with a grain of salt.

The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy.

As the testing started a good while ago, about when v3.0.1 released in fact, and many encodes were already completed by the time v3.0.2 was out, I used the following encoder versions for this test: v2.0.0, v2.1.2, v2.2.1, v2.3.0 and v3.0.1. The different SvtAv1EncApp binaries were compiled directly from their respective source codes found on the release section of the SVT-AV1 Gitlab repository, using Clang 19.1.7 and the provided Build/linux/build.sh script with the following command: build.sh cc=clang cxx=clang++ jobs=$(nproc) enable-lto static native release. The testing machine is now comprised of an i7 12700F, whose E-cores have been disabled to avoid scheduler-related issues, with 2x8GB of TOTL 3200MHz CL14 DDR4 RAM, in Arch Linux with kernel 6.12.17 and the performance governor enabled. I tried my best to run most encodes in the same session without rebooting, but a few issues made it so I had to re-run some in a new session. However, this does not concern this part of the article, so we'll revisit this in the next one.

There is an exception to one of my above statement. As you may know, there is a feature that significantly impacted SVT-AV1's competitiveness: variance boost! This feature can provide good efficiency improvements by increasing quality in low-contrast areas in frames, at little to no performance cost when properly bitrate normalized. This feature was in the work during the v1.7.0 days, but it only got merged to git relatively shortly after v2.0.0 released. As I wanted to enable varboost by default for this entire testing (as I had expressed in the conclusion of the last blog post), I decided to manually patch the v2.0.0 source code as provided on the release page with the following two commits: "Introduce the variance boost feature" and "Do not adjust picture QP/qindex value with variance boost on". Of course, the feature slightly evolved since then, but this little modification still allowed me to test the evolution of the encoder performance with this key feature on.

We're almost ready to deep dive into an ocean of metrics, graphs, and revelations! (Okay, maybe not revelations... but hopefully a few surprises.)

Samples​

The samples are the same as last time:

  • 17s Avatar The Way Of Water (trailer 3) clip sourced from thedigitaltheater.com (1920x808p with 4 columns and 4 rows of pure black borders to fix XPSNR, 23.976fps).
  • 6s Ducks Take Off clip sourced from xiph.org (1280x720p, 50fps).
  • 3s Fallout 4 clip sourced from another encoder fellow (1920x1080p, 60fps).
  • 8s Minecraft clip sourced from xiph.org (1920x1080p, 60fps).
  • 8s Sol Levante HDR clip sourced from opencontent.netflix.com (3840x2160p, 24fps). This one is pretty educative as SVT-AV1's behavior isn't influenced by the existence (or lack thereof) of HDR metadata in a source.
  • 21s Suzume (trailer 2) clip sourced from thedigitaltheater.com (seems to have been deleted since) (1920x808p, 23.976fps).
  • 13s The Mandalorian (trailer 2) clip sourced from thedigitaltheater.com (1920x800p, 23.976fps).

Visual comparisons​

comp_showcase

Throughout this blog post, you’ll find slow.pics links that provide various visual comparisons between presets.

The "High Quality" (noted HQ) and "Low Quality" (noted LQ) links showcase comparisons at two different quality targets. These have been carefully handcrafted to be as size-normalized as possible, given the available encodes. We want to be focusing on encodes with minimal bitrate deviation for a fair comparison.

Feel free to double-check the bitrate of each frame or scene (as written in the top-left corner of each screenshot) to make a more informed observation, keeping the size difference in mind when comparing the encodes.

Use the arrow keys and numpad to navigate between screenshots. Alternatively, you can click on "Slider comparison" and select two sources if you prefer comparing this way.

Quality Target & Encoding Settings​

All clips have been encoded in a wide quality range, from --crf 10 to --crf 50, with values determined using the previously described formula.

--preset X --enable-variance-boost 1 are the only parameters used here, in conjunction with the CRF values. We will compare various varboost parameters' combinations in Part 2 of the article, but for now, the defaults are used. Obviously, it wouldn't be fun without me realizing, as I'm writing these words, that I forgot to use --hierarchical-levels 4 like last time, but well, it is what it is. Nothing that significant.

Else, the SVT-AV1 defaults were used. The ones worth mentioning are:

  • --tune 1: tune PSNR
  • --aq-mode 2: variance deltaq
  • --enable-qm 0: quantisation matrices disabled
  • --enable-tf 1: temporal filtering enabled
  • --tf-strength 3: default temporal filtering strength
  • --sharpness 0: default deblock and rate distortion mode decision
  • --fast-decode 0: decode optimizations disabled

And more, like CDEF and restoration enabled, overlays and film-grain disabled...

Anyway, time to kick things off with the presets comparisons!

SVT-AV1 v3.0.x Presets Comparisons (-1 -> 10)​

If all you care about is what preset to use in the latest available version, this is the section for you!

In the following graphs, you may find comparisons between all SVT-AV1 v3.0.x presets, ranging from the slowest --preset -1 to the current fastest --preset 10.

Since v2.3.0 (and unlike in v2.1.x and v2.2.x), preset 6 made its return as its own separate preset: it is not mapped to preset 7 anymore. However, the maximum preset has become 10, and anything above it is effectively mapped to 10.

Efficiency​

---> Here are the full efficiency graphs:

Preview

Quite a lot of data eh? So much so that readability is impacted. Hence why we'll also focus on two quality targets to better understand what's exactly going on.

---> Thus, let's look at the same data but zooming in on the "high quality" range (defined here as CRF10 through 23):

Preview

---> And now, let's zoom in on the "low quality" range (defined here as CRF28 through 50):

Preview

As we can see, presets 9 and 10 can be quite unpredictable depending on the clip. Their efficiency curves are also clearly not monotonic, with unexpected efficiency regressions happening at very low CRFs (high bitrates), especially visible on SSIMU2. This behavior is likely enhanced by the harmonic scoring, but it is also observable on Butteraugli and W-VMAF to a lesser extent. When looking at preset 10 in the low quality range, the curves can be so chaotic on the SSIMU2 side we can easily conclude there's a quality consistency issue at play, that is to say that certain frames score significantly lower than others.

---> Therefore, let's remove presets 9 and 10 from the equation and have a better look at the "usable" presets, from 8 and below:

Preview

---> Same, but focusing on the "high quality" range (CRF10 -> 23):

Preview

---> And the "low quality" range (CRF28 -> 50):

Preview

Efficiency wise, we instantly notice presets 2 and below are grouped together very tightly, while every other presets appear to be more evenly spaced. Consistency, and thus SSIMU2 harmonic scores in particular, tend to get really bad below CRF38 or 44 depending on the clip, which at least teaches us that SVT-AV1 begins to struggle around these quality targets whatever the chosen preset (in its almost-stock configuration).

Something interesting seems to be happening with preset 8 where it scores better than preset 7 on certain clips, even going neck and neck with preset 6 at times according to W-VMAF. That is something worth investigating visually, in the following visual comparisons between the three presets:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

Do you agree with W-VMAF's numbers here? I think I do, to an extent.

---> Let's take a closer look at presets 4 and below to see if we can better observe the differences between the slower modes:

Preview

---> At "high quality" (CRF10 -> 23):

Preview

---> And at "low quality" (CRF28 -> 50):

Preview

It is better, but still clear the differences are relatively small. Though, we can also put BD-rate numbers to these differences thanks to psy-ex's metrics!

The following table show the BD-rate regressions of preset 0 through 4, with preset -1 as the reference, when averaged across all four metrics and all seven clips:

PresetBD-rate regression (vs P-1)
00.97%
13.79%
27.82%
317.08%
422.41%

Presets 0 through 2 are pretty close to the reference preset -1 and we notice a huge jump from preset 2 to 3, with preset 4 relatively scoring very close to preset 3.

We will put these numbers into perspective after having looked at the performance of every presets.

Speed​

Efficiency numbers are great, but they only tell half the story. The relative speed differences between presets can paint a drastically different picture of the situation and change our entire interpretation of the results so far.

In the following graphs, you may find speed comparisons of the different presets, with either a linear or a logarithmic scale. The latter is useful to better visualize the large speed variations between presets, as it compresses the scale and makes smaller differences more apparent.

I must reiterate my previous disclaimer that these numbers should be taken with a grain of salt. They only represent the speed of the given presets at each CRF value over a single run on a given machine. Making multiple runs could help eliminate small, undesired variations, but they'd come at the cost of time and electricity. It is not reasonable here considering the scale of this blog post, sacrifices must unfortunately be made. To help mitigate this issue, the processor temperature is closely monitored and the fans actively adjust to prevent overheating. I also make sure no background tasks are running and that only a single encoder instance is running at all times. This won't prevent some outliers from passing through, but they are usually easier to detect.

---> That said, let's begin by comparing the performance of all presets, from -1 to 10:

Preview

First thing first, SVT-AV1 scales exceptionally well, from well above realtime speeds, to painfully placebo speeds. Looking at the graphs with logarithmic scales, we can see that encoding speeds, the slower presets we go, tend to increase in an exponential manner. Unfortunately for us, efficiency doesn't usually follow that trend! We do notice a substancial gap between presets 4 and 5 however.

---> Let's also look at the performance of just presets -1 through 4, considered the slower, non-realtime presets:

Preview

On the logarithmic scale, the presets are impressively evenly spaced, except for preset -1 which is somewhat closer to preset 0.

Now then, we need to combine what we learned about efficiency and speed to interpret the results in an informative way!

Interpretation (TLDR)​

Despite the repeated presets shifting, some things never change.

For good modes, presets 2 and 4 still offer the best bang for your buck in balancing efficiency and speed. Preset 2 is usually slightly more than 2x faster than preset 1 and about 2x slower than preset 3 while offering close to preset 1 efficiency and largely better efficiency than preset 3. Preset 4 happens to be the "slowest" good preset that's still very competitive, with little efficiency differences against preset 3, making it a valuable choice for anyone wanting a good balance of quality and speed for non-realtime usecases.

For realtime modes, any preset from 5 and above will do as long as it reaches realtime on your system! However, consistency takes a huge hit at presets 9 and 10, rendering those two presets unappealing compared to hardware solutions that are likely to be quite competitive for this usecase. Preset 8, which strikes an excellent efficiency to speed ratio for what it is, is the last preset I would deem truly usable in SVT-AV1 v3.0.x.

SVT-AV1 v2.0.0 -> v3.0.x Initial Presets Comparisons (-1 -> 10)​

In this section, we will first examine the presets -1 to 10 range, independently of the presets shifting happening between the tested encoder versions. In a following section, we'll make more targeted comparisons taking into account the shifts.

preset -1: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Efficiency wise, preset -1 has not improved from v2.0.0 to v3.0.x, rather it has stagnated. At worst, it regressed of exactly 2.0% BD-rate on one metric in one given clip. On average, the regression is closer to 0.5%. The good news though is that the preset became up to 65% faster vs v2.0.0! That's the best case scenario, as on average across the studied quality range, it has gotten faster of about 37.5%, which is still an amazing trade-off!

It is fascinating to see how very different the presets' speed behave with CRF between versions, as at CRF50, the preset is barely faster on v3.0.x than it was on v2.0.0.

Now, feel free to look at the following comparisons to see if you can spot much of a difference between the different versions at preset -1!

---> Preset -1 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 0: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Here, we have an interesting case where preset 0 efficiency got slightly better since v2.0.0 (by about 1.75%), though it remained largely unchanged after v2.1.x. There's one funny exception: according to all metrics, it peaked at HQ on the Minecraft clip during v2.1.x and regressed ever since. Speed wise, the preset took a hit in v2.1.x, improved a lot in v2.2.x almost closing the gap with v2.0.0, and regressed again in v2.3.0 and especially in v3.0.x. The regressions are not a good look, even though by in large, they're rather insignificant.

Can you spot the differences in the comparisons below though?

---> Preset 0 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 1: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

We start seeing more diverse results with preset 1. Versions after v2.0.0 tend to fare better efficiency wise, but each version present different strengths and weaknesses depending on the clip and metric. The speed graphs are quite chaotic, though the preset seems to have consistently gotten slightly slower and slower since the v2.0.0 days. Taking all of this into account, the overall interpretation would still be that the preset behavior is mostly unchanged from before.

Let's play at "spot the differences"! After looking at the following comps, tell me what you think of the new trade-offs!

---> Preset 1 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 2: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

On Minecraft, we notice (especially at HQ) a trend of preset 2 progressively becoming worse after v2.1.x, by about 2.5% in fact vs v3.0.x. On other clips, it has either stagnated or slightly improved, except for v3.0.x where it seems to score consistently last or second to last. Relatively, the differences are small, so broadly speaking, the preset is again very similar to what it used to be. Well, except for its consistent speed improvement of 10-40% since v2.0.0, making the new trade-off considerably more appealing.

Does the following comparisons help quantify the speed improvements in the new version? Well, no... they can't, but it's nice to have them anyway! Check them out!

---> Preset 2 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 3: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

At last, efficiency changes that can be considered significant! The presets shifting is at play here, and it will only become more evident as we progress towards the faster presets. With v3.0.x having fewer presets than some earlier versions, the SVT-AV1 team spread them out more to fill the gaps, completing the shifting initiated in v2.3.0.

Preset 3 in v3.0.x almost consistently scores last, though that is less pronounced at Low Quality. On average, its BD-rate is 2.7% lower in v3.0.x vs v2.0.0. However, partly thanks to that, it received a hefty speed boost of about 24% since!

Take a look at the graphs below, does this seem worth it to you?

---> Preset 3 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 4: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Same goes for preset 4! Our slow preset of choice has become 29% faster on average, at the cost of a 3.0% efficiency loss.

With that in mind and the comps available right below, have your opinion of preset 4 changed?

---> Preset 4 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 5: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Unsurprisingly, the spreading continues! The gaming clips are the most affected here, showing an efficiency decrease of 6.1% and 14.3% respectively. Then again, the average across all seven clips is quite a bit lower at 4.5% lower efficiency. Speed is improved by 44%, so the trade-off sounds reasonable.

I'm starting to lose inspiration to introduce the comparisons, so simply have a look!

---> Preset 5 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 6: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

More of the same! Though, the shift seems to have been initiated in v2.1.x, so preset 6 in v3.0.x is quite different from its older sibling from v2.0.0. Compared to v2.0.0, preset 6 is now both 47% faster and 8.8% lower efficiency, while compared to v2.1.2 instead, it is roughly 27% faster and 2.6% lower efficiency. The gaming clips are the most affected again, showcasing that complex content suffers more as you move to higher presets.

Find the visual comparisons right after, if you're interested.

---> Preset 6 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 7: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

With preset 7, it is clear the downward shift in efficiency was progressively set in motion since v2.3.0. Compared to v2.0.0, v3.0.x's preset 7 sees an 11.6% efficiency hit, traded for a 68% speed improvement, not too bad. Versus v2.3.0, the new preset 7 has a 5% worse efficiency and only 14% better speeds. This difference is arguably the least appealing of every presets so far...

How did the preset evolve visually though? The answer is in the following comparisons!

---> Preset 7 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 8: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Preset 8's trade-offs are significantly altered in v3.0.x. This is relatively proportionate to the gain in speed, but we may have expected better. Efficiency wise, it is 15.8% lower and 48% faster than in v2.0.0. To be fair, as the presets are starting to stray too much from each other, we should probably compare it to a faster v2.0.0 preset like 9.

Look up the visual comparisons while keeping that in mind.

---> Preset 8 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 9: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Similar to preset 8, it is becoming really hard to make a fair comparison between very disparate presets, especially when it is starting to break at extreme CRF values. It is clear v3.0.x's preset 9 is built differently than v2.0.0's, it plays in a completely different league. I will not elaborate further at this point, as we will revisit this preset in the next section.

There are still visual comparisons, if you're curious:

---> Preset 9 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

preset 10: v2.0.0 -> v3.0.x​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Unlike the previous two presets, the new preset 10 can somewhat be compared to its previous iterations. After disregarding v2.2.1's which is the one to behave distinctly here, v3.0.x's preset 10 show similar characteristics across the quality range. For what it's worth, the preset is 38.5% less efficient but 39.1% faster compared to v2.0.0. It should be noted that as we go faster, the gap between fast presets is more likely to shrink on shorter clips due to the low encode times (in the order of a few seconds). As the time taken by the encoder to initialize isn't quite negligible anymore, using longer clips would likely increase the lead of preset 10 in v3.0.x vs v2.0.0 as encoding speeds would have gotten enough time to stabilize. This is obviously out of the scope of this blog post, but still, the analysis in the next section will help shed light on how the fastest preset in v3.0.x compares to the fastest one in v2.0.0.

I invite you to take a look at the visual comparisons if you want to see how preset 10 evolved from v2.0.0 to v3.0.x.

---> Preset 10 Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

SVT-AV1 v2.0.0 vs v3.0.x Selective Presets Comparisons​

Some presets cannot be directly compared anymore, as the reduction in presets since v2.0.0 means the gap had to be filled, and the fastest presets were the most affected by this move. I decided to focus on SVT-AV1 v3.0.x's presets 8, 9 and 10 here, then we'll proceed do something new for this blog post series...

v3.0.x's preset 10 vs v2.0.0's preset 13​

How does v3.0.x's fastest preset fares against v2.0.0's own?

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

Well, the answer is quite nicely! Depending on the clip and metric, v3.0.x is overall either lightly ahead or lightly behind, while staying equally fast! The trade-offs are mildly different, as v3.0.x is capable of being even slightly faster, at the cost of some efficiency. It's fair to say the dev team managed to preserve the performance of the fastest mode despite said reduction in presets. Preset 13 was always considered an experimental mode reserved for convex hull purposes, so it's my assumption that the dev team had no real intention of making the fastest preset even faster if accuracy was impacted further than it already is.

Here are some visual comparisons to visualize the situation! Did you miss them?

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

v3.0.x's preset 9 vs v2.0.0's presets 11 & 12​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

In a nutshell, preset 9 in v3.0.x fares in between v2.0.0's presets 11 and 12 in efficiency, though usually closer to preset 11 overall, while usually being closer in speed to preset 12 (notable exception on the gaming clips)! That's a free efficiency or speed boost for any non-gamer upgrading!

Screenshots... screenshots everywhere! Find them right below:

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

v3.0.x's preset 8 vs v2.0.0's presets 9 & 10​

---> Compression efficiency graphs, full quality range:

Preview

---> Compression efficiency graphs, "high quality" range (CRF10 -> 23):

Preview

---> Compression efficiency graphs, "low quality" range (CRF28 -> 50):

Preview

---> Speed graphs:

Preview

---> Interpretation:

As with preset 9 previously, preset 8 in v3.0.x usually competes closer to v2.0.0's preset 9 in efficiency, while performing one tier higher with speeds comparable to v2.0.0's preset 10. The results are more chaotic on the gaming clips but one clear advantage of preset 8 is its robust consistency, unlike v2.0.0's preset 10 (and v3.0.x's two faster presets).

Here's our last set of comps for the day:

---> Visual Comparisons:

HQLQ
Avatar (HQ)Avatar (LQ)
Ducks (HQ)Ducks (LQ)
Fallout (HQ)Fallout (LQ)
Minecraft (HQ)Minecraft (LQ)
Sol Levante (HQ)Sol Levante (LQ)
Suzume (HQ)Suzume (LQ)
Mandalorian (HQ)Mandalorian (LQ)

SVT-AV1 v2.0.0 -> v3.0.x General BD-rate Evolution​

Before we wrap up on the presets analysis, I'm going to attempt an exercise widely used for comparing video encoder performance in both academic research and industry benchmarks. BD-rate (Bjøntegaard Delta rate) calculates the bitrate savings between two encoders at equivalent quality levels, giving you a single percentage that represents compression efficiency gains. You've already seen these numbers throughout the post. While BD-rate has its limitations (it assumes rate-distortion curves follow specific mathematical models and can struggle with very different encoder behaviors), it remains the most widely accepted metric for encoder comparisons. Psy-ex's metrics conveniently outputs BD-rate numbers when running its benchmarking scripts, so this served as the basis for what we're about to do.

The graphs in this section plot BD-rate (%) against encoding time (ms), showing you the classic speed-vs-efficiency trade-off that defines encoder development.

The first batch of graphs uses SVT-AV1 v2.0.0's preset -1 as the reference:

Preview

We can instantly see the limitations of this approach. Using accurate results is vital for these graphs to make any sense, and having such bumpy curves is proof that despite my efforts in ensuring the measured encoding times were correct, this methodology is too sensitive to even the smallest of deviations. That being said, I'm not helped by the very behavior of the encoder I'm benchmarking: as we've seen in earlier speed graphs, SVT-AV1's performance doesn't scale linearly with CRF values. That is usually the expected behavior of encoders, and prior SVT-AV1 versions used to be like this too (cf. first deep dive article). This entails that when calculating the average BD-rate across the entire quality range, the shape of the speed curve can seriously throw off the results.

Anyway, let's try to interpret these results a bit. The faster presets can be found at the top left of the graph, and the slower ones in the bottom right. We can still generally notice the brighter colors tend to be above the darker ones (representing the newer versions), meaning the trade-offs did in fact improve overall. If we take preset -1 as an easy-to-analyze example, we can confirm our previous findings that it did in fact get faster and faster with versions at little to no efficiency impact.

What happens if we take a radically different preset as the reference though? Well, I got you covered with the following graphs, using SVT-AV1 v2.0.0's preset 10 (the default if unspecified) as the reference:

Preview

In this case, it tends to straighten the curves and make any sort of analysis harder. It does not help the situation and there's not much more to say with these graphs. At least, it gives a different perspective of the results.

All in all, let's just say this was a fine experiment and a good learning opportunity. I'll think of ways I can do this better by next time.

SVT-AV1 v2.0.0 -> v3.0.x Conclusion​

So what do you think of SVT-AV1's evolution from v2.0.0 to v3.0.x? Do you find it underwhelming? Well, that's almost expected. The SVT-AV1 dev team has been hard at work to reduce the amount of presets. As that's been a focus for quite some time, I'm not sure the initial reasons for this are still valid today.

Anyway, what we're mostly seeing are small but free speed improvements from smart trade-off decisions. It's worth noting that versions 2.0.0 and 3.0.0 weren't actually major feature milestones. The version numbers jumped because of breaking API changes that aren't backward compatible with previous releases.

Beyond the incremental performance gains, analysis of the different changelogs reveals a clear development strategy focused on specific areas. The dev team has invested heavily in ARM optimizations and memory requirements reduction, worked on the fast-decode feature to further reduce decoding cycles, and streamlined the architecture by removing the decoder component entirely. While many of these changes might not influence quality metrics, they represent important steps toward broader SVT-AV1 adoption by facilitating integration for actors in the industry and ensuring cross-platform consistency.

The biggest changes since v2.0.0 probably lie in the parameters originating from SVT-AV1-PSY that were recently introduced.

Presets Analysis TLDR​

So SVT-AV1 v3.0.0 delivers some nice speed gains across the board. I will refrain from giving numbers to which exact presets possess the best efficiency-to-speed trade-offs due to my speed accuracy concerns, but I can confidently say presets 2 and 4 remain the efficiency champions, giving you excellent quality without completely destroying your encode times. Presets 5-8 strike a good balance, trading a bit of efficiency for significantly quicker encodes.

This should give you a good foundation for picking your go-to preset(s) in v3.0.x.

SVT-AV1 v3.0.x Parameters Revisited​

This blog post is already pretty long... Although we won't revisiting every encoder parameter like we did back in the first deep dive on v1.8.0, we will concentrate on a few important ones, some of them coming straight from the SVT-AV1-PSY project!

EDIT: This section has been developed further in Part 2 of the article, now available here!

Closing Thoughts​

Today, we tested five SVT-AV1 versions, ranging from v2.0.0 to v3.0.x, to quantify their relative efficiency and speed. New metrics were introduced and the format from the last blog post entry was iterated upon to improve the overall quality and flow of this present article. We used this opportunity to experiment and learn encoding knowledge along the way, I hope it was valuable to you too! This is only the first part of this benchmarking session, as we'll deep dive variance boost and a few other exciting features in the next one. Revisiting certain key parametres in such a context has been on my mind for quite a while, so I hope you will look forward to Part 2!

Your feedback and suggestions are always welcome as I work to improve this blog format. Do you have a request for me? Something you'd like to see fixed or added? Let me know what you think on socials or in the different communities I'm active in!

Thanks for reading and see you soon!

I want to extend my thanks to the people who contributed, directly or indirectly, to the making of this article, including Gianni Rosato (gb82), Line (Lumen), Soda, Emre, Bolu, Julio Barba, the people behind slow.pics for hosting thousands of screenshots each time I make these articles, the SVT-AV1 dev team for the work on this amazing encoder, and probably others I'm forgetting...

Consider supporting me by making a donation on my Ko-Fi page, to reward my efforts and to compensate for the electricity bills of weeks of non-stop encoding.

]]> video compression benchmarks <![CDATA[Fast Computation of SSIMULACRA2 on GPUs: A Performance Evaluation]]> https://av1.wiki/blog/turbo-metrics-performance https://av1.wiki/blog/turbo-metrics-performance Thu, 21 Nov 2024 00:00:00 GMT In this blog entry, we will evaluate the performance of TurboMetrics, a software which provides a GPU-accelerated implementation of SSIMULACRA2, and promises to save a lot of time in its computation.

Video encoding is a time-consuming task by itself. When it comes to assessing the resulting encodes, objective quality metrics are often used, providing advantages over subjective measurements, e.g., faster evaluation processes. Nevertheless, the computation of some of these metrics is still somewhat slow. This is the case for SSIMULACRA2, one of the most popular objective metrics for image and video assessment.

Basics​

Before we start with the evaluation, we will first cover some fundamental (and not-so-fundamental) concepts.

SSIMULACRA2​

SSIMULACRA2, which is sometimes abbreviated as "ssimu2", is a perceptual metric for images, based on the concept of the multi-scale structural similarity index measure (MS-SSIM). In the words of its developer, it is "[based on MS-SSIM], computed in a perceptually relevant color space, adding two other (asymmetric) error maps, and aggregating using two different norms". A total of 54 error maps are computed over multiple downscalings (from 1:1 to 1:32) of the image to assess (distorted) and its source image (reference). These error maps are then added using a weighted sum to produce the final SSIMULACRA2 score. The weight of each error map is tuned based on a large set of subjective scores from different image benchmarks.

SSIMULACRA2 scores are in the range [-inf, 100]. They are reported to correlate to subjective visual quality scores as follows:

  • 30 = low quality
  • 50 = medium quality
  • 70 = high quality
  • 90 = very high quality; likely impossible to distinguish from the original when viewed at 1:1 from a normal viewing distance

One special consideration when using SSIMULACRA2 to assess video quality is that, even though the developers state that a score of 90 corresponds to a "visually lossless" image, for videos, this threshold is considered to be lower, around 80. This is because it is harder to notice artifacts on individual frames, given the short period of time the viewer perceives them (usually, between 1/24 and 1/60 of a second, 0.04167 and 0.0167 seconds, respectively).

The reference implementation for SSIMULACRA2 can be found here. Other popular implementations include the Rust implementation, and the Zig implementation.

Although technically speaking SSIMULACRA2 is an image-focused metric, it has gained popularity as a video assessment metric due to its reliability in correlating to subjective measurements. It is considered to provide better results than other historically more popular metrics, such as PSNR, (MS-)SSIM, and even VMAF. In that context, each video frame is treated as a separate image. SSIMULACRA2 scores are computed for each of the video's frames independently, and the average is taken as the video's score. Other useful statistics include the standard deviation (which correlates to the consistency of the quality of the video), the median, the 5th percentile, and the 95th percentile. Although SSIMULACRA2 is considered to be quite reliable for video assessment, it is worth noting that, being a purely image-based metric, it disregards any temporal information present in videos, which might be seen as a disadvantage. Other metrics, such as XPSNR, were developed with video assessment as their main objective, and do consider temporal information.

As stated previously, SSIMULACRA2 computes 54 error maps per image assessed. This entails a high amount of computation, especially for videos, which usually contain tens of thousands of frames (images). That explains its rather slow execution time. Speeding up these computations would be very much desired, if possible.

In-depth: SSIMULACRA2 computation​

The summary of the algorithm for SSIMULACRA2 is the following, as reported here:

  1. Get the frame pair (reference and distorted).
  2. Convert the frames to linear RGB (using 32-bit floating-point).
  3. For each scale (1:1 to 1:32, downsampling by 2 every step: 6 different scales):
    1. Downscale by the scale number (if needed).
    2. Convert the frames to XYB color space.
    3. Blur the following pictures, using a recursive Gaussian blur: (reference * reference), (distorted * distorted), (reference * distorted), (reference), (distorted).
    4. Compute 1-SSIM, artifact and detail_loss error scores from the 5 blurred images and the original frame pair; this yields 3 error maps.
    5. The error maps are reduced to a single number using the 1-norm and 4-norm.
  4. From this, we get 6 scores for each scale (6) and color component (3), totaling 108 scores.
  5. The scores are added using a weighted sum.
  6. The final value is processed through a non-linear function and clipped to render a score between 100 and -infinite.

SSIMULACRA2 subsampling​

Given that the metric is quite slow to compute, it is often computed over a subset (subsample) of the video frames, instead of over all of them. For example, a stride of 3 could be used, to compute SSIMULACRA2 scores for 1 frame out of every 3 in the video, skipping the computation for the other 2 frames. This is expected to speedup the computation by a factor of ×3.

However, this evidently induces an error in the score, as the average of the scores of a subsample of the video frames is not guaranteed to be equal to the average of the scores of all the frames. The smaller the subsample (i.e., the higher the computation stride), the higher this error will be. Nevertheless, it has been reported that small strides usually produce averages significantly close to the real average of all the frames. Thus, it is pretty safe to do this. To minimize the error of this sampling method, it has been stated that "as long as you don't sample in a power of two ([i.e., every] 2, 4, 8... [frames]), you'll be fine, as you want to pick frames from every temporal layer [of the video]". Other sources further recommend picking strides that are prime numbers (e.g., 1 every 3, 5, 7, 11... frames), since "prime numbers reduce the likelihood of sampling bias due to constant frame rate".

TurboMetrics​

TurboMetrics is a WIP software suite developed by the very talented Gui-Yom/LimelioN. It is "a collection of video related libraries and tools oriented at performance and hardware acceleration". It comes as a collection of Rust libraries/crates, and some include command-line applications to interact with them. There are currently no public releases of the suite in the dedicated "Releases" page of its GitHub repository, but you can download the code and compile it yourself. The latest tagged version of the software, at the moment of writing, is v0.2.2.

TurboMetrics focuses mainly on NVIDIA GPUs using CUDA. However, the developer is actively working on adding support for other hardware (e.g., AMD and Intel GPUs). Nevertheless, I only have experience with NVIDIA GPUs, so I cannot tell how the progress is going on that front.

Among the tools, there is a working SSIMULACRA2 implementation that uses CUDA to leverage GPU acceleration, ssimulacra2-cuda. We will focus exclusively on it from now on. We will use this tool to compute SSIMULACRA2 scores for certain video samples. However, it is worth noting that the tool is also able to compute other metrics, such as SSIM and PSNR. It can even compute multiple metrics on the same rum.

ssimulacra2-cuda is stated to be "close to the original implementation, and with close results". Also, the developer has said that "[r]ight now, [it] can compute SSIMULACRA2 orders of magnitude faster than the ssimulacra2_rs implementation". Therefore, we can expect the tool to compute SSIMULACRA2 scores for videos much faster than any CPU implementation out there, but provide scores that are not quite the same as the ones those CPU implementations would give, including the reference one. In this blog entry, we will test that second claim, concerning the speed of TurboMetrics in computing SSIMULACRA2.

In-depth: Implementation differences from the reference​

As reported here, the GPU implementation of SSIMULACRA2 provided in TurboMetrics differs from the reference implementation in the following ways that can affect the accuracy of the reported scores:

  • The conversion from YUV to linear RGB might not be yielding the same results as other tools.
  • Explicit Fused Multiply-Add (FMA) operations are used when possible, as the GPU can leverage them for higher performance.
  • The order of operations might not be the same, as some calculations have been rearranged.
  • Floating-point operations might not yield the same results, especially with the approximated math functions used in the GPU (like powf).

Additionally, the differences in scores are amplified by the following features of the SSIMULACRA2 computation:

  • The original weights were computed by fitting the error scores to Mean Opinion Score (MOS), and deviation in the error scores is amplified by them.
  • The final non-linear function is rather steep (it is a cubic function, x³), and, thus, score variations are amplified again.

Performance evaluation​

Setup​

In this experiment, we will evaluate TurboMetrics in terms of performance; that is, the amount of ssimu2 scores computed per unit of time. For that, an episode of an animated series (~25 minutes), encoded with H.264, has been chosen as sample video. However, the kind of video content used, and most of its technical characteristics (e.g., codec, encoding parameters, etc.) except for resolution, should be irrelevant, as the same computations are performed for every video of specific frame dimensions. In this case, the video has a resolution of 1080p (1920×1080 pixels per frame), with 8 bits per pixel, and YUV color space with 4:2:0 chroma subsampling.

The video source and example distorted encode have been compared using ssimulacra2_rs, vszip, and TurboMetrics. ssimulacra2_rs and vszip are both CPU implementations of SSIMULACRA2, the former written in Rust and the latter in Zig (but invoked from VapourSynth/Python scripts). It has been reported that the Zig implementation is considerably faster than the Rust one, and we can check that out as well!

The CPU implementations have been executed on an Intel i5-11400 CPU (base frequency of 2.60 GHz, turbo frequency of 4.40 GHz), using 6 CPU cores (-f 6 parameter in ssimulacra2_rs, -t 6 parameter in ssimulacravszip.py). TurboMetrics has been executed both on an NVIDIA GTX 1060 GPU and an NVIDIA RTX 4060 GPU. The three programs have been executed for the same video pair with varying frame strides: 1, 3, 5, 7, 11, 13, 17, 19, and 23 (not skipping frames, plus every prime number lower than 24 frames / 1 second, excluding 2).

For the CPU and RTX 4060 executions, the distorted video was a custom AV1 encode of the reference video. For the GTX 1060, as it does not have AV1 decoding hardware support, the H.264 reference video was compared to itself (the computations being performed should be equivalent anyway).

Results and discussion​

The exact results of this experiment are shown in the following table:

Stridessimulacra2_rs FPSvszip FPSGTX 1060 GPU FPSRTX 4060 GPU FPS
13.3714.6346.7085.64
39.5537.67133.86245.73
515.1549.34164.34350.01
720.4556.37197.17400.94
1130.2364.46222.98458.61
1334.5466.65239.17488.83
1740.5169.23249.58520.30
1942.8269.99258.71525.44
2345.4388.60264.85546.14

Here they are in graph form, for better visualization:

CPU vs GPUs decoded FPS

On a superficial level, we can already make some observations:

  • SSIMULACRA2 computation with ssimulacra2_rs is quite slow: 3.37 frames per second (without skipping frames) when using 6 threads. That is like 0.56 frames per second per thread. I know the CPU used is no beast by any measure, but beast CPUs are quite expensive and not that common (that is something to keep in mind). Those processing speeds are comparable to some of the slower-ish presets of SVT-AV1.
    • vszip is, in fact, considerably faster: Between ×4.34 (full video computation) and ×1.63 (stride of 19) with the same amount of threads. However, it seems that vszip does not scale as well as the other implementations (except for that weird performance boost with a stride of 23).
  • GPU computation of SSIMULACRA2 scores is impressive, considering the CPU performances:
    • Compared to ssimulacra2_rs: Speedups for the GTX 1060 (an 8-year-old GPU) are between ×14.01 (stride of 3) and ×5.83 (stride of 23). Speedups for the RTX 4060 are between ×25.73 (stride of 3) and ×12.02 (stride of 23).
    • Compared to vszip: Speedups for the GTX 1060 are between ×3.70 (stride of 19) and ×2.99 (stride of 23). Speedups for the RTX 4060 are between ×7.51 (stride of 17) and ×5.85 (stride of 1 / full video computation).
    • It is amazing seeing how GPUs not only achieve real-time SSIMULACRA2 computation of entire videos, but more than ×1.94 and ×3.56 real-time processing, even (GTX and RTX, respectively).

Nevertheless, the most interesting observation is made when we use computation strides bigger than 1: The performance does not scale linearly with the stride, as it would be expected. This happens with both the CPU and GPUs, although it is more noticeable in the case of the GPUs due to their higher performance.

Why is that? Well, there is actually a good explanation: The reported FPS are decoded FPS. That is, the speed in which the software is decoding and processing all the video frames. All the frames in the video bitstream are decoded and reconstructed as images, but the SSIMULACRA2 score is only computed for some of them. SSIMULACRA2 computation affects decoding speed in that the decoding of each frame only starts after the processing of the previous has finished (kind of, this is not be entirely true for multithreaded CPU implementations, but we can ignore that). The faster the SSIMULACRA2 processing is done, the sooner the next frame starts decoding. And there is no faster processing than not doing any processing at all, that is, skipping the frame! However, the decoding itself is a process that has to be done regardless, and the time it takes should not be disregarded. Although decoding as a process is considerably faster than SSIMULACRA2 computation, it starts becoming the bottleneck when you both compute SSIMULACRA2 so fast, and skip computing frames.

Considerations about video decoding performance​

According to NVIDIA:

  • The GTX 1060 is able to decode H.264 1080p YUV 4:2:0 video at up to 696 frames per second.
  • The RTX 4060 is able to decode H.264 1080p YUV 4:2:0 video at up to 883 frames per second.
  • The RTX 4060 is able to decode AV1 1080p YUV 4:2:0 video at up to 1005 frames per second.

Being proprietary NVIDIA technology, served as a black box through their API, we cannot be sure how the NVDEC engines are implemented and what exactly are their limitations under scenarios such as these (i.e., concurrent decoding of multiple video streams of same or different codecs). We can only trust the information NVIDIA provides. NVIDIA states that all GeForce products consist of a single NVDEC; however, we cannot be certain how its resources are managed or shared when working with multiple simultaneous decoding contexts. In our case, considering two videos are being decoded at the same time:

  • The GTX 1060 decodes up to 529.7 H.264 (2×264.85) frames per second when using a stride of 23 frames, which corresponds to 75% of its reported peak decoding performance for H.264.
  • The RTX 4060 decodes up to 546.14 H.264 + 546.14 AV1 (1092.28 total) frames per second when using a stride of 23 frames, which corresponds to 61.85% and 54.34% of its reported peak decoding performance for H.264 and AV1 respectively. Notice how, even though the NVDEC hardware resources are reportedly being shared between the two decodes, both are able to reach more than 50% peak performance at the same time.

We can just focus on processed FPS, that is, only not-skipped frames processed per second, or SSIMULACRA2 scores computed per second, to see how the processing performance degrades with increasing strides.

Stridessimulacra2_rs FPSvszip FPSGTX 1060 GPU FPSRTX 4060 GPU FPS
13.3714.6346.7085.64
33.1812.5644.6281.91
53.039.8732.8770.00
72.928.0528.1757.28
112.755.8620.2741.69
132.665.1318.4037.60
172.384.0714.6830.61
192.253.6813.6227.65
231.983.8511.5223.75

Here they are in graph form, for better visualization:

CPU vs GPUs processed FPS

It can be seen how the actual SSIMULACRA2 processing performance quickly decreases, as more and more relative time gets dedicated to decoding the videos.

One should also note that the NVDEC hardware is the same for all GPUs in the same generation (e.g., all the RTX 40 GPUs). Therefore, even if the computation of the SSIMULACRA2 scores of a video could still be accelerated further by using more powerful GPUs, e.g., RTX 4070/4080/4090, that would only be significant when using small strides. As the stride increases, the video decoding will become the bottleneck for all those GPUs sooner or later, and all of them would perform similarly to the RTX 4060. In other words: rather than spending more money on a more powerful GPU to compute SSIMULACRA2 faster, it is much more worthwhile to just increase the stride; especially since the error incurred from using a stride greater than 1 usually is statistically almost insignificant.

Conclusions​

As a quick summary of the experiment, we can highlight:

  • GPU processing greatly accelerates SSIMULACRA2 computation, with modern GPUs enabling speeds faster than real-time for most 1080p content (~85 fps), including gaming; and old GPUs still providing speeds faster than real-time (~46 fps) for popular, lower-fps 1080p video, such as movies and series.
  • As SSIMULACRA2 computations accelerate, video decoding speed becomes a relevant factor, to the point of being the main bottleneck when the computation stride is large enough. It is not worthwhile to increase the computation stride too much, as you will quickly get diminishing returns in performance.

Further discussion: Possible improvements​

One may think that a possible solution to the decoding speed bottleneck, which does not require waiting until the next-generation GPUs implement faster decoding hardware, would be to make the software decode only the frames that are going to be processed. Nevertheless, that approach comes with significant caveats: In modern video coding, frames cannot be decoded independently of each other, and a full group-of-pictures (GOP) would need to be decoded to reconstruct any of its frames. Usually, GOPs are from 1 to 10 seconds long (24 to 240 frames in our test case), depending on the encoding settings. Using any computation stride smaller than the GOP size would not see any benefit from this technique; and greater computation strides would only see limited benefits. (Aside from the possible accuracy/representativeness concerns associated to using such big strides.) Additionally, when leveraging GPU programming, the actual hardware is often treated like a black box, and the available APIs may limit considerably the control the user has over it.

Another, more useful approach one may think of, would be to exploit computation overlapping: To keep decoding frames while the SSIMULACRA2 scores are being computed, storing the decoded frames that are to be processed in a buffer. This seems to be partially implemented in TurboMetrics already, but only to some extent.

Without any modification to the SSIMULACRA2 computations, the overlap in computation and decoding would still have limited effect for low strides (where SSIMULACRA2 computation is still the bottleneck), as every time the decoded frame buffer fills completely, the decoder would have to wait for it to have free space again. In theory, the SSIMULACRA2 computations could be sped up further (maybe requiring faster but more imprecise approximations). And, ideally, TurboMetrics should aim to approach the theoretical decode FPS limit in the computation of the SSIMULACRA2 scores, thus perfectly overlapping computing and decoding. In such scenario, skipping the computation of any frames would not make sense anymore, as no speed advantage would be obtained from it. However, this is still far from being a reality, as a speedup of at least ×10.31 is needed for an RTX 4060 to match its decoding speeds with a computation stride of 1.

Short discussion on scoring error​

My original idea for this blog entry was to evaluate TurboMetrics both in terms of performance and scoring error. By "scoring error" I mean "the difference between the scores a CPU reference provides, and the ones TurboMetrics provides". However, once the first draft for that original version of this blog entry was completed, I realized I was not satisfied with the methodology followed and conclusions extracted from that second experiment. Therefore, I am putting off that evaluation for a second blog entry, that I will write once I make a new, bigger, and better, evaluation of the error.

The main concern about the error the GPU computation incurs on SSIMULACRA2 scores is that SSIMULACRA2 scores have been tuned mathematically to correlate with visual quality, while the GPU implementation's scores try to mimic those scores, but have not undergone any kind of mathematical tuning or assessment themselves. The GPU scores could be seen as "SSIMULACRA2 + random error", and that "random error" might break the desired correlation, thus reducing the reliability of the metric.

Nevertheless, it is worth noting that SSIMULACRA2 is not by any means a kind of "perfect metric" or "absolute truth" about video quality assessment. SSIMULACRA2 still is unreliable in some situations, and it does not always present a perfect correlation with visual observations. You may find here and there specific examples of SSIMULACRA2 scores that do not accurately reflect the (perceived) quality of a compressed image or video. Besides, we should keep in mind that quality is a subjective matter. In that regard, and as an example, people who do not care much about film grain might give denoised videos higher quality scores than those that SSIMULACRA2 gives. (Other metrics, such as VMAF, are more tolerant to denoising.) With all this in mind, the results provided by TurboMetrics are not necessarily "incorrect", but rather provide a kind of "alternative metric".

Just to give a preview on the error evaluation thus far, I will share the following: The preliminary results I obtained from that now-scraped evaluation seem to indicate that:

  • The scores provided by the GPU implementation of SSIMULACRA2 are statistically different to the real scores (provided by the CPU implementations).
  • No linear correlation between the real scores and the GPU scores has been found.
  • TurboMetrics might tend to under-score, compared to the real scores (i.e., the results for the frames tend to be lower than those a CPU implementation gives). This has yet to be assessed over a larger sample size to prove if it is really true, though.

I am looking for help with the error evaluation for the follow-up blog entry. Mainly, I need a considerably larger sample of SSIMULACRA2 scores for videos, computed both in CPU and GPU. I myself do not have enough material (encoded videos and/or the SSIMULACRA2 scores for all their frames) yet to do a sufficiently good evaluation, and it would take me quite some time to get it. Thus, I am looking for anyone who has saved the SSIMULACRA2 scores for all the frames of any video they had assessed, and is able to re-assess those same videos using TurboMetrics. If you think you can help, and are willing to, please contact me through the AV1 for Dummies Discord server.

Overall conclusions and final thoughts​

In this blog entry, we have tested the GPU implementation of SSIMULACRA2 available through TurboMetrics, in terms of performance (i.e., scores computed per unit of time). The main conclusions are the following:

  • All in all, TurboMetrics provides impressive acceleration of the SSIMULACRA2 computation.
  • The performance of TurboMetrics does not scale linearly with the computational stride/frame skipping. When increasing the stride, a point of diminishing returns is reached rather soon.
    • This is caused by the time taken to decode the video frames, which is non-negligible, and becomes the bottleneck when computing SSIMULACRA2 so fast and only for some of the frames.

Hardware acceleration of computationally-intensive tasks is an important topic nowadays in several fields, especially considering how powerful current-day GPUs are; and I would hope for the teams developing these quality assessment metrics to consider it as an existing option. So far, only XPSNR has seemed to be concerned about computational complexity from the beginning, and the developers' response was not to consider hardware acceleration, but rather to simplify the design of their metric.

After developing this blog entry, my own recommendation for assessing videos using SSIMULACRA2 would be to use TurboMetrics, with a stride of 3 or 5 frames. If you are willing to tolerate the error incurred by the GPU implementation, you should tolerate the error incurred by skipping frames, which probably is an order of magnitude lower. Of course, you should not take my recommendations blindly, and you should try it out yourself to see what works for you, and what you can tolerate.

Future​

TurboMetrics' development is still ongoing, and I know Gui-Yom has tons of ideas for it. To begin with, I know the following ideas are being worked on:

  • Support for GPUs of other vendors (AMD, Intel).
  • Support for FFmpeg piping, to enable working with videos of any codec.
  • GPU acceleration of XPSNR.

All these features would improve TurboMetrics significantly, while also opening the door for more tests to be performed. Additionally, as it has been mentioned in this blog entry, the SSIMULACRA2 computation can still be optimized further, including overlapping computation and decoding. Gui-Yom knows this, and plans to tackle that issue eventually.

A list of features to improve TurboMetrics' computation of SSIMULACRA2, written by the main developer, can be found here.

As I said earlier, I also plan to write a follow-up blog entry evaluating the scoring error incurred in the GPU implementation, to test the reliability of the scores provided by TurboMetrics. Additionally, it may be insightful to evaluate the error for GPU implementations of other video metrics, such as VMAF CUDA, and see if we find any similarities. I do not recall reading any analysis on the matter for other metrics, either. Nevertheless, I cannot give a date yet for when that follow-up blog entry might come out. I reckon it will take quite some time to perform the required experimentation, as well as processing and analyzing the results.

Also, this blog entry is heavily missing information on 4K video, which I believe to be of high interest. The main reason for it is that most 4K content is served using HEVC, and TurboMetrics still does not support that codec. When support for it gets added, experiments with 4K video should be conducted.

If you like TurboMetrics and have knowledge about programming, please consider contributing to it to the best of your ability. Together, we can make it even more awesome than it already is!

And, lastly, thank you very much for reading!

Special thanks​

  • Gui-Yom, for developing TurboMetrics, and also for proofreading and green-lighting this blog entry.
  • Trix, for linking me to the Python script to easily compute SSIMULACRA2 using vszip (ssimulacravszip.py), and for helping me to get it to work the way I needed.
]]> video metrics benchmarks <![CDATA[Better late than never: SVT-AV1 v2.2.x Deep Dive]]> https://av1.wiki/blog/svt-av1-third-deep-dive https://av1.wiki/blog/svt-av1-third-deep-dive Thu, 14 Nov 2024 00:00:00 GMT SVT-AV1 v2.2.0 was released in late August and a minor version v2.2.1 followed suit to adress some bugs. This blog post will focus on comparing this new encoder version to the last, on the basis of benchmarks and visual comparisons. We will quantify the new trade-offs between compression efficiency and encoding speed, so you can choose the right balance for your projects. Our metrics of choice today will be SSIMULACRA2 and XPSNR, used in conjonction with a revised methodology.

Feedback​

The biggest missed opportunity of the previous SVT-AV1 deep dives was the absence of visual comparisons. Indeed, metrics may be convenient for easily quantifying differences between encoder versions or encoding parameters, but they fail to give the information of how much these differences matter for your eyes. However, making properly useful visual comparisons isn't an easy task. Comparing two encodes of varying bitrates will bias the result against one or the other, which is not desirable. Figuring out the best way to present these comparisons and making the appropriate scripts took me weeks. Gathering all the necessary data, crafting the comparisons and proceeding to a double-checking pass took me another few weeks. These reasons explain why this blog post took so long to release, but I hope it will have been worth the wait! On that note though, I have been uploading thousands of png screenshots to slow.pics and I ended up rate-limited. This has prevented me from uploading the visual comparisons for preset 11, 12 and some of 10. I'm actively trying to fix this situation, so please understand.

Another reason for this taking so long was my decision to increase the amount of video samples while increasing the amount of CRF values tested. A grand total of 3682 encodes were done for this blog post alone, in the span of around two weeks, where my PC would be exclusively encoding 24/7. Send help.

Also, the graphs will now be using the harmonic mean instead of the arithmetic mean. Indeed, arithmetic mean scores fail to account for deviations and outliers. Using the harmonic mean implies low-scoring frames have more weight towards the final score, which adds a consistency component to the picture. That's not it though! Consistency is capital in an enjoyable watching experience. As such, each graph now possesses a SSIMULACRA2 (Harmonic) version, a XPSNR (Harmonic) version and a SSIMULACRA2 (Standard Deviation) version, in order to closely monitor variations in consistency between presets.

Lastly, the biggest complaint I have received is me exclusively using anime clips. The reason for that is pretty simple: I mostly encode anime content on my free time and am myself very little interested in other types of media. However, these blog posts have grown in exposure and I understand a majority of people is more concerned about the performance of encoders on live action content or gaming clips. Thus, this new blog post is comprised of 3 live action clips, 2 gaming clips and 2 anime clips! See the sacrifices I'm making for y'all?

Methodology​

The resources provided will include both graphs and image comparisons. Graphs offer a straightforward, objective look at efficiency across encoder parameters, using metrics as benchmarks for performance. In contrast, image comparisons display actual samples from encoded files, allowing you to assess quality firsthand. This adds a subjective dimension to the comparisons, giving you a more nuanced understanding of each preset's impact on visual quality.

The testing methodology involves using relatively short video samples with a wide range of content types, uncompressed to the y4m file format for ease of use. These lossless files are directly fed to SvtAv1EncApp, implying the performance of a single encoder instance is what's being measured here. A more serious AV1 encoding pipeline should probably be leveraging a chunked encoding approach, especially on higher core count systems. Once an encode is done, SSIMULACRA2 scores are calculated using the Zig implementation, XPSNR scores on the other hand are calculated using a ffmpeg filter, and the data is then aggregated into a final Harmonic or Standard Deviation score to create the graphs for this benchmark. The Constant Rate Factor (CRF) is plotted against encoding time, and the metrics scores against encode size (bitrate). The former may represent the efficiency as defined by the speed achieved at a certain quality target. For the latter, Bits Per Pixel scores (BPP) are calculated so that the Metric / BPP graphs may represent the compression efficiency, normalized by resolution.

How to read the graphs? For the compression efficiency ones, the closer to the top left the better. For the encoding speed ones, the closer to the left the faster. For the standard deviation ones, the closer to the bottom left the better.

The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy.

SvtAv1EncApp was compiled directly from the v2.1.2 and v2.2.1 source code using Clang 18.1.8 and the provided Build/linux/build.sh script with the following command: build.sh cc=clang cxx=clang++ jobs=$(nproc) enable-lto static native release. The testing machine is comprised of an i3 12100 in its stock configuration, with 2x8GB of 3200MHz CL14 DDR4 RAM, in Arch Linux with kernel 6.9.12 and the performance governor enabled. All encodes have been made in the same session without rebooting.

I want to give a disclaimer concerning encoding speeds. Contrary to the efficiency results which should be reproducible independantly of the machine, measuring speed is a pretty difficult endeavor, with increased risks for errors. The performance numbers I mention may differ for you depending on the hardware configuration at hand.

Samples & Encoding Settings​

The samples are as follows:

  • 17s Avatar The Way Of Water (trailer 3) clip sourced from thedigitaltheater.com (1916x804p, 23.976fps).
  • 6s Ducks Take Off clip sourced from xiph.org (1280x720p, 50fps).
  • 3s Fallout 4 clip sourced from another encoder fellow (1920x1080p, 60fps).
  • 8s Minecraft clip sourced from xiph.org (1920x1080p, 60fps).
  • 8s Sol Levante HDR clip sourced from opencontent.netflix.com (3840x2160p, 24fps). This one is pretty educative as SVT-AV1's behavior isn't influenced by the existence (or lack thereof) of HDR metadata in a source.
  • 21s Suzume (trailer 2) clip sourced from thedigitaltheater.com (1920x808p, 23.976fps).
  • 13s The Mandalorian (trailer 2) clip sourced from thedigitaltheater.com (1920x800p, 23.976fps).

All clips have been encoded in a wide quality range, from --crf 10 to --crf 50, by increments of 2, with the exception of preset -1 that uses increments of 4.

--preset X --hierarchical-levels 4 are the only parameter used here, in conjunction with the CRF values. I have been asked to use --hierarchical-levels 4 by fellow SVT-AV1-PSY developers to force smaller mini GOPs, more appropriate for testing.

Else, the SVT-AV1 defaults were used. The ones worth mentioning are:

  • --tune 1: tune PSNR
  • --aq-mode 2: variance deltaq
  • --enable-qm 0: quantisation matrices disabled
  • --irefresh-type 2: closed GOP
  • --enable-tf 1: temporal filtering enabled And more, like CDEF and restoration enabled, overlays and film-grain disabled...

Visual comparisons​

comp_showcase

Throughout this blog post, you’ll find slow.pics links that provide various visual comparisons between presets.

  • The “full” links offer comparisons across the entire quality range for each source.
  • The HQ (High Quality), MQ (Medium Quality), and LQ (Low Quality) links showcase more targeted comparisons. These have been carefully handcrafted to be as size-normalized as possible, given the available encodes. We want to be focusing on encodes with minimal bitrate deviation for a fair comparison.

Feel free to double-check the bitrate of each frame or scene to make a more informed observation, keeping the size difference in mind when comparing the encodes.

Use the arrow keys and numpad to navigate between screenshots. Alternatively, you can click on "Slider comparison" and select two sources if you prefer comparing this way.

Without further ado, let's start with the first comparisons!

Presets comparisons (-1 -> 12)​

In the following graphs, you may find comparisons between all SVT-AV1 presets, ranging from the slowest --preset -1 to the fastest --preset 12.

Just like in v2.1.x, preset 6 and 13 do not exist in v2.2.x and are instead mapped to preset 7 and 12 respectively.

Efficiency​

  • First of all, the complete efficiency graphs:
Preview

You may notice something odd going on with the Avatar results using XPSNR. I have tried to understand the cause, without success. For the remainder of this blog post, the Avatar XPSNR results will be omitted. I will continue investigating and aim to have a workaround in place for next time.

Anyway, this graph may be impressive, but difficult to read. So let's analyse different quality targets.

  • The same graphs but focusing on the "high quality" range (CRF10 -> 22):
Preview
  • Same, but now focusing on the "medium quality" range (CRF24 -> 36):
Preview
  • And lastly, focusing on the "low quality" range (CRF38 -> 50):
Preview
  • If we now focus on presets 4 and below, where it's more difficult to discern the differences between presets, we get this at "high quality":
Preview
  • This at "medium quality":
Preview
  • And the following at "low quality":
Preview

Speed​

  • Let's now compare the speed of all presets:
Preview

Unusable, right?

  • Then, here is what it looks like with a logarithmic scale:
Preview

Interpretation​

It appears as if once again preset 2 through preset 4 remain the most balanced presets all-around in an efficient encoding scenario, with preset 3 not offering much improvements over preset 4 in average scores but nicely improving on consistency instead, and preset 2 offering a nice efficiency and consistency uplift on top.

In this release again, the quality gap between preset 2 and preset 1 is pretty narrow, and the speed penalty from preset 1 onward continuously increases, ending up close to ~2x. In comparison, the penalty of going from preset 3 to preset 2 is closer to ~1.5x. As such, using preset 1 is entering placebo territory and it is usually not recommended to waste precious encoding resources on preset 0 and preset -1. This especially applies at medium to high quality, though at extremely low quality targets, like the CRF40-50 range, we can still see appreciable gains from these placebo presets in some clips.

As for the faster presets, presets 5 to 10 are usually grouped on the graphs focusing on average scores and the ones focusing on consistency. They tend to stand apart from their slower counterparts by just a bit. Though preset 10 can be worryingly close to preset 11 on some occasions. They are all viable for your real-time needs. The rule is the same as usual: go the slowest you can bear that still achieves your goal!

Presets 11 and 12 are especially inefficient and inconsistent, and to be avoided at all costs. If possible, forget they even exist, as it's probably better to use a comparably fast (or faster) competing codec. They could still be of use in a convex-hull scenario, but in the case of realtime transcoding, you will be better off with some hardware solution like the ones found in RTX 4000 or Arc GPUs.

TLDR​

The same conclusions as the previous blog posts can be made: clear quality gains can be observed as we decrease presets, until preset 2, however the effectiveness of dropping presets is noticeably less and less important as quality is increased.

SVT-AV1 v2.1.x vs v2.2.x presets comparisons:​

In this section, we’ll examine the efficiency and speed differences across presets when upgrading from SVT-AV1 2.1.x to 2.2.x. This comparison should bring a new level of nuance to our results, highlighting both incremental improvements and any notable shifts in performance.

SVT-AV1 v2.1.x brought some nice improvements over v2.0.0, but does v2.2.x bring appreciable improvements in the presets trade-offs this time around as well? Let's find out!

preset -1: v2.1.x vs v2.2.x​

  • Let's start things off with the battle of the placebos, with the Compression efficiency & consistency at "high to medium-ish quality" (CRF10 -> 30):
Preview
  • Along with the Compression efficiency & consistency at "medium-ish to low quality" (CRF34 -> 50):
Preview

Basically no changes at all, except a slight regression on Minecraft.

  • What about their speeds though?:
Preview

Well preset -1 basically became 15 to 25% faster, not bad at all!

  • Preset -1 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (LQ)

preset 0: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Efficiency wise, this new preset 0 is close to unchanged from the old preset 0, but its consistency improved slightly in a few clips at high quality and decreased in one clip at low quality.

  • Speed graphs:
Preview

Preset 0's speed sees an improvement of about 20% at best. Overall, preset 0 got a proper upgrade!

  • Preset 0 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 1: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Preset 1 is mostly unchanged but sees another slight regression in Minecraft at high quality.

  • Speed graphs:
Preview

Depending on the clip, speed is mostly unchanged or ever so slighty improved. Preset 1 is a bit stagnant this release.

  • Preset 1 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 2: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Preset 2's efficiency has regressed at high quality on some clips, improved in some and stayed the same in others. Except in one clip, consistency seems to have improved all around. At low to medium quality targets, efficiency is mostly unchanged, same for consistency. Not exactly noteworthy.

  • Speed graphs:
Preview

Speed was improved by about 10-20%. Not a bad showcase, for sure.

  • Preset 2 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 3: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Practically, it's a wash.

  • Speed graphs:
Preview

Still, preset 3 got slightly faster, I'm happy to report this is a speedup!

  • Preset 3 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 4: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Preset 4 sees a consistent though small improvement in average scores and standard deviation across the entire quality range on basically all clips.

  • Speed graphs:
Preview

Unfortunately, it got slower as a result. If you remember, v2.1.0 did the exact contrary over v2.0.0, I wonder if preset 4 simply took back the place it previously had...

  • Preset 4 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 5: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Both metrics say preset 5 regressed slightly to moderately, though surprisingly its consistency is basically unchanged.

  • Speed graphs:
Preview

The result of this regression is an impressive speedup of up to 25%.

  • Preset 5 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 6: v2.1.x vs v2.2.x​

Preset 6 is mapped to preset 7 in v2.2.x.

preset 7: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Preset 7 is close to unchanged in v2.2.x.

  • Speed graphs:
Preview

It still got some slight to moderate speedups though, which can be appreciated.

  • Preset 7 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 8: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

The efficiency and consistency of preset 8 has improved at high quality.

  • Speed graphs:
Preview

And we can observe a speed increase of around 10%. Some crazy speed deviations can be noticed in Sol Levante.

  • Preset 8 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 9: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Efficiency and consistency stayed mostly the same.

  • Speed graphs:
Preview

Speed improved by a few percents at most. Preset 9 has stagnated over v2.1.x.

  • Preset 9 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 10: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Preset 10 received the most efficiency and consistency improvements out of all presets in this release. What will be the cost of such drastic change though?

  • Speed graphs:
Preview

Well, not that much all things considered! Preset 10's speed did decrease from barely anything to 20% in the most extreme situation, but its improvements well outshine its speed regression. Overall, preset 11 went from borderline unusable to becoming an interesting new fast real-time preset. This is pretty huge in my opinion, as it offers a new kind of trade-off no other AV1 encoder or prior SVT-AV1 versions did.

  • Preset 10 visual comparisons:

Avatar (full), Avatar (HQ), Avatar (MQ), Avatar (LQ)

Ducks (full), Ducks (HQ), Ducks (MQ), Ducks (LQ)

Fallout (full), Fallout (HQ), Fallout (MQ), Fallout (LQ)

Minecraft (full), Minecraft (HQ), Minecraft (MQ), Minecraft (LQ)

Sol Levante (full), Sol Levante (HQ), Sol Levante (MQ), Sol Levante (LQ)

Suzume (full), Suzume (HQ), Suzume (MQ), Suzume (LQ)

The Mandalorian (full), The Mandalorian (HQ), The Mandalorian (MQ), The Mandalorian (LQ)

preset 11: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

The new preset 11 places itself between the old preset 11 and the old preset 12 efficiency and consistency wise.

  • Speed graphs:
Preview

Unsurprisingly, its speed is also in-between the old preset 11 and the old preset 12. I'm unsure this new trade-off helps in anything.

preset 12: v2.1.x vs v2.2.x​

  • Compression efficiency & consistency graphs, high quality range:
Preview
  • Compression efficiency & consistency graphs, medium quality range:
Preview
  • Compression efficiency & consistency graphs, low quality range:
Preview

Preset 12 is almost unchanged from the previous release.

  • Speed graphs:
Preview

Same speed wise. So, no improvements at all for the two fastest presets. They remain all around very bad performers. In hindsight, it doesn't matter that I couldn't upload the preset 11 & 12 visual comparisons, because there is literally nothing to see.

preset 13: v2.1.x vs v2.2.x​

Preset 13 is mapped to preset 12 in v2.2.x.

TLDR​

With v2.2.x, we observed new efficiency/speed trade-offs for a good amount of presets. Some presets, like -1 and 0, received significant speed improvements at no efficiency cost. Presets 3 and 7 received more reasonable speedups. Presets 2, 8 and 11 have seen new trade-offs that are mostly beneficial. Preset 10 was deeply revamped and replaces preset 9 in my book as the fastest, still viable, real-time preset. Preset 4 seems to have returned to the state it was in v2.0.0. On the other hand, preset 5 seemed to have regressed slightly, and presets 1, 9 and 12 are basically unchanged from v2.1.x.

Conclusion​

The release of SVT-AV1 v2.2.x brings some welcome speed improvements. Presets 2 through 4 continue to lead in efficiency for AV1 encoding, delivering top-tier quality and compression. Meanwhile, presets 5 through 10 offer solid alternatives for those who find presets 2 through 4 too slow, balancing quality with noticeably faster encoding times.

Hopefully, this comprehensive third deep dive has given you a helpful starting point for choosing settings when encoding with the latest SVT-AV1(-PSY) v2.2.x.

Future​

Once more, this testing focused on establishing the new presets dynamics, however I haven't revisited the different SVT-AV1 parameters since v1.8.0. A few meaningful features have been added since, like variance boost, and with this overhauled methodology, the conclusion made in that blog post may be different now. I think it will be worth to revisit this in the future, maybe in the next blog post for v2.3.0? Yes, I'm fully aware I'm late because v2.3.0 has already been out for two whole weeks. Even if the frontend of this blog post doesn't seem to have radically changed, my entire workflow has tremendously evolved since last time. It may have taken me since mid-August to manage to complete this blog post, but my efforts should allow me to produce a follow-up faster.

I am conscious of this blog post's limitations. First of all, I observed some odd behaviors from XPSNR on certain clips which I haven't been able to pinpoint yet. It would also give me nonsensical standard deviation results, which is the reason why only the standard deviation in SSIMU2 scores was given. Second of all, SVT-AV1's own behavior starts to get messy when you approach the SSIMULACRA2 0 score, rendering all the data in that region pretty much useless. Plus, aggregating these metrics scores take forever, so I'm looking into ways to accelerate the process, for instance by offloading the work to my GPU using turbo-metrics. Also, I'm stuck between wanting to increase the amount of data points in each graphs to get more detailed results and having to keep everything readable. This time around I had to seperate each graph into three quality levels, but that's already too much for my liking. I will look into improving myself on the data presentation front. I feel like I have to streamline this formula to make it more digestible for everyone.

Please, I'm open to your remarks and suggestions to improve on this blog post formula.

That said, here are my plans for future blog posts:

  • a follow-up v2.3.0 article that also revisits the useful SVT-AV1 parameters. I'm planning for this to release before 2025.
  • an article focused on giving you encoding tips and explaining common AV1 encoding knowlegde is still planned for someday.
  • a future article focused on observing the evolution of all software AV1 encoders since the beginning of their development, as well as comparisons with vpxenc, AVM (development ground for AV2) and VVenC.
  • and many more...

Thanks for reading!

Support me by making a donation on my Ko-Fi page, as a reward for my efforts and to compensate for the electricity bills of two whole weeks of non-stop encoding.

]]> video compression benchmarks <![CDATA[Codec Wiki: One Year Later]]> https://av1.wiki/blog/codec-wiki-one-year-later https://av1.wiki/blog/codec-wiki-one-year-later Sat, 20 Jul 2024 00:00:00 GMT Around a month ago, Codec Wiki celebrates it's very first anniversary as it officially hits one year old (Wow, that's a very long time!).

First and foremost, I apologize for this delayed celebration post, I have been very busy for the past month, y'know, real life things. Second of all, I would like to say THANK YOU for everyone who helped build this wiki from the ground up and writing down the entries, you know who you are!

Stats​

Thanks to everyone's efforts, over the course of 12 months Codec Wiki has grown into the (almost) go-to documentation and guide for newcomers just starting out in the quirky and wild world of multimedia encoding. So here are some pretty graphs to look at our progress so far!

Plausible​

As you may know, we run our own Plausible Analytics instance. Which is a self-hosted, open source, and privacy-respecting analytics. Currently hosted in Singapore and available for everyone to view here.

Plausible Analytics

7.7K unique total visitors for a whole year may not be much but it's a great start!

Let's not forget those who use uBlock Origin or similar to block the analytics script. So the visitor count could be two or three times more!

Google Search Console​

For those who are unfamiliar, Google Search Console is used for viewing well.. Google search data for your website. This data is unfortunately private and only the owner and those invited can see the data, no setting available for public view. Probably for the better.

Google Search Console

And for our top 10 search queries in the last 28 days:

Search Queries page 1 Search Queries page 2

With pretty much zero competition in this niche of a market, it skyrocketed Codec Wiki to the top of search results for everything multimedia related 😅.

A Brief History​

Around a year ago, I started work on an unofficial "wiki" for all things encoding, it initially started as a half-serious joke because I was annoyed there were no good documentation available anywhere on the internet on the real application and usage of encoders such as "Should I use tune SSIM in x encoder?". With the search results you'll be presented with being one or more of these:

  • Doesn't exist.
  • Mentioned in the FFmpeg mailing list but nothing else.
  • PDF research papers on IEEE and similar.
  • Outdated information on Doom9 or Multimedia Wiki.
  • "The documentation are in the code blocks".
  • Link rot.
  • Abandoned, or mostly focused on other topics.
  • Focused on a specific type of storytelling medium.
  • Actual good explanation in an old forgotten site (rare).

Some or maybe even most of these results still appear up to this day.

Frustrated with how scarce information is available on the open internet and combined with the fact that most, if not all of them only exist in non-search-engine-reachable private chat communities such as Discord where you'll have to join and scroll around older conversations.

Prone to data erasure as Discord can nuke these servers for whatever reason they please and years of "documentation" is lost in an instant. It is simply not a good platform for a plethora of valuable information.

That's why I took initiative.

Bumps in the Road​

There were also generally a lot of distrust and pushbacks when this project first began, particularly from those who are more experienced with encoding. Which is understandable because everyone who attempted before burned out and never finished it.

But Codec Wiki stood the test of time with constant updates through active collaboration of everyone involved, nowadays those voices have mostly subsided or even joined the project themselves. And I am grateful for that.

Closing Statement​

Once again, THANK YOU VERY MUCH for those who are involved in this project and making the once pipe dream of a one-stop-shop user-friendly documentation for encoding into something that's possible.

Let's continue documenting the most crucial yet invisible driving force of internet traffic, pages by pages.

Together we are strong, divided we burn out.


"There are times when a critic truly risks something, and that is the discovery and defense of the new. The world is often unkind to new talent, new creations. The new needs friends."

— Anton Ego

]]> anniversary history <![CDATA[AV1 for Dummies]]> https://av1.wiki/blog/av1-for-dummies https://av1.wiki/blog/av1-for-dummies Mon, 24 Jun 2024 00:00:00 GMT AV1 for Dummies is a comprehensive, legible guide on how to get started with AV1 at any experience level. Whether you're on Windows using your first video encoding program, or a seasoned Linux user looking to optimize your encoding pipeline, this guide has you covered.

Compare

Introduction​

AV1 is a royalty-free video codec developed by the Alliance for Open Media. It is designed to replace VP9 and presently competes with H.266. AV1 is known for its high compression efficiency, which the marketing will have you believe reduces file sizes by up to 50% compared to H.264 and up to 30% compared to H.265 across the board. It is supported by several major browsers and is widely used across many streaming services and video platforms.

Why AV1?​

Before we dive in, it is important to understand why you may want to use AV1 instead of other codecs. The reality is that AV1 is not better than H.264/5 in every single scenario; video encoding is a complicated field, and the best codec for you will depend on your specific needs. AV1 excels in:

  • Low to medium-high fidelity encoding
  • Higher resolution encoding
  • Encoding content with very little grain or noise
  • Slow, non-realtime contexts (e.g. offline encoding)

The enumeration above still consists of broad strokes, but the point is to understand that AV1 is not a silver bullet. It will not automatically make your videos smaller while preserving your desired quality. To make things more difficult, the x264 & x265 encoders are very mature, while AV1 encoding efforts designed to meet the extremely complicated needs of the human eye are still in their infancy.

This guide focuses almost entirely on tools great for offline, non-realtime encoding for various kinds of content. Streaming is an altogether different application of AV1, and is only covered sparsely here.

Do's & Don'ts​

Due to a lot of misunderstandings about codecs and compression, there are a lot of common misconceptions that are held regarding video encoding. We'll start by outlining some bad practices:

  • Don't encode the same video multiple times. This is a common mistake made by people new to video encoding. Every time you encode a video, you lose additional quality due to generation loss. This is because video codecs are lossy, and every time you encode a video, you lose more information. This is why it is important to keep the original video file if you frequently re-encode it.
  • Don't blindly copy settings from others without understanding them. What works for one person's content and workflow may not work for yours. Even the default settings on many encoders are not optimal for most content.
  • Don't assume that higher bitrate equates to better quality. Inefficient encoding can waste bits without improving visual quality, and efficient encoding can make lower bitrate video look drastically better than higher bitrate video using the same codec.
  • Don't assume all encoders/presets/settings/implementations are created equal. Even given two encoding frameworks that use the same underlying encoder, you may achieve different results given encoder version mismatches or subtly different settings used under the hood.
  • Don't use unnecessarily slow presets/speeds unless you have a specific need and ample time. While slower presets improve encoding efficiency most of the time, the quality gains reach a point of diminishing returns beyond a certain point. Use the slowest preset you can tolerate, not the slowest preset available.
  • Don't blindly trust metric scores. It is unfortunate how trusted VMAF is considering how infrequently it correlates with visual fidelity in practice now that it has become so popular. Even the beloved SSIMULACRA2 is not a perfect one-to-one with the human eye.

Now, let's move on to some good practices:

  • Experiment with different settings and compare the results.
  • Consider your content type when choosing encoding settings. Film, animation, and sports all have different characteristics that benefit from distinct approaches.
  • Try to use CRF for offline encoding, as opposed to CBR or VBR. While the latter two are effective for precisely targeting a particular bitrate, CRF is more effective at targeting a specific quality level efficiently.
  • Always use 10-bit color, even with an 8-bit source. AV1's internal workings are much more suited to 10-bit color, and you are almost always guaranteed quality improvements with zero compatibility penalty as 10-bit color is part of AV1's baseline profile.
  • Consider using grain synthesis for grainy content, as AV1 can struggle with preserving film grain efficiently.
  • Keep your encoding software up-to-date; the encoding world moves quickly.

Tools​

Since writing the last guide, there are now several versatile tools available for AV1 encoding. Below, we've assembled some tables of popular encoding tools and their features.

GUI​

Here's the key:

  • Encoder(s): The AV1 encoder(s) the software uses or is able to use. This is between SVT-AV1, aomenc, rav1e, SVT-AV1-PSY, and various aomenc forks. We'll get into this more later.
  • Ease of Use: How intuitive the software is, especially for beginners.
  • Complexity: How flexible the software can be for advanced users.
  • Efficiency: How "good" the underlying encoder(s) are. This is more subjective, but tools with bad defaults, misleading options, or slow release cycles won't score well here.
NamePlatform(s)Encoder(s)Ease of UseComplexityEfficiency
AviatorLinuxSVT-AV1-PSY●●●●●●○○○○●●●●●
NMKODERWindowsAny●●●○○●●●●○●●○○○
Av1ation StationAnyAny●●●○○●●●●●●●●●○
StaxRipWindowsAny●●○○○●●●●●●●●●○
NEAV1EWindowsAny●●●○○●●●○○●●○○○
HandbrakeAnySVT-AV1●●●●○●●○○○●●○○○
FastFlixAnyAny●●●●○●●●○○●●○○○
rAV1atorLinuxrav1e●●●●○●●●○○●○○○○
AutocompressorWebSVT-AV1●●●●●●○○○○●●●○○

Our top pics are:

  • For beginners: Aviator. If you are on Linux and you want zero hassle whatsoever, Aviator is simple but highly effective. As a Flatpak application, it ships a bundled SVT-AV1-PSY binary that is up-to-date, compiled efficiently, and handled with extensively tested defaults. It is hard to go wrong with Aviator even as an advanced user, despite its simplicity.
  • For advanced users: Av1ation Station. Av1ator Station is thoughtfully designed to be your one-stop shop for video encoding. Even though it is brand new, it has proven itself to be a capable and reliable solution for advanced video encoders that have a lot of settings to tweak and encodes to keep track of.
  • For Windows users: StaxRip. StaxRip is a long-standing, well-maintained, and highly flexible video encoding tool that supports a plethora of encoders and formats. It comes bundled with SVT-AV1-PSY and is a great choice for Windows users who want a powerful and versatile tool for video encoding.

CLI​

In the terminal, advanced encoders can explore a much greater degree of complexity than many GUIs allow. That being said, ease of use is still a consideration when managing complex command-line encoding workflows. Let's begin with the key:

  • We're omitting the "Platform(s)" column here, as this section targets Linux users. As a Windows user, WSL has you covered - as a Mac user, you're likely to experience parity with Linux (most of the time).
NameFramework(s)Ease of UseComplexityFeaturesScene Detection
SvtAv1EncAppNone●○○○○●●○○○●○○○○No
FFmpegItself●●○○○●●●●●●●●●○No
Av1anFFmpeg●○○○○●●●●●●●●○○Yes
rAV1ator CLIAv1an●●●●○●●●●○●●●●○Yes
alabamaEncoderFFmpeg●●○○○●●●●●●●●●●Yes

Our top picks are:

  • For beginners: rAV1ator CLI. rAV1ator CLI essentially walks you through the process of writing Av1an commands. It can install binaries for you, save your previous commands, and detect scenes via Av1an. This makes it a great choice for beginners who want to learn the ropes of AV1 encoding without diving into the deep end headfirst. For advanced users, it makes rapidly writing, testing, & cataloging Av1an encoding commands much simpler.
  • For advanced users: alabamaEncoder. alabamaEncoder is a powerful and flexible tool that allows you to encode video with FFmpeg and AV1. It is highly configurable and supports a mind-bending array of powerful features that would be hard to find elsewhere. While the tool is still new, it is already a great choice for advanced users who want to push the boundaries of what is possible to incorporate into an encoding workflow.

Conclusion​

In conclusion, the best AV1 tool for you will depend on your needs and experience level. If you are a beginner, Aviator is a great choice for Linux users, while StaxRip is a good option for Windows users. For advanced users, Av1ation Station is a powerful and versatile tool that can handle numerous encoding tasks. If you prefer the command line, rAV1ator CLI is a great choice for beginners, while alabamaEncoder is a powerful tool for advanced users. No matter which tool you choose, it is hard to go wrong in the modern AV1 encoding landscape when it comes to your utility of choice.

Encoders​

The world of AV1 encoding is diverse and complex, with several open-source encoders available, each bringing its own set of strengths, weaknesses, and unique features to the table. In this section, we'll dive deep into the characteristics of four major AV1 encoders:

  1. SVT-AV1,
  2. rav1e,
  3. aomenc (libaom), and
  4. SVT-AV1-PSY

Understanding these encoders is crucial for making informed decisions about what best suits your specific encoding needs.

SVT-AV1​

SVT-AV1 (Scalable Video Technology for AV1) is an AV1 encoder library and application developed by Intel, Netflix, and others. It has gained significant popularity in the encoding community due to its impressive balance of speed, quality, and scalability.

Links:

  1. Performance & Scalability

    • SVT-AV1 is renowned for its encoding speed, particularly at higher speed presets.
    • It leverages parallel processing, making it exceptionally efficient on multi-core systems. Fun fact: SVT-AV1's parallel processing is lossless, so it doesn't compromise quality for speed.

  2. Quality-to-Speed Ratio

    • SVT-AV1 strikes an impressive balance between encoding speed and output quality.
    • At faster presets, it usually outperforms other encoders in quality per unit of encoding time.
    • While it may not achieve the absolute highest quality per bit possible, its quality is generally considered impressive for its speed.

  3. Flexibility

    • SVT-AV1 offers a wide range of encoding options and presets, allowing fine-tuned control over the encoding process.
    • It provides 14 presets (0-13), with 0 being the slowest and highest quality, and 13 being the fastest but lowest quality.
    • Advanced options allow users to adjust parameters like hierarchical levels, intra-refresh type, and tuning modes.

  4. Continuous Development

    • SVT-AV1 receives frequent updates and optimizations, with new releases often coming alongside big changes.
    • The open-source nature of the project encourages community contributions and rapid feature development.

SVT-AV1 is an excellent choice for a wide range of encoding scenarios. It's particularly well-suited for:

  • High-volume encoding operations where speed is crucial
  • Live or near-live encoding of high-resolution content
  • Scenarios where a balance between quality and encoding speed is required
  • Users with multi-core systems who want to leverage their hardware efficiently

Some downsides include:

  • Higher memory usage compared to other encoders
  • The developers assess quality via its performance on traditional legacy metrics, which harms its perceptual fidelity ceiling.

rav1e​

rav1e is an AV1 encoder written in Rust & Assembly. Developed by the open-source community alongside Xiph, it brings a unique approach to AV1 encoding with its focus on safety and correctness.

Links:

  1. Safety & Reliability

    • Being written in Rust, rav1e emphasizes memory safety and thread safety.
    • This focus on safety translates to a more stable and reliable encoding process, with reduced risks of crashes or undefined behavior.

  2. High Fidelity

    • At high fidelity targets - an area where AV1 usually lacks - rav1e is a strong contender compared to other encoders.
    • It excels in preserving fine details and textures, making it a good choice for high-fidelity encoding.

  3. Quality

    • While not typically matching aomenc or SVT-AV1 in pure compression efficiency, rav1e can produce high-quality output videos.
    • It often achieves a good balance between quality and encoding time, especially at medium-speed settings.

  4. Perceptually Driven

    • rav1e's development is driven by visual fidelity, without relying heavily on metrics.
    • This focus on perceptual quality leads to a stronger foundation for future potential improvements in visual quality, as well as making the encoder very easy to use as it does not require excessive tweaking.

rav1e is well-suited for:

  • Projects where stability is paramount
  • Users who prioritize a community-driven, open-source development approach
  • Encoding tasks where a balance between quality and speed is needed, but the absolute fastest speeds are not required

Some limitations of rav1e include:

  • Lagging development compared to other encoders
  • Slower encoding speeds compared to SVT-AV1 at similar quality & size
  • Fewer advanced options compared to other encoders

aomenc (libaom)​

aomenc, based on the libaom library, is the reference encoder for AV1. Developed by the Alliance for Open Media (AOM), it is the benchmark for AV1 encoding quality and compliance.

Links:

  1. Encoding Quality

    • aomenc is widely regarded as the gold standard for AV1 encoding quality.
    • It often achieves high compression efficiency among AV1 encoders, especially at slower speed settings.
    • The encoder squeezes out nearly every last bit of efficiency from the AV1 codec, making it ideal for archival purposes or when quality per bit is critical.

  2. Encoding Speed

    • aomenc is generally the slowest among major AV1 encoders.
    • It offers 13 CPU speed levels (0-12), but even at its fastest settings, it's typically slower than other encoders at their slower settings.
    • The slow speed is often considered a trade-off for its high compression efficiency.

  3. Extensive Options

    • As the reference implementation, aomenc offers the most comprehensive encoding options.
    • It provides fine-grained control over nearly every aspect of the AV1 encoding process.
    • Advanced users can tweak many parameters to optimize for specific content types or encoding scenarios.

  4. Flexibility

    • Being the reference encoder, aomenc produces highly standards-compliant AV1 bitstreams that take advantage of the full arsenal of AV1 features.
    • It supports 4:2:0 and 4:4:4 chroma subsampling, 8- to 12-bit color depth, and various other advanced features that more specialized encoders like SVT-AV1 do not support.

aomenc is ideal for:

  • Scenarios where achieving the highest possible quality is the primary goal
  • Archival encoding where compression efficiency is crucial
  • Research and development in video compression
  • Encoding projects where encoding time is not a significant constraint

Some drawbacks of aomenc include:

  • Unresponsive development driven by legacy metrics, leading to slower adoption of new techniques and ignoring improvements communicated by people outside the Google development team
  • Cripplingly difficult to use for beginners, with a culture of cargo-culting settings
  • Slow encoding speeds compared to other AV1 encoders, which has less of an impact on the quality of the output than it used to compared to maturing encoders like SVT-AV1

SVT-AV1-PSY​

SVT-AV1-PSY is a community fork of the SVT-AV1 encoder focused on psychovisual optimizations to enhance perceived visual quality. It aims at closing the distance between SVT-AV1's high speeds and the perceptual quality of aomenc's slow brute force approach.

Links:

  1. Perceptual Optimizations

    • SVT-AV1-PSY introduces various psychovisual enhancements to improve the perceived quality of encoded video.
    • These optimizations often result in output that looks better to the human eye, even if it might not always score as well in objective metrics.

  2. Additional Features

    • Introduces new options like variance boost, which can help maintain detail in high-contrast scenes.
    • Offers alternative curve options for more nuanced control over the encoding process.
    • Extends the CRF (Constant Rate Factor) range to 70 (from 63 in mainline SVT-AV1), allowing for extremely low-bitrate encodes.
    • Introduces additional tuning options, including a new "SSIM with Subjective Quality Tuning" mode that can improve perceived quality.

  3. Visual Fidelity Focus

    • Aims to produce more visually pleasing results, sometimes at the expense of metric performance.
    • Includes options like sharpness adjustment and adaptive film grain synthesis which can significantly impact the visual characteristics of the output.
    • Features modified defaults driven by perceptual quality considerations.

  4. Extended HDR Support

    • Includes built-in support for Dolby Vision & HDR10+ encoding.
    • This makes it particularly useful for encoding HDR content without requiring additional post-processing steps or external tools.

  5. Performance

    • Based on SVT-AV1, it retains the performance characteristics of its parent encoder.
    • Adds super slow presets (-2 and -3) for research purposes and extremely high-quality encoding. These additional presets can be useful for creating reference encodes or applications where encoding time is not a concern.

SVT-AV1-PSY is particularly well-suited for:

  • Encoding scenarios where subjective visual quality is prioritized over pure metric performance
  • HDR content encoding in Dolby Vision or HDR10+
  • Users who want fine-grained control over psychovisual aspects of encoding
  • Projects that require a balance between the speed of SVT-AV1 and enhanced visual quality
  • Encoding challenging content with complex textures or high-contrast scenes

Some drawbacks are:

  • Everything that applies to SVT-AV1, including the lack of support for 4:4:4 chroma subsampling and 12-bit color depth that are useful in specific scenarios

Conclusion​

While SVT-AV1 is known for being fast, aomenc is renowned for its high-quality output, and rav1e is recognized for its safety and reliability, each encoder has strengths and weaknesses. The best encoder for you will depend on your specific needs and priorities.

As this guide is focused on offline encoding, SVT-AV1-PSY combines aomenc's traditional perceptual strength with SVT-AV1's speed. Like rav1e, it is easy to use due to strong default settings that prevent cargo culting, and overall it can be considered the best of all three worlds. It is actively developed by a team of responsive community members, and it is a great choice for most users who want a balance between quality per bit, speed, and ease of use.

The "best" encoder often depends on your use case, content type, and target audience. It's always worth experimenting with different encoders and settings to find the optimal balance for your needs. Many advanced users even employ multiple encoders in their workflows, choosing the most appropriate tool for each specific task or content type.

Final Conclusion​

The AV1 ecosystem has grown to the point where a single comprehensive guide cannot effectively cover the entire extent of the available tools and techniques for AV1 encoding across every use case. If you want more detail about a particular tool, where to acquire it, or how to compile an encoder, you can find that information throughout the various wiki entries linked on this page.

Reading can get you far, but it is natural to have questions. Please don't hesitate to connect with the team behind the Codec Wiki and many of these tools via our AV1 for Dummies Discord server. We're happy to help you with anything you need, and your questions and feedback help the wiki grow and improve. We hope you enjoy your journey into AV1 encoding, and we wish you the best of luck in your encoding endeavors!

]]> video compression <![CDATA[Observing SVT-AV1 v2.1.0's improvements: A New Deep Dive]]> https://av1.wiki/blog/svt-av1-second-deep-dive https://av1.wiki/blog/svt-av1-second-deep-dive Sun, 19 May 2024 00:00:00 GMT SVT-AV1, the most scalable AV1 encoder, has received a new update and one may wonder if the old presets recommendation still holds today. We will delve into that in this blog post, based on a series of speed and visual quality benchmarks with SSIMULACRA2 and XPSNR of SVT-AV1 2.1.0 on a corpus of varied animated clips.

Feedback​

I unfortunately never got to update the previous blog post with the image comparisons, and some people expressed concerns that this testing may not be representative of live action content. For the former, I will have to ask for your patience again, because this blog post won't contain image comparisons either, but they are being actively worked on. As for the latter, please be reassured that this testing in its entirety is perfectly representative of any modern content people typically encode: the diversity of japanese animation is rich and the content specifically chosen for this benchmark is relatively complex. From 3DCG to extremely noisy clips, we are far from the easy-to-compress static scenes of some slice-of-life show.

I have also decided to complement this benchmark of another psychovisually-driven metric (XPSNR) so that double-checking is made easier. Thus, each graph possesses a SSIMULACRA2 version and a XPSNR version. Don't hesitate to switch between one another!

Methodology​

The resources available will range from graphs to image comparisons (WIP). The former has the advantage of being easily understandable, showcasing pure efficiency comparisons between encoder parameters using metrics as the reference, while the latter are image samples from the files encoded during the tests that enable you to check quality for yourself, adding another layer of subjective interpretation to these comparisons.

The testing methodology involves using relatively short video samples from a wide range of modern anime genre, which have been either losslessly encoded with x264 --qp 0 for ease of use or losslessly cut from their source. These lossless files are then pipped into SvtAv1EncApp directly, meaning we are measuring the performance of a single encoder instance and not leveraging chunked encoding like any actual final AV1 encoding pipeline should. Once an encode is done, SSIMULACRA2 scores are calculated using the Zig implementation, XPSNR scores on the other hand are calculated using a ffmpeg filter, and lots of useful data are aggregated to make the graphs for this benchmark, including encoding time, encode size (bitrate), and metrics scores. Bits per pixel scores (BPP) are calculated so that the Metric / BPP graphs may represent the closest we have to real efficiency.

The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy.

SvtAv1EncApp was compiled directly from the v2.0.0 and v2.1.0 source code using the provided Build/linux/build.sh script, Clang 16.0.6, and Profile-Guided Optimization (PGO). The testing machine is comprised of an i3 12100 with 16GB of 3200MHz CL14 DDR4 RAM in Arch Linux with kernel 6.7.7 and the performance governor enabled. All encodes have been made in the same session without rebooting.

Samples​

The samples are as follows:

  • 11s Blame! clip which sports 3DCG action with lots of grain, effects and high-contrast elements.
  • 13s Blue Lock clip which sports rapid camera movements, complex geometry and high-contrast elements.
  • 5s Spy x Family first ending sequence with an extremely high amount of dynamic noise. New most complex source of this set.
  • 12s Jigokuraku (Hell's Paradise) flashback clip with huge static grain in a very dark scenery and some action.
  • 5s The Garden of Sinners clean but fast-paced 3DCG scene with explosions.

The resolution of every clip is 1080p, except for the first one which is 1920x804.

All clips have been encoded in a wide quality range, from --crf 6 to --crf 46, by increments of 4.

Without further ado, let's start with the first comparisons!

Presets comparisons (-1 -> 13 12)​

In the following graphs, you may find comparisons between all SVT-AV1 presets, ranging from the slowest --preset -1 to the fastest --preset 12.

Yes, you heard that right. Preset 7 and 13 are no more in v2.1.0. This new update, like the previous one, mostly consisted of optimizing the presets trade-offs. The devs have made the choice to map preset 7 to preset 6 and preset 13 to preset 12 due to the lack of spacing between the new presets. We will discuss the implications of this further ahead.

--preset X is the only parameter used here, in conjunction with the CRF values. That means everything else is default. The defaults worth mentioning are:

  • --tune 1: tune PSNR
  • --aq-mode 2: variance deltaq
  • --enable-qm 0: quantisation matrices disabled
  • --irefresh-type 2: closed GOP
  • --enable-tf 1: temporal filtering enabled

And more, like CDEF and restoration enabled, overlays and film-grain disabled...

Efficiency​

  • First of all, here are the full efficiency graphs:
Preview

This is all very cool, but visually bloated.

  • Now the same graphs but focusing on the "high quality" range (CRF6 -> 22):
Preview
  • Same, but now focusing on the "low quality" range (CRF26 -> 46):
Preview
  • If we now focus on presets 4 and below, where it's more difficult to discern the differences between presets, we get this at "high quality":
Preview
  • And the following at "low quality":
Preview

Speed​

  • Let's now see speed comparisons between all presets:
Preview

Once is not custom, preset -1 is so abysmally slow it makes the graph unusable.

  • Same, but without the placebo preset -1:
Preview
  • Lastly, here is what it looks like with a logarithmic scale:
Preview

Interpretation​

As for interpreting the results, it would seem like preset 2 and preset 4 remain all-around very balanced presets, with preset 3 being in a nice in-between spot, an improvement over v2.0.0's preset 3 in a way. We will better understand the reasons for this when we'll compare the new version to the last, a bit after.

The quality gap between preset 2 and preset 1 is usually pretty narrow, however the speed penalty from going to preset 1 is ~2x, when the penalty of going from preset 3 to preset 2 is closer to ~1.5x. As such, preset 1 enters placebo territory, and considering the very little benefits of going any lower than it, compared to the huge performance loss of even lower presets, I advise you not to waste encoding resources on preset 0 and preset -1. This applies especially at medium to high quality, however at extremely low quality like the CRF40 range, we can still see some small gains from these placebo presets.

When we start talking about faster presets though, things are pretty different from previous versions: presets 5 to 9 behave similarly on the graphs and seem to stand apart from their slower counterparts by just a bit. If you can bear the speed of preset 4, you should definitely be going for it, however if fast encoding is a necessity, for example in the case of realtime transcoding or streaming, presets 5 through 9 will serve you right with great efficiency/speed trade-offs between one another. No preset in that range particularly stands out from the others, so simply pick one depending on your performance needs.

Presets 10 to 12 are pretty inefficient, and to be avoided if possible. They can still be of use in a convex-hull scenario, but in the case of realtime transcoding, you may be better off with a hardware encoder like the ones found in RTX 4000 or Arc GPUs, especially since SVT-AV1's target bitrate mode is even less efficient than CRF mode.

TLDR​

The same conclusions as the previous blog post can be made: clear quality gains can be observed as we decrease presets, until preset 2, however the effectiveness of dropping presets is noticeably less and less important as quality is increased.

In the next part, we will evaluate the differences in efficiency and speed of every presets when updating from SVT-AV1 2.0.0 to 2.1.0, which should enable an increase of nuance from the previous results alone.

SVT-AV1 v2.0.0 vs v2.1.0 presets comparisons:​

Two months ago, I conducted a similar test to this one to compare the presets evolution between versions 1.8.0 and 2.0.0. The results were pretty unsatisfying: I noticed that presets -1 to 8 in v2.0.0 performed like the old presets 0 to 9 did in v1.8.0. We basically saw an efficiency regression at a given preset, and speedups did not follow suit as well as we would have anticipated. All in all, it wasn't all that bad, it suffice to say you could simply drop a preset from before and you were good to go again. What was more concerning however is that the release note claimed important speedups that did not impact efficiency and my testing proved otherwise. My theory is that due to the dev team testing methodology, which consist of mostly pretty low resolution clips and non-psychovisual metrics like PSNR, SSIM or bad psychovisual metrics like VMAF, it's very well possible they were tricked into thinking they introduced improvements as they tweaked the presets when in reality the metrics simply didn't notice the quality degradation. Such issue is an additional reason why the industry should adopt more competent metrics, ones that better correlate with the human vision, to improve encoders in more impactful ways and better avoid pointless regressions.

So the question for today's testing is: have the SVT-AV1 devs redeemed themselves and actually improved the presets trade-offs this time around? Let's find out!

preset -1: v2.0.0 vs v2.1.0​

  • Let's start off with a battle of the placebos, with the efficiency at "high quality":
Preview
  • And the efficiency at "low quality":
Preview

Yes, this is a bit underwhelming, but you can't just improve the best an encoder has to offer with just tweaking right?

  • Now, let's compare their respective speeds:
Preview

Let's be grateful it became ever so slightly faster, I guess.

preset 0: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

Overall, efficiency wise, this new preset 0 places itself in-between old preset -1 and 0

  • Speed graphs:
Preview

Interestingly enough, its speed is much closer to the old preset 0 than to the old preset -1. This means preset 0 was genuinely improved over v2.0.0!

preset 1: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

In efficiency, this new preset 1 is often equal to old preset 0, else in-between old preset 0 and 1.

  • Speed graphs:
Preview

We observe that the new preset is a bit closer to old preset 1 speeds than it is to old preset 0 speeds. Good news!

preset 2: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

Oh well, that's awkward.

  • Speed graphs:
Preview

Speed was left untouched too, meaning preset 2 is unchanged in v2.1.0.

preset 3: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

The new preset 3's efficiency is the same as the old one.

  • Speed graphs:
Preview

However, the preset got slightly faster, so this is a speedup!

preset 4: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

We can observe that preset 4 got slightly to moderately worse efficiency wise.

  • Speed graphs:
Preview

Fortunately, the consequence of that slight efficiency decrease is a big performance improvement!

preset 5: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

Preset 5 seems to have gotten ever so slightly worse efficiency wise.

  • Speed graphs:
Preview

Yet it became slightly faster, this is overall a good trade-off.

preset 6: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

The new preset 6 has a huge responsibility: being able to compensate in the absence of its preset 7 sibling. It seems to performs in-between old preset 6 and 7, usually closer to old 7.

  • Speed graphs:
Preview

Preset 6 is now ever so slightly slower to old 7, this is an interesting trade-off, overall a win over old 7.

preset 7: v2.0.0 vs v2.1.0​

Again, there is no preset 7. Actually, it's preset 6 that disappeared but I'm not remaking the graphs just for fun. If you select preset 6, you will be granted the following message: Svt[warn]: Preset M6 is mapped to M7.

always_has_been

preset 8: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

In efficiency, this new preset 8 is sometimes equal or slightly worse to the old 8, and sometimes equal or slightly worse than old 7...

  • Speed graphs:
Preview

Overall, the speed is pretty much unchanged from old 8. It looks like a slight regression, that's pretty disappointing.

preset 9: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

The new preset 9 is the same as ever, ever so slightly better in some scenario but nothing groundbreaking.

  • Speed graphs:
Preview

Its speed remains the same, sometimes ever so slightly slower. Basically the preset is pretty much unchanged, which may as well be a relief, as the last usable preset of the encoder.

preset 10: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

Preset 10 is slightly to moderately worse efficiency wise.

  • Speed graphs:
Preview

Its speed is mostly the same, sometimes ever so slightly faster. It's a wash, avoid this preset at all costs!

preset 11: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

Preset 11's efficiency is untouched.

  • Speed graphs:
Preview

Preset 11's speed is unchanged as well.

preset 12: v2.0.0 vs v2.1.0​

  • Efficiency graphs, high quality:
Preview
  • Efficiency graphs, low quality:
Preview

Just as preset 6, preset 12 is now mapped to 13, and unsurprisingly, its efficiency is equal to old 13.

  • Speed graphs:
Preview

The speeds seem to be in-between old 12 and 13, so potentially a slight speedup. Still, nothing to get excited at.

TLDR​

From these extensive comparisons, it appears that some presets have received genuine improvements in their respective efficiency/speed trade-off. Some presets, like -1, 0, 1 and 3, received the most significant improvements, followed by 4, 5 and 6 with overall beneficial new trade-offs. Preset 12 got slightly faster too. On the other hand, presets 8 and 10 seemed to have regressed slightly, and presets 2 and 9 are perfectly unchanged from v2.0.0.

Conclusion​

SVT-AV1 2.1.0 introduced some welcomed improvements. Presets 2 through 4 remain the king of optimal AV1 encoding, while presets 5 through 9 stand as good options for the people that find 2-4 to be too slow for their liking.

Let's be honest a second, not much as changed in SVT-AV1 since the first blog post. There was no need to redo all the parameter testing for the simple reason that their behavior remained the same, as did the conclusions drawn from them. I hope this article wasn't disappointing in a sense... Still, be reassured, this was just an appetizer, there will be more in the near future!

By the way, did you know that the SVT-AV1-PSY project was initiated a few months ago? Its defaults were tailored according to the testing done in the last blog post, allowing a free efficiency boost for anyone not keen to tweak their encoders. Furthermore, SVT-AV1-PSY introduced a sharpness parameter to control distortion, a quarter-step quantizer for more CRF precision, a new subjective SSIM tune, Dolby Vision support, frame luma bias, and some other knobs to improve the appeal and consistency of your encodes. It is actively maintained by a group of talented people, including the main dev of the aom-av1-lavish fork of aomenc. Some of the changes are being backported to mainline SVT-AV1 due to the increased interest of the mainline devs. Please check it out!

Hopefully, this comprehensive second deep dive should give you a helpful new starting point for choosing settings when encoding with the latest SVT-AV1(-PSY) 2.1.0.

Future​

My plans for the future regarding the blog post include:

  • polishing this blog post.
  • a follow-up article in the relatively near future about giving you encoding tips and explaining common AV1 encoding knowlegde, for instance showcasing why film grain synthesis is a game-changer or why chunked encoding can prove beneficial to your encoding pipeline.
  • an article focused on observing the evolution of SVT-AV1 since the beginning of its development, as well as comparisons with current aomenc, rav1e and SVT-AV1-PSY, including a quick look at the current state of AVM (development ground for AV2) in comparison to VVC's state.

Thanks for reading!

]]> video compression benchmarks <![CDATA[Encoding Animation with SVT-AV1: A Deep Dive]]> https://av1.wiki/blog/svt-av1-deep-dive https://av1.wiki/blog/svt-av1-deep-dive Sat, 30 Dec 2023 00:00:00 GMT This blog post is based on a series of visual quality benchmarks with SSIMULACRA2 and speed benchmarks of SVT-AV1 1.8.0 on a corpus of animated clips.

The resources available will range from graphs to image comparisons (WIP). The former has the advantage of being easily understandable, showcasing pure efficiency comparisons between encoder parameters using metrics as the reference, while the latter are image samples from the encoded files during the tests that enable you to check quality for yourself and add another layer of subjective interpretation to these comparisons.

The testing methodology involves using relatively short video samples from a wide range of modern anime genre, which have been either losslessly encoded with x264 --qp 0 for ease of use or losslessly cut from their source. These lossless files are then pipped into SvtAv1EncApp directly, meaning we are measuring the performance of a single encoder instance and not leveraging chunked encoding like any actual final AV1 encoding pipeline should. Once an encode is done, SSIMULACRA2 scores are calculated using the Zig implementation and lots of useful data are aggregated to make the graphs for this benchmark, including encoding time, encode size (bitrate), and SSIMULACRA2 scores. Bits per pixel scores (BPP) are calculated so that the Metric / BPP graphs may represent the closest we have to real efficiency.

The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy.

SvtAv1EncApp was compiled directly from the v1.8.0 source code using the provided Build/linux/build.sh script, Clang 16.0.6, and Profile-Guided Optimization (PGO). The testing machine is comprised of an i7 8750H running at 35W with 16GB of 2666MHz DDR4 RAM in Arch Linux with kernel 6.6.6 and the performance governor enabled. All encodes have been made in the same session without rebooting.

This testing was conducted within the AV1 Weeb Edition Discord server, which is focused on encoding animated content in AV1.

Samples​

The samples are as follows:

  • 11s Blame! clip which sports 3DCG action with lots of grain, effects and high-contrast elements. Most complex source of this set.
  • 13s Blue Lock clip which sports rapid camera movements and rotations + high-contrast elements.
  • 15s Fate/Grand Order: Babylonia relatively slow-paced clip with lots of effects still. Easiest source of this set but easy sources still give interesting data.
  • 22s Jigokuraku (Hell's Paradise) flashback clip with huge static grain in a very dark scenery and some action.
  • 14s Kaguya-sama opening sequence with lots of effects and fast change of scenery. The resolution of every clip is 1080p, except for the first one which is 1920x804.

All clips have been encoded in a wide quality range, from --crf 8 to --crf 43.

Without further ado, let's start with the first comparisons!

Presets comparisons (-1 -> 13)​

In the following graphs, you may find comparisons between all SVT-AV1 presets, ranging from the slowest --preset -1 to the fastest --preset 13. Please remember that these two extreme presets are meant for development purposes and as such should not be used in normal encoding conditions. You will soon understand why.

--preset X is the only parameter used here, in conjunction with the CRF values. That means everything else is default. The defaults worth mentioning are:

  • --tune 1: tune PSNR

  • --aq-mode 2: variance deltaq

  • --enable-qm 0: quantisation matrices disabled

  • --irefresh-type 2: closed GOP

  • --enable-tf 1: temporal filtering enabled And more, like CDEF and restoration enabled, overlays and film-grain disabled...

  • First of all, here are the efficiency graphs:

Preview

This could be too much information.

  • Now the same graphs but focusing on the "high quality" range (CRF8 -> 23):
Preview
  • Same again but without presets 9 to 13 for better clarity:
Preview
  • Now for the "low quality" range (CRF28 -> 43):
Preview
  • Same but without presets 9 to 13 for better clarity:
Preview
  • Let's now see speed comparisons between all presets:

speed_overall

As we can see, preset -1 is so abysmally slow it makes the graph unusable (BTW, notice the 1e6 in the lower right corner, it is obviously not encoding at 1 to 4 ms, but at 1 000 000 to 4 000 000 ms)

  • Here is what it looks like with a logarithmic scale:
Preview
  • Now the speed graphs but with SSIMU2 on the y-axis instead of BPP: (logarithmic scale)
Preview
  • Here are speeds graphs for preset 1 to 6 with a linear scale:
Preview

One interpretation we can have is that presets 2 to 4 have actually pretty close scores (pretty much the same at HQ, 2 points at max in the low quality range) but preset 2 is 2x slower than preset 4. The quality gap between preset 2 and preset 1 is even narrower but the speed penalty is also ~2x.

That makes it hard to recommend preset 1, while preset 3 is in a strange middle ground between 2 and 4 where it makes little sense to choose it over those two: it's better to choose preset 2 for the higher efficiency at 1.5x slower speeds, or preset 4 for the 1.33 to 1.5x higher speeds. At least the option exists.

A similar observation can be made between preset 5 and 6. They can be so close in scores in many samples (especially at HQ) while also being close in speeds that preset 5 becomes rather obsolete most of the time.

TLDR​

Clear quality gains can be observed as we decrease presets, until the very last one, however the effectiveness of dropping presets becomes less and less impressive the higher in quality you go.

  • For instance, in worst-case scenario, we observe that (for the CRF23 to CRF8 range), preset 4 only loses at maximum 2 SSIMU2 points compared to preset -1 while being 50-60x faster. Though to be fair, the speed loss from preset 4 to preset 0 is "only" 5-10x for a maximum SSIMU2 difference close to 1.5 point.

  • From CRF43 to CRF28, the difference between preset 4 and preset -1 can be as much as 5 SSIMU2 points, so lower presets become more attractive.

Tunes comparisons​

In the following graphs, you may find comparisons between SVT-AV1 tunes, from the default --tune 1 (PSNR) to the other two tunes: --tune 0 (VQ) and --tune 2 (SSIM).

Except for the tunes, --preset 4 is set due to its good balance of quality and speed, in conjunction with the CRF values. That means everything else is default. The defaults have been mentioned earlier above.

  • Let's compare the efficiency of every tunes:
Preview
  • Now let's focus on the "high quality" range (CRF8 -> 23):
Preview
  • And the "low quality" range (CRF28 -> 43):
Preview
  • And here is the speed difference:
Preview
  • Graphs comparing the tunes individually between each others will be made available soon.
  • The image comparisons will make the conclusion quite more nuanced, stay tuned (heheh) for that.

TLDR​

SSIMU2 favors tune 1 and 2 above tune 0. At high quality, tune 1 and 2 are matched, while at low quality tune 2 takes the edge. Tune 0 is sometimes a match for the other two on certain clips and other times fare pretty badly. For some reason, tune 0 is quite a bit faster now, compared to 1 and 2 which are basically the same.

Keep in mind that I have observed multiple times in the past that tune 0 kept more fine detail and was sharper than the other tunes, at the expense of potential artifacting, like ringing and distortion. It may very well explain why the metric doesn't like its results. The image comparisons may give a different interpretation than what we concluded here, please stay tuned for these subjective comparisons.

Parameters comparisons​

In the following graphs, you may find comparisons between many SVT-AV1 parameters. Additional graphs focusing on the high and low qualities will be made available later down the line

--preset 4 is used here due to its good balance of quality and speed, in conjunction with the CRF values. That means everything else is default. The defaults have been mentioned earlier above.

--tile-rows 1 --tile-columns 1 vs default --tile-rows 0 --tile-columns 0​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

tiles here are both slightly harmful and slower.

--aq-mode 0 vs default --aq-mode 2​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Except for the Jigokuraku clip, aq-mode 0 is harmful in the eyes of SSIMU2, while being slower at low CRF levels, and sometimes a match or faster at high CRF levels.

--aq-mode 1 vs default --aq-mode 2​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

aq-mode 1 fares closer to aq-mode 2 than aq-mode 0 did, both in quality and speed, but is still overall inferior according to SSIMU2

--enable-cdef 0 vs default --enable-cdef 1​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

According to SSIMU2, disabling CDEF barely impact efficiency. But as its a pretty demanding tool, there's a slight speed benefit of having it disabled too. I advise you to take these results with a grain of salt until the image comparisons, because in anime particularly, CDEF can be beneficial for the line-art.

--enable-dg 0 vs default --enable-dg 1​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Dynamic GoP control yields bit-perfect results in all clips except for Blue Lock and Jigokuraku. There is no speed benefit to disabling it except in clips where it is in use. Let's not jump to conclusions too easily, the image comparisons will tell if it's "safe" to keep the setting disabled at all times or not.

--enable-dlf 0 vs default --enable-dlf 1​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Deblocking loop filter can be slightly beneficial in some scenarios. In reverse, it is never harmful, so it is recommended to keep it default.

--fast-decode 1 vs default --fast-decode 0​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

fast-decode 1 is pretty harmful in the Fate clip and slightly harmful in the rest. There is a speed benefit of enabling it though.

--irefresh-type 1 vs default --irefresh-type 2​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Finally something interesting to analyse!

  • irefresh-type 1 is either a match or beneficial compared to irefresh-type 2 at high CRF levels.
  • irefresh-type 1 either wins or lose to irefresh-type 2 at low CRF levels depending on the clip. As such, it is NOT recommended to blindly set irefresh-type to 1 at "high quality" as it might be harmful depending on the content.
  • irefresh-type 1 is always slower compared to irefresh-type 2, so I might argue it is more safe to leave irefresh-type default at "high quality" than not.
  • According to the content type of those clips, it appears that irefresh-type 1 may benefit extremely grainy content while default irefresh-type 2 is better suited for cleanish content. This needs to be confirmed with moar testing though.

--lookahead 0 vs default --lookahead -1 (auto)​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

--lookahead 60 vs default --lookahead -1 (auto)​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

--lookahead 120 (max) vs default --lookahead -1 (auto)​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

lookahead seems to behave strangely when set...

  • lookahead 0 shifts quality around a lot and it is difficult to draw conclusions but there's a clear speed drawback of disabling lookahead.
  • lookahead 60 is perfectly bit-perfect to every clips EXCEPT for some damn reason in Jigokuraku at CRF23
  • lookahead 120 is somehow also bit-perfect, but this time in every clips and every CRF levels. Both 60 and 120 don't see much speed differences.

Soooo.... this behavior is so odd I don't advise to set any lookahead value. Let the encoder decide.

--enable-overlays 1 vs default --enable-overlays 0​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

overlays do not seem to either improve efficiency or performance.

--enable-qm 1 vs default --enable-qm 0​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Enabling quantization matrices alone increase efficiency at "high quality" with no real speed impact.

--enable-qm 1 --qm-min 0 vs --enable-qm 1​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Setting qm-min to 0 on top of enabling quantization matrices can be beneficial in some clips at no added compute time.

I will re-tests many QMs ranges in the future, but I doubt it changed much from v1.7.0 where 0 was the most appropriate choice for most content.

--enable-restoration 0 vs default --enable-restoration 1​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Even though the efficiencies are very similar, nothing is bit-perfect here. So according to SSIMU2, the loop restoration filter isn't necessarily useful. However, just like CDEF, it's a pretty demanding tool, so disabling it yields some performance improvements. Let's take these with a grain of salt until the image comparisons.

--scm 0 vs default --scm 2 (content adaptive)​

In all the clips, the results are bit-perfect and there is no notable performance difference.

--scm 1 vs default --scm 2 (content adaptive)​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Interestingly enough, screen content tools seem to improve efficiency according to SSIMU2, at the cost of a huge performance regression. After the image comparisons are published, I will conduct additional testing on this.

--enable-tf 0 vs default --enable-tf 1​

  • Efficiency graphs:
Preview
  • Speed graphs:
Preview

Disabled temporal filtering can sometimes improve efficiency slightly at "high quality", however it is very much clip dependent. It also improves performance slightly. The image comparisons will give another perspective to these results.

--enable-tpl-la 0 vs default --enable-tpl-la 1​

In all the clips, the results are bit-perfect and there is no notable performance difference.

superres:​

Kinda lazy to make, share and comment so much graphs for something that can be told in two lines...

All superres variants are freaking useless as they:

  • do not improve efficiency
  • decrease encoding speeds
  • decrease decoding speeds when either bitrate or quality normalized.

Early TLDR on parameters results:​

For a previous test with SVT-AV1 1.7.0 I did all parameters tests with --preset 6 --tune 2, now I did them with --preset 4 --tune 1. Mostly everything tested on v1.7.0 still stands today, but now we have more valuable data. Images comparisons are still needed to give more context to some results, so the conclusion presented here remains early as they are 100% based on SSIMU2 results and will require more analysis down the line.

Here is a quick run down of how each parameter affect encoding:

  • --tile-rows --tile-columns should never be used (except for decreasing decode complexity)
  • --aq-mode 2 is the most efficient / fastest
  • --enable-cdef 0 might improve performance at almost no efficiency loss (needs more thorough testing)
  • --enable-dg and --enable-dlf barely do anything
  • --fast-decode 1 decreases efficiency, improves encoding times, and the decoding gains still need to be determined.
  • --irefresh-type should be kept default at high CRF values and for cleanish content at low CRF values, but can be set to 1 at low CRF values for extremely grainy content
  • --lookahead should be kept default
  • --enable-overlays 1 does not improve efficiency, slight speed regression as well
  • --enable-qm 1 --qm-min 0 should be set for increased efficiency especially at low CRF values at no perf cost
  • --enable-restoration barely does anything but disabling yields better performance (needs visual confirmation)
  • --scm 1 screen content tools can improve efficiency with a big performance trade-off (needs more thorough testing)
  • --enable-tf 0 is a mixed bag efficiency wise but improves performance
  • superres & resize please don't.

Conclusion​

The extensive benchmarking reveals that obviously the slower presets provide better efficiency, with diminishing returns past preset 2. However, preset 4 provides a good balance of quality and speed for most use cases. Presets 6 and 8 are good options for the people that find 4 to be too slow for their liking.

The default settings tend to provide good results, but some tweaks like enabling quantization matrices can further optimize efficiency. Parameters like tile encoding and super resolution modes are not beneficial. Overall, SVT-AV1 continues to be a competitive option for encoding animation in AV1, with its speed being a notable strength.

Further testing with more content samples would help solidify these findings. The image comparisons will also provide additional subjective evaluation to complement the objective metrics used here, and we can discover the potential usefulness of Tune 0 that may betray what the metrics suggest. Overall, this comprehensive deep dive should give encoders a helpful starting point for choosing settings when encoding animation with the latest SVT-AV1 1.8.0. Thanks for reading!

]]> video compression benchmarks <![CDATA[Embedding the Un-Embeddable]]> https://av1.wiki/blog/embedding-the-un-embeddable https://av1.wiki/blog/embedding-the-un-embeddable Sun, 29 Oct 2023 00:00:00 GMT Feature image A 567.14 MB, 12 min 11 s, 2K (2,048 x 858), VP9 + Opus, 6.51 Mbps average, Blender short film "Cosmos Laundromat"

A Scenario​

While chatting in your favorite Discord servers & group chats, you may see a friend send a weird link. You might even consider it suspicious on first glance. It is a video featuring an image of a movie poster with a play button that is almost begging to be clicked. Naturally, you click it.

It loads for a second, and to your surprise it is a full-length, 90-minute (sometimes even two hour)-long unauthorized copy of a movie. If you don't know exactly what is going on, you probably sit there dumbfounded as a pixel perfect HD movie plays back. You may have expected a stereotypically muddy, blocky, laggy shitpost, but this has defied your expectations.

stolen.shoes

The truth is, there are multiple site that do this. Currently, there are five at the time of writing. Below is a list the ones I am currently familiar with:

The big question is, how do they work? Let's get to dissecting.

But First, a Quick Disclosure​

The Codec Wiki unequivocally condemns any form of piracy, including the unauthorized distribution of copyrighted content. This blog post is intended to educate & inform. You may not use the tools discussed to infringe upon the intellectual property rights of content creators without serious legal risk. We encourage our readers to respect copyright laws & use the tools we discuss here appropriately.

How it Works​

The entire scheme is actually very simple, as it is all just HTML meta tags (If you are familiar with web development, this is all a walk in the park).

The technology's inner working can be divided into two distinct parts. First, let's see how it works on the website's end.

The Website's End​

If you view each website's source, you will find this specific line in each one but they may have a different order:

<meta property="og:image" content="https://example.com/i/someimageforthumbnail.jpg">
<meta property="og:type" content="video.other">
<meta property="og:video:url" content="https://example.com/v/video.mp4">
<meta property="og:video:width" content="1920">
<meta property="og:video:height" content="1080">

These are the head parts of HTML, which dictate metadata for the document itself such as what the website title/name is, cosmetic embed, defining the site's icon, etc. They are usually found in between the <html> and <body> tags. Here's an example of a static HTML site serving one specific video:

<!DOCTYPE html>
<html>
<head>
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <link rel="shortcut icon" type="image/x-icon" href="../favicon.ico"/>
  <title>some embed site</title>
  <meta property="og:image" content="https://www.themoviedb.org/t/p/original/3U0vksjSY9LWe3Dqfr6xkgd3iQP.jpg">
  <meta property="og:type" content="video.other">
  <meta property="og:video:url" content="https://upload.wikimedia.org/wikipedia/commons/3/36/Cosmos_Laundromat_-_First_Cycle_-_Official_Blender_Foundation_release.webm">
  <meta property="og:video:width" content="2048">
  <meta property="og:video:height" content="858">
</head>
<body>
  <h1>Hi</h1>
  <p>Just your friendly neighborhood video embed site</p>
  </body>
</html>

< br /> These interactive sites usually deploy a live script, like a Javascript framework. Examples are NodeJS, ExpressJS, Svelte, etc. These are used to parse video and thumbnails realtime so they can be embedded on Discord (or potentially other platforms).

Discord's End​

Traditionally, Discord's media embedder will impose it's own video embed size limit (50 MiB) when a user sends a direct video link as usual. But in this case Discord will embed the thumbnail first, not the video. You could say the link "tricks" Discord by showing a "false face" first.

Strengths & Limitations​

After a combination of countless hours of observation, rigorous testing throughout the period of a year, and conversations with the sites' creators, the current strengths & limitations of this exploit are enumerated below.

Strengths​

  • You can embed non-web compatible codecs such as HEVC in MP4/MOV, but the user must be using a compatible browser. Thorium or Safari version 13 or greater will work for HEVC playback.
  • There is no maximum size. You could embed a video the size of a raw Bluray, although I do not condone this unless you have the necessary legal permissions to do so or you're uploading a Creative Commons licensed movie like Big Buck Bunny while adhering to the restrictions of the applicable Creative Commons license. This also means you can send high bitrate gaming clips to your friends without any restrictions, assuming you already have a place to upload them.

Limitations​

  • You can only use hotlinks, which means direct linking to the video itself ending in the appropriate file extension such as .mp4. Cloud services like Google Drive or OneDrive will not work for storage.
  • You cannot use Discord's CDN (cdn.discordapp.com) as the video source. I assume this is because of Discord's proxy blocking embeds over 50 MiB, but only discord.nfp.is can do this, as it proxies cdn.discordapp.com itself.
  • You cannot embed videos in any resolutions higher than 3840 x 2160, Discord imposes a hard limit for this on all video after it was discovered that some videos could play normally but then be maliciously scaled to ridiculous resolutions during playback to crash Discord.

Differences between Sites​

As mentioned before, there are five known sites at the time of writing. They all serve the same function, but one may interest you more than another due to slight differences in features & functionality.

Here are the sites, each with one noteworthy special benefit:

That concludes the technical overview! Next, let's cover the history of this exploit.

The Lore​

Dwayne​

In around April of 2022, a Reddit user going by the name of u/CreativeGamer03 posted a video on r/discordapp of a link where a GIF of Dwayne "The Rock" Johnson plays caption with "Is this a GIF or is it a video?" When played, a low-quality music video of Rick Astley's "Never Gonna Give You Up" plays.

The link used is now unfortunately removed.

Discovery​

On 23rd June 2022, a Discord user Clybius on the AV1 Community server asked people for VP9 or H.264 videos that were over 100 MB in size. At the time the current 500 MB nitro tier did not exist. They then decided to use a 59 minute 1080p sample video of nature scenery from around the world with a thumbnail featuring a GIF of a waterfall to test the exploit. It worked.

He tried shortly afterward with AV1. Eureka, it also worked:

AV1

Clybius confirmed that this could be patched if discovered. He cites having had the idea from the Dwayne Johnson example above, but forgetting about it for a couple of months. So, it seems this entire concept stemmed from a silly rickroll.

Dwayne

The Experiments & Interactive Site​

After the discovery of AV1 embedding, experimentation brought about the discovery that any video codec will work as long as the user can decode/play the codec and the container/extension is an MP4, MOV, or WebM. These are all traditionally web-compatible containers. If you're interested in learning about containers, please see the Containers section on the Terminology page.

This applies to HEVC, ProRes, xHE-AAC, and other bizarre codecs that are rarely seen on the Web.

While experimentating, Clybius converted one their idle domains stolen.shoes into an interactive embedder that provided a textbox for a video URL, a thumbnail URL, a width value, & a height value for the desired video. This would be the first website for Discord embedding.

Virality​

It's not long before people outside of the AV1 Community discovered stolen.shoes, and its popularity increased rapidly. Its use usually involved the illicit distribution of full-length, unauthorized copies of movies; this sometimes happened very shortly after some movies were released. There were a couple notable instances of this happenening that caused quite the stir online each time.

  • The first instance featured the DreamWorks sequel of "Puss in Boots (2011)", "Puss in Boots: The Last Wish (2022)". A 1080p video sourced from a streaming site was the first wake up call that attracted attention to the existence of these embed sites. This example used stolen.shoes.

puss

  • The second instance was when highly-anticipated animated film "The Super Mario Bros. Movie (2023)" produced by Illumination, Universal Studios, and Nintendo was spread around Discord. It was first spotted as a Cam (A camera recording by someone in theaters), then as it went out on streaming services a different link appeared but spread faster and with upgraded 1080p quality. Both used stolen.shoes as the embed site.

mario

  • The third instance is very recent as of the day this was posted. A streaming-service sourced "Five Nights at Freddy's (2023)" was spread around since the movie released both in theaters and streaming service (Peacock) day one, and it gained steam extremely fast as most people had not seen it yet. Currently, this illegal novelty is gaining hundreds of upvotes within the r/discordapp subreddit. The copy seems to be a compressed 720p encode. This example used discord.nfp.is.

fnaf

Note the ones listed here are the ones that I saw become extremely popular. There may be lesser known links that have been spread around privately or just did not cause enough noise for me to notice. Some less popular examples I've noticed, featuring more illicit copyrighted content distribution:

  • Top Gun Maverick (2022)
  • The SpongeBob trilogy (2005/2015/2020)
  • Spider-Man: Across the Spider-Verse (2023)

Closing​

The ability to embed unusually large videos on Discord has enabled both positive and negative use cases. On the one hand, it allows high-quality content to be shared easily among friends. However, it has also facilitated mass copyright infringement by empowering virtually anyone with a Discord accound to freely spread pirated movies.

While this is fascinating from a technical perspective, embedding techniques like these tread a fine ethical line. As with anything, it is important to be mindful of how our actions affect others, and I should remind everyone that content creators deserve to be compensated for their work. As users, we should support them by accessing their content via legitimate platforms.

It is hard to say how long this exploit will continue to be usable. Instead of enabling piracy, which may cause Discord to be more likely to patch this exploit if they see it as a serious threat, let's instead use these capabilities responsibly to share our own creations, gaming highlights, and other media which we can share legally. Given some thoughtfulness, perhaps we can find a fair balance between respecting copyright law and appeasing Discord's sensibilities while allowing some creative flexibility on the platform.

Thank you for reading this blog post, I hope you learned something!

]]> video discord <![CDATA[AV1 Encoding for Dummies]]> https://av1.wiki/blog/av1-encoding-for-dummies https://av1.wiki/blog/av1-encoding-for-dummies Sun, 03 Sep 2023 00:00:00 GMT This guide will show you how to encode in AV1 the right and optimal way. Yes, you using standalone libaom, libsvtav1, and librav1e from FFmpeg or even piping yuv4mpeg into mainline aomenc are all unoptimal.

Outdated Information

While a lot of the information presented in this guide is still relevant and correct, the AV1 ecosystem has changed dramatically since this guide's inception which has demanded a rewrite. Please see the AV1 for Dummies blog post for more information.

Compare

In this guide, we'll be installing Av1an for chunked encoding and infinite threading, because the current state of AV1 encoders, except for SVT-AV1, unfortunately lacks threading and will only use very low amount of cores, which hampers speeds. The only caveat to this approach is RAM consumption, encoding 2160p (4K) with aomenc with 4 workers could take upwards of 16GB of RAM! So do keep this in mind.

Installing the Tools​

Given all of the different operating systems that people use on a day to day basis and the various different encoding workflows that exist, there are a number of ways to do this.

Jump to: Windows | macOS | Linux

Microsoft Windows​

The GUI Way​

  1. Install NMKODER which is a GUI front-end to av1an with all dependencies installed.
  2. You're done, you can skip to the encoding part
Almost abandonware

Since Nmkoder already ships everything by default and its last release was 29th March 2022. You need to manually update all encoders and tools to get better encoding speeds. Missing out on updates will result in your encodes being sub-optimal.

The WSL2 Way​

(Recommended)

If you're not already familiar with WSL2, the The Windows Subsystem for Linux (WSL) is a feature of the Windows operating system that allows you to run a Linux file system, along with Linux command-line tools and GUI apps, directly on Windows. This lets Linux distributions run on bare metal without managing any virtual machines, so encoding performance is very good.

The easiest way to encode with WSL2 is to use rAV1ator CLI, an interactive TUI for Av1an. An ArchWSL2 installation tutorial is provided here.

The Automated Way​

There is now a batch script for automating the install process, which can be found here. The instructions are in the README file.

caution

The script will download outdated version encoders and tools such as aom-av1-psy and MKVToolNix v76.0, if you are fine with these you can proceed.

The Manual Way​

  1. Install Python 3.10.x, this will change so consult from the Vapoursynth website if you're reading this from the future from here and select "Windows Installer 64-bit". Upon installation check the tick for adding Python to PATH like so Python PATH)

  2. Download and install Vapoursynth from here and select "VapourSynth64-RXX.exe"

  3. Open the terminal and type vsrepo.py install lsmas ffms2 to install some plugins for Av1an to work.

  4. Download MKVToolNix from here, select "mkvtoolnix-64bit-XX.X.X-setup.exe", and install (Also available on winget!)

  5. Download Av1an from here (SELECT LATEST AND CLICK THE "ASSETS" DROPDOWN)

  6. Download shared libraries FFmpeg from gyan.dev

  7. Download a pre-built fork of Aomenc (aom-av1-lavish) which has neat stuff such as sane defaults, new tunes, optimizations, etc. This can be downloaded for Windows here (Current as of Sept 6, 2023)

info

If you opt to compile aomenc yourself, you can view the instructions on how to do that here.

  1. Move Av1an, FFmpeg (Including the FFmpeg DLLs), and aomenc to somewhere preferable, eg C:\Encoding.
  2. Add the folder AND MKVTOOLNIX INSTALLATION FOLDER to the Windows PATH environment.

macOS​

macOS is very similar to Linux, although there aren't any GUI tools for AV1 encoding that I can comfortably recommend.

Homebrew + Macports for Av1an + rav1e: Note that some commands may have to be run with sudo, which I won't explicitly include for security reasons.

Installing the Homebrew package manager is a well documented process at this point:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

As is installing MacPorts. Install the relevent .pkg for your macOS version from the MacPorts Project website: www.macports.org/install.php

Now, you can run the following commands:

brew update && brew upgrade
brew install rav1e aom mkvtoolnix ffmpeg
# Usually you must run MacPorts commands for package installations as root
port upgrade outdated
port install av1an

This is the easiest way to get everything set up & working to produce AV1 video with rav1e or mainline aomenc & Av1an. You can check that things are installed by running the following commands & parsing their output:

% av1an --version
av1an 0.4.1-unstable (rev e10880d) (Release)

* Compiler
  rustc 1.70.0 (LLVM 16.0)

* Target Triple
  aarch64-apple-darwin

* Date Info
  Commit Date:  2023-06-25

* VapourSynth Plugins
  systems.innocent.lsmas : Not found
  com.vapoursynth.ffms2  : Not found
% rav1e --version | grep "release" -C 1
rav1e 0.6.6 () (release)
rustc 1.69.0 (84c898d65 2023-04-16) (built from a source tarball) aarch64-apple-darwin
% aomenc --help | grep "AOMedia" -C 3

Included encoders:

    av1    - AOMedia Project AV1 Encoder 3.6.1 (default)

        Use --codec to switch to a non-default encoder.

Notice systems.innocent.lsmas : Not found in the Av1an output. This means you won't be able to use the lsmash chunking method through vapoursynth & may instead have to rely on hybrid chunking, through -m hybrid. This is slower & takes up disk space while encoding, but still works. A sample Av1an command with this basic installation may look like this:

av1an -i "input" -y --resume --verbose --split-method av-scenechange -m hybrid -c mkvmerge -e rav1e --force -v " --tiles 8 -s 4 --quantizer 80 --no-scene-detection" --photon-noise 7 --chroma-noise --pix-format yuv420p10le -w 8 -o "output.mkv"

Building From Source

If you want lsmash support, aom-av1-lavish instead of mainline, or anything else that isn't covered by the more basic installation, you'll have to compile from source. Things are very similar to Linux, with a few oddities:

  • macOS sometimes doesn't have a /usr/local/bin by default. You can fix this by doing mkdir /usr/local/bin.
  • Homebrew installs everything in its own directory structure. If you're building things from source that rely on libraries from vapoursynth, zimg, lsmash, etc, make sure to copy them from /opt/homebrew/lib to /usr/local/lib. Finding them is a matter of ls | grep "keyword" & copying what looks reasonable to be associated with the tool you're using.
  • Building most things from source will have instructions for *nix which work for both macOS & Linux. Even if it says Linux, there's a good chance it'll work on macOS as well, & it is always worth trying Linux build instructions on Mac. I won't be going through building every encoding tool & dependency from source, as it is generally much more intuitive than Windows, but building Av1an is worth detailing here just as an example.
brew install git rust nasm
git clone https://github.com/master-of-zen/Av1an
cd Av1an
RUSTFLAGS="-C target-cpu=native" cargo build --release
cd .. && cd target/release
cp av1an /usr/local/bin

More Difficult: Building aom-av1-lavish from Source

If you want to make the most out of your hardware & eke out every last drop of quality, it may be worth building aom-av1-lavish from source. The first step is to clone it from the Endless Merging branch:

git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging
cd aom-av1-lavish

Now, you need to make some manual changes to the source code until Clybius merges this commit.

  • Add the line #include "aq_variance.h" at line 19 in av1/encoder/encodeframe_utils.c
  • Comment out line 2546 in av1/encoder/speed_features.c. This line is const int qindex_thresh_cdef_sf_s1_s3_l2[2] = { 92, 48 }; & becomes // const int qindex_thresh_cdef_sf_s1_s3_l2[2] = { 92, 48 };.

Now you can continue to build according to the Linux instructions below. Obviously you'll need cmake, which you can install with homebrew along with any other tools you may need. While still in the aom-av1-lavish directory:

mkdir -p aom_build && cd aom_build
cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native"
make -j$(nproc)
# This may need to be run as root:
make install

Now you can run aomenc --help | grep "AOMedia" -C 3 to see if lavish installed. If you're getting the same output as above, you may need to copy the aomenc executable to /opt/local/bin, /usr/local/bin, & /opt/homebrew/bin if you already installed mainline aomenc. Running the version info command again, the correct output should look something like this:

% aomenc --help | grep AOMedia -C 3

Included encoders:

    av1    - AOMedia Project AV1 Encoder Psy v3.6.0 (default)

        Use --codec to switch to a non-default encoder.

Notice how it says AOMedia Project AV1 Encoder Psy instead of AOMedia Project AV1 Encoder. You should be all set after this to start using aom-av1-lavish & following the current parameter meta as outlined below.

Linux​

info

Yet again, try using Arch. It's way easier.

The GUI Way​

  • Install Aviator (SVT-AV1 + FFmpeg) or rAV1ator basically same thing but Av1an + rav1e. Both are only available as Flatpaks. Keep in mind Aviator ships with SVT-AV1 and rAV1ator with rav1e instead of aomenc/AOM-AV1, which I will not be covering here.

The TUI Way​

(Recommended)

  • Install rav1ator-cli, a TUI for using Av1an meant to be easy to use. Much more flexible than the GUI options & can work with a number of encoders. See this page for more info. Can be easily used on any distro.

The Compiling Route​

Ubuntu​

The guide below is targeted towards 22.04, packages and other things may be different on other versions. First Install Rust via rustup first, as apt version of Rust is severely outdated, then you can continue.

Install dependencies:

sudo apt install wget python unzip unrar build-essential meson autoconf automake libtool git nasm yasm python3-dev python3-pip cython3 libass-dev libqt5websockets5-dev libfftw3-dev libtesseract-dev ffmpeg libavcodec-dev libavformat-dev libswscale-dev libavutil-dev libswresample-dev libmediainfo-dev mkvtoolnix mediainfo perl nasm yasm git cmake libavutil-dev libavcodec-dev libavformat-dev libavdevice-dev libavfilter-dev libswscale-dev libswresample-dev libpostproc-dev llvm libclang-dev libssl-dev

Install l-smash:

git clone https://github.com/l-smash/l-smash.git
cd l-smash
./configure --enable-shared --extra-cflags="-march=native"
make -j$(nproc)
sudo make install

Install zimg:

git clone --recursive https://github.com/sekrit-twc/zimg.git
cd zimg
./autogen.sh
./configure
make -j$(nproc)
sudo make install

Install ImageMagick:

git clone https://github.com/ImageMagick/ImageMagick
cd ImageMagick
./configure
make -j$(nproc)
sudo make install

Install Vapoursynth R63:

wget https://github.com/vapoursynth/vapoursynth/archive/refs/tags/R63.zip
unzip R63.zip
cd vapoursynth-R63
./autogen.sh
./configure CFLAGS="-march=native" CXXFLAGS="-march=native" --libdir=/usr/lib
make -j$(nproc)
sudo make install
sudo mkdir /usr/lib/vapoursynth
sudo ldconfig

The plugin directory will be located in /usr/lib/vapoursynth.

Install L-SMASH-Works Vapoursynth Plugin:

git clone https://github.com/AkarinVS/L-SMASH-Works -b ffmpeg-4.5
cd L-SMASH-Works/VapourSynth && mkdir build && cd build
meson .. --optimization=3 --default-library=static -Db_lto=true -Dc_args="-march=native" -Dcpp_args="-march=native"
ninja -j$(nproc)
sudo cp libvslsmashsource.so /usr/lib/vapoursynth/
danger

L-SMASH-Works doesn't work on aarch64, it is recommended to use other plugins instead.

Install FFMS2 Vapoursynth Plugin:

git clone https://github.com/FFMS/ffms2
cd ffms2
./autogen.sh
./configure CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native"
make -j$(nproc)
sudo cp src/core/.libs/libffms2.so src/core/.libs/libffms2.so.5 src/core/.libs/libffms2.so.5.0.0 /usr/lib/vapoursynth

Install Av1an:

git clone https://github.com/master-of-zen/Av1an
cd Av1an
RUSTFLAGS="-C target-cpu=native" cargo build --release
sudo cp target/release/av1an /usr/local/bin

When there's no errors, proceed to compiling aom-av1-lavish.

Arch​

Install dependencies:

sudo pacman -S vapoursynth ffmpeg av1an mkvtoolnix-gui git perl cmake ninja meson nasm vapoursynth-plugin-lsmashsource ffms2

you're done, proceed.

Compiling aom-av1-lavish​

git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging
cd aom-av1-lavish && mkdir -p aom_build && cd aom_build
cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt"
make -j$(nproc)
sudo make install

Encoding​

The moment you've all been waiting for, let's just get into it. Here's an example recommended parameter as of now (09/03/23) [MM/DD/YY]:

av1an -x 300 -i input.mkv -w 4 -e aom -c mkvmerge --resume -m lsmash --photon-noise=10 --set-thread-affinity=2 --verbose -a " -an " -f " -an " -v " --bit-depth=10 --cpu-used=4 --end-usage=q --cq-level=24 --threads=2 --tile-columns=0 --tile-rows=0 --lag-in-frames=64 --tune-content=psy --tune=ssim --enable-keyframe-filtering=1 --disable-kf --kf-max-dist=9999 --enable-qm=1 --deltaq-mode=0 --aq-mode=0 --quant-b-adapt=1 --enable-fwd-kf=0 --arnr-strength=1 --sb-size=dynamic --enable-dnl-denoising=0 " -o "output.mkv"
Parameter Meta

It is strongly recommended to join the AV1 Discord server to get the latest updates on what to use and which to set, as it's the only easily reachable place for everything AV1 & encoding tips in general.

Now let's dissect it one-by-one

Av1an parameters:

  • -i Input.

  • -x 300 Sets scene split length to 300 frames, you can increase it for more quality at the tradeoff of video seekability.

  • -w 4 Specifies the amount of "workers" or amount of encoders working on the video.

  • --verbose Sets logging to verbose.

  • --resume Resumes the encode even when you haven't encoded yet. I strongly recommend leaving this if you resume a lot since you can accidentally delete your whole progress (There's no delete confirmation feature.. yet) if you "resumed" without the parameter in place.

  • -e aom Specifies we're using aomenc encoder which should be the default option.

  • -c mkvmerge Specifies we're using mkvmerge (MKVToolNix) to concatenate the parts when done, you can specify with ffmpeg if you want to but this is the best method.

  • -m lsmash Specifies we're using l-smash (Vapoursynth plugin) to split the videos, this is also the best method because ffms2 causes video lag (Tested a year ago, might change now) and other methods just suck (Slow and not worth it, learned the hard way). You can attempt to use ffms2 when inputting VC-1 videos as it is not possible with l-smash (Or convert it to lossless with x264 qp 0).

  • -f " -an " -f Stands for ffmpeg parameters, -an is to remove all audio since its better to encode and merge it separately. To crop use -f " -an -vf crop=1920:800 " for example to crop the video to 1920x800.

  • -v " " Is where you put the encoder's parameters in.

  • -a " -an " FFmpeg audio encoding options, we're removing it cause we can always add it later. But if you want to, you can also encode directly. Here's an example for encoding to Opus using libopus assuming stereo: -a " -c:a libopus -b:a 128k ".

  • --photon-noise=10 AV1 grain synthesis, which is a technique where the encoder puts fake grain in so it looks more natural and potentially hiding video artifacts (cause grain is hard to encode and explodes bitrate usage because of their randomness), 5-8 for almost none to little grain, 10-14 for medium, 15+ heavy, 20+ extremely heavy, 30+ for extremely grainy 90s live action films.

  • --set-thread-affinity=2 Pins the thread to the encoder, aligns with --threads=2 in the encoder parameter so set them accordingly.

aomenc parameters:

  • --bit-depth=10 We're using 10bit because it makes the video smaller and reduces banding.

  • --cpu-used=4 This is the preset which ranges from 0-9, you can go to 3 if you want more efficiency, 2 if you have a lot of time, 4 is the sweet spot, and 6 if you want speed. Don't go above 6 (Worst efficiency) or even 0 (It would take WEEKS to finish).

  • --end-usage=q --cq-level=24 This specifies that we are going to use a knockoff version of CRF level similar to x264/x265 encoders, in this case CRF 24.

  • --threads=2 Sets the amount of threads the encoder can use, aligns with --set-thread-affinity in Av1an.

  • --tile-columns=0 --tile-rows=0 This is the tiles options, where the encoder splits the videos into tiles to encode faster, see the image below (Yellow lines):

Tiling
Tile usage

Do NOT use tiles for 1080p and below, use 1 tile-columns at 1440p (2K), 2 tile-columns and 1 tile-rows for 2160p (4K)

  • --lag-in-frames=64 Similar to x264/x265 rc-lookahead. Sets a number of frames to look ahead for frametype and ratecontrol, allowing for better compression decision making. Setting to a value greater than 64 is generally not considered useful.

  • --aq-mode adaptive quantization mode, 0 is better most of the time

  • --tune-content=psy --tune=ssim As the name suggests they are tunes that affect the video output, for the better, and for the worst

Tunes to use

Set tune-content to animation if you're encoding above cq-level=30 A.K.A lower quality, despite it's name Set tune-content to psy for everything else, do not use if you encode above cq-level=30 For tune, this is a bit tricky. For now, the meta seems to be ssim, but back then it was lavish which is considered THE best tune because it's based on butteraugli. Now it's fallen behind because its more blurry than ssim, and before that it was butteraugli, and then ipq_vmaf_psy, and finally just ipq. If you use any of the VMAF tunes, you need to specify --vmaf-model-path= to where you put it.

  • --enable-keyframe-filtering=1 We're setting it to 1 because of compatibility reasons, 2 is more efficient but there are seeking issues and FFmpeg for some reason can't input it.

  • --sb-size=dynamic Allows the encoder to use 128x128 block partitioning besides 64x64 which gives an efficiency boost, ignore it.

  • --deltaq-mode set to 0 because its just better.

  • --arnr-strength=1 Controls how strong the filtering will be, 1 is good for 3D Pixar CGI-like and 2D animation, use 4 if you're doing live action content. Using maximum at higher bitrates would just result in a blurry mess.

  • --disable-kf --enable-fwd-kf=0 We're disabling keyframes cause Av1an already did scene detection, so we wont have to.. And it speeds things up.

  • --kf-max-dist=9999 Maximum keyframe interval, we're setting it at the highest possible value since av1an's scene detection keyframe interval is already 240 by default

  • --enable-chroma-deltaq=1 --enable-qm=1 --quant-b-adapt=1 Parameters that give you free efficiency boost.

  • --enable-dnl-denoising=0 Disables the encoder's built-in denoising technique when grain synthesis is enabled, you can optionally set it to 1 when you have a pretty noisy video since it works quite well.

Concatenation Error on Linux

Run ulimit -n 200000, resume, and it should concatenate just fine. If it still errors, head to the encode directory > encode, and run mkvmerge @../options.json

Merging Everything​

Once you're done just encode your audio using ffmpeg (or just passthrough it), subtitles should be carried along with your video output, and merge them in MKVToolNix! Don't want Matroska files? That's fine, you can use FFmpeg or MP4Box to output into mp4, just keep in mind that PGS/SUP/VOBSUB subtitles are not supported and Opus audio support is still experimental.

Tips & Tricks​

  • --denoise-noise-level=10 Alternative to photon-noise, slower than photon-noise and is the OG grain synthesis method, performs okay and just serves as an alternative. Don't attempt to use it at high values (>12) since it creates noticeable grain patterns.

  • --arnr-maxframes to set max reference frames that will be used to filter the encode, higher values would make the video blurrier at high fidelity but look better at lower bitrates.

  • --butteraugli-resize-factor=2 if you use any of the butteraugli-based tunes (lavish, butteraugli) to speed it up without much losses and --butteraugli-intensity-target=250 to match the content light level.

Final Thoughts​

Encoding has always been about experimentation for the best, there is really no "One size fits all" for encoding content, as they differ from scene complexity, how it's captured (2D/Real life), film grain, dark scenes, etc. So experiment away for your specific type of content!

Guide originally hosted on https://rentry.co/AV1, rewrite and migration by Simulping.

]]> video compression <![CDATA[Reducing Image Load Online]]> https://av1.wiki/blog/site-optimization https://av1.wiki/blog/site-optimization Fri, 21 Jul 2023 00:00:00 GMT A big part of understanding any multimedia codec technology is knowing the application for such technology. For images, a big use case is web delivery. Compared to other multimedia, images are incredibly popular on the Web & knowing how to serve them properly can be a massive boon to your website's traffic as well as less of a headache for users on slower connections or who are under bandwidth constraints. The most disappointing part is that images are often poorly done on the web; all too frequently will you run into a site serving massive photographic PNGs for no reason, or photography sites serving photographs fresh out of the editing software with no thought put into their final delivery. A little effort, patience, & knowledge will go a long way toward improving the user experience for individuals using your site, & this article will illustrate some of the basics.

caution

These instructions are for photographic images; other kinds of images, like non-photographic, artwork, pixel art, etc. should likely be handled differently.

danger

Many images won't load properly unless your browser supports JXL, AVIF, & proper ICCv2 color management. This is for demonstration purposes only & shouldn't represent an actual common website experience. If you're curious anyway, the following browsers can display the contents of this page perfectly:

Fire & Forget​

First, we'll illustrate what not to do, which is fortunately not incredibly difficult to avoid. Taking an image straight out of your editing software at a massive size will often bloat the size & resolution to something that isn't generally usable for a website regardless of the codec you're using & its quality per bit. It can be argued there are specific use cases that demand incredible resolution & fidelity coexist on the Web, but we won't be covering those here. Here's an example of a bloated image:

exported straight from Darktable at JPEG q90, with no scaling

bloated_jpeg

2.2 MB

Massive Improvement​

The easiest way to have a large improvement without doing much work is to simply resize the image before serving it. Even if you exported a lossy JPEG, resizing should remove a lot of artifacts. The way to perceive a worst-case for an image's size on a site is to inspect the image element's width & height, which should give us an estimate of how large we should make our image. Any larger than this value is unreasonable since we're overfilling the element's size for no reason & the image is being scaled down anyway.

box-size-mac Inspect Element in Firefox. The Mac used to take this screenshot has a relatively high display resolution of 2560x1664. Because Macs scale things differently, we're probably going to want to double the horizontal resolution here.

The width is the most important value here, so our new image is going to be exported with a width of 1699 pixels. This new image, encoded at JPEG q90 with cjpegli, looks like this:

smaller_jpeg

Obviously, there's lost fidelity compared to the original, but considering this is so much smaller, it is worth the trade-off for many. It is also worth noting we are using an improved jpeg encoder in the form of cjpegli, although that is secondary to the resize. If it doesn't look as good as you want it to, you can always scale the resolution up a bit, though currently, it looks plenty passable for its size.

2.2 MB -> 233 kB

Lazy Loading​

A bonus tip is to add the loading="lazy" attribute to your picture tag to allow the image to load only when scrolled to by a user. This doesn't save bandwidth, but it improves the user experience by loading images further down the page only when necessary. An example may look like this:

<picture>
    <img src="/images/jpeg_fallback.jpg" alt="alt text" width="XX" height="YY" loading="lazy" />
</picture>

New Codecs​

If you desire further improvement, it may be time to consider using a newer codec like AVIF or JPEG-XL. These options will compress far more effectively than JPEG, with the only trade-off being browser support. We're not going to consider WebP or HEIC, since WebP is not competitive enough with JPEG for photographic imagery (often being worse) & HEIC has been superseded by AVIF - which sees greater support anyhow - & is not royalty free, effectively preventing widespread Web adoption forever. Again, we're just considering lossy compression for photographic images; it is a different story with WebP elsewhere, as it performs well on non-photographic content & is almost always better than PNG for 8-bit lossless compression. So, we are left with JXL & AVIF for now.

Fallbacks​

AVIF sees widespread support, but JPEG-XL isn't quite there yet with Web support as Google continues to push AVIF (it is debatable if it ever will be outside the Apple ecosystem). Even with AVIF, adoption isn't remotely close to JPEG, so it is worth providing a fallback. This can look like the following example:

<picture>
    <source srcset="/img/jxl_image.jxl" type="image/jxl" />
    <source srcset="/img/avif_image.avif" type="image/avif" />
    <source srcset="/img/webp_fback.webp" type="image/webp" />
    <img src="/images/jpeg_fallback.jpg" alt="alt text" width="XX" height="YY" loading="lazy" />
</picture>

Here is a JXL falling back to an AVIF falling back to a WebP falling back to a JPEG. Pretty intense to have this many fallbacks unless you're really after the ultimate compression ratio, but it is certainly an option. AVIF & JPEG alone will probably be enough for most.

Compression Efficacy​

Let's look at how our image examples compare to the original with our new codec selection. We'll be aiming for high visual fidelity, so around the same quality as our initial JPEG encoded with cjpegli (which scores ~83.01 with the SSIMULACRA2 visual fidelity metric).

smaller_jxl

137.0 kB JPEG-XL image, encoded with cjxl lossless.png out.jxl -d 1.49 -e 9. Score: ~83.04 3.06s user time

smaller_avif

124.8 kB AVIF image, encoded with avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=16 -a tune=ssim lossless.png out.avif. Score: ~83.03 7.54s user time

JXL also supports lossless transcoding of JPEG images. This means every pixel is identical, the image just has a smaller filesize than the original JPEG; if you can use JXL, you can transcode existing JPEGs losslessly on your site & save some bandwidth that way. The JPEG transcode below gives a higher SSIMULACRA2 score than the original for some reason, but I'll chalk that up to a decoding inconsistency between how the ssimulacra2 program decodes JPEG & JXL. Either way, the scores are fairly close.

smaller_jxl_jpeg-recomp

189.4 kB JPEG-XL image from JPEG, encoded with cjxl input.jpg input-recomp.jxl -d 0.0 -e 9 --brotli_effort=11. Score: ~84.92 (???) 0.67s user time

The final trick we can use, while not a new codec at all, still increases quality per bit. Encoding an XYB JPEG with cjpegli encodes with the perceptual XYB colorspace using an ICC profile to modify the original JPEG colors, avoiding JPEG's normal YCbCr which isn't perceptually optimized for the human visual system. Using XYB, we can afford identical quality with less bitrate than normal JPEG. This has universal compatibility, but not every application understands how to handle the XYB color profile (although color-managed modern browsers should be fine).

smaller_jpeg_xyb

208.3 kB XYB JPEG, encoded with cjpegli lossless.png out.jpg --xyb -d 1.155. Score: ~83.04 0.10s user time

In this particular instance, AVIF seems to be the overall winner. This isn't always the case due to JXL's superiority at higher fidelity & with more detailed images, but according to SSIMULACRA2, AVIF has the best quality per bit with this image. You can use your own eyes to further clarify your choice, though. It is worth mentioning that as these were encoded from a 16-bit source PNG, the JXL image is the only one that maintains the full original bit depth, & AVIF isn't fast to encode.

Responsive Images​

Displaying an image that is too large for a viewport is a waste of bandwidth, & displaying an image that's too small for the viewport leaves fidelity to be desired. Luckily, we have the Responsive Image Linter that can help us figure out which image sizes we should be using.

responsive_image_linter

In our fire & forget example, we see that we are serving an image that is far too large. We already know that, but now we can see that given various viewport sizes we could be serving images that have respective widths of 270px, 958px, 1350px, 1660px, & 1916px to optimize for delivery to a variety of different devices. Here's how we'd write that in HTML:

<picture>
  <source type="image/jxl" srcset="/img_270.jxl 270w, /img_958.jxl 958w, /img_1350.jxl 1350w, /img_1660.jxl 1660w, /img_1916.jxl 1916w" sizes="(min-width: 2000px) 1916px, (min-width: 1700px) 1660px, (min-width: 1400px) 1350px, (min-width: 1000px) 958px, calc(100vw - 24px)" />
  <source type="image/avif" srcset="/img_270.avif 270w, /img_958.avif 958w, /img_1350.avif 1350w, /img_1660.avif 1660w, /img_1916.avif 1916w" sizes="(min-width: 2000px) 1916px, (min-width: 1700px) 1660px, (min-width: 1400px) 1350px, (min-width: 1000px) 958px, calc(100vw - 24px)" />
  <img width="1699" height="1136" alt="alt text" srcset="/img_270.jpg 270w, /img_958.jpg 958w, /img_1350.jpg 1350w, /img_1660.jpg 1660w, /img_1916.jpg 1916w" sizes="(min-width: 2000px) 1916px, (min-width: 1700px) 1660px, (min-width: 1400px) 1350px, (min-width: 1000px) 958px, calc(100vw - 24px)" src="/fallback.jpg" />
</picture>

It is worth noting that this example above & the example below aren't perfect implementations of a responsive image given the conditions of this site, but the general concept still applies. Some things to note:

  • srcset = the images available to your browser to serve, & their respective widths
  • sizes = the conditions given to the browser explaining under what conditions should it serve which image
  • (min-width: XXXpx) YYYpx = Given the viewport is at least XXX wide, serve an image of YYY horizontal resolution. The browser will pick an image from srcset that is CSS pixels * display scaling.
  • calc(100vw - 24px) = Usually preceded by a (min-width) condition. Specifies a value the browser should calculate on its own to pick the closest option from the srcset. Let's say we have (min-width: 997px) calc(75vw - 257px). This means given the viewport is at least 997px wide, calculate 0.75 * the current viewport resolution - 257 to find the closest image in the srcset to fit the number of pixel specified.
alt text

That's all! Massive thanks to Auto-Rez Media Technologies for the inspiration behind this article & explicit permission to use their Reduce Your Page's Image Load blog post when writing this entry. I have confirmed with their leadership that this wiki entry can be safely licensed under CC BY-SA 4.0.

]]> image web compression