HEVC over Twitter

“If a picture’s worth 1000 words, how much of a picture can you fit in 140 characters?”. This was the idea behind the Twitter image compression challenge set up in 2009. The challenge was to develop a codec that can efficiently compress an image into 140 characters (limit of a tweet on twitter) with good reconstruction quality. Boojum’s nanocrunch.cpp and Sam Hocevar’s img2twit were apparently two of the best codecs that came out of the challenge.

The High Efficiency Video Coding (HEVC) standard was finalized in January 2013. Compared to the earlier video coding standards such as the H.264/AVC, HEVC can reduce the bit rate by about 50% for equal perceptual video quality. An important feature of HEVC is its rich set of intra modes which makes it an attractive candidate for coding still images as well. It has been shown to reduce the bitrate by about 17% compared to H.264/AVC, 22% compared to JPEG 2000, 32% compared to JPEG XR, 34% compared to WebP and 44% compared to JPEG for the same visual quality.

In this blog post, we try to evaluate how HEVC would fare in the Twitter image compression challenge. The idea is to find out if it is possible for HEVC to compress an image to 140 characters (the twitter message length) and still recover a recognizable image. Using the 20902 characters available in the CJK Unified Ideographs block, the number of bits that can be represented using 140 characters is:

p=. $n_{bits} = \frac{140 log(20902)}{log(2)}=2009.18$ bits that is around $\frac{2009.18}{8} \approx 251$ bytes

So we had to compress the image to a maximum of 251 bytes. We used the Mona_Lisa image for test purposes. A scaled down version (192x288) of the original image is shown in Fig. 1.

Figure 1: The original Mona_Lisa image (22,854 bytes)

The default encoder_intra_main.cfg file was first used as the encoder configuration file.

Figure 2: Original (left) vs. HEVC default intra compressed (right)

Using the default HM 12.0 encoding parameters, the image was compressed to 2,259 bytes which was roughly about 10 times fewer bytes than used to store the original image. The visual quality was good (see Fig. 2) but 2,259 bytes were too many for a tweet. So we tried different encoder parameters to reduce the size of the bit stream and kept an eye on the reconstruction quality.

The first encoder parameter that we played with was the obvious RateControl. It is disabled by default. Enabling the RateControl allowed us to reduce the bit stream size to only 247 bytes. The reconstruction quality was poor (see Fig. 3). Not a surprise. But the good thing is that 247 bytes can be packed into 137 CJK characters which can then be easily sent as a tweet.

Figure 3: Original (left), HEVC default intra (centre) and HEVC Rate control enable (right)

Next, we see whether disabling RateControl and playing with other encoding parameters can improve the visual quality while keeping the size of the bit stream minimal. The first obvious choice for achieving maximum compression was the Quantization Parameter (QP) value. We set it to the maximum value of 51 (default: 32). Compared to rate control, the size increased to 278 bytes but the visual quality improved slightly (see Fig. 4).

Figure 4: HEVC rate control, 247 bytes (left) vs. HEVC max. QP, 278 bytes (right)

We then reduced the maximum CU width and height to 32 (default: 64) and the depth of the Coding Unit (CU) to 3 (default: 4). It saved us an extra byte while maintaining somewhat similar visual quality (see Fig. 5). The size was still a bit too large for a tweet.

Figure 5: HEVC rate control, 247 bytes (left) vs. HEVC max. QP, 278 bytes (centre), vs. HEVC max. QP + max. CU size 32x32, 277 bytes (right)

We found that, additionally, by disabling the Sample Adaptive Offset (SAO) filter reduces the bit stream size to 274 bytes (Fig. 6).

Figure 6: HEVC rate control, 247 bytes (left) vs. HEVC max. QP + max. CU size 32x32, 277 bytes (centre), vs. HEVC max. QP + max. CU size 32x32 + disable SAO, 274 bytes (right)

By disabling rate control and manually testing different encoder parameters for efficiently compressing the image, it was found that 274 bytes was possibly the minimum number of bytes we could represent the image with. Though the quality of the image was slightly better than that obtained using rate control but it was still over 23 bytes the limit of a tweet. So the image obtained with HEVC rate control was finally selected as the image to be sent via tweet.

Next, in Figure 7, we compared the results of some of the best candidate codecs for the twitter image compression challenge.

Figure 7: img2twit at ~250 bytes (left), DLI at 243 bytes (centre), and HEVC at 247 bytes (right)

It can be seen that compared to img2twit, the results for HEVC are clearly better while they are also comparable to DLI (which, according to shreevatsa is a record-holding “optimizing general-purpose lossy image codec”.

Please note that this is intended as an interesting blog post, not an extensive evaluation. It is impressive that HEVC, usually associated with much higher resolution moving images, performs so well here. Given more time to optimize we think it should be possible to improve upon the best HEVC image shown here. Please get in touch if you think you can do better, either with HEVC or another compression scheme - impress us and we’ll publish your results as an update.