Semantic Changes in Learned Image Compression (SCLIC)
Novel learning-based image compression algorithms achieve high perceptual quality at unprecedented compression rates. However, these methods tend to prioritize preserving the overall appearance of images, sometimes leading to reconstructions that appear plausible and of high quality, but are semantically different from the original input. We refer to this phenomenon as miscompression, which describes semantic changes in image details induced by neural compression.
The illustration below provides an example of a miscompression, where the digit 8 from the original image (left) is reconstructed as a 6 in the compressed version (right).
Uncompressed image, taken from CLIC2020
Neural compressed image, compressed with HiFiC
The objectives of the SCLIC research project are to quantitatively investigate the extent and characteristics of miscompression and to identify the factors that contribute to the problem. Ultimately, we aim to inform the development of mitigation strategies that facilitate the applicability of neural compression in everyday contexts.
The project is funded by the Tiroler Nachwuchsforscher*innenförderung (TNF).
References
- Hofer, N. and Böhme, R. A Taxonomy of Miscompressions: Preparing Image Forensics for Neural Compression. In IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, Rome, Italy, 2024. [Preprint]
- Mentzer, F., Toderici, G.D., Tschannen, M., and Agustsson, E. High-fidelity generative image compression. Advances in Neural Information Processing Systems, 33, (2020), 11913–11924.
- George, T., Wenzhe, S., Radu, T., et al. Workshop and challenge on learned image compression (CLIC2020). 2020. http://www.compression.cc Conference on Computer Vision and Pattern Recognition.