1. [Publications](/publications) 2. ꟻLIP: A Difference Evaluator for Alternating Images # ꟻLIP: A Difference Evaluator for Alternating Images ![](/sites/default/files/styles/wide/public/publications/flip_image2.png?itok=J1T0SmXT) **Abstract:** Image quality measures are becoming increasingly important in the field of computer graphics. For example, there is currently a major focus on generating photorealistic images in real time by combining path tracing with denoising, for which such quality assessment is integral. We present ꟻLIP, which is a difference evaluator with a particular focus on the differences between rendered images and corresponding ground truths. Our algorithm produces a map that approximates the difference perceived by humans when alternating between two images. ꟻLIP is a combination of modified existing building blocks, and the net result is surprisingly powerful. We have compared our work against a wide range of existing image difference algorithms and we have visually inspected over a thousand image pairs that were either retrieved from image databases or generated in-house. We also present results of a user study which indicate that our method performs substantially better, on average, than the other algorithms.To facilitate the use of ꟻLIP, we provide source code in C++, CUDA, MATLAB, NumPy/SciPy, and PyTorch. Please check out our [gallery of images and error maps](https://research.nvidia.com/node/3525). **Hindsight:** in the paper we advocated for the weighted median, computed from the weighted histogram, but this is not ideal. If a single number is to be used, then we recommend using the mean ꟻLIP value instead. **Errata:** p. 10: "... followed by division by 100" should be "... followed by division by 116." **High Dynamic Range (HDR) images:** This version of ꟻLIP handles only low dynamic range images. If you wish to evaluate errors in HDR images, confer the paper [Visualizing Errors in Rendered High Dynamic Range Images](https://research.nvidia.com/publication/2021-05_HDR-FLIP), where a version of ꟻLIP that handles HDR images is introduced. **How to write an ꟻ:** HTML: ꟻ Unicode: [fileformat.info](https://www.fileformat.info/info/unicode/char/a7fb/index.htm) LaTeX: \\usepackage{mathtools} \\usepackage{xspace} \\newcommand{\\FLIP}{\\protect\\reflectbox{F}LIP\\xspace} ## Authors [Pontus Ebelin](/person/pontus-ebelin) Jim Nilsson (NVIDIA) [Tomas Akenine-Möller](/person/tomas-akenine-moller) Magnus Oskarsson (Lund University) Kalle Åström (Lund University) Mark D. Fairchild (Rochester Institute of Technology) ## Publication Date Monday, July 13, 2020 ## Published in [High Performance Graphics 2020](https://www.highperformancegraphics.org/2020/) ## Research Area [Computer Graphics](/research-area/computer-graphics) ## External Links [Link to images and error maps](http://research.nvidia.com/node/3525) [Link to supplementary materials](https://drive.google.com/file/d/1HDuLzNSUozTFe624tByfLbaHXnmql2-C/view?usp=sharing) [DOI](https://doi.org/10.1145/3406183) [GitHub project](https://github.com/NVlabs/flip) ## Uploaded Files [Paper](https://research.nvidia.com/sites/default/files/node/3260/FLIP_Paper.pdf "Open file in new window")21.04 MB [Bibtex](https://d1qx31qr3h6wln.cloudfront.net/publications/flip.bib.zip "Open archive in new window")522 bytes [HPG presentation (.mp4)](https://d1qx31qr3h6wln.cloudfront.net/publications/2020-07-14%20FLIP_HPG.mp4 "Open video in new window")251.29 MB [MATLAB code \[C++, CUDA, Python, and PyTorch code available on GitHub\]](https://research.nvidia.com/sites/default/files/pubs/2020-07_FLIP/flip-matlab_0.zip "Open archive in new window")536.27 KB ## Award Wolfgang Straßer Award for 3rd best paper, High Performance Graphics 2020 ## Copyright Copyright by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax [+1 (212) 869-0481](tel:+12128690481), or . The definitive version of this paper can be found at ACM's Digital Library .