1. [Publications](/index.php/publications)
2. Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
 
 # Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection

  ![](/sites/default/files/styles/wide/public/publications/Slide1.PNG?itok=5gmcnXHX)

 Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these artifacts without resorting to complex acquisition mechanisms like alternating exposures or costly processing that are typical of high dynamic range (HDR) imaging. We propose a transformer-based deep neural network (DNN) to infer the missing HDR details. In an ablation study, we show the importance of using a multiscale DNN and train it with the proper cost function to achieve state-of-the-art quality. To aid the reconstruction of the overexposed areas, our DNN takes a reference frame from the past as an additional input. This leverages the commonly occurring temporal instabilities of autoexposure to our advantage: since well-exposed details in the current frame may be overexposed in the future, we use reinforcement learning to train a reference frame selection DNN that decides whether to adopt the current frame as a future reference. Without resorting to alternating exposures, we obtain therefore a causal, HDR hallucination algorithm with potential application in common video acquisition settings.



 ## Authors



Yazhou Xing (HKUST)

[Amrita Mazumdar](/index.php/person/amrita-mazumdar)

Anjul Patney (NVIDIA)

[Chao Liu](/index.php/person/chao-liu)

[Hongxu Danny Yin](/index.php/person/danny-yin)

Qifeng Chen (HKUST)

[Jan Kautz](/index.php/person/jan-kautz)

[Iuri Frosio](/index.php/person/iuri-frosio)

 

 

 ## Publication Date



Tuesday, August 29, 2023

 

 ## Published in



[Arxiv](https://arxiv.org/abs/2308.15462)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/index.php/research-area/machine-learning-artificial-intelligence)

[Computational Photography and Imaging](/index.php/research-area/computational-photography-imaging)

[Computer Vision](/index.php/research-area/computer-vision)

 

 

 ## External Links



[Arxiv paper](https://arxiv.org/abs/2308.15462)

 

 

 ## Uploaded Files



[Arxiv paper](https://d1qx31qr3h6wln.cloudfront.net/publications/2308.15462.pdf "Open file in new window")11.43 MB