Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays

Abstract

We present Play4D, a new method for accelerating and interactive free-viewpoint video streaming for virtual reality and light field displays. Our method is based on the observation that the face of a puppeteer is often visible in the background of the video, even when the puppeteer is not visible in the foreground. We train a deep neural network to detect the presence of a puppeteer in the background of a video, and we use this network to score the likelihood of impersonation. Our method achieves state-of-the-art performance on a new dataset of AI-based impersonation videos, and it is able to detect subtle impersonation techniques that other methods miss.