Google AI researchers propose an AI-based method for creating flythrough videos from single-view photos

Google AI Researchers Propose an Artificial Intelligence Based Method for Learning Perpetual Scene Generation from Single-View Photos

Our planet is beautiful, with its majestic mountains, stunning seascapes and tranquil forests. Imagine yourself as a bird, flying over three-dimensional, intricately detailed landscapes. Can computers learn to reproduce this type of visual experience? Current techniques, which combine different perspectives from photos, typically allow only a limited amount of camera movement. The majority of earlier research could only extrapolate the scene content from a limited range of views, corresponding to subtle head movements.

Google Research, Cornell Tech and UC Berkeley presented a method for learning how to create flythrough videos that are unrestricted, starting with a single image. This capability is learned by a collection single images without the use of camera poses or multiple views of each scene. The method is able to take a single picture and create long camera trajectory of hundreds of views with realistic and diverse contents, even though the user has never seen a video in training. This method is in contrast to the latest cutting-edge techniques for supervised view creation, which require posed multi-view film and have better performance and synthesis.

Fundamentally, they learn to create flythroughs gradually. They first compute a single-image depth map using techniques such as those shown in the figure. They use the depth map created after rendering the image from a different camera perspective, as shown in the middle.


Google AI Researchers Propose An Artificial Intelligence-Based Method For Learning Perpetual View Generation of Natural Scenes Solely From Single-View Photos