Unlocking AI’s Potential: Transferring artistic features from 2D images to 3D scenes with Cornell and Adobe’s Artistic Radiance Fields

New Computer Vision Research from Cornell and Adobe Proposes An Artificial Intelligence (AI), Method to Transfer the Artistic Features of an Arbitrary Style Image into a 3D Scene

Art is an extremely fascinating and complex discipline. The creation of artistic images can be a difficult and time-consuming task that also requires considerable expertise. Imagine if this problem was extended to other dimensions than the image plane. For example, time (with animated content) and 3D space (with virtual environments or sculptures). This presents new challenges and constraints, which this paper addresses.

The results of 2D stylization in the past have been based on video content that has been split into frames. The generated frames often achieve high-quality styling, but the video generated is often distorted by flickering artifacts. The lack of coherence in the frames is the cause. They also do not look at the 3D environment which increases the difficulty of the task. Some works that focus on 3D styling suffer from inaccurate geometric reconstructions of triangle or point cloud meshes, and lack of style detail. This is due to the differences in geometrical properties between the starting mesh and the produced mesh. The style is then applied after the linear transformation.

Artistic Radiance Fields, or ARF as it is called in the proposed method, can be used to transfer artistic features from an individual 2D image into a 3D real-world scene. This results in artistic renderings of novel views that are faithful to input style images (Fig. 1).

Source:

Latest Computer Vision Research From Cornell and Adobe Proposes An Artificial Intelligence (AI) Method To Transfer The Artistic Features Of An Arbitrary Style Image To A 3D Scene