Conference on Computer Vision and Pattern Recognition (CVPR 2021)
Authors  Erik Franz, Barbara Solenthaler, Nils Thuerey

We propose a novel method to reconstruct volumetric flows from sparse views via a global transport formulation. Instead of obtaining the space-time function of the observations, we reconstruct its motion based on a single initial state. In addition, we introduce a learned self-supervision that constrains observations from unseen angles. These visual constraints are coupled via the transport constraints and a differentiable rendering step to arrive at a robust end-to-end reconstruction algorithm. This makes the reconstruction of highly realistic flow motions possible, even from only a single input view. We show with a variety of synthetic and real flows that the proposed global reconstruction of the transport process yields an improved reconstruction of the fluid motion.

Supplemental Video
Supplemental Video (YouTube)

Fig. 1: Overview: We propose a novel algorithm that reconstructs the motion of a single initial state over the full course of an input sequence, i.e., its global transport. Based on a learned self-supervision, our algorithm yields a realistic motion for highly under-constrained scenarios such as a single input view.

Supplemental video. YouTube version.

Fig. 2: Example reconstruction using one of the ScalarFlow captures.