Unsupervised object-centric video generation and decomposition in 3D

P.M. Henderson, C. Lampert, ArXiv:2007.06705 (n.d.).

Preprint | Submitted | English
Department
Abstract
A natural approach to generative modeling of videos is to represent them as a composition of moving objects. Recent works model a set of 2D sprites over a slowly-varying background, but without considering the underlying 3D scene that gives rise to them. We instead propose to model a video as the view seen while moving through a scene with multiple 3D objects and a 3D background. Our model is trained from monocular videos without any supervision, yet learns to generate coherent 3D scenes containing several moving objects. We conduct detailed experiments on two datasets, going beyond the visual complexity supported by state-of-the-art generative approaches. We evaluate our method on depth-prediction and 3D object detection---tasks which cannot be addressed by those earlier works---and show it out-performs them even on 2D instance segmentation and tracking.
Publishing Year
Date Published
2020-07-07
Journal Title
arXiv:2007.06705
Article Number
2007.06705
Page
29
IST-REx-ID

Cite this

Henderson PM, Lampert C. Unsupervised object-centric video generation and decomposition in 3D. arXiv:200706705.
Henderson, P. M., & Lampert, C. (n.d.). Unsupervised object-centric video generation and decomposition in 3D. ArXiv:2007.06705.
Henderson, Paul M, and Christoph Lampert. “Unsupervised Object-Centric Video Generation and Decomposition in 3D.” ArXiv:2007.06705, n.d.
P. M. Henderson and C. Lampert, “Unsupervised object-centric video generation and decomposition in 3D,” arXiv:2007.06705. .
Henderson PM, Lampert C. Unsupervised object-centric video generation and decomposition in 3D. arXiv:2007.06705.
Henderson, Paul M., and Christoph Lampert. “Unsupervised Object-Centric Video Generation and Decomposition in 3D.” ArXiv:2007.06705, 2007.06705.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 2007.06705

Search this title in

Google Scholar