Warp to the Future: Joint Forecasting of Features and Feature Motion (CROSBI ID 689835)
Prilog sa skupa u zborniku | izvorni znanstveni rad | međunarodna recenzija
Podaci o odgovornosti
Šarić, Josip ; Oršić, Marin ; Antunović, Tonći ; Vražić, Sacha ; Šegvić, Siniša
engleski
Warp to the Future: Joint Forecasting of Features and Feature Motion
We address anticipation of scene development by forecasting semantic segmentation of future frames. Several previous works approach this problem by F2F (feature-to-feature) forecasting where future features are regressed from observed features. Different from previous work, we consider a novel F2M (feature-to-motion) formulation, which performs the forecast by warping observed features according to regressed feature flow. This formulation models a causal relationship between the past and the future, and regularizes inference by reducing dimensionality of the forecasting target. However, emergence of future scenery which was not visible in observed frames can not be explained by warping. We propose to address this issue by complementing F2M forecasting with the classic F2F approach. We realize this idea as a multi-head F2MF model built atop shared features. Experiments show that the F2M head prevails in static parts of the scene while the F2F head kicks-in to fill-in the novel regions. The proposed F2MF model operates in synergy with correlation features and outperforms all previous approaches both in short-term and mid-term forecast on the Cityscapes dataset.
Computer vision, semantic segmentation, semantic forecasting
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
Podaci o prilogu
1-10.
2020.
objavljeno
Podaci o matičnoj publikaciji
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Institute of Electrical and Electronics Engineers (IEEE)
Podaci o skupu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)
poster
14.06.2020-19.06.2020
Seattle (WA), Sjedinjene Američke Države