Naslov
Warp to the Future: Joint Forecasting of Features and Feature Motion

Autori
Šarić, Josip ; Oršić, Marin ; Antunović, Tonći ; Vražić, Sacha ; Šegvić, Siniša

Vrsta, podvrsta i kategorija rada
Radovi u zbornicima skupova, cjeloviti rad (in extenso), znanstveni

Izvornik
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) / - : Institute of Electrical and Electronics Engineers (IEEE), 2020, 1-10

Skup
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)

Mjesto i datum
Seattle (WA), Sjedinjene Američke Države, 14.06.2020. - 19.06.2020

Vrsta sudjelovanja
Poster

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
Computer vision, semantic segmentation, semantic forecasting

Sažetak
We address anticipation of scene development by forecasting semantic segmentation of future frames. Several previous works approach this problem by F2F (feature-to-feature) forecasting where future features are regressed from observed features. Different from previous work, we consider a novel F2M (feature-to-motion) formulation, which performs the forecast by warping observed features according to regressed feature flow. This formulation models a causal relationship between the past and the future, and regularizes inference by reducing dimensionality of the forecasting target. However, emergence of future scenery which was not visible in observed frames can not be explained by warping. We propose to address this issue by complementing F2M forecasting with the classic F2F approach. We realize this idea as a multi-head F2MF model built atop shared features. Experiments show that the F2M head prevails in static parts of the scene while the F2F head kicks-in to fill-in the novel regions. The proposed F2MF model operates in synergy with correlation features and outperforms all previous approaches both in short-term and mid-term forecast on the Cityscapes dataset.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo

POVEZANOST RADA