Naslov
Densely connected normalizing flows

Autori
Grcić, Matej ; Grubišić, Ivan ; Šegvić, Siniša

Vrsta, podvrsta i kategorija rada
Radovi u zbornicima skupova, cjeloviti rad (in extenso), znanstveni

Izvornik
Advances in Neural Information Processing Systems / Ranzato, Marc'Aurelio ; Beygelzimer, Alina ; Nguyen, K. ; Liang, Percy S. ; Wortman Vaughan, Jennifer ; Dauphin, Yann - Online : NeurIPS, 2021, 1-15

Skup
35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Mjesto i datum
Toronto, Kanada; online, 06.12.2021. - 14.12.2021

Vrsta sudjelovanja
Poster

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
normalizing flows, dense connectivity

Sažetak
Normalizing flows are bijective mappings between inputs and latent representations with a fully factorized distribution. They are very attractive due to exact likelihood evaluation and efficient sampling. However, their effective capacity is often insufficient since the bijectivity constraint limits the model width. We address this issue by incrementally padding intermediate representations with noise. We precondition the noise in accordance with previous invertible units, which we describe as cross-unit coupling. Our invertible glow-like modules increase the model expressivity by fusing a densely connected block with Nyström self-attention. We refer to our architecture as DenseFlow since both cross-unit and intra-module couplings rely on dense connectivity. Experiments show significant improvements due to the proposed contributions and reveal state-of-the-art density estimation under moderate computing budgets.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo

POVEZANOST RADA