Nalazite se na CroRIS probnoj okolini. Ovdje evidentirani podaci neće biti pohranjeni u Informacijskom sustavu znanosti RH. Ako je ovo greška, CroRIS produkcijskoj okolini moguće je pristupi putem poveznice www.croris.hr
izvor podataka: crosbi !

TVOR: Finding Discrete Total Variation Outliers Among Histograms (CROSBI ID 288232)

Prilog u časopisu | izvorni znanstveni rad | međunarodna recenzija

Banić, Nikola ; Elezović, Neven TVOR: Finding Discrete Total Variation Outliers Among Histograms // IEEE access, 9 (2021), 1807-1832. doi: 10.1109/ACCESS.2020.3047342

Podaci o odgovornosti

Banić, Nikola ; Elezović, Neven

engleski

TVOR: Finding Discrete Total Variation Outliers Among Histograms

Pearson's chi-squared test can detect outliers in the data distribution of a given set of histograms. However, in fields such as demographics (for e.g. birth years), outliers may be more easily found in terms of the histogram smoothness where techniques such as Whipple's or Myers' indices handle successfully only specific anomalies. This paper proposes smoothness outliers detection among histograms by using the relation between their discrete total variations (DTV) and their respective sample sizes. This relation is mathematically derived to be applicable in all cases and simplified by an accurate linear model. The deviation of the histogram's DTV from the value predicted by the model is used as the outlier score and the proposed method is named Total Variation Outlier Recognizer (TVOR). TVOR requires no prior assumptions about the histograms' samples' distribution, it has no hyperparameters that require tuning, it is not limited to only specific patterns, and it is applicable to histograms with the same bins. Each bin can have an arbitrary interval that can also be unbounded. TVOR finds DTV outliers easier than Pearson's chi-squared test. In case of distribution outliers, the opposite holds. TVOR is tested on real census data and it successfully finds suspicious histograms. The source code is given at https://github.com/DiscreteTotalVariation/TVOR.

age heaping ; anomaly detection ; discrete total variation ; expected value ; fitting ; histogram ; Myers' index ; outlier detection ; Pearson's chi-squared test ; total variation ; Whipple's index

nije evidentirano

nije evidentirano

nije evidentirano

nije evidentirano

nije evidentirano

nije evidentirano

Podaci o izdanju

9

2021.

1807-1832

objavljeno

2169-3536

10.1109/ACCESS.2020.3047342

Povezanost rada

Matematika, Računarstvo

Poveznice
Indeksiranost