engleski

Novel statistical parameters for model quality estimation

The choice of model quality evaluation parameters is a very important decision in selecting the best model developed in an attempt to relate property/activity of molecules with their structure described by molecular descriptors. Quality evaluation parameters are statistical parameters, like correlation coefficient (R) or standard error of estimate (S) together with other analogous parameters, calculated between an experimental set of values and those estimated or predicted by the model. The size of data set (i.e. the number of compounds in data set) and the number of optimized parameters in the model determine the number of degrees of freedom of the problem, which is further used in assessing the significance of statistical parameters and confidence interval. However, in many problems in chemistry or life sciences, the distribution of data is drastically skewed, having in data set only a few active compounds and a lot of inactive ones. In that case, standard model quality evaluation parameters (R, S) could be over-optimistic. If the data set is very large, the obtained parameters will be highly significant. To overcome this problem the concept of chance correlation is introduced. Because standard parameters like R or S do not include information about the chance accuracy, novel parameters are defined that take it into account. Their values give information about the real contribution of the model over the most probable chance accuracy. An overview and comparison of these parameters will be given and their usefulness will be illustrated on several two-state classification problems. Possibility of generalization of these parameters to classification problems with more than two classes will be analyzed.

model validation ; model quality evaluation ; chance accuracy ; classification problem

Participant - lecturer: Bono Lučić