engleski

Can cognitive reflection measurement be improved? Psychometric properties of the longer Cognitive reflection test

Cognitive reflection test (CRT) assesses reflective thinking ability, i.e. the ability to resist reporting the response that first comes to mind and coming up with correct answer after thinking about the problem more carefully. The distribution of responses on these tasks is bimodal: people give either correct or intuitive, incorrect response. However, the test has several important shortcomings. First, it consists of 3 items only which negatively affects its reliability. Second, the items are generally difficult, often leading to the floor effect. Finally, the test became familiar within general population which impairs its usability. This study aims to address these problems, i.e. to develop and validate a longer version of the test that will be informative for a broader range of the underlying ability than the original. 240 participants solved 17 CRT tasks and, to check the convergent validity of the test, syllogistic reasoning tasks, cognitive ability test, numeracy test and active open-minded thinking questionnaire (AOT). Based on low inter- correlations and discrimination parameters, four items were omitted so that the final version of the test consists of 13 tasks (CRT-13). Compared to the original test (CRT-3), CRT13 had substantially higher reliability, and preliminary IRT analysis showed that it is informative for a broader range of the latent trait. Correlations between CRT-13 and other measured variables were all in the expected direction and of expected magnitude. Finally, to confirm that CRT-13 is not just another measure of numeracy and cognitive ability, we conducted the regression analysis with syllogistic reasoning score as the outcome. CRT-13 explained significant proportion of the variance in syllogistic reasoning even after controlling for numeracy, cognitive ability and AOT, suggesting that the CRT-13 is a valid measure of cognitive reflection, capable of measuring a wider range of the trait with high precision than the original version.

cognitive reflection test ; cognitive ability ; numeracy ; IRT

nije evidentirano