engleski

Maintaining Exam Security Through Digitalisation: An Application of Psychometric Forensics to Improve Validity

One of the greatest concerns and challenges in the digitalization of assessments - even without the pandemic - is ensuring that a test remains secure and there are no aberrant test-taking behaviors. The most common and famous of these is cheating, though there are other types. Many presentations and products focus on the delivery aspect, such as remote proctoring, and rightly so. But additionally, what approaches can a test sponsor take to evaluate the integrity of their assessments by analyzing the results? This can provide some icing on the test security cake. Moreover, many of these apply to non-digital exams. We will begin with an overview of the types of psychometric forensics in the scientific literature, and what sort of aberrant behaviors they might be appropriate to identify. For example, some seek to identify examinees that are copying (similarity), while others seek to identify those who had pre-knowledge. We will then discuss a research study that compared indices in evaluating assessment integrity for national assessments in a European country. Specifically, we sought to compare the percentage and overlap of examinees flagged for cheating using an index that is based on a comparison of all examinee’s responses, compared to an index that compares only incorrect responses. We then discuss how the results were evaluated at an aggregate level to identify schools or classrooms that warranted further investigation, in concert with other statistics.

assessment, validity, cheating, psychometric forensics

nije evidentirano