Kraus, Elisabeth Barbara and Wild, Johannes and Hilbert, Sven (2024) Using Interpretable Machine Learning for Differential Item Functioning Detection in Psychometric Tests. APPLIED PSYCHOLOGICAL MEASUREMENT, 48 (4-5). pp. 167-186. ISSN 0146-6216, 1552-3497
Full text not available from this repository. (Request a copy)Abstract
This study presents a novel method to investigate test fairness and differential item functioning combining psychometrics and machine learning. Test unfairness manifests itself in systematic and demographically imbalanced influences of confounding constructs on residual variances in psychometric modeling. Our method aims to account for resulting complex relationships between response patterns and demographic attributes. Specifically, it measures the importance of individual test items, and latent ability scores in comparison to a random baseline variable when predicting demographic characteristics. We conducted a simulation study to examine the functionality of our method under various conditions such as linear and complex impact, unfairness and varying number of factors, unfair items, and varying test length. We found that our method detects unfair items as reliably as Mantel-Haenszel statistics or logistic regression analyses but generalizes to multidimensional scales in a straight forward manner. To apply the method, we used random forests to predict migration backgrounds from ability scores and single items of an elementary school reading comprehension test. One item was found to be unfair according to all proposed decision criteria. Further analysis of the item's content provided plausible explanations for this finding. Analysis code is available at: https://osf.io/s57rw/?view_only=47a3564028d64758982730c6d9c6c547.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | psychometrics; machine learning; interpretable machine learning; random forest; test fairness; differential item functioning |
| Subjects: | 000 Computer science, information & general works > 004 Computer science 300 Social sciences > 300 Social sciences 400 Language > 400 Language, Linguistics |
| Divisions: | Human Sciences > Institut für Bildungswissenschaft > Professur für Methoden der empirischen Bildungsforschung - Prof. Dr. Sven Hilbert Languages and Literatures > Institut für Germanistik > Lehrstuhl für Didaktik der Deutschen Sprache und Literatur (Prof. Dr. Anita Schilcher) |
| Depositing User: | Dr. Gernot Deinzer |
| Date Deposited: | 30 Oct 2025 07:35 |
| Last Modified: | 30 Oct 2025 07:35 |
| URI: | https://pred.uni-regensburg.de/id/eprint/64592 |
Actions (login required)
 |
View Item |
