doi:

DOI: 10.3724/SP.J.1042.2017.00701

Advances in Psychological Science (心理科学进展) 2017/25:4 PP.701-712

Advantages and illustrations of application of response time model in psychological and educational testing


Abstract:
The use of computer-based assessments makes it possible to collect response time in psychological and educational testing contexts. The collection of response time has significant implications for improving psychological and educational testing theories and practice. The author first introduces five advantages of utilizing response time, including its ability to distinguish test takers' genuine ability from their speed, investigate the speed-accuracy trade-off, detect aberrant behaviors during the testing, improve the efficiency of item selection methods in the computerized adaptive testing (CAT), and compare within-individual test-taking strategies. Then, the model features of response time models with different construction orientations are elaborated in the context of psychological and educational testing, and these models are evaluated comprehensively. Moreover, current applications of the response time models in psychological and educational testing are systematically reviewed and analyzed. Finally, directions for future research in response time in psychological and educational testing are discussed.

Key words:response time,psychological and educational testing,item response theory,response time model

ReleaseDate:2017-05-12 17:00:09



郭磊. (2014). 变长认知诊断计算机化自适应测验: 终止规则、曝光控制及题库质量监控技术(博士学位论文). 北京师范大学.

李海江, 杨娟, 贾磊, 张庆林. (2011). 不同自尊水平者的注意偏向. 心理学报, 43, 907-916.

杨惠兰, 何先友, 赵雪汝, 张维. (2015). 权力的概念隐喻表征: 来自大小与颜色隐喻的证据. 心理学报, 47, 939-949.

Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society. Series B (Methodological), 26, 211-252.

Entink, R. H. K., Fox, J. P., & van der Linden, W. J. (2009). A multivariate multilevel approach to the modeling of accuracy and speed of test takers. Psychometrika, 74, 21-48.

Entink, R. H. K., van der Linden, W. J., & Fox, J. P. (2009). A Box-Cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62, 621-640.

Fan, Z. W., Wang, C., Chang, H. H., & Douglas, J. (2012). Utilizing response time distributions for item selection in CAT. Journal of Educational and Behavioral Statistics, 37, 655-670.

Ferrando, P. J. (2006). Person-item distance and response time: An empirical study in personality measurement. Psicológica, 27, 137-148.

Ferrando, P. J., & Lorenzo-Seva, U. (2007). An item response theory model for incorporating response time data in binary personality items. Applied Psychological Measurement, 31, 525-543.

Finkelman, M. D., Kim, W., Weissman, A., & Cook, R. J. (2014). Cognitive diagnostic models and computerized adaptive testing: Two new item-selection methods that incorporate response times. Journal of Computerized Adaptive Testing, 2, 59-76.

Gulliksen, H. (1950). Theory of mental tests. New York: John Wiley.

Lee, Y. H., & Chen, H. W. (2011). A review of recent response-time analyses in educational testing. Psychological Test and Assessment Modeling, 53, 359-379.

Lee, Y. H., & Ying, Z. L. (2015). A mixture cure-rate model for responses and response times in time-limit tests. Psychometrika, 80, 748-775.

Loeys, T., Legrand, C., Schettino, A., & Pourtois, G. (2014). Semi-parametric proportional hazards models with crossed random effects for psychometric response times. British Journal of Mathematical and Statistical Psychology, 67, 304-327.

Luce, R. D. (1986). Response times. New York: Oxford University Press.

Metzler, J., & Shepard, R. N. (1974). Tranformational studies of the internal representation of three-dimensional objects. In Solso, R. L. (Ed.). Theories in cognitive psychology: the Loyola symposium (pp. 147-201). Oxford, England: Lawrence Erlbaum.

Mislevy, R. J., Wingersky, M. S., Irvine, S. H., & Dann, P. L. (1991). Resolving mixtures of strategies in spatial visualization tasks. British Journal of Mathematical and Statistical Psychology, 44, 265-288.

Ranger, J. (2013). Modeling responses and response times in personality tests with rating scales. Psychological Test and Assessment Modeling, 55, 361-382.

Ranger, J., & Kuhn. J. T. (2012a). A flexible latent trait model for response times in tests. Psychometrika, 77, 31-47.

Ranger, J., & Kuhn, J. T. (2012b). Improving item response theory model calibration by considering response times in psychological tests. Applied Psychological Measurement, 36, 214-231.

Ranger, J., Kuhn, J. T., & Gaviria, J. L. (2015). A race model for responses and response times in tests. Psychometrika, 80, 791-810.

Ranger, J., & Ortner, T. M. (2011). Assessing personality traits through response latencies using item response theory. Educational and Psychological Measurement, 71, 389-406.

Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333-367.

Rouder, J. N., Province, J. M., Morey, R. D., Gomez, P., & Heathcote, A. (2015). The lognormal race: A cognitive- process model of choice and latency with desirable psychometric properties. Psychometrika, 80, 491-513.

Tatsuoka, K. K., & Tatsuoka, M. M. (1980). A model for incorporating response-time data in scoring achievement tests. In Weiss, D. J. (Ed.). Proceedings of the 1979 computerized adaptive testing conference (pp. 236-256). Minneapolis, MN: University of Minnesota, Department of Psychology, Psychometric Methods Program.

Thissen, D. (1983). Timed testing: An approach using item response theory. In Weiss, D. J. (Ed.). New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179-203). New York: Academic Press.

van der Linden, W. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181-204.

van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287-308.

van der Linden, W. J. (2008). Using response times for item selection in adaptive testing. Journal of Educational and Behavioral Statistics, 33, 5-20.

van der Linden, W. J. (2009a). A bivariate lognormal response-time model for the detection of collusion between test takers. Journal of Educational and Behavioral Statistics, 34, 378-394.

van der Linden, W. J. (2009b). Conceptual issues in response- time modeling. Journal of Educational Measurement, 46, 247-272.

van der Linden, W. J. (2011a). Test design and speededness. Journal of Educational Measurement, 48, 44-60.

van der Linden, W. J. (2011b). Setting time limits on tests. Applied Psychological Measurement, 35, 183-199.

van der Linden, W. J., Breithaupt, K., Chuah, S. C., & Zhang, Y. W. (2007). Detecting differential speededness in multistage testing. Journal of Educational Measurement, 44, 117-130.

van der Linden, W. J., & Guo, F. M. (2008). Bayesian procedures for identifying aberrant response-time patterns in adaptive testing. Psychometrika, 73, 365-384.

van der Linden, W. J., van Krimpen-Stroop, E. M. L. A. (2003). Using response times to detect aberrant responses in computerized adaptive testing. Psychometrika, 68, 251-265.

van der Linden, W. J., & Lewis, C. (2014). Bayesian checks on cheating on tests. Psychometrika, 80, 689-706.

van der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R. A., & Borsboom, D. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118, 339-356.

Wang, C., Chang, H. H. & Douglas, J. A. (2013). The linear transformation model with frailties for the analysis of item response times. British Journal of Mathematical and Statistical Psychology, 66, 144-168.

Wang, C., Fan, Z. W., Chang, H. H., & Douglas, J. (2013). A semiparametric model for jointly analyzing response times and accuracy in computerized testing. Journal of Educational and Behavioral Statistics, 38, 381-417.

Wang, C., & Xu, G. J. (2015). A mixture hierarchical model for response times and response accuracy. British Journal of Mathematical and Statistical Psychology, 68, 456-477.

Wang, T. Y., & Hanson, B. A. (2005). Development and calibration of an item response model that incorporates response time. Applied Psychological Measurement, 29, 323-339.

Wang, T. Y., & Zhang, J. W. (2006). Optimal partitioning of testing time: Theoretical properties and practical implications. Psychometrika, 71, 105-120.