doi:

DOI: 10.3724/SP.J.1041.2017.01125

Acta Psychologica Sinica (心理学报) 2017/49:8 PP.1125-1136

Simulated data comparison of the predictive validity between bi-factor and high-order models


Abstract:
Psychological and educational researchers are often confronted with multifaceted constructs which are comprised of several related dimensions. Bi-factor and high order factor models, which were regarded as two important methods to measure the multifaceted constructs, had been widely applied in many research fields. Generally speaking, a high order factor model was nested in the respective bi-factor model. These two models were equivalent, however, only when the proportionality constraints were imposed. Many researchers compared these two models from the perspective of model-fit. Empirical or simulation studies which were based on empirical data almost overwhelmingly found that the bi-factor model was better than the high-order factor model in model fit. In contrast, however, several simulation studies found that there was no significant difference between these two models in model fit indices. Hence, it is rather difficult to conclude which model is the best one. In addition to model fit, predictive validity is also an important aspect of models' construct validity with no researcher comparing the predictive validity of these two models so far. It is thus important to compare these two models from the perspective of predictive validity, and to find out which of these two models is the better one.
Based on the comparison of model fit between bi-factor and high-order factor models, this research focuses on the differences in the predictive accuracy of these two models in two studies using the Monte Carlo simulation method. Study One compared the difference on model fit indices between the two models under different levels of factor loadings (0.4, 0.5, 0.55, 0.6, 0.7). Their difference in predictive accuracy when the criterion was an exogenous variable was compared. To ensure the conclusions have wider generalizability, Study Two focused on the comparison of the predictive validity between the two models under the same experimental conditions when the criterion was an endogenous variable. The sample size was fixed at 1000 in all condition.
The results of the simulation study suggested that, the bi-factor model and high order factor model both perfectly fitted these generated data under any conditions, and there was no significant difference between two models in model fit. When the criterion was an exogenous variable, the biases of the structural coefficients for both bi-factor model and high order factor model were small. In both models, it was more than ten percent bias in one case only, with the probability 1/80. It can be concluded that the two models' structural coefficients were unbiased estimates. When the criterion was an endogenous variable, the structural coefficient biases of the high order factor model were all within 10%. There were, however, about 50% of the structural coefficient of the bi-factor model had a bias greater than 10%.
Generally speaking, for bi-factor models, they can determine domain factors' effect size through its factor loadings. When compared to regular SEMs, such as high-order factor models, however, starting values for the models might have to be provided for convergence. For high order models, they are more compact and superior in parameter estimation. To conclude, compared with high order factor models, bi-factor models were superior in determining its effect size through domain factors' loading but they had with no advantage in model fit. High-order factor models were better than bi-factor model when the general and domain factors were used to predict criterion.

Key words:bi-factor model,high-order factor model,model-fit,structural coefficient bias

ReleaseDate:2017-08-31 10:09:57



Aiken, L. R., & Groth-Marnat, G. (2011). Psychological testing and assessment (12th ed.). Boston:Allyn & Bacon. (Original work published 2005).

[艾肯, 格罗恩-马纳特. (2011). 艾肯心理测量与评估 (张厚璨, 赵守盈 译). 北京:中国人民大学出版社.]

Bandalos, D. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling:A Multidisciplinary Journal, 9(1), 78-102.

Bentler, P. M. (1995). EQS 6 structural equations program manual. Encino, CA:Multivariate Software.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester:John Wiley and Sons.

Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J.-P., & Zhang, Z. G. (2012). Modeling general and specific variance in multifaceted constructs:A comparison of the bifactor model to other approaches. Journal of Personality, 80(1), 219-251.

Chen, F. F., Jing, Y. M., Hayes, A., & Lee, J. M. (2013). Two concepts or two Approaches? A bifactor analysis of psychological and subjective well-being. Journal of Happiness Studies, 14(3), 1033-1068.

Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189-225.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302.

Gillmore, M. R., Hawkins, J. D., Catalano, R. F., Jr., Day, L. E., Moore, M., & Abbott, R. (1991). Structure of problem behaviors in preadolescence. Journal of Consulting and Clinical Psychology, 59(4), 499-506.

Gu, H. L., Wen, Z. L., & Fang, J. (2014). Bi-factor models:A new measurement perspective of multidimensional constructs. Journal of Psychological Science, 37(4), 973-979.

[顾红磊, 温忠麟, 方杰. (2014). 双因子模型:多维构念测量的新视角. 心理科学, 37(4), 973-979.]

Gustafsson, J.-E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28(4), 407-434.

Holzinger, K. J., & Swineford, F. (1937). The bi-factor method. Psychometrika, 2(1), 41-54.

Howard, J. L., Gagné, M., Morin, A. J. S., & Forest, J. (2016). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. Journal of Management, in press.

Hull, J. G., Lehn, D. A., & Tedlie, J. C. (1991). A general approach to testing multifaceted personality constructs. Journal of Personality and Social Psychology, 61(6), 932-945.

Hyland, P., Boduszek, D., Dhingra, K., Shevlin, M., & Egan, A. (2014). A bifactor approach to modelling the rosenberg self esteem scale. Personality and Individual Differences, 66, 188-192.

Jennrich, R. I., & Bentler, P. M. (2012). Exploratory bi-factor analysis:The oblique case. Psychometrika, 77(3), 442-454.

Li, Z. H., Yin, X. Y., Cai, T. S., & Zhu, C. Y. (2013). The structure of dispositional optimism:Ttraditional factor models and bifactor model. Chinese Journal of Clinical Psychology, 21(1), 45-47, 105.

[黎志华, 尹霞云, 蔡太生, 朱翠英. (2013). 特质乐观的结构:传统因素模型与双因素模型. 中国临床心理学杂志, 21(1), 45-47, 105.]

Morgan, G., Hodge, K. J., Wells, K. E., & Watkins, M. M. (2015). Are fit indices biased in favor of bi-factor models in cognitive ability research?:A comparison of fit in correlated factors, higher-order, and bi-factor models via monte carlo simulations. Journal of Intelligence, 3(1), 2-20.

Mulaik, S. A., & Quartetti, D. A. (1997). First order or higher order general factor? Structural Equation Modeling:A Multidisciplinary Journal, 4(3), 193-211.

Musek, J. (2007). A general factor of personality:Evidence for the big one in the five-factor model. Journal of Research in Personality, 41(6), 1213-1233.

Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52(3), 431-462.

Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667-696.

Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations:Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544-559.

Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16(S1), 19-31.

Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling:A bifactor perspective. Educational and Psychological Measurement, 73(1), 5-26.

Rindskopf, D., & Rose, T. (1988). Some theory and applications of confirmatory second-order factor analysis. Multivariate Behavioral Research, 23(1), 51-67.

Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models:Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137-150.

Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22(1), 53-61.

Spearman, C. (1927). The abilities of man:Their nature and measurement. New York:MacMillan.

Wang, D. F., & Cui, H. (2005). Explorations of Chinese personality. Beijing, China:Social Sciences Academic Press.

[王登峰, 崔红. (2005). 解读中国人的人格. 北京:社会科学文献出版社.]

Watters, C. A., Keefer, K. V., Kloosterman, P. H., Summerfeldt, L. J., & Parker, J. D. A. (2013). Examining the structure of the internet addiction test in adolescents:A bifactor approach. Computers in Human Behavior, 29(6), 2294-2302.

Yung, Y.-F., Thissen, D., & McLeod, L. D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64(2), 113-128.