Alpha Coefficient for Validating Scales Scores' Reliability: Facts and Misconceptions

Abohashem, Elsayed

doi:10.21608/jsre.2021.137152

Alpha Coefficient for Validating Scales Scores' Reliability: Facts and Misconceptions

Document Type : Original Article

Author

Elsayed Abohashem

King Saud University

10.21608/jsre.2021.137152

Abstract

Abstract
The current research aims to document and analyze the most common misconceptions about the alpha coefficient among researchers, and the most important of these beliefs: (1) Alpha was first developed by Cronbach , (2) the use of alpha without checking its Assumption, (3) the Alpha equals reliability , (4) the high value of the alpha coefficient is an indication of internal consistency and homogeneity, (5) the reliability of the scale grades can be improved by deleting some items, (6) the alpha coefficient value must be equal to or greater than (0.7 or 0.8), (7) the alpha coefficient It is the best among all published reliability coefficients, (8) the alpha is a fixed property to describe the scale, (9) the alpha coefficient measures the internal consistency of the gauge, (10) the largest alpha coefficient is always the best, (11) the alpha coefficient value runs from zero to one, (12) alpha coefficient statistically significant , (13) the increase in the number of items leads to high values for the alpha coefficient, (14) the increase in the number of alternatives leads to high values of the alpha coefficient, (15) the difference in the direction of the item leads to high values of the alpha coefficient, (16) the increase in size The sample leads to high values for the alpha coefficient. Lastly, some recommendations are offered to correct and address these popular misconceptions held by researchers about Alpha coefficient.

Keywords

Main Subjects

Educational psychology

References

Anastasi, A. & Urbina, S. (1997). Psychological Testing. (7^th ed.). Upper Saddle River, New Jersey: Prentice Hall.

Bonett, D. & Wright,T.(2014). Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Journal of Organizational Behavior, 36(1), 3-15.

BrckaLorenz, A., Chiang, Y. & Nelson Laird, T. (2013). Internal Consistency Reliability, FSSE Psychometric Portfolio, 160, 1-4

Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 3(3), 296-322.

Christmann, A. & Aelst, S. (2006). Robust estimation of Cronbach’s alpha. Journal of Multivariate Analysis, 97, 1660 – 1674.

Cho, E., & Kim, S. (2015). Cronbach’s Coefficient Alpha: Well Known but Poorly Understood. Organizational Research Methods, 18(2), 207-230.

Cho, E., (2016) “Making reliability reliable: A systematic approach to reliability coefficients,” Organizational Research Methods, 19 (4), 651–682.

Choudhury, S. & Bhattacharjee , D. (2014). Optimal number of scale points in likert type scales for quantifying compulsive buying behaviour. Asian Journal of Management Research, 4(3), 432 -440.

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98-104.

Cronbach, L. J & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391-418.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.

Davenport, E. C., Davison, M. L., Liou, P. Y., & Love, Q. U. (2015). Reliability, Dimensionality, and Internal Consistency as Defined by Cronbach: Distinct Albeit Related Concepts. Educational Measurement: Issues and Practice, 34(4), 4-9.

Drost, E. (2011). Validity and Reliability in Social Science Research. Education Research and Perspectives, 38(1):105-124.

Dunn,T. Baguley, T. and Brunsden, W. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Psychological Society, 105(3), 399-412.

Ercan ,I. Yazici,B. Sigirli ,D. Ediz ,B. and Kan,I. (2007).Examining Cronbach Alpha, Theta, Omega Reliability Coefficients According to the Sample Size. Journal of Modern Applied Statistical Methods. 6(1), 291-303.

Finch, W.& French, B. (2019). Educational and Psychological Measurement. New York, Taylor & Francis.

George, D. & Mallery, P (2020). IBM SPSS Statistics 26 Step by Step: A Simple Guide and Reference. 6^th ed. Taylor & Francis.

Graham, J. (2006). Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability What They Are and How to Use Them. Educational and Psychological Measurement, 66(6) ,930-944.

Hinton, P., McMurray,I.& Brownlow, C.(2014). SPSS Explained. East Sussex, Second edition, England, Routledge Inc.

Hogan, T. (2015). Psychological Testing a Practical Introduction. Third Edition. John Wiley & Sons.

Javali, S., Gudaganavar, N. and Raj, S. (2011) Effect of Varying Sample Size in Estimation of Coefficients of Internal Consistency. Downloaded from http://www.webmedcentral.com on 22-Dec-2011.

Kopalle, P. K., & Lehmann, D. R. (1997). Alpha inflation? The impact of eliminating scale items on Cronbach’s alpha. Organizational Behavior and Human Decision Processes, 70(3), 189- 197.

Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria. Organizational Research Methods, 9(2), 202-220.

Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 4(2), 73–79.

McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433.

Morera, O. &Stokes, S. (2016). Coefficient α as a Measure of Test Score Reliability: Review of 3 Popular Misconceptions. Statistics,106(3),458-461.

Nunnally , J. & Berstein , I.( 1994). Psychometric Theory (3^rd ed.). New York, McGraw Hill.

Panayides,P.(2013). Coefficient Alpha Interpret with Caution. Europe's Journal of Psychology, 9(4), 687-696.

Peterson, R. A. (1994). A meta-analysis of Cronbach’s coefficient alpha. Journal of Consumer Research, 21, 381-391.

Price, L. (2017). Psychometric Methods Theory into Practice. New York, Guilford Press

Raykov, T. & Marcoulides, G. (2011). Introduction to Psychometric Theory. New York, Taylor & Francis Group.

Raykov, T. (2004). Point and interval estimation of reliability for multiple components measuring instruments via linear constraint covariance Structure modeling. Structural Equation Modeling, 11(3), 342-356.

Raykov, T. (2007). Reliability if deleted, not “alpha if deleted”: Evaluation of scale reliability following component deletion. British Journal of Mathematical and Statistical Psychology, 60(2), 201-216.

Raykov, T. (2008). Alpha if item deleted: A note on criterion validity loss in scale revision if maximizing coefficient alpha. British Journal of Mathematical and Statistical Psychology, 61(2), 275-285.

Revelle,W.and Zinbarg, R(2009). Coefficients alpha, beta, omega and the GLB: Comments on Sijtsma. Psychometrical, 74(1), 145-154.

Ritter, N.L. (2010). Understanding a Widely Misunderstood Statistic: Cronbach’s α. Paper presented at the annual meeting of the Southwest Educational Research Association, New Orleans, February 18, 2010.

Rodriguez, M and Maeda, Y. (2006). Meta-Analysis of Coefficient Alpha. Psychological Methods, 11(3), 306-322.

Salvucci, S., Walter, E., Conley, V., Fink, S., & Saba, M. (1997). Measurement error studies at the National Center for Education Statistics (NCES). Washington D. C.: U. S. Department of Education.

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350- 353.

Schmidt, F. L.; & Hunter, J. E., (1996). Measurement error in psychological research: Lessons from 26 research scenario. Psychological Methods, 1(2), 199-223.

Serbetar I., & SedlarI. (2016). Assessing Reliability of a Multi-Dimensional Scale by Coefficient Alpha. Journal of Elementary Education, 9(1/2), 189-196.

Sharma, B (2016). A focus on reliability in developmental research through Cronbach’s Alpha among medical, dental and paramedical professionals, Asian Pac. J. Health Sci., 3 (4):271-278.

Sijtsma, K. (2015). Delimiting coefficient alpha from internal consistency and unidimensionality. Educational Measurement: Issues and Practices, 34(4), 10–13.

Souza, A., Alexandre, N. & Guirardello, E. (2017). Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia e Servicos de Saude, Brasília, 26(3),649-659.

Spearman, C. (1910). Correlation calculated from faulty data. British Journal Psychology, 3(3), 271-295.

Spiliotopoulou, G. (2009). Reliability reconsidered: Cronbach's alpha and paediatric assessment in occupational therapy. Australian Occupational Therapy, 56, 150- 155.

Streiner, D. L. (2003A). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99-103.

Streiner, D. L. (2003b). Being inconsistent about consistency: When coefficient alpha does and doesn't matter. Journal of Personality Assessment, 80(3), 217–222.

Taber, K. (2018). The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education. 48, 1273-1296.

Tan, S. (2009). Misuses of KR-20 and Cronbach’s Alpha Reliability Coefficients. Education and Science, 34(152), 101- 112.

Tang, W, Cui, Y, Babenko , O (2014).Internal Consistency: Do We Really Know What It Is and How to Assess It? Journal of Psychology and Behavioral Science, 2(2) ,205-220.

Vaske, J. J., Beaman, J., & Sponarski, C. (2016). Rethinking internal consistency in Cronbach's alpha. Leisure Sciences, 39(2), 1-11.

Weng, L. (2004). Impact of the Number of Response Categories and Anchor Labels on Coefficient Alpha and Test-Retest Reliability. Educational and Psychological Measurement, 64(6),956-972.

Wilkinson, L. and the Task Force on Statistical Inference APA Board of Scientific Affairs (1999). Statistical Methods in Psychology Journals Guidelines and Explanations. American Psychological, 54(8), 594-604.

Wong,C. , Pang, K., Shi, J. & Mao , Y.(2011). Differences between odd number and even number response formats: Evidence from mainland Chinese respondents. Asian Pacific Journal of Management, 28(6), 379–399.

Yang, Y., Green, S. (2011). Coefficient alpha: A reliability coefficient for the 21 century? Journal of Psychoeducational Assessment, 29(4), 377–392.

Zumbo, B. Gadermann, A. and Zeisser, C. (2007). Ordinal Versions of Coefficients Alpha and Theta for Likert Rating Scales. Journal of Modern Applied Statistical Methods.6 (1),21-29.