A Comparison of the Mantle-Hansel and Raju's Signed Area Measure Index Methods for Detecting Differential performance of Item of the learning styles index questionnaire

Document Type : Original Article

Author

Abstract

Abstract
The study aimed to a comparison of the Mantle-Hansel (MH) chi-square and Raju's Signed Area Measure index (RSAI) between the curves of the item properties with the (2PLM) Methods for Detecting Differential performance of Item of Learning Styles Index Questionnaire (LSIQ) considering Felder-Silverman model. It also aimed to find out the extent of agreement between the two methods of detecting (DIF) according to gender variable. The questionnaire included (44) items and the sample consisted of (813) students of Taibah University. Here are the most important findings:
Uniform DIF was detected in (3) items, equals (6.8%) using (MH) method; two items, favouring male group, and one item, favouring female group, but when using the (RSAI) method, a uniform DIF was detected in two items, equals (4.5%); one item,  favouring male group, and one item,  favouring female group.
The percentage of agreement between the (MH) and the (RSAI) methods was high, as the two methods reveal two items at a rate of (66.6%) out of (3) items that showed (DIF) due to gender variable. Moreover, the value of the coefficient of agreement kappa (0.79) and is considered a substantial agreement, The qualitative analysis that carried out to the three items having uniform (DIF) revealed that the nature of Humanistic education and difference in gender between male and female may result in showing uniform (DIF) in these items. Therefore, we are going to maintain them within the remaining items of the questionnaire.
 
 

Keywords

Main Subjects

References

المراجع الأجنبيَّة
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. DC: American Educational Research Association.
Cameron, I. M., Scott, N. W., Adler, M., & Reid, I. C. (2014). A comparison of three methods of assessing differential item functioning (DIF) in the Hospital Anxiety Depression Scale: Ordinal logistic regression, Rasch analysis and the Mantel chi-square procedure. Quality of Life Research, 23(10), 2883-2888. https://doi.org/10.1007/s11136-014-0719-3
Camilli, G. (2006). Test fairness. Educational Measurement, 4, 221-256
Cánovas, R., Espínola, M., Iribarne, L., & Cimadevilla, J. M. (2008). A new virtual task to evaluate human place learning. Behavioural Brain Research, 190(1), 112-118. https://doi.org/10.1016/j.bbr.2008.02.024
Cassidy, S. (2004). Learning styles: An overview of theories, models, and measures. Educational psychology, 24(4), 419-444. https://doi.org/10.1080/01443410 42000228834
Chen, N. S., Liu, C. C., Kinshuk, & Wei, C. W. (2011). Effects of matching teaching strategy to thinking style on learner’s quality of reflection in an online learning environment. Computers & Education, 56(1), 53-64. https://doi.org/10.1016/j.compedu.2010.08.021
Chiu, P. C. (2008). The effect of English proficiency on mathematics performance: A comparison of Item Response Theory-based area and Mantel-Haenszel methods (Doctoral dissertation, University of Kansas)
Cook, D. A. (2005). Reliability and validity of scores from the index of learning styles. Academic Medicine, 80(10), S97-S101
Escarlos, J. A., & Escarlos, G. S. (2018). Gender Differences in Learning Style Preferences Among Veterinary Medicine Students. International Journal of Scientific & Technology Research, 7(6), 50-53.
Elena Oliveri, M., Lawless, R., Robin, F., & Bridgeman, B. (2018). An exploratory analysis of differential item functioning and its possible sources in a higher education admissions context. Applied Measurement in Education, 31(1), 1-16 https://doi.org/10.1080/08957347.2017.1391258
Felder, R. M., & Silverman, L. K. (1988). Learning and teaching styles in Engineering Education. Engineering education, 78(7), 674-681.
Felder, R. M. & Spurlin, J. (2005). Applications, reliability and validity of the index of learning styles. International Journal of Engineering Education, 21(1): 103–112.
Genovese, J. E. (2004). The Index of Learning Styles: An Investigation of its Reliability and Concurrent Validity with the Preference Test. Individual Differences Research, 2(3), 169-172.
Gómez-Benito, J., & Navas-Ara, M. J. (2000). A Comparison of χ2, RFA and IRT Based Procedures in the Detection of DIF. Quality and Quantity, 34(1), 17-31. https://doi.org/10.1023/A:1004703709442
Gómez-Benito, J., Hidalgo, M. D., & Guilera, G. (2010). Bias in measurement instruments. Fair tests. Papeles del Psicólogo, 31(1), 75-84
Gur, R. C., Turetsky, B. I., Matsui, M., Yan, M., Bilker, W., Hughett, P., & Gur, R. E. (1999). Sex differences in brain gray and white matter in healthy young adults: Correlations with cognitive performance. Journal of Neuroscience, 19(10), 4065-4072. https://doi.org/10.1523/JNEUROSCI.19-10-04065.1999
Guyer, R., & Thompson, N.A. (2014). User’s Manual for Xcalibre item response theory calibration software, version 4.2.2 and later. Woodbury MN: Assessment Systems Corporation.
Halpern, D. F., Benbow, C. P., Geary, D.C., Hyde, J.S., & Gernsbacher, M. A. (2007). The Science of Sex Differences in Science and Mathematics. Psychological Science in the Public Interest, 8(1), 1-51. https://doi.org/10.1111/j.1529-1006.2007.00032.x
Hambleton, R. K. & Swaminathan, H. (1990). Item Response Theory: Principles and Applications. Boston, MA: Kluwer-Nijhoff Publishing.
Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods. Applied Measurement in Education, 2(4), 313-334. https://doi.org/10.1207/s15324818ame0204_4
Hambleton, R. K., Swaminathan, H., & Rogers H. J.  (1991). Fundamentals of item response theory.  Newbury Park, CA: Sage Publications, Inc.
 
Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. Test validity, 129-145.
León, I., Tascón, L., & Cimadevilla, J. M. (2016). Age and gender-related differences in a spatial memory task in humans. Behavioural Brain Research, 306, 8-12. https://doi.org/10.1016/j.bbr.2016.03.008Get rights and content
Mašić, A., Polz, E., & Bećirović, S. (2020). The Relationship Between Learning Styles, GPA, School Level and Gender. European Researcher, 11(1), 51-60. https://dx.doi.org/ 10.13187/er.2020.1.51
Michaelides, P. (2008). "An Illustration of a Mantel-Haenszel Procedure to Flag Misbehaving Common Items in Test Equating". Practical Assessment, Research, and Evaluation, 13(7), 1-13, https://scholarworks.umass.edu/pare/vol13/iss1/7
Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53(4), 495-502. https://doi.org/10.1007/BF02294403
Raju, N. S., Drasgow, F., & Slinde, J. A. (1993). An empirical comparison of the area methods, Lord's chi-square test, and the Mantel-Haenszel technique for assessing differential item functioning. Educational and Psychological Measurement, 53(2), 301-314. https://doi.org/10.1177/0013164493053002001
Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14(2), 197-207. https://doi.org/10.1177/014662169001400208
Rosati, P.A. (1997). Gender differences in the learning preferences of engineering students, In Proceedings of the 1997 American Society for Engineering Education annual conference & exposition.
Salubayba, T. (2013). Differential item functioning detection in reading comprehension test using Mantel Hanzel Item Response Theory and Logical Data Analysis. The International Journal of Social Sciences. 14(1). 76-82.
Sandstrom, N. J., Kaufman, J., & Huettel, S. A. (1998). Males and Females use different distal cues in a virtual environment navigation task. Cognitive Brain Research, 6(4), 351-360. https://doi.org/10.1016/S0926-6410 (98)00002-0
Skaggs, G., & Lissitz, R. W. (1992). The consistency of detecting item bias across different test administrations: Implications of another failure. Journal of Educational Measurement, 29(3), 227-242. https://doi.org/10.1111/j.1745-3984.1992.tb00375.x
Spss, I. (2017). IBM SPSS Statistics for Windows, version 25. Armonk, NY: IBM SPSS Corp.
Uiterwijk, H., & Vallen, T. (2005). Linguistic sources of item bias for second-generation immigrants in Dutch tests. Language Testing, 22(2), 211-234. https://doi.org/10.1191/0265532205lt301oa
Van Zwanenberg, N., Wilkinson, L. J., & Anderson, A. (2000). Felder and Silverman's Index of Learning Styles and Honey and Mumford's Learning Styles Questionnaire: How do they compare, and do they predict academic performance? Educational Psychology, 20(3), 365-380. https://doi.org/10.1080/713663743
Yilmaz, A. E., & Aktas, S. (2018). Ridit and exponential type scores for estimating the kappa statistic. Kuwait Journal of Science, 45(1), 89-99. https://doi.org/ 10.10800266 4760903012666/.
Zwick, R. (2012). A review of ETS differential item functioning assessment procedures: Flagging rules, minimum sample size requirements, and criterion refinement. ETS Research Report Series, (1), i-30. https://doi.org/10.1002/j.2333-8504.2012.tb02290.x
Zywno, M. S. (2003). A contribution to validation of score meaning for Felder-Soloman’s index of learning styles. In Proceedings of the 2003 American Society for Engineering Education annual conference & exposition, 119, 1-5. Washington, DC: American Society for Engineering Education

Files

Share

How to cite

Statistics