Keywords
cross-lingual assessment
dimensionality
invariance
test structure
validity
Estructura de la prueba
validez
evaluación en varios idiomas
examinados bilingues
dimensionalidad
invariancia
How to Cite
Abstract
Malay- and English-language versions of a mathematics exam were analyzed for structural equivalence byadministering both versions to a group of Malay-English bilingual students. The analysis and comparison of test structure was determined using both DIMTEST and weighted multidimensional scaling. The assessment was found to be unidimensional and to possess similar structure across the two language versions. Implications of this study suggest bilingual examinees can be used to evaluate the invariance of test structure across translated test forms. Future research should explore situations where bilingual examinees can be used to link different language versions of assessments for monolingual populations.References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association.
Baer, J., Baldi, S., Ayotte, K., Green, P. J., & McGrath, D. (2007). The reading literacy of U.S. fourth-grade students in an international context: Results from the 2001 and 2006 Progress in International Reading Literacy Study (PIRLS). (NCES-2008-017). Washington, DC: U.S. Department of Education. http://nces.ed.gov/pubs2008/2008017.pdf
Boldt, R. F. (1969). Concurrent validity of the PAA and SAT for bilingual Dade County high school volunteers. (College Entrance Examination Board Research and Development Report 68-69, No. 3). Princeton, NJ: Educational Testing Service.
Carroll, J. D., & Chang, J. J. (1970) Analysis of individual differences in multidimensional scaling via an N-way generalization of Eckart-Young decomposition. Psychometrika, 35, 283-319.CTB/McGrawBHill (1988). Spanish assessment of basic education: Technical report. Monterey, CA: McGraw Hill
Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting test into multiple languages and cultures. In R. K. Hambleton, R. Merenda, & C. Spielberger (Eds.) Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Hillsdale, NJ: Lawrence Erlbaum.
Hambleton, R. K., Merenda, P.F. & Spielberger, C.D. (2005). Adapting educational and psychological tests for cross-cultural assessment. Hillsdale, NJ: Lawrence Erlbaum.International Test Commission (2010). Guidelines for translating and adapting tests. Retrieved from http://www.intestcom.org
Millsap, R. E. (2007). Invariance in measurement and prediction revisited. Psychometrika, 72(4), 461-473.
Mullis, I.V.S., Martin, M.O., & Foy, P. (2008). TIMSS 2007 international mathematics report: Findings from IEA’s trends in international mathematics and science study at the fourth and eighth grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
Ong, S. L., & Sireci, S. G. (2008). Using bilingual students to link and evaluate different language versions of an exam. US-China Education Review, 5, 37-46.
Organisation for Economic Co-operation and Development. (2006). Literacy skills for the world of tomorrow—further results from PISA 2003. Paris: Author.
Prieto, A. J. (1992). A method for translation of instruments to other languages. Adult Education Quarterly, 43(1), 1-14.
Sireci, S. G. (1997). Problems and issues in linking assessments across languages. Educational Measurement: Issues and Practice, 16(1), 12-19.
Sireci, S. G. (2005). Using bilinguals to evaluate the comparability of different language versions of a test. In R.K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 117-138). Hillsdale, NJ: Lawrence Erlbaum.
Sireci, S. G. (2011). Evaluating test and survey items for bias across languages and cultures. In D. Matsumoto and F. van de Vijver (Eds.), Cross-cultural research methods in psychology (pp. 216-243). Oxford, UK: Oxford University Press
Sireci, S. G., Bastari, B., & Allalouf, A. (1998, August). Evaluating construct equivalence across adapted tests. Invited paper presented at the annual meeting of the American Psychological Association (Division 5), San Francisco, CA.
Sireci, S. G. & Berberoglu, G. (2000). Using bilingual respondents to evaluate translated-adapted items. Applied Measurement in Education, 35 (2), 229-259.
Sireci, S. G., & Khaliq, S. N. (2002, April). An analysis of the psychometric properties of dual language test forms. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
Sireci, S. G., Patsula, L., & Hambleton, R. K. (2005). Statistical methods for identifying fl awed items in the test adaptations process. In R.K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 93-115). Hillsdale, NJ: Lawrence Erlbaum.
Statistical Package for Social Sciences (SPSS). (2007). Multidimensional Scaling (PROXSCAL). SPSS CategoriesTM 16.0 (pp. 64-78). Chicago, IL: SPSS, Inc. Available online at: http://support.spss.com/ProductsExt/SPSS/Documentation/SPSSforWindows/index.html#16
Statistics Canada & Organization for Economic Co-operation and Development (OECD) (2005). Learning a living: First results of the adult literacy and life skills survey. Paris: Minister of Industry, Canada, and OECD. http://www.oecd.org/edu/highereducationandadultlearning/41529631.pdf.
Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589-617.
Stout, W., Douglas, J., Junker, B., & Roussos, L. (1993). DIMTEST Manual. University of Illinois at Urbana-Champaign: Department of Statistics.
van de Vijver, F. J. R., & Poortinga, Y. H. (2005). Conceptual and methodological issues in adapting tests. In R.K. Hambleton, P. Merenda, & C. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 39-63). Hillsdale, NJ: Lawrence Erlbaum.