Abstract
The detection of differential item functioning (DIF) is fundamental to ensure instruments’ invariance, and, therefore, a better estimate of the construct being measured across the different groups of examinees. The purpose of this research was to provide substantive hypothesizes related to possible sources of DIF, comparing students reported with accommodations for ADHD (focal group) and students with no accommodations (reference group), using the Standardized-P Difference and the Mantel Haenszel (MH) methods. Data from the Costa Rican national high school exit tests in Spanish and Math, from the year 2004, in public schools, were analyzed. First, these two methods were used to detect items with DIF, and then, using a more qualitative approach, drew hypotheses related to possible sources of DIF. Some degree of agreement was verified between the two different empirical methods, being Mantel-Haenszel more sensitive. In the Spanish test, DIF was hypothesized to be caused by the length and wording of the options, and the literary and non-literary texts in the stem. In Math, possible sources of DIF involved vocabulary, wording, the transition of verbal to mathematical language, the visuospatial item organization, and the drawing of graphs and geometrical figures. At the methodological level, complementing the statistical analyses with the judges’ criteria was helpful to identify possible sources of irrelevant variance in the construct measured by these tests. The hypotheses must be interpreted with caution, though, since the number of items detected as exhibiting DIF was relatively small.References
Abedi, Jamal, Hofstetter, Carolyn, Baker, Eva and Lord, Carol. (2001). NAEP Math performance and test accommodations: Interactions with student language background. Retrieved from http://www.cse.ucla.edu/products/Reports/TR536.pdf
Aguerri, María Ester, Blum, Diego, Picón, Jimena y Galibert, María Silvia. (2010). Reglas de detección del funcionamiento diferencial del ítem. Estudio del efecto del tamaño de muestra en presencia de DIF paralelo. II Congreso Internacional de Investigación y Práctica Profesional en Psicología XVII Jornadas de Investigación Sexto Encuentro de Investigadores en Psicología del MERCOSUR. Facultad de Psicología - Universidad de Buenos Aires, Buenos Aires. Recuperado de http://www.aacademica.org/000-031/920
American Educational Research Association (AERA), American Psychological Association (APA) and National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, Estados Unidos de América: Author.
Anastasi, Anne y Urbina, Susana. (1998). Tests psicológicos (7ª. ed.). Juárez, México: Prentice Hall.
Angoff, William. (1993). Perspectives on Differential Item Functioning Methodology. En Paul Holland and Howard Wainer (Eds.), Differential item functioning (pp. 3-23). New Jersey, United States of America: Lawrence Erlbaum Associates.
Attorresi, Horacio, Galibert, María Silvia, Zanelli, Marta, Lozzia, Gabriela y Aguerri, María Ester. (2003). Error tipo I en el análisis del funcionamiento diferencial del ítem basado en la diferencia de los parámetros de dificultad. Revista Psicológica, 24(2), 289-306. Retrieved from http://www.redalyc.org/articulo.oa?id=16924207
Bandeira, Wagner. (2002). Detección del funcionamiento diferencial del ítem (DIF) en test de rendimiento. Aportaciones teóricas y metodológicas (Tesis doctoral, Universidad Complutense de Madrid). Recuperado de http://biblioteca.ucm.es/tesis/edu/ucm-t26457.pdf
Bandeira, Wagner. (2003). Descripción de los principales métodos para detectar el funcionamiento diferencial del ítem (DIF) en el área de la evaluación educativa. Revista de Pedagogía Bordón, 55(2), 177-188.
Camilli, Gregory. (1993). The case against item bias detection techniques based on internal criteria: Do item bias procedure obscure test fairness issues? In Holland, Paul and Wainer, Howard (Eds.), Differential item functioning (pp. 321-335). New Jersey, United States of America: Lawrence Erlbaum Associates.
Cea, M° Ángeles. (1999). Metodología cuantitativa: Estrategias y técnicas de investigación social. Madrid, España: Editorial Síntesis.
Carvajal, Jorge y Poggio, Andrew. (2006, abril). Studying equivalence of Spanish language versions of a large scale assessment: Differential item functioning in the cognitive and affective domain. Work present in Annual Meeting of the National Council on Measurement in Education. San Francisco, United States of America.
Donoghue, John, Holland, Paul and Thayer, Dorothy. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and Standardization measures of differential item functioning. In Paul Holland and Howard Wainer (Eds.), Differential item functioning (pp. 137-166). New Jersey, United States of America: Lawrence Erlbaum Associates.
Dorans, Neil and Holland, Paul. (1993). DIF detection and description: Mantel-Haenszel and standardization. In Paul Holland and Howard Wainer (Eds.), Differential item functioning (pp. 35–66). New Jersey, United States of America: Lawrence Erlbaum Associates.
Elosúa, Paula y López, Alicia. (1999). Funcionamiento diferencial de los ítems y sesgo en la adaptación de dos pruebas verbales. Revista Psicológica, 20, 23-40. Retrieved from http://www.uv.es/revispsi/articulos1.99/elosua.pdf
Gómez-Benito, Juana, Hidalgo, María Dolores y Guilera, Georgina. (2010). El sesgo de los instrumentos de medición. Tests justos. Papeles del Psicólogo, 31(1) 75-84. Recuperado de http://www.cop.es/papeles
Gómez-Benito, Juana, Balluerka, Nekane, González, Andrés, Widaman, Keith F. and Padilla, José Luis. (2017). Detecting differential item functioning in behavioral indicators across parallel forms. Psicothema, 29(1), 91-95. doi: 10.7334/psicothema2015.112
Hidalgo Montesinos, María Dolores, López Pina, José Antonio y Sánchez Meca, Julio. (1997). Error tipo I y potencia de las pruebas chi-cuadrado en el estudio del funcionamiento diferencial de los ítems. Revista de Investigación educativa, 15(1), 149-168.
Hidalgo Montesinos, María Dolores, Galindo Garre, Francisca, Inglés Saura, Cándido José, Campoy Menéndez, Guillermo y Ortiz Soria, Beatriz. (1999). Estudio del funcionamiento diferencial de los ítems en una escala de habilidades sociales para adolescentes. Revista Anales de psicología, 15(2), 331-342. Retrieved from http://www.um.es/analesps/v15/v15_2pdf/17v98_14mdhidalg.PDF
Koretz, Daniel. (1997). The assessment of students with disabilities in Kentucky. Retrieved from http://cresst.org/wp-content/uploads/TECH431.pdf
Koretz, Daniel y Barton, Karen. (2003). Assessing students with disabilities: Issues and evidence. Retrieved from http://research.cse.ucla.edu/reports/TR587.pdf
Longford, Nicholas, Holland, Paul y Thayer, Dorothy. (1993). Stability of the MH D-DIF Statistics Across Populations. In Paul Holland and Howard Wainer (Eds.), Differential item functioning (pp. 255–276). New Jersey, United States of America: Lawrence Erlbaum Associates.
Messick, Samuel. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Research, 18(2), 5-11.
Messick, Samuel. (1995). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and practice, 14(4), 5-8.
Ministerio de Educación Pública (MEP). (1998). Políticas, normativa y procedimientos para el acceso a la educación de los estudiantes con necesidades educativas especiales (Reimpresión de la 1ª. ed.). San José, Costa Rica: Author.
Montero, Eiliana. (1993). Linguistic and cultural influences on differential item functioning for Hispanic examinees in a standardized secondary level achievement test (Unpublished doctoral thesis in Educational Research). The Florida State University, Tallahasse, Florida, USA.
Moreno Oliver, Francesc. (2001). Análisis psicopedagógico de los alumnos de educación secundaria obligatoria con problemas de comportamiento en el contexto escolar (Tesis Doctoral, Universitat Autònoma de Barcelona). Retrieved from http://www.tdx.cat/handle/10803/5411
Muñiz, José. (1990). Teoría de Respuesta a los Ítemes. Madrid, España: Ediciones Pirámide S.A.
Padilla, José Luis, González, Andrés y Pérez, Cristino. (1998). Diferencias instruccionales y funcionamiento diferencial de los ítems: Acuerdo entre el método Mantel – Haenszel y la regresión logística. Revista Psicológica, 19, 201-215. Retrieved from http://www.uv.es/revispsi/articulos3.98/padilla.pdf
Penfield, Randall. (2013). DIFAS 5.0. Differential item functioning analysis system. User’s Manual. Retrieved from http://soe.uncg.edu/wp-content/uploads/2015/12/DIFASManual_V5.pdf
Penfield, Randall and Camilli, Gregory. (2006). Differential Item Functioning and Item Bias. In S. Sinharay and C.R. Rao (Eds.). Handbook of Statistics. Psychometrics (Vol. 26; pp. 125-167). Amsterdam, Holanda: Elsevier.
Prieto, Gerardo. (2013). Análisis del Funcionamiento Diferencial de los Ítems de una prueba de Comprensión Lectora del Español como segunda lengua. Recuperado de http://www.alte.org/attachments/pdfs/files/conferencia_gpa_qwroz.pdf
Rubiales, Josefina, Bakker, Liliana, Russo, Daiana and González, Rocío (2016). Desempeño en funciones ejecutivas y síntomas comórbidos asociados en niños con Trastorno por déficit de atención con hiperactividad (TDAH). Revista CES Psicología, 9(2), 99-113, doi: http://dx.doi.org/10.21615/cesp.9.2.7
Sireci, Stephen G., Scarpati, Stanley E. y Li, Shuhong. (2005). Test accommodations for students with disabilities: An analysis of the interaction hypothesis. Review of Educational Research, 75(4), 457-490.
Thompson, Sandra, Blount, Amanda y Thurlow, Martha. (2002). A summary of research on the effects of test accommodations: 1999 through 2001 (Technical Report 34). Recuperado de https://nceo.umn.edu/docs/OnlinePubs/TechReport34.pdf
Villalobos, Ericka y Morales, Krissia. (2002). Niños con déficit de atención: Orientación a padres y docentes. San José, Costa Rica: Editorial Universidad Estatal a Distancia.
Wainer, Howard. (1993). Model-Based Standardized Measurement of an Item's Differential Impact. In Paul Holland and Howard Wainer (Eds.), Differential item functioning (pp. 123–135). New Jersey, United States of America: Lawrence Erlbaum Associates.
Zieky, Michael. (1993). Practical questions in the use of DIF statistics in test development. In Paul Holland and Howard Wainer (Eds.), Differential item functioning (pp. 337-347). New Jersey, United States of America: Lawrence Erlbaum Associates.