Keywords
modelos jerárquicos bayesianos
efectos de capacitación
prueba de admisión a la Universidad
modelos multinivel
Impact evaluation
Bayesian hierarchical models
effects of training
university admission test
multilevel models
How to Cite
Abstract
A quasi-experimental design with pre and post- test was used to estimate training effects for the University of Costa Rica’s admission test, a standardized exam that measures reasoning abilities in mathematical and verbal contexts. Four secondary public schools from the metropolitan central area of the country participated in the study; two of them were randomly assigned to the intervention group and the other two to the control group, with 61 students in the first group and 80 in the second. The intervention consisted of five three-hour training sessions, using a written guide developed by a pedagogy expert with a constructivist approach. Before and after measures were reduced test forms of the real admission test from the year 2014. The dependent variable was the difference between the two measures. The effect of the training was estimated using a multilevel Bayesian regression model with a significant magnitude of 3.5 percentage points.References
Baydar, N. (1990). Effects of coaching on the validity of the SAT: Results of a simulation study. In W. W. Wilingham, C. Lewis, R. Morgan, and L. Ramist (Eds.), Predicting college grades: An analysis of institutional trends over two decades. Princeton, New Jersey: ETS.
Bock, R.D. (Ed.) (1989). Multilevel Analysis of Educational Data. San Diego: Academic Press.
Browne, W.J., & Draper D. (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1(3), 473-514.
Burns, G. N., Siers, B. P., & Christiansen, N. D. (2008). Effects of providing pre-test information and preparation materials on applicant reactions to selection procedures. International Journal of Selection and Assessment, 16(1), 73-77.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research on teaching. In N. L. Gage (Ed.), Handbook of research on teaching (pp. 171–246). Chicago, IL: Rand McNally.
Carey, K. (2000). A multilevel modeling approach to analysis of patient costs under managed care. Health Economics, 9, 435-446.
Cohen, A. D. (2006). The coming of age of research on test taking strategies. Language Assessment Quarterly, 3(4) 307-331.
Crocker, L. (2005). Teaching for the test: How and why test preparation is appropriate. In R. P. Phelps (Ed.), Defending standardized testing (pp. 159–174). Mahwah, NJ: Lawrence Erlbaum Associates.
Fernández, A. & Del Valle, R. (2013). Desigualdad educativa en Costa Rica: La brecha entre estudiantes de colegios públicos y privados. Análisis con los resultados de la evaluación internacional PISA. Revista CEPAL, 111.
Flippo, R. F., Becker, M. J., & Wark, D. M. (2000). Preparing for and taking tests. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1, 515-533.
Gelman, A. & Hill, J. (2007). Data Analysis Using Regression and Multilevel/Hierarchical Models.Cambridge: Cambridge University Press.
Gilks, W., Richardson, S., & Spiegelhalter, D. (1996). Markov Chain Monte Carlo in Practice. Londres: Chapman and Hall.
Goldstein, H. (1987). Multilevel models in education and social research. New York:Oxford University Press.
Goldstein, H. (1995). Multilevel Statistical Models.Londres: Edward Arnold.
Goldstein, H., Browne, W., & Rasbash, J. (2002). Multilevel modelling of medical data. Statistics in Medicine, 15, 3291-3315.
Harmon, M. G., Morse, D. T., & Morse, L. W. (1996). Confirmatory factor analysis of the Gibb Experimental Test of Testwiseness. Educational and Psychological Measurement, 56(2), 276-286.
Koriat, A., & Bjork, R. A. (2006). Illusions of competence during study can be remedied by manipulations that enhance learners’ sensitivity to retrieval conditions at test. Memory & Cognition,34(5),959-972.
Kulik, J. A., Bangert-Drowns, R. L., & Kulik, C. (1984). Effectiveness of coaching for aptitude tests. Psychological Bulletin, 95(2), 179-188.
Lambert, P., Sutton, A., Burton, P., Abrams, R., Jones, D. (2005). How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Statistics in Medicine, 24, 2401-2428.
Leyland, A., & Goldstein, H. (2001). Multilevel Modeling of Health Statistics. Willey: Chichester.
Martínez-Cardenoso, J., Muniz, J., & García Cueto, E. (2000). Efecto del entrenamiento sobre las propiedades psicométricas de los tests. Psicothema, 12(Suppl2) 2000, 363-367.
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18 (2), 5-11.
Millman, J., Bishop, C. H., & Ebel, R. (1965). Analysis of test wiseness in the cognitive domain. Educational and Psychological Measurement, 18, 787–790.
Ministerio de Educación Pública de Costa Rica (MEP) (2012). Informe Nacional de Factores Asociados al Rendimiento Académico en las Pruebas Nacionales Diagnósticas, III Ciclo de la Educación General Básica, 2010. San José, Costa Rica: Dirección de Gestión y Evaluación de la Calidad, Departamento de Evaluación Académica y Certificación, Ministerio de Educación Pública.
Montero, E., Rojas, S., Rodino, A. & Zamora, E. (2012). Costa Rica en las pruebas PISA 2009 de competencia lectora y alfabetización matemática. Cuarto Informe Estado de la Educación. San José: Programa Estado de la Nación.
Montero, E. (2014). Dimensiones de lectura como predictoras de los puntajes en las pruebas PISA-Costa Rica-2009 y PAA de la UCR: Evidencias de regresiones corregidas por estructura multinivel. Ponencia presenta en las Jornadas de Investigación 2014 del Instituto de Investigaciones Psicológicas de la Universidad de Costa Rica.
Moreira, T. E. (2009). Relación entre factores individuales e institucionales con el rendimiento en matemática: Un análisis multivariado. Avances en Medición, 7, 115-128
Morse, D.T. (1998). The Relative Difficulty of Selected Test-Wiseness Skills among College Students. Educational and Psychological Measurement, 58(3), 399-408.
Powers, D. E. (1993). Coaching for the SAT: A summary of the summaries and an update. Educational Measurement: Issues and Practice, 12, 24-39.
Powers, D. E., & Rock, D. A. (1999). Effects of coaching on SAT I: Reasoning Test scores. Journal of Educational Measurement, 36(2), 93-118.
Raudenbush, S. W., & Bryk, A. S. (2002) Hierarchical Linear Models. Applications and Data Analysis Methods.Second Edition. Londres: Sage Publications.
Raudenbush, S.W., & Willms, J.D. (Eds.). (1991). Schools, Classrooms, and Pupils: International Studies of Schooling from a Multilevel Perspective. San Diego: Academic Press.
Rice, N., & Jones, A. (1997). Multilevel models and health economics. Health Economics, 6, 561-575.
Rice, N., & Leyland, A. (1996). Multilevel models: Applications to health data. Journal of Health Services Research and Policy, 1, 154-164.
Rojas, L. (2004). Factores Asociados a la Repitencia de los y las Estudiantes que Cursan Sétimo Año en Colegios Académicos, Diurnos y Públicos: Un Modelo de Análisis de Niveles Múltiples. Tesis doctoral. San José, Costa Rica: Universidad Estatal a Distancia.
Samson, G.E. (1985). Effects of Training in Test-Taking Skills on Achievement Test Performance: A Quantitative Synthesis. The Journal of Educational Research, 78(5), 261-266.
Sarnacki, R.E. (1979). An Examination of Test-Wiseness in the Cognitive Test Domain. Review of Educational Research, 49(2), 252-279.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.
Shulman, L. S. (2013). When Coaching and Testing Collide. Carnegie Perspectives: Current news views and links form the Carnegie Foundation http://www.carnegiefoundation.org/perspectives/when-coaching-and-testing-collide
Snijders, T., & Bosker, R. (1999). Multilevel Analysis. Londres: Sage Publications.
Trejos, J.D. (2010). Indicadores sobre equidad en la educación para Costa Rica. Tercer Informe Estado de la Educación.San José: Programa Estado de la Nación.