# Analysis of statistical methods to evaluate the performance of simulation models in horticultural crops

## Authors

• Freddy Soto-Bravo Universidad de Costa Rica
• María Isabel González-Lutz Universidad de Costa Rica.

## Keywords:

statistical evaluation, crop modelling, linear regression, analysis of differences

## Abstract

Introduction. Every simulation model must be calibrated and validated, in order to avoid speculative and inaccurate conclusions. The methods to evaluate simulation models are usually applied “by habit”, without specifying  basic methodological details which leads to the use of terminology and symbology that could cause confusion Objective. The objective in the present study was to analyze the different statistical methods employed to evaluate the performance of simulation models in agriculture, and thus propose which is the most suitable from the practical point of view. Materials and Methods. Statistical methods based on difference and regression analysis, between measured and simulated values were analyzed. Regarding the difference analysis group, the used methods were root mean square error (RMSE), mean absolute error (MAE), relative error (RE), adjustment index (d), me bianas error (MBE) and the model efficiency (E). In the case of the regression analysis the intercept, linear regression (b) and determination (R2) coefficients, and the estimation confidence limits were scrutinize. Results. The ER, d and E, are measures which objective is the comparison between different models to simulate a given variable, instead of evaluating the performance of the model as such. The root square mean error usually used to evaluate differences between observed and simulated values is different from the RMSE regression. The different cases illustrated with the “Eurotate_N” model demonstrated the apropriate practical application of the regression analysis as statistical tool to evaluate its capacity to simulate fruit yield, volumetric soil moisture, evapotranspiration and dry matter in tomato crop under greenhouse. Conclusion. The most appropriate statistical method proposed to evaluate a simulation model in tomato was the regression analysis.

## References

Cannavo, P.R., S. Parnaudeau, and V. Reau. 2008. Modeling N dynamics to assess enviromental impacts of cropped soils. Adv. Agron. 97:131-174. doi:10.1016/S0065-2113(07)00004-1

Doltra, J., and P. Muñoz. 2010. Simulation of nitrogen leaching from a fertigated crop rotation in a Mediterranean climate using the EU-Rotate_N and Hydrus-2D models. Agric. Water Manag. 97:277-285. doi:10.1016/j.agwat.2009.09.019

Draper, N., and H. Smith. 1998. Applied regression analysis. 3rd ed. Wiley Interscience, NY, USA.

Fernández, M., F. Orgaz, E. Fereres, J.C. López, A. Céspedes, J. Pérez, S. Bonachela, y M. Gallardo. 2001. Programación del riego de cultivos hortícolas bajo invernadero en el sudeste español. Caja Rural, Almería, ESP.

Flavelle, P. 1992. A quantitative measure of model validation and its potential use for regulatory purpose. Adv. Water Resour. 15:5-13. doi:10.1016/0309-1708(92)90028-Z

Fox, D.G. 1981. Judging air quality model performance: A summary of the AMS workshop on dispersion model performance. Bull. Am. Meteorol. Soc. 62:599-609.

Galloway, J., F. Dentener, D. Capone, E. Boyer, R. Howarth, S. Seitzinger, G. Asner, C. Cleveland, P. Green, E. Holland, D. Karl, F. Michaels, J. Porter, A. Townsend, and C. Voromarty. 2004. The global nitrogen cycle: Past, present and future. Biogeochemistry 70:153-226. doi:10.1007/s10533-004-0370-0

Guo, R., C. Nendel, C. Rahn, C. Jiang, and Q. Chen. 2010. Tracking nitrogen losses in a greenhouse crop rotation experiment in North China using the EU-Rotate_N simulation model. Environ. Pollut. 158:2218-2229. doi:10.1016/j.envpol.2010.02.014

Mayer, D.G., M.A. Stuart, and A.J. Swain. 1994. Regression of real word data on model output: an appropiate overall test of validity. Agric. Syst. 45:93-104. doi:10.1016/S0308-521X(94)90282-8

Meinke, H., P. Struik, J. Vos, and W. Van-der-Werf. 2008. Modelling that bridges scales and connects disciplines. In: H. Van-Keulen et al., editors, 40 years theory and model at Wageningen UR. Wageningen University, and Research Centre, Wageningen, NED. p. 37.

Mendenhall, W., R. Scheaffer, y D. Wackerly. 1986. Estadística matemática con aplicaciones. 2a ed. Grupo Editorial

Iberoamérica, Andalucía, ESP.

Nash, J.E., and J.V Sutcliffe. 1970. River Flow forecasting through conceptual models. Part I-A discussion of principles. J. Hydrol. 10:282-290. doi:10.1016/0022-1694(70)90255-6

Rahn, C.R., K. Zhang, R. Lillywhite, C. Ramos, J. Doltra, J.M. de-Paz, H. Riley, M. Fink, C. Nendel, K. Thorup-Kristensen, A. Pedersen, F. Piro, A. Venezia, C. Firth, U. Schmutz, F. Rayns, and K. Strohmeyer. 2010. Eu-Rotate_N - a decision support system - to predict environmental and economic consequences of the management of nitrogen fertiliser in crop rotations. Europ. J. Hort. Sci. 75(1):20-32.

Reckhow, K.H., J.T. Clements, and R.C. Dodds. 1990. Statistical evaluation of mechanistic water-quality models. J. Environ. Eng. 116:250-268. doi:10.1061/(ASCE)0733-9372(1990)116:2(250)

Soto, F., M. Gallardo, C. Giménez, T. Peña-Fleitas, and R.B. Thompson. 2014. Simulation of tomato growth, water and N dynamics using the EU-Rotate_N model in Mediterranean greenhouses with drip irrigation and fertigation. Agric. Water Manag. 132:46-59. doi:10.1016/j.agwat.2013.10.002

Stockle, C.O., J. Kjelgaard, and G. Bellocchi. 2004. Evaluation of estimated weather data for calculating Penman-Monteith reference crop evapotranspiration. Irrig. Sci. 23:39-46. doi:10.1007/s00271-004-0091-0

Sun, Y., K. Hu, K. Zhang, L. Jiang, and Y. Xu. 2012. Simulation of nitrogen fate for greenhouse cucumber grown under different water and fertilizer management usingthe EU-Rotate N model. Agric. Water Manag. 112:21-32. doi:10.1016/j.agwat.2012.06.001

Thornley, J.H., and I.M. Johnson. 2000. Plant and crop modelling: A mathematical approach to plant and crop physiology. The Blackburn Press, NJ, USA.

Tilman, D., K.G. Cassman, P.A. Matson, R. Naylor, and S. Polasky. 2002. Agricultural sustainability and intensive production practices. Nature 418:671-677. doi:10.1038/nature01014

Willmott, C.J. 1982. Some comments on the evaluation of model performance. Bull. Am. Meteorol. Soc. 63:1309-1313. doi:10.1175/1520-0477(1982)063<1309:SCOTEO>2.0.CO;2

Yang, J., D.J. Greenwood, D.L. Rowell, G.A. Wadsworth, and I.G. Burns. 2000. Statistical methods for evaluating a crop nitrogen simulation model, N_ABLE. Agric. Syst. 64:37-53. doi:10.1016/S0308-521X(00)00010-X

Yang, J.M., J.Y. Yang, S. Liu, and G. Hoogenboom. 2014. An evaluation of the statistical methods for testing the performance of crop models with observed data. Agric. Syst. 127:81-89. doi:10.1016/j.agsy.2014.01.008

2019-05-01

## How to Cite

Soto-Bravo, F., & González-Lutz, M. I. (2019). Analysis of statistical methods to evaluate the performance of simulation models in horticultural crops. Agronomía Mesoamericana, 30(2), 517–534. https://doi.org/10.15517/am.v30i2.33839

Technical Notes