Analysis of statistical methods to evaluate the performance of simulation models in horticultural crops
DOI:
https://doi.org/10.15517/am.v30i2.33839Keywords:
statistical evaluation, crop modelling, linear regression, analysis of differencesAbstract
Introduction. Every simulation model must be calibrated and validated, in order to avoid speculative and inaccurate conclusions. The methods to evaluate simulation models are usually applied “by habit”, without specifying basic methodological details which leads to the use of terminology and symbology that could cause confusion Objective. The objective in the present study was to analyze the different statistical methods employed to evaluate the performance of simulation models in agriculture, and thus propose which is the most suitable from the practical point of view. Materials and Methods. Statistical methods based on difference and regression analysis, between measured and simulated values were analyzed. Regarding the difference analysis group, the used methods were root mean square error (RMSE), mean absolute error (MAE), relative error (RE), adjustment index (d), me bianas error (MBE) and the model efficiency (E). In the case of the regression analysis the intercept, linear regression (b) and determination (R2) coefficients, and the estimation confidence limits were scrutinize. Results. The ER, d and E, are measures which objective is the comparison between different models to simulate a given variable, instead of evaluating the performance of the model as such. The root square mean error usually used to evaluate differences between observed and simulated values is different from the RMSE regression. The different cases illustrated with the “Eurotate_N” model demonstrated the apropriate practical application of the regression analysis as statistical tool to evaluate its capacity to simulate fruit yield, volumetric soil moisture, evapotranspiration and dry matter in tomato crop under greenhouse. Conclusion. The most appropriate statistical method proposed to evaluate a simulation model in tomato was the regression analysis.
Downloads
References
Cannavo, P.R., S. Parnaudeau, and V. Reau. 2008. Modeling N dynamics to assess enviromental impacts of cropped soils. Adv. Agron. 97:131-174. doi:10.1016/S0065-2113(07)00004-1
Doltra, J., and P. Muñoz. 2010. Simulation of nitrogen leaching from a fertigated crop rotation in a Mediterranean climate using the EU-Rotate_N and Hydrus-2D models. Agric. Water Manag. 97:277-285. doi:10.1016/j.agwat.2009.09.019
Draper, N., and H. Smith. 1998. Applied regression analysis. 3rd ed. Wiley Interscience, NY, USA.
Fernández, M., F. Orgaz, E. Fereres, J.C. López, A. Céspedes, J. Pérez, S. Bonachela, y M. Gallardo. 2001. Programación del riego de cultivos hortícolas bajo invernadero en el sudeste español. Caja Rural, Almería, ESP.
Flavelle, P. 1992. A quantitative measure of model validation and its potential use for regulatory purpose. Adv. Water Resour. 15:5-13. doi:10.1016/0309-1708(92)90028-Z
Fox, D.G. 1981. Judging air quality model performance: A summary of the AMS workshop on dispersion model performance. Bull. Am. Meteorol. Soc. 62:599-609.
Galloway, J., F. Dentener, D. Capone, E. Boyer, R. Howarth, S. Seitzinger, G. Asner, C. Cleveland, P. Green, E. Holland, D. Karl, F. Michaels, J. Porter, A. Townsend, and C. Voromarty. 2004. The global nitrogen cycle: Past, present and future. Biogeochemistry 70:153-226. doi:10.1007/s10533-004-0370-0
Guo, R., C. Nendel, C. Rahn, C. Jiang, and Q. Chen. 2010. Tracking nitrogen losses in a greenhouse crop rotation experiment in North China using the EU-Rotate_N simulation model. Environ. Pollut. 158:2218-2229. doi:10.1016/j.envpol.2010.02.014
Mayer, D.G., M.A. Stuart, and A.J. Swain. 1994. Regression of real word data on model output: an appropiate overall test of validity. Agric. Syst. 45:93-104. doi:10.1016/S0308-521X(94)90282-8
Meinke, H., P. Struik, J. Vos, and W. Van-der-Werf. 2008. Modelling that bridges scales and connects disciplines. In: H. Van-Keulen et al., editors, 40 years theory and model at Wageningen UR. Wageningen University, and Research Centre, Wageningen, NED. p. 37.
Mendenhall, W., R. Scheaffer, y D. Wackerly. 1986. Estadística matemática con aplicaciones. 2a ed. Grupo Editorial
Iberoamérica, Andalucía, ESP.
Nash, J.E., and J.V Sutcliffe. 1970. River Flow forecasting through conceptual models. Part I-A discussion of principles. J. Hydrol. 10:282-290. doi:10.1016/0022-1694(70)90255-6
Rahn, C.R., K. Zhang, R. Lillywhite, C. Ramos, J. Doltra, J.M. de-Paz, H. Riley, M. Fink, C. Nendel, K. Thorup-Kristensen, A. Pedersen, F. Piro, A. Venezia, C. Firth, U. Schmutz, F. Rayns, and K. Strohmeyer. 2010. Eu-Rotate_N - a decision support system - to predict environmental and economic consequences of the management of nitrogen fertiliser in crop rotations. Europ. J. Hort. Sci. 75(1):20-32.
Reckhow, K.H., J.T. Clements, and R.C. Dodds. 1990. Statistical evaluation of mechanistic water-quality models. J. Environ. Eng. 116:250-268. doi:10.1061/(ASCE)0733-9372(1990)116:2(250)
Soto, F., M. Gallardo, C. Giménez, T. Peña-Fleitas, and R.B. Thompson. 2014. Simulation of tomato growth, water and N dynamics using the EU-Rotate_N model in Mediterranean greenhouses with drip irrigation and fertigation. Agric. Water Manag. 132:46-59. doi:10.1016/j.agwat.2013.10.002
Stockle, C.O., J. Kjelgaard, and G. Bellocchi. 2004. Evaluation of estimated weather data for calculating Penman-Monteith reference crop evapotranspiration. Irrig. Sci. 23:39-46. doi:10.1007/s00271-004-0091-0
Sun, Y., K. Hu, K. Zhang, L. Jiang, and Y. Xu. 2012. Simulation of nitrogen fate for greenhouse cucumber grown under different water and fertilizer management usingthe EU-Rotate N model. Agric. Water Manag. 112:21-32. doi:10.1016/j.agwat.2012.06.001
Thornley, J.H., and I.M. Johnson. 2000. Plant and crop modelling: A mathematical approach to plant and crop physiology. The Blackburn Press, NJ, USA.
Tilman, D., K.G. Cassman, P.A. Matson, R. Naylor, and S. Polasky. 2002. Agricultural sustainability and intensive production practices. Nature 418:671-677. doi:10.1038/nature01014
Willmott, C.J. 1982. Some comments on the evaluation of model performance. Bull. Am. Meteorol. Soc. 63:1309-1313. doi:10.1175/1520-0477(1982)063<1309:SCOTEO>2.0.CO;2
Yang, J., D.J. Greenwood, D.L. Rowell, G.A. Wadsworth, and I.G. Burns. 2000. Statistical methods for evaluating a crop nitrogen simulation model, N_ABLE. Agric. Syst. 64:37-53. doi:10.1016/S0308-521X(00)00010-X
Yang, J.M., J.Y. Yang, S. Liu, and G. Hoogenboom. 2014. An evaluation of the statistical methods for testing the performance of crop models with observed data. Agric. Syst. 127:81-89. doi:10.1016/j.agsy.2014.01.008
Downloads
Additional Files
Published
How to Cite
Issue
Section
License
1. Proposed policy for open access journals
Authors who publish in this journal accept the following conditions:
a. Authors retain the copyright and assign to the journal the right to the first publication, with the work registered under the attribution, non-commercial and no-derivative license from Creative Commons, which allows third parties to use what has been published as long as they mention the authorship of the work and upon first publication in this journal, the work may not be used for commercial purposes and the publications may not be used to remix, transform or create another work.
b. Authors may enter into additional independent contractual arrangements for the non-exclusive distribution of the version of the article published in this journal (e.g., including it in an institutional repository or publishing it in a book) provided that they clearly indicate that the work was first published in this journal.
c. Authors are permitted and encouraged to publish their work on the Internet (e.g. on institutional or personal pages) before and during the review and publication process, as it may lead to productive exchanges and faster and wider dissemination of published work (see The Effect of Open Access).
