IDEAS home Printed from https://ideas.repec.org/a/eee/agisys/v127y2014icp81-89.html
   My bibliography  Save this article

An evaluation of the statistical methods for testing the performance of crop models with observed data

Author

Listed:
  • Yang, J.M.
  • Yang, J.Y.
  • Liu, S.
  • Hoogenboom, G.

Abstract

Calibration and evaluation are two important steps prior to the application of a crop simulation model. The objective of this paper was to review common statistical methods that are being used for crop model calibration and evaluation. A group of deviation statistics were reviewed, including root mean squired error (RMSE), normalize-RMSE (nRMSE), mean absolute error (MAE), mean error (E), paired-t, index of agreement (d), modified index of agreement (d1), revised index of agreement (d1′), modeling efficiency (EF) and revised modeling efficiency (EF1). A case study of the statistical evaluation was conducted for the DSSAT Cropping System Model (CSM) using 10 experimental datasets for maize, peanut, soybean, wheat and potato from Brazil, China, Ghana, and the USA. The results indicated that R2 was not a good statistic for model evaluation because it is insensitive to regression coefficients (α and β) of the linear model y=α+βx+ε. However, linear regression can be used for model evaluation (test H0: α=0, β=1) if auto-correlation, normality and heteroskedasticaity of the error term (ε) are tested or the proper data transfers are made. The results also illustrated that statistical evaluation of total dataset across treatments might be insufficient. Hence the evaluation of each treatment is necessary to make the right conclusion, especially when evaluating soil water content under different planting date treatments and soil mineral N under different N treatments. Co-variability analysis among dimensionless statistics (d, d1, d1′, EF and EF1) recommended that d and EF are inflated by the sum of squares-based deviations, i.e., the larger deviations contribute more weight on the statistic than the smaller deviation due to the squared term. However, EF had a larger range and a clear physical meaning at EF=0, making it superior to d. Values of d=0.75 were obtained from regression with all positive values of EF (EF⩾0), indicating that values of d⩾0.75 and EF⩾0 should be the minimum values for plant growth evaluation. Values of d⩾0.60 and EF⩾−1.0 should be the minimum values for soil outputs evaluation combined with t-test due to the fact that the soil parameters in the DSSAT SOIL module are difficult to calibrate compared with plant growth parameters because of no sufficient observed soil dataset. Due to the statistical nature, no single statistic is more robust over others but some statistics are highly correlated. Therefore, several statistics may be used from each of the following correlated groups (RMSE, MAE), (E, t-test), (d, d1, d1′) and (EF, EF1) in one assessment of model evaluation so that a representative statistical conclusion can be obtained with respect to model performance.

Suggested Citation

  • Yang, J.M. & Yang, J.Y. & Liu, S. & Hoogenboom, G., 2014. "An evaluation of the statistical methods for testing the performance of crop models with observed data," Agricultural Systems, Elsevier, vol. 127(C), pages 81-89.
    • Handle: RePEc:eee:agisys:v:127:y:2014:i:c:p:81-89
    DOI: 10.1016/j.agsy.2014.01.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0308521X14000171
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.agsy.2014.01.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Liu, S. & Yang, J.Y. & Zhang, X.Y. & Drury, C.F. & Reynolds, W.D. & Hoogenboom, G., 2013. "Modelling crop yield, soil water content and soil temperature for a soybean–maize rotation under conventional and conservation tillage systems in Northeast China," Agricultural Water Management, Elsevier, vol. 123(C), pages 32-44.
    2. Rinaldi, Michele & Ventrella, Domenico & Gagliano, Caterina, 2007. "Comparison of nitrogen and irrigation strategies in tomato using CROPGRO model. A case study from Southern Italy," Agricultural Water Management, Elsevier, vol. 87(1), pages 91-105, January.
    3. Yang, J. & Greenwood, D. J. & Rowell, D. L. & Wadsworth, G. A. & Burns, I. G., 2000. "Statistical methods for evaluating a crop nitrogen simulation model, N_ABLE," Agricultural Systems, Elsevier, vol. 64(1), pages 37-53, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Zhuo Ting & Yang, J.Y. & Drury, C.F. & Hoogenboom, G., 2015. "Evaluation of the DSSAT-CSM for simulating yield and soil organic C and N of a long-term maize and wheat rotation experiment in the Loess Plateau of Northwestern China," Agricultural Systems, Elsevier, vol. 135(C), pages 90-104.
    2. Jiang, Yiwen & Zhang, Lanhui & Zhang, Baoqing & He, Chansheng & Jin, Xin & Bai, Xiao, 2016. "Modeling irrigation management for water conservation by DSSAT-maize model in arid northwestern China," Agricultural Water Management, Elsevier, vol. 177(C), pages 37-45.
    3. Dutta, S. K & Laing, Alison M. & Kumar, S. & Gathala, Mahesh K. & Singh, Ajoy K. & Gaydon, D.S. & Poulton, P., 2020. "Improved water management practices improve cropping system profitability and smallholder farmers’ incomes," Agricultural Water Management, Elsevier, vol. 242(C).
    4. Wang, Haidong & Cheng, Minghui & Liao, Zhenqi & Guo, Jinjin & Zhang, Fucang & Fan, Junliang & Feng, Hao & Yang, Qiliang & Wu, Lifeng & Wang, Xiukang, 2023. "Performance evaluation of AquaCrop and DSSAT-SUBSTOR-Potato models in simulating potato growth, yield and water productivity under various drip fertigation regimes," Agricultural Water Management, Elsevier, vol. 276(C).
    5. OKORIE, Benedict Odinaka & NIRAJ, Yadav, 2022. "Effects Of Different Tillage Practices On Soil Fertility Properties: A Review," International Journal of Agriculture and Environmental Research, Malwa International Journals Publication, vol. 8(1), February.
    6. Cammarano, D. & Ronga, D. & Di Mola, I. & Mori, M. & Parisi, M., 2020. "Impact of climate change on water and nitrogen use efficiencies of processing tomato cultivated in Italy," Agricultural Water Management, Elsevier, vol. 241(C).
    7. Shuang Liu & Yuru Gao & Huilin Lang & Yong Liu & Hong Zhang, 2022. "Effects of Conventional Tillage and No-Tillage Systems on Maize ( Zea mays L.) Growth and Yield, Soil Structure, and Water in Loess Plateau of China: Field Experiment and Modeling Studies," Land, MDPI, vol. 11(11), pages 1-14, October.
    8. Anshuman Gunawat & Devesh Sharma & Aditya Sharma & Swatantra Kumar Dubey, 2022. "Assessment of climate change impact and potential adaptation measures on wheat yield using the DSSAT model in the semi-arid environment," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 111(2), pages 2077-2096, March.
    9. Kadiyala, M.D.M. & Jones, J.W. & Mylavarapu, R.S. & Li, Y.C. & Reddy, M.D., 2015. "Identifying irrigation and nitrogen best management practices for aerobic rice–maize cropping system for semi-arid tropics using CERES-rice and maize models," Agricultural Water Management, Elsevier, vol. 149(C), pages 23-32.
    10. Pringle, M.J. & Marchant, B.P. & Lark, R.M., 2008. "Analysis of two variants of a spatially distributed crop model, using wavelet transforms and geostatistics," Agricultural Systems, Elsevier, vol. 98(2), pages 135-146, September.
    11. Yang, Xuan & Zheng, Lina & Yang, Qian & Wang, Zikui & Cui, Song & Shen, Yuying, 2018. "Modelling the effects of conservation tillage on crop water productivity, soil water dynamics and evapotranspiration of a maize-winter wheat-soybean rotation system on the Loess Plateau of China using," Agricultural Systems, Elsevier, vol. 166(C), pages 111-123.
    12. Yang, J. & Rowell, D. L. & Burns, I. G. & Guttormsen, G. & Riley, H. & Wadsworth, G. A., 2002. "Modification and evaluation of the crop nitrogen model N_ABLE using Norwegian field data," Agricultural Systems, Elsevier, vol. 72(3), pages 241-261, June.
    13. Bai, Yu & Gao, Jinhua, 2021. "Optimization of the nitrogen fertilizer schedule of maize under drip irrigation in Jilin, China, based on DSSAT and GA," Agricultural Water Management, Elsevier, vol. 244(C).
    14. Zhou, Hong & Zhao, Wen zhi, 2019. "Modeling soil water balance and irrigation strategies in a flood-irrigated wheat-maize rotation system. A case in dry climate, China," Agricultural Water Management, Elsevier, vol. 221(C), pages 286-302.
    15. Zhang, Kerou & Peng, Changhui & Zhu, Qiuan & Li, Mingxu & Yan, Zhongqing & Li, Meng & Yan, Liang & Zhang, Xiaodong & Wang, Jinzhi & Li, Yong & Kang, Enze & Song, Hanxiong & Kang, Xiaoming, 2022. "Estimating natural nitrous oxide emissions from the Qinghai–Tibetan Plateau using a process-based model: Historical spatiotemporal patterns and future trends," Ecological Modelling, Elsevier, vol. 466(C).
    16. Laike Kebede & Melesse Temesgen & Abebe Fanta & Asfaw Kebede & Johan Rockström & Assefa M. Melesse, 2023. "Effect of Locally Adapted Conservation Tillage on Runoff, Soil Erosion, and Agronomic Performance in Semiarid Rain-Fed Farming in Ethiopia," Land, MDPI, vol. 12(3), pages 1-15, March.
    17. Rana, G. & Katerji, N. & Lazzara, P. & Ferrara, R.M., 2012. "Operational determination of daily actual evapotranspiration of irrigated tomato crops under Mediterranean conditions by one-step and two-step models: Multiannual and local evaluations," Agricultural Water Management, Elsevier, vol. 115(C), pages 285-296.
    18. Ruishi Si & Yumeng Yao & Xueqian Zhang & Qian Lu & Noshaba Aziz, 2022. "Exploring the Role of Contiguous Farmland Cultivation and Adoption of No-Tillage Technology in Improving Transferees’ Income Structure: Evidence from China," Land, MDPI, vol. 11(4), pages 1-25, April.
    19. Ai, Pengrui & Ma, Yingjie & Hai, Ying, 2021. "Influence of jujube/cotton intercropping on soil temperature and crop evapotranspiration in an arid area," Agricultural Water Management, Elsevier, vol. 256(C).
    20. Kiani, Mina & Gheysari, Mahdi & Mostafazadeh-Fard, Behrouz & Majidi, Mohammad Mahdi & Karchani, Kazem & Hoogenboom, Gerrit, 2016. "Effect of the interaction of water and nitrogen on sunflower under drip irrigation in an arid region," Agricultural Water Management, Elsevier, vol. 171(C), pages 162-172.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:agisys:v:127:y:2014:i:c:p:81-89. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/agsy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.