IDEAS home Printed from
   My bibliography  Save this article

Least Squares and Entropy: A Penalty Function Perspective


  • Paul V. Preckel


Mathematical measures of entropy as defined by Shannon and cross entropy as defined by Kullback and Leibler are currently in vogue in the field of econometrics, primarily due to the comprehensive work of Golan, Judge, and Miller. An alternative interpretation of the entropy measure as a penalty function over deviations is presented, and a number of parallels are drawn with least squares estimators. It is demonstrated that both approaches may be applied to the general linear model. The causes of differences in estimated parameter values are described, and some suggestions for the formulation of entropy-based econometric problems are presented. Copyright 2001, Oxford University Press.

Suggested Citation

  • Paul V. Preckel, 2001. "Least Squares and Entropy: A Penalty Function Perspective," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 83(2), pages 366-377.
  • Handle: RePEc:oup:ajagec:v:83:y:2001:i:2:p:366-377

    Download full text from publisher

    File URL:
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Bennett, Richard & Blaney, Ralph, 2002. "Social consensus, moral intensity and willingness to pay to address a farm animal welfare issue," Journal of Economic Psychology, Elsevier, vol. 23(4), pages 501-520, August.
    2. Robert Innes, 1999. "Self-Policing and Optimal Law Enforcement When Violator Remediation is Valuable," Journal of Political Economy, University of Chicago Press, vol. 107(6), pages 1305-1325, December.
    3. Boland, Philip J. & Proschan, Frank, 1988. "Multivariate arrangement increasing functions with applications in probability and statistics," Journal of Multivariate Analysis, Elsevier, vol. 25(2), pages 286-298, May.
    4. Milgrom, Paul & Shannon, Chris, 1994. "Monotone Comparative Statics," Econometrica, Econometric Society, vol. 62(1), pages 157-180, January.
    5. Boehlje, Michael, 1996. "Industrialization of Agriculture: What are the Implications?," Choices, Agricultural and Applied Economics Association, vol. 11(1).
    6. John A. Miranowski & Bruce A. Babcock, 2004. "Genetic Information in Agricultural Productivity and Product Development," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 86(1), pages 73-87.
    7. Jayson L. Lusk & John A. Fox & Ted C. Schroeder & James Mintert & Mohammad Koohmaraie, 2001. "In-Store Valuation of Steak Tenderness," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 83(3), pages 539-550.
    8. Milgrom, Paul & Roberts, John, 1996. "The LeChatelier Principle," American Economic Review, American Economic Association, vol. 86(1), pages 173-179, March.
    9. Garen, John, 1988. "Compensating Wage Differentials and the Endogeneity of Job Riskiness," The Review of Economics and Statistics, MIT Press, vol. 70(1), pages 9-16, February.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Heckelei, Thomas & Britz, Wolfgang, 2000. "Positive Mathematical Programming with Multiple Data Points: A Cross-Sectional Estimation Procedure," Cahiers d'Economie et de Sociologie Rurales (CESR), INRA (French National Institute for Agricultural Research), vol. 57.
    2. Kaplan, Jonathan D. & Howitt, Richard E. & Farzin, Y. Hossein, 2003. "An information-theoretical analysis of budget-constrained nonpoint source pollution control," Journal of Environmental Economics and Management, Elsevier, vol. 46(1), pages 106-130, July.
    3. Buschena, David E. & Atwood, Joseph A., 2011. "Evaluation of similarity models for expected utility violations," Journal of Econometrics, Elsevier, vol. 162(1), pages 105-113, May.
    4. R. Carter Hill & Randall C. Campbell, 2001. "Maximum Entropy Estimation in Economic Models with Linear Inequality Restrictions," Departmental Working Papers 2001-11, Department of Economics, Louisiana State University.
    5. Arndt, Channing & Robinson, Sherman & Tarp, Finn, 2002. "Parameter estimation for a computable general equilibrium model: a maximum entropy approach," Economic Modelling, Elsevier, vol. 19(3), pages 375-398, May.
    6. Balcombe, Kelvin, 2006. "Cross-Entropy Estimation of Linear Cointegrated Equations," MPRA Paper 15100, University Library of Munich, Germany.
    7. You, Liangzhi & Wood, Stanley & Wood-Sichra, Ulrike, 2009. "Generating plausible crop distribution maps for Sub-Saharan Africa using a spatially disaggregated data fusion and optimization approach," Agricultural Systems, Elsevier, vol. 99(2-3), pages 126-140, February.
    8. Rui Fragoso & Maria Leonor da Silva Carvalho, 2013. "Estimation of cost allocation coefficients at the farm level using an entropy approach," Journal of Applied Statistics, Taylor & Francis Journals, vol. 40(9), pages 1893-1906, September.
    9. Heckelei, Thomas & Mittelhammer, Ronald C. & Jansson, Torbjorn, 2008. "A Bayesian Alternative To Generalized Cross Entropy Solutions For Underdetermined Econometric Models," Discussion Papers 56973, University of Bonn, Institute for Food and Resource Economics.
    10. Heckelei, Thomas & Wolff, Hendrik, 2002. "A Methodological Note on the Estimation of Programming Models," 2002 International Congress, August 28-31, 2002, Zaragoza, Spain 24896, European Association of Agricultural Economists.
    11. Esteban Fernández-Vázquez, 2014. "Estimating the effect of technological factors from samples affected by collinearity: a data-weighted entropy approach," Empirical Economics, Springer, vol. 47(2), pages 717-731, September.
    12. Hansen, Heiko & Surry, Yves R., 2006. "Die Schatzung Verfahrensspezifischer Faktoreinsatzmengen Fur Die Landwirtschaft In Deutschland," 46th Annual Conference, Giessen, Germany, October 4-6, 2006 14959, German Association of Agricultural Economists (GEWISOLA).
    13. You, Liangzhi & Wood, Stanley & Wood-Sichra, Ulrike, 2007. "Generating plausible crop distribution and performance maps for Sub-Saharan Africa using a spatially disaggregated data fusion and optimization approach:," IFPRI discussion papers 725, International Food Policy Research Institute (IFPRI).

    More about this item


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:ajagec:v:83:y:2001:i:2:p:366-377. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Oxford University Press) or (Christopher F. Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.