IDEAS home Printed from https://ideas.repec.org/a/inm/orisre/v14y2003i2p127-145.html
   My bibliography  Save this article

Measuring Information Technology Payoff: A Meta-Analysis of Structural Variables in Firm-Level Empirical Research

Author

Listed:
  • Rajiv Kohli

    (Department of Management, University of Notre Dame, Notre Dame, Indiana 46556)

  • Sarv Devaraj

    (Department of Management, University of Notre Dame, Notre Dame, Indiana 46556)

Abstract

Payoffs from information technology (IT) continue to generate interest and debate both among academicians and practitioners. The extant literature cites inadequate sample size, lack of process orientation, and analysis methods among the reasons some studies have shown mixed results in establishing a relationship between IT investment and firm performance.In this paper we examine the structural variables that affect IT payoff through a meta analysis of 66 firm-level empirical studies between 1990 and 2000. Employing logistic regression and discriminant analyses, we present statistical evidence of the characteristics that discriminate between IT payoff studies that observed a positive effect and those that did not. In addition, we conduct ordinary least squares (OLS) regression on a continuous measure of IT payoff to examine the influence of structural variables on the result of IT payoff studies.The results indicate that the sample size, data source (firm-level or secondary), and industry in which the study is conducted influence the likelihood of the study finding greater improvements on firm performance. The choice of the dependent variable(s) also appears to influence the outcome (although we did not find support for process-oriented measurement), the type of statistical analysis conducted, and whether the study adopted a cross-sectional or longitudinal design. Finally, we present implications of the findings and recommendations for future research.

Suggested Citation

  • Rajiv Kohli & Sarv Devaraj, 2003. "Measuring Information Technology Payoff: A Meta-Analysis of Structural Variables in Firm-Level Empirical Research," Information Systems Research, INFORMS, vol. 14(2), pages 127-145, June.
    • Handle: RePEc:inm:orisre:v:14:y:2003:i:2:p:127-145
    DOI: 10.1287/isre.14.2.127.16019
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/isre.14.2.127.16019
    Download Restriction: no
    File URL: https://libkey.io/10.1287/isre.14.2.127.16019?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peter Nijkamp & Piet Rietveld & Laura Spierdijk, 2000. "A Meta-Analytic Comparison of Determinants of Public Transport Use: Methodology and Application," Environment and Planning B, , vol. 27(6), pages 893-903, December.
    2. Catherine J. Morrison, 2000. "Assessing The Productivity Of Information Technology Equipment In U.S. Manufacturing Industries," The Review of Economics and Statistics, MIT Press, vol. 79(3), pages 471-481, August.
    3. Dale W. Jorgenson, 2001. "Information Technology and the U.S. Economy," Higher School of Economics Economic Journal Экономический журнал Высшей школы экономики, CyberLeninka;Федеральное государственное автономное образовательное учреждение высшего образования «Национальный исследовательский университет «Высшая школа экономики», vol. 5(1), pages 3-34.
    4. Thompson SH, Teo & Poh Kam, Wong, 1998. "An Empirical Study of the Performance Impact of Computerization in the Retail Industry," Omega, Elsevier, vol. 26(5), pages 611-621, October.
    5. Dudley, Leonard & Lasserre, Pierre, 1989. "Information as a substitute for inventories," European Economic Review, Elsevier, vol. 33(1), pages 67-88, January.
    6. Anandhi S. Bharadwaj & Sundar G. Bharadwaj & Benn R. Konsynski, 1999. "Information Technology Effects on Firm Performance as Measured by Tobin's q," Management Science, INFORMS, vol. 45(7), pages 1008-1024, July.
    7. M. Lynne Markus & Daniel Robey, 1988. "Information Technology and Organizational Change: Causal Structure in Theory and Research," Management Science, INFORMS, vol. 34(5), pages 583-598, May.
    8. Dale W. Jorgenson & Kevin J. Stiroh, 2000. "Raising the Speed Limit: U.S. Economic Growth in the Information Age," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 31(1), pages 125-236.
    9. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    10. Koski, Heli, 1999. "The implications of network use, production network externalities and public networking programmes for firm's productivity," Research Policy, Elsevier, vol. 28(4), pages 423-439, April.
    11. Tridas Mukhopadhyay & Surendra Rajiv & Kannan Srinivasan, 1997. "Information Technology Impact on Process Output and Quality," Management Science, INFORMS, vol. 43(12), pages 1645-1659, December.
    12. Stephen D. Oliner & Daniel E. Sichel, 2000. "The Resurgence of Growth in the Late 1990s: Is Information Technology the Story?," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 3-22, Fall.
    13. Sanjeev Dewan & Chung-ki Min, 1997. "The Substitution of Information Technology for Other Factors of Production: A Firm Level Analysis," Management Science, INFORMS, vol. 43(12), pages 1660-1675, December.
    14. Bei-Hung Chang & Stuart Lipsitz & Christine Waternaux, 2000. "Logistic regression in meta-analysis using aggregate data," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(4), pages 411-424.
    15. Kar Yan Tam, 1998. "The Impact of Information Technology Investments on Firm Performance and Evaluation: Evidence from Newly Industrialized Economies," Information Systems Research, INFORMS, vol. 9(1), pages 85-98, March.
    16. Stoneman, Paul & Kwon, Myung Joong, 1996. "Technology Adoption and Firm Profitability," Economic Journal, Royal Economic Society, vol. 106(437), pages 952-962, July.
    17. Peter Weill, 1992. "The Relationship Between Investment in Information Technology and Firm Performance: A Study of the Valve Manufacturing Sector," Information Systems Research, INFORMS, vol. 3(4), pages 307-333, December.
    18. Daniel Robey & Marie-Claude Boudreau, 1999. "Accounting for the Contradictory Organizational Consequences of Information Technology: Theoretical Directions and Methodological Implications," Information Systems Research, INFORMS, vol. 10(2), pages 167-185, June.
    19. Byungtae Lee & Anitesh Barua, 1999. "An Integrated Assessment of Productivity and Efficiency Impacts of Information Technology Investments: Old Data, New Analysis and Evidence," Journal of Productivity Analysis, Springer, vol. 12(1), pages 21-43, August.
    20. Donald Siegel, 1997. "The Impact Of Computers On Manufacturing Productivity Growth: A Multiple-Indicators, Multiple-Causes Approach," The Review of Economics and Statistics, MIT Press, vol. 79(1), pages 68-78, February.
    21. Belleflamme, Paul, 2001. "Oligopolistic competition, IT use for product differentiation and the productivity paradox," International Journal of Industrial Organization, Elsevier, vol. 19(1-2), pages 227-248, January.
    22. Erik Brynjolfsson & Lorin M. Hitt, 2000. "Beyond Computation: Information Technology, Organizational Transformation and Business Performance," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 23-48, Fall.
    23. repec:bla:jindec:v:46:y:1998:i:2:p:257-79 is not listed on IDEAS
    24. Anitesh Barua & Byungtae Lee, 1997. "An Economic Analysis of the Introduction of an Electronic Data Interchange System," Information Systems Research, INFORMS, vol. 8(4), pages 398-422, December.
    25. Maryellen R. Kelley, 1994. "Productivity and Information Technology: The Elusive Connection," Management Science, INFORMS, vol. 40(11), pages 1406-1425, November.
    26. Byrd, T. A. & Marshall, T. E., 1997. "Relating information technology investment to organizational performance: a causal model analysis," Omega, Elsevier, vol. 25(1), pages 43-56, February.
    27. Erik Brynjolfsson & Lorin Hitt, 1996. "Paradox Lost? Firm-Level Evidence on the Returns to Information Systems Spending," Management Science, INFORMS, vol. 42(4), pages 541-558, April.
    28. Baba Prasad & Patrick T. Harker, 1997. "Examining the Contribution of Information Technology Toward Productivity and Profitability in U.S. Retail Banking," Center for Financial Institutions Working Papers 97-09, Wharton School Center for Financial Institutions, University of Pennsylvania.
    29. Rai, A. & Patnayakuni, R. & Patnayakuni, N., 1996. "Refocusing where and how IT value is realized: An empirical investigation," Omega, Elsevier, vol. 24(4), pages 399-412, August.
    30. Anitesh Barua & C. H. Sophie Lee & Andrew B. Whinston, 1996. "The Calculus of Reengineering," Information Systems Research, INFORMS, vol. 7(4), pages 409-428, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefan Schweikl & Robert Obermaier, 2020. "Lessons from three decades of IT productivity research: towards a better understanding of IT-induced productivity effects," Management Review Quarterly, Springer, vol. 70(4), pages 461-507, November.
    2. Sarv Devaraj & Rajiv Kohli, 2003. "Performance Impacts of Information Technology: Is Actual Usage the Missing Link?," Management Science, INFORMS, vol. 49(3), pages 273-289, March.
    3. Guido Schryen, 2010. "Preserving Knowledge on IS Business Value," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 2(4), pages 233-244, August.
    4. Chen, Yueh H. & Lin, Winston T., 2009. "Analyzing the relationships between information technology, inputs substitution and national characteristics based on CES stochastic frontier production models," International Journal of Production Economics, Elsevier, vol. 120(2), pages 552-569, August.
    5. Sinan Aral & Peter Weill, 2007. "IT Assets, Organizational Capabilities, and Firm Performance: How Resource Allocations and Organizational Differences Explain Performance Variation," Organization Science, INFORMS, vol. 18(5), pages 763-780, October.
    6. Neeraj Mittal & Barrie R. Nault, 2009. "Research Note ---Investments in Information Technology: Indirect Effects and Information Technology Intensity," Information Systems Research, INFORMS, vol. 20(1), pages 140-154, March.
    7. Gianfranco E. Atzeni & OA Carboni, 2004. "ICT productivity and firm propensity to innovative investment: learning effect evidence from italian micro data," Working Paper CRENoS 200414, Centre for North South Economic Research, University of Cagliari and Sassari, Sardinia.
    8. Fred V. Carstensen & William F. Lott & Stan McMillen, 2003. "The Economic Impact of Connecticut's Information Technology Industry," CCEA Studies 2003-02, University of Connecticut, Connecticut Center for Economic Analysis.
    9. Rajiv D. Banker & Robert J. Kauffman, 2004. "50th Anniversary Article: The Evolution of Research on Information Systems: A Fiftieth-Year Survey of the Literature in Management Science," Management Science, INFORMS, vol. 50(3), pages 281-298, March.
    10. Radhakrishnan, Abirami & Zu, Xingxing & Grover, Varun, 2008. "A process-oriented perspective on differential business value creation by information technology: An empirical investigation," Omega, Elsevier, vol. 36(6), pages 1105-1125, December.
    11. Saggi Nevo & Michael Wade & Wade D. Cook, 2010. "An empirical study of IT as a factor of production: The case of Net-enabled IT assets," Information Systems Frontiers, Springer, vol. 12(3), pages 323-335, July.
    12. Vincent J. Shea & Kevin E. Dow & Alain Yee-Loong Chong & Eric W. T. Ngai, 0. "An examination of the long-term business value of investments in information technology," Information Systems Frontiers, Springer, vol. 0, pages 1-15.
    13. S J Ho & S K Mallick, 2010. "The impact of information technology on the banking industry," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(2), pages 211-221, February.
    14. Prasad, Acklesh & Heales, Jon, 2010. "On IT and business value in developing countries: A complementarities-based approach," International Journal of Accounting Information Systems, Elsevier, vol. 11(4), pages 314-335.
    15. Erik Brynjolfsson & Lorin M. Hitt, 2003. "Computing Productivity: Firm-Level Evidence," The Review of Economics and Statistics, MIT Press, vol. 85(4), pages 793-808, November.
    16. Hyunbae Chun & Sung‐Bae Mun, 2006. "Substitutability and Accumulation of Information Technology Capital in U.S. Industries," Southern Economic Journal, John Wiley & Sons, vol. 72(4), pages 1002-1015, April.
    17. Young Bong Chang & Vijay Gurbaxani, 2013. "An Empirical Analysis of Technical Efficiency: The Role of IT Intensity and Competition," Information Systems Research, INFORMS, vol. 24(3), pages 561-578, September.
    18. Vincent J. Shea & Kevin E. Dow & Alain Yee-Loong Chong & Eric W. T. Ngai, 2019. "An examination of the long-term business value of investments in information technology," Information Systems Frontiers, Springer, vol. 21(1), pages 213-227, February.
    19. Sinan Aral & Erik Brynjolfsson & Marshall Van Alstyne, 2007. "Information, Technology and Information Worker Productivity: Task Level Evidence," NBER Working Papers 13172, National Bureau of Economic Research, Inc.
    20. Chou, Yen-Chun & Hao-Chun Chuang, Howard & Shao, Benjamin B.M., 2014. "The impacts of information technology on total factor productivity: A look at externalities and innovations," International Journal of Production Economics, Elsevier, vol. 158(C), pages 290-299.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:14:y:2003:i:2:p:127-145. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.