IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/2307.html
   My bibliography  Save this paper

Surveying surveys and questioning questions - learning from World Bank experience

Author

Listed:
  • Recanatini, Francesca
  • Wallsten, Scott J.
  • Lixin Colin Xu

Abstract

The World Bank has increasingly focused on firm-level surveys to build the data foundation needed for accurate policy analysis in developing and transition economies. The authors take stock of some recent Bank surveys, and discuss how to improve their results. Lessons on data issues, and hypothesis testing: 1) Use panel data, if possible. 2) Have enough information about productivity to estimate a production function. 3) Avoid the paradigm of"list the severity of the obstacle/problem on a scale of 1 to 5". Instead, ask for data on specific dimensions of the problem that will shed light on alternative hypothesis and policy recommendations. 4) Pick particular disaggregated industries, and sample those industries in each survey. 5) Identify the most important interventions of interest, and consider how you will empirically identify specific changes by picking instruments useful for doing so. Lessons on questionnaire design: a) Incorporate only one idea or dimension in each question. Do not ask, in one question, about the"quality, integrity, and efficiency"of services, for example. b) Consider the costs and benefits of numeric scales compared with adjectival scales. Scales in which each point is labeled may be more precise than numeric scales in which only the end points are labeled. But responses are very sensitive to the exact adjective chosen, and it may be impossible to translate adjectives precisely across languages, making it impossible to compare responses across countries. c) Recognize that the share of respondents expressing opinions will be biased upward if the survey does not include a middle ("indifferent"or"don't know") category, and downward if it does include the middle category. d) When asking degree-of-concern and how-great-an-obstacle question, consider first asking a filter question (such as"Do you believe this regulation is an obstacle or not?"). If the answer isyes, then ask how severe the obstacle is. e) Be aware of the effects of context. The act of asking questions can affect the answers given on subsequent, related questions. f) Think carefully about how to ask sensitive questions. Consider using a self-administered module for sensitive questions. alternatively, a randomized response mechanisms may be a useful, truth-revealing mechanism.

Suggested Citation

  • Recanatini, Francesca & Wallsten, Scott J. & Lixin Colin Xu, 2000. "Surveying surveys and questioning questions - learning from World Bank experience," Policy Research Working Paper Series 2307, The World Bank.
  • Handle: RePEc:wbk:wbrwps:2307
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2000/04/24/000094946_00040806105193/Rendered/PDF/multi_page.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lazear, Edward P & Rosen, Sherwin, 1981. "Rank-Order Tournaments as Optimum Labor Contracts," Journal of Political Economy, University of Chicago Press, vol. 89(5), pages 841-864, October.
    2. Rosen, S., 1990. "Contracts and Market for Executives," University of Chicago - Economics Research Center 90-12, Chicago - Economics Research Center.
    3. Willis, Robert J., 1987. "Wage determinants: A survey and reinterpretation of human capital earnings functions," Handbook of Labor Economics, in: O. Ashenfelter & R. Layard (ed.), Handbook of Labor Economics, edition 1, volume 1, chapter 10, pages 525-602, Elsevier.
    4. Devarajan, Shantayanan & Easterley, William R. & Pack, Howard, 2001. "Is investment in Africa too low or too high : macro and micro evidence," Policy Research Working Paper Series 2519, The World Bank.
    5. Gyimah-Brempong, Kwabena & Traynor, Thomas L, 1999. "Political Instability, Investment and Economic Growth in Sub-Saharan Africa," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 8(1), pages 52-86, March.
    6. Shirley, Mary M & Xu, Lixin Colin, 2001. "Empirical Effects of Performance Contracts: Evidence from China," Journal of Law, Economics, and Organization, Oxford University Press, vol. 17(1), pages 168-200, April.
    7. Winston, Clifford, 1993. "Economic Deregulation: Days of Reckoning for Microeconomists," Journal of Economic Literature, American Economic Association, vol. 31(3), pages 1263-1289, September.
    8. Rosen, Sherwin, 1986. "Prizes and Incentives in Elimination Tournaments," American Economic Review, American Economic Association, vol. 76(4), pages 701-715, September.
    9. Shleifer, Andrei & Vishny, Robert W, 1997. "A Survey of Corporate Governance," Journal of Finance, American Finance Association, vol. 52(2), pages 737-783, June.
    10. John Haltiwanger & Russell Cooper & Laura Power, 1999. "Machine Replacement and the Business Cycle: Lumps and Bumps," American Economic Review, American Economic Association, vol. 89(4), pages 921-946, September.
    11. Tomas Philipson, 1997. "Data Markets and the Production of Surveys," Review of Economic Studies, Oxford University Press, vol. 64(1), pages 47-72.
    12. Jan Willem Gunning & Taye Mengistae, 2001. "Determinants of African Manufacturing Investment: the Microeconomic Evidence," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 10(suppl_2), pages 48-80.
    13. Blank, Lorraine & Grosh, Margaret, 1999. "Using Household Surveys to Build Analytic Capacity," World Bank Research Observer, World Bank Group, vol. 14(2), pages 209-227, August.
    14. Olley, G Steven & Pakes, Ariel, 1996. "The Dynamics of Productivity in the Telecommunications Equipment Industry," Econometrica, Econometric Society, vol. 64(6), pages 1263-1297, November.
    15. John Haltiwanger, 1997. "Measuring and analyzing aggregate fluctuations: the importance of building from microeconomic evidence," Review, Federal Reserve Bank of St. Louis, issue May, pages 55-78.
    16. Garey Ramey & Joel Watson, 1997. "Contractual Fragility, Job Destruction, and Business Cycles," The Quarterly Journal of Economics, Oxford University Press, vol. 112(3), pages 873-911.
    17. Shirley, Mary M & Xu, Lixin Colin, 1998. "Information, Incentives, and Commitment: An Empirical Analysis of Contracts between Government and State Enterprises," Journal of Law, Economics, and Organization, Oxford University Press, vol. 14(2), pages 358-378, October.
    18. Mark E. Doms & Timothy Dunne, 1998. "Capital Adjustment Patterns in Manufacturing Plants," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(2), pages 409-429, April.
    19. Jenkins, Carolyn, 1998. "Determinants of Private Investment in Zimbabwe," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 7(1), pages 34-61, March.
    20. Stone, Andrew H.W., 1992. "Listening to firms : how to use firm-level surveys to assess constraints on private sector development," Policy Research Working Paper Series 923, The World Bank.
    21. Ricardo J. Caballero & Eduardo M. R. A. Engel & John C. Haltiwanger, 1995. "Plant-Level Adjustment and Aggregate Investment Dynamics," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 26(2), pages 1-54.
    22. Mortensen, Dale T., 1994. "The cyclical behavior of job and worker flows," Journal of Economic Dynamics and Control, Elsevier, vol. 18(6), pages 1121-1142, November.
    23. Theodore Groves & Yongmiao Hong & John McMillan & Barry Naughton, 1994. "Autonomy and Incentives in Chinese State Enterprises," The Quarterly Journal of Economics, Oxford University Press, vol. 109(1), pages 183-209.
    24. Caves, Richard E., 1989. "International differences in industrial organization," Handbook of Industrial Organization, in: R. Schmalensee & R. Willig (ed.), Handbook of Industrial Organization, edition 1, volume 2, chapter 21, pages 1225-1250, Elsevier.
    25. Lixin Colin Xu, 2000. "Control, Incentives and Competition: The Impact of Reform on Chinese State‐owned Enterprises," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 8(1), pages 151-173, March.
    26. Groves, Theodore & Yongmiao Hong & John McMillan & Barry Naughton, 1995. "China's Evolving Managerial Labor Market," Journal of Political Economy, University of Chicago Press, vol. 103(4), pages 873-892, August.
    27. Dollar, David & Hallward-Driemeier, Mary, 2000. "Crisis, Adjustment, and Reform in Thailand's Industrial Firms," World Bank Research Observer, World Bank Group, vol. 15(1), pages 1-22, February.
    28. Andrew Atkeson & Patrick Kehoe, 1997. "Industry Evolution and Transition: A Neoclassical Benchmark," NBER Working Papers 6005, National Bureau of Economic Research, Inc.
    29. Jeffrey R. Campbell, 1997. "Measuring and analyzing aggregate fluctuations: the importance of building from microeconomic evidence - commentary," Review, Federal Reserve Bank of St. Louis, issue May, pages 83-86.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jensen, Nathan M. & Li, Quan & Rahman, Aminur, 2007. "Heard melodies are sweet, but those unheard are sweeter : understanding corruption using cross-national firm-level surveys," Policy Research Working Paper Series 4413, The World Bank.
    2. Manning, Nick & Mukherjee, Ranjana & Gokcekus, Omer, 2000. "Public officials and their institutional environment - an analytical model for assessing the impact of institutional change on public sector performance," Policy Research Working Paper Series 2427, The World Bank.
    3. Clarke, George, 2011. "Lying about firm performance: Evidence from a survey in Nigeria," MPRA Paper 35382, University Library of Munich, Germany.
    4. Clarke, George R.G., 2011. "How Petty is Petty Corruption? Evidence from Firm Surveys in Africa," World Development, Elsevier, vol. 39(7), pages 1122-1132, July.
    5. Hallward-Driemeier, Mary & Wallsten, Scott & Lixin Colin Xu, 2003. "The investment climate and the firm : firm-level evidence from China," Policy Research Working Paper Series 3003, The World Bank.
    6. Hallward-Driemeier, Mary, 2001. "Firm-level survey provides data on Asia's corporate crisis and recovery," Policy Research Working Paper Series 2515, The World Bank.
    7. Jensen, Nathan M & Rahman, Aminur, 2011. "The silence of corruption : identifying underreporting of business corruption through randomized response techniques," Policy Research Working Paper Series 5696, The World Bank.
    8. World Bank, 2008. "Bulgaria - Investment Climate Assessment : Volume 2. Detailed Report," World Bank Publications - Reports 7868, The World Bank Group.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:eee:labchp:v:3:y:1999:i:pb:p:2711-2805 is not listed on IDEAS
    2. Aivazian, Varouj A. & Ge, Ying & Qiu, Jiaping, 2005. "Corporate governance and manager turnover: An unusual social experiment," Journal of Banking & Finance, Elsevier, vol. 29(6), pages 1459-1481, June.
    3. Chong-En Bai & Qiao Liu & Joe Lu & Frank Song & Junxi Zhang, 2003. "Corporate Governance and Market Valuation in China," William Davidson Institute Working Papers Series 2003-564, William Davidson Institute at the University of Michigan.
    4. John Haltiwanger & Russell Cooper & Laura Power, 1999. "Machine Replacement and the Business Cycle: Lumps and Bumps," American Economic Review, American Economic Association, vol. 89(4), pages 921-946, September.
    5. Marcela Eslava & John Haltiwanger & Adriana Kugler & Maurice Kugler, 2010. "Factor Adjustments after Deregulation: Panel Evidence from Colombian Plants," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 378-391, May.
    6. Douglas W Dwyer, 1995. "Technology Locks, Creative Destruction And Non-Convergence In Productivity Levels," Working Papers 95-6, Center for Economic Studies, U.S. Census Bureau.
    7. Douglas Dwyer, 1998. "Technology Locks, Creative Destruction, and Non-Convergence in Productivity Levels," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(2), pages 430-473, April.
    8. Lucia Foster & John C. Haltiwanger & C. J. Krizan, 2001. "Aggregate Productivity Growth: Lessons from Microeconomic Evidence," NBER Chapters, in: New Developments in Productivity Analysis, pages 303-372, National Bureau of Economic Research, Inc.
    9. repec:cvs:starer:9724 is not listed on IDEAS
    10. Pinar Celikkol Geylani & Spiro E. Stefanou, 2008. "Linking Investment Spikes and Productivity Growth: U.S. Food Manufacturing Industry," Working Papers 08-36, Center for Economic Studies, U.S. Census Bureau.
    11. Zhang, Le-Yin, 2004. "The Roles of Corporatization and Stock Market Listing in Reforming China's State Industry," World Development, Elsevier, vol. 32(12), pages 2031-2047, December.
    12. Hu, Fang & Tan, Weiqiang & Xin, Qingquan & Yang, Sixian, 2013. "How do market forces affect executive compensation in Chinese state-owned enterprises?," China Economic Review, Elsevier, vol. 25(C), pages 78-87.
    13. Bai, Chong-En & Liu, Qiao & Lu, Joe & Song, Frank M. & Zhang, Junxi, 2004. "Corporate governance and market valuation in China," Journal of Comparative Economics, Elsevier, vol. 32(4), pages 599-616, December.
    14. Bruno Crépon & Richard Duhautois, 2003. "Ralentissement de la productivité et réallocations d'emplois : deux régimes de croissance," Économie et Statistique, Programme National Persée, vol. 367(1), pages 69-82.
    15. Lixin Colin Xu & Tian Zhu & Yi‐min Lin, 2005. "Politician control, agency problems and ownership reform," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 13(1), pages 1-24, January.
    16. Gourio, Francois & Kashyap, Anil K, 2007. "Investment spikes: New facts and a general equilibrium exploration," Journal of Monetary Economics, Elsevier, vol. 54(Supplemen), pages 1-22, September.
    17. Raouf Boucekkine & Bruno de Oliveira Cruz, 2015. "Technological Progress and Investment: A Non-Technical Survey," AMSE Working Papers 1519, Aix-Marseille School of Economics, France.
    18. Tor Eriksson, 2005. "Managerial pay and executive turnover in the Czech and Slovak Republics," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 13(4), pages 659-677, October.
    19. Gebreeyesus, Mulu, 2009. "Inactions and Spikes of Investment in Ethiopian Manufacturing Firms: Empirical Evidence on Irreversibility and Non-convexities," MERIT Working Papers 2009-061, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    20. Caballero, Ricardo J., 1999. "Aggregate investment," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 12, pages 813-862, Elsevier.
    21. Austan Goolsbee & David B. Gross, 1997. "Estimating Adjustment Costs with Data on Heterogeneous Capital Goods," NBER Working Papers 6342, National Bureau of Economic Research, Inc.
    22. Russell W. Cooper & John C. Haltiwanger, 2006. "On the Nature of Capital Adjustment Costs," Review of Economic Studies, Oxford University Press, vol. 73(3), pages 611-633.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:2307. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.