Surveying surveys and questioning questions - learning from World Bank experience
The World Bank has increasingly focused on firm-level surveys to build the data foundation needed for accurate policy analysis in developing and transition economies. The authors take stock of some recent Bank surveys, and discuss how to improve their results. Lessons on data issues, and hypothesis testing: 1) Use panel data, if possible. 2) Have enough information about productivity to estimate a production function. 3) Avoid the paradigm of"list the severity of the obstacle/problem on a scale of 1 to 5". Instead, ask for data on specific dimensions of the problem that will shed light on alternative hypothesis and policy recommendations. 4) Pick particular disaggregated industries, and sample those industries in each survey. 5) Identify the most important interventions of interest, and consider how you will empirically identify specific changes by picking instruments useful for doing so. Lessons on questionnaire design: a) Incorporate only one idea or dimension in each question. Do not ask, in one question, about the"quality, integrity, and efficiency"of services, for example. b) Consider the costs and benefits of numeric scales compared with adjectival scales. Scales in which each point is labeled may be more precise than numeric scales in which only the end points are labeled. But responses are very sensitive to the exact adjective chosen, and it may be impossible to translate adjectives precisely across languages, making it impossible to compare responses across countries. c) Recognize that the share of respondents expressing opinions will be biased upward if the survey does not include a middle ("indifferent"or"don't know") category, and downward if it does include the middle category. d) When asking degree-of-concern and how-great-an-obstacle question, consider first asking a filter question (such as"Do you believe this regulation is an obstacle or not?"). If the answer isyes, then ask how severe the obstacle is. e) Be aware of the effects of context. The act of asking questions can affect the answers given on subsequent, related questions. f) Think carefully about how to ask sensitive questions. Consider using a self-administered module for sensitive questions. alternatively, a randomized response mechanisms may be a useful, truth-revealing mechanism.
|Date of creation:||31 Mar 2000|
|Contact details of provider:|| Postal: 1818 H Street, N.W., Washington, DC 20433|
Phone: (202) 477-1234
Web page: http://www.worldbank.org/
More information through EDIRC
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Lazear, Edward P & Rosen, Sherwin, 1981.
"Rank-Order Tournaments as Optimum Labor Contracts,"
Journal of Political Economy,
University of Chicago Press, vol. 89(5), pages 841-864, October.
- Edward P. Lazear & Sherwin Rosen, 1979. "Rank-Order Tournaments as Optimum Labor Contracts," NBER Working Papers 0401, National Bureau of Economic Research, Inc.
- Sherwin Rosen, 1990. "Contracts and the Market for Executives," NBER Working Papers 3542, National Bureau of Economic Research, Inc.
- Willis, Robert J., 1987. "Wage determinants: A survey and reinterpretation of human capital earnings functions," Handbook of Labor Economics, in: O. Ashenfelter & R. Layard (ed.), Handbook of Labor Economics, edition 1, volume 1, chapter 10, pages 525-602 Elsevier.
- Devarajan, Shantayanan & Easterley, William R. & Pack, Howard, 2001. "Is investment in Africa too low or too high : macro and micro evidence," Policy Research Working Paper Series 2519, The World Bank.
- Gyimah-Brempong, Kwabena & Traynor, Thomas L, 1999. "Political Instability, Investment and Economic Growth in Sub-Saharan Africa," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 8(1), pages 52-86, March.
- Winston, Clifford, 1993. "Economic Deregulation: Days of Reckoning for Microeconomists," Journal of Economic Literature, American Economic Association, vol. 31(3), pages 1263-1289, September.
- Sherwin Rosen, 1985. "Prizes and Incentives in Elimination Tournaments," NBER Working Papers 1668, National Bureau of Economic Research, Inc.
- Shleifer, Andrei & Vishny, Robert W, 1997. " A Survey of Corporate Governance," Journal of Finance, American Finance Association, vol. 52(2), pages 737-783, June.
- Andrei Shleifer & Robert W. Vishny, 1995. "A Survey of Corporate Governance," Harvard Institute of Economic Research Working Papers 1741, Harvard - Institute of Economic Research.
- Andrei Shleifer & Robert W. Vishny, 1996. "A Survey of Corporate Governance," NBER Working Papers 5554, National Bureau of Economic Research, Inc.
- Shirley, Mary & Lixin Colin Xu, 1998. "The empirical effects of performance contracts: evidence from China," Policy Research Working Paper Series 1919, The World Bank.
- Russell Cooper & John Haltiwanger & Laura Power, 1995. "Machine Replacement and the Business Cycle: Lumps and Bumps," Papers 0062, Boston University - Industry Studies Programme.
- Russell Cooper & John Haltiwanger & Laura Power, 1995. "Machine Replacement and the Business Cycle: Lumps and Bumps," NBER Working Papers 5260, National Bureau of Economic Research, Inc.
- Tomas Philipson, 1997. "Data Markets and the Production of Surveys," Review of Economic Studies, Oxford University Press, vol. 64(1), pages 47-72.
- Blank, Lorraine & Grosh, Margaret, 1999. "Using Household Surveys to Build Analytic Capacity," World Bank Research Observer, World Bank Group, vol. 14(2), pages 209-227, August.
- Olley, G Steven & Pakes, Ariel, 1996. "The Dynamics of Productivity in the Telecommunications Equipment Industry," Econometrica, Econometric Society, vol. 64(6), pages 1263-1297, November.
- G. Steven Olley & Ariel Pakes, 1992. "The Dynamics of Productivity in the Telecommunications Equipment Industry," NBER Working Papers 3977, National Bureau of Economic Research, Inc.
- George S Olley & Ariel Pakes, 1992. "The Dynamics Of Productivity In The Telecommunications Equipment Industry," Working Papers 92-2, Center for Economic Studies, U.S. Census Bureau.
- John Haltiwanger, 1997. "Measuring and analyzing aggregate fluctuations: the importance of building from microeconomic evidence," Review, Federal Reserve Bank of St. Louis, issue May, pages 55-78.
- Garey Ramey & Joel Watson, 1997. "Contractual Fragility, Job Destruction, and Business Cycles," The Quarterly Journal of Economics, Oxford University Press, vol. 112(3), pages 873-911.
- Shirley, Mary M & Xu, Lixin Colin, 1998. "Information, Incentives, and Commitment: An Empirical Analysis of Contracts between Government and State Enterprises," Journal of Law, Economics and Organization, Oxford University Press, vol. 14(2), pages 358-378, October.
- Shirley, Mary M. & L. Colin Xu, 1997. "Information, incentives, and commitment : an empirical analysis of contracts between government and state enterprises," Policy Research Working Paper Series 1769, The World Bank.
- Mark E. Doms & Timothy Dunne, 1998. "Capital Adjustment Patterns in Manufacturing Plants," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(2), pages 409-429, April.
- Mark Doms & Timothy Dunne, 1994. "Capital Adjustment Patterns in Manufacturing Plants," Working Papers 94-11, Center for Economic Studies, U.S. Census Bureau.
- Jenkins, Carolyn, 1998. "Determinants of Private Investment in Zimbabwe," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 7(1), pages 34-61, March.
- Ricardo J. Caballero & Eduardo M. R. A. Engel & John C. Haltiwanger, 1995. "Plant-Level Adjustment and Aggregate Investment Dynamics," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 26(2), pages 1-54.
- Mortensen, Dale T., 1994. "The cyclical behavior of job and worker flows," Journal of Economic Dynamics and Control, Elsevier, vol. 18(6), pages 1121-1142, November.
- Theodore Groves & Yongmiao Hong & John McMillan & Barry Naughton, 1994. "Autonomy and Incentives in Chinese State Enterprises," The Quarterly Journal of Economics, Oxford University Press, vol. 109(1), pages 183-209.
- Caves, Richard E., 1989. "International differences in industrial organization," Handbook of Industrial Organization, in: R. Schmalensee & R. Willig (ed.), Handbook of Industrial Organization, edition 1, volume 2, chapter 21, pages 1225-1250 Elsevier.
- Lixin Colin Xu, 2000. "Control, Incentives and Competition: The Impact of Reform on Chinese State-owned Enterprises," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 8(1), pages 151-173, March.
- Groves, Theodore & Yongmiao Hong & John McMillan & Barry Naughton, 1995. "China's Evolving Managerial Labor Market," Journal of Political Economy, University of Chicago Press, vol. 103(4), pages 873-892, August.
- Dollar, David & Hallward-Driemeier, Mary, 2000. "Crisis, Adjustment, and Reform in Thailand's Industrial Firms," World Bank Research Observer, World Bank Group, vol. 15(1), pages 1-22, February.
- Andrew Atkeson & Patrick Kehoe, 1997. "Industry Evolution and Transition: A Neoclassical Benchmark," NBER Working Papers 6005, National Bureau of Economic Research, Inc.
- Jeffrey R. Campbell, 1997. "Measuring and analyzing aggregate fluctuations: the importance of building from microeconomic evidence - commentary," Review, Federal Reserve Bank of St. Louis, issue May, pages 83-86. Full references (including those not matched with items on IDEAS)