Surveying surveys and questioning questions - learning from World Bank experience
AbstractThe World Bank has increasingly focused on firm-level surveys to build the data foundation needed for accurate policy analysis in developing and transition economies. The authors take stock of some recent Bank surveys, and discuss how to improve their results. Lessons on data issues, and hypothesis testing: 1) Use panel data, if possible. 2) Have enough information about productivity to estimate a production function. 3) Avoid the paradigm of"list the severity of the obstacle/problem on a scale of 1 to 5". Instead, ask for data on specific dimensions of the problem that will shed light on alternative hypothesis and policy recommendations. 4) Pick particular disaggregated industries, and sample those industries in each survey. 5) Identify the most important interventions of interest, and consider how you will empirically identify specific changes by picking instruments useful for doing so. Lessons on questionnaire design: a) Incorporate only one idea or dimension in each question. Do not ask, in one question, about the"quality, integrity, and efficiency"of services, for example. b) Consider the costs and benefits of numeric scales compared with adjectival scales. Scales in which each point is labeled may be more precise than numeric scales in which only the end points are labeled. But responses are very sensitive to the exact adjective chosen, and it may be impossible to translate adjectives precisely across languages, making it impossible to compare responses across countries. c) Recognize that the share of respondents expressing opinions will be biased upward if the survey does not include a middle ("indifferent"or"don't know") category, and downward if it does include the middle category. d) When asking degree-of-concern and how-great-an-obstacle question, consider first asking a filter question (such as"Do you believe this regulation is an obstacle or not?"). If the answer isyes, then ask how severe the obstacle is. e) Be aware of the effects of context. The act of asking questions can affect the answers given on subsequent, related questions. f) Think carefully about how to ask sensitive questions. Consider using a self-administered module for sensitive questions. alternatively, a randomized response mechanisms may be a useful, truth-revealing mechanism.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by The World Bank in its series Policy Research Working Paper Series with number 2307.
Date of creation: 31 Mar 2000
Date of revision:
Environmental Economics&Policies; Decentralization; Labor Policies; Economic Theory&Research; ICT Policy and Strategies; Environmental Economics&Policies; Economic Theory&Research; ICT Policy and Strategies; Banks&Banking Reform; Statistical&Mathematical Sciences;
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- G. Steven Olley & Ariel Pakes, 1992. "The Dynamics of Productivity in the Telecommunications Equipment Industry," NBER Working Papers 3977, National Bureau of Economic Research, Inc.
- Caves, Richard E., 1989. "International differences in industrial organization," Handbook of Industrial Organization, in: R. Schmalensee & R. Willig (ed.), Handbook of Industrial Organization, edition 1, volume 2, chapter 21, pages 1225-1250 Elsevier.
- Jenkins, Carolyn, 1998. "Determinants of Private Investment in Zimbabwe," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 7(1), pages 34-61, March.
- Lazear, Edward P & Rosen, Sherwin, 1981.
"Rank-Order Tournaments as Optimum Labor Contracts,"
Journal of Political Economy,
University of Chicago Press, vol. 89(5), pages 841-64, October.
- Gyimah-Brempong, Kwabena & Traynor, Thomas L, 1999. "Political Instability, Investment and Economic Growth in Sub-Saharan Africa," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 8(1), pages 52-86, March.
- Sherwin Rosen, 1990. "Contracts and the Market for Executives," NBER Working Papers 3542, National Bureau of Economic Research, Inc.
- Andrei Shleifer & Robert W. Vishny, 1995.
"A Survey of Corporate Governance,"
Harvard Institute of Economic Research Working Papers
1741, Harvard - Institute of Economic Research.
- Shirley, Mary M. & L. Colin Xu, 1997.
"Information, incentives, and commitment : an empirical analysis of contracts between government and state enterprises,"
Policy Research Working Paper Series
1769, The World Bank.
- Shirley, Mary M & Xu, Lixin Colin, 1998. "Information, Incentives, and Commitment: An Empirical Analysis of Contracts between Government and State Enterprises," Journal of Law, Economics and Organization, Oxford University Press, vol. 14(2), pages 358-78, October.
- Mark Doms & Timothy Dunne, 1994.
"Capital Adjustment Patterns in Manufacturing Plants,"
94-11, Center for Economic Studies, U.S. Census Bureau.
- Mark E. Doms & Timothy Dunne, 1998. "Capital Adjustment Patterns in Manufacturing Plants," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(2), pages 409-429, April.
- Groves, Theodore, et al, 1994. "Autonomy and Incentives in Chinese State Enterprises," The Quarterly Journal of Economics, MIT Press, vol. 109(1), pages 183-209, February.
- Groves, Theodore & Yongmiao Hong & John McMillan & Barry Naughton, 1995. "China's Evolving Managerial Labor Market," Journal of Political Economy, University of Chicago Press, vol. 103(4), pages 873-92, August.
- Jeffrey Campbell, 1997. "Measuring and analyzing aggregate fluctuations: the importance of building from microeconomic evidence - commentary," Review, Federal Reserve Bank of St. Louis, issue May, pages 83-86.
- Lixin Colin Xu, 2000. "Control, Incentives and Competition: The Impact of Reform on Chinese State-owned Enterprises," The Economics of Transition, The European Bank for Reconstruction and Development, vol. 8(1), pages 151-173, March.
- Russell Cooper & John Haltiwanger & Laura Power, 1995.
"Machine Replacement and the Business Cycle: Lumps and Bumps,"
NBER Working Papers
5260, National Bureau of Economic Research, Inc.
- John Haltiwanger & Russell Cooper & Laura Power, 1999. "Machine Replacement and the Business Cycle: Lumps and Bumps," American Economic Review, American Economic Association, vol. 89(4), pages 921-946, September.
- Russell Cooper & John Haltiwanger & Laura Power, 1995. "Machine Replacement and the Business Cycle: Lumps and Bumps," Papers 0062, Boston University - Industry Studies Programme.
- Ricardo J. Caballero & Eduardo M. R. A. Engel & John C. Haltiwanger, 1995. "Plant-Level Adjustment and Aggregate Investment Dynamics," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 26(2), pages 1-54.
- George S Olley & Ariel Pakes, 1992. "The Dynamics Of Productivity In The Telecommunications Equipment Industry," Working Papers 92-2, Center for Economic Studies, U.S. Census Bureau.
- John C. Haltiwanger, 1997. "Measuring and analyzing aggregate fluctuations: the importance of building from microeconomic evidence," Review, Federal Reserve Bank of St. Louis, issue May, pages 55-78.
- Sherwin Rosen, 1985.
"Prizes and Incentives in Elimination Tournaments,"
NBER Working Papers
1668, National Bureau of Economic Research, Inc.
- Dollar, David & Hallward-Driemeier, Mary, 2000. "Crisis, Adjustment, and Reform in Thailand's Industrial Firms," World Bank Research Observer, World Bank Group, vol. 15(1), pages 1-22, February.
- Winston, Clifford, 1993. "Economic Deregulation: Days of Reckoning for Microeconomists," Journal of Economic Literature, American Economic Association, vol. 31(3), pages 1263-89, September.
- Willis, Robert J., 1987. "Wage determinants: A survey and reinterpretation of human capital earnings functions," Handbook of Labor Economics, in: O. Ashenfelter & R. Layard (ed.), Handbook of Labor Economics, edition 1, volume 1, chapter 10, pages 525-602 Elsevier.
- Shirley, Mary & Lixin Colin Xu, 1998. "The empirical effects of performance contracts: evidence from China," Policy Research Working Paper Series 1919, The World Bank.
- Mortensen, Dale T., 1994. "The cyclical behavior of job and worker flows," Journal of Economic Dynamics and Control, Elsevier, vol. 18(6), pages 1121-1142, November.
- Philipson, Tomas, 1997. "Data Markets and the Production of Surveys," Review of Economic Studies, Wiley Blackwell, vol. 64(1), pages 47-72, January.
- Andrew Atkeson & Patrick Kehoe, 1997. "Industry Evolution and Transition: A Neoclassical Benchmark," NBER Working Papers 6005, National Bureau of Economic Research, Inc.
- Devarajan, Shantayanan & Easterley, William R. & Pack, Howard, 2001. "Is investment in Africa too low or too high : macro and micro evidence," Policy Research Working Paper Series 2519, The World Bank.
- Blank, Lorraine & Grosh, Margaret, 1999. "Using Household Surveys to Build Analytic Capacity," World Bank Research Observer, World Bank Group, vol. 14(2), pages 209-27, August.
- Ramey, Garey & Watson, Joel, 1997. "Contractual Fragility, Job Destruction, and Business Cycles," The Quarterly Journal of Economics, MIT Press, vol. 112(3), pages 873-911, August.
- Clarke, George, 2008.
"How petty is petty corruption? Evidence from firm survey in Africa,"
15073, University Library of Munich, Germany, revised 24 Aug 2008.
- Clarke, George R.G., 2011. "How Petty is Petty Corruption? Evidence from Firm Surveys in Africa," World Development, Elsevier, vol. 39(7), pages 1122-1132, July.
- Jensen, Nathan M & Rahman, Aminur, 2011. "The silence of corruption : identifying underreporting of business corruption through randomized response techniques," Policy Research Working Paper Series 5696, The World Bank.
- Hallward-Driemeier, Mary, 2001. "Firm-level survey provides data on Asia's corporate crisis and recovery," Policy Research Working Paper Series 2515, The World Bank.
- Hallward-Driemeier, Mary & Wallsten, Scott & Lixin Colin Xu, 2003. "The investment climate and the firm : firm-level evidence from China," Policy Research Working Paper Series 3003, The World Bank.
- Jensen, Nathan M. & Li, Quan & Rahman, Aminur, 2007. "Heard melodies are sweet, but those unheard are sweeter : understanding corruption using cross-national firm-level surveys," Policy Research Working Paper Series 4413, The World Bank.
- World Bank, 2008. "Bulgaria - Investment Climate Assessment : Volume 2. Detailed Report," World Bank Other Operational Studies 7868, The World Bank.
- Manning, Nick & Mukherjee, Ranjana & Gokcekus, Omer, 2000. "Public officials and their institutional environment - an analytical model for assessing the impact of institutional change on public sector performance," Policy Research Working Paper Series 2427, The World Bank.
- Clarke, George, 2011. "Lying about firm performance: Evidence from a survey in Nigeria," MPRA Paper 35382, University Library of Munich, Germany.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Roula I. Yazigi).
If references are entirely missing, you can add them using this form.