IDEAS home Printed from https://ideas.repec.org/p/wiw/wiwrsa/ersa03p406.html
   My bibliography  Save this paper

An alternative method to the scrambled Halton sequence for removing correlation between standard Halton sequences in high dimensions

Author

Listed:
  • Stephane Hess
  • John Polak

Abstract

Halton sequences were first introduced in the 1960s as an alternative to pseudo-random number sequences, with the aim of providing better coverage of the area of integration and negative correlation in the simulated probabilities between observations. This is needed in order to achieve variance reduction when using simulation to approximate an integral that does not have a closed-form expression. Such integrals arise in many areas of regional science, for example in the evaluation and estimation of certain types of discrete choice models. While the performance of standard Halton sequences is very good in low dimensions, problems with correlation have been observed between sequences generated from higher primes. This can cause serious problems in the estimation of models with high-dimensional integrals (e.g., models of aspects of spatial choice, such as route or location). Various methods have been proposed to deal with this; one of the most prominent solutions is the scrambled Halton sequence, which uses special predetermined permutations of the coefficients used in the construction of the standard sequence. In this paper, we conduct a detailed analysis of the ability of scrambled Halton sequences to remove the problematic correlation that exists between standard Halton sequences for high primes in the two-dimensional space. The analysis shows that although the scrambled sequences exhibit a lower degree of overall correlation than the standard sequences, for some choices of primes, correlation remains at an unacceptably high level. This paper then proposes an alternative method, based on the idea of using randomly shuffled versions of the one-dimensional standard Halton sequences in the construction of multi-dimensional sequences. We show that the new shuffled sequences produce a significantly higher reduction in correlation than the scrambled sequences, without loss of quality of coverage. Another substantial advantage of this new method is that it can, without any modifications, be used for any number of dimensions, while the use of the scrambled sequences requires the a-priori computation of a matrix of permutations, which for high dimensional problems could lead to significant runtime disadvantages. Repeated runs of the shuffling algorithm will also produce different sequences in different runs, which nevertheless maintain the same quality of one-dimensional coverage. This is not at all the case for the scrambled sequences. In view of the clear advantages in its ability to remove correlation, combined with its runtime and generalization advantages, this paper recommends that this new algorithm should be preferred to the scrambled Halton sequences when dealing with high correlation between standard Halton sequences.

Suggested Citation

  • Stephane Hess & John Polak, 2003. "An alternative method to the scrambled Halton sequence for removing correlation between standard Halton sequences in high dimensions," ERSA conference papers ersa03p406, European Regional Science Association.
  • Handle: RePEc:wiw:wiwrsa:ersa03p406
    as

    Download full text from publisher

    File URL: https://www-sre.wu.ac.at/ersa/ersaconfs/ersa03/cdrom/papers/406.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521747387, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chiou, Lesley & Walker, Joan L., 2007. "Masking identification of discrete choice models under simulation methods," Journal of Econometrics, Elsevier, vol. 141(2), pages 683-703, December.
    2. Campbell, Danny & Hutchinson, W. George & Scarpa, Riccardo, 2006. "Using Discrete Choice Experiments to Derive Individual-Specific WTP Estimates for Landscape Improvements under Agri-Environmental Schemes: Evidence from the Rural Environment Protection Scheme in Irel," Sustainability Indicators and Environmental Valuation Working Papers 12220, Fondazione Eni Enrico Mattei (FEEM).
    3. Danny Campbell, 2007. "Willingness to Pay for Rural Landscape Improvements: Combining Mixed Logit and Random‐Effects Models," Journal of Agricultural Economics, Wiley Blackwell, vol. 58(3), pages 467-483, September.
    4. Danny Campbell & George Hutchinson & Riccardo Scarpa, 2006. "Using mixed logit models to derive individual-specific WTP estimates for landscape improvements under agri-environmental schemes: evidence from the Rural Environment Protection Scheme in Ireland," Working Papers 0607, Rural Economy and Development Programme,Teagasc.
    5. Lorenzo Cappellari & Stephen P. Jenkins, 2006. "Calculation of multivariate normal probabilities by simulation, with applications to maximum simulated likelihood estimation," Stata Journal, StataCorp LP, vol. 6(2), pages 156-189, June.
    6. Danny Campbell & W George Hutchinson & Riccardo Scarpa, 2009. "Using Choice Experiments to Explore the Spatial Distribution of Willingness to Pay for Rural Landscape Improvements," Environment and Planning A, , vol. 41(1), pages 97-111, January.
    7. Danny Campbell & W. George Hutchinson & Riccardo Scarpa, 2006. "Lexicographic Preferences in Discrete Choice Experiments: Consequences on Individual-Specific Willingness to Pay Estimates," Working Papers 2006.128, Fondazione Eni Enrico Mattei.
    8. Danny Campbell & W. George Hutchinson & Riccardo Scarpa, 2006. "Lexicographic Preferences in Discrete Choice Experiments: Consequences on Individual-Specific Willingness to Pay Estimates," Working Papers 2006.128, Fondazione Eni Enrico Mattei.
    9. Danny Campbell & George Hutchinson & Riccardo Scarpa, 2006. "Integrating landscape improvement indices and discrete choice experiments: evidence from the Rural Environment Protection Scheme in Ireland," Working Papers 0609, Rural Economy and Development Programme,Teagasc.
    10. Stephane Hess & John W. Polak, 2004. "An analysis of parking behaviour using discrete choice models calibrated on SP datasets," ERSA conference papers ersa04p60, European Regional Science Association.
    11. Campbell, Danny, 2007. "Combining mixed logit models and random effects models to identify the determinants of willingness to pay for rural landscape improvements," 81st Annual Conference, April 2-4, 2007, Reading University, UK 7975, Agricultural Economics Society.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhifeng Gao & Ted C. Schroeder, 2009. "Consumer responses to new food quality information: are some consumers more sensitive than others?," Agricultural Economics, International Association of Agricultural Economists, vol. 40(3), pages 339-346, May.
    2. Cheng, Leilei & Yin, Changbin & Chien, Hsiaoping, 2015. "Demand for milk quantity and safety in urban China: evidence from Beijing and Harbin," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 59(2), April.
    3. Wen, Chieh-Hua & Huang, Chia-Jung & Fu, Chiang, 2020. "Incorporating continuous representation of preferences for flight departure times into stated itinerary choice modeling," Transport Policy, Elsevier, vol. 98(C), pages 10-20.
    4. Johannes Buggle & Thierry Mayer & Seyhun Orcan Sakalli & Mathias Thoenig, 2023. "The Refugee’s Dilemma: Evidence from Jewish Migration out of Nazi Germany," The Quarterly Journal of Economics, Oxford University Press, vol. 138(2), pages 1273-1345.
    5. Christelis, Dimitris & Dobrescu, Loretti I. & Motta, Alberto, 2020. "Early life conditions and financial risk-taking in older age," The Journal of the Economics of Ageing, Elsevier, vol. 17(C).
    6. Ortega, David L. & Wang, H. Holly & Wu, Laping & Hong, Soo Jeong, 2015. "Retail channel and consumer demand for food quality in China," China Economic Review, Elsevier, vol. 36(C), pages 359-366.
    7. Tina Birgitte Hansen & Jes Sanddal Lindholt & Axel Diederichsen & Rikke Søgaard, 2019. "Do Non-participants at Screening have a Different Threshold for an Acceptable Benefit–Harm Ratio than Participants? Results of a Discrete Choice Experiment," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 12(5), pages 491-501, October.
    8. Doyle, Orla & Fidrmuc, Jan, 2006. "Who favors enlargement?: Determinants of support for EU membership in the candidate countries' referenda," European Journal of Political Economy, Elsevier, vol. 22(2), pages 520-543, June.
    9. Tovar, Jorge, 2012. "Consumers’ Welfare and Trade Liberalization: Evidence from the Car Industry in Colombia," World Development, Elsevier, vol. 40(4), pages 808-820.
    10. Pereira, Pedro & Ribeiro, Tiago, 2011. "The impact on broadband access to the Internet of the dual ownership of telephone and cable networks," International Journal of Industrial Organization, Elsevier, vol. 29(2), pages 283-293, March.
    11. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    12. Potoglou, Dimitris & Palacios, Juan & Feijoo, Claudio & Gómez Barroso, Jose-Luis, 2015. "The supply of personal information: A study on the determinants of information provision in e-commerce scenarios," 26th European Regional ITS Conference, Madrid 2015 127174, International Telecommunications Society (ITS).
    13. Sant'Anna, Ana Claudia & Bergtold, Jason & Shanoyan, Aleksan & Caldas, Marcellus & Granco, Gabriel, 2021. "Deal or No Deal? Analysis of Bioenergy Feedstock Contract Choice with Multiple Opt-out Options and Contract Attribute Substitutability," 2021 Conference, August 17-31, 2021, Virtual 315289, International Association of Agricultural Economists.
    14. Simon P. Anderson & André de Palma, 2012. "Competition for attention in the Information (overload) Age," RAND Journal of Economics, RAND Corporation, vol. 43(1), pages 1-25, March.
    15. Mtimet, Nadhem & Ujiie, Kiyokazu & Kashiwagi, Kenichi & Zaibet, Lokman & Nagaki, Masakazu, 2011. "The effects of Information and Country of Origin on Japanese Olive Oil Consumer Selection," 2011 International Congress, August 30-September 2, 2011, Zurich, Switzerland 114642, European Association of Agricultural Economists.
    16. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    17. Boyce, Christopher & Czajkowski, Mikołaj & Hanley, Nick, 2019. "Personality and economic choices," Journal of Environmental Economics and Management, Elsevier, vol. 94(C), pages 82-100.
    18. Choi, Andy S., 2013. "Nonmarket values of major resources in the Korean DMZ areas: A test of distance decay," Ecological Economics, Elsevier, vol. 88(C), pages 97-107.
    19. Doherty, Edel & Campbell, Danny, 2011. "Demand for improved food safety and quality: a cross-regional comparison," 85th Annual Conference, April 18-20, 2011, Warwick University, Coventry, UK 108791, Agricultural Economics Society.
    20. Schleich, Joachim & Faure, Corinne & Guetlein, Marie-Charlotte & Tu, Gengyang, 2020. "Conveyance, envy, and homeowner choice of appliances," Energy Economics, Elsevier, vol. 89(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wiw:wiwrsa:ersa03p406. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Gunther Maier (email available below). General contact details of provider: http://www.ersa.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.