Advanced Search
MyIDEAS: Login

An alternative method to the scrambled Halton sequence for removing correlation between standard Halton sequences in high dimensions

Contents:

Author Info

  • Stephane Hess

    ()

  • John Polak

    ()

Registered author(s):

    Abstract

    Halton sequences were first introduced in the 1960s as an alternative to pseudo-random number sequences, with the aim of providing better coverage of the area of integration and negative correlation in the simulated probabilities between observations. This is needed in order to achieve variance reduction when using simulation to approximate an integral that does not have a closed-form expression. Such integrals arise in many areas of regional science, for example in the evaluation and estimation of certain types of discrete choice models. While the performance of standard Halton sequences is very good in low dimensions, problems with correlation have been observed between sequences generated from higher primes. This can cause serious problems in the estimation of models with high-dimensional integrals (e.g., models of aspects of spatial choice, such as route or location). Various methods have been proposed to deal with this; one of the most prominent solutions is the scrambled Halton sequence, which uses special predetermined permutations of the coefficients used in the construction of the standard sequence. In this paper, we conduct a detailed analysis of the ability of scrambled Halton sequences to remove the problematic correlation that exists between standard Halton sequences for high primes in the two-dimensional space. The analysis shows that although the scrambled sequences exhibit a lower degree of overall correlation than the standard sequences, for some choices of primes, correlation remains at an unacceptably high level. This paper then proposes an alternative method, based on the idea of using randomly shuffled versions of the one-dimensional standard Halton sequences in the construction of multi-dimensional sequences. We show that the new shuffled sequences produce a significantly higher reduction in correlation than the scrambled sequences, without loss of quality of coverage. Another substantial advantage of this new method is that it can, without any modifications, be used for any number of dimensions, while the use of the scrambled sequences requires the a-priori computation of a matrix of permutations, which for high dimensional problems could lead to significant runtime disadvantages. Repeated runs of the shuffling algorithm will also produce different sequences in different runs, which nevertheless maintain the same quality of one-dimensional coverage. This is not at all the case for the scrambled sequences. In view of the clear advantages in its ability to remove correlation, combined with its runtime and generalization advantages, this paper recommends that this new algorithm should be preferred to the scrambled Halton sequences when dealing with high correlation between standard Halton sequences.

    Download Info

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
    File URL: http://www-sre.wu-wien.ac.at/ersa/ersaconfs/ersa03/cdrom/papers/406.pdf
    Download Restriction: no

    Bibliographic Info

    Paper provided by European Regional Science Association in its series ERSA conference papers with number ersa03p406.

    as in new window
    Length:
    Date of creation: Aug 2003
    Date of revision:
    Handle: RePEc:wiw:wiwrsa:ersa03p406

    Contact details of provider:
    Postal: Welthandelsplatz 1, 1020 Vienna, Austria
    Web page: http://www.ersa.org

    Related research

    Keywords:

    This paper has been announced in the following NEP Reports:

    References

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
    as in new window
    1. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521747387, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as in new window

    Cited by:
    1. Danny Campbell & George Hutchinson & Riccardo Scarpa, 2006. "Using mixed logit models to derive individual-specific WTP estimates for landscape improvements under agri-environmental schemes: evidence from the Rural Environment Protection Scheme in Ireland," Working Papers 0607, Rural Economy and Development Programme,Teagasc.
    2. Cappellari, Lorenzo & Jenkins, Stephen P., 2006. "Calculation of Multivariate Normal Probabilities by Simulation, with Applications to Maximum Simulated Likelihood Estimation," IZA Discussion Papers 2112, Institute for the Study of Labor (IZA).
    3. Stephane Hess & John W. Polak, 2004. "An analysis of parking behaviour using discrete choice models calibrated on SP datasets," ERSA conference papers ersa04p60, European Regional Science Association.
    4. Danny Campbell & George Hutchinson & Riccardo Scarpa, 2006. "Lexicographic preferences for rural environmental landscape improvements: implications on individual-specific willingness to pay estimates," Working Papers 0610, Rural Economy and Development Programme,Teagasc.
    5. Lesley Chiou & Joan Walker, 2005. "Masking Identification of Discrete Choice Models under Simulation Methods," Occidental Economics Working Papers 5, Occidental College, Department of Economics, revised May 2006.
    6. Chiou, Lesley & Walker, Joan L., 2007. "Masking identification of discrete choice models under simulation methods," Journal of Econometrics, Elsevier, vol. 141(2), pages 683-703, December.
    7. Danny Campbell & W. George Hutchinson & Riccardo Scarpa, 2006. "Using Discrete Choice Experiments to Derive Individual-Specific WTP Estimates for Landscape Improvements under Agri-Environmental Schemes: Evidence from the Rural Environment Protection Scheme in Irel," Working Papers 2006.26, Fondazione Eni Enrico Mattei.

    Lists

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    Statistics

    Access and download statistics

    Corrections

    When requesting a correction, please mention this item's handle: RePEc:wiw:wiwrsa:ersa03p406. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Gunther Maier).

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.