An alternative method to the scrambled Halton sequence for removing correlation between standard Halton sequences in high dimensions
AbstractHalton sequences were first introduced in the 1960s as an alternative to pseudo-random number sequences, with the aim of providing better coverage of the area of integration and negative correlation in the simulated probabilities between observations. This is needed in order to achieve variance reduction when using simulation to approximate an integral that does not have a closed-form expression. Such integrals arise in many areas of regional science, for example in the evaluation and estimation of certain types of discrete choice models. While the performance of standard Halton sequences is very good in low dimensions, problems with correlation have been observed between sequences generated from higher primes. This can cause serious problems in the estimation of models with high-dimensional integrals (e.g., models of aspects of spatial choice, such as route or location). Various methods have been proposed to deal with this; one of the most prominent solutions is the scrambled Halton sequence, which uses special predetermined permutations of the coefficients used in the construction of the standard sequence. In this paper, we conduct a detailed analysis of the ability of scrambled Halton sequences to remove the problematic correlation that exists between standard Halton sequences for high primes in the two-dimensional space. The analysis shows that although the scrambled sequences exhibit a lower degree of overall correlation than the standard sequences, for some choices of primes, correlation remains at an unacceptably high level. This paper then proposes an alternative method, based on the idea of using randomly shuffled versions of the one-dimensional standard Halton sequences in the construction of multi-dimensional sequences. We show that the new shuffled sequences produce a significantly higher reduction in correlation than the scrambled sequences, without loss of quality of coverage. Another substantial advantage of this new method is that it can, without any modifications, be used for any number of dimensions, while the use of the scrambled sequences requires the a-priori computation of a matrix of permutations, which for high dimensional problems could lead to significant runtime disadvantages. Repeated runs of the shuffling algorithm will also produce different sequences in different runs, which nevertheless maintain the same quality of one-dimensional coverage. This is not at all the case for the scrambled sequences. In view of the clear advantages in its ability to remove correlation, combined with its runtime and generalization advantages, this paper recommends that this new algorithm should be preferred to the scrambled Halton sequences when dealing with high correlation between standard Halton sequences.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by European Regional Science Association in its series ERSA conference papers with number ersa03p406.
Date of creation: Aug 2003
Date of revision:
Contact details of provider:
Postal: Welthandelsplatz 1, 1020 Vienna, Austria
Web page: http://www.ersa.org
This paper has been announced in the following NEP Reports:
- NEP-ALL-2004-02-29 (All new papers)
- NEP-DCM-2004-02-29 (Discrete Choice Models)
- NEP-ECM-2004-03-03 (Econometrics)
- NEP-GEO-2004-02-29 (Economic Geography)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Train,Kenneth E., 2009.
"Discrete Choice Methods with Simulation,"
Cambridge University Press, number 9780521766555, October.
- Lesley Chiou & Joan Walker, 2005. "Masking Identification of Discrete Choice Models under Simulation Methods," Occidental Economics Working Papers 5, Occidental College, Department of Economics, revised May 2006.
- Danny Campbell & W. George Hutchinson & Riccardo Scarpa, 2006. "Using Discrete Choice Experiments to Derive Individual-Specific WTP Estimates for Landscape Improvements under Agri-Environmental Schemes: Evidence from the Rural Environment Protection Scheme in Irel," Working Papers 2006.26, Fondazione Eni Enrico Mattei.
- Lorenzo Cappellari & Stephen P. Jenkins, 2006.
"Calculation of multivariate normal probabilities by simulation, with applications to maximum simulated likelihood estimation,"
StataCorp LP, vol. 6(2), pages 156-189, June.
- Lorenzo Cappellari & Stephen P. Jenkins, 2006. "Calculation of Multivariate Normal Probabilities by Simulation, with Applications to Maximum Simulated Likelihood Estimation," Discussion Papers of DIW Berlin 584, DIW Berlin, German Institute for Economic Research.
- Cappellari, Lorenzo & Jenkins, Stephen P., 2006. "Calculation of multivariate normal probabilities by simulation, with applications to maximum simulated likelihood estimation," ISER Working Paper Series 2006-16, Institute for Social and Economic Research.
- Cappellari, Lorenzo & Jenkins, Stephen P., 2006. "Calculation of Multivariate Normal Probabilities by Simulation, with Applications to Maximum Simulated Likelihood Estimation," IZA Discussion Papers 2112, Institute for the Study of Labor (IZA).
- Danny Campbell & George Hutchinson & Riccardo Scarpa, 2006.
"Lexicographic preferences for rural environmental landscape improvements: implications on individual-specific willingness to pay estimates,"
0610, Rural Economy and Development Programme,Teagasc.
- Danny Campbell & W. George Hutchinson & Riccardo Scarpa, 2006. "Lexicographic Preferences in Discrete Choice Experiments: Consequences on Individual-Specific Willingness to Pay Estimates," Working Papers 2006.128, Fondazione Eni Enrico Mattei.
- Danny Campbell & George Hutchinson & Riccardo Scarpa, 2006. "Using mixed logit models to derive individual-specific WTP estimates for landscape improvements under agri-environmental schemes: evidence from the Rural Environment Protection Scheme in Ireland," Working Papers 0607, Rural Economy and Development Programme,Teagasc.
- Chiou, Lesley & Walker, Joan L., 2007. "Masking identification of discrete choice models under simulation methods," Journal of Econometrics, Elsevier, vol. 141(2), pages 683-703, December.
- Stephane Hess & John W. Polak, 2004. "An analysis of parking behaviour using discrete choice models calibrated on SP datasets," ERSA conference papers ersa04p60, European Regional Science Association.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Gunther Maier).
If references are entirely missing, you can add them using this form.