IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v144y2020ics0167947319302051.html
   My bibliography  Save this article

Approximate filtering of conditional intensity process for Poisson count data: Application to urban crime

Author

Listed:
  • Santitissadeekorn, Naratip
  • Lloyd, David J.B.
  • Short, Martin B.
  • Delahaies, Sylvain

Abstract

The primary focus is a sequential data assimilation method for count data modelled by an inhomogeneous Poisson process. In particular, a quadratic approximation technique similar to the extended Kalman filter is applied to develop a sub-optimal, discrete-time, filtering algorithm, called the extended Poisson–Kalman filter (ExPKF), where only the mean and covariance are sequentially updated using count data via the Poisson likelihood function. The performance of ExPKF is investigated in several synthetic experiments where the true solution is known. In numerical examples, ExPKF provides a good estimate of the “true” posterior mean, which can be well-approximated by the particle filter (PF) algorithm in the very large sample size limit. In addition, the experiments demonstrate that the ExPKF algorithm can be conveniently used to track parameter changes; on the other hand, a non-filtering framework such as a maximum likelihood estimation (MLE) would require a statistical test for change points or implement time-varying parameters. Finally, to demonstrate the model on real-world data, the ExPKF is used to approximate the uncertainty of urban crime intensity and parameters for self-exciting crime models. The Chicago Police Department’s CLEAR (Citizen Law Enforcement Analysis and Reporting) system data is used as a case study for both univariate and multivariate Hawkes models. An improved goodness of fit measured by the Kolmogorov–Smirnov (KS) statistics is achieved by the filtered intensity. The potential of using filtered intensity to improve police patrolling prioritisation is also tested. By comparing with the prioritisation based on MLE-derived intensity and historical frequency, the result suggests an insignificant difference between them. While the filter is developed and tested in the context of urban crime, it has the potential to make a contribution to data assimilation in other application areas.

Suggested Citation

  • Santitissadeekorn, Naratip & Lloyd, David J.B. & Short, Martin B. & Delahaies, Sylvain, 2020. "Approximate filtering of conditional intensity process for Poisson count data: Application to urban crime," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    • Handle: RePEc:eee:csdana:v:144:y:2020:i:c:s0167947319302051
    DOI: 10.1016/j.csda.2019.106850
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947319302051
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2019.106850?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. D. A. Stephens, 1994. "Bayesian Retrospective Multiple‐Changepoint Identification," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 43(1), pages 159-178, March.
    2. Jakob Gulddahl Rasmussen, 2013. "Bayesian Inference for Hawkes Processes," Methodology and Computing in Applied Probability, Springer, vol. 15(3), pages 623-642, September.
    3. Dassios, Angelos & Zhao, Hongbiao, 2013. "Exact simulation of Hawkes process with exponentially decaying intensity," LSE Research Online Documents on Economics 51370, London School of Economics and Political Science, LSE Library.
    4. Jamie Olson & Kathleen Carley, 2013. "Exact and approximate EM estimation of mutually exciting hawkes processes," Statistical Inference for Stochastic Processes, Springer, vol. 16(1), pages 63-80, April.
    5. Mohler, G. O. & Short, M. B. & Brantingham, P. J. & Schoenberg, F. P. & Tita, G. E., 2011. "Self-Exciting Point Process Modeling of Crime," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 100-108.
    6. Steffen Unkel & C. Paddy Farrington & Paul H. Garthwaite & Chris Robertson & Nick Andrews, 2012. "Statistical methods for the prospective detection of infectious disease outbreaks: a review," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(1), pages 49-82, January.
    7. Eric W. Fox & Martin B. Short & Frederic P. Schoenberg & Kathryn D. Coronges & Andrea L. Bertozzi, 2016. "Modeling E-mail Networks and Inferring Leadership Using Self-Exciting Point Processes," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(514), pages 564-584, April.
    8. G. O. Mohler & M. B. Short & Sean Malinowski & Mark Johnson & G. E. Tita & Andrea L. Bertozzi & P. J. Brantingham, 2015. "Randomized Controlled Field Trials of Predictive Policing," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1399-1411, December.
    9. Santitissadeekorn, N. & Short, M.B. & Lloyd, D.J.B., 2018. "Sequential data assimilation for 1D self-exciting processes with application to urban crime data," Computational Statistics & Data Analysis, Elsevier, vol. 128(C), pages 163-183.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cavaliere, Giuseppe & Lu, Ye & Rahbek, Anders & Stærk-Østergaard, Jacob, 2023. "Bootstrap inference for Hawkes and general point processes," Journal of Econometrics, Elsevier, vol. 235(1), pages 133-165.
    2. Chenlong Li & Zhanjie Song & Wenjun Wang, 2020. "Space–time inhomogeneous background intensity estimators for semi-parametric space–time self-exciting point process models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(4), pages 945-967, August.
    3. Mohammed A. A. Al-qaness & Ahmed A. Ewees & Hong Fan & Mohamed Abd Elaziz, 2020. "Optimized Forecasting Method for Weekly Influenza Confirmed Cases," IJERPH, MDPI, vol. 17(10), pages 1-12, May.
    4. Philip A. White & Alan E. Gelfand, 2021. "Generalized Evolutionary Point Processes: Model Specifications and Model Comparison," Methodology and Computing in Applied Probability, Springer, vol. 23(3), pages 1001-1021, September.
    5. Santitissadeekorn, N. & Short, M.B. & Lloyd, D.J.B., 2018. "Sequential data assimilation for 1D self-exciting processes with application to urban crime data," Computational Statistics & Data Analysis, Elsevier, vol. 128(C), pages 163-183.
    6. Liu, Chenguang, 2020. "Statistical inference for a partially observed interacting system of Hawkes processes," Stochastic Processes and their Applications, Elsevier, vol. 130(9), pages 5636-5694.
    7. Haberman, Cory P. & Hatten, David & Carter, Jeremy G. & Piza, Eric L., 2021. "The sensitivity of repeat and near repeat analysis to geocoding algorithms," Journal of Criminal Justice, Elsevier, vol. 73(C).
    8. Carter, Jeremy G. & Mohler, George & Raje, Rajeev & Chowdhury, Nahida & Pandey, Saurabh, 2021. "The Indianapolis harmspot policing experiment," Journal of Criminal Justice, Elsevier, vol. 74(C).
    9. Rummens, Anneleen & Hardyns, Wim, 2021. "The effect of spatiotemporal resolution on predictive policing model performance," International Journal of Forecasting, Elsevier, vol. 37(1), pages 125-133.
    10. Mohler, George & Carter, Jeremy & Raje, Rajeev, 2018. "Improving social harm indices with a modulated Hawkes process," International Journal of Forecasting, Elsevier, vol. 34(3), pages 431-439.
    11. Baichuan Yuan & Frederic P. Schoenberg & Andrea L. Bertozzi, 2021. "Fast estimation of multivariate spatiotemporal Hawkes processes and network reconstruction," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(6), pages 1127-1152, December.
    12. Alex Reinhart & Joel Greenhouse, 2018. "Self‐exciting point processes with spatial covariates: modelling the dynamics of crime," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 67(5), pages 1305-1329, November.
    13. Alsenafi, Abdulaziz & Barbaro, Alethea B.T., 2018. "A convection–diffusion model for gang territoriality," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 510(C), pages 765-786.
    14. Doyo G Enki & Paul H Garthwaite & C Paddy Farrington & Angela Noufaily & Nick J Andrews & Andre Charlett, 2016. "Comparison of Statistical Algorithms for the Detection of Infectious Disease Outbreaks in Large Multiple Surveillance Systems," PLOS ONE, Public Library of Science, vol. 11(8), pages 1-25, August.
    15. Emmanuel Bacry & Jean-Francois Muzy, 2014. "Second order statistics characterization of Hawkes processes and non-parametric estimation," Papers 1401.0903, arXiv.org, revised Feb 2015.
    16. Anatoliy Swishchuk & Aiden Huffman, 2020. "General Compound Hawkes Processes in Limit Order Books," Risks, MDPI, vol. 8(1), pages 1-25, March.
    17. Jens Ludwig & Sendhil Mullainathan, 2021. "Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System," Journal of Economic Perspectives, American Economic Association, vol. 35(4), pages 71-96, Fall.
    18. Dewei Wang & Chendi Jiang & Chanseok Park, 2019. "Reliability analysis of load-sharing systems with memory," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 25(2), pages 341-360, April.
    19. Xunjie Cheng & Tao Chen & Yang Yang & Jing Yang & Dayan Wang & Guoqing Hu & Yuelong Shu, 2018. "Using an innovative method to develop the threshold of seasonal influenza epidemic in China," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-13, August.
    20. Christin Schröder & Luis Alberto Peña Diaz & Anna Maria Rohde & Brar Piening & Seven Johannes Sam Aghdassi & Georg Pilarski & Norbert Thoma & Petra Gastmeier & Rasmus Leistner & Michael Behnke, 2020. "Lean back and wait for the alarm? Testing an automated alarm system for nosocomial outbreaks to provide support for infection control professionals," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-15, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:144:y:2020:i:c:s0167947319302051. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.