IDEAS home Printed from https://ideas.repec.org/p/vnm/wpdman/40.html
   My bibliography  Save this paper

Pack light on the move: Exploitation and exploration in a dynamic environment

Author

Listed:
  • Marco LiCalzi

    (Department of Management, Università Ca' Foscari Venezia)

  • Davide Marchiori

    (Department of Management, Università Ca' Foscari Venezia)

Abstract

This paper revisits a recent study by Posen and Levinthal (2012) on the exploration/exploitation tradeoff for a multi-armed bandit problem, where the reward probabilities undergo random shocks. We show that their analysis suffers two shortcomings: it assumes that learning is based on stale evidence, and it overlooks the steady state. We let the learning rule endogenously discard stale evidence, and we perform the long run analyses. The comparative study demonstrates that some of their conclusions must be qualified.

Suggested Citation

  • Marco LiCalzi & Davide Marchiori, 2013. "Pack light on the move: Exploitation and exploration in a dynamic environment," Working Papers 4, Department of Management, Università Ca' Foscari Venezia.
    • Handle: RePEc:vnm:wpdman:40
    as

    Download full text from publisher

    File URL: http://virgo.unive.it/wpideas/storage/2013wp4.pdf
    File Function: First version, 2013
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Leigh Tesfatsion & Kenneth L. Judd (ed.), 2006. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 2, number 2.
    2. Duffy, John, 2006. "Agent-Based Models and Human Subject Experiments," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 19, pages 949-1011, Elsevier.
    3. James G. March, 1991. "Exploration and Exploitation in Organizational Learning," Organization Science, INFORMS, vol. 2(1), pages 71-87, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Friederike Wall, 2016. "Agent-based modeling in managerial science: an illustrative survey and study," Review of Managerial Science, Springer, vol. 10(1), pages 135-193, January.
    2. Yoo, Seung Han, 2014. "Learning a population distribution," Journal of Economic Dynamics and Control, Elsevier, vol. 48(C), pages 188-201.
    3. Pascal Seppecher & Isabelle Salle & Dany Lang, 2019. "Is the market really a good teacher?," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 299-335, March.
    4. Moulet, Sonia & Rouchier, Juliette, 2008. "The influence of seller learning and time constraints on sequential bargaining in an artificial perishable goods market," Journal of Economic Dynamics and Control, Elsevier, vol. 32(7), pages 2322-2348, July.
    5. Giuseppe Attanasi & Samuele Centorrino & Ivan Moscati, 2011. "Double Auction Equilibrium and Efficiency in a Classroom Experimental Search Market," LERNA Working Papers 11.03.337, LERNA, University of Toulouse.
    6. Michael Pickhardt, "undated". "A few can do – Ethical behavior and the provision of public goods in an agent-based model," Working Papers 201037, Institute of Spatial and Housing Economics, Munster Universitary.
    7. Michael Neugart & Matteo G. Richiardi, 2012. "Agent-based models of the labor market," LABORatorio R. Revelli Working Papers Series 125, LABORatorio R. Revelli, Centre for Employment Studies.
    8. Olivier Brandouy & Angelo Corelli & Iryna Veryzhenko & Roger Waldeck, 2012. "A re-examination of the “zero is enough” hypothesis in the emergence of financial stylized facts," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 7(2), pages 223-248, October.
    9. Mikhail Anufriev & Jasmina Arifovic & John Ledyard & Valentyn Panchenko, 2013. "Efficiency of continuous double auctions under individual evolutionary learning with full or limited information," Journal of Evolutionary Economics, Springer, vol. 23(3), pages 539-573, July.
    10. Flaminio Squazzoni, 2010. "The impact of agent-based models in the social sciences after 15 years of incursions," History of Economic Ideas, Fabrizio Serra Editore, Pisa - Roma, vol. 18(2), pages 197-234.
    11. Shu‐Heng Chen & Shu G. Wang, 2011. "Emergent Complexity In Agent‐Based Computational Economics," Journal of Economic Surveys, Wiley Blackwell, vol. 25(3), pages 527-546, July.
    12. Oliver Baumann, 2015. "Models of complex adaptive systems in strategy and organization research," Mind & Society: Cognitive Studies in Economics and Social Sciences, Springer;Fondazione Rosselli, vol. 14(2), pages 169-183, November.
    13. Hommes, Cars H., 2006. "Heterogeneous Agent Models in Economics and Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186, Elsevier.
    14. Shu-Heng Chen & Umberto Gostoli, 2017. "Coordination in the El Farol Bar problem: The role of social preferences and social networks," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 12(1), pages 59-93, April.
    15. Hommes, Cars, 2011. "The heterogeneous expectations hypothesis: Some evidence from the lab," Journal of Economic Dynamics and Control, Elsevier, vol. 35(1), pages 1-24, January.
    16. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    17. repec:wyi:journl:002151 is not listed on IDEAS
    18. Frank M. A. Klingert & Matthias Meyer, 2012. "Effectively combining experimental economics and multi-agent simulation: suggestions for a procedural integration with an example from prediction markets research," Computational and Mathematical Organization Theory, Springer, vol. 18(1), pages 63-90, March.
    19. Narine Udumyan & Juliette Rouchier & Dominique Ami, 2014. "Integration of Path-Dependency in a Simple Learning Model: The Case of Marine Resources," Computational Economics, Springer;Society for Computational Economics, vol. 43(2), pages 199-231, February.
    20. Desmarchelier, Benoît & Regis, Paulo José & Salike, Nimesh, 2018. "Product space and the development of nations: A model of product diversification," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 34-51.
    21. Julissa Alexandra Galarza-Villamar & Mariette McCampbell & Cees Leeuwis & Francesco Cecchi, 2021. "Adding Emergence and Spatiality to a Public Bad Game for Studying Dynamics in Socio-Ecological Systems (Part I): The Design of Musa-Game for Integrative Analysis of Collective Action in Banana Disease," Sustainability, MDPI, vol. 13(16), pages 1-21, August.

    More about this item

    Keywords

    adaptation; learning; turbulence; multi-armed bandit problem;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vnm:wpdman:40. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Marco LiCalzi (email available below). General contact details of provider: https://edirc.repec.org/data/mdvenit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.