IDEAS home Printed from https://ideas.repec.org/p/ter/wpaper/00197.html

Evaluating Creative Work with Artificial Intelligence: Evidence from Constrained Innovation Tasks

Author

Listed:
  • Valerio Fedele Addis
  • Giuseppe Attanasi
  • Giovanni Di Bartolomeo
  • Michele Mariella
  • Valentina Peruzzi

Abstract

We study whether a large language model can reliably evaluate human creativity in constrained, innovation-like tasks. Using expert-generated creative outputs from a validated experiment with workers in cultural and creative industries, we embed ChatGPT as an evaluator and benchmark its assessments against expert human judgments obtained through the Consensual Assessment Technique. In Study 1, we show that AI-based creativity evaluations exhibit internal consistency comparable to that of expert judges across repeated and independent runs, even under conservative scenarios. Replacing a human judge with an AI evaluator does not reduce inter-rater reliability across drawing, mathematical, and verbal tasks. In Study 2, we find that AI evaluations are systematically structured along fluency, flexibility, originality, and elaboration, with task-specific weighting of these dimensions. Overall, the results indicate that AI can serve as a reliable and structured evaluator of creativity in constrained innovation environments.

Suggested Citation

  • Valerio Fedele Addis & Giuseppe Attanasi & Giovanni Di Bartolomeo & Michele Mariella & Valentina Peruzzi, 2026. "Evaluating Creative Work with Artificial Intelligence: Evidence from Constrained Innovation Tasks," CIMEO Working Paper Series 197, Centre for Investigation and Modelling of Experimental Observations (CIMEO).
    • Handle: RePEc:ter:wpaper:00197
    as

    Download full text from publisher

    File URL: https://www.dipecodir.it/wpcom/data/wp_no_197_2026.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Giuseppe Attanasi & Ylenia Curci & Patrick Llerena & Maria del Pino Ramos-Sosa & Adriana Carolina Pinate & Giulia Urso, 2019. "Looking at Creativity from East to West: Risk Taking and Intrinsic Motivation in Socially and Culturally Diverse Countries," Working Papers of BETA 2019-38, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    2. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    3. Roberts, Deborah L. & Candi, Marina, 2024. "Artificial intelligence and innovation management: Charting the evolving landscape," Technovation, Elsevier, vol. 136(C).
    4. Giuseppe Attanasi & Michela Chessa & Sara Gil-Gallen & Patrick Llerena, 2021. "A survey on experimental elicitation of creativity in economics," Revue d'économie industrielle, De Boeck Université, vol. 0(2), pages 273-324.
    5. Hoegl, Martin & Gibbert, Michael & Mazursky, David, 2008. "Financial constraints in innovation projects: When is less more?," Research Policy, Elsevier, vol. 37(8), pages 1382-1391, September.
    6. Gary Charness & Daniela Grieco, 2023. "Creativity and Corporate Culture," The Economic Journal, Royal Economic Society, vol. 133(653), pages 1846-1870.
    7. Gary Charness & Daniela Grieco, 2019. "Creativity and Incentives," Journal of the European Economic Association, European Economic Association, vol. 17(2), pages 454-496.
    8. Laura Abrardi & Carlo Cambini & Laura Rondi, 2022. "Artificial intelligence, firms and consumer behavior: A survey," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 969-991, September.
    9. Murro, Pierluigi & Peruzzi, Valentina, 2026. "Credit constraints and open innovation strategies," Journal of Economic Behavior & Organization, Elsevier, vol. 245(C).
    10. Uri Gneezy & Katharina Laske & Marina Schröder, 2021. "Creative solutions: Expertise versus Crowd Sourcing," Economics Bulletin, AccessEcon, vol. 41(4), pages 2580-2586.
    11. Nola Hewitt-Dundas, 2006. "Resource and Capability Constraints to Innovation in Small and Large Plants," Small Business Economics, Springer, vol. 26(3), pages 257-277, April.
    12. Joseph Lampel & Theresa Lant & Jamal Shamsie, 2000. "Balancing Act: Learning from Organizing Practices in Cultural Industries," Organization Science, INFORMS, vol. 11(3), pages 263-269, June.
    13. Dan Ariely & Uri Gneezy & George Loewenstein & Nina Mazar, 2009. "Large Stakes and Big Mistakes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 451-469.
    14. Charness, Gary & Grieco, Daniela, 2022. "Creativity and ambiguity tolerance," Economics Letters, Elsevier, vol. 218(C).
    15. Hanna Hottenrott & Bettina Peters, 2012. "Innovative Capability and Financing Constraints for Innovation: More Money, More Innovation?," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1126-1142, November.
    16. Attanasi, Giuseppe & Curci, Ylenia & Llerena, Patrick & Urso, Giulia, 2026. "Intrinsic vs. extrinsic motivators on creative collaboration: The effect of sharing rewards," Journal of Economic Behavior & Organization, Elsevier, vol. 245(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ciccarone, Giuseppe & Di Bartolomeo, Giovanni & Peruzzi, Valentina & Signore, Maria Luigia, 2026. "Creativity meets social capital: Theory and field evidence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 124(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Attanasi, Giuseppe & Curci, Ylenia & Llerena, Patrick & Urso, Giulia, 2026. "Intrinsic vs. extrinsic motivators on creative collaboration: The effect of sharing rewards," Journal of Economic Behavior & Organization, Elsevier, vol. 245(C).
    2. Giuseppe Attanasi & Michela Chessa & Carlo Ciucani & Sara Gil Gallen, 2020. "Children's GrI-Creativity: Effects of Limited Resources in Creative Drawing," GREDEG Working Papers 2020-34, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    3. Prissé, Benjamin & Deng, Ruotong & Ho, Jun Quan & Jayasekara, Dinithi & Ouangraoua, Chris, 2026. "The creativity of artificial intelligence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 121(C).
    4. Attanasi, Giuseppe & Chessa, Michela & Ciucani, Carlo, 2026. "Creative drawings and treatment conditions: gender differences in children," SocArXiv 5ejdc_v1, Center for Open Science.
    5. Giuseppe Attanasi & Michela Chessa & Sara Gil-Gallen & Patrick Llerena, 2021. "A survey on experimental elicitation of creativity in economics," Revue d'économie industrielle, De Boeck Université, vol. 0(2), pages 273-324.
    6. Michela Chessa & Benjamin Prissé, 2024. "The Evaluation of Creativity," GREDEG Working Papers 2024-16, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    7. Grieco, Daniela & Llerena, Patrick & Maltese, Anne-Gaëlle, 2026. "Creativity and task perception," Journal of Economic Behavior & Organization, Elsevier, vol. 241(C).
    8. repec:beh:jbepv1:v:7:y:2023:i:2:p:21-32 is not listed on IDEAS
    9. Erik Sarrazin, 2026. "Incentives and Creativity in Groups — Experimental Evidence on Creative Processes and Dimensions," Working Papers 2601, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 21 May 2026.
    10. Maria-Magdalena Rosu & Rodica Ianole-Călin, 2024. "Are we there yet? Resilience training as a catalyst for student creativity," Journal of Behavioral Economics for Policy, Society for the Advancement of Behavioral Economics (SABE), vol. 8(S2), pages 51-60, December.
    11. Giuseppe Attanasi & Giuseppe Ciccarone & Giovanni Di Bartolomeo, 2026. "The Bright Side of Imprecision: Creativity Meets Rational Inattention," Working Papers in Public Economics 268, Department of Economics and Law, Sapienza University of Rome.
    12. Rodet, Cortney S., 2026. "Assessing the interaction of empathy and diversity in creative teams," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 121(C).
    13. Anne-Gaëlle Maltese & Sara Gil-Gallen & Patrick Llerena, 2023. "Disentangling the role of surface and deep-level variables on individuals’ and groups’ creative performance: A cross-level experimental evidence," Working Papers of BETA 2023-19, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    14. Blaufus, Kay & Piehl, Kevin & Schröder, Marina, 2026. "Input versus output incentives in idea generation—An experimental analysis," European Economic Review, Elsevier, vol. 186(C).
    15. Ciccarone, Giuseppe & Di Bartolomeo, Giovanni & Peruzzi, Valentina & Signore, Maria Luigia, 2026. "Creativity meets social capital: Theory and field evidence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 124(C).
    16. Giuseppe Attanasi & Ylenia Curci & Patrick Llerena & Maria del Pino Ramos-Sosa & Adriana Carolina Pinate & Giulia Urso, 2019. "Looking at Creativity from East to West: Risk Taking and Intrinsic Motivation in Socially and Culturally Diverse Countries," Working Papers of BETA 2019-38, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    17. Katharina Laske & Nathalie Römer & Marina Schröder, 2024. "Piece-Rate Incentives and Idea Generation – An Experimental Analysis," CESifo Working Paper Series 11594, CESifo.
    18. Tao, Xionghua & Yu, Chen, 2026. "Convertible bonds and corporate innovation: incentive or resistance?," The North American Journal of Economics and Finance, Elsevier, vol. 85(C).
    19. Glenn Dutcher & Cortney S. Rodet, 2022. "Which two heads are better than one? Uncovering the positive effects of diversity in creative teams," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 31(4), pages 884-897, November.
    20. Laske, Katharina & Schröder, Marina, 2017. "Quantity, Quality and Originality: The Effects of Incentives on Creativity," VfS Annual Conference 2017 (Vienna): Alternative Structures for Money and Banking 168151, Verein für Socialpolitik / German Economic Association.
    21. Noémi Berlin & Jan Dul & Marco Gazel & Louis Lévy-Garboua & Todd Lubart, 2023. "Creative Cognition as a Bandit Problem," Documents de travail du Centre d'Economie de la Sorbonne 23002, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • M14 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - Corporate Culture; Diversity; Social Responsibility
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ter:wpaper:00197. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Giovanni Di Bartolomeo (email available below). General contact details of provider: https://edirc.repec.org/data/cmrosit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.