IDEAS home Printed from https://ideas.repec.org/p/ter/wpaper/00197.html

Evaluating Creative Work with Artificial Intelligence: Evidence from Constrained Innovation Tasks

Author

Listed:
  • Valerio Fedele Addis
  • Giuseppe Attanasi
  • Giovanni Di Bartolomeo
  • Michele Mariella
  • Valentina Peruzzi

Abstract

We study whether a large language model can reliably evaluate human creativity in constrained, innovation-like tasks. Using expert-generated creative outputs from a validated experiment with workers in cultural and creative industries, we embed ChatGPT as an evaluator and benchmark its assessments against expert human judgments obtained through the Consensual Assessment Technique. In Study 1, we show that AI-based creativity evaluations exhibit internal consistency comparable to that of expert judges across repeated and independent runs, even under conservative scenarios. Replacing a human judge with an AI evaluator does not reduce inter-rater reliability across drawing, mathematical, and verbal tasks. In Study 2, we find that AI evaluations are systematically structured along fluency, flexibility, originality, and elaboration, with task-specific weighting of these dimensions. Overall, the results indicate that AI can serve as a reliable and structured evaluator of creativity in constrained innovation environments.

Suggested Citation

  • Valerio Fedele Addis & Giuseppe Attanasi & Giovanni Di Bartolomeo & Michele Mariella & Valentina Peruzzi, 2026. "Evaluating Creative Work with Artificial Intelligence: Evidence from Constrained Innovation Tasks," CIMEO Working Paper Series 197, Centre for Investigation and Modelling of Experimental Observations (CIMEO).
    • Handle: RePEc:ter:wpaper:00197
    as

    Download full text from publisher

    File URL: https://www.dipecodir.it/wpcom/data/wp_no_197_2026.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joseph Lampel & Theresa Lant & Jamal Shamsie, 2000. "Balancing Act: Learning from Organizing Practices in Cultural Industries," Organization Science, INFORMS, vol. 11(3), pages 263-269, June.
    2. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    3. Roberts, Deborah L. & Candi, Marina, 2024. "Artificial intelligence and innovation management: Charting the evolving landscape," Technovation, Elsevier, vol. 136(C).
    4. Dan Ariely & Uri Gneezy & George Loewenstein & Nina Mazar, 2009. "Large Stakes and Big Mistakes," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 451-469.
    5. Hoegl, Martin & Gibbert, Michael & Mazursky, David, 2008. "Financial constraints in innovation projects: When is less more?," Research Policy, Elsevier, vol. 37(8), pages 1382-1391, September.
    6. Laura Abrardi & Carlo Cambini & Laura Rondi, 2022. "Artificial intelligence, firms and consumer behavior: A survey," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 969-991, September.
    7. Hanna Hottenrott & Bettina Peters, 2012. "Innovative Capability and Financing Constraints for Innovation: More Money, More Innovation?," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1126-1142, November.
    8. Attanasi, Giuseppe & Curci, Ylenia & Llerena, Patrick & Urso, Giulia, 2026. "Intrinsic vs. extrinsic motivators on creative collaboration: The effect of sharing rewards," Journal of Economic Behavior & Organization, Elsevier, vol. 245(C).
    9. Uri Gneezy & Katharina Laske & Marina Schröder, 2021. "Creative solutions: Expertise versus Crowd Sourcing," Economics Bulletin, AccessEcon, vol. 41(4), pages 2580-2586.
    10. Nola Hewitt-Dundas, 2006. "Resource and Capability Constraints to Innovation in Small and Large Plants," Small Business Economics, Springer, vol. 26(3), pages 257-277, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giuseppe Attanasi & Michela Chessa & Sara Gil-Gallen & Patrick Llerena, 2021. "A survey on experimental elicitation of creativity in economics," Revue d'économie industrielle, De Boeck Université, vol. 0(2), pages 273-324.
    2. Liu, Duan & Li, Zhiyuan & He, Hongbo & Hou, Wenxuan, 2021. "The determinants of R&D smoothing with asset sales: Evidence from R&D-intensive firms in China," International Review of Economics & Finance, Elsevier, vol. 75(C), pages 76-93.
    3. Abdin, Joynal & Sharma, Abhijit & Trivedi, Rohit & Wang, Chengang, 2024. "Financing constraints, intellectual property rights protection and incremental innovation: Evidence from transition economy firms," Technological Forecasting and Social Change, Elsevier, vol. 198(C).
    4. Giuseppe Attanasi & Michela Chessa & Carlo Ciucani & Sara Gil Gallen, 2020. "Children's GrI-Creativity: Effects of Limited Resources in Creative Drawing," GREDEG Working Papers 2020-34, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    5. Runsen Yuan & Chunling Li & Xinjie Cao & Nian Li & Nosherwan Khaliq, 2022. "Research on the Influence of Mixed-Ownership Reform on Exploratory Innovation of SOEs: The Mediation Effect of Agency Conflict and Financing Constraint," SAGE Open, , vol. 12(2), pages 21582440221, May.
    6. García-Quevedo, José & Segarra-Blasco, Agustí & Teruel, Mercedes, 2018. "Financial constraints and the failure of innovation projects," Technological Forecasting and Social Change, Elsevier, vol. 127(C), pages 127-140.
    7. Maurizio Pugno, 2024. "Creativity, well-being, and economic development: An evolutionary approach," Journal of Evolutionary Economics, Springer, vol. 34(1), pages 205-225, January.
    8. Heller, David, 2024. "Financial market integration and the effects of financing constraints on innovation," Research Policy, Elsevier, vol. 53(4).
    9. Lovász, Anna & Cukrowska-Torzewska, Ewa & Rigó, Mariann & Szabó-Morvai, Ágnes, 2022. "Gender differences in the effect of subjective feedback in an online game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    10. Kyle Myers & Wei Yang Tham, 2023. "Money, Time, and Grant Design," Papers 2312.06479, arXiv.org.
    11. Guangyuan Lu & Xiong Bai & Xiaoyun Zhang, 2025. "How Does Financial Support Affect Firms’ Innovation and Total Factor Productivity: A Quasi-Natural Experiment in China," Sustainability, MDPI, vol. 17(1), pages 1-33, January.
    12. Juliao-Rossi, Jorge & Forero-Pineda, Clemente & Losada-Otalora, Mauricio & Peña-García, Nathalie, 2020. "Trajectories of innovation: A new approach to studying innovation performance," Journal of Business Research, Elsevier, vol. 115(C), pages 322-333.
    13. Krzyzanowski, Jan & Walz, Uwe, 2021. "Bank regulation, lending and patenting: Evidence from the EBA capital exercise," SAFE Working Paper Series 330, Leibniz Institute for Financial Research SAFE.
    14. Susanne Hügel & Markus Kreutzer, 2019. "The Impact Of Organisational Slack On Innovative Work Behaviour: How Do Top Managers And Employees Differ?," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 24(03), pages 1-44, April.
    15. Keupp, Marcus Matthias & Gassmann, Oliver, 2013. "Resource constraints as triggers of radical innovation: Longitudinal evidence from the manufacturing sector," Research Policy, Elsevier, vol. 42(8), pages 1457-1468.
    16. Attanasi, Giuseppe & Chessa, Michela & Ciucani, Carlo, 2026. "Creative drawings and treatment conditions: gender differences in children," SocArXiv 5ejdc_v1, Center for Open Science.
    17. Amankwah-Amoah, Joseph & Hinson, Robert E., 2019. "Contextual influences on new technology ventures: A study of domestic firms in Ghana," Technological Forecasting and Social Change, Elsevier, vol. 143(C), pages 289-296.
    18. Giuseppe Attanasi & Ylenia Curci & Patrick Llerena & Maria del Pino Ramos-Sosa & Adriana Carolina Pinate & Giulia Urso, 2019. "Looking at Creativity from East to West: Risk Taking and Intrinsic Motivation in Socially and Culturally Diverse Countries," Working Papers of BETA 2019-38, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    19. Shaikh, Ibrahim A. & O'Brien, Jonathan Paul & Peters, Lois, 2018. "Inside directors and the underinvestment of financial slack towards R&D-intensity in high-technology firms," Journal of Business Research, Elsevier, vol. 82(C), pages 192-201.
    20. Radu Cezara-Georgiana, 2025. "AI and Human Bias: The Future of Jobs in the Labor Market," Proceedings of the International Conference on Business Excellence, Sciendo, vol. 19(1), pages 3423-3431.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • M14 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - Corporate Culture; Diversity; Social Responsibility
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ter:wpaper:00197. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Giovanni Di Bartolomeo (email available below). General contact details of provider: https://edirc.repec.org/data/cmrosit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.