Document Type
Article
Publication Date
11-27-2025
Publisher
Springer Nature
Abstract
The Computational Creativity literature stresses the importance of evaluation in computational creativity systems in addition to generative capabilities. But, what about machines that only evaluate? When it comes to co-creative systems, humans often take on the primary evaluative role, while machines assist with the generation of creative artifacts. In this paper, we propose flipping the paradigm, envisioning machines that (only) evaluate humans in the creative space. We call such machines Creative Evaluators whose end goal is to evaluate human-made artifacts and provides feedback to aid humans in the creative process, while the machine itself refrains from directly generating any content. We present a co-creative framework for designing Creative Evaluators, which includes the tension between originality and quality of creative artifacts, and the need for explainability and fairness in machines that take on an evaluative role for human-made artifacts. To illustrate this framework, we present Titlevator, a machine that evaluates titles for AI papers, while also providing feedback to the user on the originality and the quality of the paper being evaluated.
Recommended Citation
Bossou, K., & Ackerman, M. (2025). Creative Evaluators: When Machines Evaluate Us. New Generation Computing, 44(1), 3. https://doi.org/10.1007/s00354-025-00302-8
Comments
Open access to this article is funded by Santa Clara University Library.
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.