Grupo Alarcos · Journal article · 2026
Practical Insights Into AI System Product Quality Evaluation
Wallace Albertini, Jesús Ramón Oviedo Lama, Eduardo Motta, Julia Araujo, Marina Araujo, Moisés Rodríguez Monje, Helio Lopes, Kalinowski, M., Mario Piattini Velthuis
This study investigates the application of the ISO/IEC 25059 standard in the quality evaluation of artificial intelligence (AI) systems, focusing on functional suitability. Through a case study evaluating an AI system developed for analyzing images of oil platform decks, we discuss the practical nuances of evaluating functional correctness. We share information on specific properties defined to operationalize the evaluation. We emphasize the need to thoroughly discuss acceptable specification limits, especially in safety-critical contexts. Furthermore, the representativeness and sample size of the test cases should be systematically considered to reflect the profile of real-world operational conditions.