Critical thinking and artificial intelligence in academia: A qualitative matrix analysis procedure for evaluating AI systems

Abstract

This work introduces the Matrix AI Systems Analysis Procedure (MASIA), a qualitative, matrix-based method designed to evaluate the performance and quality of generative artificial intelligence systems within academic settings. MASIA centers on the analysis of three key components in AI-generated responses: narrative synthesis, source usage, and the formulation of new prompts. By doing so, it fosters critical thinking among users and offers valuable tools for both teaching and research.The procedure defines variables and analytical parameters that enable the comparison of different AI systems, thereby supporting informed decision-making in scholarly and research environments. Furthermore, MASIA integrates ethical considerations, including traceability, proper attribution, and plagiarism prevention, making it a flexible instrument adaptable to various academic needs and projects. The chapter concludes that MASIA is a straightforward yet powerful tool for enhancing critical thinking, improving teaching and learning processes, and providing a foundation for comparative research on artificial intelligence in academia.