A web-based self-assessment model for evaluating multidisiciplinary cancer teams in Spain: development and validation pilot study

Abstract

Background: Tumor boards constitute the main consensus and clinical decision-making body of multidisciplinary teams (MDTs) in cancer care. With the increasing clinical complexity of treatment options (eg, targeted therapies, multimodal treatments) and the progressive incorporation of new areas of intervention (eg, survivorship care), tumor boards are now required to play a central role in all cancer processes. However, although frameworks are in place to evaluate MDT quality, only few web-based tools are available for this purpose; indeed, no web-based MDT evaluation tools have been developed for or adapted to the Spanish National Health System. Objective: The first aim of this study was to develop a web-based self-assessment model (Autoevaluación de Equipos Multidisciplinares de Atención al Cáncer [AEMAC]) for evaluating multidisciplinary cancer teams in Spain and the second aim was to validate this tool by testing its metric properties, acceptability, and usability. Methods: We designed and validated the AEMAC program in 3 stages. In the first stage (research), we reviewed the available scientific evidence and performed a qualitative case study of good practice in multidisciplinary care within the Spanish National Health System (n=4 centers and 28 health care professionals). The results were used to define the thematic areas and quality criteria for the self-evaluation model, which were then discussed and validated by a group of experts. The second stage (development) involved the technological development of a web app that would be accessible from any mobile device. In the third stage (piloting and validation), we conducted 4 pilot tests (n=15 tumor boards, 243 professionals) and used the results to analyze the acceptability and usefulness of the tool. Results: We designed a self-assessment model based on 5 thematic areas encompassing a total of 25 quality components, which users rated on a 3-option development scale. The evaluation process, which was managed entirely from the web app, consisted of individual self-assessment, group prioritization, and creation of an improvement plan. Cronbach alpha (.86), McDonald's omega (0.88), and various fit indices (comparative fit index between 0.95 and 1 and goodness-of-fit index between 0.97 and 0.99 for all 5 aspects) confirmed internal consistency. The mean rating for overall satisfaction with the tool and for consistency between the content of the tool and the reality of tumor boards was 7.6 out of 10. Conclusions: The results obtained during the period of research and piloting of the AEMAC program showed that it has an appropriate structure and metric properties and could therefore be implemented in a real context and generalized to other hospitals. As a virtual tool, it helps to measure the key aspects of MDT quality, such as effectiveness of collaboration and communication, leadership, and the organizational environment.