TORCH D8.3 - Research Assessment Report

Abstract

This report outlines the impacts of implementing a connected RRI strategy on research assessment and provides recommendations on change evaluation. It focuses on SPA1 Working Towards reforming research assessment and more specifically, pays special attention to Rewards and Recognition as a central element in the reform of Research Assessment. It provides background information on the current environment of Research Assessment. It describes Developing a CHARM-EU Research Assessment system as linked with Recognition and Rewards at UU. It then discusses the core elements of a connected RRI in relation to Reform of research assessment, analysing its impacts on different levels and concludes with recommendations for change evaluation.

For CHARM-EU and its member universities, reforming research assessment is both a dynamic evolving activity and a prerequisite to advance our collaboration and bring our innovation ideas into practice. Whilst our alliance has entered a new phase with the start of CHARM-EIGHT, the research and innovation dimension addressed by TORCH is coming to its first major milestone with the conclusion of the project, giving us an opportunity to reflect on the work completed and its impact, both intended and unintended, as well as the ramifications for the future. The main objective of this report is to outline the impacts of implementing a connected RRI strategy on research assessment and provide recommendations on change evaluation. This report can contribute to different work packages in CHARM-EIGHT, such as those directed toward professional development and rewarding excellence in teaching or the work towards a transdisciplinary and team-oriented doctoral programme.

As described in D8.4, the second half of TORCH is characterized by an integrative approach focusing on the identification and development of five strategic priority areas (SPAs). One of these is Working Towards the Reform of Research Assessment, which includes rewards, recognition and human resource strategies in relation to research assessment. Given the timeframe in TORCH, the complexity of the Reform of Research Assessment and the establishment of a large coalition to prepare and share reform approaches and experiences, we decided not to pilot Research Assessment within the project timeline. Instead, this topic is addressed in a proposed action plan (see Description of AP1 in Deliverable 9.2) to be carried out at an appropriate time.

This report focuses on SPA1 Working Towards reforming research assessment - and more specifically, pays special attention to Rewards and Recognition as a central element in the reform of Research Assessment. Our work in previous work packages and deliverables (specifically D8.1) has identified the area of recruitment and promotion of staff as being one of the central hubs of interconnected RRI, which is at the core of new Rewards and Recognition practices. At this, our approach is primarily informed by the implementation of innovative practices at Utrecht University, one of the TORCH and CHARM partners, and the member institution with the largest amount of work already conducted on research assessment reform. As a Dutch university, UU is part of a national process to reform rewards and recognition processes. UU is actively advancing this field and is in a natural and logical position to share its experiences and expertise with the alliance partners. UU is also highlighted as an example of best practice in the LERU Position Paper on Research Assessment and by the authors of the mid-term review of TORCH.

We have found that our existing practices to recognize and nurture RRI practices are still fragmented, and existing recognition and reward structures overall still form a source of disincentives to engage with RRI. For research assessment this means we identify a clear need to screen and uncover where recognition and rewards structures in fact function as disincentives to RRI and what changes could be made to make these structures more suited to facilitate RRI. One aim of Research assessment should thus lie in actively encouraging RRI activities (such as by recognizing a broader set of contributions) by recognizing them as relevant activities for researchers and ideally by providing career paths for researchers who excel at such activities. Furthermore, our CHARM alliance could help provide a community of like-minded researchers with opportunities to exchange experiences. To help overcome the fragmentation of existing initiatives, a long-term vision and commitment to develop institutional support for transdisciplinary science and public engagement are needed. Research assessment reforms have stressed the need for more team-based evaluation approaches, which could be a valuable step in addressing fragmentation.

All five TORCH partners have signed the CoARA agreement and have thus committed themselves to implement relevant organizational changes and share practices towards a reform of research assessment. Beyond this, our alliance is inspired by the commitments and principles spelled out in CoARA. After discussing this suggestion and weighing the different options to design, launch, implement and evaluate pilot actions, the TORCH PMT decided that rather than developing piloting activities on research assessment, it was advisable to define a strategy, an action plan. We recognised that the final phase of the TORCH project coincided directly with the rapid development of CoARA and we considered it better to delay our piloting activities until the basic elements of CoARA became clearer.