Identifying Metrics for Measuring Research Culture at the University of Leeds

Abstract

Evaluating research quality, environment, and impact has become standard in Higher Education (HE) and is largely based on quantitative data and expert assessments. Data-driven evaluations that focus on high-level statistics or conventional outputs can compromise the recognition of a wider range of research outputs and outcomes by a more diverse range of contributors. Hence, the mechanisms for evaluating research must be applicable and inclusive of a wide range of research activities. In contrast, research culture covers a vast breadth of areas, from career development, career pathways, reward, and recognition, to research integrity and equality. Most of these areas are not easily measurable, with capacity and capability limitations compounding the challenge. Clearly, there is a wealth of measurement options, which many research institutions are currently grappling with to best suit their local context.  However, there are concerns as to whether it is appropriate or even possible to measure research cultural change. Concentrating too heavily on metrics rather than the changes themselves may pose additional barriers to the cultural change we desire. Thus, we argue that the adopted measures must be nuanced for context and for success relative to where we started and what we collectively understand as being measured.

Here we discuss the University of Leeds’ process of selecting metrics to measure research culture change over the next five years. We share how we engaged with the SCOPE framework to identify, shortlist, and probe potential metrics across the four strategic objectives we have identified are best placed to enhance our research culture. From an initial list of more than 80 metrics we have been able to narrow down to just five robust metrics that we feel, with regular monitoring, will maintain adaptability, resilience, and rigour.

This paper aims to provide open and transparent insight into how we have chosen to measure our change in research culture, in order to: benefit the wider sector; foster the sharing of best practices and avoid duplication of efforts. Thus, capturing the true essence of what we at the University of Leeds think it means to change culture.