Overcoming the journal impact factor and transforming research assessment, a perspective from the Open University of Catalonia
The Open University of Catalonia is a signatory to the San Francisco Declaration on Research Assessment (DORA). The university’s DORA Working Group explains how by becoming a signatory, the university is now working towards a fresh approach to research assessment.
Most universities and quality agencies still use the journal impact factor, first and foremost, to assess the quality of research. It is applied to evaluate the productivity of researchers or research groups, as well as to award accreditations, promotions or funding.
However, many scientists, departments and international institutions publicly criticise how these types of assessment policies are flimsy, simplistic and even prevent the preservation of epistemic diversity and scientific pluralism. Research quality cannot be measured by such an indicator. Ever more often, we hear voices argue that research is about much more than the journals and that publishing cannot be the main and only motive for research. Impact factor journals must not become simply a mechanism for rewards.
This criticism is gaining ground as the world is facing bigger and more complicated global challenges, many linked to in the United Nations' 2030 Agenda, and not all solutions are in view. If researchers are asked to simply meet the impact factor criteria set by quality agencies, knowledge generation will become more and more a race in which the role of society and the contribution to social transformation is clearly minimised.
Against this backdrop, the San Francisco Declaration on Research Assessment (DORA) was established in 2012. It is a worldwide initiative criticising the misuse of the journal impact factor and promoting the exploration of valid alternatives for assessing research quality transparently and accurately. While the journal impact factor is where DORA points its critical focus, on a more general level it also questions research assessment based on exclusionary, bibliometric and mechanised notions of excellence.
While the UOC is conditioned by the rules of specific institutional arrangements (i.e. public agencies in Spain), as a DORA signatory the university is supported in promoting internal changes that can inspire the transformation of research assessment models in a wider environment. DORA allows universities to reach mutual agreements on institutional changes that will affect research staff and publishers alike.
On top of this, in order to increase our contribution to social transformation, the UOC has elaborated an institution-wide Open Knowledge Action Plan that aims to make the university a central node for open and global knowledge. This plan includes proposals to rethink research assessment and transform our evaluation methods using more qualitative, transparent, fair, inclusive and socially relevant formulas that take into consideration not only research quality, but also the societal impact of our research. The plan proposes to work towards a research assessment that redefines what scientific merit means and recognises and values a wider range of achievements, not only the journal where it is published.
Currently, we are working on three main aspects in order to transform our institution and comply with the principles in DORA. First, we are creating a space where we can foster conversations within our academic community about how we understand research quality and how research should be assessed at our institution. Second, we are analysing all internal research funding schemes, identifying incoherencies with DORA and embedding changes towards the DORA principles. Third, we are participating in international forums, like those provided by the European University Association, in order to advocate for systemic change related to finding new ways of doing research evaluation.
Clara Riera, Daniel Aranda, Israel Rodríguez and Nadja Gmelch, on behalf of the Open University of Catalonia DORA Working Group.
The Open University of Catalonia, an EUA member, contributed to the 2019 EUA Open Science and Open Access Survey on Research Assessment, which aims to inform and strengthen the discussion by gathering and sharing information about current and future university approaches to research assessment.
“Expert Voices” is an online platform featuring original commentary and analysis on the higher education and research sector in Europe. It offers EUA experts, members and partners the opportunity to share their expertise and perspectives in an interactive and flexible exchange on key topics in the field.
All views expressed in these articles are those of the authors and do not necessarily reflect those of EUA.
