Support to reforms of research assessment in the European Research Area

Topic description

ExpectedOutcome:

Projects are expected to contribute to the following expected outcomes:

• Support and contribute to the implementation of institutional changes for reforming research assessment (i.e., the assessment of research projects, researchers, research units, and research institutions), in line with Action 3 of the ERA Policy Agenda actions and in support of Action 1;
• Connection of existing organisations and initiatives for reforms of research assessment, facilitating the exchange of information and mutual learning, and stimulating consensus building among the stakeholders;
• Recommendations for policy-makers, research funding and performing organisations, higher education institutions and other research and innovation actors on how to best implement institutional changes to research assessment;
• Global outreach of European efforts to reform research assessment and reinforced international cooperation on evolutions in research assessment.

These targeted outcomes in turn contribute to medium and long-term impacts:

• Research proposals and researchers evaluated in an unbiased manner on their intrinsic merits and performance rather than on the number of publications and where these are published;
• Researchers evaluated based on a broader range of research outputs and tasks (including open science practices);
• Researchers benefit from attractive careers, regardless of gender or other social characteristics;
• Modernised higher education sector, benefitting from improvements to the research assessment systems (including for performance-based funding).

The way research projects, researchers, research units, and research institutions are assessed is fundamental for a well-functioning research and innovation system. The research and innovation system is undergoing major transformations with diversification of desired research outputs not restricted anymore to publications, and of research tasks and required skills; with a culture of sharing of knowledge and tools and of open collaboration (including societal engagement) becoming mainstream; and with a growing need of multi-disciplinary approaches and collaboration to tackle ever more complex scientific questions and societal challenges. However, the current system often uses limited methods to assess the quality, performance and impact of research, favouring quantity of results of individual researchers and the impact factor of the venue where they are published.

Several institutions including research funders and universities, in Europe and beyond, are currently reforming the assessment systems of their research and researchers. At a global level, the San Francisco Declaration on Research Assessment^[1] (DORA) aims at improving how the output of scientific research is evaluated. The Recommendation on Open Science^[2] adopted by the United Nations Educational, Scientific and Cultural Organization (UNESCO) in November 2021 calls for a review of research assessment systems to align them with the principles of open science. The Global Research Council (GRC) held a conference^[3] in 2020 on “Responsible Research Assessment” and established a dedicated working group in 2021. The European Research Area ERA Policy Agenda for 2022-2024^[4], adopted by the Council of the European Union on 26 November 2021, includes a priority action for reforming the assessment systems for research, researchers and institutions to improve their quality, performance and impact. Accordingly, the European Commission is taking steps to facilitate an agreement between research funders, research performing organisations, national/regional evaluation agencies or authorities, and other stakeholders like learned societies, to reform research assessment criteria and processes in willing organisations, along commonly agreed principles and actions. In 2021, consultation of stakeholders^[5] identified convergence on 10 principles expected to guide the reform of research assessment^[6].

The establishment of a coalition of committed organisations is expected to accelerate changes towards research assessment systems that promote qualitative judgement with unbiased peer-review, supported by a more responsible use of quantitative indicators. The reformed research assessment systems should consider a more diverse set of research cultures and outputs, by valuing not only publications but also other research outputs such as data sets, software, models, workflows, methods, etc., and proper conduct including integrity and gender equality, equal opportunities and inclusiveness. The reformed systems are expected to reward open collaboration as well as early knowledge and data sharing, as these practices enable good science^[7]. It should also reward the diversity of tasks of researchers, and consider the different contributions to the work of teams.

This action aims at supporting the reform of the assessment of research projects, researchers, research units, and research institutions. The action consists of three parts, all of which must be addressed:

• The first part aims at supporting the operation of the coalition. This includes support to the sharing of information, mutual learning and awareness raising, across individual organisations, umbrella organisations, and initiatives, involved in the coalition and beyond, including with national and regional authorities. The action is expected to support workshops, working groups and other tools to: raise awareness, exchange experience and disseminate information on institutional changes, contribute to the development and piloting of metrics needed for research assessment; identify, monitor and showcase case studies, including good practices and lessons learnt, and develop and share guidance and recommendations for research organisations and policy-makers;
• The second part involves financial support to third parties, by launching ‘cascading grant’ call(s) to support institutions from across the ERA, notably those engaged in the coalition approach, to implement sustainable institutional changes to reform and improve research assessment criteria and processes. This may require support services to be developed and provided to the beneficiary third party organisations. The ‘cascading grant’ mechanism is expected to contribute to institutional changes in a significant number of organisations (e.g., 40-50 individual organisations involved, of different types and across different geographical areas). As such, a significant proportion of the funding should be allocated to this mechanism, and one or more call(s) for proposals should be launched;
• The third part aims at international cooperation on research assessment. The action will envisage international cooperation with entities outside the EU Member States and Associated Countries. The action will promote internationally the European efforts, will support exchange of information and explore opportunities for aligning policies, and will seek to attract non-European organisations to join the coalition.

The proposals should develop close cooperation, share knowledge and evidence, and build on various institutional, national and international initiatives, including DORA, GRC, UNESCO and the European Open Science Cloud (EOSC). Proposals should also build on the results of projects funded under earlier Framework Programme actions. Notably, applicants are expected to cooperate with relevant projects funded under call topic HORIZON-WIDERA-2021-ERA-01-45, call topic HORIZON-WIDERA-2021-ERA-01-81, call topic HORIZON-WIDERA-2022-ERA-01-51, call topic HORIZON-INFRA-2022-EOSC-01-01, to ensure synergies and complementarity of outcomes. Further collaborations are expected to emerge at a later stage, e.g. resulting from the call topic HORIZON-WIDERA-2024-ERA-01-03 of Horizon Europe

The actions should envisage a duration appropriate to the ambition and complexity of the proposed topic, but not exceed 36 months.

[1] https://sfdora.org/

[2] https://unesdoc.unesco.org/ark:/48223/pf0000379949.locale=en

[3] https://www.globalresearchcouncil.org/news/responsible-research-assessment/

[4] Annex to the Council Conclusions on the future governance of the European Research Area of 26 November 2021 https://data.consilium.europa.eu/doc/document/ST-14308-2021-INIT/en/pdf

[5] Scoping report “Towards a reform of the research assessment system”:

https://data.europa.eu/doi/10.2777/707440

[6] The 10 principles identified during the stakeholders consultation and listed in the scoping report are: Ethics and integrity; Freedom of scientific research; Autonomy of research organisations; Independence and transparency of the data, infrastructure and criteria; Quality; Contribution to advancing knowledge and the (potential) impact of research results; Diversity of research activities, practices, outputs; Variety of scientific disciplines, research approaches, research career stages; Diversity in research roles and careers, and team science; and Gender equality, equal opportunities and inclusiveness

[7] Open science practices include: early and open sharing of research (for example through preregistration, registered reports, pre-prints, or crowd-sourcing); research output management; measures to ensure reproducibility of research outputs; providing open access to research outputs (such as publications, data, software, models, algorithms, and workflows); participation in open peer-review; and involving all relevant knowledge actors including citizens, civil society and end users in the co creation of R&I agendas and contents (such as citizen science).