CONTEXT:
The integration of responsibility into evaluation practices stood as a focal point within the SUPER_MoRRI project’s agenda, prominently featured during the project’s annual event in April 2021. To facilitate collaborative learning about Responsible Research and Innovation (RRI) and RRI-like initiatives beyond Europe, we organized a series of webinars grouped by regions, covering the Americas, Asia/Pacific, and Africa/Middle East, with a fourth session dedicated to reflecting on key themes emerging from the previous discussions.
Each regional webinar commenced with a plenary session featuring three country-based panel presentations, offering insights into local perspectives on responsible research evaluation practices. These presentations served as a catalyst for subsequent in-depth discussions within smaller breakout sessions, attended by representatives from research funding organizations, research performing institutions, and policy-makers, fostering an environment conducive to shared insights and collaborative dialogue.
The combined total of participants from thirty different countries, including Australia, Austria, Botswana, Colombia, Germany, Ghana, Japan, Kenya, India, Iran, Mexico, Netherlands, Rwanda, Spain, and the USA, nearly reached 100 individuals across the three regional sessions. This diverse representation not only facilitated the exchange of valuable experiences from various corners of the globe but also laid the foundation for a burgeoning stakeholder network poised to influence research and innovation through responsible evaluation practices.
LESSONS LEARNED:
Through these discussions, it becomes evident that there are substantial disparities in the social, economic, political, and geographical characteristics of the participating countries worldwide, reflecting the varying developmental stages of their science systems. However, despite these differences, recurring themes emerged when addressing responsible evaluation, highlighting the common threads that unite the global community in the pursuit of ethical research and innovation practices.
Lesson 1: Use of universal indicators
The widespread application of universal indicators appears unsuitable in various contexts. In discussions surrounding responsible practices, no nation wishes to employ or generate information that might undermine cultural diversity or fuel inter-regional competition to meet rigid metrics. Nevertheless, several countries express a valid apprehension concerning the subjectivity inherent in evaluations. This concern is especially pertinent in regions where the professionalization of research management and evaluation processes is still in its nascent stages. In these contexts, metrics alone may prove insufficient in comprehending the intricate dynamics of national knowledge production systems. Nonetheless, there remains a pressing need for metrics to substantiate the allocation of limited funds for scientific endeavors.
Lesson 2: Change takes time
The consensus among most countries was that altering policy in pursuit of responsible research practices is not particularly challenging; the real hurdle lies in shifting the culture. While new legislation and regulations can enforce change, they must align with people’s intrinsic motivations for responsibility, and the transformation should be gradual. Abrupt shifts may force compliance, but as highlighted by officials from various continents, excessive pressure frequently results in research integrity issues.
Lesson 3: Change requires investment
Introducing responsible evaluation practices into research necessitates a significant commitment in terms of both financial resources and human capital. Funding plays a pivotal role in this equation, particularly when it comes to training research managers and high-level evaluators for science systems worldwide. The concept of evidence-based policy within the scientific realm is meaningful only when individuals possess the qualifications to generate and utilize such evidence. Achieving this competence entails substantial investments in both time and financial resources.
Lesson 4: Responsibility beyond standardization
Efforts advocating for the implementation of standardized evaluation systems, particularly those relying on internationally established indicators, often evoke apprehension when responsibility is the focal point. Many evaluation models predominantly rely on audit-like or checkbox-ticking methodologies, which frequently fail to yield positive impacts on local scientific landscapes. True responsibility in evaluation only materializes when it embraces active engagement from researchers and stakeholders within the evaluation framework. This involvement enables them to contribute perspectives to the indicators and champion the assessment of locally relevant research alongside the internationally anticipated output from scientists. In this regard, it becomes imperative to recognize and account for the unique realities of each country when defining what responsibility means within their specific context.