Page: Assessing Funding Applications
Research assessment processes in focus
Research assessment is a core activity that has high importance of what science gets done, by whom, and in what ways. Research funders are at the forefront of various efforts to improve research assessment and to promote responsible assessment cultures and practices.
There are two key aspects to consider in monitoring openness and responsibility in research assessment:
- What past and proposed future contributions are evaluated in making a decision on whether to fund a grant proposal?
- How is the assessment process designed and conducted by the research funding organisation?
For example, grant applications are increasingly expected to demonstrate their relevance to societal stakeholders and describe measures by which research results will be shared. Involving non-academic experts in assessment processes to help form judgements about such expectations has thus become a concern for many funders.
Other aspects of the way in funders conduct research assessment processes include the elimination of gender bias from assessment processes.
Research funders enhance openness and responsibility in the conduct of research assessment through three main activities:
- Composition of assessment panels
- Selection of reviewers
- Training or guidance support for reviewers and assessors
Composition of assessment panels refers to the selection of panel members to participate in assessment panels that make recommendations regarding the funding of grant applications. This category includes inviting assessors from different disciplines and from different types of organisations, the setting up of gender balanced assessment panels, and the presence of societal stakeholders and experts on these panels.
Selection of reviewers refers to the inclusion of diverse experts in the initial reading, review, and scoring of grant applications. This can include international and national experts, reviewers with scientific and/or societal stakeholder expertise in the topic, and selections based on gender representation, and other criteria.
Training or guidance support refers to all areas in which the RFO supports reviewers and assessors by providing specific training, guidelines, or instructions on how to conduct a responsible assessment. This includes training or guidance on how avoid unconscious biases, and for mitigation of gender biases to prevent discrimination against women grant applicants. This includes guidance on assessing achievement relative to opportunities, to better take account of career breaks or periods of underemployment due to maternity or other caring responsibilities that can impact on women researchers’ careers.
How research funders support open and responsible research assessment
A majority of research funders support responsible research assessment through the composition of their assessment panels. A slightly smaller majority of funders support responsible research assessment by providing training or guidance to grant application reviewers and/or assessment panel members. Criteria for the selection of diverse grant application reviewers is another popular approach to responsible assessment processes among RFOs.
An overall assessment of grant assessment processes considers how many open and responsible elements are part of a research funders’ approach to assessment.
Overall assessment of research funders’ grant assessment processes
How the overall assessment works (Do we include this?)
Analyses of policy documents, main research programme call documents, and expert interviews conducted with research funders, are the basis for an overall qualitative assessment of the openness and responsibility of research funders’ grant assessment processes.
Descriptions of assessment categories:
| Descriptor 1 | Descriptor 2 |
Responsible + | Non-academic experts usually/always included as reviewers and/or members of assessment panels | Gender balanced panels, guidance/training on open and responsible aspects, trans/interdisciplinarity valued, no conflict of interest (COI) |
Responsible | Non-academic experts sometimes included as reviewers and/or members of assessment panels | Gender balanced panels, Guidance/training on RRI-related aspects, trans/interdisciplinarity valued, no COI |
Balanced | Gender balanced panels/reviewers or non-academic expert included as reviewers and/or members of assessment panels | Interdisciplinarity valued, no COI |
Basic | Gender equality and/or interdisciplinarity valued | No COI |
HERE: Honeycombs
Resources
A range of open resources are available are available for those who are interested in open and responsible research funding.
Detailed descriptions of the collection of the data and information about open and responsible research funding shown at PROMISE4era.eu can be found here.
Open data on national research funding can be found at COKI
A range of position papers and other resources about trends in research funding from the perspective of public research funding organisations can be found at Science Europe.
Private research foundations’ activities and discussions about funding can be found the European Foundation Centre.