Tools to Advance Research Assessment (TARA) (with DORA) (2021-2023)
(Funded by Arcadia, a charitable fund of Lisbet Rausing & Peter Baldwin)
TARA aims to accelerate research assessment reform through:
Systematic review of new or innovative research assessment policies and practices at academic institutions, primarily in the United States and Europe
Analysis & assessment of new or innovative research assessment policies and practices in the form of an interactive online dashboard to track criteria and standards for academic advancement
Development and testing of tools and resources to help institutions move forward with new policies and practices to improve research assessment in the United States and Europe
A Community of Practice in responsible research across Europe (funded by the Wellcome Trust) (2022-2024)
The project, in collaboration with main applicant Marcus Munafó (University of Bristol) and co-applicant Bart Penders (Maastricht University) builds on the framework provided by the Singapore Statement on Research Integrity.
Understanding different responsible research frameworks will provide a deeper insight into what constitutes responsible research practice across disciplines, and the extent to which these frameworks can be applied across disciplines – the life and biomedical sciences, the social sciences, and humanities. The 2-year project will also establish a responsible research Community of Practice – a group of people who "share a concern or a passion for something they do and learn how to do it better as they interact regularly" – initially across the UK Reproducibility Network of 20 UK HEIs, and an additional 8 continental European HEIs.
How Evaluation shapes Ocean Science – A Multi-Scale Ethnography of Fluid Knowledge (2019-2024)
(Funded by the European Research Council (ERC), grant agreement No. 805550)
The FLUIDKNOWLEDGE project will investigate the past, present and future of evaluating ocean science. Ocean science, like many fields, is under great pressure to be both scientifically excellent, and relevant to industry, and relevant to the future of the planet. How do steering efforts toward interdisciplinary engagement and societal relevance relate to other norms and criteria of scientific quality (e.g. excellence, global competitiveness) in actual practice? The team will examine the intricate epistemic consequences that evaluation might have on concrete practices of knowledge creation by combining longitudinal quantitative analysis with rich ethnographic studies in different national contexts. They will also develop concepts to theoretically grasp the constitutive potential of research evaluation, based on such multilevel approaches.
Stimulating Academic Gatekeeper Engagement in responsible research assessment (SAGE) (funded by the ZonMw Fostering Responsible Research Practices program) (2019-2020)
The project “Optimizing the responsible researcher” (see below) has enriched the knowledge base about the shared values of responsible biomedical research and the various ways in which these ideals are (or are not) put into practice. In “SAGE” we translate our results to the practice of organizing and evaluating biomedical research. We focus on the ‘gatekeepers’ at University Medical Centres who can contribute to changing the criteria that are used to evaluate and reward researchers. SAGE results in: a user-oriented workshop and report for each of our two partner institutes; a two-day executive course for decision-makers in biomedicine that targets the leadership of all Dutch university medical centres; and instruction videos on responsible evaluation for the purpose of the course that will be made openly available as online resource after the course.
Sarah de Rijcke (CWTS)
Guus Dix (CWTS)
Stephen Curry (Imperial College) - Advisor
James Wilsdon (Sheffield University) - Advisor
SOPs4RI – Standard Operating Procedures for Research Integrity (2019-2022)
(Funded by the EC through the SwafS-03-2018 program)
Coordinators: Niels Mejlgaard and Mads Sorensen (Aarhus University)
The aim of the Standard Operating Procedures for Research Integrity (SOPs4RI) project is to promote excellent research and a strong research integrity culture that aligns with the principles and norms of the ‘European Code of Conduct for Research Integrity’, and to counter research misconduct. The overall objective of SOPs4RI is to create a toolbox which fosters research integrity and prevents, detects, and handles research misconduct for European research performing organizations (RPOs) and research funding organizations (RFOs). The project involves a mixed-methods, co-creative approach to the development and empirical validation of standard operating procedures (SOPs) and guidelines that will make it possible for RPOs and RFOs to create and implement Research Integrity Promotion Plans.
'Optimizing the responsible researcher: towards fair and constructive academic advancement'
(funded by the ZonMw Fostering Responsible Research Practices program) (2017-2019)
The project aimed to describe the optimal profile of researchers in terms of their propensity to foster responsible conduct in research. This profile will be compared with existing academic incentive and reward systems.
Recent years have seen high-profile initiatives to improve current criteria for assessing academic achievements (e.g., the Leiden Manifesto, the Metric Tide, Science in Transition, DORA, METRICS, Reward Alliance). Some institutions are implementing improved and innovative incentive and reward systems. It is yet unknown whether these systems will counter unintended effects of evaluation systems and unwarranted uses of performance metrics. Plus, help to foster responsible conduct of research by selecting the scientists with a multidimensional profile (i.e., more than a good publication and citation record) and a skill-set that enables them to undertake and supervise both innovative and societally relevant research.
The project resulted in an evidence-based framework and a set of concrete policy recommendations for designing (or adapting) academic reward systems aimed at fostering excellent, socially responsible research.
Sarah de Rijcke (Principal investigator, CWTS)
Paul Wouters (CWTS)
James Wilsdon (Sheffield)
Frank Miedema (UMC Utrecht, Science in Transition),
Joeri Tijdink (VUmc) + Guus Dix (CWTS)
Lex Bouter (Project adviser, VUmc)
Anna Boyksen Fellowship 2016-2018 (TU Munich Institute for Advanced Study)
with Ruth Müller (Munich Center for Technology in Society - MCTS)
The TUM-IAS is the key feature of the TUM's institutional strategy to promote top-level research in the Excellence Initiative by the German federal and state governments. TUM-IAS awards Fellowships to distinguished researchers and gives them the necessary time and financial support to explore new venues, to develop novel research areas and to establish intensive international collaborations.
Target Group of the Anna Boyksen fellowships are outstanding scientists from outside TUM who intend to explore gender- and diversity-relevant problems with regard to the natural and engineering sciences together with a TUM research group.
The fellowship period will be devoted to prolonging the collaboration with prof. Ruth Müller at the Munich Centre for Technology in Society (MCTS, TUM). The joint project for the Anna Boyksen Fellowship will analyze interactions between evaluation criteria and knowledge production in the engineering sciences.
Promoting Integrity as an Integral Dimension of Excellence in Research (PRINTEGER) (until 2018)
(funded by the EC through a H2020 COST Action)
Coordinators: Hub Zwart and Willem Halffman (Radboud University Nijmegen)
Science and Evaluation Studies group members: Thed van Leeuwen, Paul Wouters, Sarah de Rijcke
The goal of the project was to encourage a research culture that treats integrity as an integral part of doing research, instead of an externally driven steering mechanism. In order to stimulate integrity and responsible research, new forms of governance are needed that are firmly grounded in and informed by research practice. Concretely, the work entailed in the project consisted of:
A systematic review of integrity cultures and practices;
An analysis and assessment of current challenges, pressures, and opportunities for research integrity in a demanding and rapidly changing research system;
The development and testing of tools and policy recommendations enabling key players to effectively address issues of integrity, specifically directed at science policy makers, research managers and future researchers.