Sarah de Rijcke

How Evaluation shapes Ocean Science – A Multi-Scale Ethnography of Fluid Knowledge (2019-2024)

(Funded by the European Research Council (ERC), grant agreement No. 805550) 

 

The FLUIDKNOWLEDGE project will investigate the past, present and future of evaluating ocean science. Ocean science, like many fields, is under great pressure to be both scientifically excellent, and relevant to industry, and relevant to the future of the planet. How do steering efforts toward interdisciplinary engagement and societal relevance relate to other norms and criteria of scientific quality (e.g. excellence, global competitiveness) in actual practice? The team will examine the intricate epistemic consequences that evaluation might have on concrete practices of knowledge creation by combining longitudinal quantitative analysis with rich ethnographic studies in different national contexts. They will also develop concepts to theoretically grasp the constitutive potential of research evaluation, based on such multilevel approaches.

 

https://www.fluid-knowledge.com/

​

Stimulating Academic Gatekeeper Engagement in responsible research assessment (SAGE) (funded by the ZonMw Fostering Responsible Research Practices program) (2019-2020)

​

The project “Optimizing the responsible researcher” (see below) has enriched the knowledge base about the shared values of responsible biomedical research and the various ways in which these ideals are (or are not) put into practice. In “SAGE” we translate our results to the practice of organizing and evaluating biomedical research. We focus on the ‘gatekeepers’ at University Medical Centres who can contribute to changing the criteria that are used to evaluate and reward researchers. SAGE results in: a user-oriented workshop and report for each of our two partner institutes; a two-day executive course for decision-makers in biomedicine that targets the leadership of all Dutch university medical centres; and instruction videos on responsible evaluation for the purpose of the course that will be made openly available as online resource after the course.

 

Project members:

​

• Sarah de Rijcke (CWTS)

• Guus Dix (CWTS)

• Stephen Curry (Imperial College) - Advisor

• James Wilsdon (Sheffield University) - Advisor​

SOPs4RI – Standard Operating Procedures for Research Integrity (2019-2022)

(Funded by the EC through the SwafS-03-2018 program) 

 

Coordinators: Niels Mejlgaard and Mads Sorensen (Aarhus University)

​

The aim of the Standard Operating Procedures for Research Integrity (SOPs4RI) project is to promote excellent research and a strong research integrity culture that aligns with the principles and norms of the ‘European Code of Conduct for Research Integrity’, and to counter research misconduct. The overall objective of SOPs4RI is to create a toolbox which fosters research integrity and prevents, detects, and handles research misconduct for European research performing organizations (RPOs) and research funding organizations (RFOs). The project involves a mixed-methods, co-creative approach to the development and empirical validation of standard operating procedures (SOPs) and guidelines that will make it possible for RPOs and RFOs to create and implement Research Integrity Promotion Plans.

'Optimizing the responsible researcher: towards fair and constructive academic advancement'

(funded by the ZonMw Fostering Responsible Research Practices program) (2017-2019)

 

The project aimed to describe the optimal profile of researchers in terms of their propensity to foster responsible conduct in research. This profile will be compared with existing academic incentive and reward systems.

 

Recent years have seen high-profile initiatives to improve current criteria for assessing academic achievements (e.g., the Leiden Manifesto, the Metric Tide, Science in Transition, DORA, METRICS, Reward Alliance). Some institutions are implementing improved and innovative incentive and reward systems. It is yet unknown whether these systems will counter unintended effects of evaluation systems and unwarranted uses of performance metrics. Plus, help to foster responsible conduct of research by selecting the scientists with a multidimensional profile (i.e., more than a good publication and citation record) and a skill-set that enables them to undertake and supervise both innovative and societally relevant research.

 

The project resulted in an evidence-based framework and a set of concrete policy recommendations for designing (or adapting) academic reward systems aimed at fostering excellent, socially responsible research.

​

Team members:

​

• Sarah de Rijcke (Principal investigator, CWTS)

• Paul Wouters (CWTS)

• James Wilsdon (Sheffield)

• Frank Miedema (UMC Utrecht, Science in Transition),

• Joeri Tijdink (VUmc) + Guus Dix (CWTS)

• Lex Bouter (Project adviser, VUmc)

Anna Boyksen Fellowship 2016-2018 (TU Munich Institute for Advanced Study)

with Ruth Müller (Munich Center for Technology in Society - MCTS)

 

The TUM-IAS is the key feature of the TUM's institutional strategy to promote top-level research in the Excellence Initiative by the German federal and state governments. TUM-IAS awards Fellowships to distinguished researchers and gives them the necessary time and financial support to explore new venues, to develop novel research areas and to establish intensive international collaborations.

 

Target Group of the Anna Boyksen fellowships are outstanding scientists from outside TUM who intend to explore gender- and diversity-relevant problems with regard to the natural and engineering sciences together with a TUM research group.

 

The fellowship period will be devoted to prolonging the collaboration with prof. Ruth Müller at the Munich Centre for Technology in Society (MCTS, TUM). The joint project for the Anna Boyksen Fellowship will analyze interactions between evaluation criteria and knowledge production in the engineering sciences.

Promoting Integrity as an Integral Dimension of Excellence in Research (PRINTEGER) (until 2018)

(funded by the EC through a H2020 COST Action)

 

Coordinators: Hub Zwart and Willem Halffman (Radboud University Nijmegen)

Science and Evaluation Studies group members: Thed van Leeuwen, Paul Wouters, Sarah de Rijcke

 

The goal of the project was to encourage a research culture that treats integrity as an integral part of doing research, instead of an externally driven steering mechanism. In order to stimulate integrity and responsible research, new forms of governance are needed that are firmly grounded in and informed by research practice. Concretely, the work entailed in the project consisted of:

​

1. A systematic review of integrity cultures and practices;

2. An analysis and assessment of current challenges, pressures, and opportunities for research integrity in a demanding and rapidly changing research system;

3. The development and testing of tools and policy recommendations enabling key players to effectively address issues of integrity, specifically directed at science policy makers, research managers and future researchers.

 

Consortium partners: Free University Brussels, the University of Tartu (Estonia), Oslo and Akershus University College, Leiden University, and the Universities of Bonn, Bristol, and Trento.

Measuring science: The use of metrics in assessing impact, innovation and excellence in modern academia (funded by The Swedish Research Council)

 

Björn Hammarfelt & Sarah de Rijcke

 

This project concerned the governance of science with an emphasis on how research is made auditable through the use of quantitative techniques; especially bibliometrics. Publication patterns as well as questionnaires were used in order to examine how bibliometric evaluation influences the practices of scholars. Studies were also made of university rankings and their use in promoting the ‘excellent’ university. In tying the practical, institutional, and political levels together the project aimed to provide a unique perspective on the emergence of a ‘metrics culture´ in academia. Using a theoretical stance in theories about the ´audit´ and ´risk´ society it aimed to take the analysis beyond discussions about different models of evaluating, or indicators of impact, and to place it in a larger context of social and organizational change. This broader perspective is required in order to understand the co-evolution of governance systems and techniques of measurement; a perspective where ‘metrics’ is not only a symptom but also a practice that in itself contributes to these developments.

Defining standards of intellectual quality in Dutch legal scholarship

 

Wolfgang Kaltenbrunner & Sarah de Rijcke

 

This project drew on a comparison of ongoing debates about research evaluation in three Dutch Law faculties. The project aimed to analyse the implications of the need to create new alignments between research and science policy on the one hand (including the requirement of (‘societal relevance’), and the everyday articulation work legal scholars engage in to bring research to closure on the other.

​

The impact of indicators: How evaluation shapes biomedical knowledge production (2012-2015)

 

Alex Rushforth & Sarah de Rijcke

 

In this exploratory ethnographic study we analyzed how performance indicators and research evaluation link to biomedical knowledge production in the Netherlands. Fieldwork was carried out at two University Medical Centres (UMCs) and at three research groups within each centre: a molecular cell biology laboratory, a surgical oncology laboratory, and a medical statistics group. Conducting fieldwork in two UMCs enabled a focus on how the institutional context shapes the dynamics of evaluation and indicator usage. In addition, we identified three broad registers of biomedical knowledge production, following institutionalized distinctions between basic, translational and applied research at each UMC. This sampling logic was also based on an assumption that different sub-fields of biomedicine pursue quite distinct patterns of knowledge production, for instance in terms of publication and citation practices. Theoretically, the project draws on and contributes to historical, organisational, and sociological studies of accountability practices in science.

​