Summer Seminar Scientific Progress – Individual and Collective

Description

The last years have brought a new wave of cross-disciplinary interest in research itself, often motivated by concerns about potential threats to scientific progress such as hyper-competition, fraud, questionable research practices, poor research documentation, publication bias, and insufficient cooperation within and between disciplines.

The goal of this seminar is to bring together recent work from epistemology and philosophy of science on scientific progress and meta-scientific perspectives from a broad range of areas like sociology, psychology, economics, complexity science, science studies and cultural and evolutionary anthropology, fostering an active exchange between disciplines and viewpoints.

Program

Please find the program here.

The 3-day program is organized around three themes (see below) and comprises invited talks from an interdisciplinary group of international speakers, structured discussion rounds between participants, and poster sessions. In addition to substantive contributions, a central aim of the seminar is to examine concepts and assumptions that are relevant across fields. We will discuss differences in the conceptualisation of core notions such as (for example) progress, knowledge, publication bias, and their operationalisation in empirical studies and simulation models. 

Theme 1: What is scientific progress?

Theme 2: From individual to collective knowledge generation – funding, peer review, publication process, citation, retraction, canonisation

Theme 3: The interplay of individual and collective in scientific progress – competition, cooperation, team science

Format

The seminar will be held on location in Amsterdam at the Vrije Universiteit Amsterdam. It is not possible to attend the seminar through live streaming. The talks of the speakers will be recorded (not the discussions) and posted on our website shortly after the seminar.

Poster Presentations and Lightning Talks

On Thursday, 25th August, several researchers will give a poster presentation or lightning talk about their research. Find their abstracts below.

Joshua Luczak (Singapore Management University) – Scientific Progress and Robust Bayesian Confirmation

Aran Arslan and Frank Zenker (Bogazici University, Istanbul) – Cohen’s error-rates and the body of knowledge in behavioral science

Dominika Czerniawska, Richelle Boone and Simcha Jong (Leiden University) – Scientific Collaboration under the Shadow of a War: The Nuclear Fusion Community after the Russian Invasion of Ukraine

Serdal Tümkaya (Middle East Technical University) – Thomas Kuhn and his cumulative progress through scientific revolutions

Yanmeng Xing, Ying Fan, Roberta Sinatra and An Zeng (An Zeng GuangDong University of Technology) – The neglected risk of working in a big scientific group

Andrew M. Demetriou, Amy van der Meijden, Jos Sloof, Mattheo de Wit, Emiel Witting, Andreea Zlei and Cynthia C. S. Liem (Delft University of Technology) – Alexandria: a Proof-of-Concept Publication Platform that Treats Academic Outputs like Software Artifacts

Sakshi Ghai (University of Cambridge) – Is science updating the definition of human diversity?

Speakers

Darrell Rowbottom: Is scientific progress just what you’d like? The literature on (cognitive) scientific progress in the philosophy of science is dominated by competing theories, involving general claims that progress is constituted, wholly or centrally, by increases in knowledge, or verisimilitude, or understanding, or problem-solving capacity, or some appropriate surrogate. (Invent a new theory, and get a new paper published.) But underlying these theories is typically an unspoken, and invariably an undefended, presumption: that there is an objective, temporally invariant, ‘aim of science’. The fundamental idea is that when this aim of science is furthered, then progress occurs. So the relevant authors typically disagree, ultimately, on what said aim is.

However, no philosopher has yet succeeded in making sense of the notion of an aim of science; and the most promising accounts thereof suggest that sociological (rather than philosophical) study should be used to determine such an aim and that it would only be temporally invariant as a matter of contingency.

Given also the widespread disagreement about what constitutes scientific progress, moreover, we should be open to considering an alternative view which has, rather remarkably, been neglected; I call it ‘cognitive scientific progress anti-realism’. This view, which is analogous to moral anti-realism, is that the values involved in assessing whether progress has occurred in science are fundamentally subjective (and perhaps by extension intersubjective) in character. Cognitive progress is what you (or we) make it. Or so I shall argue. Darrell Rowbottom is a Professor of Philosophy at Lingnan University in Hong Kong.

Felipe Romero: The conceptual origins of metascience: fashion, revolution, or spin-off? The term ‘metascience’ precedes the replication crisis. However, only now is metascience becoming institutionalized. This institutionalization and its perils require philosophical attention. How did metascience emerge? Where does the novelty of metascience lie? How does metascience relate to other fields that take science as their subject matter? In this talk, I explore different models of discipline creation and change, and seek to understand whether they can make sense of the emergence of metascience. After examining these models, I suggest that we should question the increasingly popular perception of metascience as a fully authoritative field. Felipe Romero is an Assistant Professor in the Department of Theoretical Philosophy at the University of Groningen.

James Evans: The Social, Biographical and Demographical Locus of Innovation. In this talk, I explore the social, biographical and demographical locus of innovation in science, scholarship and technological development. With teams growing in all areas of scientific and scholarly research, I first explore the relationship between team structure and the character of innovation they produce. Drawing on 89,575 self-reports of team member research activity underlying scientific publications, I show how individual activities cohere into broad roles of (1) leadership through the direction and presentation of research and (2) support through data collection, analysis and discussion. The hidden hierarchy of a scientific team is characterized by its lead (or L)-ratio of members playing leadership roles to total team size. The L-ratio is validated through correlation with imputed contributions to the specific paper and to science as a whole, which I use to effectively extrapolate the L-ratio for 16,397,750 papers where roles are not explicit. I show that relative to flat, egalitarian teams, tall, hierarchical teams produce less novelty and more often develop existing ideas; increase productivity for those on top and decrease it for those beneath; increase short-term citations but decrease long-term influence. These effects hold within-person — the same person on the same-sized team produces science much more likely to disruptively innovate if they work on a flat, high L-ratio team. These results suggest the critical role flat teams play for sustainable scientific advance and the training and advancement of scientists–tall teams and older scientists have more influence (and receive more credit) than ever before.

I then explore how with rising life expectancies around the world and an older scientific workforce than ever before, what aging means for individual scientists and what aging scientists mean for scientific and technological progress? Prior research focuses on star scientists, their changing dates and rates of breakthrough success across history. I examine how all scientists and scholars age in terms of how their stream of ideas and contributions relate to the evolving frontier of knowledge, and how demographically aging fields relate to field-level progress. Analyzing more than 244 million scholars across 241 million articles across the last two centuries, here I show that for all fields, periods, and impact levels, scientists’ research ideas and references age linearly over time, their research is less likely to disrupt the state of science and more likely to criticize emerging work. Early success accelerates scientist aging; while changing institutions and fields and collaborating with young scientists slows it. These patterns aggregate within fields such that those with a higher proportion of older scientists experience a lower churn of ideas and more rapid individual aging, suggesting a universal link between aging, activity, and advance. The work demonstrates how tracking the demography of scientists can forecast areas of growth and maturity, and suggests how managing it through policy could help modulate science between crystallization and chaos. James Evans is a Professor of Sociology, Director of the Knowledge Lab, and Faculty Director of Computational Social Science at the University of Chicago.

Insa Lawler: Scientific progress and idealization. Intuitively, science progresses from truth to truth. A glance at history quickly reveals that this idea is mistaken. We often learn from scientific theories that turned out to be false. This paper focuses on a different challenge: Idealizations are deliberately and ubiquitously used in science. Scientists thus work with assumptions that are known to be false. Any account of scientific progress needs to account for this widely accepted scientific practice. I examine how the four dominant accounts—the problem-solving account, the truthlikeness account, the epistemic account, and the noetic account—can cope with the challenge from idealization, with an eye on indispensable idealizations. One upshot is that, on all accounts, idealizations can promote progress. Only some accounts allow them to constitute progress. Insa Lawl is an Assistant Professor of Philosophy at the University of North Carolina at Greensboro.

Hanne Andersen: Collective Knowledge Production – The Importance of Granularity. Science has developed immensely over the last century. For example, in addressing a wider and wider range of topics, and requiring more and more resources, scientific research has become increasingly collaborative and interdisciplinary. In this talk, I shall analyze some of the epistemic implications of this development. I shall focus on how groups of different kinds and different sizes can provide different conditions for collective knowledge production, and I shall discuss how philosophical analyses of knowledge production in groups can provide input for ongoing debates in science policy on research organization, research funding, and strategic research planning. Hanne Andersen is a Professor in Philosophy of Science and Head of Department of Science Education at the University of Copenhagen.

Hannah Rubin: Structural Causes of Citation Gaps. The social identity of a researcher can affect their position in a community, as well as the uptake of their ideas. In many fields, members of underrepresented or minority groups are less likely to be cited, leading to citation gaps. Though this empirical phenomenon has been well-studied, empirical work generally does not provide insight into the causes of citation gaps. I will argue, using mathematical models, that citation gaps are likely due in part to the structure of academic communities. The existence of these ‘structural causes’ has implications for attempts to lessen citation gaps and for proposals to make academic communities more efficient (e.g. by eliminating pre-publication peer review). These proposals have the potential to create feedback loops, amplifying current structural inequities. Hannah Rubin is an Assistant Professor of Philosophy at the Department of Philosophy at the University of Notre Dame.

Arnout van de Rijt: The “Proposal Effect” in Science Funding. Grants tend to go to people who won grants before. This may be (1) because of a Matthew effect in science funding or (2) because some are better scientists than others (or both). In either case, the implication is that if in a funding competition panelists did not actually read the full proposal texts, they would end up selecting a not-too-different set of winners. We report on a field experiment conducted by The Dutch Research Council (NWO) in collaboration with the authors in an early-career competition for awards of 800,000 euros. At the preselection stage of the competition, proposals were evaluated by the regular panel as well as by a shadow panel. A random half of shadow panelists were shown a CV and only a one-paragraph summary of the proposed research, while the other half were shown a CV and a full proposal. We find that two shadow panelists did not agree less on the merit of an application when one did not have access to the full proposal text than when both had access. These results suggests that lengthy proposal texts are mostly redundant and of little consequence in the selection of applications, at least at the preselection stage of the grant review process, where most applications get discarded. Arnout van de Rijt is a Professor of Sociology at the European University Institute in Italy.

Aydin Mohseni: Intervention and Backfire in the Replication Crisis. How should we assess proposals for changes in the norms of scientific practice? I present a modeling framework that addresses essential aspects of this question and I employ the framework to illustrate unobvious implications for standard, proposed methodological interventions in the replication crisis. In particular, I show how interventions intended to lower false discovery rates can exhibit diminished efficacy and even backfire under imminently plausible conditions. While such results should be combined with empirical data to make reliable inferences, I argue that these should already demonstrate the promise of the proposed framework in elucidating
unrecognized facets of the logic of intervention. Aydin Mohseni is a Postdoctoral Research Fellow at the Center for Philosophy of Science at the University of Pittsburgh.

Roberta Sinatra: Quantifying the biases of scientific impact. Every day our life is made easier by efficient measures and algorithms that search and rank scientific information. Yet, these measures and algorithms have an issue: they are trained on citations, which are ingrained with human biases. Therefore the output is inherently biased too, creating inequalities, raising concerns of discrimination, even harming economic growth. In this talk, I will present research focusing on quantifying bias in publication data. The overarching goal of this research is to uncover the bias mechanisms that, given the same quality, drive different citation trajectories, and use them (1) to create fair measures and algorithms, (2) to improve our understanding of the scientific enterprise. Roberta Sinatra is an Associate Professor of Data Science at ITU Copenhagen.

Mike Thicke: Modes of collective knowledge production in science. How should scientists be organized to best achieve collective epistemic progress? I consider three modes of organization: an entrepreneurial mode, a centralized mode, and a hybrid mode where a centralized body both sets the research agenda and synthesizes research outputs. Entrepreneurial science—where scientists have the freedom to choose their own agenda and where collective knowledge is an emergent property of the publication process—is the paradigmatic mode of research today. I argue that many pathologies of science can be attributed to this mode, which suggests that we ought to look for other ways of doing science collectively. Mike Thicke is a Researcher in Philosophy of Science and Social Epistemology, MESH Research at Michigan State University.

Daniël Lakens: Which research is valuable enough to do well? In recent years scientists in many disciplines have had to raise the bar to improve the reliability of their findings. The increasing realization that fields should perform more replication studies, as well as design studies with larger samples, reduces the number of research questions we can reliably study, compared to our beliefs about this a decade ago. This raises questions concerning research prioritization. Which studies should fields perform, and how should they decide upon these studies? Are individual scientists able to make these choices, or does research prioritization require collective decision making, for example at consensus conferences? Would increasing coordination in the selection of research questions facilitate scientific progress, or hinder it? And how should we shape reward structures in academia to move towards collective agreements about research prioritization, in fields where it is deemed to be beneficial? Daniël Lakens is an Associate Professor in the Human-Technology Interaction group at Eindhoven University of Technology.

Leo Tiokhin: Science as an evolutionary process. Science involves generation of variation (ideas, methods), transmission of information within and between generations, and mechanisms for selection that favor some variants over others (journals, funding agencies, rules that scientists use to evaluate ideas). Science thus satisfies the 3 conditions necessary for the operation of evolution by means of natural selection: variation, heritability, differential reproductive success. In this talk, I will illustrate how an evolutionary perspective can shed light on key issues relevant to the question of how to reform science to improve collective scientific progress. Leo Tiokhin is a postdoc at Eindhoven University of Technology & Research Fellow at the University of Amsterdam Institute for Advanced Study.

Duygu Uygun-Tunç: Collective vice in science: Lessons from the credibility crisis. In this talk I examine the role of epistemically virtuous scientists in the success (and failure) of science as a social institution characterized by predominantly epistemic ends; that is, in collective epistemic virtue (and vice). I analyze several structural explanations of the epistemic success of science which rule out virtue attributions to scientists vis-a-vis a case of collective epistemic vice; i.e., the credibility crisis in the social and behavioral sciences. The question as to the role of epistemic virtue in science is embedded in a broader debate on how the goals of epistemically virtuous communities and epistemically virtuous individuals are related.

On one extreme end of the individual-collective virtue debate there is the conservative or traditional position that individual epistemic virtue is necessary and sufficient for collective epistemic virtue. Let us call this position virtue absolutism. On the other extreme end there is the recently much more popular position that individual virtue is neither necessary nor sufficient for collective virtue, and even individual vice can possibly contribute to the success of science. Let us call this virtue reductionism. This latter position has been defended by many in the philosophy of science with different arguments that aim to explain the success of science in terms of its social structure, most notably the arguments from the invisible hand and division of cognitive labor by Hull and Kitcher respectively. These arguments maintain that the success of science is primarily due to the incentive structures of its credit economy, thanks to which scientists can (and sometimes better) serve collective epistemic good by pursuing credit instead of genuinely epistemic aims such as truth or knowledge. The same conclusion is defended in the sociology of science in reference to scientific norms, most notably by Merton. The sociological argument maintains that scientists behave in a way that serves the institutional goals of science not because they are epistemically virtuous people but because they comply with the rules and norms enjoined by the scientific institution, which are thus the real cause behind its success.

While acknowledging (contra virtue-absolutism) that divergent motivations and behaviors might also serve the collective goals of science, I argue (contra virtue-reductionism) that the presence of a significant proportion of epistemically virtuous scientists in a scientific community is a necessary condition for it to manifest collective epistemic virtue. I analyze a set of major arguments for virtue-reductionism, and offer objections in reference to meta-scientific research and theorizing on the credibility crisis. Firstly, the same incentive structures correlate with collective epistemic vice as well as virtue, which begs for additional explanatory factors.

Secondly, scientists respond to incentives or sanctions differently depending on both individual and contextual factors. In some cases the most salient explanation of the heterogeneous behavior of scientists in relation to situational factors are features pertaining to their epistemic characters. Thirdly, many equally important aspects of the success of science cannot be explained without reference to individual epistemic virtue. Scientific progress is multifaceted: besides productivity and impact we also value reliability and rigor, without which the former can be meaningless. However, there is no single social mechanism that can achieve these all at once, because there are trade-offs between values pertaining to the context of discovery and those pertaining to the context of justification. While successful individual contributions to discovery-oriented collective goals need not manifest epistemic virtue, successful contributions to justification-oriented goals do.

Lastly and most importantly, neither incentive structures nor institutional norms can be established, maintained, or reformed in a way that reliably serves collective epistemic ends of science without there being a significant proportion of epistemically virtuous scientists. Duygu Uygun-Tunç is a Marie Curie Cofund Fellow at Middle East Technical University and visiting scholar at University of California, Irvine.

Registration

Registration is closed.

Organising committee

Anne Scheel, Postdoctoral Researcher, Philosophy Department, Vrije Universiteit Amsterdam

Jeroen de Ridder, Associate Professor of Philosophy, Philosophy Department, Vrije Universiteit Amsterdam

Vincent Traag, Senior Researcher, Centre for Science and Technology Studies (CWTS), Leiden University

Duygu Uygun-Tunç, Marie Curie Cofund Fellow at Middle East Technical University, Ankara, and Visiting Scholar at University of California, Irvine

Daniël Muller, Project Manager at Abraham Kuyper Center for Science and the Big Questions, Vrije Universiteit Amsterdam

Cancellation policy

In case Covid restrictions make it necessary to switch to an online only format, full reimbursements of the tickets will be issued. Cancellation for any other reason is possible until August 15, minus administration costs (max. 11 euros).

Privacy policy

Find our privacy policy here.

Contact

For questions or more information, please contact:

Daniël Muller, Project Manager Abraham Kuyper Center
@: d.muller2@vu.nl