Explore the vast range of titles available.

An international team of biologists, philosophers, and historians of science explores the critically important process of replication in biological and biomedical research. Without replication, the trustworthiness of scientific research remains in doubt. Although replication is increasingly recognized as a central problem in many scientific disciplines, repeating the same scientific observations of experiments or reproducing the same set of analyses from existing data is remarkably difficult. In this important volume, an international team of biologists, philosophers, and historians of science addresses challenges and solutions for valid replication of research in medicine, ecology, natural history, agriculture, physiology, and computer science. After the introduction to important concepts and historical background, the book offers paired chapters that provide theoretical overviews followed by detailed case studies. These studies range widely in topics, from infectious-diseases and environmental monitoring to museum collections, meta-analysis, bioinformatics, and more. The closing chapters explicate and quantify problems in the case studies, and the volume concludes with important recommendations for best practices. -- Provided by publisher.

Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when using the same procedures with adequate statistical power. Two of the articles in this special section question the value of direct replication by other laboratories. In this commentary, I discuss the problematic implications of some of their assumptions and argue that direct replication by multiple laboratories is the only way to verify the reliability of an effect.

Although there are surely multiple contributors to the replication crisis in psychology, one largely unappreciated source is a neglect of basic principles of measurement. We consider 4 sacred cows-widely shared and rarely questioned assumptions-in psychological measurement that may fuel the replicability crisis by contributing to questionable measurement practices. These 4 sacred cows are: (a) we can safely rely on the name of a measure to infer its content; (b) reliability is not a major concern for laboratory measures; (c) using measures that are difficult to collect obviates the need for large sample sizes; and (d) convergent validity data afford sufficient evidence for construct validity. For items a and d, we provide provisional data from recent psychological journals that support our assertion that such beliefs are prevalent among authors. To enhance the replicability of psychological science, researchers will need to become vigilant against erroneous assumptions regarding both the psychometric properties of their measures and the implications of these psychometric properties for their studies. Bien qu'il soit certain que de nombreux facteurs contribuent à la crise de la reproductibilité en psychologie, l'un d'entre eux, largement méconnu, est la négligence des principes de base de la mesure. Nous examinons quatre principes « intouchables » de la mesure en psychologie - des hypothèses largement diffusées et rarement remises en question - qui, en rendant les pratiques de mesure discutables, peuvent alimenter la crise de la reproductibilité. Ces quatre intouchables sont les suivants : (A) nous pouvons nous fier en toute confiance au nom d'une mesure pour en déduire le contenu; (b) la fiabilité n'est pas une préoccupation majeure pour les mesures en laboratoire; (c) le recours à des mesures qui sont difficiles à recueillir écarte le besoin d'échantillons de taille plus importante; (d) des données convergentes sur la validité constituent des éléments de preuve suffisants de la validité conceptuelle. Pour les éléments a et d, nous fournissons des données provisoires issues de revues de psychologie récentes qui soutiennent notre affirmation selon laquelle de telles croyances prévalent parmi les auteurs. Afin d'améliorer la reproductibilité de la science de la psychologie, les chercheurs devront être vigilants face aux suppositions erronées concernant les propriétés psychométriques de ces mesures et aux répercussions de ces propriétés psychométriques pour leurs études. Public Significance Statement This article outlines four widely held but erroneous measurement assumptions that may adversely affect the accuracy and replicability of psychological findings. The effects of questionable measurement practices stemming from these assumptions are discussed, and new data bearing on the prevalence of these assumptions in academic journals are presented. In addition, this article offers several potential remedies that researchers and journals can implement to improve the measurement of psychological constructs.

This paper discusses recent trends in the use of replications in economics. We include the results of recent replication studies that have attempted to identify replication rates within the discipline. These studies generally find that replication rates are relatively low. We then consider obstacles to undertaking replication studies and highlight replication initiatives in psychology and political science, behind which economics appears to lag.

Recent controversies in psychology have spurred conversations about the nature and quality of psychological research. One topic receiving substantial attention is the role of replication in psychological science. Using the complete publication history of the 100 psychology journals with the highest 5-year impact factors, the current article provides an overview of replications in psychological research since 1900. This investigation revealed that roughly 1.6% of all psychology publications used the term replication in text. A more thorough analysis of 500 randomly selected articles revealed that only 68% of articles using the term replication were actual replications, resulting in an overall replication rate of 1.07%. Contrary to previous findings in other fields, this study found that the majority of replications in psychology journals reported similar findings to their original studies (i.e., they were successful replications). However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. Moreover, despite numerous systemic biases, the rate at which replications are being published has increased in recent decades.

After a decade of implementing open science practices as a principal investigator, mentor, data repository founder, and editor-in-chief, I have learned that the question is not whether researchers should adopt these practices but how to adapt them meaningfully. This commentary, based on a talk given at the 2024 Canadian Society for Brain, Behaviour, and Cognitive Science conference, argues for two key principles: First, open science implementation must be context-dependent rather than one-size-fits-all, and second, practical research realities require flexible approaches to idealized policies. Through personal examples, from my evolution with preregistration from \"recipe\" to \"guide\" during COVID-19 research to challenges with Registered Reports using existing data sets, I show how open science practices work best when researchers approach them as evolving tools rather than rigid rules. I also discuss field-specific differences in open science uptake between psychology and education and the importance of equity considerations in implementation. The commentary concludes with concrete recommendations for researchers and journals, emphasizing that sustainable open science requires meeting researchers where they are while maintaining transparency and scientific rigour. Après avoir consacré une décennie à la promotion et à la mise en œuvre des pratiques de la science ouverte en tant que chercheuse principale, mentore, fondatrice d'une base de données et rédactrice en chef, j'ai réalisé qu'il ne s'agit pas de savoir si les chercheurs doivent adopter ces pratiques, mais plutôt de comprendre comment les intégrer de manière significative. Ce commentaire, basé sur une présentation donnée lors de la conférence 2024 de la Société canadienne des sciences du cerveau, du comportement et de la cognition, plaide en faveur de deux principes clés : premièrement, la mise en œuvre de la science ouverte doit être adaptée au contexte plutôt que fondée sur une approche universelle et, deuxièmement, les réalités pratiques de la recherche exigent des approches flexibles face aux politiques idéalisées. À travers des exemples personnels - de mon évolution dans l'utilisation de la préinscription, passée d'une « recette » à un « guide » pendant mes recherches liées à la COVID-19, aux défis rencontrés avec les rapports enregistrés utilisant des ensembles de données existants -, je montre que les pratiques de science ouverte fonctionnent le mieux lorsque les chercheurs les considèrent comme des outils en constante évolution plutôt que comme des règles rigides. Je discute également des différences spécifiques au domaine dans l'adoption de la science ouverte entre la psychologie et l'éducation, ainsi que de l'importance des considérations d'équité dans la mise en œuvre. Le commentaire se termine par des recommandations concrètes à l'intention des chercheurs et des revues, en soulignant que la durabilité de la science ouverte repose sur une approche qui rejoint les chercheurs là où ils en sont, tout en maintenant la transparence et la rigueur scientifique. Public Significance Statement Open science practices like sharing research data and preplanning studies can make research more trustworthy and useful, but researchers often struggle with how to implement these practices in real-world situations. This commentary shows that rather than having rigid, one-size-fits-all rules for open science, we need flexible approaches that work for different types of research while still maintaining transparency and scientific quality. These insights can help Canadian researchers, funding agencies, and journals develop more practical and equitable policies that actually support better science rather than creating barriers for researchers with different resources or research contexts.

We make a two-pronged proposal to (i) strengthen the incentives for replication work and (ii) better organize and draw attention to the replications that are conducted. First we propose that top journals publish short “replication reports.” These reports could summarize novel work replicating an existing high-impact paper, or they could highlight a replication result embedded in a wider-scope published paper. Second, we suggest incentivizing replications with the currency of our profession: citations. Enforcing a norm of citing replication work alongside the original would provide incentives for replications to both authors and journals.

There are many high-quality resources available which describe best practices in the implementation of both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Yet, partly owing to the complexity of these procedures, confusion persists among psychologists with respect to the implementation of EFA and CFA. Primary among these misunderstandings is the very mathematical distinction between EFA and CFA. The current paper uses a brief example to illustrate the difference between the statistical models underlying EFA and CFA, both of which are particular instantiations of the more general common factor model. Next, important considerations for the implementation of EFA and CFA discussed in this paper include the need to account for the categorical nature of item-level observed variables in factor analyses, the use of factor analysis in studies of the psychometric properties of new tests or questionnaires and previously developed tests, decisions about whether to use EFA or CFA in these contexts, and the importance of replication of factor analytic models in the ongoing pursuit of validation. De nombreuses ressources de haute qualité existent pour décrire les meilleures pratiques en matière de mise en œuvre de l'analyse factorielle exploratoire (AFE) et de l'analyse factorielle confirmatoire (AFC). Or, en partie dû à la complexité de ces procédures, une certaine confusion persiste entre les psychologues quant à la mise en œuvre de l'AFE et de l'AFC. L'une des principales sources de ces malentendus réside dans la distinction mathématique entre l'AFE et l'AFC. Le présent article utilise un bref exemple pour illustrer la différence entre les modèles statistiques sous-jacents à l'AFE et l'AFC, lesquels sont tous deux des instanciations particulières du modèle factoriel plus général. Ensuite, d'importantes considérations relatives à la mise en œuvre de l'AFE et de l'AFC, abordées dans le présent article, incluent la nécessité de tenir compte de la nature catégorique de variables observées au niveau des items dans les analyses factorielles, l'utilisation de l'analyse factorielle dans l'étude de propriétés psychométriques de nouveaux tests ou questionnaires et de tests élaborés dans le passé, des décisions quant à la procédure la plus appropriée - soit l'AFE ou l'AFC - dans ces contextes et l'importance de la reproduction de modèles d'analyse factorielle dans la poursuite de la validation en cours.

Psychological science is hard. This short article focuses on two issues. One has to do with the importance of understanding statistical power and how post hoc data explorations and selective reporting can lead to exaggerated estimates of the size of effects and the strength of relationships (which in turn contribute to replication failures). The other topic is tools research psychologists can use to improve the reproducibility of their procedures and analyses. The article closes with a comment on the deeper challenge of improving the usefulness and testability of theories in psychology. La science de la psychologie est une discipline exigeante. Ce court article se concentre sur deux enjeux. Le premier concerne l'importance de comprendre la puissance statistique et la façon dont les explorations post hoc des données et les rapports sélectifs peuvent entraîner des surestimations de la taille des effets et de la force des relations (contribuant ainsi aux échecs de réplication). Le second porte sur les outils que les psychologues peuvent utiliser pour améliorer la reproductibilité de leurs procédures et analyses. La conclusion de l'article aborde le défi complexe lié à l'amélioration de la pertinence et de la validité des théories psychologiques. Public Significance Statement Psychologists (and other life scientists) often use statistical analyses of data collected from samples of participants to make inferences about causal and correlational relationships in populations. This article aimed to improve understanding of constraints on the uses of inferential statistical tests and to encourage psychological scientists to strive to make their statistical analyses appropriate, transparent and reproducible. The article closes with a brief comment on the importance of theory development in psychological science.

We extend the basic theory of kriging, as applied to the design and analysis of deterministic computer experiments, to the stochastic simulation setting. Our goal is to provide flexible, interpolation-based metamodels of simulation output performance measures as functions of the controllable design or decision variables, or uncontrollable environmental variables. To accomplish this, we characterize both the intrinsic uncertainty inherent in a stochastic simulation and the extrinsic uncertainty about the unknown response surface. We use tractable examples to demonstrate why it is critical to characterize both types of uncertainty, derive general results for experiment design and analysis, and present a numerical example that illustrates the stochastic kriging method.