Journal Article

Issues with data and analyses

Allison, David B., Kaiser, Kathryn A., Brown, Andrew W.

2018

Overview

Some aspects of science, taken at the broadest level, are universal in empirical research. These include collecting, analyzing, and reporting data. In each of these aspects, errors can and do occur. In this work, we first discuss the importance of focusing on statistical and data errors to continually improve the practice of science. We then describe underlying themes of the types of errors and postulate contributing factors. To do so, we describe a case series of relatively severe data and statistical errors coupled with surveys of some types of errors to better characterize the magnitude, frequency, and trends. Having examined these errors, we then discuss the consequences of specific errors or classes of errors. Finally, given the extracted themes, we discuss methodological, cultural, and systemlevel approaches to reducing the frequency of commonly observed errors. These approaches will plausibly contribute to the self-critical, self-correcting, ever-evolving practice of science, and ultimately to furthering knowledge.

• Publisher: National Academy of Sciences
• Subject: Anthropology / Data analysis / Data Collection - standards / Data Collection - statistics & numerical data / Data processing / Empirical analysis / Errors / Humans / Physical Sciences / Quality Control / Reproducibility of Results / Research Design - standards / Research Design - statistics & numerical data / Sackler on Improving the Reproducibility of Scientific Research / Science - standards / Science - statistics & numerical data / Scientific Experimental Error / Social Sciences / Statistics / Statistics as Topic - standards