Journal Article

Assessing and adjusting for bias in ecological analysis using multiple sample datasets

Li, Qingfeng

2025

Overview

Background Ecological analysis utilizes group-level aggregate measures to investigate the complex relationships between individuals or groups and their environment. Despite its extensive applications across various disciplines, this approach remains susceptible to several biases, including ecological fallacy. Methods Our study identified another significant source of bias in ecological analysis when using multiple sample datasets, a common practice in fields such as public health and medical research. We show this bias is proportional to the sampling fraction used during data collection. We propose two adjustment methods to address this bias: one that directly accounts for the sampling fraction and another based on measurement error models. The effectiveness of these adjustments is evaluated through formal mathematical derivations, simulations, and empirical analysis using data from the 2014 Kenya Demographic and Health Survey. Results Our findings reveal that the sampling fraction bias can lead to significant underestimation of true relationships when using aggregate measures from multiple sample datasets. Both adjustment methods effectively mitigate this bias, with the measurement-error-adjusted estimator showing particular robustness in real-world applications. The results highlight the importance of accounting for sampling fraction bias in ecological analyses to ensure accurate inference. Conclusion Beyond the ecological fallacy uncovered by Robinson’s seminar work, our research identified another critical bias in ecological analysis that is likely just as prevalent and consequential. The proposed adjustment methods provide potential tools for researchers to adjust for this bias, thereby improving the validity of ecological inferences. This study underscores the need for caution when pooling aggregate measures from multiple sample datasets and offers potential solutions to enhance the reliability of ecological analyses in various research domains. Clinical trial number Not applicable.

• Publisher: BioMed Central,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Adjustment / Aggregate measures / Analysis / Bias / Causal analysis / Causality / Computer Simulation / Data Interpretation, Statistical / Data science methodologies / Datasets as Topic / Ecological analysis / Ecological research / Ecological studies / Ecology - methods / Health Sciences / Health Surveys / Humans / Kenya / Medical research / Medicine / Medicine & Public Health / Medicine, Experimental / Methods / Models, Statistical / Sample datasets / Sampling fraction bias / Set (Psychology) / Sociology / Statistical Theory and Methods / Statistics for Life Sciences / Technology application / Theory of Medicine/Bioethics