Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
1,248
result(s) for
"multiple comparisons"
Sort by:
Multiple Comparisons Using R
Controlling multiplicity in experiments ensures better decision making and safeguards against false claims. Adopting a unifying theme based on maximum statistics, this self-contained introduction describes the common underlying theory of multiple comparison procedures through numerous examples. It covers a range of multiple comparison procedures, from the Bonferroni method and Simes' test to resampling and adaptive design methods. The book also presents a detailed description of available software implementations in R. The R packages and source code for the analyses are available at http://CRAN.R-project.org.
eBook
The Control of the False Discovery Rate in Multiple Testing under Dependency
Benjamini and Hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems. A simple procedure was given there as an FDR controlling procedure for independent test statistics and was shown to be much more powerful than comparable procedures which control the traditional familywise error rate. We prove that this same procedure also controls the false discovery rate when the test statistics have positive regression dependency on each of the test statistics corresponding to the true null hypotheses. This condition for positive dependency is general enough to cover many problems of practical interest, including the comparisons of many treatments with a single control, multivariate normal test statistics with positive correlation matrix and multivariate t. Furthermore, the test statistics may be discrete, and the tested hypotheses composite without posing special difficulties. For all other forms of dependency, a simple conservative modification of the procedure controls the false discovery rate. Thus the range of problems for which a procedure with proven FDR control can be offered is greatly increased.
Journal Article
Dataset decay and the problem of sequential analyses on open datasets
Open data allows researchers to explore pre-existing datasets in new ways. However, if many researchers reuse the same dataset, multiple statistical testing may increase false positives. Here we demonstrate that sequential hypothesis testing on the same dataset by multiple researchers can inflate error rates. We go on to discuss a number of correction procedures that can reduce the number of false positives, and the challenges associated with these correction procedures.
Journal Article
Beyond Bonferroni: Less conservative analyses for conservation genetics
Studies in conservation genetics often attempt to determine genetic differentiation between two or more temporally or geographically distinct sample collections. Pairwise p-values from Fisher’s exact tests or contingency Chi-square tests are commonly reported with a Bonferroni correction for multiple tests. While the Bonferroni correction controls the experiment-wise α, this correction is very conservative and results in greatly diminished power to detect differentiation among pairs of sample collections. An alternative is to control the false discovery rate (FDR) that provides increased power, but this method only maintains experiment-wise α when none of the pairwise comparisons are significant. Recent modifications to the FDR method provide a moderate approach to determining significance level. Simulations reveal that critical values of multiple comparison tests with both the Bonferroni method and a modified FDR method approach a minimum asymptote very near zero as the number of tests gets large, but the Bonferroni method approaches zero much more rapidly than the modified FDR method. I compared pairwise significance from three published studies using three critical values corresponding to Bonferroni, FDR, and modified FDR methods. Results suggest that the modified FDR method may provide the most biologically important critical value for evaluating significance of population differentiation in conservation genetics.␣Ultimately, more thorough reporting of statistical significance is needed to allow interpretation of biological significance of genetic differentiation among populations.
Journal Article
An Algorithm for a Letter-Based Representation of All-Pairwise Comparisons
All-pairwise comparisons among a set of t treatments or groups are one of the most frequent tasks in applied statistics. Users of statistical software are accustomed to the familiar lines display, in which treatments that do not differ significantly, are connected by a common line or letter. Availability of the lines display is restricted mainly to the balanced analysis of variance setup. This limited availability is at stark variance with the diversity of statistical methods and models, which call for multiple comparisons. This article describes a general method for graphically representing any set of t(t−1)/2 all-pairwise significance statements (p values) for t treatments by a familiar letter display, which is applicable regardless of the underlying data structure or the statistical method used for comparisons. The method reproduces the familiar lines display in case of the balanced analysis of variance. Its broad applicability is demonstrated using data from an international multienvironment wheat yield trial and from a fish catching survey.
Journal Article
Multiple Comparison Procedures for Exponential Mean Lifetimes Compared with Several Controls
Under heteroscedasticity, we propose one-stage multiple comparison procedures for several treatment groups compared with several control groups in terms of exponential mean lifetimes. The simultaneous confidence intervals including one-sided and two-sided confidence intervals for the difference between the mean lifetime from the i-th treatment group and the mean lifetime from the j-th control group are developed in this research. The required critical values are obtained and tabulated for the practical use of users. The experimenters can use these simultaneous confidence intervals to determine whether the treatment mean lifetimes are better than several controls or worse than several controls under a specified confidence level. Finally, one example of comparing the mean duration of remission using four drugs for treating leukemia is used for the aims of illustrations.
Journal Article
When to adjust alpha during multiple testing
Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically (mindlessly) assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses.
Journal Article
Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test
Multiple comparisons tests (MCTs) include the statistical tests used to compare groups (treatments) often following a significant effect reported in one of many types of linear models. Due to a variety of data and statistical considerations, several dozen MCTs have been developed over the decades, with tests ranging from very similar to each other to very different from each other. Many scientific disciplines use MCTs, including >40,000 reports of their use in ecological journals in the last 60 years. Despite the ubiquity and utility of MCTs, several issues remain in terms of their correct use and reporting. In this study, we evaluated 17 different MCTs. We first reviewed the published literature for recommendations on their correct use. Second, we created a simulation that evaluated the performance of nine common MCTs. The tests examined in the simulation were those that often overlapped in usage, meaning the selection of the test based on fit to the data is not unique and that the simulations could inform the selection of one or more tests when a researcher has choices. Based on the literature review and recommendations: planned comparisons are overwhelmingly recommended over unplanned comparisons, for planned non-parametric comparisons the Mann-Whitney-Wilcoxon U test is recommended, Scheffé’s S test is recommended for any linear combination of (unplanned) means, Tukey’s HSD and the Bonferroni or the Dunn-Sidak tests are recommended for pairwise comparisons of groups, and that many other tests exist for particular types of data. All code and data used to generate this paper are available at: https://github.com/stevemidway/MultipleComparisons .
Journal Article