Explore the vast range of titles available.

A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

Analysis of multiple secondary outcomes in a clinical trial leads to an increased probability of at least one false significant result among all secondary outcomes studied. In this paper, we question the notion that that if no multiplicity adjustment has been applied to multiple secondary outcome analyses in a clinical trial, then they must necessarily be regarded as exploratory. Instead, we argue that if individual secondary outcome results are interpreted carefully and precisely, there is no need to downgrade our interpretation to exploratory. This is because the probability of a false significant result for each comparison, the per-comparison wise error rate, does not increase with multiple testing. Strong effects on secondary outcomes should always be taken seriously and must not be dismissed purely on the basis of multiplicity concerns.

Cumulative meta-analyses are prone to produce spurious P < 0.05 because of repeated testing of significance as trial data accumulate. Information size in a meta-analysis should at least equal the sample size of an adequately powered trial. Trial sequential analysis (TSA) corresponds to group sequential analysis of a single trial and may be applied to meta-analysis to evaluate the evidence. Six randomly selected neonatal meta-analyses with at least five trials reporting a binary outcome were examined. Low-bias heterogeneity-adjusted information size and information size determined from an assumed intervention effect of 15% were calculated. These were used for constructing trial sequential monitoring boundaries. We assessed the cumulative z-curves' crossing of P = 0.05 and the boundaries. Five meta-analyses showed early potentially spurious P < 0.05 values. In three significant meta-analyses the cumulative z-curves crossed both boundaries, establishing firm evidence of an intervention effect. In two nonsignificant meta-analyses the cumulative z-curves crossed P = 0.05, but never the boundaries, demonstrating early potentially spurious P < 0.05 values. In one nonsignificant meta-analysis the cumulative z-curves never crossed P = 0.05 or the boundaries. TSAs may establish when firm evidence is reached in meta-analysis.

The Mantel test, based on comparisons of distance matrices, is commonly employed in comparative biology, but its statistical properties in this context are unknown. Here, we evaluate the performance of the Mantel test for two applications in comparative biology: testing for phylogenetic signal, and testing for an evolutionary correlation between two characters. We find that the Mantel test has poor performance compared to alternative methods, including low power and, under some circumstances, inflated type-l error. We identify a remedy for the inflated type-l error of three-way Mantel tests using phylogenetic permutations; however, this test still has considerably lower power than independent contrasts. We recommend that use of the Mantel test should be restricted to cases in which data can only be expressed as pairwise distances among taxa.

One of the most important statistical analyses when designing animal and human studies is the calculation of the required sample size. In this review, we define central terms in the context of sample size determination, including mean, standard deviation, statistical hypothesis testing, type I/II error, power, direction of effect, effect size, expected attrition, corrected sample size, and allocation ratio. We also provide practical examples of sample size calculations for animal and human studies based on pilot studies, larger studies similar to the proposed study—or if no previous studies are available—estimated magnitudes of the effect size per Cohen and Sawilowsky.

A response to Reinhart & Rinella (2016) and Rinella & Reinhart (2017) ‘A common soil handling technique can generate incorrect estimates of soil biota effects on plants’ and ‘Mixing soil samples across experi- mental units ignores uncertainty and generates incorrect estimates of soil biota effects on plants’

Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically (mindlessly) assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses.

Identifying species interactions and detecting when ecological communities are structured by them is an important problem in ecology and biogeography. Ecologists have developed specialized statistical hypothesis tests to detect patterns indicative of community-wide processes in their field data. In this respect, null model approaches have proved particularly popular. The freedom allowed in choosing the null model and statistic to construct a hypothesis test leads to a proliferation of possible hypothesis tests from which ecologists can choose to detect these processes. Here, we point out some serious shortcomings of a popular approach to choosing the best hypothesis for the ecological problem at hand that involves benchmarking different hypothesis tests by assessing their performance on artificially constructed data sets. Terminological errors concerning the use of Type I and Type II errors that underlie these approaches are discussed. We argue that the key benchmarking methods proposed in the literature are not a sound guide for selecting null hypothesis tests, and further, that there is no simple way to benchmark null hypothesis tests. Surprisingly, the basic problems identified here do not appear to have been addressed previously, and these methods are still being used to develop and test new null models and summary statistics, from quantifying community structure (e.g., nestedness and modularity) to analyzing ecological networks.

Background Null Hypothesis Significance Testing (NHST) has been well criticised over the years yet remains a pillar of statistical inference. Although NHST is well described in terms of statistical models, most textbooks for non-statisticians present the null and alternative hypotheses ( H 0 and H A , respectively) in terms of differences between groups such as ( μ 1  =  μ 2 ) and ( μ 1  ≠  μ 2 ) and H A is often stated to be the research hypothesis. Here we use propositional calculus to analyse the internal logic of NHST when couched in this popular terminology. The testable H 0 is determined by analysing the scope and limits of the P -value and the test statistic’s probability distribution curve. Results We propose a minimum axiom set NHST in which it is taken as axiomatic that H 0 is rejected if P -value< α . Using the common scenario of the comparison of the means of two sample groups as an example, the testable H 0 is {( μ 1  =  μ 2 ) and [( x ¯ 1 ≠ x ¯ 2 ) due to chance alone]}. The H 0 and H A pair should be exhaustive to avoid false dichotomies. This entails that H A is ¬{( μ 1  =  μ 2 ) and [( x ¯ 1 ≠ x ¯ 2 ) due to chance alone]}, rather than the research hypothesis ( H T ). To see the relationship between H A and H T , H A can be rewritten as the disjunction H A : ({( μ 1  =  μ 2 ) ∧ [( x ¯ 1 ≠ x ¯ 2 ) not due to chance alone]} ∨ {( μ 1  ≠  μ 2 ) ∧ [ ( x ¯ 1 ≠ x ¯ 2 ) not due to ( μ 1  ≠  μ 2 ) alone]} ∨ {( μ 1   ≠   μ 2 ) ∧ [( x ¯ 1 ≠ x ¯ 2 ) due to ( μ 1   ≠   μ 2 ) alone]} ). This reveals that H T (the last disjunct in bold) is just one possibility within H A . It is only by adding premises to NHST that H T or other conclusions can be reached. Conclusions Using this popular terminology for NHST, analysis shows that the definitions of H 0 and H A differ from those found in textbooks. In this framework, achieving a statistically significant result only justifies the broad conclusion that the results are not due to chance alone, not that the research hypothesis is true. More transparency is needed concerning the premises added to NHST to rig particular conclusions such as H T . There are also ramifications for the interpretation of Type I and II errors, as well as power, which do not specifically refer to H T as claimed by texts.

The use of linear mixed effects models (LMMs) is increasingly common in the analysis of biological data. Whilst LMMs offer a flexible approach to modelling a broad range of data types, ecological data are often complex and require complex model structures, and the fitting and interpretation of such models is not always straightforward. The ability to achieve robust biological inference requires that practitioners know how and when to apply these tools. Here, we provide a general overview of current methods for the application of LMMs to biological data, and highlight the typical pitfalls that can be encountered in the statistical modelling process. We tackle several issues regarding methods of model selection, with particular reference to the use of information theory and multi-model inference in ecology. We offer practical solutions and direct the reader to key references that provide further technical detail for those seeking a deeper understanding. This overview should serve as a widely accessible code of best practice for applying LMMs to complex biological problems and model structures, and in doing so improve the robustness of conclusions drawn from studies investigating ecological and evolutionary questions.