Explore the vast range of titles available.

The γ-FDP and k-FWER multiple testing error metrics, which are tail probabilities of the respective error statistics, have become popular recently as alternatives to the FDR and FWER. We propose general and flexible stepup and stepdown procedures for testing multiple hypotheses about sequential (or streaming) data that simultaneously control both the type I and II versions of γ-FDP, or k-FWER. The error control holds regardless of the dependence between data streams, which may be of arbitrary size and shape. All that is needed is a test statistic for each data stream that controls the conventional type I and II error probabilities, and no information or assumptions are required about the joint distribution of the statistics or data streams. The procedures can be used with sequential, group sequential, truncated, or other sampling schemes. We give recommendations for the procedures' implementation including closed-form expressions for the needed critical values in some commonly-encountered testing situations. The proposed sequential procedures are compared with each other and with comparable fixed sample size procedures in the context of strongly positively correlated Gaussian data streams. For this setting we conclude that both the stepup and stepdown sequential procedures provide substantial savings over the fixed sample procedures in terms of expected sample size, and the stepup procedure performs slightly but consistently better than the stepdown for γ-FDP control, with the relationship reversed for k-FWER control.

The probability of false discovery proportion (FDP) exceeding γ∈ [0,1), defined as γ-FDP, has received much attention as a measure of false discoveries in multiple testing. Although this measure has received acceptance due to its relevance under dependency, not much progress has been made yet advancing its theory under such dependency in a nonasymptotic setting, which motivates our research in this article. We provide a larger class of procedures containing the stepup analog of, and hence more powerful than, the stepdown procedure in Lehmann and Romano [Ann. Statist. 33 (2005) 1138-1154] controlling the γ-FDP under similar positive dependence condition assumed in that paper. We offer better alternatives of the stepdown and stepup procedures in Romano and Shaikh [IMS Lecture Notes Monogr. Ser. 49 (2006a) 33-50, Ann. Statist. 34 (2006b) 1850-1873] using pairwise joint distributions of the null p-values. We generalize the notion of γ-FDP making it appropriate in situations where one is willing to tolerate a few false rejections or, due to high dependency, some false rejections are inevitable, and provide methods that control this generalized γ-FDP in two different scenarios: (i) only the marginal p-values are available and (ii) the marginal p-values as well as the common pairwise joint distributions of the null p-values are available, and assuming both positive dependence and arbitrary dependence conditions on the p-values in each scenario. Our theoretical findings are being supported through numerical studies.

Consider the multiple testing problem of testing null hypotheses$H_{1},\\ldots ,H_{S}$. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. But if s is large, control of the FWER is so stringent that the ability of a procedure that controls the FWER to detect false null hypotheses is limited. It is therefore desirable to consider other measures of error control. This article considers two generalizations of the FWER. The first is the k-FWER, in which one is willing to tolerate k or more false rejections for some fixed k ≥ 1. The second is based on the false discovery proportion (FDP), defined to be the number of false rejections divided by the total number of rejections (and defined to be 0 if there are no rejections). Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300] proposed control of the false discovery rate (FDR), by which they meant that, for fixed α, E (FDP) ≤ α. Here, we consider control of the FDP in the sense that, for fixed γ and α, P{FDP > γ} ≤ α. Beginning with any nondecreasing sequence of constants and p-values for the individual tests, we derive stepup procedures that control each of these two measures of error control without imposing any assumptions on the dependence structure of the p-values. We use our results to point out a few interesting connections with some closely related stepdown procedures. We then compare and contrast two FDP-controlling procedures obtained using our results with the stepup procedure for control of the FDR of Benjamini and Yekutieli [Ann. Statist. 29 (2001) 1165-1188].

The concept of k-FWER has received much attention lately as an appropriate error rate for multiple testing when one seeks to control at least k false rejections, for some fixed k ≥ 1. A less conservative notion, the k-FDR, has been introduced very recently by Sarkar [Ann. Statist. 34 (2006) 394-415], generalizing the false discovery rate of Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300]. In this article, we bring newer insight to the k-FDR considering a mixture model involving independent p-values before motivating the developments of some new procedures that control it. We prove the k-FDR control of the proposed methods under a slightly weaker condition than in the mixture model. We provide numerical evidence of the proposed methods' superior power performance over some k-FWER and k-FDR methods. Finally, we apply our methods to a real data set.

In a multiple testing problem where one is willing to tolerate a few false rejections, procedure controlling the familywise error rate (FWER) can potentially be improved in terms of its ability to detect false null hypotheses by generalizing it to control the k-FWER, the probability of falsely rejecting at least k null hypotheses, for some fixed k > 1. Simes' test for testing the intersection null hypothesis is generalized to control the k-FWER weakly, that is, under the intersection null hypothesis, and Hochberg's stepup procedure for simultaneous testing of the individual null hypotheses is generalized to control the k-FWER strongly, that is, under any configuration of the true and false null hypotheses. The proposed generalizations are developed utilizing joint null distributions of the k-dimensional subsets of the p-values, assumed to be identical. The generalized Simes' test is proved to control the k-FWER weakly under the multivariate totally positive of order two (MTP₂) condition [J. Multivariate Analysis 10 (1980) 467-498] of the joint null distribution of the p-values by generalizing the original Simes' inequality. It is more powerful to detect k or more false null hypotheses than the original Simes' test when the p-values are independent. A stepdown procedure strongly controlling the k-FWER, a version of generalized Holm's procedure that is different from and more powerful than [Ann. Statist. 33 (2005) 1138-1154] with independent p-values, is derived before proposing the generalized Hochberg's procedure. The strong control of the k-FWER for the generalized Hochberg's procedure is established in situations where the generalized Simes' test is known to control its k-FWER weakly.

Consider the multiple testing problem of testing k null hypotheses, where the unknown family of distributions is assumed to satisfy a certain monotonicity assumption. Attention is restricted to procedures that control the familywise error rate in the strong sense and which satisfy a monotonicity condition. Under these assumptions, we prove certain maximin optimality results for some well-known stepdown and stepup procedures.

Testing multiple null hypotheses in two stages to decide which of these can be rejected or accepted at the first stage and which should be followed up for further testing having had additional observations is of importance in many scientific studies. We develop two procedures, each with two different combination functions, Fisher's and Simes’, to combine p -values from two stages, given prespecified boundaries on the first-stage p -values in terms of the false discovery rate (FDR) and controlling the overall FDR at a desired level. The FDR control is proved when the pairs of first- and second-stage p -values are independent and those corresponding to the null hypotheses are identically distributed as a pair (p ₁, p ₂) satisfying the p -clud property. We did simulations to show that (1) our two-stage procedures can have significant power improvements over the first-stage Benjamini–Hochberg (BH) procedure compared to the improvement offered by the ideal BH procedure that one would have used had the second stage data been available for all the hypotheses, and can continue to control the FDR under some dependence situations, and (2) can offer considerable cost savings compared to the ideal BH procedure. The procedures are illustrated through a real gene expression data. Supplementary materials for this article are available online.