Explore the vast range of titles available.

We present a new method for post-selection inference for ℓ₁ (lasso)-penalized likelihood models, including generalized regression models. Our approach generalizes the post-selection framework presented in Lee et al. (2013). The method provides P-values and confidence intervals that are asymptotically valid, conditional on the inherent selection done by the lasso. We present applications of this work to (regularized) logistic regression, Cox’s proportional hazards model, and the graphical lasso. We do not provide rigorous proofs here of the claimed results, but rather conceptual and theoretical sketches. Les auteurs présentent une nouvelle méthode d’inférence post-sélection pour les modèles de vraisemblance avec une pénalité ℓ₁(lasso). Leur approche généralise le cadre d’inférence post-sélection de Lee et coll. (2013). Leur méthode génère des p-values et des intervalles de confiance qui sont asymptotiquement valides conditionnellement à la sélection inhérente au lasso. Les auteurs présentent une application de ces résultats à la régression logistique (régularisée), au modèle à risques proportionnels de Cox et au lasso graphique. Ils ne présentent pas de preuves rigoureuses des résultats avanés, mais plutôt une esquisse conceptuelle et théorique.

Statistical hypothesis testing has been widely criticized by ecologists in recent years. I review some of the more persistent criticisms of P values and argue that most stem from misunderstandings or incorrect interpretations, rather than from intrinsic shortcomings of the P value. I show that P values are intimately linked to confidence intervals and to differences in Akaike's information criterion (ΔAIC), two metrics that have been advocated as replacements for the P value. The choice of a threshold value of ΔAIC that breaks ties among competing models is as arbitrary as the choice of the probability of a Type I error in hypothesis testing, and several other criticisms of the P value apply equally to ΔAIC. Since P values, confidence intervals, and ΔAIC are based on the same statistical information, all have their places in modern statistical practice. The choice of which to use should be stylistic, dictated by details of the application rather than by dogmatic, a priori considerations.

Ecologists frequently ask questions that are best addressed with a model comparison approach. Under this system, the merit of several models is considered without necessarily requiring that (1) models are nested, (2) one of the models is true, and (3) only current data be used. This is in marked contrast to the pragmatic blend of Neyman-Pearson and Fisherian significance testing conventionally emphasized in biometric texts (Christensen 2005), in which (1) just two hypotheses are under consideration, representing a pairwise comparison of models, (2) one of the models, H sub(0), is assumed to be true, and (3) a single data set is used to quantify evidence concerning H sub(0).

Multiple-hypothesis testing involves guarding against much more complicated errors than single-hypothesis testing. Whereas we typically control the type I error rate for a single-hypothesis test, a compound error rate is controlled for multiple-hypothesis tests. For example, controlling the false discovery rate FDR traditionally involves intricate sequential p-value rejection methods based on the observed data. Whereas a sequential p-value method fixes the error rate and estimates its corresponding rejection region, we propose the opposite approach-we fix the rejection region and then estimate its corresponding error rate. This new approach offers increased applicability, accuracy and power. We apply the methodology to both the positive false discovery rate pFDR and FDR, and provide evidence for its benefits. It is shown that pFDR is probably the quantity of interest over FDR. Also discussed is the calculation of the q-value, the pFDR analogue of the p-value, which eliminates the need to set the error rate beforehand as is traditionally done. Some simple numerical examples are presented that show that this new approach can yield an increase of over eight times in power compared with the Benjamini-Hochberg FDR method.

In randomized experiments, Fisher-exact P values are available and should be used to help evaluate results rather than the more commonly reported asymptotic P values. One reason is that using the latter can effectively alter the question being addressed by including irrelevant distributional assumptions. The Fisherian statistical framework, proposed in 1925, calculates a P value in a randomized experiment by using the actual randomization procedure that led to the observed data. Here, we illustrate this Fisherian framework in a crossover randomized experiment. First, we consider the first period of the experiment and analyze its data as a completely randomized experiment, ignoring the second period; then, we consider both periods. For each analysis, we focus on 10 outcomes that illustrate important differences between the asymptotic and Fisher tests for the null hypothesis of no ozone effect. For some outcomes, the traditional P value based on the approximating asymptotic Student’s t distribution substantially subceeded the minimum attainable Fisher-exact P value. For the other outcomes, the Fisher-exact null randomization distribution substantially differed from the bell-shaped one assumed by the asymptotic t test. Our conclusions: When researchers choose to report P values in randomized experiments, 1) Fisher-exact P values should be used, especially in studies with small sample sizes, and 2) the shape of the actual null randomization distribution should be examined for the recondite scientific insights it may reveal.

Background In medical research and practice, the p -value is arguably the most often used statistic and yet it is widely misconstrued as the probability of the type I error, which comes with serious consequences. This misunderstanding can greatly affect the reproducibility in research, treatment selection in medical practice, and model specification in empirical analyses. By using plain language and concrete examples, this paper is intended to elucidate the p -value confusion from its root, to explicate the difference between significance and hypothesis testing, to illuminate the consequences of the confusion, and to present a viable alternative to the conventional p -value. Main text The confusion with p -values has plagued the research community and medical practitioners for decades. However, efforts to clarify it have been largely futile, in part, because intuitive yet mathematically rigorous educational materials are scarce. Additionally, the lack of a practical alternative to the p -value for guarding against randomness also plays a role. The p -value confusion is rooted in the misconception of significance and hypothesis testing. Most, including many statisticians, are unaware that p -values and significance testing formed by Fisher are incomparable to the hypothesis testing paradigm created by Neyman and Pearson. And most otherwise great statistics textbooks tend to cobble the two paradigms together and make no effort to elucidate the subtle but fundamental differences between them. The p -value is a practical tool gauging the “strength of evidence” against the null hypothesis. It informs investigators that a p -value of 0.001, for example, is stronger than 0.05. However, p -values produced in significance testing are not the probabilities of type I errors as commonly misconceived. For a p -value of 0.05, the chance a treatment does not work is not 5%; rather, it is at least 28.9%. Conclusions A long-overdue effort to understand p -values correctly is much needed. However, in medical research and practice, just banning significance testing and accepting uncertainty are not enough. Researchers, clinicians, and patients alike need to know the probability a treatment will or will not work. Thus, the calibrated p -values (the probability that a treatment does not work) should be reported in research papers.

Increasing attention has been drawn to the misuse of statistical methods over recent years, with particular concern about the prevalence of practices such as poor experimental design, cherry picking and inadequate reporting. These failures are largely unintentional and no more common in ecology than in other scientific disciplines, with many of them easily remedied given the right guidance. Originating from a discussion at the 2020 International Statistical Ecology Conference, we show how ecologists can build their research following four guiding principles for impactful statistical research practices: (1) define a focussed research question, then plan sampling and analysis to answer it; (2) develop a model that accounts for the distribution and dependence of your data; (3) emphasise effect sizes to replace statistical significance with ecological relevance; and (4) report your methods and findings in sufficient detail so that your research is valid and reproducible. These principles provide a framework for experimental design and reporting that guards against unsound practices. Starting with a well‐defined research question allows researchers to create an efficient study to answer it, and guards against poor research practices that lead to poor estimation of the direction, magnitude, and uncertainty of ecological relationships, and to poor replicability. Correct and appropriate statistical models give sound conclusions. Good reporting practices and a focus on ecological relevance make results impactful and replicable. Illustrated with two examples—an experiment to study the impact of disturbance on upland wetlands, and an observational study on blue tit colouring—this paper explains the rationale for the selection and use of effective statistical practices and provides practical guidance for ecologists seeking to improve their use of statistical methods. Resumen (Spanish) La atención acerca del uso incorrecto de métodos estadísticos en ecología y áreas relacionadas ha aumentado mucho en los últimos años, con especial énfasis en la incidencia elevada de malas prácticas en el diseño de los estudios, así como también en el reporte inadecuado de los hallazgos, incluyendo resultados parciales o sesgados. Estos fallos son en gran parte involuntarios y, en general, podrían prevenirse con la instrucción adecuada. Con orígen en un taller de trabajo en el marco de la Conferencia Internacional de Ecología Estadística en 2020, proponemos una serie de cuatro principios orientadores para el desarrollo de investigaciones con base en buenas prácticas estadísticas. Estas pautas son: (1) Definir una pregunta de investigación bien enfocada, para luego planificar el muestreo y el análisis de datos para darle respuesta; (2) Concebir un modelo estadístico que contemple la distribución y las estructuras de dependencia en los datos; (3) Enfatizar la interpretación de los resultados en base a la magnitud de los efectos, y reemplazar la significancia estadística tradicional con la relevancia en términos ecológicos; y (4) Reportar los métodos y los hallazgos con suficiente detalle para que éstos puedan ser reproducibles y la investigación pueda validarse. Comenzar con una pregunta bien definida permite diseñar un estudio eficiente para su respuesta, en tanto que la ejecución de análisis estadísticos apropiados es fundamental para extraer información sólida y confiable. Por otro lado, la interpretación en base a la relevancia ecológica confiere mayor impacto y utilidad a los hallazgos, y el reporte adecuado de todos los procedimientos genera transparencia y permite replicación del estudio. La implementación de estas prácticas permite evitar fallas comunes que, en consecuencia, llevan a evaluaciones pobres de la magnitud, dirección, e incertidumbre asociada a los efectos ecológicos que se pretenden elucidar. En suma, estos principios proporcionan un marco para el diseño y conducción de estudios en base a buenas prácticas. Ilustrado con dos ejemplos—un experimento que evalúa el impacto de perturbaciones en humedales de tierras altas, y un estudio observacional acerca de la coloración del Herrerillo Común—este artículo argumenta cómo la incorporación los cuatro principios llevará a la ejecución de prácticas de investigación efectivas, y representa una guía práctica para mejorar el uso y aplicación de métodos estadísticos en Ecología y áreas afines. Resumo (Portuguese) Nos últimos anos assistimos a uma cada vez maior chamada de atenção para a utilização inadequada de métodos estatísticos, em particular no que diz respeito a práticas comuns como delineamentos experimentais deficientes, escolha seletiva de resultados ou a inadequada apresentação de resultados. Estas falhas são geralmente não intencionais, e muitas podem ser evitadas com formação adequada. No seguimento de uma sessão de Discussão na Conferencia Internacional de Ecologia Estatística de 2020, sugerimos que os ecólogos podem estruturar a sua investigação suportados por 4 princípios básicos para desenvolverem práticas de investigação estatística impactantes: (1) Definir uma questão de investigação objetiva, e planear a amostragem e a análise para lhe poder responder; (2) Desenvolver um modelo que incorpora a distribuição e as estruturas de dependência dos dados recolhidos; (3) Focar em tamanhos de efeitos para substituir o foco, da significância estatística, para a relevância ecológica; (4) Reportar os métodos e os resultados com detalhe suficiente para que a investigação seja válida e possível de reproduzir. Estes princípios fornecem um paradigma para delineamento experimental e formas de comunicação que nos protegem contra práticas duvidosas. Partir de uma questão bem definida ajuda os investigadores a delinear um estudo que a possa responder, e protege contra más práticas comuns que levam a uma estimação inadequada da direção, magnitude das, e incerteza associada às, relações ecológicas de interesse, e consequentes resultados difíceis ou impossíveis de replicar. Modelos estatísticos adequados geram conclusões sólidas. Boas práticas no reportar dos resultados e o foco na relevância ecológica geram resultados impactantes e replicáveis. Suportado por dois exemplos reais—uma experiência para avaliar o impacto de perturbações em zonas húmidas de terras altas, e um estudo observacional sobre coloração em chapim azul—este artigo apresenta uma estratégia para a seleção e utilização de metodologias estatísticas eficientes e providencia conselhos práticos para ecólogos que pretendam melhorar a sua utilização de análises estatísticas. Zusammenfassung (German) In den letzten Jahren wurde zunehmend auf den Missbrauch statistischer Methoden aufmerksam gemacht. Solch Missbrauch beinhaltet Praktiken wie schlechtes Versuchsdesign, Rosinenpickerei und unzureichende Berichterstattung. Diese Fehler sind größtenteils unbeabsichtigt und kommen in der Ökologie nicht häufiger vor als in anderen wissenschaftlichen Disziplinen. Viele von diesen Fehlern können mit der richtigen Anleitung einfach behoben werden. Ausgehend von einer Diskussion an der International Statistical Ecology Conference 2020 zeigen wir, wie Ökologen ihre Forschung nach vier Leitprinzipien für wirksame statistische Forschungspraktiken aufbauen können: (1) Definiere eine fokussierte Forschungsfrage und plane dann die Stichprobenziehung und Datenanalyse zu ihrer Beantwortung; (2) Entwickle ein Modell, das die Verteilung und Abhängigkeiten der Daten berücksichtigt; (3) Betone die Effektgrößen, um statistische Signifikanz durch ökologische Relevanz zu ersetzen; und (4) Gib die Methoden und Ergebnisse ausreichend detailliert an, damit die Forschungsergebnisse gültig und reproduzierbar sind. Diese Prinzipien bilden einen Rahmen für die Gestaltung und Berichterstattung von Studien, der gegen unlautere Praktiken Schutz bietet. Eine klar definierten Forschungsfrage ermöglicht es Forschern eine effiziente Studie zur Beantwortung dieser Frage zu erstellen, und schützt vor schlechten Forschungspraktiken. Schlechte Forschungspraktiken führen zu einer schlechten Schätzung der Richtung, des Ausmaßes und der Unsicherheit ökologischer Verhältnisse, und zu schlechter Reproduzierbarkeit. Richtige und geeignete statistische Modelle hingegen liefern fundierte Schlussfolgerungen. Gute Berichterstattungspraktiken und Konzentration auf ökologische Relevanz sorgen für wirkungsvolle Ergebnisse, die reproduzierbar sind. Wir illustrieren das Rahmenwerk mit zwei Beispielen: einem Experiment zur Untersuchung der Auswirkungen von Störungen auf Feuchtgebiete und eine Beobachtungsstudie zur Blaumeisenfärbung. Dieser Artikel erklärt die Gründe für die Auswahl und Verwendung effektiver statistischer Verfahren und liefert eine praktische Anleitung für Ökologen, die ihre Verwendung statistischer Methoden verbessern möchten. (Polish) W ostatnich latach coraz więcej uwagi przywiązuje się do błędnego wykorzystania metod statystycznych, ze szczególnym naciskiem na stosowanie tak wątpliwych technik, jak niewłaściwe planowanie eksperymentów, selektywne raportowanie wyników czy brak transparentności w publikowaniu wyników analiz. Problemy te nie wynikają z reguły z intencjonalnych działań, a ich występowanie w badaniach ekologicznych nie różni się zbytnio od innych dziedzin. Wiele z nich można łatwo wyeliminować stosując się do pewnych podstawowych zasad. Na podstawie dyskusji zapoczątkowanej w 2020 podczas Międzynarodowej Konferencji Ekologii Statystycznej (ISEC) formułujemy zestaw czterech rekomendacji, które mogą pomóc ekologom raportować wyniki analiz statystycznych w bardziej transparentny i poprawny sposób. Proponowane cztery zasady to: (1) Najpierw zaprojektuj dobrze zdefiniowane pytanie badawcze, dopiero potem zaplanuj strategię próbkowania i analizy zebranych danych; (2) Stwórz model statystyczny biorący pod uwagę rozkład twoich danych i obecne w nich zależności; (3) Połóż silniejszy nacisk na wielkości efektów, a nie istotność statystyczną—pozwoli to lepiej uwypukl

The purpose of this paper is to propose methodologies for statistical inference of low dimensional parameters with high dimensional data. We focus on constructing confidence intervals for individual coefficients and linear combinations of several of them in a linear regression model, although our ideas are applicable in a much broader context. The theoretical results that are presented provide sufficient conditions for the asymptotic normality of the proposed estimators along with a consistent estimator for their finite dimensional covariance matrices. These sufficient conditions allow the number of variables to exceed the sample size and the presence of many small non‐zero coefficients. Our methods and theory apply to interval estimation of a preconceived regression coefficient or contrast as well as simultaneous interval estimation of many regression coefficients. Moreover, the method proposed turns the regression data into an approximate Gaussian sequence of point estimators of individual regression coefficients, which can be used to select variables after proper thresholding. The simulation results that are presented demonstrate the accuracy of the coverage probability of the confidence intervals proposed as well as other desirable properties, strongly supporting the theoretical results.

In the sparse linear regression setting, we consider testing the significance of the predictor variable that enters the current lasso model, in the sequence of models visited along the lasso solution path. We propose a simple test statistic based on lasso fitted values, called the covariance test statistic, and show that when the true model is linear, this statistic has an Exp(1) asymptotic distribution under the null hypothesis (the null being that all truly active variables are contained in the current lasso model). Our proof of this result for the special case of the first predictor to enter the model (i.e., testing for a single significant predictor variable against the global null) requires only weak assumptions on the predictor matrix X. On the other hand, our proof for a general step in the lasso path places further technical assumptions on X and the generative model, but still allows for the important high-dimensional case p>n, and does not necessarily require that the current lasso model achieves perfect recovery of the truly active variables. Of course, for testing the significance of an additional variable between two nested linear models, one typically uses the chi-squared test, comparing the drop in residual sum of squares (RSS) to a $\\chi _1^2$ distribution. But when this additional variable is not fixed, and has been chosen adaptively or greedily, this test is no longer appropriate: adaptivity makes the drop in RSS stochastically much larger than $\\chi _1^2$ under the null hypothesis. Our analysis explicitly accounts for adaptivity, as it must, since the lasso builds an adaptive sequence of linear models as the tuning parameter λ decreases. In this analysis, shrinkage plays a key role: though additional variables are chosen adaptively, the coefficients of lasso active variables are shrunken due to the l₁ penalty. Therefore, the test statistic (which is based on lasso fitted values) is in a sense balanced by these two opposing properties—adaptivity and shrinkage—and its null distribution is tractable and asymptotically Exp(1).

The false discovery rate (FDR) is a multiple hypothesis testing quantity that describes the expected proportion of false positive results among all rejected null hypotheses. Benjamini and Hochberg introduced this quantity and proved that a particular step-up p-value method controls the FDR. Storey introduced a point estimate of the FDR for fixed significance regions. The former approach conservatively controls the FDR at a fixed predetermined level, and the latter provides a conservatively biased estimate of the FDR for a fixed predetermined significance region. In this work, we show in both finite sample and asymptotic settings that the goals of the two approaches are essentially equivalent. In particular, the FDR point estimates can be used to define valid FDR controlling procedures. In the asymptotic setting, we also show that the point estimates can be used to estimate the FDR conservatively over all significance regions simultaneously, which is equivalent to controlling the FDR at all levels simultaneously. The main tool that we use is to translate existing FDR methods into procedures involving empirical processes. This simplifies finite sample proofs, provides a framework for asymptotic results and proves that these procedures are valid even under certain forms of dependence.