Explore the vast range of titles available.

We consider the problem of A-B testing when the impact of the treatment is marred by a large number of covariates. Randomization can be highly inefficient in such settings, and thus we consider the problem of optimally allocating test subjects to either treatment with a view to maximizing the precision of our estimate of the treatment effect. Our main contribution is a tractable algorithm for this problem in the online setting, where subjects arrive, and must be assigned, sequentially, with covariates drawn from an elliptical distribution with finite second moment. We further characterize the gain in precision afforded by optimized allocations relative to randomized allocations, and show that this gain grows large as the number of covariates grows. Our dynamic optimization framework admits several generalizations that incorporate important operational constraints such as the consideration of selection bias, budgets on allocations, and endogenous stopping times. In a set of numerical experiments, we demonstrate that our method simultaneously offers better statistical efficiency and less selection bias than state-of-the-art competing biased coin designs. This paper was accepted by Noah Gans, stochastic models and simulation .

In this paper, we introduce evidence propagation operations on influence diagrams, a concept of the value of evidence to measure the impact/value of new observations/experimentation, and a concept of the value of revelation. Evidence propagation operations are critical for the computation of the value of evidence, general update and inference operations in normative expert systems that are based on the influence diagram (generalized Bayesian network) paradigm. The value of evidence allows us to compute the outcome sensitivity directly defined as the maximum difference among the values of evidence, and the value of perfect information, as the expected value of the values of evidence. We define the value of revelation as the optimal value of the values of evidence. We discuss the relationship between the value of revelation and the value of control. We also discuss implementation issues related to computation of the value of evidence and the value of perfect information.

We may learn from our mistakes, but Deborah Mayo argues that, where experimental knowledge is concerned, we haven't begun to learn enough. Error and the Growth of Experimental Knowledge launches a vigorous critique of the subjective Bayesian view of statistical inference, and proposes Mayo's own error-statistical approach as a more robust framework for the epistemology of experiment. Mayo genuinely addresses the needs of researchers who work with statistical analysis, and simultaneously engages the basic philosophical problems of objectivity and rationality. Mayo has long argued for an account of learning from error that goes far beyond detecting logical inconsistencies. In this book, she presents her complete program for how we learn about the world by being \"shrewd inquisitors of error, white gloves off.\" Her tough, practical approach will be important to philosophers, historians, and sociologists of science, and will be welcomed by researchers in the physical, biological, and social sciences whose work depends upon statistical analysis.

Expert opinion and judgment enter into the practice of statistical inference and decision-making in numerous ways. Indeed, there is essentially no aspect of scientific investigation in which judgment is not required. Judgment is necessarily subjective, but should be made as carefully, as objectively, and as scientifically as possible. Elicitation of expert knowledge concerning an uncertain quantity expresses that knowledge in the form of a (subjective) probability distribution for the quantity. Such distributions play an important role in statistical inference (for example as prior distributions in a Bayesian analysis) and in evidence-based decision-making (for example as expressions of uncertainty regarding inputs to a decision model). This article sets out a number of practices through which elicitation can be made as rigorous and scientific as possible. One such practice is to follow a recognized protocol that is designed to address and minimize the cognitive biases that experts are prone to when making probabilistic judgments. We review the leading protocols in the field, and contrast their different approaches to dealing with these biases through the medium of a detailed case study employing the SHELF protocol. The article ends with discussion of how to elicit a joint probability distribution for multiple uncertain quantities, which is a challenge for all the leading protocols. Supplementary materials for this article are available online.

Large-scale group decision-making (LSGDM) deals with complex decision- making problems which involve a large number of decision makers (DMs). Such a complex scenario leads to uncertain contexts in which DMs elicit their knowledge using linguistic information that can be modelled using different representations. However, current processes for solving LSGDM problems commonly neglect a key concept in many real-world decision-making problems, such as DMs’ regret aversion psychological behavior. Therefore, this paper introduces a novel consensus based linguistic distribution LSGDM (CLDLSGDM) approach based on a statistical inference principle that considers DMs’ regret aversion psychological characteristics using regret theory and which aims at obtaining agreed solutions. Specifically, the CLDLSGDM approach applies the statistical inference principle to the consensual information obtained in the consensus process, in order to derive the weights of DMs and attributes using the consensus matrix and adjusted decision-making matrices to solve the decision-making problem. Afterwards, by using regret theory, the comprehensive perceived utility values of alternatives are derived and their ranking determined. Finally, a performance evaluation of public hospitals in China is given as an example in order to illustrate the implementation of the designed method. The stability and advantages of the designed method are analyzed by a sensitivity and a comparative analysis.

It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid \"post-selection inference\" by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all submodels. By purchasing \"simultaneity insurance\" for all possible submodels, the resulting post-selection inference is rendered universally valid under all possible model selection procedures. This inference is therefore generally conservative for particular selection procedures, but it is always less conservative than full Scheffé protection. Importantly it does not depend on the truth of the selected submodel, and hence it produces valid inference even in wrong models. We describe the structure of the simultaneous inference problem and give some asymptotic results.

Haavelmo (1944) proposed a probabilistic structure for econometric modeling, aiming to make econometrics useful for decision making. His fundamental contribution has become thoroughly embedded in econometric research, yet it could not answer all the deep issues that the author raised. Notably, Haavelmo struggled to formalize the implications for decision making of the fact that models can at most approximate actuality. In the same period, Wald (1939, 1945) initiated his own seminal development of statistical decision theory. Haavelmo favorably cited Wald, but econometrics did not embrace statistical decision theory. Instead, it focused on study of identification, estimation, and statistical inference. This paper proposes use of statistical decision theory to evaluate the performance of models in decision making. I consider the common practice of as-if optimization: specification of a model, point estimation of its parameters, and use of the point estimate to make a decision that would be optimal if the estimate were accurate. A central theme is that one should evaluate as-if optimization or any other model-based decision rule by its performance across the state space, listing all states of nature that one believes feasible, not across the model space. I apply the theme to prediction and treatment choice. Statistical decision theory is conceptually simple, but application is often challenging. Advancing computation is the primary task to complete the foundations sketched by Haavelmo and Wald.

Inference and decision making under uncertainty are key processes in every autonomous system and numerous robotic problems. In recent years, the similarities between inference and decision making triggered much work, from developing unified computational frameworks to pondering about the duality between the two. In spite of these efforts, inference and control, as well as inference and belief space planning (BSP) are still treated as two separate processes. In this paper we propose a paradigm shift, a novel approach which deviates from conventional Bayesian inference and utilizes the similarities between inference and BSP. We make the key observation that inference can be efficiently updated using predictions made during the decision making stage, even in light of inconsistent data association between the two. We developed a two staged process that implements our novel approach and updates inference using calculations from the precursory planning phase. Using autonomous navigation in an unknown environment along with iSAM2 efficient methodologies as a test case, we benchmarked our novel approach against standard Bayesian inference, both with synthetic and real-world data (KITTI dataset). Results indicate that not only our approach improves running time by at least a factor of two while providing the same estimation accuracy, but it also alleviates the computational burden of state dimensionality and loop closures.

Monitoring the impacts of anthropogenic threats and interventions to mitigate these threats is key to understanding how to best conserve biodiversity. Ecologists use many different study designs to monitor such impacts. Simpler designs lacking controls (e.g. Before–After (BA) and After) or pre‐impact data (e.g. Control–Impact (CI)) are considered to be less robust than more complex designs (e.g. Before–After Control‐Impact (BACI) or Randomized Controlled Trials (RCTs)). However, we lack quantitative estimates of how much less accurate simpler study designs are in ecology. Understanding this could help prioritize research and weight studies by their design's accuracy in meta‐analysis and evidence assessment. We compared how accurately five study designs estimated the true effect of a simulated environmental impact that caused a step‐change response in a population's density. We derived empirical estimates of several simulation parameters from 47 ecological datasets to ensure our simulations were realistic. We measured design performance by determining the percentage of simulations where: (a) the true effect fell within the 95% Confidence Intervals of effect size estimates, and (b) each design correctly estimated the true effect's direction and magnitude. We also considered how sample size affected their performance. We demonstrated that BACI designs performed: 1.3–1.8 times better than RCTs; 2.9–4.2 times versus BA; 3.2–4.6 times versus CI; and 7.1–10.1 times versus After designs (depending on sample size), when correctly estimating true effect's direction and magnitude to within ±30%. Although BACI designs suffered from low power at small sample sizes, they outperformed other designs for almost all performance measures. Increasing sample size improved BACI design accuracy, but only increased the precision of simpler designs around biased estimates. Synthesis and applications. We suggest that more investment in more robust designs is needed in ecology since inferences from simpler designs, even with large sample sizes may be misleading. Facilitating this requires longer‐term funding and stronger research–practice partnerships. We also propose ‘accuracy weights’ and demonstrate how they can weight studies in three recent meta‐analyses by accounting for study design and sample size. We hope these help decision‐makers and meta‐analysts better account for study design when assessing evidence. Foreign Language Japanese 生物多様性の保全を効果的に行うためには、人為的脅威の影響や保全対策の効果を適切に評価することが重要となる。生態学ではこのような評価を行うために、様々な研究デザインが用いられている。対照区が存在しないBefore‐After (BA)デザインやAfterデザイン、また処理以前のデータが存在しないControl‐Impact (CI)デザインなど簡素な研究デザインは、Before‐After Control‐Impact (BACI)デザインやランダム化比較試験（RCTs: Randomized Controlled Trials)などの複雑なデザインよりも頑健さに劣ると考えられている。しかしながら、生態学においてこれら簡素な研究デザインがどれだけ正確度に劣るのか、定量的な評価はこれまで行われていない。研究デザインの正確度を定量的に評価することで、メタ解析やエビデンスの評価を行う際に、用いられた研究デザインの正確度に基づいて各研究の優先順位付けや重み付けを行うことが可能になるだろう。 本研究では、環境変化が個体群密度に及ぼす影響を、5種類の研究デザインがどれだけ正確に推定することができるのか、シミュレーションを用いて検討した。より現実に即した状況を再現するため、シミュレーションで用いたパラメータは、47の生態学的データから抽出した。各研究デザインの正確度は、シミュレーションにおいて、（１）推定された効果サイズの95％信頼区間に真の効果が含まれる割合、（２）推定された効果が真の効果の方向・程度と一致した割合、を算出することによって評価した。またサンプルサイズの違いが各研究デザインの正確度に及ぼす影響も検討した。 シミュレーションの結果、BACIデザインはランダム化比較試験に対して1.3–1.8倍、BAデザインに対して2.9–4.2倍、CIデザインに対して3.2–4.6倍、Afterデザインに比較すると7.1–10.1倍も正確に真の効果を推定できる（推定された効果が真の効果の方向と一致し、且つ真の効果の ± 30%内に含まれる）ことが明らかになった（比較値のばらつきはサンプルサイズによる）。BACIデザインの正確度はサンプルサイズが小さい場合には低下したが、それでもほとんどの指標において他のデザインよりも高い正確度を示していた。サンプルサイズを増やすことでBACIデザインの正確度は向上したが、他の研究デザインでは偏った推定値の精度が向上するだけであった。 Synthesis and applications. 例えサンプルサイズが十分であったとしても、簡素なデザインに基づいた推論は正確でない可能性があるため、生態学においてもより頑健な研究デザインの利用を推進していく必要があると考えられる。頑健な研究デザインの利用を推進するためには、長期に渡る研究資金の確保や、研究と実践の間でのより強固な連携が必要となるだろう。本研究では更にこれらの結果に基づいて、メタ解析において研究デザインとサンプルサイズに基づいて各研究の重み付けをする手法を提案し、近年行われた３つのメタ解析を用いてその実用例を提示した。これらの結果は、意思決定者やメタ解析を行う研究者が、研究デザインを考慮したエビデンスの評価を行うために有用となるだろう。 We suggest that more investment in more robust designs is needed in ecology since inferences from simpler designs, even with large sample sizes may be misleading. Facilitating this requires longer‐term funding and stronger research–practice partnerships. We also propose ‘accuracy weights’ and demonstrate how they can weight studies in three recent meta‐analyses by accounting for study design and sample size. We hope these help decision‐makers and meta‐analysts better account for study design when assessing evidence.

We examined features of citizen science that influence data quality, inferential power, and usefulness in ecology. As background context for our examination, we considered topics such as ecological sampling (probability based, purposive, opportunistic), linkage between sampling technique and statistical inference (design based, model based), and scientific paradigms (confirmatory, exploratory). We distinguished several types of citizen science investigations, from intensive research with rigorous protocols targeting clearly articulated questions to mass-participation internet-based projects with opportunistic data collection lacking sampling design, and examined overarching objectives, design, analysis, volunteer training, and performance. We identified key features that influence data quality: project objectives, design and analysis, and volunteer training and performance. Projects with good designs, trained volunteers, and professional oversight can meet statistical criteria to produce high-quality data with strong inferential power and therefore are well suited for ecological research objectives. Projects with opportunistic data collection, little or no sampling design, and minimal volunteer training are better suited for general objectives related to public education or data exploration because reliable statistical estimation can be difficult or impossible. In some cases, statistically robust analytical methods, external data, or both may increase the inferential power of certain opportunistically collected data. Ecological management, especially by government agencies, frequently requires data suitable for reliable inference. With standardized protocols, state-of-the-art analytical methods, and well-supervised programs, citizen science can make valuable contributions to conservation by increasing the scope of species monitoring efforts. Data quality can be improved by adhering to basic principles of data collection and analysis, designing studies to provide the data quality required, and including suitable statistical expertise, thereby strengthening the science aspect of citizen science and enhancing acceptance by the scientific community and decision makers. Examinamos las características de la ciencia ciudadana que influyen sobre la calidad de datos, el poder inferencial, y la utilidad en la ecología. Consideramos temas como el muestreo ecológico (basado en probabilidad, deliberado, oportunista), la conexión entre la técnica de muestreo y la inferencia estadística (basada en diseño, basada en modelo) y los paradigmas científicos (confirmatorio, exploratorio) como trasfondo contextual para nuestra evaluación. Distinguimos varios tipos de investigación de ciencia ciudadana, desde investigación intensiva con protocolos rigurosos enfocados en preguntas claramente articuladas hasta proyectos de participación masiva en plataformas de internet con recolección de datos oportunistas carentes de un diseño de muestreo, y examinamos los objetivos generales, el diseño, el análisis, y la preparación de los voluntarios y el desempeño. Identificamos características clave que influyen sobre la calidad de los datos: los objetivos del proyecto, el diseño y el análisis, y la preparación y el desempeño de los voluntarios. Los proyectos con buenos diseños, voluntarios preparados, y supervisión profesional pueden cumplir con criterios estadísticos para producir datos de alta calidad con un fuerte poder inferencial, y por lo tanto son muy adecuados para los objetivos de investigación ecológica. Los proyectos con una recolección oportunista de datos, un diseño de muestreo ínfimo o nulo, y una preparación mínima de los voluntarios son más adecuados para los objetivos generales relacionados con la educación pública o la exploración de datos ya que la estimación estadística confiable puede ser complicada o imposible. En algunos casos los métodos analíticos estadísticamente sólidos, los datos externos, o ambos, pueden incrementar el poder inferencial de ciertos datos recolectados de manera oportunista. El manejo ecológico, en especial el que realizan las agencias gubernamentales, requiere frecuentemente de datos apropiados para una inferencia confiable. Con protocolos estandarizados, métodos analíticos modernos, y programas supervisados correctamente, la ciencia ciudadana puede contribuir de forma valiosa a la conservación al incrementar el alcance de los esfuerzos de monitoreo para una especie. La calidad de datos puede mejorarse si se adhiere a los principios básicos de la recolección y análisis de datos, se diseñan los estudios para que proporcionen la calidad requerida de datos, y si se incluye una pericia estadística adecuada, fortaleciendo así el aspecto científico de la ciencia ciudadana y aumentando su aceptación dentro de la comunidad científica y con quienes toman las decisiones. 本研究分析了生态学中影响数据质量.、推论统计效カ和有用性的公民科学的特征。检验的背景包括如生 态学抽样(基于概率的抽样、目的抽样、机会抽样X 抽样技术与统计推断(基于设计或基于模型) 的联系，以及 科学范式(验怔性或探索性) 等话题。我们区分出不同类型的公民科学调查，从有清晰明确的问题及严格实验规 范的深入研究，到缺少抽样设计、投机型数据收集的基于互联网的大规模参与项目；并研究了项目的总体目标、 设计、分析、志愿者培训和实现情况。本研究确定了影响数据质量的关键特征，包括项目目标、设计和分析、 志愿者培训和实现情況。拥有良好的设计、训练有素的志愿者和专业监督的项目通常符合统计学标准，能获得 推论统计效カ强的高质量数据，因此可以达到生态学研究目标。而投机型数据收集、很少或没有进行抽样设计 且志愿者培训非常有限的项目，更适合与公共教育或数据探索相关的一般性目标，因为它们很难或不可能进行可 靠的统计估计。在一些情况下，统计上強健的分析方法和 外部数据也会増加某些投机型数据的推论效力。 生态管理特别是来自政府机构的管理，常常需要那些适合进行可靠统计推论的数据。当采用标准化的实验规 范、最先进的分析方法和良好监督的程序时，公民科学可以扩大物种监测范围，为保护做出重要贡献。坚持数据 收集和分析的基本原则、设计研究方案来提供所需的高质量数据，并恰当运用统计学知识，能够增强数据质置 从而加强公民科学的科学性，提高科学界和决策者对公民科学的接受度。