Explore the vast range of titles available.

Datasets comprised of large sets of both predictor and outcome variables are becoming more widely used in research. In addition to the well-known problems of model complexity and predictor variable selection, predictive modelling with such large data also presents a relatively novel and under-studied challenge of outcome variable selection. Certain outcome variables in the data may not be adequately predicted by the given sets of predictors. In this paper, we propose the method of Sparse Multivariate Principal Covariates Regression that addresses these issues altogether by expanding the Principal Covariates Regression model to incorporate sparsity penalties on both of predictor and outcome variables. Our method is one of the first methods that perform variable selection for both predictors and outcomes simultaneously. Moreover, by relying on summary variables that explain the variance in both predictor and outcome variables, the method offers a sparse and succinct model representation of the data. In a simulation study, the method performed better than methods with similar aims such as sparse Partial Least Squares at prediction of the outcome variables and recovery of the population parameters. Lastly, we administered the method on an empirical dataset to illustrate its application in practice.

We investigate high-dimensional nonconvex penalized regression, where the number of covariates may grow at an exponential rate. Although recent asymptotic theory established that there exists a local minimum possessing the oracle property under general conditions, it is still largely an open problem how to identify the oracle estimator among potentially multiple local minima. There are two main obstacles: (1) due to the presence of multiple minima, the solution path is nonunique and is not guaranteed to contain the oracle estimator; (2) even if a solution path is known to contain the oracle estimator, the optimal tuning parameter depends on many unknown factors and is hard to estimate. To address these two challenging issues, we first prove that an easy-to-calculate calibrated CCCP algorithm produces a consistent solution path which contains the oracle estimator with probability approaching one. Furthermore, we propose a high-dimensional BIC criterion and show that it can be applied to the solution path to select the optimal tuning parameter which asymptotically identifies the oracle estimator. The theory for a general class of nonconvex penalties in the ultra-high dimensional setup is established when the random errors follow the sub-Gaussian distribution. Monte Carlo studies confirm that the calibrated CCCP algorithm combined with the proposed high-dimensional BIC has desirable performance in identifying the underlying sparsity pattern for high-dimensional data analysis.

This article explores estimation and inference in a regression kink model with an unknown threshold. A regression kink model (or continuous threshold model) is a threshold regression constrained to be everywhere continuous with a kink at an unknown threshold. We present methods for estimation, to test for the presence of the threshold, for inference on the regression parameters, and for inference on the regression function. A novel finding is that inference on the regression function is nonstandard since the regression function is a nondifferentiable function of the parameters. We apply recently developed methods for inference on nondifferentiable functions. The theory is illustrated by an application to the growth and debt problem introduced by Reinhart and Rogoff, using their long-span time-series for the United States.

In some supervised learning settings, the practitioner might have additional information on the features used for prediction. We propose a new method that leverages this additional information for better prediction. The method, which we call the feature-weighted elastic net (“fwelnet”), uses these “features of features” to adapt the relative penalties on the feature coefficients in the elastic net penalty. In our simulations, fwelnet outperforms the lasso in terms of the test mean squared error, and usually gives an improvement in terms of the true positive rate or false positive rate for feature selection. We also compare this method with the group lasso and Bayesian estimation. Lastly, we apply the proposed method to the early prediction of preeclampsia, where fwelnet outperforms the lasso in terms of the 10-fold cross-validated area under the curve (0.84 vs. 0.80, respectively), and suggest how fwelnet might be used for multi-task learning.

We propose a Monte Carlo algorithm to sample from high dimensional probability distributions that combines Markov chain Monte Carlo and importance sampling. We provide a careful theoretical analysis, including guarantees on robustness to high dimensionality, explicit comparison with standard Markov chain Monte Carlo methods and illustrations of the potential improvements in efficiency. Simple and concrete intuition is provided for when the novel scheme is expected to outperform standard schemes. When applied to Bayesian variable-selection problems, the novel algorithm is orders of magnitude more efficient than available alternative sampling schemes and enables fast and reliable fully Bayesian inferences with tens of thousand regressors.

This paper proposes the hybrid system model identified by a PWARX (piecewise affine autoregressive exogenous) model for modeling human driving behavior. In the proposed model, the mode segmentation is carried out automatically and the optimal number of modes is decided by a novel methodology based on consistent variable selection. In addition, model flexibility is added within the ARX (autoregressive exogenous) partitions in the form of statistical variable selection. The proposed method is able to capture both the decision-making and motion-control facets of the driving behavior. The resulting model is an optimal basal model which is not affected by the choice of data, where the explanatory variables are allowed to vary within each ARX region, thus, allowing a higher-level understanding of the motion-control aspect of the driving behavior, as well as explaining the driver’s decision-making. The proposed model is applied to model the car-following driving task based on real-road driving data, as well as to ROS-CARLA-based car-following simulation and compared to Gipp’s driver model. Obtained results that show better performance both on prediction performance and mimicking actual real-road driving demonstrates and validates the usefulness of the model.

Variable selection for structured covariates lying on an underlying known graph is a problem motivated by practical applications, and has been a topic of increasing interest. However, most of the existing methods may not be scalable to high-dimensional settings involving tens of thousands of variables lying on known pathways such as the case in genomics studies. We propose an adaptive Bayesian shrinkage approach which incorporates prior network information by smoothing the shrinkage parameters for connected variables in the graph, so that the corresponding coefficients have a similar degree of shrinkage. We fit our model via a computationally efficient expectation maximization algorithm which scalable to highdimensional settings (p ~ 100,000). Theoretical properties for fixed as well as increasing dimensions are established, even when the number of variables increases faster than the sample size. We demonstrate the advantages of our approach in terms of variable selection, prediction, and computational scalability via a simulation study, and apply the method to a cancer genomics study.

I use longitudinal survey data from commercial fishing deckhands in the Alaskan Bering Sea to provide new insights on empirical methods commonly used to estimate compensating wage differentials and the value of statistical life (VSL). The unique setting exploits intertemporal variation in fatality rates and wages within worker-vessel pairs caused by a combination of weather patterns and policy changes, allowing identification of parameters and biases that it has only been possible to speculate about in more general settings. I show that estimation strategies common in the literature produce biased estimates in this setting, and decompose the bias components due to latent worker, establishment, and job-match heterogeneity. The estimates also remove the confounding effects of endogenous job mobility and dynamic labor market search, narrowing a conceptual gap between search-based hedonic wage theory and its empirical applications. I find that workers' marginal aversion to fatal risk falls as risk levels rise, which suggests complementarities in the benefits of public safety policies. Supplementary materials for this article are available online.

Variable selection methods have been extensively developed for and applied to cancer genomics data to identify important omics features associated with complex disease traits, including cancer outcomes. However, the reliability and reproducibility of the findings are in question if valid inferential procedures are not available to quantify the uncertainty of the findings. In this article, we provide a gentle but systematic review of high-dimensional frequentist and Bayesian inferential tools under sparse models which can yield uncertainty quantification measures, including confidence (or Bayesian credible) intervals, p values and false discovery rates (FDR). Connections in high-dimensional inferences between the two realms have been fully exploited under the “unpenalized loss function + penalty term” formulation for regularization methods and the “likelihood function × shrinkage prior” framework for regularized Bayesian analysis. In particular, we advocate for robust Bayesian variable selection in cancer genomics studies due to its ability to accommodate disease heterogeneity in the form of heavy-tailed errors and structured sparsity while providing valid statistical inference. The numerical results show that robust Bayesian analysis incorporating exact sparsity has yielded not only superior estimation and identification results but also valid Bayesian credible intervals under nominal coverage probabilities compared with alternative methods, especially in the presence of heavy-tailed model errors and outliers.

We consider the task of fitting a regression model involving interactions among a potentially large set of covariates, in which we wish to enforce strong heredity. We propose FAMILY , a very general framework for this task. Our proposal is a generalization of several existing methods, such as VANISH , hierNet , the all-pairs lasso, and the lasso using only main effects. It can be formulated as the solution to a convex optimization problem, which we solve using an efficient alternating directions method of multipliers (ADMM) algorithm. This algorithm has guaranteed convergence to the global optimum, can be easily specialized to any convex penalty function of interest, and allows for a straightforward extension to the setting of generalized linear models. We derive an unbiased estimator of the degrees of freedom of FAMILY , and explore its performance in a simulation study and on an HIV sequence dataset. Supplementary materials for this article are available online.