Explore the vast range of titles available.

Nowadays, High dimensional data are quickly increasing in many areas because of the development of new technology which helping to collect data with a large number of variables in order to better understanding for a given phenomenon of interest. Multiple Linear Regression is a famous technique used to investigate the relationship between one dependent variable and one or more of independent variables and analyzing the effects of them. Fitting this model requests assumptions, one of them is large sample size. High dimensional data does not satisfy this assumption because the sample size is small compared to the number of explanatory variables (k). Consequently, the results of traditional methods to estimate the model can be misleading. Regularization or shrinkage techniques (e.g., LASSO) have been proposed to estimate this model in this case. Nonparametric method was proposed to estimate this model. Average mean square error and root mean square error criteria are used to assess the performance of nonparametric; LASSO and OLS methods in the case of simulation study and analyzing the real dataset. The results of simulation study and the analysis of real data set show that nonparametric regression method is outperformance of LASSO and OLS methods to fit this model with high dimensional data.

Due to the inherent uncertainty of wind conditions as well as the price unpredictability in the competitive electricity market, wind power producers are exposed to the risk of concurrent fluctuations in both price and volume. Therefore, it is imperative to develop strategies to effectively stabilize their revenues, or cash flows, when trading wind power output in the electricity market. In light of this context, we present a novel endeavor to construct multivariate derivatives for mitigating the risk of fluctuating cash flows that are associated with trading wind power generation in electricity markets. Our approach involves leveraging nonparametric techniques to identify optimal payoff structures or compute the positions of derivatives with fine granularity, utilizing multiple underlying indexes including spot electricity price, area-wide wind power production index, and local wind conditions. These derivatives, referred to as mixed derivatives, offer advantages in terms of hedge effectiveness and contracting efficiency. Notably, we develop a methodology to enhance the hedge effects by modeling multivariate functions of wind speed and wind direction, incorporating periodicity constraints on wind direction via tensor product spline functions. By conducting an empirical analysis using data from Japan, we elucidate the extent to which the hedge effectiveness is improved by constructing mixed derivatives from various perspectives. Furthermore, we compare the hedge performance between high-granular (hourly) and low-granular (daily) formulations, revealing the advantages of utilizing a high-granular hedging approach.

Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve theminimax rates of convergence (up to log n-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constraints such as (generalized) additive models. While there is a lot of flexibility in the network architecture, the tuning parameter is the sparsity of the network. Specifically, we consider large networks with number of potential network parameters exceeding the sample size. The analysis gives some insights into why multilayer feedforward neural networks perform well in practice. Interestingly, for ReLU activation function the depth (number of layers) of the neural network architectures plays an important role, and our theory suggests that for nonparametric regression, scaling the network depth with the sample size is natural. It is also shown that under the composition assumption wavelet estimators can only achieve suboptimal rates.

This paper is a selective review of the regularization methods scattered in statistics literature. We introduce a general conceptual approach to regularization and fit most existing methods into it. We have tried to focus on the importance of regularization when dealing with today's high-dimensional objects: data and models. A wide range of examples are discussed, including nonparametric regression, boosting, covariance matrix estimation, principal component estimation, subsampling.[PUBLICATION ABSTRACT]

We develop and apply an approach for analyzing multi-curve data where each curve is driven by a latent state process. The state at any particular point determines a smooth function, forcing the individual curve to “switch” from one function to another. Thus each curve follows what we call a switching nonparametric regression model. We develop an EM algorithm to estimate the model parameters. We also obtain standard errors for the parameter estimates of the state process. We consider three types of hidden states: those that are independent and identically distributed, those that follow a Markov structure, and those that are independent but with distribution depending on some covariate(s). A simulation study shows the frequentist properties of our estimates. We apply our methods to a building’s power usage data. Les auteures développent et mettent en application une approche d’analyse de données multicourbes où chaque courbe est générée par un processus latent. L’état d’un point particulier détermine une fonction lisse, forçnt les courbes individuelles à passer d’une fonction à l’autre. Chaque courbe suit ainsi ce que les auteures appellent un modèle de régression non paramétrique intermittent. Elles développent un algorithme EM pour estimer les paramètres et obtiennent les erreur-types pour les estimateurs des paramètres du modèle d’états. Les auteures considèrent trois types d’états cachés: ceux qui sont indépendants et identiquement distribués, ceux qui suivent une structure de Markov, et ceux qui sont indépendants mais dont la distribution dépend de covariables. Elles présentent une simulation afin de montrer les propriétés fréquentistes de leurs estimateurs et appliquent leur méthode à des données réelles de consommation d’énergie de bâtiments.

Assuming that a smoothness condition and a suitable restriction on the structure of the regression function hold, it is shown that least squares estimates based on multilayer feedforward neural networks are able to circumvent the curse of dimensionality in nonparametric regression. The proof is based on new approximation results concerning multilayer feedforward neural networks with bounded weights and a bounded number of hidden neurons. The estimates are compared with various other approaches by using simulated data.

Recent results in nonparametric regression show that deep learning, that is, neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper, we show that we can get similar results also for least squares estimates based on simple fully connected neural networks with ReLU activation functions. Here, either the number of neurons per hidden layer is fixed and the number of hidden layers tends to infinity suitably fast for sample size tending to infinity, or the number of hidden layers is bounded by some logarithmic factor in the sample size and the number of neurons per hidden layer tends to infinity suitably fast for sample size tending to infinity. The proof is based on new approximation results concerning deep neural networks.

Decision tree ensembles are an extremely popular tool for obtaining high-quality predictions in nonparametric regression problems. Unmodified, however, many commonly used decision tree ensemble methods do not adapt to sparsity in the regime in which the number of predictors is larger than the number of observations. A recent stream of research concerns the construction of decision tree ensembles that are motivated by a generative probabilistic model, the most influential method being the Bayesian additive regression trees (BART) framework. In this article, we take a Bayesian point of view on this problem and show how to construct priors on decision tree ensembles that are capable of adapting to sparsity in the predictors by placing a sparsity-inducing Dirichlet hyperprior on the splitting proportions of the regression tree prior. We characterize the asymptotic distribution of the number of predictors included in the model and show how this prior can be easily incorporated into existing Markov chain Monte Carlo schemes. We demonstrate that our approach yields useful posterior inclusion probabilities for each predictor and illustrate the usefulness of our approach relative to other decision tree ensemble approaches on both simulated and real datasets. Supplementary materials for this article are available online.

Kernel ridge regression (KRR) is a standard method for performing nonparametric regression over reproducing kernel Hilbert spaces. Given n samples, the time and space complexity of computing the KRR estimate scale as 𝓞(n3) and 𝓞(n2), respectively, and so is prohibitive in many cases. We propose approximations of KRR based on m-dimensional randomized sketches of the kernel matrix, and study how small the projection dimension m can be chosen while still preserving minimax optimality of the approximate KRR estimate. For various classes of randomized sketches, including those based on Gaussian and randomized Hadamard matrices, we prove that it suffices to choose the sketch dimension m proportional to the statistical dimension (modulo logarithmic factors). Thus, we obtain fast and minimax optimal approximations to the KRR estimate for nonparametric regression. In doing so, we prove a novel lower bound on the minimax risk of kernel regression in terms of the localized Rademacher complexity.

Standard regression approaches assume that some finite number of the response distribution characteristics, such as location and scale, change as a (parametric or nonparametric) function of predictors. However, it is not always appropriate to assume a location/scale representation, where the error distribution has unchanging shape over the predictor space. In fact, it often happens in applied research that the distribution of responses under study changes with predictors in ways that cannot be reasonably represented by a finite dimensional functional form. This can seriously affect the answers to the scientific questions of interest, and therefore more general approaches are indeed needed. This gives rise to the study of fully nonparametric regression models. We review some of the main Bayesian approaches that have been employed to define probability models where the complete response distribution may vary flexibly with predictors. We focus on developments based on modifications of the Dirichlet process, historically termed dependent Dirichlet processes, and some of the extensions that have been proposed to tackle this general problem using nonparametric approaches.