Journal Article

On distribution-weighted partial least squares with diverging number of highly correlated predictors

Zhu, Li-Ping, Zhu, Li-Xing

2009

Overview

Because highly correlated data arise from many scientific fields, we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated. For this, we first develop a distribution-weighted least squares estimator that can recover directions in the central subspace, then use the distribution-weighted least squares estimator as a seed vector and project it onto a Krylov space by partial least squares to avoid computing the inverse of the covariance of predictors. Thus, distrbution-weighted partial least squares can handle the cases with high dimensional and highly correlated predictors. Furthermore, we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares. For theoretical investigation, we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate O{n¹/²/ log (n)} and o(n¹/³) respectively where n is the sample size. When there are no other constraints on the covariance of predictors, the rates n¹/² and n¹/³ are optimal. We also propose a Bayesian information criterion type of criterion to estimate the dimension of the Krylov space in the partial least squares procedure. Illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non-ellipticity and works well even in 'small n-large p' problems.

• Publisher: Oxford, UK : Blackwell Publishing Ltd,Blackwell Publishing Ltd,Blackwell Publishing,Blackwell,Royal Statistical Society,Oxford University Press
• Subject: Algorithms / Analysis of covariance / Bayesian analysis / Central subspace / Collinearity / Consistent estimators / Constraints / Convergence / Correlation / covariance / Criteria / Data / data collection / Dimensionality / Dimensionality reduction / Distribution / Distribution function / Distribution theory / Eigenvalues / Estimating techniques / Estimation / Estimators / Exact sciences and technology / General topics / Grammatical aspect / Inverse regression / Investigations / Least squares / Least squares estimation / Least squares method / Linear inference, regression / Mathematics / Modeling / Normality / Parameter estimation / Partial least squares / Probability and statistics / Probability theory and stochastic processes / Random variables / Sample size / Sciences and techniques of general use / Simulation / Standard deviation / Statistical methods / Statistics / Studies