Asset Details
Do you wish to reserve the book?
SINGULARITY, MISSPECIFICATION AND THE CONVERGENCE RATE OF EM
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
SINGULARITY, MISSPECIFICATION AND THE CONVERGENCE RATE OF EM
Do you wish to request the book?
SINGULARITY, MISSPECIFICATION AND THE CONVERGENCE RATE OF EM
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
SINGULARITY, MISSPECIFICATION AND THE CONVERGENCE RATE OF EM
2020
Overview
A line of recent work has analyzed the behavior of the Expectation-Maximization (EM) algorithm in the well-specified setting, in which the population likelihood is locally strongly concave around its maximizing argument. Examples include suitably separated Gaussian mixture models and mixtures of linear regressions. We consider over-specified settings in which the number of fitted components is larger than the number of components in the true distribution. Such mis-specified settings can lead to singularity in the Fisher information matrix, and moreover, the maximum likelihood estimator based on n i.i.d. samples in d dimensions can have a nonstandard
O
(
(
d
/
n
)
1
4
)
rate of convergence. Focusing on the simple setting of two-component mixtures fit to a d-dimensional Gaussian distribution, we study the behavior of the EM algorithm both when the mixture weights are different (unbalanced case), and are equal (balanced case). Our analysis reveals a sharp distinction between these two cases: in the former, the EM algorithm converges geometrically to a point at Euclidean distance of
O
(
(
d
/
n
)
1
2
)
from the true parameter, whereas in the latter case, the convergence rate is exponentially slower, and the fixed point has a much lower
O
(
(
d
/
n
)
1
4
)
accuracy. Analysis of this singular case requires the introduction of some novel techniques: in particular, we make use of a careful form of localization in the associated empirical process, and develop a recursive argument to progressively sharpen the statistical rate.
Publisher
Institute of Mathematical Statistics
Subject
