Stochastic Systems Group
Home Research Group Members Programs  
Demos Calendar Publications Mission Statement Alumni

SSG Seminar Abstract

Covariance, Subspace, and Intrinsic Cramer-Rao Bounds

Dr. Steven T. Smith
Technical Staff, MIT Lincoln Laboratory

Algorithms and systems analysis for signal detection, location, and classification all rely on covariance-based methods. But how well understood is the important problem of covariance matrix estimation? What does "accuracy" mean in the context of covariance matrices? This talk addresses these questions at their deepest level, and provides powerful new tools and insights, as well as well as some startling surprises. The covariance matrix problem is framed as an intrinsic estimation problem on the space of positive definite (covariance) matrices, which has the structure of a homogeneous or quotient space, not a vector space - the necessary setting for classical Cramer-Rao bounds. Covariance matrix estimation accuracy bounds are derived from an intrinsic derivation of the Cramer-Rao bound on arbitrary Riemannian manifolds (another new development), and compared to the accuracy achieved by standard methods involving the sample covariance matrix (SCM). Estimator efficiency is discussed from different, novel, viewpoints. Remarkably, it is shown that that from an intrinsic perspective, the SCM is a biased and inefficient estimator; the bias corresponds to the SCM's poor estimation quality at low sample support - this contradicts the well-known fact that E[SCM] = R because the linear expectation operator implicitly treats the covariance matrices as points in a real vector space, compared to the intrinsic treatment of positive-definite Hermitian matrices used in this talk. The accuracy bound on unbiased covariance matrix estimators is shown to be about (10/log10)*n/sqrt(K) decibels, where n is the matrix order and K is the sample support. Thus a connection is established between estimation loss for covariance matrices, and the well-known Reed-Mallett-Brennan detection loss in adaptive filtering problems. Simple, closed-form expressions for all results are presented, and compared to numerical evidence based on Monte Carlo simulations. The analysis approach developed is directly applicable to many other estimation problems on manifolds encountered in signal processing and elsewhere, such as estimating rotation matrices in computer vision and estimating subspace basis vectors in blind source separation. Finally, several intriguing open questions pertaining to the possibility of improved covariance matrix estimation methods at low sample support are presented.

Biography: Steven Thomas Smith was born in La Jolla, CA in 1963. He received the B.A.Sc. degree in electrical engineering and mathematics from the University of British Columbia, Vancouver, BC in 1986 and the Ph.D. degree in applied mathematics from Harvard University, Cambridge, MA in 1993. From 1986 to 1988 he was a research engineer at ERIM, Ann Arbor, MI, where he developed morphological image processing algorithms. He is currently a senior member of the technical staff at MIT Lincoln Laboratory, which he joined in 1993. His research involves algorithms for adaptive signal processing, detection, and tracking to enhance radar and sonar systems performance. He has taught signal processing courses at Harvard and for the IEEE. His most recent work addresses intrinsic estimation and superresolution bounds, mean and variance CFAR, advanced tracking methods, and space-time adaptive processing algorithms. He was an associate editor of the IEEE Transactions on Signal Processing (2000-2002), and received the SIAM outstanding paper award in 2001.

Problems with this site should be emailed to