Nonparametric Statistics Bibliography
This is a bibliography regarding nonparametric density estimators and
nonparametric statistics. In particular methods based on kernel
density estimates are emphasized.

Nonparametric Density Estimators

ParzenRosenblatt Density Estimation

General

E. Parzen,
On estimation of a probability density function and mode,
Annals of Mathematical Statistics 33 (1962), 10651076

Murray Rosenblatt,
Remarks on some nonparametric estimates of a density function,
Annals of Mathematical Statistics 27 (1956), no. 3, 832837

Consistency

M.O. Ioffe and V.Ya. Katkovnik,
Necessary and sufficient conditions of almost sure convergence
of kernel estimates of probability density and its derivatives,
Avtomatika i Telemekhanika 47 (1986), no. 12, 3342

V.N. Vapnik and A.R. Stefanyuk,
Nonparametric methods for reconstructing probability densities
Avtomatika i Telemekhanika 39 (1978), no. 8, 3852

Bandwidth Selection
Some discussion with regards to ML and crossvalidation is found in
[devroye87, p.126]
while
[fukunaga90, p.~261]
discusses bandwidth selection for a special class of densities. A more theoretic
discussion is found in
[devroye96, ch. 25]

Luc Devroye,
A course in density estimation,
Progress in Probability and Statistics, vol. 14, Birkhauser, Boston, 1987.

Luc Devroye,
A probabilistic theory of pattern recognition,
Applications of Mathematics, vol.31, SpringerVerlag, New York, 1996

Keinosuke Fukunaga,
Introduction to statistical pattern recognition,
Computer Science and Scientific Computing, Academic Press, Inc. Boston,
1990.

Peter Hall and J.S. Marron,
Local minima in crossvalidation functions,
Journal of the Royal Statistical Society. Series B 53 (1991),
no. 1, 245252

Peter Hall, Simon Sheather, M.C. Jones, and J.S. Marron,
On optimal databased bandwidth selection in kernel density estimation,
Biometrika, 78 (1991), no. 2, 263269

Wolfgang Hardle, J.S. Marron, and Wand M.C.,
Bandwidth choice for density derivatives
Journal of the Royal Statistical Society. Series B 52 (1990),
no. 1, 223232

J.S. Marron,
An asymptotically efficient solution to the bandwidth problem of
kernel density estimation,
Annals of Statistics 13 (1985), no. 3, 10111023.

J.S. Marron,
A comparison of crossvalidation techniques in density estimation,
Annals of Statistics 15 (1987), no. 1, 152162.

Byeong U. Park and J.S. Marron,
Comparison of datadriven bandwidth selectors,
Journal of the American Statistical Association 84 (1990),
no. 409, 6672.

S.J. Sheather and M.C. Jones,
A reliable databased bandwidth selection method for kernel density
estimation,
Journal of the Royal Statistical Society. Series B 53 (1991),
no. 3, 683690.

kNN Nearest Neighbor Estimation
This includes the (hard to find) reports which originally propose the
kNN density estimate [1,2] as well as a journal
article which also derives the consistency of the estimator
[3]. General consistency results are derived in
[4] and [5].

General

E. Fix and L.J. Hodges,
Discriminatory analysis, nonparametric disctrimination, consistency
properties,
Report 4, School of Aviation Medicine, Randolph Field, Texas, 1951,
Project 2149004

E. Fix and L.J. Hodges,
Nonparametric discrimination small sample performance,
Report 11, School of Aviation Medicine, Randolph Field, Texas, 1952,
Project 2149004

D.O. Loftsgaarden and C.P. Quesenberry,
A nonparametric estimate of a multivariate density function,
Annals of Mathematical Statistics 36 (1965), no. 3, 10491051.

Daid S. Moore and James W. Yackel,
Consistency properties of nearest neighbor density function estimators,
Annals of Statistics 5 (1977), no. 1, 143154.

Charles J. Stone,
Consistend nonparametric regression,
Annals of Statistics 5 (1977), no. 4, 595620.

Applications to Discriminant Analysis/Pattern Recognition
KNN asymptotics relative to Bayes error [3]
[6, p.100,103,123].
Parzen classifier asymptotics relative to Bayes error
[7] [8].
Convergence of Bayes error estimates
[2] [4] [5, p. 150, theorem 10.1] [7] [11].

Niall H. Anderson, Peter Hall, and D.M. Titterington,
Twosample test statistics for measuring descrepancies between two
multivariate probability density functions using kernelbased
density estimates,
Journal of Multivariate Analysis 50 (1994), 4154.

A. Antos, L. Devroye, and L. Gyorfi,
Lower Bounds for Bayes Error Estimation,
IEEE Transactions on Pattern Analysis and Machine Intelligence
vol. 21, no. 7, Jul. 1999, pp. 643645.

Thomas M. Cover and Peter E. Hart,
Nearest neighbor pattern classification,
IEEE Transactions on Information Theory IT13 (1967), no. 1,
2127

L. Devroye,
Automatic Pattern Recognition: A Study of the Probability of Error,
IEEE Transactions on Pattern Analysis and Machine Intelligence
vol. 10, no. 4, Jul. 1988, pp. 530543.

Luc Devroye, Lazlo Gyorfi, and Gabor Lugosi,
A probabilistic theory of pattern recognition,
Applications of Mathematics, vol. 31, SpringerVerlag, New York, 1996.

R.O. Duda and P.E. Hart,
Pattern classification and scene analysis,
John Wiley and Sons, 1973.

Stanley C. Fralick and Richard W. Scott,
Nonparametric bayesrisk estimation,
IEEE Transactions on Information Theory IT17 (1971),
no. 4, 440444.

K. Fukunaga and D. Hummels,
LeaveOneOut Procedures for Nonparametric Error Estimates,
IEEE Transactions on Pattern Analysis and Machine Intelligence
vol. 11, no. 4, Apr. 1989, pp. 421423.

Rudolf Gruubel,
Estimation of density functionals,
Ann. Inst. Statist. Math. 46 (1994), no. 1, 6775.

A. Jain, R. Duin, and J. Mao,
Statistical Pattern Recognition: A Review,
IEEE Transactions on Pattern Analysis and Machine Intelligence
vol. 17, no. 1, Jan. 2000, pp. 437.

Y. Yang,
Minimax Nonparametric Classification  Part I: Rates of Convergence,
IEEE Transactions on Information Theory,
vol. 45, no. 7, Nov. 1999, pp. 22712284.

Nonparametric Estimation of Entropy
Recent survey of nonparametric entropy estimators [2].
Law of large numbers estimator is proposed in [1] although
the proof is incorrect. The proof might be corrected by showing that
the denominator term which is erroneously moved outside of the
integral converges to a constant (using a Lorentz series). An integral
approach is proposed in [6]. A better discusion is found
in [5].

General

Ibrahim A. Ahmad and PiErh Lin,
A nonparametric estimation of the entropy for absolutely continuous
distributions,
IEEE Transactions on Information Theory 22 (1976), no. 3,
372375.

J. Beirlant, E.J. Dudewicz, L. Gyorfi, and E.C. van der Meulen,
Nonparametric entropy estimation: An overview,
Intern. J. Math. Stat. Sci. 6 (1997), no. 1, 1739

Georges A. Darbellay and Igor Vajda,
Estimation of the information by an adaptive partitioning of the
observation space,
IEEE Transactions on Information Theory 45 (1999), no. 4,
13151321.

Paul P.B. Eggermont and Vincent N. LaRiccia,
Best asymptotic normality of the kernel density entropy estimator
for smooth densities,
IEEE Transactions on Information Theory 45 (1999), no. 4,
13211327.

Harry Joe,
Estimation of entropy and other functionals of a multivariate
density,
An. Inst. Statist. Math. 41 (1989), no. 4, 683697.

Abdelkader Mokkadem,
Estimation of the etnropy and information of aboslutely
continuous random variables,
IEEE Transactions on Information Theory 35 no.1 (1989), 193196.
Last update: 3/9/00
ihleratmit.edu