Publication Type

Journal Article

Publication Date

7-2013

Abstract

Shrinkage methods have been shown to be effective for classification problems. As a form of regularization, shrinkage through penalization helps to avoid overfitting and produces accurate classifiers for prediction, especially when the dimension is relatively high. Despite the benefit of shrinkage on classification accuracy of resulting classifiers, in this article, we demonstrate that shrinkage creates biases on classification probability estimation. In many cases, this bias can be large and consequently yield poor class probability estimation when the sample size is small or moderate. We offer some theoretical insights into the effect of shrinkage and provide remedies for better class probability estimation. Using penalized logistic regression and proximal support vector machines as examples, we demonstrate that our proposed refit method gives similar classification accuracy and remarkable improvements on probability estimation on several simulated and real data examples.

Keywords

Bias, High dimension, Refit, Regularization

Discipline

International Economics | Labor Economics

Research Areas

Econometrics

Publication

American Statistician

Volume

67

Issue

3

First Page

134

Last Page

142

ISSN

0003-1305

Identifier

10.1080/00031305.2013.817356

Additional URL

http://doi.org/10.1080/00031305.2013.817356

Download

Share

COinS