Patch-Based Principal Covariance Discriminative Learning for Image Set Classification

Image set classification has attracted increasing attention with respect to the use of significant amounts of within-set information. The covariance matrix is a natural and effective descriptor for describing image sets. Non-singular covariance matrices, known as symmetric positive definite (SPD) matrices, are regarded as points on a Riemannian manifold. A common method of classifying points on a manifold is to explicitly map the SPD matrices from a Riemannian manifold to Euclidean space, such as in the covariance discriminative learning (CDL) method. However, the disadvantages of the CDL method are as follows: 1) the method models the whole image set as a covariance matrix, whereas if there are insufficient set samples or merely one set, the within-class information studied by the discriminative learning may not be utilized well and 2) when the original sample covariance matrices are of high dimensionality, the computational cost is non-trivial. To address these problems of CDL, we propose to exploit the maximal linear patch to cluster image sets into multiple subsets (local patches), which could provide substantially more within-class information. Moreover, we refine the manifold formed by the SPD matrices to a lower dimensionality and more discriminative manifold by collaboratively applying principal component analysis to all training sets. Experiments are performed on face recognition and objection categorization tasks; extensive comparison results illustrate the considerable effectiveness of our proposed method.