Histogram of Oriented Principal Components for Cross-View Action Recognition

Existing techniques for 3D action recognition are sensitive to viewpoint variations because they extract features from depth images which are viewpoint dependant. In contrast, we directly process pointclouds for cross-view action recognition from novel unknown and unseen views. We propose Histogram of Oriented Principal Components (HOPC) descriptor which is robust to noise, viewpoint, scale and action speed variations. HOPC is extracted at a point by projecting all three eigenvectors of the pointcloud within a local spatio-temporal support volume, scaled by their corresponding eigenvalues, on the vertices of a regular dodecahedron. HOPC is also used to detect Spatio-Temporal Keypoints (STK) in 3D pointcloud sequences so that HOPC descriptors from only the relevant regions are used for action recognition. Finally, we propose a global descriptor that is extracted from the normalized spatio-temporal distribution of STKs in 4-D. STK-Distribution and local HOPC descriptors were evaluated for cross-view human action recognition on the Northwestern-UCLA and UWA3D Multiview Activity II datasets. Comparison with six cross-view action recognition algorithms shows that both descriptors individually outperform all existing techniques. Since HOPC and STK-Distribution capture complimentary information, their combination significantly improves classification accuracy which is over 22% higher than the nearest competitor on both datasets.