The BACON Approach for Rank-Deficient Data

Athanassios Kondylis, Ali S. Hadi, Mark Werner

Abstract


Rank-deficient data are not uncommon in practice. They result from highly collinear variables and/or high-dimensional data. A special case of the latter occurs when the number of recorded variables exceeds the number of observations. The use of the BACON algorithm for outlier detection in multivariate data is extended here to include rank-deficient data. We present two approaches to identifying outliers in rank-deficient data based on the original BACON algorithm. The first algorithm projects the data onto a robust subspace of reduced dimension, while the second employs a ridge type regularization on the covariance matrix. Both algorithms are tested on real as well as simulated data sets with good results in terms of their effectiveness in outlier detection. They are also examined in terms of computational efficiency and found to be very fast, with particularly good scaling properties for increasing dimension.

Keywords


High-dimensional data, Mahalanobis distance, Outlier detection, Spatial median.

Full Text:

PDF


DOI: http://dx.doi.org/10.18187/pjsor.v8i3.514

Refbacks

  • There are currently no refbacks.




Copyright (c)

Title

The BACON Approach for Rank-Deficient Data

Keywords

High-dimensional data, Mahalanobis distance, Outlier detection, Spatial median.

Description

Rank-deficient data are not uncommon in practice. They result from highly collinear variables and/or high-dimensional data. A special case of the latter occurs when the number of recorded variables exceeds the number of observations. The use of the BACON algorithm for outlier detection in multivariate data is extended here to include rank-deficient data. We present two approaches to identifying outliers in rank-deficient data based on the original BACON algorithm. The first algorithm projects the data onto a robust subspace of reduced dimension, while the second employs a ridge type regularization on the covariance matrix. Both algorithms are tested on real as well as simulated data sets with good results in terms of their effectiveness in outlier detection. They are also examined in terms of computational efficiency and found to be very fast, with particularly good scaling properties for increasing dimension.

Date

2012-07-01

Identifier


Source

Pakistan Journal of Statistics and Operation Research; Vol 8. No. 3, 2012



Print ISSN: 1816-2711 | Electronic ISSN: 2220-5810