Minimum-overlap clusterings and the sparsity of overcomplete decompositions of binary matrices

Abstract

Given a set of n binary data points, a widely used technique is to group its features into k clusters. In the case where n < k, the question of how overlapping are the clusters becomes of interest. In this paper we approach the question through matrix decomposition, and relate the degree of overlap with the sparsity of one of the resulting matrices. We present analytical results regarding bounds on this sparsity, and a heuristic to estimate the minimum amount of overlap that an exact grouping of features into k clusters must have. As shown below, adding new data will not alter this minimum amount of overlap.