University of Negev Researchers Develop DeepDPM: A Deep Clustering Algorithm With An Unknown Number Of Clusters

Clustering is an essential unsupervised learning job in which class labels are not accessible, unlike in the supervised situation of classification. Furthermore, the number of classes represented by K and their relative sizes are unknown in the totally unsupervised environment on which this research focuses.

Clustering tasks have not been overlooked by Deep Learning (DL). Large and high-dimensional datasets are typically clustered better and more efficiently by DL approaches than by traditional clustering methods. However, while nonparametric approaches have advantages over parametric methods (methods that require a known K) in classical clustering, there are just a few nonparametric deep clustering methods.

Unfortunately, the latter is neither scalable nor effective enough. It is advantageous to be able to deduce the latent K. Parametric approaches may perform poorly if K is not accurately estimated. In both balanced and unbalanced datasets, using the wrong K can have a major negative impact on parametric approaches.

Changing K during training has beneficial optimization implications; for example, breaking a single cluster into two changes many data labels at the same time. When K is unknown, a popular workaround is to utilize model selection, which involves running a parametric technique multiple times with different K values over a wide range and then selecting the best K using an unsupervised criterion. K could be a valuable quantity in and of itself. 

When K is uncertain, Bayesian nonparametric (BNP) mixture models, such as the Dirichlet Process Mixture (DPM), provide an elegant, data-adaptive solution for clustering. However, due to the high computing cost associated with DPM inference, only a few studies have attempted to employ it in conjunction with deep clustering.

Researchers at the University of Negev attempted to bridge this gap in a recent paper by developing DeepDPM, a powerful deep nonparametric approach. They advocated effectively combining the benefits of DL and DPM. Despite its unfair edge, DeepDPM yields result equivalent to top parametric approaches even when K is known.

DeepDPM, the suggested approach, leverages cluster splits and merges to modify K, as well as a dynamic design to handle such changes. For Expectation-Maximization algorithms, it also employs a novel amortized inference technique. DeepDPM can be integrated into clustering-based deep pipelines.

DeepDPM is differentiable for the majority of the training, unlike an offline clustering step. Across a variety of datasets and metrics, DeepDPM surpasses existing nonparametric clustering algorithms (both classical and deep). It also smoothly manages class imbalance and scales effectively with massive datasets.

DeepDPM was compared to both parametric and nonparametric approaches by the team. The MNIST, USPS, and Fashion-MNIST datasets, as well as their unbalanced counterparts, were used in the study. DeepDPM almost equally dominates across all datasets and measures, and its performance benefit only grows in imbalanced scenarios, according to the findings. It was also observed that nonparametric methods were less influenced by the imbalance than parametric methods. DeepDPM, like most clustering techniques, would struggle to recover if the input features were weak. Parametric approaches may also be a somewhat better alternative if K is known and the dataset is balanced.

Conclusions

A deep nonparametric clustering approach, a dynamic architecture that adapts to shifting K values, was described by researchers at the University of Negev. Deep and non-deep nonparametric approaches were outperformed by the method, which yielded State of the Art results. The method’s resistance to both class imbalance and the initial K was demonstrated by the researchers. They were the first approach of its sort to post findings on ImageNet, demonstrating the scalability of DeepDPM.

This Article Is Based On The Research Paper 'DeepDPM: Deep Clustering With an Unknown Number of Clusters'. All Credit For This Research Goes To The Researchers of This Project. Check out the paper and codes.

Please Don't Forget To Join Our ML Subreddit