Unsupervised Representation Learning by Predicting Random Distances
Deep neural networks have gained tremendous success in a broad range of machine learning tasks due to its remarkable capability to learn semantic-rich features from high-dimensional data. However, they often require large-scale labelled data to successfully learn such features, which significantly hinders their adaption into unsupervised learning tasks, such as anomaly detection and clustering, and limits their applications into critical domains where obtaining massive labelled data is prohibitively expensive. To enable downstream unsupervised learning on those domains, in this work we propose to learn features without using any labelled data by training neural networks to predict data distances in a randomly projected space. Random mapping is a highly efficient yet theoretical proven approach to obtain approximately preserved distances. To well predict these random distances, the representation learner is optimised to learn class structures that are implicitly embedded in the randomly projected space. Experimental results on 19 real-world datasets show our learned representations substantially outperform state-of-the-art competing methods in both anomaly detection and clustering tasks.
The authors discuss the a novel technique, called Random Distance Prediction, to learn rich features from domains where massive data are hard to produce; in particular, they focus on the two tasks of anomaly detection and clustering. The paper is well written and understandable by a non-specialistic audience; introduction and references are adequate, and the theoretical analysis is reasonably explained, although the two optional losses should have been discussed more deeply. Results are fairly supporting the authors* claim: however, the improvement in performances w.r.t. alternative approaches are limited in most cases, and the contribute of the optional losses is somehow unclear and inconsistent across different datasets. It would be interesting also to check how stable is the method (i.e. the losses) for different underlying network architectures, and also how RDP compares to very basic approaches employing dimensionality reduction algorithms such as t-SNE or UMAP.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Unsupervised Representation Learning by Predicting Random Distances
Deep neural networks have gained tremendous success in a broad range of machine learning tasks due to its remarkable capability to learn semantic-rich features from high-dimensional data. However, they often require large-scale labelled data to successfully learn such features, which significantly hinders their adaption into unsupervised learning tasks, such as anomaly detection and clustering, and limits their applications into critical domains where obtaining massive labelled data is prohibitively expensive. To enable downstream unsupervised learning on those domains, in this work we propose to learn features without using any labelled data by training neural networks to predict data distances in a randomly projected space. Random mapping is a highly efficient yet theoretical proven approach to obtain approximately preserved distances. To well predict these random distances, the representation learner is optimised to learn class structures that are implicitly embedded in the randomly projected space. Experimental results on 19 real-world datasets show our learned representations substantially outperform state-of-the-art competing methods in both anomaly detection and clustering tasks.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors discuss the a novel technique, called Random Distance Prediction, to learn rich features from domains where massive data are hard to produce; in particular, they focus on the two tasks of anomaly detection and clustering. The paper is well written and understandable by a non-specialistic audience; introduction and references are adequate, and the theoretical analysis is reasonably explained, although the two optional losses should have been discussed more deeply. Results are fairly supporting the authors* claim: however, the improvement in performances w.r.t. alternative approaches are limited in most cases, and the contribute of the optional losses is somehow unclear and inconsistent across different datasets. It would be interesting also to check how stable is the method (i.e. the losses) for different underlying network architectures, and also how RDP compares to very basic approaches employing dimensionality reduction algorithms such as t-SNE or UMAP.
assistant
The authors discuss the a novel technique, called Random Distance Prediction, to learn rich features from domains where massive data are hard to produce; in particular, they focus on the two tasks of anomaly detection and clustering. The paper is well written and understandable by a non-specialistic audience; introduction and references are adequate, and the theoretical analysis is reasonably explained, although the two optional losses should have been discussed more deeply. Results are fairly supporting the authors* claim: however, the improvement in performances w.r.t. alternative approaches are limited in most cases, and the contribute of the optional losses is somehow unclear and inconsistent across different datasets. It would be interesting also to check how stable is the method (i.e. the losses) for different underlying network architectures, and also how RDP compares to very basic approaches employing dimensionality reduction algorithms such as t-SNE or UMAP.