publications

publications by categories in reversed chronological order. generated by jekyll-scholar.

2024

Geometric Encoder Regularization for Autoencoders

Philipp Nazari

Preprint, 2024

Abs PDF

In this study we adapt existing decoder-based geometric regularization methods for autoencoders to depend only on the encoder. This shift aligns more naturally with the primary function of autoencoders in generating low-dimensional embeddings. Specifically, we replace the pullback metric with a pushforward. Although this pushforward does not account for all of the geometric information encoded in the data manifold, it is applicable to a much broader spectrum of models.
Entropy Aware Message Passing in Graph Neural Networks

Philipp Nazari, Oliver Lemke, Davide Guidobene, and 1 more author

arXiv preprint arXiv:2403.04636, 2024

Abs PDF

Deep Graph Neural Networks struggle with oversmoothing. This paper introduces a novel, physics-inspired GNN model designed to mitigate this issue. Our approach integrates with existing GNN architectures, introducing an entropy-aware message passing term. This term performs gradient ascent on the entropy during node aggregation, thereby preserving a certain degree of entropy in the embeddings. We conduct a comparative analysis of our model against state-of-the-art GNNs across various common datasets.

2023

Geometric Autoencoders-What You See is What You Decode

Philipp Nazari, Sebastian Damrich, and Fred A Hamprecht

In International Conference on Machine Learning, 2023

Abs PDF

Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can help unfolding the data. However, such expressive networks can achieve low reconstruction error even when the latent representation is distorted. To avoid such misleading visualizations, we propose first a differential geometric perspective on the decoder, leading to insightful diagnostics for an embedding’s distortion, and second a new regularizer mitigating such distortion. Our “Geometric Autoencoder” avoids stretching the embedding spuriously, so that the visualization captures the data structure more faithfully. It also flags areas where little distortion could not be achieved, thus guarding against misinterpretation.