Publication Type

Conference Proceeding Article

Version

submittedVersion

Publication Date

12-2021

Abstract

The construction and theoretical analysis of the most popular universally consistent nonparametric density estimators hinge on one functional property: smoothness. In this paper we investigate the theoretical implications of incorporating a multi-view latent variable model, a type of low-rank model, into nonparametric density estimation. To do this we perform extensive analysis on histogram-style estimators that integrate a multi-view model. Our analysis culminates in showing that there exists a universally consistent histogram-style estimator that converges to any multi-view model with a finite number of Lipschitz continuous components at a rate of ˜O(1/3√n) in L1 error. In contrast, the standard histogram estimator can converge at a rate slower than 1/d√n on the same class of densities. We also introduce a new nonparametric latent variable model based on the Tucker decomposition. A rudimentary implementation of our estimators experimentally demonstrates a considerable performance improvement over the standard histogram estimator. We also provide a thorough analysis of the sample complexity of our Tucker decomposition-based model and a variety of other results. Thus, our paper provides solid theoretical foundations for extending low-rank techniques to the nonparametric setting.

Keywords

Density estimation, Low-rank methods, Tensor methods, Tucker decomposition, statistical guarantees, bias-variance analysis.

Discipline

Theory and Algorithms

Research Areas

Intelligent Systems and Optimization

Publication

Advances in Neural Information Processing Systems (NeurIPS 2021): December 7-10, Virtual: Proceedings

Volume

34

First Page

12180

Last Page

12193

ISBN

9781713845393

Publisher

NIPs Foundation

City or Country

San Diego

Additional URL

https://arxiv.org/abs/2204.00930

Download

Share

COinS