A Tensor Framework for Learning in Structured Domains

Abstract: Machine Learning (ML) for structured data aims to build ML models which can handle structured data (e.g. trees). In this context, many popular ML models build a succinct representation of the input structure by learning how to aggregate its constituents. While learning complex aggregation functions is desirable to increase the expressiveness of the learned representations, easy-to-compute functions are usually preferred for computational reasons. To overcome this limitation, we propose a general framework for learning in structured domains which has tensor theory at its backbone.

In this presentation, we show how tensors arise naturally in ML models for structured data, and, most importantly, how to leverage tensor decompositions to make the tensor approach computationally feasi-ble. We focus on three tensor decompositions (CP, HOSVD, TT), by investigating how the selection of the tensor decomposition (and its ranks) affects the ML model behaviour. The results obtained on synthetic and NLP datasets demonstrate that the application of tensors in ML for structured data is a promising research direction.