Seminario

Stability of Deep Neural Networks via discrete rough paths

Seminario periodico del Dipartimento di Matematica
1 dicembre 2022
Orario di inizio 
14:30
PovoZero - Via Sommarive 14, Povo (Trento)
Aula seminari "-1" (Povo 0) e via Zoom (contattare dept.math@unitn.it per le credenziali)
Destinatari: 
Comunità universitaria
Comunità studentesca UniTrento
Partecipazione: 
Ingresso libero
Online
Email per prenotazione: 
Referente: 
Prof. Michele Coghi
Contatti: 
Università degli Studi Trento 38123 Povo (TN) - Staff Dipartimento di Matematica
+39 04 61/281508-1625-1701-1980-3898
Speaker: 
Nikolas Tapia (WIAS Berlin)

Abstract: Using rough path techniques, we provide a priori estimates for the output of Deep Residual Neural Networks in terms of both the input data and the (trained) network weights. As trained network weights are typically very rough when seen as functions of the layer, we propose to derive stability bounds in terms of the total p-variation of trained weights for any p∈[1,3]. Unlike the C1-theory underlying the neural ODE literature, our estimates remain bounded even in the limiting case of weights behaving like Brownian motions, as suggested in [Cohen-Cont-Rossier-Xu, "Scaling Properties of Deep Residual Networks”, 2021]. Mathematically, we interpret residual neural network as solutions to (rough) difference equations, and analyse them based on recent results of discrete time signatures and rough path theory. Based on joint work with C. Bayer and P. K. Friz.