In this paper, we show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of mean absolute error (MAE) between the predicted and expected feature vectors is upper bounded by the sum of an approximation error, an estimation error, and an optimization error. Leveraging upon error decomposition techniques in statistical learning theory and non-convex optimization theory, we derive upper bounds for each of the three aforementioned errors and impose necessary constraints on DNN models. Moreover, we assess our theoretical results through a set of image de-noising and speech enhancement experiments. Our proposed upper bounds of MAE for DNN based vector-to-vector regression are corroborated by the experimental results and the upper bounds are valid with and without the 'over-parametrization' technique.

Qi J., Du J., Siniscalchi S.M., Ma X., Lee C.-H. (2020). Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network-Based Vector-to-Vector Regression. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 68, 3411-3422 [10.1109/TSP.2020.2993164].

Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network-Based Vector-to-Vector Regression

Siniscalchi S. M.
Supervision
;
2020-05-01

Abstract

In this paper, we show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of mean absolute error (MAE) between the predicted and expected feature vectors is upper bounded by the sum of an approximation error, an estimation error, and an optimization error. Leveraging upon error decomposition techniques in statistical learning theory and non-convex optimization theory, we derive upper bounds for each of the three aforementioned errors and impose necessary constraints on DNN models. Moreover, we assess our theoretical results through a set of image de-noising and speech enhancement experiments. Our proposed upper bounds of MAE for DNN based vector-to-vector regression are corroborated by the experimental results and the upper bounds are valid with and without the 'over-parametrization' technique.
mag-2020
Qi J., Du J., Siniscalchi S.M., Ma X., Lee C.-H. (2020). Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network-Based Vector-to-Vector Regression. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 68, 3411-3422 [10.1109/TSP.2020.2993164].
File in questo prodotto:
File Dimensione Formato  
Analyzing_Upper_Bounds_on_Mean_Absolute_Errors_for_Deep_Neural_Network-Based_Vector-to-Vector_Regression.pdf

Solo gestori archvio

Descrizione: main document
Tipologia: Versione Editoriale
Dimensione 728.19 kB
Formato Adobe PDF
728.19 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
2008.05459v1.pdf

accesso aperto

Tipologia: Pre-print
Dimensione 5.83 MB
Formato Adobe PDF
5.83 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10447/639407
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 40
  • ???jsp.display-item.citation.isi??? 32
social impact