In this paper, we present a multi-layer learning approach to the language model (LM) adaptation problem by making use of multi-objective programming (MOP). The overall objective function of conventional MAP-based LM adaptation is implicitly a composition of two objective functions: The first objective is concerned with the maximum likelihood estimation of the model parameters from the indomain data while the second objective is concerned with an appropriate representation of prior information obtained from a general purpose corpus. In this paper, we separate these individual objective functions, which are at least partially conflicting, and take an MOP approach to LM adaptation. The resulting MOP problem is solved in an iterative manner such that each objective is optimized one after another with constraints on the others. This iterative solution can be represented as a multi-layer learning problem in each layer of which only one objective is minimized with constraints on others. In estimating an n-gram LM, number of the layers is given by 2× n with one hidden unit per layer. The inputs to the hidden units are LMs of order up to n that are estimated either from the general purpose corpus or from the in-domain data. When solved this way, the target LM is in the form of a log-linear interpolation of component LMs. In our preliminary experiments with bigram LMs, the proposed approach slightly outperformed linear interpolation. In our ongoing work with trigram LMs, we expect the proposed approach to outperform linear interpolation in terms of both the perplexity and the automatic speech recognition work error rate.
S. YAMAN, S. M. SINISCALCHI, AND C.-H. LEE (2009). A Multi-Objective Programming-Based Approach to Language Model Adaptation. In NIPS.
A Multi-Objective Programming-Based Approach to Language Model Adaptation
S. M. SINISCALCHI;
2009-01-01
Abstract
In this paper, we present a multi-layer learning approach to the language model (LM) adaptation problem by making use of multi-objective programming (MOP). The overall objective function of conventional MAP-based LM adaptation is implicitly a composition of two objective functions: The first objective is concerned with the maximum likelihood estimation of the model parameters from the indomain data while the second objective is concerned with an appropriate representation of prior information obtained from a general purpose corpus. In this paper, we separate these individual objective functions, which are at least partially conflicting, and take an MOP approach to LM adaptation. The resulting MOP problem is solved in an iterative manner such that each objective is optimized one after another with constraints on the others. This iterative solution can be represented as a multi-layer learning problem in each layer of which only one objective is minimized with constraints on others. In estimating an n-gram LM, number of the layers is given by 2× n with one hidden unit per layer. The inputs to the hidden units are LMs of order up to n that are estimated either from the general purpose corpus or from the in-domain data. When solved this way, the target LM is in the form of a log-linear interpolation of component LMs. In our preliminary experiments with bigram LMs, the proposed approach slightly outperformed linear interpolation. In our ongoing work with trigram LMs, we expect the proposed approach to outperform linear interpolation in terms of both the perplexity and the automatic speech recognition work error rate.File | Dimensione | Formato | |
---|---|---|---|
A-Multi-Objective-Programming-Based-Approach-to-Language-Model-Adaptation.pdf
Solo gestori archvio
Tipologia:
Versione Editoriale
Dimensione
210.79 kB
Formato
Adobe PDF
|
210.79 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.