In this paper, we present our initial efforts for building a code-switching (CS) speech recognition system leveraging existing acoustic models (AMs) and language models (LMs), i.e., no training required, and specifically targeting intra-sentential switching. To achieve such an ambitious goal, new mechanisms for foreign pronunciation generation and language model (LM) enrichment have been devised. Specifically, we have designed an automatic approach to obtain high quality pronunciation of foreign language (FL) words in the native language (NL) phoneme set using existing acoustic phone decoders and an LSTM-based grapheme-to-phoneme (G2P) model. Improved accented pronunciations have thus been obtained by learning foreign pronunciations directly from data. Furthermore, a code-switching LM was deployed by converting the original NL LM into a CS LM using translated word pairs and borrowing statistics for the NL LM. Experimental evidence clearly demonstrates that our approach better deals with accented foreign pronunciations than techniques based on human labeling. Moreover, our best system achieves a 55.5% relative word error rate reduction from 34.4%, obtained with a conventional monolingual ASR system, to 15.3% on an intra-sentential CS task without harming the monolingual recognition accuracy.

Huang, Z., Zhuang, X., Liu, D., Xiao, X., Zhang, Y., Siniscalchi, S.M. (2019). Exploring Retraining-free Speech Recognition for Intra-sentential Code-switching. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings (pp. 6066-6070). IEEE [10.1109/ICASSP.2019.8682478].

Exploring Retraining-free Speech Recognition for Intra-sentential Code-switching

Siniscalchi, Sabato Marco
2019-01-01

Abstract

In this paper, we present our initial efforts for building a code-switching (CS) speech recognition system leveraging existing acoustic models (AMs) and language models (LMs), i.e., no training required, and specifically targeting intra-sentential switching. To achieve such an ambitious goal, new mechanisms for foreign pronunciation generation and language model (LM) enrichment have been devised. Specifically, we have designed an automatic approach to obtain high quality pronunciation of foreign language (FL) words in the native language (NL) phoneme set using existing acoustic phone decoders and an LSTM-based grapheme-to-phoneme (G2P) model. Improved accented pronunciations have thus been obtained by learning foreign pronunciations directly from data. Furthermore, a code-switching LM was deployed by converting the original NL LM into a CS LM using translated word pairs and borrowing statistics for the NL LM. Experimental evidence clearly demonstrates that our approach better deals with accented foreign pronunciations than techniques based on human labeling. Moreover, our best system achieves a 55.5% relative word error rate reduction from 34.4%, obtained with a conventional monolingual ASR system, to 15.3% on an intra-sentential CS task without harming the monolingual recognition accuracy.
2019
978-1-4799-8131-1
Huang, Z., Zhuang, X., Liu, D., Xiao, X., Zhang, Y., Siniscalchi, S.M. (2019). Exploring Retraining-free Speech Recognition for Intra-sentential Code-switching. In ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings (pp. 6066-6070). IEEE [10.1109/ICASSP.2019.8682478].
File in questo prodotto:
File Dimensione Formato  
0006066.pdf

Solo gestori archvio

Tipologia: Versione Editoriale
Dimensione 198.65 kB
Formato Adobe PDF
198.65 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
2109.00921v1.pdf

accesso aperto

Tipologia: Post-print
Dimensione 201.42 kB
Formato Adobe PDF
201.42 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10447/649557
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 5
  • ???jsp.display-item.citation.isi??? 4
social impact