Acoustic adaptation in cross database audio visual SHMMtraining for phonetic spoken term detection
Visual information in the form of lip movements of the speaker has been shown to improve the performance of speech
recognition and search applications. In our previous work,
we proposed cross database training of synchronous hidden
Markov models (SHMMs) to make use of external large and
publicly available audio databases in addition to the relatively small given audio visual database. In this work, the cross database training approach is improved by performing an additional audio adaptation step, which enables audio visual SHMMs to benefit from audio observations of the external audio models before adding visual modality to them. The proposed approach outperforms the baseline cross database training approach in clean and noisy environments in terms of phone recognition accuracy as well as spoken term detection (STD) accuracy.