CLASP
The Centre for Linguistic Theory and Studies in Probability

Attention Models in Deep Learning for Machine Translation

Abstract: In the last number of years deep learning models have made a significant impact across a range of fields. Machine Translation is one such area of research. The development of the encoder-decoder architecture and its extension to include an attention mechanism has led to deep learning models achieving state of the art MT results for a number of langauge pairs.

However, an open question in deep learning for MT is what is the best attention mechanism to use. This talk will begin by reviewing the current state of the art in deep learning for MT. The second half of the talk will present a novel attention based encoder-decoder architecture for MT. This novel architecture is the result of collaborative research between John Kelleher, Giancarlo Salton, and Robert J. Ross.

Lecturer: John Kelleher is a lecturer in the School of Computing at the Dublin Institute of Technology and a researcher at the Adapt research center. He currently supervises research projects in a number of areas including machine translation, activity recognition and discovery, dialogue systems, computational models of spatial language, and music transcription.

For the last number of years the majority of his research has used a machine learning methodology, and in 2015 he published a textbook on machine learning with MIT Press. John’s collaborators on this research are Giancarlo Salton, who is a PhD student at the Dublin Institute of Technology, and Robert Ross who is a senior lecturer in the School of Computing at the Dublin Institute of Technology.