CENG 534

Deep Learning for Natural Language Processing

Natural language processing (NLP) is one of the most important technologies of the information age. In traditional NLP, task-specific feature engineering and language-specific solutions were common. Recently, deep learning approaches have obtained very high performance across many different NLP tasks and multilingual solutions have been introduced. This course covers cutting-edge research in deep learning applied to NLP. Topics include word vector representations, window-based neural networks, recurrent neural networks, long-short-term-memory models, recursive neural networks, convolutional neural networks as well as some very novel models involving a memory component. A term project to implement, train, test, and visualize a custom neural network solution to a large scale NLP problem will be given.

Week Topics
1 Intro to NLP and Deep Learning
2 Simple Word Vector representations: word2vec, GloVe
3 Advanced word vector representations: language models, softmax, single layer networks
4 Neural Networks and backpropagation — for named entity recognition
5 Practical tips: gradient checks, overfitting, regularization, activation functions, details
6 Recurrent neural networks — for language modeling and other tasks
7 Recursive neural networks — for parsing
8 Review Session for Midterm
9 Convolutional neural networks — for sentence classification
10 Guest Lecture: Speech Recognition
11 Guest Lecture: Machine Translation
12 Guest Lecture: Seq2Seq and Large Scale DL
13 The future of Deep Learning for NLP: Dynamic Memory Networks
14 Project Presentations