CENG 467
NATURAL LANGUAGE UNDERSTANDING AND GENERATION
This course focuses on deep learning methods for natural language understanding and generation. It covers tokenization and pre-training for language models, the transformer architecture, machine translation, parsing, summarisation, question answering, instruction fine tuning, ethics in NLP, in context learning and retrieval-augmented generation.
The main objective of the course is to provide students with a basic knowledge of natural language processing, machine learning and programming with a more advanced knowledge of existing deep learning techniques for natural language understanding and generation. The target audience is senior undergraduate students. The course covers the foundations for the deep learning for natural language processing graduate course.
The Natural Language Understanding and Generation course contributes to students’ ability to conduct research and develop products in the field of Generative Artificial Intelligence at the graduation stage.
- Apply deep learning methods for natural language processing to industrial problems
- Be aware of the social implications of natural language processing and know the relevant ethical terminology
- Use different language model architectures in problem solving
- List different natural language processing tasks and their features
Course Materials: Speech and Language Processing, D. Jurafsky and J.H. Martin, 3rd Ed., 2025.
Midterm: 30%
Assignments: 30%
Final Exam: 40%
| Week | Topic |
| 1 | Introduction |
| 2 | Machine Translation and Conditional Language Models |
| 3 | Modeling Data and Words |
| 4 | Feedforward Language Models, Recurrent Neural Networks |
| 5 | Sequence-to-sequence Models with Attention, Transformers and Pretrained Language Models |
| 6 | Parsing |
| 7 | Instruction Fine Tuning and RLHF and Parameter Efficient Fine Tuning |
| 8 | Summarisation |
| 9 | Evaluation of generation and translation |
| 10 | Ethics in NLP |
| 11 | In Context Learning |
| 12 | Question Answering |
| 13 | Retrieval-Augmented Generation |
| 14 | Diffusion Models for NLP |
- CENG 400
- CENG 411
- CENG 415
- CENG 416
- CENG 418
- CENG 421
- CENG 422
- CENG 424
- CENG 431
- CENG 432
- CENG 433
- CENG 434
- CENG 435
- CENG 436
- CENG 437
- CENG 441
- CENG 442
- CENG 443
- CENG 444
- CENG 451
- CENG 452
- CENG 461
- CENG 462
- CENG 463
- CENG 464
- CENG 465
- CENG 471
- CENG 472
- CENG 473
- CENG 481
- CENG 482
- CENG 483
- CENG 484
- CENG 485
- CENG 486
- CENG 487
- CENG 488
- CENG 499