Foundations of Large Language Models

Students are introduced to foundational concepts and advanced techniques in large language models (LLMs) to understand the pivotal role of LLMs in natural language processing (NLP). The course covers a range of topics, including transformer architecture, and examines key models such as generative pre-trained transformer (GPT), bidirectional encoder representations from transformers (BERT), and text-to-text transfer transformer (T5). Practical skills are developed in employing these models for tasks like text generation, language translation, sentiment analysis, and building chatbots and conversational agents. Methods for working with LLMs include tokenization, dependency parsing, embedding generation, and fine-tuning of pre-trained models for specialized applications, as well as prompt engineering, data augmentation, and model evaluation to enhance performance. The course curriculum incorporates machine learning and deep learning techniques relevant to LLMs, including attention mechanisms and neural network optimizations. Students gain hands-on experience conducting experiments on platforms like Google Colab and implementing projects utilizing libraries such as Hugging Face transformers in Python.