Back close

Course Detail

Course Name  Introduction to NLP, Transformers & LLM
Course Code 24AIM203
Program B.Tech. in Artificial Intelligence (AI) and Data Science (Medical Engineering)
Semester III
Credits 3
Campus Coimbatore

Syllabus

Unit 1

Overview of Natural Language Processing (NLP) and its applications in AI and biomedical engineering, historical perspective and evolution of NLP. Understanding linguistic fundamentals: syntax, semantics, morphology, and phonetics.Tokenization, stemming, and lemmatization. Text Preprocessing and Feature Extraction: Techniques for cleaning and preprocessing textual data,Feature extraction methods for representing text data, including bag-of-words and TF-IDF. Introduction to popular NLP libraries such as NLTK, spaCy, and Hugging Face Transformers.

Unit 2

Statistical and Machine Learning Approaches: Overview of statistical and machine learning approaches in NLP. Sentiment analysis, named entity recognition, and part-of-speech tagging. Introduction to Language Models (LM): Understanding the concept of language modeling. Overview of traditional language models such as N-grams. Neural Networks for NLP: Introduction to neural networks in the context of NLP, Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks. Introduction to Transfer Learning in NLP: Overview of transfer learning and its application in NLP, Introduction to pre-trained language models.

Unit 3

Introduction to Large Language Models (LLM) : Overview of large language models such as GPT (Generative Pre-trained Transformer) and BERT (Bidirectional Encoder Representations from Transformers). Pre-training Language Models: Understanding the pre-training process for language models, Exploration of model architectures and training strategies. Fine-tuning Language Models: Techniques and considerations for fine-tuning pre-trained language models,Applications of fine-tuned models in specific domains, including biomedical engineering. Ethical Considerations in NLP: Discussion on ethical challenges and biases in NLP, Strategies for mitigating biases and ensuring responsible AI in NLP applications.

Unit 4

Applications in AI and Biomedical Engineering: NLP in Healthcare: Applications of NLP in healthcare and biomedical engineering, Case studies on text-based analysis of medical literature, electronic health records, and patient data. Biomedical Text Mining: Techniques for extracting information from biomedical texts, Exploration of literature mining and knowledge discovery in biomedicine. Advanced NLP Applications: Overview of advanced NLP applications in AI, including chatbots, summarization, and question-answering systems, Practical projects applying NLP techniques to real-world scenarios. Future Trends in NLP and LLM: Exploration of emerging trends in NLP and Large Language Models. Discussion on the potential impact of future advancements in AI and biomedical engineering.

Course Objectives and Outcomes

Course Objectives:

  • Understand the foundational concepts of Natural Language Processing (NLP) and Large Language Models (LLM), including linguistic fundamentals and preprocessing techniques.
  • Explore advanced NLP techniques, such as sentiment analysis, named entity recognition, and neural networks, as well as gain insights into language modeling.
  • Acquire hands-on experience with popular NLP libraries and tools, and comprehend the principles behind pre-trained language models.
  • Apply NLP and LLM concepts to real-world applications in AI and biomedical engineering, with a focus on ethical considerations and responsible AI.

Course Outcomes:

After completing this course, students should be able to:
CO1: Employ various NLP preprocessing techniques, including tokenization and feature extraction, to analyze and manipulate textual data effectively.
CO2: Apply advanced NLP techniques, such as sentiment analysis and part-of-speech tagging, and understand the architecture and training strategies of neural networks for NLP.
CO3: Utilize pre-trained language models for tasks like language understanding and generation, and fine-tune models for domain-specific applications.
CO4: Develop NLP solutions in AI and biomedical engineering, considering ethical implications and biases, ensuring responsible and impactful use of language models.

CO-PO mapping:

CO/PO

PO1

PO2

PO3

PO4

PO5

PO6

PO7

PO8

PO9

PO10

PO11

PO12

PSO1

PSO2

PSO3

CO1

3

1

1

1

3

2

2

2

1

CO2

3

2

3

3

2

3

2

2

2

2

3

3

3

CO3

2

2

2

2

1

3

1

2

2

2

2

1

2

CO4

3

3

3

3

2

3

2

2

2

2

3

3

3

Text Books / Reference Books

  1. Dan Jurafsky, James H. Martin, “Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition,” 3rd Edition, Pearson, 2019
  2. Lane, Howard, Hapke, “Natural Language Processing in Action,” 1st Edition, Manning Publications, 2019
  3. Rajalingappaa Shanmugamani, “Hands-On Natural Language Processing with Python: A practical guide to applying deep learning architectures to your NLP applications,” 1st Edition, Packt Publishing, 2018
  4. Alexander Rush, “Transformers in Natural Language Processing,” 1st Edition, O’Reilly Media, 2021

DISCLAIMER: The appearance of external links on this web site does not constitute endorsement by the School of Biotechnology/Amrita Vishwa Vidyapeetham or the information, products or services contained therein. For other than authorized activities, the Amrita Vishwa Vidyapeetham does not exercise any editorial control over the information you may find at these locations. These links are provided consistent with the stated purpose of this web site.

Admissions Apply Now