Introduction to Big Data and Hadoop: Introduction, characteristics, sources, and types of Big Data; Challenges in Big Data processing; Distributed computing concepts; NoSQL databases and data models; Hadoop ecosystem: Hadoop Distributed File System [HDFS], Hadoop YARN, MapReduce framework, and Hadoop ecosystem components; MapReduce programming model: Mapper, Reducer, data flow, and real-world applications.
Apache Spark, Hive, and Pig: Spark architecture, execution model, Resilient Distributed Datasets (RDDs), transformations and actions, DataFrames, Spark SQL, and data preprocessing; Apache Hive: Architecture, Hive data types, file formats, Hive Query Language (HQL), and User Defined Functions (UDFs); Apache Pig: Architecture, Pig Latin language, data processing, and analytics using Pig.
Machine Learning and Analytics using Spark: Spark MLlib, Machine learning pipelines, feature engineering, model building, and evaluation; Regression, Classification, Clustering, and Recommendation techniques using Spark; Spark Structured Streaming for real-time analytics; Performance optimization in Spark applications; Applications and case studies of Big Data Analytics in healthcare, finance, social media, smart cities, and Internet of Things (IoT).