Publication

Abstract : The work, titled “Optimizing Large Language Models for Edge Devices Using Knowledge Distillation”, aims to develop an efficient and lightweight text classification system tailored for edge devices such as smartphones and embedded systems. While Large Language Models (LLMs) offer high accuracy in natural language processing tasks, their computational and memory demands render them unsuitable for direct deployment on resource-constrained edge hardware. To address this limitation, the project employs knowledge distillation, where a compact student model is trained to emulate the behavior of a larger teacher model, preserving performance while significantly reducing complexity. Furthermore, the system incorporates quantization techniques to compress model parameters into lower-bit representations, enhancing memory efficiency and inference speed. Together, these approaches facilitate real-time, on-device disaster tweet classification without reliance on cloud infrastructure. The methodology demonstrates strong potential for broader applications, including healthcare alerts, cybersecurity threat detection, and filtering of misinformation.