Publication

Abstract : Imagine a world where educational videos not only explain concepts but also generate real-time, context-aware in-sights in every frame of the video. We present a real-time AI-powered educational assistant designed to enhance learning by automatically generating detailed explanations for educational videos using Large Visual Language models (LVLMs). The model starts by capturing frames from short videos using OpenCV. This is to ensure that every single information in every frame is collected and processed. The frames which are extracted using OpenCV will be encoded into Base64 format, as LVLMs could process base64 encoded data more efficiently than the normal pixelates. The LVLM (ChatGPT-4 vision) is an API based model where entire architecture is internally handled by OpenAI to make the developers easy to use the model. This model helps to extract the data from the frames under one second interval capturing contextually relevant textual descriptions. This approach assists the user to clarify their doubts under the same portal rather than going out of the platform and getting distracted. To validate the system’s performance the evaluation metrics pitches in, BERT Score, ROUGE and BLEU scores are used. The model achieved a BERT Score F1 of 0.7772, demonstrating strong semantic alignment with human references. The BLEU score of 0.3074 further confirms the system’s ability to generate well-structured and coherent explanations. Additionally, ROUGE-1 (F1: 0.3088), ROUGE-2 (F1: 0.0915), and ROUGE-L (F1: 0.2215) highlight its effectiveness in capturing meaningful word and phrase overlaps. Looking ahead, planning to inculcate support for longer videos, incorporating multimodal learning elements like audio transcriptions to enrich explanations further. This helps bridging the gap between passive video consumption and interactive AI-driven education.