Publication

Abstract : IoT is one of the most efficient tools for real-time monitoring of various systems. However, faulty sensors, unreliable communication, interference, and synchronization issues often result in significant missing data in IoT-based sensor systems. This missing data leads to considerable performance degradation and inaccuracies in data analytics. Consequently, we were motivated to identify optimal imputation algorithms for real-time time-series data. In this study, we compared various statistical and machine learning models with generative AI models for data imputation in the context of water demand data. Specifically, we applied the following methods to the BFWDN dataset: Mean Imputation, Regression, Expectation–Maximization, K-Nearest Neighbors (KNN), Feed-forward Neural Network, GAIN, VAE, DLIN, XGBoost, Kriging, and Spatio-Temporal GAIN. We evaluated their performance using a comprehensive set of metrics, such as RMSE, MAE, NRMSE, Jensen-Shannon Distance, Kolmogorov–Smirnov (K-S) Test Statistic, K-S Test p-value, Estimation Bias, Empirical Standard Error, Coverage Rate, AUC, C-index, Gower’s Distance, Similarity Index, and Reconstruction Loss (RL). The results indicate that generative models, including GAIN, VAE, and especially Spatio-Temporal GAIN or DLIN, outperform all other approaches even when facing high missing data rates (80–90%).