Classifying Indonesian Hoax News Titles with SVM, XGBoost, and BiLSTM

Abstract

This study investigates the automated detection of hoaxes related to President Jokowi in Indonesian news by analyzing only news titles, aiming for efficient detection and reduced traffic to harmful websites. We compared the performance of traditional (SVM, XGBoost) and deep learning (BiLSTM) algorithms, with and without Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance in a dataset scraped from trusted news sources (CNN Indonesia, Detik News) and a fact-checking platform (turnbackhoax.id). The results indicate that BiLSTM generally outperformed SVM and XGBoost, demonstrating the potential of deep learning for this task. However, applying SMOTE negatively impacted BiLSTM's performance, suggesting overfitting. Notably, precision consistently exceeded recall across all models, indicating high reliability in identifying hoaxes but a potential for missing a significant number of actual hoaxes. This highlights a trade-off between avoiding false positives and ensuring comprehensive detection. The findings also suggest that language-specific characteristics influence algorithm effectiveness. This research contributes to developing efficient and accurate tools for combating misinformation in the Indonesian online environment, emphasizing the importance of title-based analysis and careful consideration on data balancing.