Sentiment Analysis of IKD Application Reviews on Play Store Using Random Forest

Kelvin H.; Erlin; Yenny Desnelita; Dwi Oktarina

doi:10.22146/jnteti.v14i3.20473

Kelvin H. Department of Information Systems, Faculty of Computer Science, Institut Bisnis dan Teknologi Pelita Indonesia, Pekanbaru 28127, Indonesia
Erlin Department of Informatics Engineering, Faculty of Computer Science, Institut Bisnis dan Teknologi Pelita Indonesia, Pekanbaru 28127, Indonesia
Yenny Desnelita Department of Information Systems, Faculty of Computer Science, Institut Bisnis dan Teknologi Pelita Indonesia, Pekanbaru 28127, Indonesia
Dwi Oktarina Department of Information Systems, Faculty of Computer Science, Institut Bisnis dan Teknologi Pelita Indonesia, Pekanbaru 28127, Indonesia

DOI: https://doi.org/10.22146/jnteti.v14i3.20473

Keywords: Sentiment Analysis, Text Classification, IKD Application, Random Forest, SMOTE

Abstract

The rapid growth of digital applications in population administration services has increased the importance of sentiment analysis to understand user perceptions more deeply. This study focuses on the Digital population identity (Identitas Kependudukan Digital, IKD), a digital identity application developed by the Indonesian government. It aims to classify user reviews of the IKD application into positive, neutral, and negative sentiments using the random forest algorithm. The dataset consisted of 28,134 user reviews from the Google Play Store, including usernames, review texts, timestamps, and star ratings. The research stages included data preprocessing, labeling, handling missing values, and text processing (cleansing, tokenizing, stopword removal, and stemming). The data were divided into 80% training and 20% testing sets. The best-performing model used the parameters: max_depth=None, max_features=log2, min_samples_leaf=1, min_samples_split=2, and n_estimators=300, achieving an average accuracy of 83.78%. To address class imbalance, the synthetic minority oversampling technique (SMOTE) was applied, resulting in improved performance with an accuracy of 86.29%. Evaluation metrics before SMOTE showed 83.85% accuracy, 80.40% precision, 83.85% recall, and 81.73% F1 score. After SMOTE, precision increased to 81.22%, while accuracy and recall slightly decreased to 80.86%, with an F1 score of 81.03%. Furthermore, sentiment trend analysis using N-gram techniques (unigram, bigram, trigram) was conducted to identify frequently mentioned topics and user concerns. These insights support the research objective of guiding application improvements aligned with user needs and enhancing the overall digital service experience.

References

R.W. Sasongko, “Implementasi identitas kependudukan digital di Kabupaten Bandung,” J. Regist., vol. 5, no. 1, pp. 69–86, Sep. 2023, doi: 10.33701/jurnalregistratie.v5i1.3148.

P.C. Ardilia, “Optimalisasi pendampingan layanan administrasi kependudukan melalui program Kalimasada untuk mewujudkan tertib adminduk studi di Kelurahan Tembok Dukuh Kota Surabaya,” PRAJA Obs., J. Penelit. Adm. Publik, vol. 2, no. 3, pp. 63–68, May 2022.

A. Widiyarta and I. Humaidah, “Implementasi aktivasi identitas kependudukan digital (IKD) dalam mendorong digitalisasi di Kelurahan Jepara Kota Surabaya,” J. Ilm. Wahana Pendidik., vol. 9, no. 18, pp. 43–51, Sep. 2023, doi: 10.5281/zenodo.8310255.

M.N. Muttaqin and I. Kharisudin, “Analisis sentimen pada ulasan aplikasi Gojek menggunakan metode support vector machine dan k nearest neighbor,” UNNES J. Math., vol. 10, no. 2, pp. 22–27, Nov. 2021, doi: 10.15294/ujm.v10i2.48474.

N.L.P.C. Savitri, R.A. Rahman, R. Venyutzky, and N.A. Rakhmawati, “Analisis klasifikasi sentimen terhadap sekolah daring pada Twitter menggunakan supervised machine learning,” JuTISI (J. Tek. Inform. Sist. Inf.), vol. 7, no. 1, pp. 47–58, Apr. 2021, doi: 10.28932/jutisi.v7i1.3216.

A. Firdaus and W.I. Firdaus, “Text mining dan pola algoritma dalam penyelesaian masalah informasi: (Sebuah ulasan),” JUPITER, J. Penelit. Ilmu Teknol. Komput., vol. 13, no. 1, pp. 66–78, Apr. 2021.

A.C. Müller and S. Guido, Introduction to Machine Learning with Python: A Guide for Data Scientists. Sebastopol, CA, USA: O’Reilly Media, 2017.

B.B. Baskoro, I. Susanto, and S. Khomsah, “Analisis sentimen pelanggan hotel di Purwokerto menggunakan metode random forest dan TF-IDF (Studi kasus: Ulasan pelanggan pada situs TRIPADVISOR),” J. Inform. Inf. Syst. Softw. Eng. Appl. (INISTA), vol. 3, no. 2, pp. 21–29, May 2021, doi: 10.20895/inista.v3i2.218.

A. Komarudin and A.M. Hilda, “Analisis sentimen ulasan aplikasi identitas kependudukan digital pada Play Store menggunakan metode naïve Bayes,” Comput. Sci. (CO-SCIENCE), vol. 4, no. 1, pp. 28–36, Jan. 2024, doi: 10.31294/coscience.v4i1.2955.

P.K. Sari and R.R. Suryono, “Komparasi algoritma support vector machine dan random forest untuk analisis sentimen metaverse,” J. Mnemon., vol. 7, no. 1, pp. 31–39, Feb. 2024, doi: 10.36040/mnemonic.v7i1.8977.

R.A. Lestari, A. Erfina, and W. Jatmiko, “Penerapan algoritma support vector machine pada analisis sentimen terhadap identitas kependudukan digital,” J. Teknol. Inf. Ilmu Komput., vol. 10, no. 5, pp. 1063–1070, Oct. 2023, doi: 10.25126/jtiik.20231057264.

M. Ulfa, R.H. Kusumodestoni, and A. Sucipto, “Analisis sentimen review aplikasi identitas kependudukan digital di Google Play Store menggunakan KNN,” J. Inform. Teknol. Sains (JINTEKS), vol. 6, no. 4, pp. 1155–1165, Nov. 2024, doi: 10.33558/jinteks.v6i4.4963.

A. Roihan, P.A. Sunarya, and A.S. Rafika, “Pemanfaatan machine learning dalam berbagai bidang: Review paper,” IJCIT (Indones. J. Comput. Inf. Technol.), vol. 5, no. 1, pp. 75–82, May 2020, doi: 10.31294/ijcit.v5i1.7951.

R. Wati, S. Ernawati, and H. Rachmi, “Pembobotan TF-IDF menggunakan naïve Bayes pada sentimen masyarakat mengenai isu kenaikan BIPIH,” J. Manaj. Inform. (JAMIKA), vol. 13, no. 1, pp. 84–93, Apr. 2023, doi: 10.34010/jamika.v13i1.9424.

Erlin et al., “Dampak SMOTE terhadap kinerja random forest classifier berdasarkan data tidak seimbang,” Matrik: J. Manaj. Tek. Inform. Rekayasa Komput., vol. 21, no. 3, pp. 677–690, Jul. 2022, doi: 10.30812/matrik.v21i3.1726.

A. Mulianti, Y. Chrisnanto, and H. Ashaury, “Optimalisasi klasifikasi support vector machine dengan SMOTE: Studi kasus ulasan pengguna aplikasi Alfagift,” J. Pekommas, vol. 9, no. 2, pp. 249–258, Dec. 2024, doi: 10.56873/jpkm.v9i2.5583.

R. Chairunnisa, Indriati, and P.P. Adikara, “Analisis sentimen terhadap karyawan dirumahkan pada media sosial Twitter menggunakan fitur n-gram dan pembobotan augmented TF-IDF probability dengan k-nearest neighbour,” J. Pengemb. Teknol. Inf. Ilmu Komput., vol. 6, no. 4, pp. 1960–1965, Apr. 2022.

Identitas Kependudukan Digital. (2025). Kementerian Dalam Negeri. Accessed: March. 10, 2025. [Online]. Available: https://play.google.com/store/apps/details?id=gov.dukcapil.mobile_id&hl=id

E. Christy and K. Suryowati, “Analisis klasifikasi status bekerja penduduk Daerah Istimewa Yogyakarta menggunakan metode random forest,” J. Stat. Ind. Komputasi, vol. 6, no. 1, pp. 69–76, Jan. 2021.

D. Normawati and S.A. Prayogi, “Implementasi naïve Bayes classifier dan confusion matrix pada analisis sentimen berbasis teks pada Twitter,” J-SAKTI (J. Sains Komput. Inform.), vol. 5, no. 2, pp. 697–711, Sep. 2021, doi: 10.30645/j-sakti.v5i2.369.