slot gacor https://vta.kemenhub.go.id/assets/
kentangbet
ROBUSTNESS EVALUATION OF GRADIENT BOOSTING MODELS FOR GRADUATION PREDICTION UNDER COHORT-BASED DISTRIBUTION SHIFTS | Daniswara | JIKO (Jurnal Informatika dan Komputer)

ROBUSTNESS EVALUATION OF GRADIENT BOOSTING MODELS FOR GRADUATION PREDICTION UNDER COHORT-BASED DISTRIBUTION SHIFTS

Rifandito Daniswara, Chanifah Indah Ratnasari

Abstract


Student graduation rate is a critical performance indicator for higher education institutions, particularly in accreditation assessment. Early prediction of on-time graduation supports academic planning and quality assurance. Although prior studies report high predictive accuracy using conventional cross-validation, limited attention has been given to robustness under cohort-based distribution shifts. This study evaluates the robustness of three gradient boosting models—Histogram-Based Gradient Boosting (HGB), Extreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM)—for predicting on-time graduation using structured academic trajectory data from 370 labeled instances across three cohorts. Two validation strategies were employed: a stratified 80:20 split for aggregated evaluation and Leave-One-Group-Out (LOGO) validation to simulate cohort-based distribution shifts. Under stratified evaluation, all models achieved macro F1-scores above 0.74, with HGB obtaining the highest score (0.7568). However, LOGO evaluation revealed substantial performance degradation, with mean F1-scores below 0.51 and increased variability across cohorts. XGBoost demonstrated comparatively better stability under distribution shifts. These findings indicate that high predictive accuracy under random splits does not guarantee cross-cohort robustness, highlighting the importance of distribution-aware validation for reliable deployment in educational data mining.

References


Badan Akreditasi Nasional Perguruan Tinggi, “IAPS 5.0,” 2025. Available: https://www.banpt.or.id/wp-content/uploads/2025/06/Peraturan-BAN-PT-Nomor-13-Tahun-2025-ttg-IAPS-5.0.pdf

E. Haryatmi and S. Pramita Hervianti, “Penerapan Algoritma Support Vector Machine Untuk Model Prediksi Kelulusan Mahasiswa Tepat Waktu,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 5, no. 2, pp. 386–392, Apr. 2021, doi: 10.29207/resti.v5i2.3007.

N. Yustira, D. Witarsyah, and E. Sutoyo, “Implementasi Algoritma NaÏve Bayes Classification Untuk Klasifikasi Kelulusan Mahasiswa Tepat Waktu,” eProceedings of Engineering, vol. 8, no. 5, Oct. 2021. Available: https://openlibrarypublications.telkomuniversity.ac.id/index.php/engineering/article/view/16721

E. Novianto, A. Hermawan, and D. Avianto, “Klasifikasi Algoritma K-Nearest Neighbor, Naive Bayes, dan Decision Tree untuk Prediksi Status Kelulusan Mahasiswa S1,” Rabit : Jurnal Teknologi dan Sistem Informasi Univrab, vol. 8, no. 2, pp. 146–154, Jul. 2023, doi: 10.36341/rabit.v8i2.3434.

G. A. J. Satvika, I. N. Sukajaya, and I. G. A. Gunadi, “Improving k-nearest neighbor performance using permutation feature importance to predict student success in study,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 35, no. 3, pp. 1835–1844, Sep. 2024, doi: 10.11591/ijeecs.v35.i3.pp1835-1844.

I. Nirmala, H. Wijayanto, and K. A. Notodiputro, “Prediction of Undergraduate Student’s Study Completion Status Using MissForest Imputation in Random Forest and XGBoost Models,” ComTech: Computer, Mathematics and Engineering Applications, vol. 13, no. 1, pp. 53–62, Feb. 2022, doi: 10.21512/comtech.v13i1.7388.

A. J. Fernandez-Garcia, J. C. Preciado, F. Melchor, R. Rodriguez-Echeverria, J. M. Conejero, and F. Sanchez-Figueroa, “A real-life machine learning experience for predicting university dropout at different stages using academic data,” IEEE Access, vol. 9, pp. 133076–133090, 2021, doi: 10.1109/ACCESS.2021.3115851.

M. Windarti and P. T. Prasetyaninrum, “Prediction Analysis Student Graduate Using Multilayer Perceptron,” in Proceedings of the International Conference on Online and Blended Learning 2019 (ICOBL 2019), Paris, France: Atlantis Press, May 2020, pp. 53–57. doi: 10.2991/assehr.k.200521.011.

A. Salam and J. Zeniarja, “Classification of deep learning convolutional neural network feature extraction for student graduation prediction,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 32, no. 1, pp. 335–341, Oct. 2023, doi: 10.11591/IJEECS.V32.I1.PP335-341.

P. W. Koh et al., “WILDS: A Benchmark of in-the-Wild Distribution Shifts,” Jul. 2021, doi: 10.48550/arXiv.2012.07421.

Z. Shi et al., “Effective robustness against natural distribution shifts for models with different training data,” in Proceedings of the 37th International Conference on Neural Information Processing Systems, in NIPS ’23. Red Hook, NY, USA: Curran Associates Inc., Dec. 2023. doi: 10.5555/3666122.3669339.

Z. Liu, J. Van Niekerk, and H. Rue, “Leave-group-out cross-validation for latent Gaussian models,” Jul. 2025, doi: 10.57645/20.8080.02.25.

A. Adin, E. T. Krainski, A. Lenzi, Z. Liu, J. Martínez-Minaya, and H. Rue, “Automatic cross-validation in structured models: Is it time to leave out leave-one-out?,” Spat. Stat., vol. 62, Aug. 2024, doi: 10.1016/j.spasta.2024.100843.

T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 785–794. doi: 10.1145/2939672.2939785.

G. Ke et al., “LightGBM: a highly efficient gradient boosting decision tree,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, in NIPS’17. Red Hook, NY, USA: Curran Associates Inc., 2017, pp. 3149–3157. doi: 10.5555/3294996.3295074.

F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Jun. 2018, doi: 10.48550/arXiv.1201.0490.

M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Inf. Process. Manag., vol. 45, no. 4, pp. 427–437, Jul. 2009, doi: 10.1016/j.ipm.2009.03.002.

T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition (Springer Series in Statistics), 2nd ed. NY: Springer New York, 2009. doi: 10.1007/978-0-387-84858-7.

Z. C. Lipton, C. Elkan, and B. Naryanaswamy, “Optimal Thresholding of Classifiers to Maximize F1 Measure,” in Machine Learning and Knowledge Discovery in Databases, F. and H. E. and M. R. Calders Toon and Esposito, Ed., Berlin, Heidelberg: Springer Berlin Heidelberg, 2014, pp. 225–239. doi: 10.1007/978-3-662-44851-9_15.

A. Agresti, Categorical data analysis. Wiley, 2013.

I. Gulrajani and D. Lopez-Paz, “In Search of Lost Domain Generalization,” 2020. Available: https://arxiv.org/abs/2007.01434




DOI: https://doi.org/10.33387/jiko.v9i1.11546

Refbacks

  • There are currently no refbacks.