Predicting National Research Project Delays: A Comparative Analysis Of Random Forest, Xgboost, And Logistic Regression
Abstract
Purpose: This study identifies the determinants of completion delay in national research projects and compares the predictive performance of Random Forest, XGBoost, and Logistic Regression in the context of the Productive Innovative Research (RISPRO) program managed by the Education Fund Management Institution (LPDP). Methods: Historical data on completed RISPRO projects from 2013 to February 2026 were integrated from selection, disbursement, reporting, monitoring, and no-cost extension databases. The final modeling data were processed through a leakage-safe workflow that combined project metadata with title-based TF-IDF features, standardized numeric variables, encoded categorical variables, and the application of SMOTE only to the training folds. Determinants were triangulated through Mann-Whitney U and chi-square tests, Logistic Regression coefficients and odds ratios, and tree-based feature importance. Model performance was evaluated using repeated stratified five-fold cross-validation, producing 25 estimates per algorithm, followed by Friedman, Wilcoxon signed-rank, and McNemar tests. Results: Project duration is the most consistent determinant: a one-standard-deviation increase, approximately 0.60 years, raises the odds of delay by 3.34 times. Funding shows a non-linear pattern, with projects funded at IDR 3-5 billion showing the highest odds of delay, 4.72 times the reference category, while projects above IDR 5 billion show lower risk. Slower second-stage disbursement and slower monitoring are also associated with higher delay risk. Random Forest achieved the highest mean F1-score, 0.907, and recall, 0.942, although the overall Friedman test showed no statistically significant difference among the three algorithms, p = 0.0692. Implications: LPDP can use these models as an early-warning instrument to prioritize assistance for projects at higher risk of delay, accelerate second-stage disbursement, and implement risk-based monitoring. Random Forest is suitable as the primary screening engine, while Logistic Regression can provide an interpretive explanatory layer for accountable public-sector decision-making.References
Arana-Barbier, P. J. (2023). The relationship between scientific production and economic growth through R&D investment: A bibliometric approach. Journal of Scientometric Research, 12(3), 596-602.
Baccarini, D. (1996). The concept of project complexity: A review. International Journal of Project Management, 14(4), 201-204. https://doi.org/10.1016/0263-7863(95)00093-3
Bank Indonesia. (n.d.). Data inflasi. Retrieved October 26, 2025, from https://www.bi.go.id/id/statistik/indikator/data-inflasi.aspx
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2021). Beyond accuracy: The role of mental models in human-AI team performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 9, 2-11.
Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13, 281-305.
Bimantara, H. C., Rokim, A., & Rilvani, E. (2025). Data mining untuk estimasi waktu produksi dan pengiriman komponen prefab berdasarkan riwayat proyek. Jurnal Media Akademik, 3(7).
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321-357. https://doi.org/10.1613/jair.953
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785-794. https://doi.org/10.1145/2939672.2939785
Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1-30.
Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7), 1895-1923. https://doi.org/10.1162/089976698300017197
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861-874. https://doi.org/10.1016/j.patrec.2005.10.010
Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675-701. https://doi.org/10.1080/01621459.1937.10503522
Giri, M., Zhang, C., Olorunnishola, A., Hada, S., & Pal, U. K. (2025). Developing a construction delay prediction model as a delay-avoidance strategy for public construction projects. Journal of Legal Affairs and Dispute Resolution in Engineering and Construction, 17(3). https://doi.org/10.1061/JLADAH.LADR-1276
Government of the Republic of Indonesia. (2021). Presidential Regulation of the Republic of Indonesia No. 111 of 2021 concerning endowment funds in education.
Han, J., Kamber, M., & Pei, J. (2012). Data mining: Concepts and techniques (3rd ed.). Morgan Kaufmann.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer.
Heagney, J. (2016). Fundamentals of project management (5th ed.). AMACOM.
Hidayatuloh, R. (2023). Evaluasi skema pendanaan riset LPDP pendekatan Triple Helix Balance Model dan Open Innovation [Master's thesis, Institut Teknologi Sepuluh Nopember].
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). Wiley.
International Organization for Standardization. (2018). ISO 31000:2018 risk management - Guidelines.
Irawan, B. (2025). Faktor-faktor yang memengaruhi kinerja tim riset pada RISPRO Invitasi Balance Triple Helix Model [Master's thesis, IPB University].
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning: With applications in R. Springer.
Jurafsky, D., & Martin, J. H. (2021). Speech and language processing (3rd ed. draft). Stanford University.
Juszczak, P., Zio, E., & Grolinger, T. (2002). The impact of data normalization on classification performance. Proceedings of the 7th International Conference on Database Systems for Advanced Applications.
Kannan, R., Abdul Halim, H. A., Ramakrishnan, K., Ismail, S., & Wijaya, D. R. (2022). Machine learning approach for predicting production delays: A quarry company case study. Journal of Big Data, 9, Article 94. https://doi.org/10.1186/s40537-022-00644-w
Kaufman, S., Rosset, S., Perlich, C., & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data, 6(4), Article 15. https://doi.org/10.1145/2382577.2382579
Kerzner, H. (2017). Project management: A systems approach to planning, scheduling, and controlling (12th ed.). Wiley.
Kleinbaum, D. G., & Klein, M. (2010). Logistic regression: A self-learning text (3rd ed.). Springer.
Kotsiantis, S., Kanellopoulos, D., & Pintelas, P. (2007). Handling missing values in classification problems: A comparative study. International Journal of Artificial Intelligence and Applications, 6(4).
Lembaga Penyelidikan Ekonomi dan Masyarakat, Fakultas Ekonomi dan Bisnis Universitas Indonesia. (2023). Laporan pengukuran dampak program pendanaan riset RISPRO LPDP. LPDP.
Lock, D. (2020). Project management (11th ed.). Routledge.
LPDP. (2020a). Pedoman pendanaan riset inovatif produktif LPDP 2020. Ministry of Finance of the Republic of Indonesia.
LPDP. (2020b). Pedoman RISPRO Kolaborasi Internasional. Ministry of Finance of the Republic of Indonesia.
LPDP. (2021a). Laporan kinerja 2020. Ministry of Finance of the Republic of Indonesia.
LPDP. (2021b). Laporan tahunan 2020. Ministry of Finance of the Republic of Indonesia.
LPDP. (2021c). Peraturan Direktur Utama LPDP No. PER-18/LPDP/2021 concerning RISPRO mandatory funding guidelines. Ministry of Finance of the Republic of Indonesia.
LPDP. (2022a). Laporan kinerja 2021. Ministry of Finance of the Republic of Indonesia.
LPDP. (2022b). Laporan tahunan 2021. Ministry of Finance of the Republic of Indonesia.
LPDP. (2023a). Laporan kinerja 2022. Ministry of Finance of the Republic of Indonesia.
LPDP. (2023b). Laporan tahunan 2022. Ministry of Finance of the Republic of Indonesia.
LPDP. (2023c). Peraturan Direktur Utama LPDP No. PER-24/LPDP/2023 concerning institutional activity cost standards. Ministry of Finance of the Republic of Indonesia.
LPDP. (2024a). Laporan kinerja 2023. Ministry of Finance of the Republic of Indonesia.
LPDP. (2024b). Laporan tahunan 2023. Ministry of Finance of the Republic of Indonesia.
LPDP. (2025a). Laporan kinerja 2024. Ministry of Finance of the Republic of Indonesia.
LPDP. (2025b). Laporan tahunan 2024. Ministry of Finance of the Republic of Indonesia.
LPDP. (2025c). Peraturan Direktur Utama LPDP No. PER-50/LPDP/2025 concerning no-cost extensions for RISPRO funding.
LPDP. (n.d.-a). Riset Inovatif Produktif (RISPRO). https://risprolpdp.kemenkeu.go.id
LPDP. (n.d.-b). Visi dan misi. Retrieved October 29, 2025, from https://lpdp.kemenkeu.go.id/tentang/visi-misi/
Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30.
Mandrekar, J. N. (2010). Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology, 5(9), 1315-1316. https://doi.org/10.1097/JTO.0b013e3181ec173d
McKinney, W. (2010). Data structures for statistical computing in Python. Proceedings of the 9th Python in Science Conference, 56-61.
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12, 153-157. https://doi.org/10.1007/BF02295996
Meredith, J. R., & Mantel, S. J. (2014). Project management: A managerial approach (8th ed.). Wiley.
Ministry of Finance of the Republic of Indonesia. (2020). Regulation No. 47/PMK.01/2020 concerning the organization and governance of LPDP.
Ministry of Finance of the Republic of Indonesia. (2025). Regulation No. 32 of 2025 concerning standard input costs for fiscal year 2026.
Ministry of Higher Education, Science, and Technology. (2025, July 31). Dukung Asta Cita, Kemdiktisaintek kuatkan riset untuk pemerataan dan pertumbuhan ekonomi.
Mitchell, T. M. (1997). Machine learning. McGraw-Hill.
Molnar, C. (2020). Interpretable machine learning: A guide for making black box models explainable. Independent publication.
Molnar, C. (2022). Interpretable machine learning (2nd ed.). Independent publication.
Movafaghpour, M. A. (2023). Predicting project delays using a new trended regression tree method. Shahed Journal of Civil Engineering.
Nguyen, L. H. (2020). Empirical analysis of a management function's failures in construction project delay. Journal of Open Innovation: Technology, Market, and Complexity, 6(2).
OECD. (2015). Frascati manual 2015: Guidelines for collecting and reporting data on research and experimental development. OECD Publishing. https://doi.org/10.1787/9789264239012-en
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825-2830.
Powers, D. M. W. (2011). Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies, 2(1), 37-63.
Prasojo, L. D., Yuliana, L., & Prihandoko, L. A. (2023). Kinerja penelitian di pendidikan tinggi: Analisis PLS-SEM terhadap suasana penelitian, kolaborasi, pendanaan, kompetensi, dan luaran.
Project Management Institute. (2017). A guide to the project management body of knowledge (PMBOK guide) (6th ed.).
Project Management Institute. (2021). A guide to the project management body of knowledge (PMBOK guide) (7th ed.).
Reuters. (2025, August 22). South Korea's Lee says plans record $25 billion government spending on research in 2026.
Sagi, O., & Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1249. https://doi.org/10.1002/widm.1249
Sahu, P., Bera, D. K., Parhi, P. K., & Kandpal, M. (2025). Smart delay prediction: Supervised machine learning solutions for construction projects. Journal of Mechanics of Continua and Mathematical Sciences, 20(6).
Sarsiti, S., & Rosid, T. (2025). Peran inovasi dan teknologi dalam meningkatkan pertumbuhan ekonomi menuju Indonesia Emas 2045. Sosial Market Journal, 7(1).
Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427-437. https://doi.org/10.1016/j.ipm.2009.03.002
Turner, J. R. (2020). Handbook of project-based management: Leading strategic change in organizations (4th ed.). McGraw-Hill Education.
Turner, J. R., & Müller, R. (2019). The handbook of project management: Leading organizational change (4th ed.). McGraw-Hill Education.
Uddin, S., Ong, S., & Lu, H. (2022). Machine learning in project analytics: A data-driven framework and case study. Scientific Reports, 12, Article 15252. https://doi.org/10.1038/s41598-022-19728-x
Vabalas, A., Gowen, E., Poliakoff, E., & Casson, A. J. (2019). Machine learning algorithm validation with a limited sample size. PLOS ONE, 14(11), e0224365. https://doi.org/10.1371/journal.pone.0224365
Vasić, J., Kecman, N., & Mladenović, I. (2016). Research and development investment as determinant of international competitiveness and economic growth in EU28 and Serbia. Economic Themes, 54(2), 282-298. https://doi.org/10.1515/ethemes-2016-0010
Wideman, R. M. (2019). Project and program risk management: A guide to managing project risks and opportunities. AEW Services.
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80-83. https://doi.org/10.2307/3001968
Wolpert, D. H. (1996). The lack of a priori distinctions between learning algorithms. Neural Computation, 8(7), 1341-1390. https://doi.org/10.1162/neco.1996.8.7.1341
Yaseen, Z. M., Ali, Z. H., Salih, S. Q., & Al-Ansari, N. (2020). Prediction of risk delay in construction projects using a hybrid artificial intelligence model. Sustainability, 12(4), Article 1514. https://doi.org/10.3390/su12041514
Zhang, L., Wu, X., & Skibniewski, M. J. (2020). Risk identification and assessment in construction projects using machine learning. Automation in Construction, 118, Article 103275. https://doi.org/10.1016/j.autcon.2020.103275
Copyright (c) 2026 Kontigensi : Jurnal Ilmiah Manajemen

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
By exercising the Licensed Rights, You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial 4.0 International Public License.
























