TY - JOUR
T1 - The accuracy of machine learning models relies on hyperparameter tuning
T2 - student result classification using random forest, randomized search, grid search, bayesian, genetic, and optuna algorithms
AU - Rimal, Yagyanath
AU - Sharma, Navneet
AU - Alsadoon, Abeer
N1 - Publisher Copyright:
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.
PY - 2024/2
Y1 - 2024/2
N2 - Hyperparameters play a critical role in analyzing predictive performance in machine learning models. They serve to strike a balance between overfitting and underfitting of research-independent features to prevent extremes. Manual tuning and automated techniques are employed to identify the optimal combination and permutation to achieve the best model performance. This study explores the pursuit of the best fit through various hyperparameters. Following Logistic Regression analysis, this research compared Random Forest, Randomized search, Grid search, Genetic, Bayesian, and Optuna machine learning model tuning for the best accuracy of prediction the student The model accuracy was further assessed using confusion matrices and Receiver Operating Characteristic—Area Under the Curve (ROC-AUC) curves for student grade classification. The genetic algorithm's recommended hyperparameter tuning yielded the highest accuracy (82.5%) and AUC-ROC score (90%) for student result classification. Manual tuning with an estimator of 300, criterion entropy, max features of sqrt, and a minimum sample leaf of 10 achieved an accuracy of 81.1%, which closely resembled the performance randomized search cross-validation algorithm. The default random forest model scored the least accuracy (78%). However, this manual tuning process took a lesser time (3.66 s) to fit the model while grid search CV tuned 941.5 s. Hence, this research made significant contributions to optimizing various machine learning models using a range of hyperparameters for grade classification.
AB - Hyperparameters play a critical role in analyzing predictive performance in machine learning models. They serve to strike a balance between overfitting and underfitting of research-independent features to prevent extremes. Manual tuning and automated techniques are employed to identify the optimal combination and permutation to achieve the best model performance. This study explores the pursuit of the best fit through various hyperparameters. Following Logistic Regression analysis, this research compared Random Forest, Randomized search, Grid search, Genetic, Bayesian, and Optuna machine learning model tuning for the best accuracy of prediction the student The model accuracy was further assessed using confusion matrices and Receiver Operating Characteristic—Area Under the Curve (ROC-AUC) curves for student grade classification. The genetic algorithm's recommended hyperparameter tuning yielded the highest accuracy (82.5%) and AUC-ROC score (90%) for student result classification. Manual tuning with an estimator of 300, criterion entropy, max features of sqrt, and a minimum sample leaf of 10 achieved an accuracy of 81.1%, which closely resembled the performance randomized search cross-validation algorithm. The default random forest model scored the least accuracy (78%). However, this manual tuning process took a lesser time (3.66 s) to fit the model while grid search CV tuned 941.5 s. Hence, this research made significant contributions to optimizing various machine learning models using a range of hyperparameters for grade classification.
KW - Automate hyperparameter tuning
KW - Bayesian
KW - Characteristic area under the curve
KW - Educational data mining
KW - Random forest
KW - Randomized search cross validation
KW - Tree-based pipeline optimization tool
UR - http://www.scopus.com/inward/record.url?scp=85185099417&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85185099417&partnerID=8YFLogxK
U2 - 10.1007/s11042-024-18426-2
DO - 10.1007/s11042-024-18426-2
M3 - Article
AN - SCOPUS:85185099417
SN - 1380-7501
SP - 74349
EP - 74364
JO - Multimedia Tools and Applications
JF - Multimedia Tools and Applications
ER -