Q-LungBench: Leakage-Aware Benchmarking of Machine Learning Models for Questionnaire-Based Lung Cancer Risk Classification

Abstract

Questionnaire-based lung cancer datasets are widely used in machine-learning studies, yet their small size, class imbalance, duplicate records, and test-guided model choices can make performance appear better than it is. We developed Q-LungBench as a reproducible, leakage-aware framework for evaluating six classifiers on the public Survey Lung Cancer dataset. After exact duplicates were removed before data partitioning, 276 unique records with 15 predictors remained. The data were divided using a stratified 80:20 development-test split. On the development set only, logistic regression, radial-basis-function support vector machine, random forest, gradient boosting, k-nearest neighbours, and Gaussian naive Bayes were tuned with five-fold stratified cross-validation. The prespecified selection criterion, mean balanced accuracy, identified logistic regression as the final model (0.854) before independent testing. On the 56-record test set, logistic regression achieved 91.07% accuracy, 94.79% balanced accuracy, 89.58% sensitivity, 100% specificity, an F1-score of 94.51%, a Matthews correlation coefficient of 0.742, a ROC-AUC of 0.971 (95% bootstrap confidence interval 0.917-1.000), and a PR-AUC of 0.995. Permutation analysis highlighted allergy, swallowing difficulty, wheezing, coughing, and chest pain as the variables on which the fitted model relied most. The findings indicate that transparency in the assessment process does not necessarily mean inferiority without any fusion or artificial oversampling afterward. However, the limited size of negative instances and the uncertain source of the dataset prevent us from making any diagnosis at this point.