A Comparative Study of XGBoost and Transformer for E-commerce Purchase Prediction under Class Imbalance
关键词:
E-Commerce Purchase Prediction, Class Imbalance, Xgboost, Transformer, SMOTE, Behavioral Sequence Modeling摘要
Class imbalance poses a persistent challenge in e-commerce purchase prediction, where positive samples rarely exceed 10% of observations. This study compares XGBoost and Transformer architectures under controlled imbalance conditions using the Tmall Repeat Purchase Prediction dataset (6.12% positive rate). A 2 × 2 factorial design crossed model type (XGBoost vs. Transformer) with balancing strategy (original imbalanced vs. SMOTE-balanced), drawing on 22 tabular features and 20-length behavioral sequences derived from user interaction logs. On the original data, XGBoost outperformed Transformer (AUC 0.657 vs. 0.631; F1 0.194 vs. 0.169). SMOTE, counter to expectations, degraded both models. Ablation experiments confirmed this was not an artifact of the sampling ratio. At extreme imbalance (1% positive rate), the pattern reversed: Transformer proved more robust, its AUC falling by only 1.7% against XGBoost’s 9.1%, and surpassing XGBoost on AUC at this level. These results challenge the common assumption that tree-based models are always preferable under class imbalance and offer practical guidance for model selection in real-world e-commerce settings.##submission.downloads##
已出版
2026-08-31
期次
栏目
Articles
##submission.license##
##submission.copyrightStatement##
##submission.license.cc.by-nc-nd4.footer####submission.howToCite##
A Comparative Study of XGBoost and Transformer for E-commerce Purchase Prediction under Class Imbalance. (2026). CPS Digital Library - Series of Conferences, 66-75. https://seriesofconference.com/index.php/SCJ/article/view/402



