Estimation of Penalized Logistic Regression Models Using Selected Statistical Methods and Deep Learning Techniques

Abstract

This study aims to investigate and classify water pollution in the Groundwater in Iraq by employing a combination of penalized statistical methods and modern artificial intelligence techniques. The study is based on real environmental data provided by the Iraqi Ministry of Environment, in addition to simulated data generated using the Monte Carlo simulation approach. The research focuses on penalized logistic regression models, including LASSO, Adaptive LASSO, SCAD, and Elastic Net, alongside gradient boosting algorithms, particularly the Extreme Gradient Boosting (XGBoost) algorithm, in order to evaluate their interpretative and predictive performance when dealing with high-dimensional and highly correlated environmental data.

The Mean Squared Error (MSE) criterion was adopted to compare the performance of the proposed methods. Simulation results demonstrated that the Elastic Net method achieved the best performance among penalized logistic regression models, yielding the lowest MSE values across all sample sizes. Moreover, the results indicated that the XGBoost algorithm outperformed the traditional Gradient Boosting algorithm, showing superior predictive accuracy and stability. In the practical application to real water quality data from the Tigris River, all studied environmental variables were found to be statistically significant in both the Elastic Net model and the XGBoost algorithm. While the two approaches showed general agreement in the direction of variable effects, XGBoost exhibited a greater ability to capture nonlinear and complex relationships among variables.

The comparative analysis revealed that the Elastic Net model is more suitable for interpretative purposes and identifying the most influential environmental factors, whereas XGBoost provides higher predictive accuracy. These findings confirm that integrating penalized statistical models with artificial intelligence techniques offers a comprehensive and effective analytical framework for studying water pollution.