PSO-UFS: A Novel Approach to Univariate Feature Selection Using Particle Swarm Optimization
DOI:
https://doi.org/10.22399/ijcesen.4033Keywords:
univariate feature selection, particle swarm optimization, interpretable classification performance, automated feature selection, data analysisAbstract
Univariate Feature Selection (UFS) traditionally involves a labor-intensive process of trial-and error, necessitating the selection of scoring functions and the determination of feature numbers. These choices can inadvertently affect both the performance and interpretability of the model. To address this challenge, we introduce Particle Swarm Optimization for Univariate Feature Selection (PSO-UFS), an innovative method that automates these crucial decisions. PSO-UFS leverages the power of Particle Swarm Optimization (PSO) to autonomously identify the optimal scoring function and feature subset that maximize a machine learning algorithm’s performance metric. Our empirical evaluations across multiple datasets demonstrate that PSO-UFS significantly outperforms traditional UFS in various performance metrics, including accuracy, precision, recall, and F1-score. Importantly, PSO-UFS generates more interpretable feature subsets, thereby enhancing the model’s comprehensibility. This advancement paves the way for broader applications in real-world scenarios where feature reduction and interpretability are paramount.
References
[1] Tengku Mazlin Tengku Ab Hamid, Roselina Sallehuddin, Zuriahati Mohd Yunos, and Aida Ali. Ensemble based filter feature selection with harmonize particle swarm optimization and support vector machine for optimal cancer classification. Machine Learning with Applications, 5:100054, 2021.
[2] Dharyll Prince M Abellana and Demelo M Lao. A new univariate feature selection algorithm based on the best–worst multi-attribute decision-making method. Decision Analytics Journal, 7:100240, 2023.
[3] Esra’a Alhenawi, Rizik Al-Sayyed, Amjad Hudaib, and Seyedali Mirjalili. Feature selection methods on gene expression microarray data for cancer classification: A systematic review. Computers in Biology and Medicine, 140:105051, 2022.
[4] Theyab R Alsenani, Safial Islam Ayon, Sayeda Mayesha Yousuf, Fahad Bin Kamal Anik, and Mohammad Ehsan Shahmi Chowdhury. Intelligent feature selection model based on particle swarm optimization to detect phishing websites. Multimedia Tools and Applications, pages 1–33, 2023.
[5] Barry Becker and Ronny Kohavi. Adult. UCI Machine Learning Repository, 1996. DOI: https://doi.org/10.24432/C5XW20.
[6] Andrea Bommert, Thomas Welchowski, Matthias Schmid, and J¨org Rahnenf¨uhrer. Benchmark of filter methods for feature selection in high-dimensional gene expression survival data. Briefings in Bioinformatics, 23(1): bbab354, 2022.
[7] Mohammed Mehdi Bouchene and Kheireddine Abainia. Classical machine learning and transformer models for offensive and abusive language classification on dziri language. In 2023 International Conference on Decision Aid Sciences and Applications (DASA), pages 116–120. IEEE, 2023.
[8] Gavin Brown, Adam Pocock, Ming-Jie Zhao, and Mikel Luj´an. Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. The journal of machine learning research, 13:27–66, 2012.
[9] Enrico Civitelli, Matteo Lapucci, Fabio Schoen, and Alessio Sortino. An effective procedure for feature subset selection in logistic regression based on information criteria. Computational Optimization and Applications, 80(1):1–32, 2021.
[10] M. Dash and H. Liu. Feature selection for clustering-a filter solution. ICDM, 1:115–122, 2000.
[11] Xuelian Deng, Yuqing Li, Jian Weng, and Jilian Zhang. Feature selection for text classification: A review. Multimedia Tools and Applications, 78:3797–3816, 2019.
12] Peter Drot´ar, Juraj Gazda, and Zdenek Sm´ekal. An experimental comparison of feature selection methods on two-class biomedical datasets. Computers in biology and medicine, 66:1–10, 2015.
[13] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[14] R. A. Fisher. Iris. UCI Machine Learning Repository, 1988. DOI: https://doi.org/10.24432/C56C76.
[15] Ahmed G Gad. Particle swarm optimization algorithm and its applications: a systematic review. Archives of computational methods in engineering, 29(5):2531–2561, 2022.
[16] I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh. Feature extraction: foundations and applications. 207, 2006. [17] Isabelle Guyon and Andr´e Elisseeff. An introduction to variable and feature selection. Journal of machine learning research, 3(Mar):1157–1182, 2003.
[18] Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene selection for cancer classification using support vector machines. Machine learning, 46:389–422, 2002.
[19] Ramesh Kumar Huda and Haider Banka. Efficient feature selection methods using pso with fuzzy rough set as fitness function. Soft Computing, pages 1–21, 2022.
[20] Meetu Jain, Vibha Saihjpal, Narinder Singh, and Satya Bir Singh. An overview of variants and advancements of pso algorithm. Applied Sciences, 12(17):8392, 2022.
[21] Shivani Jain and Anju Saha. Rank-based univariate feature selection methods on machine learning classifiers for code smell detection. Evolutionary Intelligence, 15(1):609–638, 2022.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
