The Multi-factor based Regression Test Case Prioritization using Fuzzy Logic
Abstract
The maintenance level activity generally done after the modification in the software to check whether it is functioning right or not is termed as regression testing. Test case prioritization, a key practice, involves strategically ordering test cases based on specific criteria to enhance the efficiency of fault detection within a condensed time frame. The fuzzy rule base serves as an alternative to the conventional crisp value set, offering a nuanced approach beyond binary outcomes (Yes or No). The primary objective of this research is to address critical factors often overlooked in existing literature on prioritization. Notably, prevalent approaches focus on singular factors during test case prioritization, highlighting the need for a comprehensive technique. To enhance the prioritization of test cases, there is a demand for a method that considers multi-factors or combinations thereof, ultimately increasing effectiveness. This paper introduces an innovative approach a multi-factors regression test-case prioritization technique utilizing fuzzy rules. The methodology aims to optimize the prioritization of test cases, striking a balance between effectiveness and time efficiency. Fuzzy rules are formulated to assess the effectiveness of a prioritized set of test cases in developing the proposed approach. A user-friendly tool has been developed to facilitate the application of this technique, allowing users to input relevant factors and subsequently prioritize test cases accordingly. Through extensive experiments using the developed tool, the effectiveness of the proposed approach has been validated. The results demonstrate that the priority lists of test cases generated for different projects, considering multi-factors, show greater promise compared to techniques relying solely on a single factor for prioritization.
References
\bibitem{b1}
Nayak, S., Kumar, C., Tripathi, S. et al. Regression test optimization and prioritization using Honey Bee optimization algorithm with fuzzy rule base. Soft Comput 25, 9925–9942 (2021). https://doi.org/10.1007/s00500-020-05428-z
\bibitem{b2}
N. Jazdi, P. Göhner and C. Malz, "Prioritization of Test Cases Using Software Agents and Fuzzy Logic," in 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, Montreal, Quebec Canada, 2012 pp. 483-486.
doi: 10.1109/ICST.2012.131
\bibitem{b3}
Badhera, U., and Biswas, D. (2013). Test case prioritization using fuzzy logic based on requirement prioritizing. International Journal on Computational Sciences \& Applications, 3(2).
\bibitem{b4}
H. K. N. Leung and L. White, "Insights into regression testing (software testing)," Proceedings. Conference on Software Maintenance - 1989, 1989, pp. 60-69, doi: 10.1109/ICSM.1989.65194.
\bibitem{b5}
G. Rothermel, R. H. Untch, Chengyun Chu and M. J. Harrold, "Prioritizing test cases for regression testing," in IEEE Transactions on Software Engineering, vol. 27, no. 10, pp. 929-948, Oct. 2001, doi: 10.1109/32.962562.
\bibitem{b6}
Valášková, K., Klieštik, T., \& Mišánková, M. (2014). The role of fuzzy logic in decision making process. In 2014 2nd international conference on management innovation and business innovation (Vol. 44, No. 1, pp. 143-148).
\bibitem{b7}
Hellmann, M. (2001). Fuzzy logic introduction. Université de Rennes, 1, 1-9.
\bibitem{b8}
Hettiarachchi, C., \& Do, H. (2019, July). A systematic requirements and risks-based test case prioritization using a fuzzy expert system. In 2019 IEEE 19th international conference on software quality, reliability and security (QRS) (pp. 374-385). IEEE.
\bibitem{b9}
Xu, Z., Gao, K., Khoshgoftaar, T. M., \& Seliya, N. (2014). System regression test planning with a fuzzy expert system. Information Sciences,259, 532-543
\bibitem{b10}
Yadav, D. K., \& Dutta, S. (2016, March). Test case prioritization technique based on early fault detection using fuzzy logic. In 2016 3rd international conference on computing for sustainable global development (INDIACom) (pp. 1033-1036). IEEE.
\bibitem{b11}
Tahvili, S., Afzal, W., Saadatmand, M., Bohlin, M., Sundmark, D., \& Larsson, S. (2016). Towards earlier fault detection by value-driven prioritization of test cases using fuzzy TOPSIS. In Information Technology: New Generations (pp. 745-759). Springer, Cham.
\bibitem{b12}
Silva, D., Rabelo, R., Campanha, M., Neto, P. S., Oliveira, P. A., \& Britto, R. (2016, July). A hybrid approach for test case prioritization and selection. In 2016 IEEE Congress on Evolutionary Computation (CEC) (pp. 4508-4515). IEEE.
\bibitem{b13}
Schwartz, A., \& Do, H. (2013, September). A fuzzy expert system for cost-effective regression testing strategies. In 2013 IEEE International Conference on Software Maintenance (pp. 1-10). IEEE.
\bibitem{b14}
Hettiarachchi, C., Do, H., \& Choi, B. (2016). Risk-based test case prioritization using a fuzzy expert system. Information and Software Technology, 69, 1-15.
\bibitem{b15}
Xu, Z., Gao, K., \& Khoshgoftaar, T. M. (2005, August). Application of fuzzy expert system in test case selection for system regression test. In IRI-2005 IEEE International Conference on Information Reuse and Integration, Conf, 2005. (pp. 120-125). IEEE.
\bibitem{b16}
Alakeel, A. M. (2014). Using fuzzy logic in test case prioritization for regression testing programs with assertions. The Scientific World Journal, 2014.
\bibitem{b17}
C. Hettiarachchi and H. Do, "A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System," 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), 2019, pp. 374-385, doi: 10.1109/QRS.2019.00054.
\bibitem{b18}
C. Hettiarachchi and H. Do, "A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System," 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), 2019, pp. 374-385, doi: 10.1109/QRS.2019.00054.
\bibitem{19}
Minh, Vu Trieu, Katushin, Dmitri, Antonov, Maksim and Veinthal, Renno. "Regression Models and Fuzzy Logic Prediction of TBM Penetration Rate" Open Engineering, vol. 7, no. 1, 2017, pp. 60-68. https://doi.org/10.1515/eng-2017-0012.
\bibitem{20}
Leung, H. K., \& White, L. (1989, October). Insights into regression testing (software testing). In Proceedings. Conference on Software Maintenance-1989 (pp. 60-69). IEEE.
\bibitem{21}
Yoo, S., \& Harman, M. (2012). Regression testing minimization, selection and prioritization: a survey. Software testing, verification and reliability, 22(2), 67-120.
\bibitem{22}
Elbaum, S., Malishevsky, A. G., \& Rothermel, G. (2000, August). Prioritizing test cases for regression testing. In Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis (pp. 102-112).
\bibitem{23}
Binkley, D. (1997). Semantics guided regression test cost reduction. IEEE Transactions on Software Engineering, 23(8), 498-516.
\bibitem{24}
Myers, G. J., Sandler, C.,\& Badgett, T. (2011). The art of software testing. John Wiley \& Sons.
\bibitem{25}
Stuebing, H. G. (1980). A modern facility for software production and maintenance. 1980. 14.
\bibitem{26}
Rothermel, G., \& Harrold, M. J. (1997). A safe, efficient regression test selection technique. ACM Transactions on Software Engineering and Methodology (TOSEM), 6(2), 173-210.
\bibitem{27}
Pan, J. (1999). Software testing. Dependable Embedded Systems, 5, 2006.
\bibitem{28}
Rosenblum, D. S., \& Weyuker, E. J. (1997). Using coverage information to predict the cost-effectiveness of regression testing strategies. IEEE Transactions on Software Engineering, 23(3), 146-156.
\bibitem{29}
J. S. Collofello and J. J. Buck, “Software quality assurance for maintenance,” IEEE Software, pp. 46-51, Sept. 1987
\bibitem{30}
Rothermel, G., \& Harrold, M. J. (1998). Empirical studies of a safe regression test selection technique. IEEE Transactions on Software Engineering, 24(6), 401-419.
\bibitem{31}
Srivastava, P. R. (2008). Test case prioritization. Journal of Theoretical \& Applied Information Technology, 4(3).
\bibitem{32}
Elbaum, S., Malishevsky, A. G., \& Rothermel, G. (2002). Test case prioritization: A family of empirical studies. IEEE transactions on software engineering, 28(2), 159-182.
\bibitem{33}
Elbaum, S., Rothermel, G., Kanduri, S., \& Malishevsky, A. G. (2004). Selecting a cost-effective test case prioritization technique. Software Quality Journal, 12, 185-210.
\bibitem{34}
Hao, D., Zhang, L., Zhang, L., Rothermel, G., \& Mei, H. (2014). A unified test case prioritization approach. ACM Transactions on Software Engineering and Methodology (TOSEM), 24(2), 1-31.
\bibitem{35}
Bhasin, H., Gupta, S., \& Kathuria, M. (2013). Regression testing using fuzzy logic. International journal of Computer Science and information technology, 4(2), 378-380.
\bibitem{36}
Huang, Y., Shu, T., \& Ding, Z. (2021). A learn-to-rank method for model-based regression test case prioritization. IEEE Access, 9, 16365-16382.
\bibitem{37}
Ufuktepe, E., \& Tuglular, T. (2023). Application of the law of minimum and dissimilarity analysis to Regression Test Case Prioritization. IEEE Access.
\bibitem{38}
Li, Z., Harman, M.,\& Hierons, R. M. (2007). Search algorithms for regression test case prioritization. IEEE Transactions on software engineering, 33(4), 225-237.
\bibitem{39}
Rothermel, G., Untch, R., Chu, C., \& Harrold, M. J. (2001). Test case prioritization. IEEE Transactions on Software Engineering, 27(10), 929-948.
\bibitem{40}
Malhotra, R., Kaur, A., \& Singh, Y. (2010). A regression test selection and prioritization technique. Journal of Information Processing Systems, 6(2), 235-252.
\bibitem{41}
Khatibsyarbini, M., Isa, M. A., Jawawi, D. N., \& Tumeng, R. (2018). Test case prioritization approaches in regression testing: A systematic literature review. Information and Software Technology, 93, 74-93
\bibitem{42}
Vescan, A., Chisalita-Cretu, C., Serban, C., \& Diosan, L. (2021, July). On the use of evolutionary algorithms for test case prioritization in regression testing considering requirements dependencies. In Proceedings of the 1st ACM International Workshop on AI and Software Testing/Analysis (pp. 1-8).
\bibitem{43}
Nayak, S., Kumar, C., \& Tripathi, S. (2022). Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate. Soft Computing, 26(15), 6953-6968.
\bibitem{44}
Biswas, S., Rathi, R., Dutta, A., Mitra, P., & Mall, R. (2022). A regression test case prioritization technique targeting ‘hard to detect’faults. International Journal of System Assurance Engineering and Management, 1-16.
Nayak, S., Kumar, C., Tripathi, S. et al. Regression test optimization and prioritization using Honey Bee optimization algorithm with fuzzy rule base. Soft Comput 25, 9925–9942 (2021). https://doi.org/10.1007/s00500-020-05428-z
\bibitem{b2}
N. Jazdi, P. Göhner and C. Malz, "Prioritization of Test Cases Using Software Agents and Fuzzy Logic," in 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, Montreal, Quebec Canada, 2012 pp. 483-486.
doi: 10.1109/ICST.2012.131
\bibitem{b3}
Badhera, U., and Biswas, D. (2013). Test case prioritization using fuzzy logic based on requirement prioritizing. International Journal on Computational Sciences \& Applications, 3(2).
\bibitem{b4}
H. K. N. Leung and L. White, "Insights into regression testing (software testing)," Proceedings. Conference on Software Maintenance - 1989, 1989, pp. 60-69, doi: 10.1109/ICSM.1989.65194.
\bibitem{b5}
G. Rothermel, R. H. Untch, Chengyun Chu and M. J. Harrold, "Prioritizing test cases for regression testing," in IEEE Transactions on Software Engineering, vol. 27, no. 10, pp. 929-948, Oct. 2001, doi: 10.1109/32.962562.
\bibitem{b6}
Valášková, K., Klieštik, T., \& Mišánková, M. (2014). The role of fuzzy logic in decision making process. In 2014 2nd international conference on management innovation and business innovation (Vol. 44, No. 1, pp. 143-148).
\bibitem{b7}
Hellmann, M. (2001). Fuzzy logic introduction. Université de Rennes, 1, 1-9.
\bibitem{b8}
Hettiarachchi, C., \& Do, H. (2019, July). A systematic requirements and risks-based test case prioritization using a fuzzy expert system. In 2019 IEEE 19th international conference on software quality, reliability and security (QRS) (pp. 374-385). IEEE.
\bibitem{b9}
Xu, Z., Gao, K., Khoshgoftaar, T. M., \& Seliya, N. (2014). System regression test planning with a fuzzy expert system. Information Sciences,259, 532-543
\bibitem{b10}
Yadav, D. K., \& Dutta, S. (2016, March). Test case prioritization technique based on early fault detection using fuzzy logic. In 2016 3rd international conference on computing for sustainable global development (INDIACom) (pp. 1033-1036). IEEE.
\bibitem{b11}
Tahvili, S., Afzal, W., Saadatmand, M., Bohlin, M., Sundmark, D., \& Larsson, S. (2016). Towards earlier fault detection by value-driven prioritization of test cases using fuzzy TOPSIS. In Information Technology: New Generations (pp. 745-759). Springer, Cham.
\bibitem{b12}
Silva, D., Rabelo, R., Campanha, M., Neto, P. S., Oliveira, P. A., \& Britto, R. (2016, July). A hybrid approach for test case prioritization and selection. In 2016 IEEE Congress on Evolutionary Computation (CEC) (pp. 4508-4515). IEEE.
\bibitem{b13}
Schwartz, A., \& Do, H. (2013, September). A fuzzy expert system for cost-effective regression testing strategies. In 2013 IEEE International Conference on Software Maintenance (pp. 1-10). IEEE.
\bibitem{b14}
Hettiarachchi, C., Do, H., \& Choi, B. (2016). Risk-based test case prioritization using a fuzzy expert system. Information and Software Technology, 69, 1-15.
\bibitem{b15}
Xu, Z., Gao, K., \& Khoshgoftaar, T. M. (2005, August). Application of fuzzy expert system in test case selection for system regression test. In IRI-2005 IEEE International Conference on Information Reuse and Integration, Conf, 2005. (pp. 120-125). IEEE.
\bibitem{b16}
Alakeel, A. M. (2014). Using fuzzy logic in test case prioritization for regression testing programs with assertions. The Scientific World Journal, 2014.
\bibitem{b17}
C. Hettiarachchi and H. Do, "A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System," 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), 2019, pp. 374-385, doi: 10.1109/QRS.2019.00054.
\bibitem{b18}
C. Hettiarachchi and H. Do, "A Systematic Requirements and Risks-Based Test Case Prioritization Using a Fuzzy Expert System," 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), 2019, pp. 374-385, doi: 10.1109/QRS.2019.00054.
\bibitem{19}
Minh, Vu Trieu, Katushin, Dmitri, Antonov, Maksim and Veinthal, Renno. "Regression Models and Fuzzy Logic Prediction of TBM Penetration Rate" Open Engineering, vol. 7, no. 1, 2017, pp. 60-68. https://doi.org/10.1515/eng-2017-0012.
\bibitem{20}
Leung, H. K., \& White, L. (1989, October). Insights into regression testing (software testing). In Proceedings. Conference on Software Maintenance-1989 (pp. 60-69). IEEE.
\bibitem{21}
Yoo, S., \& Harman, M. (2012). Regression testing minimization, selection and prioritization: a survey. Software testing, verification and reliability, 22(2), 67-120.
\bibitem{22}
Elbaum, S., Malishevsky, A. G., \& Rothermel, G. (2000, August). Prioritizing test cases for regression testing. In Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis (pp. 102-112).
\bibitem{23}
Binkley, D. (1997). Semantics guided regression test cost reduction. IEEE Transactions on Software Engineering, 23(8), 498-516.
\bibitem{24}
Myers, G. J., Sandler, C.,\& Badgett, T. (2011). The art of software testing. John Wiley \& Sons.
\bibitem{25}
Stuebing, H. G. (1980). A modern facility for software production and maintenance. 1980. 14.
\bibitem{26}
Rothermel, G., \& Harrold, M. J. (1997). A safe, efficient regression test selection technique. ACM Transactions on Software Engineering and Methodology (TOSEM), 6(2), 173-210.
\bibitem{27}
Pan, J. (1999). Software testing. Dependable Embedded Systems, 5, 2006.
\bibitem{28}
Rosenblum, D. S., \& Weyuker, E. J. (1997). Using coverage information to predict the cost-effectiveness of regression testing strategies. IEEE Transactions on Software Engineering, 23(3), 146-156.
\bibitem{29}
J. S. Collofello and J. J. Buck, “Software quality assurance for maintenance,” IEEE Software, pp. 46-51, Sept. 1987
\bibitem{30}
Rothermel, G., \& Harrold, M. J. (1998). Empirical studies of a safe regression test selection technique. IEEE Transactions on Software Engineering, 24(6), 401-419.
\bibitem{31}
Srivastava, P. R. (2008). Test case prioritization. Journal of Theoretical \& Applied Information Technology, 4(3).
\bibitem{32}
Elbaum, S., Malishevsky, A. G., \& Rothermel, G. (2002). Test case prioritization: A family of empirical studies. IEEE transactions on software engineering, 28(2), 159-182.
\bibitem{33}
Elbaum, S., Rothermel, G., Kanduri, S., \& Malishevsky, A. G. (2004). Selecting a cost-effective test case prioritization technique. Software Quality Journal, 12, 185-210.
\bibitem{34}
Hao, D., Zhang, L., Zhang, L., Rothermel, G., \& Mei, H. (2014). A unified test case prioritization approach. ACM Transactions on Software Engineering and Methodology (TOSEM), 24(2), 1-31.
\bibitem{35}
Bhasin, H., Gupta, S., \& Kathuria, M. (2013). Regression testing using fuzzy logic. International journal of Computer Science and information technology, 4(2), 378-380.
\bibitem{36}
Huang, Y., Shu, T., \& Ding, Z. (2021). A learn-to-rank method for model-based regression test case prioritization. IEEE Access, 9, 16365-16382.
\bibitem{37}
Ufuktepe, E., \& Tuglular, T. (2023). Application of the law of minimum and dissimilarity analysis to Regression Test Case Prioritization. IEEE Access.
\bibitem{38}
Li, Z., Harman, M.,\& Hierons, R. M. (2007). Search algorithms for regression test case prioritization. IEEE Transactions on software engineering, 33(4), 225-237.
\bibitem{39}
Rothermel, G., Untch, R., Chu, C., \& Harrold, M. J. (2001). Test case prioritization. IEEE Transactions on Software Engineering, 27(10), 929-948.
\bibitem{40}
Malhotra, R., Kaur, A., \& Singh, Y. (2010). A regression test selection and prioritization technique. Journal of Information Processing Systems, 6(2), 235-252.
\bibitem{41}
Khatibsyarbini, M., Isa, M. A., Jawawi, D. N., \& Tumeng, R. (2018). Test case prioritization approaches in regression testing: A systematic literature review. Information and Software Technology, 93, 74-93
\bibitem{42}
Vescan, A., Chisalita-Cretu, C., Serban, C., \& Diosan, L. (2021, July). On the use of evolutionary algorithms for test case prioritization in regression testing considering requirements dependencies. In Proceedings of the 1st ACM International Workshop on AI and Software Testing/Analysis (pp. 1-8).
\bibitem{43}
Nayak, S., Kumar, C., \& Tripathi, S. (2022). Analytic hierarchy process-based regression test case prioritization technique enhancing the fault detection rate. Soft Computing, 26(15), 6953-6968.
\bibitem{44}
Biswas, S., Rathi, R., Dutta, A., Mitra, P., & Mall, R. (2022). A regression test case prioritization technique targeting ‘hard to detect’faults. International Journal of System Assurance Engineering and Management, 1-16.
Authors
waqar, M. W. A., Dr. Muhammad Bilal Bashir, & Dr. Yaser Hafeez. (2024). The Multi-factor based Regression Test Case Prioritization using Fuzzy Logic. International Journal of Advanced Science and Computer Applications, 3(1). https://doi.org/10.47679/ijasca.v3i1.56
This work is licensed under a Creative Commons Attribution 4.0 International License.