Abstract
Testing is a significant part of the software development lifecycle, and generating an optimal number of test data is crucial to accelerate the overall software testing process. However, thorough and exhaustive testing is not always viable because of the combinatorial optimisation problem. In addition, limited resources, costs, and timelines impede the software testing process.
Combinatorial testing is a testing approach that facilitates the testing process by reducing the amount of test data to a feasible and manageable size. The theoretical basis of combinatorial testing affirms that a reduction in test data is possible by reducing interaction between parameters. Based on this principle, several combinatorial testing strategies have been developed over recent decades. However, most of these are not suitable in real-world situations due to their inability to support data constraints, mixed interactions of input data, and the generation of sequence arrays. A strategy is, therefore, required to meet the above-mentioned requirements.
This thesis proposes a combinatorial testing tool, known as SICT (Swarm Intelligence Combinatorial Testing), that is effective and efficient in solving combinatorial optimisation problems related to the generation of test data. SICT consists of five strategies that solve different components of the test data generation problems: SITG (Swarm Intelligence Test Generator) and mSITG (modified SITG) generate optimal number of test data; SISEQ (Swarm Intelligence Sequence Generator) and mSISEQ (modified SISEQ) generate an optimal number of test sequences; and finally, SITGU (SITG Utility) supports data constraints, mixed interaction of input data, and non-uniform parameters. These strategies utilise a swarm intelligence based heuristic search, which is useful in terms of producing an optimal result.
The performances of these strategies are compared with popular test data generation strategies, and discussion of the strategies aims to determine their applicability and effectiveness. The SICT exhibited up to 20% improvement in time complexity for test data generation, and up to 10% outperformance in space complexity for sequence generation. In addition, the constraints-provisioning feature of SITGU improves the overall usability. Finally, the performance evaluation results show that the SICT are more acceptable and useable in generating an optimal amount of test data and test sequence.
Combinatorial testing is a testing approach that facilitates the testing process by reducing the amount of test data to a feasible and manageable size. The theoretical basis of combinatorial testing affirms that a reduction in test data is possible by reducing interaction between parameters. Based on this principle, several combinatorial testing strategies have been developed over recent decades. However, most of these are not suitable in real-world situations due to their inability to support data constraints, mixed interactions of input data, and the generation of sequence arrays. A strategy is, therefore, required to meet the above-mentioned requirements.
This thesis proposes a combinatorial testing tool, known as SICT (Swarm Intelligence Combinatorial Testing), that is effective and efficient in solving combinatorial optimisation problems related to the generation of test data. SICT consists of five strategies that solve different components of the test data generation problems: SITG (Swarm Intelligence Test Generator) and mSITG (modified SITG) generate optimal number of test data; SISEQ (Swarm Intelligence Sequence Generator) and mSISEQ (modified SISEQ) generate an optimal number of test sequences; and finally, SITGU (SITG Utility) supports data constraints, mixed interaction of input data, and non-uniform parameters. These strategies utilise a swarm intelligence based heuristic search, which is useful in terms of producing an optimal result.
The performances of these strategies are compared with popular test data generation strategies, and discussion of the strategies aims to determine their applicability and effectiveness. The SICT exhibited up to 20% improvement in time complexity for test data generation, and up to 10% outperformance in space complexity for sequence generation. In addition, the constraints-provisioning feature of SITGU improves the overall usability. Finally, the performance evaluation results show that the SICT are more acceptable and useable in generating an optimal amount of test data and test sequence.
| Original language | English |
|---|---|
| Qualification | Doctor of Philosophy |
| Awarding Institution |
|
| Supervisors/Advisors |
|
| Award date | 01 Aug 2017 |
| Publication status | Published - 2017 |