Year
2024
Season
Fall
Paper Type
Master's Thesis
College
College of Computing, Engineering & Construction
Degree Name
Master of Science in Computer and Information Sciences (MS)
Department
Computing
NACO controlled Corporate Body
University of North Florida. School of Computing
First Advisor
Dr. Upulee Kanewala
Second Advisor
Dr. Karthikeyan Umapathy
Third Advisor
Dr. Anirban Ghosh
College Dean
Dr. Chip Klostermeyer
Abstract
Scientific software can be broadly defined as software written to aid in conducting scientific research activities. As this software is often written by the scientists themselves, who often learned programming to build that software, some important aspects of software development may get less attention than they would have gotten on enterprise software developed by programmers. One area of development that often gets overlooked in this regard is testing. Thus, in this study, we seek to better understand the testing processes and propose methods to improve the effectiveness of those tests. We begin with examining the software's Continuous Integration/Delivery practices. This is primarily done through GitHub Actions, which can be used to automate testing. Based on this, we examined the GitHub Actions of several scientific software practices of various sizes and scientific domains. Results of our study show that, even among the variety in the software, many aspects of their GitHub Actions Workflows are shared. We then conducted a study to evaluate the fault detection effectiveness of the tests in scientific software. Our results show that there do exist common deficiencies in testing among this scientific software, such as low if-statement branch coverage or low edge case sensitivity and lack of thorough testing of mathematical calculations. These shortcomings can potentially be improved through known testing methods. Finally, Based on our findings, we propose and explore a testing method called metamorphic testing, which makes use of necessary properties of a function that identify relationships between a that function's input and its output. We added metamorphic tests to the software's test suites, and found as much as a 10\% improvement on mutation testing results among some types of mutations. The results imply that the improvements made by metamorphic testing can be even further improved when written by someone with domain knowledge.
Suggested Citation
Roker, Kris, "Understanding and augmenting scientific software testing practices" (2024). UNF Graduate Theses and Dissertations. 1308.
https://digitalcommons.unf.edu/etd/1308
