Contextual understanding and improvement of metamorphic testing in scientific software development
Document Type
Conference Proceeding
Publication Date
10-11-2021
Abstract
Background: Metamorphic testing emerges as a simple and effective approach for testing scientific software; yet, its adoption in actual scientific software projects is less studied. Aims: In order for the practitioners to better adopt metamorphic testing in their projects, we set out to first gain a deep understanding about the current qualify assurance workflow, testing practices, and tools. Method: We propose to integrate various empirical sources, including artifact analysis, stakeholder interviews, and gap analysis from the literature. Results: Applying our approach to the Open Water Analytics Stormwater Management Model project helped to identify four new needs requiring continued and more research: (1) systematic and explicit formulation of metamorphic relations, (2) metamorphic testing examples specific to the scientific software, (3) correlating metamorphic testing with regression testing, and (4) integrating metamorphic testing with build tools like CMake and continuous integration tools like GitHub Actions. Conclusions: Integrating different empirical sources is promising for establishing a contextual understanding of software engineering practices, and for action research, such as workflow refinements and tool interventions, to be carried out in a principled manner.
Publication Title
International Symposium on Empirical Software Engineering and Measurement
Digital Object Identifier (DOI)
10.1145/3475716.3484188
ISSN
19493770
E-ISSN
19493789
ISBN
9781450386654
Citation Information
Peng Z, Kanewala U, Niu N. Contextual understanding and improvement of metamorphic testing in scientific software development. DOI: 10.1145/3475716.3484188