Mutation Testing in Industry CI: A Value- Centric Approach - A case study about the intersection of software quality and developer experience.
Ladda ner
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Software engineering and technology (MPSOF), MSc
Publicerad
2024
Författare
van Heijningen, Stefan
Wiik, Theo
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Context: Mutation testing is a robust testing technique to assess the sufficiency of
test suites. The industry is still struggling to adopt mutation testing and maximize
its benefits despite recent research suggesting it is becoming more mature.
Objectives: This study aims to share insights and recommendations to:
1. Assist developers during the integration process of mutation testing tools.
2. Present mutation testing results to maximize their benefit and minimize the
cost of using them.
Methods: We perform a case study in an industry setting. We create an experience
report that reflects on the integration process of a mutation testing tool at the
partner company. We then focus on developer experience by using ten think-aloud
sessions and semi-structured interviews to explore what information developers perceive
as useful and how the information should be presented.
Results: A CI pipeline was developed to run mutation testing nightly and upload
results to a developed dashboard. Integrating mutation testing tools is still a
challenging process. When interacting with results, developers valued interactivity,
getting an overview, and wanted to associate mutation testing results with other
contextual information of the codebase, such as code coverage and complexity. A
set of recommendations was created to facilitate integrating and using mutation
testing in industry settings.
Conclusion: As-is, mutation testing is a standalone tool that should be easier to
integrate and interact with other aspects of the codebase to become more adopted
and successful.
Beskrivning
Ämne/nyckelord
software engineering , quality assurance , test sufficiency criteria , mutation testing , continuous integration , developer experience , case study , thesis