How Do Practitioners Gain Confidence in Assurance Cases?
Abstract
Assurance Cases (ACs) are prepared to argue that the system's desired quality attributes (e.g., safety or security) are satisfied. While there is strong adoption of ACs, practitioners are often left asking an important question: are we confident that the claims made by the case are true? While many confidence assessment methods (CAMs) exist, little is known about the use of these methods in practice. This study aimed to develop understanding of current practice for AC confidence assessment: what methods are used and what barriers exist for their use? Structured interviews and an email questionnaire were used to gather data from 19 practitioners with experience contributing to real-world ACs. Open-coding was performed on transcripts. The most common CAMs were peer-review, dialectic reasoning ("defeaters"), and checklist comparison. Some practitioners also used models. Participants preferred qualitative methods and expressed concerns about quantitative CAMs. Barriers included additional work, inadequate guidance, subjectivity, and trustworthiness concerns. A gap exists between proposed methods and practitioner needs. Researchers should connect CAMs to established practices, use them for stakeholder communication, clarify application details, provide accessible guidance, and validate method trustworthiness.