2:00 - 2:30 |
Configurations Everywhere: Implications for Testing and Debugging in Practice
Many industrial systems are highly-configurable, complicating the testing and debugging process. While researchers have developed techniques to statically extract, quantify and manipulate the valid system configurations, we conjecture that many of these techniques will fail in practice. In this paper we analyze a highly-configurable industrial application and two open source applications in order to quantify the true challenges that configurability creates for software testing and debugging. We find that (1) all three applications are multi-lingual, hence static analyses need to cross programming language barriers to work, (2) there are multiple access points and methods to modify configurations, implying that practitioners need configuration traceability and should gather and merge metadata from more than one source and (3) the configuration state of an application on failure cannot be reliably determined by reading persistent data; a runtime memory dump or other heuristics must be used for accurate debugging. We conclude with a roadmap and lessons learned that will help practitioners better handle configurability now, and that may lead to new configuration-aware testing and debugging techniques in the future.
|
|
Dongpu Jin, Xiao Qu, Myra Cohen and Brian Robinson |
|
2:30 - 3:00 |
Software Feature Location in Practice: Debugging Aircraft Simulation Systems
In this paper, we report on a study that we have conducted at CAE, one of the largest civil aircraft simulation companies in the world, in which we have developed a feature location approach to help software engineers debug simulation scenarios. A simulation scenario consists of a set of software components, configured in a certain way. A simulation fails when it does not behave as intended. This is typically a sign of a configuration problem. To detect configuration errors, we propose FELODE (Feature Location for Debugging), an approach that uses a single trace combined with user queries. When applied to CAE systems, FELODE achieves in average a precision of 50% and a recall of up to 100%.
|
| Salman Hoseini, Abdelwahab Hamou-Lhadj, Patrick Desrosiers and Martin Tapp
|
3:00 - 3:30 |
Deriving Requirements Model from Textual Use Cases
Textual use cases are commonly used to represent software requirements at an initial stage. However in most of the cases, these documents are unstructured. A structured and formalized requirements model would greatly help in further analysis of these use cases. In this paper, we present an approach to derive structured requirements models from textual use case requirements in the form of process diagrams and ontology using methods based on computational linguistics. These proposed requirements models are capable of modeling both structural and behavioral entities present in a use case. We consider a corpus containing 123 actual requirements use cases created by Infosys Ltd. and translated them to process diagrams and ontology. For evaluating the performance of conversion we propose a few metrics and show that on average our linguistic engine mis-identified 2% of actions and missed out only 3% of the actions described in the input text. We have benchmarked precision and recall against Sinha et al.'s study on model extraction from use cases. Our method produced high accuracy in different categories such as Initiating Actor and Responding Actor, as well as, for most action types.
|
| Kiran Sawant, Suman Roy, Francois Plesse, Srivibha Sripathi and A. S. M. Sajeev
|
3:30 - 4:00 |
Tracking Requirements Evolution by Using Issue Tickets: A Case Study of a Document Management and Approval System
Requirements evolve throughout the software lifecycle. When requirements change, requirements engineers must determine what software artifacts could be affected. The history of and rationale for requirements evolution provides engineers some information about artifact dependencies for impact analysis. In this paper, we discuss a case study of requirements evolution for a large-scale system governed by Japanese laws and regulations. We track requirements evolution using issue tickets created in response to stakeholder requests. We provide rules to identify requirements evolution events (e.g. refine, decompose, and replace) from combinations of operations (e.g. add, change, and delete) specified in the issue tickets. We propose a Requirements Evolution Chart (REC) to visually represent requirements evolution as a series of events over time, and implement tool support to generate a REC from a series of issue tickets using our rules to identify requirements evolution events. We found that the REC supports impact analysis and compliance efforts.
|
| Shinobu Saito, Yukako Iimura, Kenji Takahashi, Aaron Massey and Annie Antón
|