Improving Student Software Projects session
June 6th, Friday, 10:30 am - 12:30 pm
Room: Hall 3
|10:30 - 11:00||State-based Monitoring and Goal-driven Project Steering: Field Study of the SEMAT Essence Framework
At Carnegie Mellon University in Silicon Valley, the graduate master program ends with a practicum project during which students serve as software engineering consultants for an industry client. In this context, students are challenged to demonstrate their ability to work on self-managing and self-organizing teams. This paper presents a field study of the Software Engineering Method and Theory (SEMAT) Essence framework. The objective is to evaluate the effectiveness of the Essence’s novel state-based monitoring and goal-driven steering approach provided by the Essence kernel alphas and their states. The researchers conducted the study on seven graduate master student teams applying the approach throughout their practicum projects. The research methodology involves weekly observation and recording of each team’s state progression and collecting students’ reflection on the application of the approach. The main result validates that the approach provides student teams with a holistic, lightweight, non-prescriptive and method-agnostic way to monitor progress and steer projects, as well as an effective structure for team reflection and risk management. The paper also validates that the Essence kernel provides an effective mechanism for monitoring and steering work common to most student software projects. This includes the work done during project initiation as well as the work done at the project or release level. Support for technical work should come from additional practices added on top of the kernel, or by extending or altering the kernel definition. The conclusion is that the approach enables students to learn to steer projects effectively by addressing the various dimensions of software engineering. Hence the approach could be leveraged in software engineering education.
|Cecile Peraire and Todd Sedano|
|Carnegie Mellon University, United States|
|11:00 - 11:30||Introduction of Continuous Delivery in Multi-Customer Project Courses
Continuous delivery is a set of practices and principles to release software faster and more frequently. While it helps to bridge the gap between developers and operations for software in production, it can also improve the communication between developers and customers in the development phase, i.e. before software is in production. It shortens the feedback cycle and developers ideally use it right from the beginning of a software development project. In this paper we describe the implementation of a customized continuous delivery workflow and its benefits in a multi-customer project course in summer 2013. Our workflow focuses on the ability to deliver software with only a few clicks to the customer in order to obtain feedback as early as possible. This helps developers to validate their understanding about requirements, which is especially helpful in agile projects where requirements might change often. We describe how we integrated this workflow and the role of the release manager into our project-based organization and how we introduced it using different teaching methods. Within three months 90 students worked in 10 different projects with real customers from industry and delivered 490 releases. After the project course we evaluated our approach in an online questionnaire and in personal interviews. Our findings and observations show that participating students understood and applied the concepts and are convinced about the benefits of continuous delivery.
|Stephan Krusche and Lukas Alperowitz|
|Technische Universitaet Muenchen, Germany|
|11:30 - 12:00||Process Mining Software Repositories from Student Projects in an Undergraduate Software Engineering Course
An undergraduate level Software Engineering courses generally consists of a team-based semester long project and emphasizes on both technical and managerial skills. Software Engineering is a practice-oriented and applied discipline and hence there is an emphasis on hands-on development, process, usage of tools in addition to theory and basic concepts. We present an approach for mining the process data (process mining) from software repositories archiving data generated as a result of constructing software by student teams in an educational setting. We present an application of mining three software repositories: team wiki (used during requirement engineering), version control system (development and maintenance) and issue tracking system (corrective and adaptive maintenance) in the context of an undergraduate Software Engineering course. We propose visualizations, metrics and algorithms to provide an insight into practices and procedures followed during various phases of a software development life-cycle. The proposed visualizations and metrics (learning analytics) provide a multi-faceted view to the instructor serving as a feedback tool on development process and quality by students. We mine the event logs produced by software repositories and derive insights such as degree of individual contributions in a team, quality of commit messages, intensity and consistency of commit activities, bug fixing process trend and quality, component and developer entropy, process compliance and verification. We present our empirical analysis on a software repository dataset consisting of 19 teams of 5 members each and discuss challenges, limitations and recommendations.
|Megha Mittal and Ashish Surekha|
|IIIT Delhi, India|
|12:00 - 12:30||Comparing Test Quality Measures for Assessing Student-Written Tests
Many educators now include software testing activities in programming assignments, so there is a growing demand for appropriate methods of assessing the quality of student-written software tests. While tests can be hand-graded, some educators also use objective performance metrics to assess software tests. The most common measures used at present are code coverage measures—tracking how much of the student’s code (in terms of statements, branches, or some combination) is exercised by the corresponding software tests. Code coverage has limitations, however, and sometimes it overestimates the true quality of the tests. Some researchers have suggested that mutation analysis may provide a better indication of test quality, while some educators have experimented with simply running every student’s test suite against every other student’s program—an “all-pairs” strategy that gives a bit more insight into the quality of the tests. However, it is still unknown which one of these measures is more accurate, in terms of most closely predicting the true bug revealing capability of a given test suite. This paper directly compares all three methods of measuring test quality in terms of how well they predict the observed bug revealing capabilities of student-written tests when run against a naturally occurring collection of student-produced defects. Experimental results show that all-pairs testing—running each student’s tests against every other student’s solution—is the most effective predictor of the underlying bug revealing capability of a test suite. Further, no strong correlation was found between bug revealing capability and either code coverage or mutation analysis scores.
|Stephen H. Edwards and Zalia Shams|
|Virginia Tech, USA|