Supporting material for the paper

Are Developers Aware of the Architectural Impact of Their Changes?


By Matheus Paixao, Jens Krinke, DongGyun Han, Chaiyong Ragkhitwetsagul and Mark Harman

1 - Abstract

Although considered one of the most important decisions in a software development lifecycle, empirical evidence on how developers perform and perceive architectural changes is still scarce. Given the large implications of architectural decisions, we do not know whether developers are aware of their changes’ impact on the software’s architecture, whether awareness leads to better changes, and whether automatically making developers aware would prevent degradation. Therefore, we use code review data of 4 open source systems to investigate the intent and awareness of developers when performing changes. We extracted 8,900 reviews for which the commits are available. 2,152 of the commits have changes in their computed architectural metrics, and 338 present significant changes to the architecture. We manually inspected all reviews for commits with significant changes and found that only in 38% of the time developers are discussing the impact of their changes on the architectural structure, suggesting a lack of awareness. Finally, we observed that developers tend to be more aware of the architectural impact of their changes when the architectural structure is improved, suggesting that developers should be automatically made aware when their changes degrade the architectural structure.

Keywords: Software Modularisation, Software Evolution, Multiobjective Search.


2 - Preprint

An preprint version of the paper can be downloaded here.

3 - Code Reviews Datasets

For each system studied in the paper we provide a CSV file. For each review we provide:

  • Review ID on Gerrit
  • URL for the webview of the review
  • ID of the commit before the review
  • ID of the commit after the review
  • Change in cohesion and coupling metrics for the system before and after the review
  • The manual classification regarding the intent of the review and the architectural awareness of developers
Download the CSV files for egit, linuxtools, couchbase-java-client and couchbase-jvm-core.

In addition, to support replication of the results for the reviews in which the webview is no longer available, we provide the description and comments of all reviews as separate files.


4 - Structural Architectures Dataset

For each review of each system, we extracted the structural architecture of the system before and after each review. This dataset can be downloaded here.

October 2017 (last update: October 2017)