Evaluating the Relevance of Testing Frameworks

Testing: When assessing the validity of testing frameworks, software development teams and hiring managers may wonder how common or relevant a particular framework has become across the industry. Researchers from the University of Genova, Italy used LinkUp's historical job listing data, collected directly from company websites since 2007 and filtered by relevant occupational (O*NET) codes, to examine the frequency of mentions of testing frameworks AssertJ and Hamcrest within job listings over a three-year period.

They found that both frameworks shared nearly the same popularity, aiding in their research surrounding how those frameworks perform across various sample groups.

Along with other identifiers such as geographic location and company, LinkUp stores the text descriptions of every job listing collected, containing insight into desired occupational tool proficiencies for particular industries or occupations.

The data made available by LinkUp concern about 110 million of ads published since 2007up to 2018, mostly written in English, but located in 220 different countries. To compare the fame of AssertJ and Hamcrest, we wanted to analyze the text, searching for such keywords. To make such searches feasible, given the flood of data, we first had to prune the irrelevant ads and keep only the interesting ones. Thus, we made a selection based on three criteria:

– Job typology: we kept only the ads for positions in the field of software development, using the O*NET field;

– Language: we restricted ourselves to the ads with an English description;

– Time frame: we restricted our analysis to the years 2015–2018 for two reasons. On the one hand, we are more interested in recent history, having a greater impact on the current trends. On the other hand, technology is changing fast, so that any research question about tools needs to target a small interval.

Leotta, M., Cerioli, M., Olianas, D. et al. Two experiments for evaluating the impact of Hamcrest and AssertJ on assertion development. Software Qual J 28, 1113–1145 (2020). https://doi.org/10.1007/s11219