Team Quality Metrics 🤦‍♀️

"How can we measure the quality of development team deliverables?"

A question born with good intentions but fraught with dangerous and meaningless measurements.


The backstory

I have recently taken on a new role and have been having discussions with various people surrounding the development team. One of these discussions centered around quality metrics.

Now before I carry on I would like to offer my simple opinion of quality metrics:

Measurements, if taken, should be taken at the team level. The entire team is responsible for the quality of the products it delivers and a failure caused by a single member is a failure shared by the team.

As part of this conversation, I was pointed at a blog post by Kostis Kapelonis, in which we found a measurement which caused some debate:

"PDWT (Percent of Developers who Write Tests) is probably the most important metric of all. There is no point in talking about software testing anti-patterns if you have zero tests in the first place. All developers in the team should write tests. A new feature should be declared done only when it is accompanied by one or more tests."

We could automate this, but would it add value?


The implementation

So now let's figure out, the crudest mechanism of extracting this measurement:
- We use git - We separate tests into files with naming/path convention

So using that I wrote a VERY rough bash script which can be run in any git repo test-contributors.sh

#!/usr/bin/env bash
function getContributors() {  
    git ls-tree -r --name-only master $1 | while read file ; do
      git log --follow --pretty=format:%an $file | sort | uniq
    done | sort | uniq
  return 0
}

allContributors=()  
while read -r user; do  
  allContributors+=("$user")
done <<< "$(getContributors ./)"

testContributorsRaw=$(getContributors) <<< $(find . -name '*test*' -o -name '*feature*' -o -name '*spec*')  
testContributors=()  
while read -r tester; do  
  testContributors+=("$tester")
done <<< "$testContributorsRaw"

htmlAllContributors=""  
for contributor in "${allContributors[@]}"; do  
  htmlAllContributors+="<p>${contributor}</p>"
done

htmlTestContributors=""  
for testContributor in "${testContributors[@]}"; do  
  htmlTestContributors+="<p>${testContributor}</p>"
done

numberOfContributors=${#allContributors[@]}  
numberOfTesters=${#testContributors[@]}  
scale=2  
percentageTesters=$(($numberOfTesters*100/$numberOfContributors | bc))

IFS=$'\n'  
cat << _EOF_  
<!DOCTYPE html>  
<html lang="en">  
<head>  
  <title>Bootstrap Example</title>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
</head>  
<body>

<div class="container">  
  <div class="jumbotron">
  <h1>👩‍⚕️ ${percentageTesters}% Contributors writing tests</h1>
  </div>
  <div class="row">
    <div class="col-sm-4">
      <h1>ALL Contributors</h1>
      ${htmlAllContributors}
    </div>
    <div class="col-sm-4">
      <h1>TEST Contributors</h1>
      ${htmlTestContributors}
  <div>
</div>

</body>  
</html>  
_EOF_  



This will yield a snippet of html which looks like this:



Conclusions

This feels like finger pointing and I don't have evidence this will add any value but it might be a good indicator if bigger problems within a project team.