CI and NCoverage

On our current project the continuous integration process demands that the code base has a minimal test coverage of 85%. If this condition is not met – guess what – the build fails. Initially we’ve started with a limit of 95%. We all knew that this was not quiet realistic but we thought there could be nothing wrong by starting with a noble attitude. As expected the effective coverage of the code was always slowly dropping to the configured limit. To the current point I am still satisfied with the result but there were some issues coming up during the daily stand up meetings:

Who’s in charge fixing the build if the limit is not met and the build fails?

From my point of view this is a no brainer. The person who actually checks in is responsible. But one member of my team had a different view. After some days off the project he joined us again, made a new class, checked it in and the build failed. From his point of view the coverage before his check in was already not sufficient because there should always be enough reserve for an empty class with a constructor to be checked in without the build failing. So he suggested we should all take some time to collectively increase the overall coverage.
I replied that if you are consequently do test first there should not even be such a class without an according test. Unfortunately until this point not every member of our team has come to the point where he is consequently doing test first. Also we did usually no coverage analysis on our local machine before checking in. This means, if the person which checks the code in is not a test-hardliner the build might. It means further that if you are close to the breaking limit of the test coverage your build will fail proportional to the number of members in your team which do not have enough experience in testing. This is really not a good option because a failing build means not only subsequent check ins will also fail if the build has not been fixed in the meantime, it means also more turbulence in the team.

For me this means that even if I manage to convince the team that if the build fails it is the one checking in who's in charge, I have to live with the fact that there’s some flow getting lost if we are near the limit.

Do we set the limit narrow enough to allow code be within the constraint which the team agrees does not have to be tested? (Like declarative code which when tested would only duplicate the declaration) Or do we tag the code with the according attributes to ignore it in the analysis and therefore are able to set the limit on a higher percentage?

From my point of view I do not like my code to be cluttered with ignore attributes just to be able to meet a self determined limit in our build process. But there were other opinions. One team member suggested that if we all agree that this code does not need to be tested why should we not tag it with such an attribute. I thinks this is a legitimate argument. Although I would not like the additional typing if we could avoid it simply by lowering the coverage limit accordingly. Eventually this is why we are configuring such a limit: because there is code which we all agree cannot be efficiently or meaningful be tested.

Bottom line we are still gathering experience how to properly handle the additional metrics of a coverage analysis in our development process.