One of the reasons why I started this discussion was because a project to get more code coverage done will be in the GSOC (Google Summer of Code) this year. Until now 2 students have made a proposal to do that (application closes this Friday). At least one project will be done, but maybe more: would be nice if we get all
tests done that we want to have, and that seems to be in reach now. Which is great! I'm not at all against tests or something like that, I was just asking myself some questions about test quality (and found some answers a.o. in Sandi's presentation). In the ideas list for this year's GSOC it said: "The goal of this project is to improve the code coverage". I am putting some question marks to blindly follow the measure of "code coverage" as it is just a quantity that doesn't say much about the quality of tests.
Recently one of the students already did some work on tests and they were merged without any critic.I commented on two afterwards, for there were some things wrong with them (some of which has now been corrected). It was another signal of the maintainers of the Framework only looking at "code coverage" not at quality of the tests. One of the first tests that was committed was for a private method; if you do them, which we are discussing here, they could better be done later, after the API is completely unit-tested (I completely agree with you about that, Amy).
At the same time there were a bunch of tests freshly committed, also for private methods, in the FoF repository (for the new ORM-possibilities). When putting a question mark there, the main reply was that the Joomla project only looks at the measure of code coverage for inclusion of FoF. Probably they would occasionally even provide useless tests, as long as it improves code coverage.
I also don't think that code quality would "automatically" improve when tests of private methods are deleted. I deliberately put that a bit back-and-white on top of my posting (although I did not use the word "automatically" there), but it was given nuance in the rest of the posting. The disadvantage of testing an implementation instead of an interface can be, that it might slow down the refactoring to another implementation. That would be counter-productive. Further: if a private method cannot be tested on the interface, that might be an indication that something is architecturally wrong. Those are all arguments to improve code (and testing) quality. And that is my main point: please don't only look at code coverage as a measure, but look at the quality of tests in the first place. The figure of 80% that Andrew mentions for instance is based on what? Why not 90% or 70%. I'd rather have 100% of the API tested, but with good tests. For if a test is not good, then it could better be deleted. It is not easy to answer the question what makes a test good or bad (or how
good or bad). I want to try to get some work done this summer for GSOC not only on "code coverage" but also on getting a list of indications of test quality.
On Wednesday, 19 March 2014 03:49:12 UTC+1, Amy Stephen wrote:
On Friday, March 14, 2014 4:38:20 PM UTC-5, Herman Peeren wrote:
Private methods should not be unit-tested. Code quality would improve if we delete those tests.
Herman - I test public methods only (not protected or private) because my API is based on public methods. So, being able to start from scratch, I used this theory. (Though I disagree that code quality automatically improves deleting unnecessary tests.)
In this case, if I were motivated to follow an approach of only testing the API for Joomla, I would first create the needed PR's to cover the public and protected methods.
Then, with full coverage in place, I would follow with a PR that removed private tests. That's when this discussion makes sense, not before.