[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [jfw] Re: Private methods should not be unit-tested



On 19 March 2014 18:51, Herman Peeren <> wrote:
> list for this year's GSOC it said: "The goal of this project is to improve
> the code coverage". I am putting some question marks to blindly follow the
> measure of "code coverage" as it is just a quantity that doesn't say much
> about the quality of tests.

Code quality is your job as a mentor. You can obviously use indicators
like PMD and the like, but there is no substitute for an experienced
eye-ball.

> Recently one of the students already did some work on tests and they were
> merged without any critic.I commented on two afterwards, for there were some
> things wrong with them (some of which has now been corrected). It was
> another signal of the maintainers of the Framework only looking at "code
> coverage" not at quality of the tests.

That's an invalid assumption but I can't respond any better since I do
not know the specific cases.

> One of the first tests that was
> committed was for a private method; if you do them, which we are discussing
> here,  they could better be done later, after the API is completely
> unit-tested (I completely agree with you about that, Amy).

As I've said, it depends on the use-case. I think it's just as bad to
blindly follow someone's advice to "delete all private tests".

> At the same time there were a bunch of tests freshly committed, also for
> private methods, in the FoF repository (for the new ORM-possibilities).

That is nothing to do with us.

> When
> putting a question mark there, the main reply was that the Joomla project
> only looks at the measure of code coverage for inclusion of FoF. Probably
> they would occasionally even provide useless tests, as long as it improves
> code coverage.

Given that FoF, last time I looked, is predominantly *untested*, any
increase in code coverage is a good thing. I thin that's a reasonable
argument when your code coverage is near zero.

> would be counter-productive. Further: if a private method cannot be tested
> on the interface, that might be an indication that something is
> architecturally wrong.

It might be, it might not. It depends how you use private methods. I
use them to prevent the developer from tampering with code I don't wan
them to have access to.

> Those are all arguments to improve code (and testing)
> quality. And that is my main point: please don't only look at code coverage
> as a measure, but look at the quality of tests in the first place.

You have to look at both.

> The
> figure of 80% that Andrew mentions for instance is based on what? Why not
> 90% or 70%. I'd rather have 100% of the API tested, but with good tests.

80% is completely arbitrary. You can choose a different number when
accepting pull requests against your personal repositories :) I
personally feel 100% is too much to expect from the average
contributor. I also think that 80% is a good indication that the code
is behaving the way it was design to and the act of refining the tests
over time is no different to refining the code itself. As we find a
bug we fix the code and the tests and over time code and test quality
improves. This is what we have been doing for many years and I believe
it's a balanced approach.

> For
> if a test is not good, then it could better be deleted.

If a test is no good it needs to be fixed.

> It is not easy to
> answer the question what makes a test good or bad (or how good or bad). I
> want to try to get some work done this summer for GSOC not only on "code
> coverage" but also on getting a list of indications of test quality.

I will be interested to see what you come up with. But please lets
concentrate on the tests we can write or improve rather than deleting
tests. I think this conversation would have been a lot shorter if
you'd simply asked this simple question in the first place "How should
I guide my GSOC students to improve our tests?".

Regards,
Andrew Eddie

-- 
Framework source code: https://github.com/joomla/joomla-framework
Visit http://developer.joomla.org for more information about developing with Joomla!
--- 
You received this message because you are subscribed to the Google Groups "Joomla! Framework Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-framework+unsubscribe AT googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-framework.