Correspondence with John Hattie

Shortly after I started this blog, after I’d done a few posts, I wrote to John Hattie at the University of Melbourne pointing out some of my concerns. One of the things I pointed out was that he claimed the ‘Effect Size’ had units of standard deviation when it can be shown mathematically that it actually has no units (and it’s fine for it to have no units as long as you realise that).

In fairness to him, he wrote back quite a long letter taking each of my points in turn. When it came to my ‘the Effect size has no units’ point he said –

“It is not correct to claim that the Effect Size has no units, it does, from -infinity to +infinity but more normally between -3 and 3”

Now, up to this point, I couldn’t quite believe all that I’d found out about the Effect Size. I would say to myself ‘the Effect Size is wrong and you’re the only one who noticed. Yeah, right!’ I was constantly searching my mind thinking ‘You’ve missed something, what have you missed?’

When I read this statement from him my mouth just dropped open.

Not only does John Hattie not know what units the ‘Effect Size’ is measured in, he doesn’t even understand what units are. What he’s quoted are not the units but the typical magnitude of the ‘Effect Size’ as found in Education research. This is an error which throws doubt on John Hattie’s basic mathematical competence.

To give you an example of how big a gaffe this is, imagine you asked a Physics Teacher what the units of speed for a car are. ‘The units of speed of a car are between 0 and 70’ they answer. No, the units of speed are miles per hour (or kilometres per hour or metres per second). That is a significant mistake and you wouldn’t have a great deal of faith in the ability of the person who said it afterwards.

Is his letter, John Hattie also admonished me, saying that he had read my blog and felt I made too many remarks about him for him to leave a comment. He said –

“. . . in Academia the criticism is of ideas not people”

Which is fine except most people have no way to gauge whether or not a Mathematical argument is correct or not so they might need to rely on other questions to guide them, questions like –

– Do other people in relevant fields use this?

– What is the competence of the person using this?

Now these questions won’t give us the definite answer to the use of the ‘Effect Size’ but what they may do is indicate an area of concern that may be worthy of further investigation.

The answer to the first question is, Mathematicians and Scientists have never heard of the Effect Size, in fact only Psychologists and Education Researchers use it.

If you’re going to use Maths that Mathematicians don’t you’re either a genius, or you don’t know what you’re doing.

John Hattie is an Arts Graduate, who doesn’t understand what units are, nor the importance of getting them correct. I’ll leave you to ponder for yourself which he is.