# Correspondence with John Hattie

Shortly after I started this blog, after I’d done a few posts, I wrote to John Hattie at the University of Melbourne pointing out some of my concerns. One of the things I pointed out was that he claimed the ‘Effect Size’ had units of standard deviation when it can be shown mathematically that it actually has no units (and it’s fine for it to have no units as long as you realise that).

In fairness to him, he wrote back quite a long letter taking each of my points in turn. When it came to my ‘the Effect size has no units’ point he said –

“It is not correct to claim that the Effect Size has no units, it does, from -infinity to +infinity but more normally between -3 and 3”

Now, up to this point, I couldn’t quite believe all that I’d found out about the Effect Size. I would say to myself ‘the Effect Size is wrong and you’re the only one who noticed. Yeah, right!’ I was constantly searching my mind thinking ‘You’ve missed something, what have you missed?’

When I read this statement from him my mouth just dropped open.

Not only does John Hattie not know what units the ‘Effect Size’ is measured in, he doesn’t even understand what units are. What he’s quoted are not the units but the typical magnitude of the ‘Effect Size’ as found in Education research. This is an error which throws doubt on John Hattie’s basic mathematical competence.

To give you an example of how big a gaffe this is, imagine you asked a Physics Teacher what the units of speed for a car are. ‘The units of speed of a car are between 0 and 70’ they answer. No, the units of speed are miles per hour (or kilometres per hour or metres per second). That is a significant mistake and you wouldn’t have a great deal of faith in the ability of the person who said it afterwards.

“. . . in Academia the criticism is of ideas not people”

Which is fine except most people have no way to gauge whether or not a Mathematical argument is correct or not so they might need to rely on other questions to guide them, questions like –

– Do other people in relevant fields use this?

– What is the competence of the person using this?

Now these questions won’t give us the definite answer to the use of the ‘Effect Size’ but what they may do is indicate an area of concern that may be worthy of further investigation.

The answer to the first question is, Mathematicians and Scientists have never heard of the Effect Size, in fact only Psychologists and Education Researchers use it.

If you’re going to use Maths that Mathematicians don’t you’re either a genius, or you don’t know what you’re doing.

John Hattie is an Arts Graduate, who doesn’t understand what units are, nor the importance of getting them correct. I’ll leave you to ponder for yourself which he is.

## 7 thoughts on “Correspondence with John Hattie”

1. I remember reading a blog which showed that the effe size wa suspect anyway because the meta data that was synthesised was apples and pears (garbage in/garbage out)… Was that one of yours?

• nobodytoo

*i*

• That blog certainly appears to make the point. I think I’ve also seen something more recently. If that’s so, there may be a consensus developing. Stats….can’t live with them, can’t live without them…..

2. Passionate Teacher

I think that your comments about John Hattie are far too harsh. The positive impact that he has had on teaching and learning is incredible and that his work is excellent. Why try and bring someone down and find errors that indeed are not there? Your comments are not true nor valid.

• I don’t know on what you’re basing the idea that his work is excellent, he uses Statistics unrecognised by Mathematicians.

If his conclusions are invalid, then the only thing that Visible Learning has had a positive impact on, is his bank balance, as he charges £450 per delegate for a days conference – http://www.visiblelearningplus.co.uk/visible-learning-conference-2014/