Correspondence with John Hattie

Shortly after I started this blog, after I’d done a few posts, I wrote to John Hattie at the University of Melbourne pointing out some of my concerns. One of the things I pointed out was that he claimed the ‘Effect Size’ had units of standard deviation when it can be shown mathematically that it actually has no units (and it’s fine for it to have no units as long as you realise that).

In fairness to him, he wrote back quite a long letter taking each of my points in turn. When it came to my ‘the Effect size has no units’ point he said –

“It is not correct to claim that the Effect Size has no units, it does, from -infinity to +infinity but more normally between -3 and 3”

Now, up to this point, I couldn’t quite believe all that I’d found out about the Effect Size. I would say to myself ‘the Effect Size is wrong and you’re the only one who noticed. Yeah, right!’ I was constantly searching my mind thinking ‘You’ve missed something, what have you missed?’

When I read this statement from him my mouth just dropped open.

Not only does John Hattie not know what units the ‘Effect Size’ is measured in, he doesn’t even understand what units are. What he’s quoted are not the units but the typical magnitude of the ‘Effect Size’ as found in Education research. This is an error which throws doubt on John Hattie’s basic mathematical competence.

To give you an example of how big a gaffe this is, imagine you asked a Physics Teacher what the units of speed for a car are. ‘The units of speed of a car are between 0 and 70’ they answer. No, the units of speed are miles per hour (or kilometres per hour or metres per second). That is a significant mistake and you wouldn’t have a great deal of faith in the ability of the person who said it afterwards.

Is his letter, John Hattie also admonished me, saying that he had read my blog and felt I made too many remarks about him for him to leave a comment. He said –

“. . . in Academia the criticism is of ideas not people”

Which is fine except most people have no way to gauge whether or not a Mathematical argument is correct or not so they might need to rely on other questions to guide them, questions like –

– Do other people in relevant fields use this?

– What is the competence of the person using this?

Now these questions won’t give us the definite answer to the use of the ‘Effect Size’ but what they may do is indicate an area of concern that may be worthy of further investigation.

The answer to the first question is, Mathematicians and Scientists have never heard of the Effect Size, in fact only Psychologists and Education Researchers use it.

If you’re going to use Maths that Mathematicians don’t you’re either a genius, or you don’t know what you’re doing.

John Hattie is an Arts Graduate, who doesn’t understand what units are, nor the importance of getting them correct. I’ll leave you to ponder for yourself which he is.





7 thoughts on “Correspondence with John Hattie

  1. I remember reading a blog which showed that the effe size wa suspect anyway because the meta data that was synthesised was apples and pears (garbage in/garbage out)… Was that one of yours?

  2. I think that your comments about John Hattie are far too harsh. The positive impact that he has had on teaching and learning is incredible and that his work is excellent. Why try and bring someone down and find errors that indeed are not there? Your comments are not true nor valid.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s