A closer look at Hattie’s top two Effect Sizes

Hattie’s top two Effect Sizes in Visible Learning are

Self-reported grades – 1.44

Piagetian programmes – 1.28

in fact these are the only two that are above 1.0

Kristen DiCerbo has had a closer look at Self-reported grades. Hat-tip to @Mrsdaedalus.

Piagetian stages were proposed by Jean Piaget. Basically, as children develop they pass through various stages of development, firstly with motor skills as babies, then thinking skills as young children.

Piagetian programmes cites only one meta-analysis, Jordan and Brownlee (1981). Unfortunately, I can’t find the full paper, only an abstract. The abstract does show two things though.

Firstly, the original studies weren’t calculated as Effect Sizes, they were calculated as correlations. Hattie has again converted correlation coefficients into Effect Sizes. The study is basically saying that kids who develop faster when they are babies (because they are more intelligent) do better at tests a few years later (because they are more intelligent). Hardly earth-shattering stuff. And the same as the Self-reported grades, this is not an intervention, there’s nothing you can do about it, it’s just a correlation.

Secondly, the students in the study had an average age of just 7 years old. Hattie has used this to extrapolate to all students aged 5 to 18. We teach pupils not to extrapolate outside the data range at GCSE.

Remember that both of these Effect Sizes were used when Hattie calculated his 0.40 average, so, if they are wrong, then so is the 0.40 hinge point. And we could have included any number of correlations in here and changed them to Effect Sizes. It just shows that his 0.40 ‘hinge point’ is completely arbitrary.

Also, it may be worth pointing out at this point the differences between the correlation coefficient and the Effect Size.

The correlation coefficient – Proposed in 1880 by Karl Pearson who is considered by many to be the Father of Mathematical Statistics and founded the first University Statistics department. Explanation of the reasoning behind it and derivation using Algebra in every Statistics textbook. Learnt by Mathematicians either at A Level or first year of University.

The Effect Size – Proposed in 1985 by Gene Glass, an Educational Psychologist. No explanation or derivation ever given even today. Appears in no Maths textbooks. No Mathematician has ever heard of it.


One thought on “A closer look at Hattie’s top two Effect Sizes

  1. I had a look at both Piagetian Stages and Self-Reported Grades a while back. Both posts are linked from https://dodiscimus.wordpress.com/2014/11/09/index-to-series-on-hatties-visible-learning/ . I have been a bit lazy about updating the original post on Piagetian Stages but have now read the Jordan & Brownlee paper (which one of them very kindly posted to me, as it was an old conference paper). It’s more-or-less what I was expecting, and you have surmised, identifying a correlation between a test to identify Piagetian stage of cognitive development with tests of maths and reading performance. My series digging into Hattie isn’t abandoned but has rather been stalled by other events; I need to pull some of the work I’ve done together for an internal university thing, and that might result in another effect or two getting posted on my blog, but then I’m going to get pulled in other directions. I might then finally get back to it but in a sense the whole Hattie wave is largely spent, and whilst I think it would be a good thing if some people delved into his work to sort the wheat from the chaff I don’t know that it’s going to be me. I know your view would be that it’s all chaff but with several strong caveats I think there is a valuable resource there that has done the usual education thing of being treated as the Holy Grail for a few years and then cast aside as worthless junk, instead of being cautiously welcomed, picked over, and built on, which would have been a better outcome.
    Best wishes

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s