In 2007, four American researchers looked at the data from seven national tests in Reading and six national tests in Maths across an age range from six to seventeen. They were looking for patterns in the Effect Sizes.
As we can see there is a clear downward trend and the hinge figure of 0.40 is never achieved again after the age of 10.
Again there is a downward trend and the figure of 0.40 is never achieved after the age of 11. The authors of the paper also found the same trend when they studied national test results for Social Studies and Science.
This means that Hattie’s hinge figure of 0.40 is spectacularly misleading. Educational research done in Primary schools will usually do better than 0.40, whereas Teachers teaching in Secondary Schools will find that their Effect Size is usually below 0.40 and gets worse the older the children are, no matter how effectively they are teaching.
To get any kind of fair comparison for educational studies, we need to know the age of the children studied, as well as their results. We can then compare fairly with the typical Effect Size for their age range, instead of a headline figure of 0.40.
One possible reason that we are seeing this pattern is that the ‘Effect Size’ is really (inversely) measuring how spread out the pupils are, not how well they are progressing.
In Year 1, there’s not as big a difference between the top and the bottom child, because even the quickest child hasn’t learned that much. This means the standard deviation (how spread out the pupils are) is small. When you divide by something small you get a big number.
In Year 11, the opposite is true, there is a large difference between the top pupils and the bottom pupils. A big spread means a large standard deviation and dividing by a large number gives you a small number.
Hat Tip to @dylanwiliam