Why the EEF report on Philosophy for children has no tests for statistical significance

I noticed a few days ago that people were expressing surprise on Twitter that the EEF report on Philosophy for children had no tests for statistical significance.

The problems with the statistics have been written about in greater detail here and also here.

OK. Maybe some people haven’t read my previous blogs or believed what I’ve said before (and it is quite shocking) so I will briefly explain again.

When Mathematicians invented modern-day Statistics in the 1930s, they needed a way to see if results from an experiment were a real effect or just randomness. (For example, I throw a coin 10 times and it comes up Heads 7 times, it’s probably just randomness. I throw a coin 100 times and it comes up Heads 70 times, it’s probably biased.) So, Mathematicians invented statistical significance and p values to separate randomness and real effects.

Now, along come some Psychologists. They said “Mathematicians are a bunch of idiots and they’re doing this all wrong, let’s invent our own way of doing things’. So they invented the Effect Size. Mathematicians and Scientists have continued using statistical significance and Psychologists and Educationalists have continued using the Effect Size. They have said repeatedly that Null Hypothesis Significance Testing (i.e. the way Mathematicians and Scientists do things) is wrong.


Statistics Hell

This kind of thing is repeated on numerous Social Science websites.

So, you’ve really got to understand this, it’s not a case of them choosing one technique over another.

The people who use the Effect Size think that statistical significance testing, i.e. the way Mathematicians and Scientists do things is wrong and they have invented their own way of doing Statistics. 

You’ve really got to grasp that to understand what I’ve been saying in my blogs.


5 thoughts on “Why the EEF report on Philosophy for children has no tests for statistical significance

    • Psychologists are ‘strongly encouraged’ i.e. forced by the American Psychological Association to use the Effect Size instead of statistical significance if they want to publish stuff in journals.

      We’ve discussed this before Andrew, you use Mathematician to mean people who just do Pure Maths, I use it to include people who do Statistics as well. I’m happy to agree to disagree as I don’t think either is ‘the’ definition but to pretend you don’t know that is a bit naughty.

      • I use “mathematician” to mean people who study maths. Using it to mean statisticians is misleading. I see no reason to stop challenging you when you are being misleading, just because I’ve done so in the past.

        I’m curious as to what you are referring to with the APA, but it is still, nevertheless, common for psychologists to use statistical significance and misleading to suggest they don’t.

  1. Just done an MSc in Ed Research Methods at Leeds Uni. In the quant. method course we covered and used significance testing as well as learning a little about effect size. It was only an introductory course, so I don’t feel qualified to argue about their relative merits, but significance testing was one of the major components of what we learned.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s