How did the inventor of the Effect Size use it? (Not the way Hattie does.)

In 1969, Psychologist Jacob Cohen released his book ‘Statistical Power Analysis for the Behavioral Sciences’. In this book Jacob Cohen introduced the Effect Size for the first time and explained how to use it.

So, how did Jacob Cohen, the inventor of the Effect Size, use it?

Image

Quick translation – I noticed that people in the Behavioral Sciences sometimes did badly designed experiments because they didn’t understand Statistics well enough, so, I decided to help them by making some easy look-up tables.

Image

Quick translation – There are four ways to do Power Analysis, but two of them are rarely needed. The two main ways you need to check your experiment before you do it are, firstly, check the Statistical Power is high enough or alternately check you have planned to test enough people.

Image

Image

Quick translation – To use the Statistical Power tables, you need to know the number of people in your experiment, the Statistical Significance you want and the Effect Size.

And here is a Statistical Power table from Jacob Cohen’s book, notice the Effect Size (d) at the top. There are dozens of pages of these tables in his book.

Image

And here he gives an example of how to use the Statistical Power tables.

Image

The other thing you need to check is the Sample Size.

Image

Quick translation – The other way to check your experiment is with the Sample Size table. To use this your need the Statistical Power, the Statistical Significance and the Effect Size.

And here is a Sample Size table, notice the Effect Size (d) at the top. Again there are dozens of pages of these tables in the book.

Image

And he gives an example of how to use the Sample Size table.

Image

Now, every modern user of the Effect Size cites Cohen and they always quote him about small, medium and large effects. This gives the impression that they are just continuing his work, yet, they are using it in a completely different way to him.

Jacob Cohen, the inventor of the Effect Size, used it to check the Statistical Power and the Sample Size of an experiment before you did the experiment. He did this using look-up tables.

Advertisements

7 thoughts on “How did the inventor of the Effect Size use it? (Not the way Hattie does.)

  1. Given what you say, does this mean (in your opinion) that the way effect sizes are used to interpret educational research now is inappropriate? What are the consequences of using them the way they are currently used?

  2. In my understanding, the use of effect sizes in meta-analysis was developed by Glass. How did he want us to use it?

    • Well, Educational Psychologist Gene Glass didn’t so much ‘develop’ the Effect Size, as, wake up one day and decide to use it in a completely different way to the inventor with no mathematical explanation whatsoever.

      He does also use it in a different way to Hattie though, so, I make it at least three different ways to use it.

  3. You have quite a few posts that basically target John Hattie’s work, in particular. I’m not here to judge whether you are on to something, but just to ask a few questions:

    1) Why are you so passionately targeting Hattie’s work, in particular?

    2) You mention that his research is flawed due to calculation errors and even using inappropriate statistical methods. What would you have done differently to determine the overall effectiveness of the strategies John Hattie has studied?

  4. Pingback: Época: desinformação em três atos – final | AVALIAÇÃO EDUCACIONAL – Blog do Freitas

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s