“How could thousands of Psychologists and Educationalists all make the same mistake? Entire fields doing incorrect Statistics. It’s simply not plausible.”
On Thursday night I read a piece called ‘The Art of being Right’ by Arthur Schopenhauer. Underneath I reproduce a few paragraphs from a section entitled ‘Appeal to Authority rather than Reason’.
“When we come to look into the matter, so-called universal opinion is the opinion of two or three people; and we should be persuaded of this if we could see the way in which it really arises.
We should find that it is two or three persons who, in the first instance, accepted it, or advanced it and maintained it; and of whom people were so good as to believe they had thoroughly tested it. Then a few other persons, persuaded beforehand that the first were men of the requisite capacity, also accepted the opinion. These, again, were trusted by many others, whose laziness suggested to them that it was better to believe at once, than to go through the troublesome task of testing the matter for themselves. Thus the number of these lazy and credulous adherents grew from day to day; for the opinion had no sooner obtained a fair measure of support than its further supporters attributed this to the fact that the opinion could only have obtained it by the cogency of its arguments. The remainder were then compelled to grant what was universally granted, so as not to pass for unruly persons who resisted opinions which everyone accepted.
Since this is what happens, where is the value of the opinion even of a hundred millions? It is no more established than a historical fact reported by a hundred chroniclers who can be proved to have plagiarised it from one another; the opinion in the end being traceable to a single individual.”
Gene Glass should be the most famous man in Education. He is the person who changed the way the ‘Effect Size’ is used and spread its new use throughout Education. He became an Educational Psychologist in 1964. In the early Seventies he was receiving Psychotherapy and decided it had helped him so much that he wanted to prove to everyone that Psychotherapy worked. He’d learned about the ‘Effect Size’ from Jacob Cohen’s book ‘Statistical Power Analysis for the Behavioral Sciences’. (Jacob Cohen originally invented the ‘Effect Size’ and wrote a 500 page book explaining how to correctly use it to find the number of people you needed for your experiment.) Glass decided to completely change the way Jacob Cohen used the ‘Effect Size’, throw away the carefully constructed statistical look-up tables and use it for a completely different reason, sticking results together. While he was doing this, Glass was also elected as the President of the American Educational Research Association. He used his Presidential address to 1,500 educational researchers to announce his new method of putting results together using the new way of using the ‘Effect Size’. How many of those researchers would have thought that there was any element of doubt in what this eminent man was telling them at this prestigious occasion? How many of them would have had the necessary expertise to tell if it was correct or not? Glass wrote a 2 page pamphlet justifying his new way (this has a few sketches on it as proof) and published an article with his wife, Mary Lee Smith, in ‘American Psychologist’. Psychologists and Educationalists all started to copy him and the new method spread throughout Psychology and Education.
So, imagine all the children of the world, underneath them, supporting them are the teachers from all the different countries, underneath them is the whole of education research and all of this, resting on his shoulders, is just one man, Gene Glass. Given that Mathematicians have never taken the remotest bit of interest in the ‘Effect Size’, are we absolutely sure he’s correct?