The Chinese Way

The Government announced today that they are spending £11 million to have 32 hub schools, bringing Chinese teachers to the UK and sending our teachers to China so we can learn how they do so well compared to us in Mathematics.

Yay, there’s a magic bullet that means the children can achieve without hard work or behaving themselves? I know we’ve been fooled many, many times before but this time it’s really true.

But wait, what’s this?  If we look at the report released by Parliament – ‘Underachievement of White Working Class children‘

 

We see that the poorest Chinese children beat the richest non-Chinese children in this country.

It’s almost as if there is something else going on and the Chinese don’t have a magic way of teaching Maths at all.

The answer is of course that the Chinese have a culture of hard work and respect for education as typified in the ‘Battle hymn of the Tiger Mother‘ book by Amy Chua. They bring this with them when they move countries enabling them to come top in our country as well.

Maybe we need to be worrying more about copying the attitudes and culture of the Chinese parents and children towards Education and a little less trying to copy a mythical, magic, Chinese way of teaching.

Significance testing

When I was investigating the ‘Effect Size’ I found lots of criticism of significance testing on Social Science websites. Remember, this is once again, Social Scientists, often but not always Psychologists, criticising the way Mathematicians and Scientists do Statistics.

This is actually a fundamental part of the ‘Effect Size’ story as their failure to understand the significance testing procedure has led directly to the ‘Effect Size’ as they try to solve a ‘problem’ that isn’t really a problem, only a misunderstanding on their part.

It is also vital to recognise that the ‘Effect Size’ isn’t just another statistical method to choose from amongst many, it is the tip of the ice-berg of a completely different ethos. The people who advocate using the ‘Effect Size’ think that the whole way Mathematicians and Scientists do Statistics is wrong so they’ve decided to  invent their own version. This has been mistakenly copied by people in Education like John Hattie.

In my next post I’ll be looking at the Maths of significance testing, but, what if you don’t know anything about Alpha levels or Type 1 and 2 errors, how could you judge? Well, a good place to start would be the mathematical credentials of the people making the criticism. So let’s have a look at the people who are criticising significance testing.

If we type in ‘Criticism of Significance testing’ into Google, the first ten results are –

http://community.dur.ac.uk/r.j.coe/teaching/critsig.htm – Number one on the list, our old friend Robert Coe, Professor of Education at Durham University

http://en.wikipedia.org/wiki/Statistics – A general article on Statistics by Wikipedia

http://www.cem.org/attachments/publications/CEMWeb037%20The%20Case%20Against%20Statistical%20Significance%20Testing.pdf – CEM, Professor Coe’s organisation publishing an article by Ronald P.Carver, Professor of Education and Psychology at the University of Missouri

http://errorstatistics.com/2012/12/24/13-well-worn-criticisms-of-significance-tests-and-how-to-avoid-them/ – Deborah Mayo, Professor of Philosophy at the University of Pennsylvania

http://www.johndcook.com/blog/2008/11/18/five-criticisms-of-significance-testing/ – John D Cook, Consultant in Applied Mathematics and Computing

http://www.uic.edu/classes/psych/psych548/fraley/ – R.Chris Fraley, Professor of Psychology at the University of Chicago

 http://www.johnmyleswhite.com/notebook/2012/05/10/criticism-1-of-nhst-good-tools-for-individual-researchers-are-not-good-tools-for-research-communities/ – John Myles White, PhD student in Psychology

http://www.andrews.edu/~rbailey/Chapter%20two/7217331.pdf – Andrews University Education department. Authors, Jeffery Gliner, retired Professor of Psychology, Associate Professor Nancy Leech, PhD in Philosophy and MA in Counselling, George Morgan, retired Professor of Education

http://lesswrong.com/lw/g13/against_nhst/ – No information

http://www.ncbi.nlm.nih.gov/pubmed/17002771 – Authors – Dr Fiona Fidler, Environmental Science, background in Psychology and Philosophy, Mark Burgman, Environmental Science, background Zoology, Geoff Cummnigs, retired professor of Psychology, Robert Buttrose, background in Philosophy, Neil Thomason, historical and philisophical studies

And so it goes on, page after page of Psychologists, Philosophers and Education Professors critisicing the way Mathematicians and Scientists do Statistics.

So, you can judge for yourself the quality of the people criticising the way Mathematicans do Maths. Though this time we do seem to have a lot of Philosophers as well as Psychologists.

Now, this is important because, their mistakes in significance testing have led to the ‘Effect Size’, which has led to Education research being done incorrectly, which has an impact on real children in real classrooms.

In my next post, I will deal with the more Mathsy side of things. I will show that their criticisms of significance testing are baseless and just show their poor understanding of Statistics.

Can we stop yet?

If you were not a Mathematician you might think that all Mathematicians are pretty much the same, however, there are three main strands to the Maths that gets taught at University. Pure, Mechanics and Statistics. A bit like Science splits into Biology, Chemistry and Physics. Pure is Algebra, proofs, very abstract things like that, whereas, Statistics is all about analysing data from the real world. Someone who was very accomplished at Pure Maths would, nevertheless, be a total beginner at Statistics as the skills and knowledge aren’t really transferable.

What we have here is a classic case of someone who is an expert in their own field, switching to a different field, forgetting they are no longer an expert, yet, still being supremely confident in their own judgement and opinion. A good analogy would be someone who does a Physics degree up to Quantum Mechanics level who then moves over to Biology. They need to go right back to the beginning and start to quietly learn the different parts of a cell. Imagine if they started to loudly disagree with accepted Biological opinion after a week of lessons. Yet, Professor Coe does disagree with accepted Statistical opinion. No Mathematician uses the Effect Size.

Professor Coe did a Pure Maths degree which had no statistics in it.

John Hattie did an Arts degree which had no statistics in it.

There’s a very simple reason they advocate the use of statistics you won’t find in any Maths textbook, their degrees contained no statistics.

Nobody in Maths uses the Effect Size.

Can we stop yet?

Who works at the Education Endowment Foundation?

The Education Endowment Foundation was set up in 2011, with a £125 million grant from the Department of Education. So, who are the Executive Team spending this money?

Chief Executive

Dr Kevan Collins, Chief Executive, PhD in Literacy Development

Grants

Eleanor Stringer, Grants Manager, BA Philosophy, Politics and Economics

Emily Yeomans, Grants Manager, BSc Biology, PGCE Science

Matthew van Poortvilet, Grants Manager, no information

Evaluations

Camilla Nevill, Evaluations Manager, BA Experimental Psychology

Elena Rosa Brown, Evaluations Officer, MA Psychology and Education

Sarah Tang, Evaluations Officer, MSc Economics of Education

Dissemination and Impact

Robbie Coleman, Research and Communications Manager, no information

James Richardson, Senior Analyst, BA Politics, PGCE Geography, MA Education, Culture and Society

Dr Jonathan Sharples, Senior Researcher, M Biochem, Biochemistry, PhD Biochemistry

Peter Henderson, Research Officer : BA History, MSc Public Policy

Sharmini Selvarajah : Deputy Head of News, BA Social and Political Science, MA Public Policy

Development and Communications

Stephen Tall, BA Modern History, MA Modern History

Rebecca Clegg. BSc Marketing

So, my question is this. If none of the people who run the Education Endowment Foundation have any qualifications in Mathematics or Statistics, exactly whose expertise are they following? Would they know if they’d made an error? You might say, well they’ve obviously asked the experts, but the experts are Mathematicians and they haven’t asked them, because, no Mathematician uses the ‘Effect Size’, so whose advice are they following? The learning of millions of children may depend on it.

Who are the authors of the EEF toolkit?

The Sutton Trust-EEF Teaching and Learning Toolkit is a summary of all the research that the EEF has done so far. So, who are the authors of this Toolkit that advises the 500,00 teachers in this country the most effective way to teach their pupils?

Authors

Professor Steve Higgins, Professor of Education at Durham University, BA Literae Humaniores (Classics – Ancient Rome, Ancient Greece, Latin, Ancient Greek and Philosophy)

Dr Maria Katsipatki, Research Associate at Durham University School of Education, BA Psychology

Dr Dimitra Kokotsaki, Lecturer at Durham University School of Education, no information about her degree but she lectures in Music education

Professor Rob Coe, Professor of Education at Durham University, BSc Mathematics, PhD Education

Dr Lee Elliott Major, Director Sutton Trust, BSc Physics, PhD Physics

Robbie Coleman, Research and Communications Manager EEF, MSc Comparative Social Policy

When I originally envisioned the EEF, I imagined dozens of Statisticians, bent over spreadsheets, all furiously discussing their statistical methodology. Yet, it appears than in the whole of the EEF only one person has any Mathematical training at all. And, he’s using Statistics that no Mathematician has ever heard of. Are we willing to bet the lives and aspirations of millions of children on one man’s say so? I’m not sure we should.

Psychology journal bans significance testing

The journal “Basic and Applied Social Psychology” has banned significance testing, confidence intervals and p values.

These are all mainstream Statistics tools with decades of use by Mathematicians.

Stephen Gorard who evaluates reports for the EEF has also recently called for the banning of these tools in Education.

Massive hat-tip to @cbokhove.