A large scale multi-school, multi-nation well-being intervention: Convincing evidence or not?

backlit beach dawn dusk
Photo by Cedric Lim on Pexels.com

A few weeks ago, 600 teachers in my district ranging from early childhood to secondary, attended a professional development event to launch a new district wide well-being initiative. The speaker was Dr Denise Quinlan, director of the NZ Institute of Well-Being and Resilience and graduate of the University of Pennsylvania Center for Positive Psychology.  Denise made the case for the introduction of well-being interventions in NZ schools and cited a key piece of research in support of her arguments. The research is a PhD thesis by Adler Alejandro, entitled “Teaching Well-Being Increases Academic Performance: Evidence from Bhutan, Mexico and Peru”  and carried out at the Center for Positive Psychology at Pennsylvania University. The thesis can be downloaded here.

Being interested in assessing the quality of the evidence behind educational interventions, especially one that brought 600 teachers together, I read the thesis. For a PhD study it is a spectacular piece of work spanning 3 nations and a huge number of schools and students. The first study was in Bhutan, a 2nd in Mexico and a 3rd in Peru. In all cases the well-being intervention caused significant increases in student well-being and academic performance. Furthermore, the research used randomisation, an active control group, single blind experimental techniques along with large samples sizes and international replication which surely means it is a convincing piece of evidence. Or is it?

In each country the intervention consisted of roughly the same ten curriculum elements, customized somewhat to the local culture. Here are the elements for Bhutan:

Bhutan

The number of different elements strikes me as problematic: Which of them, or which combination of them, caused the increase in student academic performance? It doesn’t matter, you might conclude, what matters is that it worked. But what if most of the benefits where due to one or two elements and the rest had no effect, surely we would want to know if we were trying to design a similar intervention?

But the complications don’t end there. In addition to the ten elements outlined above, teachers in the intervention group were also taught how to give written and verbal feedback to students with a positive rather than negative or corrective focus from the teacher – something Alejandro claims (without evidence) is typical of teachers. Also the teacher training was designed to be a significant personally transformative experience, presumably from a positive psychological perspective. So the possible causal effects for changes in student performance start to compound. Did positively transformed teachers teach their curriculum with more vigour than usual resulting in higher academic performance?

Pertinent to discuss also is the nature of the control group. Teachers in the control group were trained to teach human anatomy, psychology and nutrition to students, and did so for the same amount of time as the intervention group. Perhaps I am being unreasonable, but the difference between the intervention and control group programs seems too big to adequately compensate for intervention effects (eg. the power of enthused teachers doing something new with students).

The crunch question though, is does this research provide enough evidence to underpin the design and implementation of similar well-being interventions in NZ schools? Adhering closely to the design of the intervention used may give the best chance of success. But we don’t know enough about what caused the positive effects to adjust the program to our NZ context. For example most NZ secondary schools already teach a Health curriculum based around well-being. Does that mean many of our students already have good well-being skills, more so perhaps than students in Bhutan, Mexico or Peru? Is the opportunity cost of implementing this particular program with high fidelity too high compared to other possible initiatives such as improving the quality of literacy and numeracy instruction or creating knowledge-rich curricula, all of which would improve educational achievement and the life-long well-being of students?

 

Fueling teacher practice: Av-gas or Liquid Hydrogen?

selective focus photography of gasoline nozzle
Photo by rawpixel.com on Pexels.com

Aviation gas has an energy density of 33.5 MJ/L whereas liquid hydrogen only has an energy density of 9.3 MJ/L. If ideas are the fuel that drives teachers then too many are running on liquid hydrogen.  Teacher practice is based around a number of key ideas and too many smart, dedicated, caring teachers are being given the wrong ones.

A popular one is ‘it’s all about relationships.’ The implications are students will learn best when the teacher knows their background allowing a positive relationship to be formed and thus the teacher will be able to optimise their teaching and behaviour management of said student. Trouble arises because, although there is a kernel of truth in this, it is flawed as a starting point.

To begin, most teachers have between 20-30 students in their class that can only be successfully managed as a group, not as individuals. If the teacher is in a high school, that number increases to between 100-120 students, often seen all in one day. In these circumstances, the best chance of a positive relationship between a teacher and their students depends on the teacher establishing firm control of student behaviour dynamics so that effective teaching and learning can take place. The best teachers are assertively in charge, approachable yes, but able to command student attention and co-operation to foster learning. They are not setting out to befriend students as a strategy.

Knowing more about the background of students – and yes, some have very difficult backgrounds – can cause a teacher to lower expectations. This might involve going easier on them over behaviour issues or work completion, both of which are unhelpful for the student concerned. For example, a student who is weak at maths, needs more practice at maths, not less. A maths teacher knowing a student is weak at maths but doesn’t do homework because his parents fight every night is not helping them by letting them off work.

The principal of the high school I attended as a student insisted that his teachers be firm, fair and friendly – in that order. Being firm, fair and friendly is not the same as being friendly, fair and firm, which is can be the case with more student-centered approaches. The later approach can place the teacher in the position of soliciting the friendship and good-will of students.

 

 

Teaching ‘Sticky’ and ‘Non-sticky’ Kids

pexels-photo-1148998.jpeg
Photo by Sharon McCutcheon on Pexels.com

Whether we like it or not, some kids do better at school than others. A good part of this is beyond our control, determined by their genetic inheritance. In his book ‘Blueprint: How DNA Makes Us Who We Are’  Robert Plomin finds that on average, between 60 and 70% of educational achievement is down to genetics. This still leaves a significant 30-40% determined by the environment which schools, parents and peers can affect. Obviously educational achievement is strongly informed by intelligence and a useful way to look at it, conceived and researched by the psychologist Raymond Cartell, describes it as consisting of two parts – fluid intelligence and crystallised intelligence.

Fluid intelligence is the genetically determined part and consists of our working memory capacity and ability to use logic and reason to solve new unfamiliar problems without depending on prior knowledge. It is worth noting that nobody has yet found a way to improve fluid intelligence, despite the intensive effort to develop cognitive training programs. Crystallised intelligence consists of the knowledge you have in long term memory – what you know. It is this second part that parents and schools can affect, and as David Didau argues so persuasively in his book ‘Making Kids Cleverer – A manifesto for closing the advantage gap,’ increasing crystallised intelligence should be the principle aim of schooling.

One reason for the achievement or advantage gap is the amount of knowledge and vocabulary students enter school with. Students that have been read to extensively, been part of many dinner time conversations, talked at more by adults, taken on trips to libraries, museums etc., have a significant advantage. These students have more crystallised intelligence. This prior knowledge permits them to understand and retain a greater percentage of everything the teacher says, both instructions for activities and explanations of concepts. It is as if their minds possess intellectual velcro or stickiness.

Because significant numbers of children arrive at school without this useful prior knowledge or mental stickiness it is important for teachers to do two things:

  1. teach a knowledge-rich curriculum explicitly
  2. employ specific strategies to help students remember what they learn

Teaching a knowledge-rich curriculum ensures any cultural knowledge deficits are ameliorated as much as possible. Explicit teaching means not setting up learning activities which privilege students with more prior knowledge, such as inquiry based learning or semi-discovery approaches, but carefully breaking down topics and explaining them directly to students, questioning to check understanding and giving lots of guided practice using the knowledge.

Whether they admit it or not, every teacher wants, and aims for students to remember lesson content, however few take deliberate steps to help students remember what they learn. An exception would be teachers who believe they are teaching transferable general skills and that detailed content can be looked up on google as required so there is no point in remembering it, but cognitive science is very clear that we can only think with the information we have in our long term memory, making this approach more than a little flawed.

Strategies to help students remember learning such as key vocab, definitions and procedures can include frequent low-stakes tests or student self-quizzing using knowledge organisers, chorusing responses and online games such as Kahoots. The aim is for the knowledge to be transferred into student’s long term memories permanently, not just for an exam or end of topic test. To this end cumulative rather than modular end of topic tests which include content from previous topics helps cement learning.

 

 

 

Starting points: teaching other people’s kids

close up of text
Photo by Public Domain Pictures on Pexels.com

In my last year of high school my favourite subject was physics. I enjoyed mechanics the most, calculating velocity, acceleration, angular momentum etc., and the steps in logic in working them out. I remember one key strategy for solving exam style problems imparted to us, which was to always start with what you know. This meant something very specific, which was writing down in a list all the parameters for which values were given in the question and then reading carefully to figure out what parameter you were being asked to calculate.

Today starting with ‘what you know’ seems to me to be the most logical way to address the enterprise of teaching other people’s kids. Except instead of a few values given in an exam question, ‘what you know’ becomes the ‘what we know’ in a broader scientific sense. It means considering what can be reasonably concluded from empirical investigations in relevant fields – which conclusions are supported by evidence and better still supported by a plausible theory?

I should explain I use the term ‘teaching other people’s kids’ because I think if you were, for example, teaching your own kids, you’d be entitled to teach them any which way you choose, whereas when you are employed to teach other people’s kids, you and the school should be using proven methods, as befitting any professional providing a service to the public. This is where starting from ‘what we know’ in a scientific sense comes in.

In my experience most parents and many teachers are not aware that much educational practice, both historically and currently is not developed with reference to empirical evidence, but is based instead on beliefs about how students learn. For example – students learn best when they work independently, when they choose their own topics, when they learn in an authentic context or when the teacher’s lesson delivery matches their preferred learning style. Teachers and principals, must ask ‘what is the evidence for this?’

It is astounding for me to reflect, 16 years into my career, that my teacher training did not take account of nor reference knowledge from the scientific fields relevant to learning. Even by the start of this century the fields of cognitive science and educational psychology had established reliable insights to guide teachers in their practice. There were also considerable bodies of research directly speaking to what works and what doesn’t in schools and in the classroom.

Why should we be privileging science over belief as the basis for teaching? This is because, although far from perfect, it is the best hope we have of figuring out what works and why and distinguishing that from our own folk theories, anecdotes and biases. And our children deserve this. It doesn’t mean that there is a piece of research to back every decision a teacher or school makes, but that their decision making should be informed where possible by research.

 

 

 

 

Education: Arguments from Imagination or Evidence?

closeup photo of woman in black top with her index finger on top of her lower lip
Photo by Hannah Nelson on Pexels.com

Call me stupid. I definitely think I was for the first 15 years of my teaching career. Stupid, because I did not apply the reasoned, evidenced based thinking I had learned completing a PhD in a scientific field, to my new profession of secondary school teaching. Granted, the education training I received, a one year post-graduate diploma, did not encourage me in any way to do this, but did I really need to spend so long with my head in the sand before starting to question the basis of my teaching?

Better late than never, goes the saying, and several years ago I decided I needed to start finding more out about education. I started searching to find if there was a counter-narrative to the dominate educational paradigm I had learned in teacher training and continued to receive in teacher PD down the years.

Book depository is a wonderful thing and got the ball rolling with ‘The Smartest Kids in the World: And How They Got That Way’ by Amanda Ripley, Eric Kalenze’s ‘Education Is Upside-Down: Reframing Reform to Focus on the Right Problems’ and Daniel Willingham’s great book ‘Why Don’t Students Like School?: A Cognitive Scientist Answers Questions About How the Mind Works and What It Means for the Classroom. Many more excellent books followed  – David Didau and Nick Rose’s ‘What Every Teacher Needs to Know About Psychology,’ Daisy Christodolou’s ‘Seven Myths About Education’ and Doug Lemov’s ‘Teach Like a Champion.’ I should add that several excellent blogs, notably by Greg Ashman in Australia and David Didau in the UK provided many valuable insights too.

What I rapidly learned was that there was a dominant ‘group think’ or educational paradigm operating and that there are teachers and educational academics questioning it who present a valid counter-narrative. I realised that many of the accepted ‘truths’ that teachers base their collective practice on were not based on any form of empirical evidence or reasoning, but were instead what Stephen Pinker calls ‘arguments from imagination.’ Examples would be learning styles, students learn best when they discover knowledge themselves, teaching from the front is a poor strategy, getting students to memorise key knowledge is harmful and the classic 21st Century skills arguments about critical thinking, creativity and collaboration.

With my head somewhat cleared of the edu-myths I had been programmed with I returned to the empirical approach I learned as a science post-graduate, which I now feel is the most ethical starting point we can take as a profession. To begin with, what can the fields of science that deal with how humans learn and behave – cognitive science and psychology, actually tell us that will help us formulate our teaching practice? What do the substantial amounts of research into teacher practice and student achievement tell us? Is it all contradictory, as some claim, or are there commonalities that give us robust starting points? I should note my teacher training did not take this approach, in fact, a sign of respect was that you had developed a coherent set of beliefs about how students learn best.

In 2018, along with 3 of my colleagues, I spoke at the first ever ResearchEd conference in Auckland about how we must privilege scientific evidence in education (text of talk here). If we wish to solve some of the seemingly intractable problems facing NZ education, such as the māori achievement gap and the crisis in reading and maths, it is essential that we examine evidence to determine the causes and the best courses of action. In my next blog I will address the scientific evidence around reading and how it needs to be applied in the NZ context.

 

 

A Smoking Gun? Student Attendance and the Achievement Gap in NZ.

Image result for school student black and white on way to school clipart

In a previous post I discussed the New Zealand Ministry of Education’s premise that unconscious racist bias was the cause of the persistent under achievement gap for Māori versus Pakeha (NZ European) students. I argued that it was an unfalsifiable hypothesis since it cannot be measured and the possibility of modifying it unlikely. Here I present an alternative causal agent, one that can be measured and directly acted upon by schools.

Student attendance in NZ is collected by the Ministry of Education and the most up to date set of data and an accompanying report was released this year (here) . This data reveals that for the years 2011 to 2017, Pakeha were 15% more likely to attend school regularly than Māori (attending regularly is defined by the MOE as 90% or more). The Ministry’s own report provides graphs showing the strong positive correlation between student achievement and performance in Level One of NCEA, the country’s national qualification.

It seems very plausible that at least some of the underachievement gap for Māori students versus Pakeha is due to poor attendance. This is particularly significant given the percentage of internal course work in many NCEA courses – is often as high as 80 to 100%. In fact, by the Ministry’s own data, a drop in attendance from 80 to 70% corresponds to a 10% drop in the probability that they will achieve NCEA Level 1.

Clearly improving attendance is no simple challenge for schools, involving changing the attitudes and patterns of behaviour of both students and caregivers. But it is a measurable outcome that could have significant impact in reducing the underachievement gap for Māori students, perhaps more effectively than hunting down the ghosts of unconscious racial bias.

 

 

 

White privilege, White thinking, White spaces: A brief comment

cozy-ideas-people-clipart-black-and-white

 

Students of politics and the social sciences will no doubt be familiar with these concepts but, they may be unfamiliar to many (as a science graduate they were unknown to me until very recently). They have been used in articles about the NZ education system and the achievement gap for maori linklink . The purpose of this blog is to very briefly outline a little of the background to these terms.

The concept of ‘white privilege’ is associated with identity studies, collectively – critical race theory, intersectionality, postcolonial studies, and queer theory which developed out of post-modernism (which was heavily influenced by Marxism.) Key features are the identification of oppressed and oppressor groups and discrimination as the sole cause of inequality in measured outcomes. Oppressed/oppressor pairs include white/black, male/female, straight/gay, feminist/transgender, and even attractive/unattractive. In keeping with its postmodern origins, scientific evidence is often disregarded, dismissed disparagingly as the product of a white male patriarchy.

While most of us are sympathetic to social justice causes, the discourse emanating from identity studies is extreme. Some examples – gender is totally socially constructed, only white people can be racist, or closer to home, the claim by the NZ Ministry of Education that the achievement gap for maori students is due to unconscious racial bias of teachers (read about this here). Never mind the dozen or so other causal factors that might be in play.

It is not that discrimination does not exist and has not existed in the past for the aforementioned oppressed groups, but personal or systemic discrimination has become the blanket explanation and moral trigger point for explaining every disparate statistic between unequal groups in society. Stereotyping white people as privileged is unhelpful and simplistic and begs the question where do individuals of mixed heritage fit into the scheme of things?