A large scale multi-school, multi-nation well-being intervention: Convincing evidence or not?

backlit beach dawn dusk
Photo by Cedric Lim on Pexels.com

A few weeks ago, 600 teachers in my district ranging from early childhood to secondary, attended a professional development event to launch a new district wide well-being initiative. The speaker was Dr Denise Quinlan, director of the NZ Institute of Well-Being and Resilience and graduate of the University of Pennsylvania Center for Positive Psychology.  Denise made the case for the introduction of well-being interventions in NZ schools and cited a key piece of research in support of her arguments. The research is a PhD thesis by Adler Alejandro, entitled “Teaching Well-Being Increases Academic Performance: Evidence from Bhutan, Mexico and Peru”  and carried out at the Center for Positive Psychology at Pennsylvania University. The thesis can be downloaded here.

Being interested in assessing the quality of the evidence behind educational interventions, especially one that brought 600 teachers together, I read the thesis. For a PhD study it is a spectacular piece of work spanning 3 nations and a huge number of schools and students. The first study was in Bhutan, a 2nd in Mexico and a 3rd in Peru. In all cases the well-being intervention caused significant increases in student well-being and academic performance. Furthermore, the research used randomisation, an active control group, single blind experimental techniques along with large samples sizes and international replication which surely means it is a convincing piece of evidence. Or is it?

In each country the intervention consisted of roughly the same ten curriculum elements, customized somewhat to the local culture. Here are the elements for Bhutan:

Bhutan

The number of different elements strikes me as problematic: Which of them, or which combination of them, caused the increase in student academic performance? It doesn’t matter, you might conclude, what matters is that it worked. But what if most of the benefits where due to one or two elements and the rest had no effect, surely we would want to know if we were trying to design a similar intervention?

But the complications don’t end there. In addition to the ten elements outlined above, teachers in the intervention group were also taught how to give written and verbal feedback to students with a positive rather than negative or corrective focus from the teacher – something Alejandro claims (without evidence) is typical of teachers. Also the teacher training was designed to be a significant personally transformative experience, presumably from a positive psychological perspective. So the possible causal effects for changes in student performance start to compound. Did positively transformed teachers teach their curriculum with more vigour than usual resulting in higher academic performance?

Pertinent to discuss also is the nature of the control group. Teachers in the control group were trained to teach human anatomy, psychology and nutrition to students, and did so for the same amount of time as the intervention group. Perhaps I am being unreasonable, but the difference between the intervention and control group programs seems too big to adequately compensate for intervention effects (eg. the power of enthused teachers doing something new with students).

The crunch question though, is does this research provide enough evidence to underpin the design and implementation of similar well-being interventions in NZ schools? Adhering closely to the design of the intervention used may give the best chance of success. But we don’t know enough about what caused the positive effects to adjust the program to our NZ context. For example most NZ secondary schools already teach a Health curriculum based around well-being. Does that mean many of our students already have good well-being skills, more so perhaps than students in Bhutan, Mexico or Peru? Is the opportunity cost of implementing this particular program with high fidelity too high compared to other possible initiatives such as improving the quality of literacy and numeracy instruction or creating knowledge-rich curricula, all of which would improve educational achievement and the life-long well-being of students?