METHODOLOGICAL REVIEW OF MODEL-BASED COST-EFFECTIVENESS ANALYSES OF SCHOOL-BASED INTERVENTIONS TO INCREASE PUPILS’ LEVEL OF PHYSICAL ACTIVITY

REVISÃO METODOLÓGICA DE ANÁLISES DE CUSTO-EFETIVIDADE BASEADAS EM MODELOS DE INTERVENÇÕES ESCOLARES PARA AUMENTAR O NÍVEL DE ATIVIDADE FÍSICA DOS ALUNOS

ABSTRACT

A large number of school-based interventions promoting physical activity have been developed. Due to difficulties of obtaining observational data on long-term consequences such as effects and costs, modelling techniques offer opportunities to consider these. The aim of this study was to provide an overview of modelling approaches applied in economic evaluations of school-based physical activity programmes. We identify key methodological choices, challenges, and areas with a lack of evidence. A literature search was conducted to identify all relevant studies published within the last 10 years. The included studies were described with focus on main methodological aspects, including the costs, effects, and modelling techniques. Eight model-based economic analyses of school-based physical activity programmes were identified. The majority of these studies concluded that the interventions had a high probability of being cost-effective or even cost saving based on the national-specific thresholds. Although most studies did provide a description of the models, details on the methodological choices were not always transparent. Moreover, evidence on the effectiveness and inclusion of all relevant cost categories were found to be challenging. Different modelling methodologies have been used to assess the cost-effectiveness of school-based physical activity programmes. Only few studies have evaluated the long-term cost-effectiveness, and they have all challenging methodological issues.

Keywords:
Cost-effectiveness; Physical activity; Intervention

RESUMO

Um grande número de intervenções escolares que promovem a atividade física foram desenvolvidas. Devido as dificuldades de se obter dados observacionais sobre o efeito a longo prazo e as conseqüências dos custos, as técnicas de modelagem oferecem oportunidades para considerar as conseqüências econômicas e de saúde a longo prazo. O objetivo do estudo foi fornecer uma visão geral das abordagens de modelagem aplicadas em avaliações econômicas de programas de atividade física baseados na escola. Ele identifica as principais escolhas metodológicas, desafios e áreas com falta de evidências. Foi realizada uma pesquisa bibliográfica para identificar todos os estudos relevantes publicados nos últimos 10 anos. Os estudos incluídos foram descritos com foco nos principais aspectos metodológicos, incluindo os custos, efeitos e técnicas de modelagem. Oito análises econômicas baseadas em modelos de programas de atividades físicas baseadas na escola foram identificadas. A maioria desses estudos concluiu que as intervenções tinham uma alta probabilidade de serem custo-efetivas ou mesmo econômicas com base nos limites específicos nacionais. Embora a maioria dos estudos fornecesse uma descrição dos modelos, os detalhes sobre as escolhas metodológicas nem sempre foram transparentes. Além disso, as evidências sobre a eficácia e inclusão de todas as categorias de custos relevantes foram consideradas desafiadoras. Diferentes metodologias de modelagem têm sido usadas para avaliar a relação custo-efetividade de programas de atividade física baseados na escola. Apenas poucos estudos avaliaram a relação custo-efetividade a longo prazo e têm questões metodológicas desafiadoras.

Palavras-chave:
Custo-efetividade; Atividade física; Intervenção

Introduction

There are many well-documented health benefits of being physically active including reduced risk of obesity, diabetes, high blood pressure, high cholesterol, asthma and arthritis¹. In children and adolescents, regular physical activity appears to have positive impacts on health, cognition, self-esteem and academic achievement². Regular physical activity in young age may have positive impacts on the health in adulthood since physically active children and adolescents are more likely to be physically active later in life³. The importance of being physically active in childhood has also been emphasised in relation to elimination of excess weight and maintenance of normal weight. Studies have suggested that only modest interventions are necessary to achieve bodyweight change in children^4-6. Due to these benefits, the World Health Organisation (WHO) recommends that children and adolescents accumulate at least 60 minutes of moderate- to vigorous-intensity physical activity (MVPA) each day⁷. However, the proportion of children/adolescents who comply with this recommendation is often low⁸.

Because children/adolescents spend a large amount of time at schools and there is a wide reach of children from all socioeconomic backgrounds, the school-setting may play an important role in encouraging pupils to comply with the recommended phycial activity. In the U.S., the Institute of Medicine recommends all schools to provide students with 60 minutes of MVPA each day⁹. A considerably range of different school-based programs have been developed, and for many the effectiveness have been evaluated¹⁰.

Evaluation of the long-term health and cost consequences is challenging as it has to be documented that the school-based program is able to make the students more physical active during their school years but also that the school-based program is able to make the students more active after they have left the school. This requires long-term follow-up data. Having established that the program can improve the long-term level of physical activity, then changes in the risk of diseases that are related to physical inactive life styles, as well as their association to reduced health related quality of life, increased mortality risks and costs related to healthcare treatment, early retirement and death, have to be established. Although it is desirable to obtain such data from prospective studies, it is both costly and require long follow-up periods of time. Due to these methodological challenges in obtaining evidence on the long-term effectiveness and cost consequences, only a small number of programs have been subjected to a prospective health economic evaluation¹¹. The lack of observational data can to a certain extent be overcome by application of modelling techniques that allow analysts to apply information from a variety of sources and extrapolate the impact of interventions to different population groups, disease or time points¹².

The objective of this study was to review recent model-based analyses addressing the cost-effectiveness of school-based interventions aimed at increasing physical activity in children and adolescents, and to describe the variation in methods applied to model potential long-term effects and costs of the interventions. The scope of the review is restricted to studies that assess cost-effectiveness of school-based interventions and not general interventions targeted at physical activity in children. The focus of the review is on methodological choices applied in the included studies rather than an assessment of their quality.

Methodology

Search strategy

The literature search was restricted to scientific studies published within the last 10 years and included in PubMed, Web of Science and EconLit. Search terms included “child”, “school”, “physical activity” and “economic evaluation” linked with ‘AND’-operator. Related terms were included using the ‘OR’-operator (e.g. “child” included the following MeSH Terms: child, adolescent, students operationalised by: child*, adolesc*, teen*, youth, young*, scholar, pupil*, student*).

Inclusion/exclusion criteria

To be included, the considered interventions had to be school-based, i.e. designed to incorporate some kind of physical activity into one (or more) segments of the school day, including travel to and from school, before- and after-school activities, recess, lunchtime breaks, physical exercise (PE) and lectures. The review was limited to economic analyses using modelling techniques as defined by Mandelblatt et al.¹²: “In its broadest sense, the term ‘modelling’ can be taken to include anything beyond the direct application of observed data. However, in the context of economic evaluation, the term is generally understood to refer to studies that ‘employ an analytic methodology to account for events that occur over time”. This definition sets modelling techniques clearly apart from statistical models such as regression models and meta-analyses which were excluded from the review.

Only original studies (not reviews) written in English and published in scientific peer-reviewed journals were considered. Furthermore, only interventions targeting a general population of children/adolescents and not those with a specific disease or condition, were included. Physical activity programmes were also considered if they included other components (e.g. nutrition).

Data extraction

Papers identified by the defined search terms which were deemed potentially relevant based on the assessment of the titles and abstracts, were obtained as full-text and critically reviewed with focus on inclusion/exclusion criteria. The key methodological elements of the included studies were then summarized in an Excel sheet. The following information about the major characteristics of the studies were extracted: authors, year, country, intervention components, study population, study design, time horizon, perspective, comparator, discounting rates, the effects and their sources, and details of the costs (reference year, currency, cost categories), details of the modelling, uncertainty analysis, and main results.

Based on the extracted data, the methodological choices made in relation to construction of the models were described in separate sections. To describe costs, cost categories were derived from a conceptual framework developed by Wolfenstetter¹³. According to this framework, the cost dimension of economic evaluations of primary preventive physical activity programs include program development costs, program implementation costs (recruitment costs, personnel and non-personnel costs and participant time costs), and cost savings due to health effects of the intervention. The cost savings are composed of direct medical costs (utilisation of healthcare services), direct non-medical costs (e.g. cost of transportation or information costs) and indirect costs (e.g. productivity loss due to morbidity) depending on the chosen study perspective.

Results

Literature search

Figure 1 provides a summary of the results of our literature search most recently updated in August 2017. The initial PubMed search using the predefined Mesh and Title/Abstract terms returned 2312 results, which was reduced to 1264 articles after excluding studies older than 10 years, and to 1196 by excluding non-English articles. After screening of titles/abstracts of the articles, 22 potentially relevant studies were obtained as full-text, out of which 8 studies^14-21 met the inclusion criteria. The searches in the Web of Science and EconLit did not add additional studies.

Figure 1
Flowchart of the literature search

Study characteristics

Table 1 provides a summary of the key components of each study. Three of the studies took place in Australia, three in the United States, one in New Zealand, and one in Canada. The three studies from the U.S. used a ten-year timeframe for the long-term outcomes whereas other studies applied a lifetime timeframe. All the studies undertook the evaluation from a societal perspective except for the study by Rush et al.¹⁸ which used the perspective of the funding body (health care budget perspective), and Ekwaru et al.²¹ that applied a school system’s perspective. Most of the studies modelled a national implementation of the evaluated interventions, only the analyses by Wang et al.¹⁷ and Ekwaru et al.²¹ were limited to smaller cohorts of children.

All the models discounted future costs and benefits at 3% with the exception of Rush et al.¹⁸ study that used a discount rate of 3.5%. The majority of the included interventions showed a high probability of being cost-effective^18,19,21 or even cost saving based on the national-specific thresholds¹⁷.

Table 1
Description of included studies

Study/ Year/ Country	Intervention components	Study population/Age Group	Study and model type	Time Horizon	Perspective	Comparator	Discounting	Reference year and currency	Uncertainty analysis	Main Results
Moodie et al. 2009 (AUS)14	Walking School Bus: active transport (walking to and from school)	Childhood population (age 5-7 years)	CUA, Markov modelling techniques	Rest of life or 100 years for cost-offsets and DALYs, 1 year for the intervention	Societal	No intervention	3% for both costs and benefits	2001, AUD	Simulation-modeling techniques (Monte Carlo simulations)/univariate sensitivity tests	Net ICER: AUD 760,000 per DALY saved
Moodie et al. 2010 (AUS)15	Active After-school Communities (AASC) program: After-school program promoting physical activity	Primary school children (age of 5-11 years)	CUA, Markov modelling techniques	Rest of life or 100 years for cost-offsets and DALYs, 1 year for the intervention	Societal	No intervention	3% for both costs and benefits	2001, AUD	Simulation-modeling techniques (Monte Carlo simulations)/univariate sensitivity tests	Net ICER: AUD 82,000 per DALY saved
Moodie et al. 2011 (AUS)16	Travel SMART Schools (TSS): curriculum program promoting active transport	5th and 6th grade children (age of 10-11 years)	CUA, Markov modelling techniques	Rest of life or 100 years for cost-offsets and DALYs, 1 year for the intervention	Societal	No intervention	3% for both costs and benefits	2001, AUD	Simulation-modeling techniques (Monte Carlo simulations)/univariate sensitivity tests	Net ICER: AUD 117,000 per DALY saved
Wang et al. 2011 (USA)17	Interdisciplinary curriculum intervention (physical activity + nutrition)	Middle school children (11-13 years old)	CUA, extrapolation of results from a RCT	10 years for cost offsets and QALYs	Societal	No intervention	3% for costs and QALYs	2010, USD	Univariate and multivariate sensitivity analyses (Monte Carlo simulations)	Net ICER: −$2966 per QALY
Rush et al. 2014 (NZ)18	Multicomponent program (physical activity + nutrition)	Primary school children 6-8 and 9-11 years old	CUA, extrapolation lifetime model	Lifetime	Funder (governmental)	Historical controls, no intervention	3.5% for all future costs and outcomes	2011, NZD	Sensitivity analyses of cost/QALY by varying conditions of model for younger and older children	ICER: $30,438 per QALY for the younger and $24,690 per QALY for the older children
Barrett et al. 2015 (USA)19	Active PE policy	Elementary school children aged 6-11 years	CEA, Markov cohort model	10 years (2005-2015)	Modified societal	No intervention	3% for both costs and benefits	2014, USD	Probabilistic sensitivity analysis (Monte Carlo)	ICER: $0.34 per MET-hour gained; $401 per BMI unit reduced
Cradock et al. 2017 (USA)20	Six interventions promoting physical activity	Age ranging 5-14 (3-5 in Hip Hop to Health Jr.)	CEA, Individual level micro-simulation model	10 years (2005-2015)	Modified societal	No intervention	3% for both costs and benefits	2014, USD	Probabilistic sensitivity analysis (Monte Carlo)	Most cost-effective intervention: New Afterschool Programs (cost saving -$4.6 billion)
Ekwaru et al. 2017 (CA)21	Health promotion program (physical activity + nutrition)	Grade 5 children (about 10 years of age)	CUA, Markov modelling techniques	Up to 80 years among male and 84 years among female, 2 years for intervention	School system	No intervention	3% for both costs and health outcomes	2008, CAD	Probabilistic sensitivity analysis/two-way sensitivity analysis	ICER: CA$33,421 per QALY gained

• Source: Authors

Summary of the models

Moodie et al., 2009 ¹⁴ , 2010 ¹⁵ , 2011 ¹⁶

The first three of the included modelling studies were conducted as part of the “Assessing Cost-Effectiveness of Obesity”-project, which evaluated thirteen different interventions targeting prevention and reduction of obesity in Australian children and adolescents. Since the methods applied in these studies are similar, they are described once.

The national implementation of the three interventions operating in steady-state (fully implemented without workforce, infrastructure or learning-curve issues) was modelled for 1 year. The time horizon for modelling the cost-offsets and disability adjusted life years (DALYs) averted due to the interventions, was the remaining life-time or 100 years. Due to lack of evidence on effectiveness, effect from the intervention in terms of changed behaviour was based on available information (see Table 3). The relationship between behaviour change, energy balance and body mass index (BMI) was estimated using the “best available evidence”. Deterministic Markov modelling techniques were applied to estimate the number of DALYs averted due to the intervention. The change in DALYs was determined as the difference in future mortality and morbidity between the current practice and the intervention scenario derived from changes in the age-specific BMI distribution over the remaining lifetime (BMI effects were assumed to be maintained into adulthood). Using the modelled effect from interventions, cost-offsets from savings in healthcare costs from lower prevalence of obesity-related diseases were then projected.

Wang et al., 2011 ¹⁷

This study reported a cost-utility analysis (CUA) of an interdisciplinary curriculum intervention promoting healthy nutrition and physical activity among adolescents. The effect of the intervention on preventing bulimia nervosa (BN) was modelled using data from a RCT and several follow-up studies reporting duration and probability of progression from disordered weight control behaviours (DWCB) to BN. Quality-adjusted life years (QALYs) gained during a 10-year period from prevented BN were estimated based on trial-data on health-related quality of life (HRQL), recovery and relapse rates among BN patients. Savings in healthcare costs related to BN prevention were based on data from the literature. In the final analysis, the intervention’s effect on preventing BN was combined with its effect on preventing of adulthood overweight, obtained from a previously published study.

Rush et al., 2014 ¹⁸

A multicomponent (physical activity and nutrition) school-based program was developed in New Zealand’s Waikato District. To assess the effectiveness, 2011 measurements of participants’ BMI were compared to 2004 and 2006 measurements from a RCT. The intervention’s effect on BMI (assumed to decay at 1% per year after the first 5 years) was used to project a shift in the distribution of BMI if implemented nationally, using an existing cohort-based model. The two-arm model estimated the differences in likelihood of remaining in ‘good health’, getting one of 14 obesity-related chronic illnesses and dying for the intervention and control group. Estimates for gained life-years and preference-based utility weights associated with each health state were used to calculate QALY-gains. Savings in future healthcare costs from avoided chronic illnesses were estimated. Main results were presented as net incremental cost-effectiveness ratio (ICER) .

Barrett et al., 2015 ¹⁹

This study used a Markov cohort model to estimate cost-effectiveness of a national implementation of the “Active PE” policy. According to this policy, all U.S. elementary schools were required to devote 50% of PE time to MVPA. The model was calibrated for a closed cohort representing the U.S. population in 2015 and followed over a 10-year period to evaluate the shift in BMI and direct healthcare cost-offsets derived from savings in obesity-related healthcare expenditure resulting from the intervention. The authors calibrated the model with data obtained from the literature. Data from a meta-analysis of active PE trials were used to estimate change in MVPA as a result of the implementation, while two studies (a RCT and a longitudinal study) were used to estimate the change in BMI expected from a change in MVPA. The BMI changes due to the intervention were assumed to be maintained over the follow-up period. Costs of 1-year and 10-year policy implementation were also calculated for the closed cohort

Cradock et al., 2017 ²⁰

This study examined the cost-effectiveness of five different school-based (and one pre-school) interventions which previously have been found effective in increasing MVPA in children/adolescents. The authors conducted a systematic review and identified six interventions that could be used during different times of a school-day (pre-school in one of the interventions). The authors used a stochastic, discrete-time, individual-level microsimulation model to project 10-year outcomes of the national implementation of each of the interventions. A substantial number of articles was systematically reviewed to identify the effectiveness parameters applied in the model. Moreover, several national health registries provided data for the model. After converting expected increase in MVPA per day to MET (The Metabolic Equivalent of Task) -hour and BMI unit change per day (assumed to be maintained over the follow-up period), number of prevented childhood obesity cases and related savings in medical cost by each intervention over the 10-year period was projected. One of the interventions involved non-medical savings related to caregiver time.

Ekwaru et al. 2017 ²¹

This Canadian study assessed the long-term health and economic impacts of a school-based health program promoting healthy eating and active living. The authors used a Markov model to estimate the ICER for the intervention program compared with a scenario without intervention. The impact of the intervention was described in terms of changes in weight status, risk of chronic diseases, and QALY for the cohort of 10-year old children throughout the lifetime up to 80 years among male and 84 years among female. Firstly, the transition probabilities for three weight categories (normal, overweight and obese) were estimated for both intervention and control group based on data collected over two years. Secondly, the two-year outcomes were extrapolated using three scenarios on weight development during the subsequent eight years after the end of the intervention. Thirdly, health states with thirteen chronic diseases, no-chronic disease and a dead state were modelled for the three weight categories (total of 43 annual states) over the remaining lifetime. This complex transition model was then used to estimate the incremental effect in terms of prevented life years with excess weight and chronic disease, and gain in QALYs for the three scenarios.

Methodological choices

Intervention Effects

A variety of effect measures was used in the included studies. While the majority of the studies^14-16,19,20 estimated increase in time spent being physically active and subsequent changes in BMI as their primary short-term effect measure, Rush et al.¹⁸ and Ekwaru et al.²¹ used directly observed changes in BMI and weight status respectively. Wang et al.¹⁷ used change in prevalence of DWCB and weight status as health effects and combined these in the final analysis. The sources of the effects and the quality-level of evidence differed across the studies (Table 2). The short-term intermediate effects were extrapolated to the final outcomes (QALYs or DALYs) using modelling techniques in all but two studies^19,20. The effect in terms of increased MVPA in the two studies was converted to MET-hours and BMI-units based on the literature and these were then used in ICER calculation.

Table 2
Intervention effect and its source

	Effect	Source of effect
Moodie et al. 2009 (AUS)14	Extra time spent on walking to and from school	One week snapshot of the WSB program
Moodie et al. 2010 (AUS)15	Extra time spent on physical activity during after-school	AASC funding guidelines
Moodie et al. 2011 (AUS)16	Extra time spent on active transport	One week snapshot of the WSB program
Wang et al. 2011 (USA)17	Cases of DWCB prevented, composite indicator based on a directly measured BMI and a TSF (Triceps skinfolds)	Data collected as a part of a RCT
Rush et al. 2014 (NZ)18	The median difference in BMI between the intervention group and historical control group	Measurements from an evaluation study (2011) compared to measurements from a RCT (2004 and 2006)
Barrett et al. 2015 (USA)19	Extra time spent in MVPA during PE class	Meta-analysis
Cradock et al. 2017 (USA)20	Increase in MVPA per day (as a result of the 6 different interventions)	Meta-analysis; 9 experimental studies; quasi-experimental study; cluster RCT; longitudinal observational study; one group-RCT
Ekwaru et al. 2017 (CA)21	Change in weight status (APPLE schools compared to control group)	Weight status data obtained from grade 5 students attending APPLE and Non-APPLE schools at baseline and at the end of the intervention

• Source: Authors

Intervention Costs

Program development costs were only included in one²¹ of the 8 studies. Program implementation costs were explicitly reported in all but one study¹⁸. Costs of recruiting participants were assessed in the three Australian studies^14,15,16. There was a considerable variation in inclusion of the personnel and non-personnel costs, depending on the accuracy of the reporting and the type of program (Table 3). None of the studies included cost of participants’ time. Three of the studies^17,18,21 estimated the intervention costs based on the interventions’ budget information, while the remaining studies used a pathway analysis to identify all components of the intervention.

Cost savings

Cost savings in terms of direct healthcare costs were included in all but one study that used the school system’s perspective and therefore excluded them in accordance with the analytical perspective²¹. Only one study²⁰ estimated direct non-medical costs-savings which were expressed as costs of caregiver time, that would be spent by caring for the children in absence of the afterschool program. Similarly, only one study¹⁷ included labour market gains (indirect cost savings). These were calculated as the value of gained productivity due to the intervention’s effect on weight and were included in the study’s final analysis where the effect of the intervention on BN and obesity were combined.

Table 3
Intervention costs and cost savings

Type of cost *	Development costs	Recruitment costs	Program implementation costs: Personnel costs (salaries)	Non-personnel costs	Cost savings
Moodie et al. 2009 (AUS)14	Excluded	Recruitment of local governments and schools; promotional materials	Central coordination and planning; volunteer time costs (average ordinary time earning , leisure time rates); training of volunteers by Vic Roads officers	Training venue hire & catering; kit bags; curriculum materials; pecial events & theme days; insurance and police checks of volunteers	Direct medical (medical cost savings arising from future reductions in obesity-related diseases)
Moodie et al. 2010 (AUS)15	Excluded	Recruitment of teacher, Out of School Hours Service (OSHCS) and Australian Sports Commission (ASC) coordinations; program promotion	National, state, regional and site coordination & planning; teacher or child care worker salary	Equipment, hire venue & transport; after-school care fee	Direct medical (medical cost savings arising from future reductions in obesity-related diseases)
Moodie et al. 2011 (AUS)16	Excluded	School recruitment (TSS coordinator)	Central coordination (project officer); teacher travel time to training; backfill teachers during training; training of teachers	Curriculum manuals; whole-of-school events; training related costs: venue hire, vehicle costs and catering	Direct medical (medical cost savings arising from future reductions in obesity-related diseases)
Wang et al. 2011 (USA)17	Excluded	Not specified	Trainer and assistant trainer salary; reimbursement of teachers for participating in the training	Curriculum book; food during training; school funds	Direct medical costs (medical treatment costs saved per BN case prevented over 10 years + medical costs averted per case of adulthood overweight prevented); Indirect costs (cost of lost productivity averted per case of adulthood overweight prevented)
Rush et al. 2014 (NZ)18	Excluded	Not specified	Not specified	Not specified	Direct medical (reductions in ongoing healthcare and costs associated with the treatment of the chronic health conditions)
Barrett et al. 2015 (USA)19	Excluded	Not specified	State PE coordinator; salary to teacher and principal training facilitator; training time for principals	Curriculum & equipment set; training paper copies	Direct medical (reductions in obesity-related healthcare expenditures over 10 years)
Cradock et al. 2017 (USA)20	Excluded	Not specified	Federal, state, district & transportation coordination; time costs related to training of teachers, principals, recess monitors, School Wellness Champions and program staff; Wellness champion stipend	Equipment; curriculum materials; training materials; specialized CDs and handouts; installation of playground markings; food; transportation; ongoing certification; rent	Direct medical (reductions in obesity-related healthcare expenditures over 10 years); direct non-medical (time of caregivers)
Ekwaru et al. 2017 (CA)21	Research costs	Not specified	Intervention and administration staff costs; professional development	Transport	Excluded

• * Participant time costs were not included in any of the reviewed studies; ** Intervention cost per person was calculated out of budget information (2010 budget for Project Energize)
• Source: Authors

Discussion

Important methodological aspects

This review aimed at providing an overview on different modelling techniques used to assess the long-term cost-effectiveness of school-based physical activity interventions. Eight studies were identified as a result of a comprehensive literature search. The majority of the evaluated interventions were considered to be cost-effective in regards to the country-specific benchmarks. However, due to the differences in methodology of the studies, the results are difficult to compare.

Modelling

One of the fundamental features when constructing a model is to what extent the main effect parameter(s) change over time. For the studies included in this review, this relates to whether the short-term effect of interventions on physical activity will be maintained over the defined follow-up period or not. Due to the lack of evidence on children’s BMI development after the end of the interventions, assumptions had to be made in all studies. The majority of the studies based the extrapolation of the effect on the assumption that the intervention-related BMI reduction would be maintained for the entire follow-up period. The only exceptions were Rush et al.¹⁸ that applied a decay rate of 1% after the first 5 years, and Ekwaru et al.²¹ that used three different scenarios to account for this uncertainty. A full maintenance of the effect is very unlikely in reality, given the brevity of the intervention and the modest size of the effect. Therefore, this assumption should (at least) be tested in a sensitivity analysis.

Furthermore, the model simulations can be made at an aggregate level (e.g. Markov cohort model), or an individual level that allows the individuals to be tracked separately (e.g. microsimulation model)²⁵. The majority of the included studies utilized the Markov (cohort) modelling approach to simulate BMI (or weight status changes), risk of obesity-related diseases and mortality in hypothetical cohorts of individuals receiving the interventions. The main limitation of cohort-level models is that they assume homogenous individuals in the cohort. This limitation can be avoided by individual-level modelling techniques that simulate one individual at a time and account for heterogeneity of individuals by tracking the past health states of individual and modelling individual’s risk of future events^25,26. We identified only one study²⁰ that applied an individual-based model - a microsimulation model. Their model projected the costs and effectiveness of the six interventions through their impact on BMI changes, obesity prevalence, and obesity-related health care costs over ten years.

Effects

None of the studies considered other than health effects of the interventions. As a result of increased level of physical activity, the interventions might provide other positive side-effects, such as the encouragement of social cohesion, improved cognitive function and academic performance. Furthermore, the interventions are likely to have positive spill-over effects by spreading to the wider school-community, parents and the local community. Considering these broader effects could potentially improve the ICER of the interventions.

Another methodological challenge was found in the two studies evaluating active transport interventions^14-16. The interventions had a number of different objectives (e.g. reduction of congestion, accidents and pollution) and provided several potential benefits. Due to this, it could be argued that the costs associated with the intervention should be apportioned across the intervention objectives instead of being fully attributed to just one of them. Additionally, some of the interventions considered in this review were complex and combined promotion of physical activity with healthy nutrition^17,18,20,21. In this type of combined interventions, it is not possible to attribute the effect to a specific intervention component which complicates comparisons of the studies.

Finally, the data on effectiveness included in the models were obtained from a number of different sources with a varying level of evidence, which potentially impact their validity. According to the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach²⁴, the evidence obtained from RCTs, followed by controlled natural or quasi experiments, and longitudinal studies should be prioritized. The level of evidence in the included studies was generally low (see Table 2). In one of the studies, there was no empirical evidence on effectiveness at all¹⁵.

Costs

One of the key aspects of the economic evaluations is the perspective, which has to be chosen carefully and made explicit, as it defines which cost categories are appropriate to include in the analysis. Ideally, economic evaluation should employ a societal perspective and include all costs and consequences associated with the intervention. In almost all included studies, the analysis was conducted from a societal perspective. The only exceptions was Rush et al.¹⁸ with a funder’s (health care budget) and Ekwaru et al.²¹ with a school system perspective. The perspective in Barrett et al.¹⁹ and Cradock et al.²⁰ was defined as a ‘modified societal perspective’ due to the exclusion of participant time costs. Surprisingly, other studies reporting social perspective did not mention this cost category. Even though there are disagreements on how the value of time that children spend participating in intervention activities should be determined^22-23, exclusion of this cost category should (at least) be made explicit in the studies. Furthermore, despite the societal perspective, only the direct healthcare costs were considered in all included studies. Indirect cost savings were only considered in Wang et al. and direct non-medical costs only in Cradock et al.²⁰

Comparison with other studies

A number of reviews have been conducted on the economic evidence on childhood obesity primary prevention programs^27-28. There are also reviews on transferability²⁹, and quality of economic evaluations of physical activity interventions in children³⁰. The present literature review contribute to the existing evidence with the overview of modelling approaches applied in economic evaluation of the school-based interventions encouraging physical activity. Assessment of quality of these models was beyond scope of this review, but represents an area for a potential future research. A guideline developed by ISPOR could potentially be applied for this purpose³¹.

Limitations of this review

One of the limitations of the review is the time frame for the search which was restricted to the last 10 years. This restriction was a pragmatic choice, which we believe does not affect the main findings of this review. During the last ten years, greater awareness of the health benefits of physical activity and greater interest in providing school-based interventions have been seen. Moreover, the modelling techniques for economic analyses have developed dramatically within the last decade, and the importance of recent, more complex approaches have been acknowledged²⁶.

This methodological review was based on the published articles and available supplementary materials. This implies that interpretations had to be based on the description of the model construction and assumptions as it appeared in the articles. Although most articles included detailed and relevant information, a detailed technical description of the modelling was in some cases not available. However, the aim of this review was to provide an overview and not a detailed technical critique of the modelling techniques. Additionally, the review was restricted to peer-reviewed articles published in the selected databases and therefore did not consider model-based analyses reported in other forms, e.g. reports. The report form allows more detailed information to be documented but reports may be more difficult to identify in a systematic way.

Conclusion

This review shows that only a small number of school-based interventions have been evaluated in terms of long-term cost-effectiveness. Without proper economic evaluation including a consideration of the costs and long-term effects, decisions to invest in physical activity programs may be based on faith and wrong perceptions of cost and effect. The risk of investment decisions being misguided is therefore high. Disregarding potentially important long-term effects might lead to inappropriate decisions of not investing in interventions that potentially could have saved resources and improved the duration and quality of life, while the lack of cost data may carry a risk of supporting the wrong types of intervention. A further research is therefore needed to cover the long-term benefits of the school-based interventions either in form of long-term observational studies or high-quality modelling studies.

Aknowledgements

This work was finanically supported by a unrestricted research grant provided by TrygFonden, Virum, Denmark.

References

• ¹
  Warburton DE, Nicol CW, Bredin SS. Health benefits of physical activity: the evidence. CMAJ 2006;174(6):801-809. Doi: 10.1503/cmaj.051351.
  » https://doi.org/10.1503/cmaj.051351
• ²
  Janssen I, Leblanc AG. Systematic review of the health benefits of physical activity and fitness in school-aged children and youth. Int J Behav Nutr Phys Act 2010;7:40. Doi: 10.1186/1479-5868-7-40.
  » https://doi.org/10.1186/1479-5868-7-40
• ³
  Telama R, Yang X, Viikari J, Valimaki I, Wanne O, Raitakari O. Physical activity from childhood to adulthood: a 21-year tracking study. Am J Prev Med 2005;28(3):267-273. Doi: 10.1016/j.amepre.2004.12.003.
  » https://doi.org/10.1016/j.amepre.2004.12.003
• ⁴
  Wang YC, Gortmaker SL, Sobol AM, Kuntz KM. Estimating the energy gap among US children: a counterfactual approach. Pediatrics 2006;118(6):e1721-e1733. Doi: 10.1542/peds.2006-0682.
  » https://doi.org/10.1542/peds.2006-0682
• ⁵
  Wang YC, Orleans CT, Gortmaker SL. Reaching the healthy people goals for reducing childhood obesity: closing the energy gap. Am J Prev Med 2012;42(5):437-444. Doi: 10.1016/j.amepre.2012.01.018.
  » https://doi.org/10.1016/j.amepre.2012.01.018
• ⁶
  Hall KD, Butte NF, Swinburn BA, Chow CC. Dynamics of childhood growth and obesity: development and validation of a quantitative mathematical model. Lancet Diabetes Endocrinol 2013;1(2):97-105.
• ⁷
  Global Recommendations on Physical Activity for Health. Geneva: World Health Organization 2010; 2010.
• ⁸
  Hallal PC, Andersen LB, Bull FC, Guthold R, Haskell W, Ekelund U. Global physical activity levels: surveillance progress, pitfalls, and prospects. Lancet 2012;380(9838):247-257. Doi: 10.1016/S0140-6736(12)60646-1.
  » https://doi.org/10.1016/S0140-6736(12)60646-1
• ⁹
  Committee on Physical Activity and Physical Education in the School Environment, Food and Nutrition Board, Institute of Medicine. Educating the Student Body: Taking Physical Activity and Physical Education to School. Washington DC; 2013. Doi: 10.17226/18314.
  » https://doi.org/10.17226/18314
• ¹⁰
  Kriemler S, Meyer U, Martin E, Sluijs EM, Andersen LB, Martin BW. Effect of school-based interventions on physical activity and fitness in children and adolescents: a review of reviews and systematic update. Br J Sports Med 2011;45(11):923-930. Doi: 10.1136/bjsports-2011-090186. Doi: 10.1136/bjsports-2011-090186.
  » https://doi.org/10.1136/bjsports-2011-090186
• ¹¹
  Waters E, Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, et al. Interventions for preventing obesity in children. Cochrane Database Syst Rev 2011;07(12):CD001871. Doi: 10.1002/14651858.CD001871.pub3.
  » https://doi.org/10.1002/14651858.CD001871.pub3
• ¹²
  Mandelblatt JS, Fryback DG, Weinstein MC, Russell LB, Gold MR, Hadorn DC. Assessing the effectiveness of health interventions. In: Gold JES, Russell LB, Weinstein MC, editor. Cost-effectiveness in health and medicine: New York: Oxford University Press; 1996, p. 135-75.
• ¹³
  Wolfenstetter SB. Conceptual framework for standard economic evaluation of physical activity programs in primary prevention. Prev Sci 2011;12(4):435-451. Doi: 10.1007/s11121-011-0235-4.
  » https://doi.org/10.1007/s11121-011-0235-4
• ¹⁴
  Moodie M, Haby M, Galvin L, Swinburn B, Carter R. Cost-effectiveness of active transport for primary school children - Walking School Bus program. Int J Behav Nutr Phys Act 2009;6:63. Doi: 10.1186/1479-5868-6-63.
  » https://doi.org/10.1186/1479-5868-6-63
• ¹⁵
  Moodie ML, Carter RC, Swinburn BA, Haby MM. The cost-effectiveness of Australia's Active After-School Communities program. Obesity 2010;18(8):1585-1592. Doi: 10.1038/oby.2009.401.
  » https://doi.org/10.1038/oby.2009.401
• ¹⁶
  Moodie M, Haby MM, Swinburn B, Carter R. Assessing cost-effectiveness in obesity: active transport program for primary school children--TravelSMART Schools Curriculum program. J Phys Act Health 2011;8(4):503-515.
• ¹⁷
  Wang LY, Nichols LP, Austin SB. The economic effect of Planet Health on preventing bulimia nervosa. Arch Pediatr Adolesc Med 2011;165(8):756-762. Doi: 10.1001/archpediatrics.2011.105.
  » https://doi.org/10.1001/archpediatrics.2011.105
• ¹⁸
  Rush E, Obolonkin V, McLennan S, Graham D, Harris JD, Mernagh P, et al. Lifetime cost effectiveness of a through-school nutrition and physical programme: Project Energize. Obes Res Clin Pract 2014;8(2):e115-e122. Doi: 10.1016/j.orcp.2013.03.005.
  » https://doi.org/10.1016/j.orcp.2013.03.005
• ¹⁹
  Barrett JL, Gortmaker SL, Long MW, Ward ZJ, Resch SC, Moodie ML, et al. Cost Effectiveness of an Elementary School Active Physical Education Policy. Am J Prev Med 2015;49(1):148-159. Doi: 10.1016/j.amepre.2015.02.005.
  » https://doi.org/10.1016/j.amepre.2015.02.005
• ²⁰
  Cradock AL, Barrett JL, Kenney EL, Giles CM, Ward ZJ, Long MW, et al. Using cost-effectiveness analysis to prioritize policy and programmatic approaches to physical activity promotion and obesity prevention in childhood. Prev Med 2017;95 Suppl:S17-S27. Doi: 10.1016/j.ypmed.2016.10.017.
  » https://doi.org/10.1016/j.ypmed.2016.10.017
• ²¹
  Ekwaru JP, Ohinmaa A, Tran BX, Setayeshgar S, Johnson JA, Veugelers PJ. Cost-effectiveness of a school-based health promotion program in Canada: A life-course modeling approach. PloS one 2017;12(5):e0177848. Doi: 10.1371/journal.pone.0177848.
  » https://doi.org/10.1371/journal.pone.0177848
• ²²
  Drummond MF, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 2nd. ed. New York: Oxford University; 1997.
• ²³
  Drummond MFMA. Economic Evaluation in Health Care: Merging Theory with Practice. Oxford: Oxford University Press; 2001.
• ²⁴
  Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. Grade guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64(4):401-406. Doi: 10.1016/j.jclinepi.2010.07.015
  » https://doi.org/10.1016/j.jclinepi.2010.07.015
• ²⁵
  Petrou S, Gray A. Economic evaluation using decision analytical modelling: design, conduct, analysis, and reporting. BMJ 2011;342:d1766. Doi: 10.1136/bmj.d1766.
  » https://doi.org/10.1136/bmj.d1766
• ²⁶
  Marsh K, Phillips CJ, Fordham R, Bertranou E, Hale J. Estimating cost-effectiveness in public health: a summary of modelling and valuation methods. Health Econ Rev 2012;2(1):17. Doi: 10.1186/2191-1991-2-17.
  » https://doi.org/10.1186/2191-1991-2-17
• ²⁷
  John J. Economic perspectives on pediatric obesity: impact on health care expenditures and cost-effectiveness of preventive interventions. Nestle Nutr Workshop Ser Pediatr Program 2010;66:111-24. Doi: 10.1159/000318952.
  » https://doi.org/10.1159/000318952
• ²⁸
  John J, Wolfenstetter SB, Wenig CM. An economic perspective on childhood obesity: recent findings on cost of illness and cost effectiveness of interventions. Nutrition 2012;28(9):829-839. Doi: 10.1016/j.nut.2011.11.016
  » https://doi.org/10.1016/j.nut.2011.11.016
• ²⁹
  Korber K. Potential transferability of economic evaluations of programs encouraging physical activity in children and adolescents across different countries--a systematic review of the literature. Int J Environ Res Public Health 2014;11(10):10606-10621. Doi: 10.3390/ijerph111010606
  » https://doi.org/10.3390/ijerph111010606
• ³⁰
  Korber K. Quality assessment of economic evaluations of health promotion programs for children and adolescents-a systematic review using the example of physical activity. Health Econ Rev 2015;5:35. Doi: 10.1186/s13561-015-0071-5
  » https://doi.org/10.1186/s13561-015-0071-5
• ³¹
  Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices--overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--1. Med Decis Making 2012;32(5):667-677. Doi: 10.1016/j.jval.2012.06.012.
  » https://doi.org/10.1016/j.jval.2012.06.012

Publication Dates

• Publication in this collection
  29 Apr 2019
• Date of issue
  2019

History

• Received
  24 Aug 2018
• Reviewed
  26 Dec 2018
• Accepted
  31 Jan 2019