Systematic review on the use and description of measures to evaluate psychotherapeutic interventions with children and adolescents in Brazil

Revisão sistemática sobre o uso e descrição de medidas para avaliar intervenções psicoterapêuticas com crianças e adolescentes no Brasil

Abstract

Introduction:

Psychotherapeutic interventions in childhood and adolescence are recognized as a mental treatment and as a tool to reduce psychological disorders in youth and adulthood. Therefore, there is an increasing concern about evidence of effectiveness of mental disorder interventions and adequacy of measurement. The objectives of this systematic review were to investigate predominant research designs and instruments used in Brazilian studies describing psychotherapeutic interventions with children and adolescents and to examine how these instruments are described regarding evidence of validity and reliability.

Method:

Five databases were surveyed, and for each one two independent judges performed the selection of records and applied the inclusion and exclusion criteria.

Results:

The final database comprised 28 papers, in which 92 instruments were referred. Fifty-seven instruments cited did not have descriptions of evidence of validity for the Brazilian population; for 31 instruments, validity evidence was mentioned, but the study did not detail which validity parameter was used; three studies described content validity evidence for their instruments. Furthermore, information about reliability was described for only two instruments.

Conclusion:

A lack of studies in the field of child and adolescent psychotherapy was found in Brazil. There is a significant need for the field to attend both the psychometric properties and the quality of description of research instruments. The scientific production of studies focused on the evaluation of psychotherapeutic interventions may promote evidence-based psychotherapy and justify the offer of mental treatment in different contexts.

Keywords:
Evaluation; psychometrics; psychotherapy; childhood; adolescence; systematic review

Resumo

Introdução:

Intervenções psicoterápicas na infância e adolescência são reconhecidas como tratamento mental e ferramenta de redução de transtornos psicológicos na juventude e adultez. Assim, evidencia-se uma crescente preocupação com a comprovação científica acerca da efetividade de intervenções oferecidas no campo da saúde mental e com a adequação das medidas utilizadas. Os objetivos desta revisão sistemática foram investigar delineamentos de pesquisa predominantes e instrumentos utilizados em estudos brasileiros com intervenções psicoterápicas com crianças e adolescentes e examinar como esses instrumentos são descritos acerca de suas evidências de validade e confiabilidade.

Método:

Cinco bases de dados foram consultadas; para cada base, dois juízes independentes realizaram a seleção dos registros e aplicaram os critérios de exclusão e inclusão.

Resultados:

O banco final compreendeu 28 artigos, nos quais foram citados 92 instrumentos para avaliação das intervenções psicoterápicas realizadas. Dentre estes, para 57 instrumentos citados não foram descritas evidências de validade da medida para a população brasileira. Para 31 dos instrumentos citados, foi mencionada a existência de evidências de validade, sem especificar o parâmetro de validade. Três estudos descreveram evidências de validade de conteúdo dos instrumentos empregados, enquanto dois instrumentos são apresentados com informações acerca de evidências de confiabilidade.

Conclusão:

Constatou-se uma carência de estudos na área da psicoterapia com crianças e adolescentes no contexto brasileiro e a necessidade de maior atenção às propriedades psicométricas e qualidade da descrição dos instrumentos. A produção científica neste campo pode impulsionar a psicoterapia baseada em evidências e justificar a oferta desse tipo de tratamento em diferentes contextos.

Descritores:
Avaliação; psicometria; psicoterapia; infância; adolescência; revisão sistemática

Introduction

Most psychological disorders that are typically discovered when maturity is reached have an early beginning in childhood or adolescence.¹ Mental disorders are prevalent in up to 20% of children and adolescents worldwide, and 50% of mental disorders affecting adults begin during childhood.² The most commonly reported childhood psychiatric disorders - oppositional defiant disorder, attention deficit/hyperactivity disorder (ADHD), anxiety and depression - are associated with psychosocial damage such as school dropout, psychoactive substance use, risky behavior and other dysfunctions in adult life. Thus, the issue is not only individual but also social.² Despite the impact of childhood psychiatric disorders on the individuals’ overall health and their lifetime effects, psychiatric interventions in adolescence and childhood are often neglected, especially in underdeveloped countries.^2,3

According to the World Health Organization, detecting and treating mental disorders in children and adolescents decreases the likelihood of long-term illness and reduces individual, family, community and health system stress.⁴ Therefore, it is relevant to highlight the importance of investing governmental funds in research into these early development stages with the aim of reducing the duration of untreated mental illnesses in childhood and adolescence, which may last for the entire adult life.² Generally, this type of government investment ends up being targeted toward treatments that provide evidence of results. Therefore, interest in evaluating the effectiveness of available therapeutic models and psychotherapeutic interventions has increased and has given strength to evidence-based therapies. This is especially relevant considering the growing prevalence of behavioral therapies and the need for evidence in support of recognized therapeutic models.⁵

A factor that has a major impact on these types of outcome-focused studies is the adoption of appropriate measurements, given that instrument selection can help to assure the trustworthiness and consistency of studies. In terms of clinical recognition, the degree of applicability and social utility that a study might achieve is based upon the adequacy of the measure to the researcher's objectives and design.⁶ It is not uncommon for studies to fail to report information regarding the validity, reliability and responsiveness of the measures employed. Some instruments are often used and assumed to be appropriate even though they may be of poor quality.⁷ Failure to properly describe the instruments used not only hinders the possibility of replicating the study but also weakens the credibility of the results.

In countries in which English is not the native language, such as Brazil, cases of instrument adaptation and translation are more frequent than the development and creation of original measurements.⁸ The choice to adapt an existing instrument instead of developing a new one may be common in Brazil due to the possibility of comparing data in different samples and contexts, thereby allowing greater equity in the evaluation.⁹ For this reason, it is important to respect the standardization steps in the instrument adaptation process^10-12 to ensure the methodological quality that characterizes evidence-based research. The Brazilian Federal Council of Psychology, the institution that regulates the use, elaboration and commercialization of psychological tests, considers the presentation of empirical evidence of validity and precision a mandatory requirement for instruments such as scales and inventories. The Federal Council of Psychology also states that foreign tests of any nature that are translated into Portuguese must be adjusted based on studies reporting evidence of validity, precision and normative data in a Brazilian sample before they can be applied in clinical settings in Brazil.¹³

In considering evidence-based psychotherapy in Brazil, one of the most remarkable aspects is the lack of psychotherapy evaluation studies and the great variety of psychotherapeutic techniques and models that lack scientific validation but are still being offered in the country.¹⁴ To change this situation, it is essential to clarify how the results of psychotherapeutic interventions are measured and how the effectiveness and efficacy of treatments are identified among different psychopathologies.^14,15 However, it is unclear how psychologists use knowledge garnered by research in the field to make decisions regarding the selection of an evidence-based treatment, especially given the complexity of scientific literature, which may be difficult to understand for professionals outside the academy.¹⁶ This is even more true in Brazil, where there is little debate about evidence-based practice and a dearth of literature on the subject in Portuguese.¹⁷ Therefore, studies should seek to systematize the body of evidence related to psychotherapeutic processes and assessments in Brazil to qualify the interventions employed in the country and to make this body of knowledge available to clinical professionals

The objective of this study was to evaluate how Brazilian studies that researched psychotherapeutic interventions with children and adolescents based on different theoretical approaches have evaluated and measured their own results. For this purpose, we described the study designs as well as the different types of psychotherapeutic interventions and theoretical approaches adopted. In addition, we sought to investigate which instruments were used to assess interventions. Some aspects of the instruments employed were also identified (e.g., evaluation method, construct to be measured, respondent). In addition, we conducted a critical analysis of how the instruments, which are the core of effective studies, are described in the research. More specifically, we examined whether the researchers provided information (particularly regarding validity) about the instruments employed, thereby supporting their choice of measures. The purpose is not to find out whether the instrument has already been the object of previous validation studies in Brazil, but rather to determine whether studies in this field seek and describe scientific support, especially regarding psychometric properties, for the selection of the instruments.

Given this objective, the method chosen was a systematic review. The goal was to gather, recognize and synthesize different studies to gauge whether there are gaps in knowledge, inform proposals for improvement¹⁸ and provide a critical and reflexive analysis of the research results that goes beyond summarizing the data found.^19,20 This method could help to identify how studies of psychotherapeutic interventions with children and adolescents are designed and how interventions themselves are evaluated and measured. Through this systematic review, we sought to emphasize possible gaps and opportunities for development regarding methods of evaluation in psychotherapeutic interventions with children and adolescents. Hence, the main objective of this study was to generate information to strengthen evidence-based therapy in Brazil, thereby improving clinical intervention within this age group in the country.

Method

To conduct this systematic review of instruments used in Brazilian studies where psychotherapeutic interventions with children and adolescents were developed, we followed the steps described by the checklist Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA).²¹ All studies included in the final database were led by at least one Brazilian researcher, and the samples in all the papers comprised only Brazilian participants. Three resources were used for this search. First, the following databases were used to search for articles: 1) PubMed; 2) Scientific Electronic Library Online (SciELO); 3) Biblioteca Virtual em Saúde (BVS; affiliated with the Brazilian Ministry of Health); 4) Literatura Latino-Americana e do Caribe em Ciências da Saúde (LILACS); and 5) Periódicos Eletrónicos de Psicologia (PePSIC) - Latin America. The choice of keywords was based on Science Health Descriptors (Descritores em Ciências da Saúde - DeCS). Additionally, keywords not catalogued but recurrently used by psychologists and psychotherapists were added to maximize the search potential. The second resource used was an interlibrary loan system, to obtain the complete copies of papers. Third, paper authors and journal editors were contacted when the previously listed retrieval methods were unsuccessful.

Four judges conducted the searches. Two of the authors of this study reviewed the PubMed, SciELO and BVS databases, and the other two examined the LILACS and PePSIC databases. The authors conducted the searches individually, applying the inclusion and exclusion criteria and then comparing the results with their working partner. In cases where their opinions diverged regarding the inclusion or exclusion of a study, a third judge who was not working in that database assessed the case to make the final decision. The search was performed in September 2017, with no date limitations in the databases. The following inclusion criteria were applied: 1) research that described group, individual, brief or single psychotherapeutic interventions with children and adolescents based on any theoretical approach; 2) articles in English, Spanish or Portuguese; 3) research that included a Brazilian sample and/or was led by at least one Brazilian researcher; and 4) research that used instruments and assessments to evaluate the interventions performed. After applying the inclusion criteria, the following exclusion criteria were observed: 1) papers duplicated across databases; and 2) studies for which we could not retrieve the full text or abstracts (i.e., when the editor or author did not return contact or when the interlibrary loan system could not provide the required material).

In all databases, we used Portuguese and English descriptors and strings. The model string for all surveys was: "(youth OR adolescent OR child) AND (instrument OR scale OR test OR inventory OR assessment OR survey OR questionnaire OR measure) AND (psychotherapy)" in English; the same string in Portuguese was also employed. Since the databases have different search logics, the descriptor string was adapted for each database, although coherence and homogeneity were maintained among all strings. To summarize, in PubMed, we used the advanced search, inserting the complete string in the open field "Advanced Search Builder." Considering that this database has worldwide reach, the descriptor "Brazil OR Brasil" was added to the string in hopes of refining the results. The SciELO database has two search options: free entry field, which requires the selection of just one logical operator (AND or OR) and therefore does not support the use of the model string, and advanced search, which has three separate fields for the search. Thus, in the first field, we entered the descriptor "youth OR adolescent OR child," in the second "instrument OR scale OR test OR inventory OR assessment OR survey OR questionnaire OR measure," and in the third we entered the descriptor "psychotherapy." The BVS database, in contrast, provides only separate fields for the search, and each descriptor within the complete string must be added separately. Finally, Lilacs allows the complete string to be entered into the search field, whereas PePSIC does not return results when the complete string is used in the free entry field. Therefore, to amplify the search potential, we selected the broader descriptors from the string and applied them in the basic form, which contains only three fields. In the first, we entered the descriptor child, in the second adolescent, and in the third psychotherapy.

With regard to the term psychotherapeutic intervention, the present review used Kazdin's²² conceptual perspective, which describes it as having a focus on improving the individual's adaptation and helping to reduce harmful behavior, psychological symptoms or complaints through psychological processes such as social support, interpersonal influence, and learning. The author's concept ranges from classic psychotherapeutic intervention, such as individual or group therapy derived from largely widespread approaches such as psychoanalysis and cognitive behavioral therapy, to non-traditional interventions with theoretical and scientific bases, such as boot camps where children are responsible for an animal or activity. Retrospective studies were not considered.

Descriptive statistical techniques were performed using the Statistical Package for the Social Sciences version 19.0 to identify frequencies within the studies’ designs, types of psychotherapeutic intervention, theoretical approaches and instrument evaluation method (whether projective or psychometric) of the psychotherapeutic intervention.

Results

The first search yielded 1,179 records (SciELO [28], BVS [586], Lilacs [281], PePSIC [149], PubMed [135]). After applying the inclusion criteria, only 62 records remained in the database (SciELO [7], BVS [5], Lilacs [22], PePSIC [15], PubMed [13]). The exclusion criteria removed a further 34 articles from the selection, leaving only 28 records remaining in the final analysis (SciELO [5], BVS [2], Lilacs [7], PePSIC [7], PubMed [7]). Among the 28 studies that comprised the final database, only two required assistance of the third judge to determine their inclusion. The process is shown in Figure 1, which illustrates the records found, denoting results obtained using the Portuguese and English strings.

Figure 1
Search flow.

Of the 28 articles included in the final database, 60.7% (n = 17) were written in Portuguese and 7.1% (n = 2) in Portuguese and English; the remaining articles were written in English. Designs were divided into quantitative (n = 15, 53.5%) and qualitative (n = 12, 42.8%), and one article used a mixed method. Only 28.5% were published in the past 5 years (2012 to 2016; papers issued in 2017 were not found). The year with the highest number of publications (n=4) was 2015. The first article found was published in 1991. In the final sample, studies adopting the psychoanalytic (n = 10) and cognitive-behavioral (n=7) approaches predominated. Five studies sampled more than 50 participants, and only one sampled more than 100 individuals; eight articles were single case studies. The final outcome showed that 53.5% (n=15) of the studies were performed with children, 32.1% (n=9) with adolescents and 14.2% (n=4) with both children and adolescents. Considering the evaluation methods of the psychotherapeutic interventions performed, 60.7% (n=17) used psychometric instruments, 25% (n=7) projective instruments, and 14.2% (n=4) used both psychometric and projective instruments. Furthermore, 64.2% (n=18) of the studies used instruments that were answered only by the patients, disregarding the perspectives of the school, parents and psychotherapists. Thirteen studies (46.4%) assessed individual interventions, and twelve focused on group therapy (42.8%). One study developed both individual and group therapy, one focused on family therapy, and one developed a behavioral intervention for enuresis using a urine alarm.

Half of the studies adopted measures specific to the pathology observed or provided as the reason for the consultation, such as depression, anxiety, intelligence, sexual abuse or eating disorder scales. Regarding the overall assessment of patient functioning (behavior, feelings, patient experience) or treatment outcome or progress, studies employed the following instruments to measure intervention effectiveness: five studies used The Child Behavior Checklist (CBCL),²³ three the Children's Global Assessment Scale (CGAS),²⁴ and three used the 36-Item Short Form Health Survey (SF-36).²⁵ Two studies employed the Clinical Global Impression (CGI),²⁶ and another two employed the Child Psychotherapy Q-Set (CPQ).²⁷ Other instruments were employed only once among the identified articles, namely: Questionário Parental de Sintomas (Parental Symptom Questionnaire; without pointed reference), Profile of Mood States (POMS),²⁸ Development and Well-Being Assessment,²⁹ and Youth Quality of Life Instrument-Research Version (YQOL-R).³⁰

The quality of the description of the instruments used in the studies was also analyzed, or, more specifically, whether the validity or reliability of evidence within the Brazilian population was mentioned, taking into account that all studies had a Brazilian sample. Of the 92 instruments referenced in the 28 articles, 57 (61.9%) did not have descriptions about evidence of validity within the Brazilian population; for 31 instruments (33.6%), validity evidence was mentioned, but the study did not detail which validity parameter was used. Only three studies described content validity evidence for the instruments used. Furthermore, information about reliability was described for only two instruments. All results are shown in Table 1.

Table 1
Instrument description regarding evidence of validity and reliability

Author	Year of publication	Age group	Psychopathological frameworks	Instruments/Assessments	Evidence of validity	Evidence of reliability
Golfeto31	1991	Children	Psychosomatic disorder	Sandplay	–	–
Jordy &	1996	Children	ADHD	Wechsler Intelligence Scale Test for Children	–	–
Gorodscy32				Trinca Story-Drawing Test	–	–
				Concentrated Attention Test of the CEPA Battery	–	–
				Bender Gestalt Test	–	–
D'Affonseca &	2003	Children	Without specific	The Center for Epidemiological Studies Depression Scale for	–	–
Williams33			pathology/multiple	Children (CES-DC)	–	–
			complaints	Pies-Harris Self-Esteem Scale	–	–
				Child Behavior Checklist (CBCL)	EV	–
Asbahr et al.34	2005	Children &	OCD	Children's Yale-Brown Obsessive-Compulsive Scale	–	–
		adolescents		NIMH Global Obsessive Compulsive Scale	–	–
				The Clinical Global Impression	–	–
				The Children's Global Assessment Scale	–	–
				Children's Depression Inventory	–	–
				Multidimensional Anxiety Scale for Children	–	–
				Schedule for Affective Disorders and Schizophrenia	–	–
				Yale Global Tic Severity Scale	–	–
Sauaia &	2005	Children &	Sexual abuse/	The Human Figure Drawing Test	–	–
Araújo35		adolescents	mistreatment	Sandplay	–	–
Carnier et al36	2008	Adolescents	Eating disorder	Binge Eating Scale (BES)	EV	–
				Bulimic Investigatory Test of Edinburgh (BITE)	EV	–
Coelho37	2008	Children	Encopresis	Questionário de Atividade de Vida Diária/Daily Activity Questionnaire	–	–
				Questionário sobre a Encoprese/Encopresis Questionnaire	–	–
Deakin & Link38	2008	Children	Without specific pathology/multiple complaints	The Rorschach Test	–	–
Fernandes et	2008	Adolescents	Depression	Children's Depression Rating Scale	–	–
al.39				Child Behavior Checklist (CBCL) - Youth Self-Report	–	–
				Escala de Autoconceito Infanto-Juvenil (Child and Youth Self-Concept Scale; EAC-IJ)	–	–
				Child Behavior Checklist (CBCL)	–	–
Habigzang et	2008	Children	Sexual abuse/	Children's Attributions and Perceptions Scale (CAPS)	CV	–
al.40			mistreatment	Children's Depression Inventory (CDI)	EV	Paper describes internal consistency evidence and correlation with other measures
				Escala de Estresse Infantil (Child Stress Scale; ESI)	CV	–
				The State-Trait Anxiety Inventory for Children (STAI-CH)	EV	–
Deakin &Nunes41	2009	Children	Without specific pathology/multiple complaints	Bender Gestalt Test	–	–
				Wechsler Intelligence Scale Test for Children	EV	–
				The Rorschach Test	–	–
				Child Behavior Checklist (CBCL)	–	–
Habigzang et al.42	2009	Children & adolescents	Borderline disorder	Children's Attributions and Perceptions Scale (CAPS)	CV	–
				Children's Depression Inventory (CDI)	EV	Paper describes internal consistency evidence
				Escala de Estresse Infantil (Child Stress Scale; ESI)	EV	–
				The State-Trait Anxiety Inventory for Children (STAI-CH)	–	–
Lofrano-Prado et al.43	2009	Adolescents	Eating disorder	State-Trait Anxiety Inventory SF-36	EV EV	– –
				Beck Depression Inventory (BDI)	EV	–
				Body Shape Questionnaire (BSQ)	EV	–
				Binge Eating Scale (BES)	EV	–
Fonseca & Pacheco44	2010	Children	Without specific pathology/multiple complaints	Raven's Progressive Matrices Test Sequências Lógicas/Logical Sequences	– –	– –
				Tarefas de Habilidades Aritméticas/Arithmetic Skills Tasks	–	–
				Questionário Parental de Sintomas/Parental Symptom Questionnaire	–	–
Gomide45	2010	Adolescents	Aggressive behavior	Inventário de Estilos Parentais/Parenting Styles Inventory	–	–
Telles et al.46	2010	Children	Speech development disorders	HTP	–	–
Turkiewicz et al.47	2010	Adolescents	Eating disorder	Eating Disorder Examination-Questionnaire The Children's Global Assessment Scale	– –	– –
				Development and Well-being Assessment	–	–
Bolsson & Benetti48	2011	Children	Anguish and phobias	Fables Test	CV	–
Ramires & Godinho49	2011	Children	Without specific pathology/multiple complaints Manchester Child Attachment Story Task (MCAST)	Fables Test	– –	– –
Steibel et al.50	2011	Children	Without specific pathology/multiple complaints	Child Behavior Checklist (CBCL) Bender Gestalt Test The Human Figure Drawing Test Fables Test Ficha de Informação sobre a Criança/Child Information Record	– – – – –	– – – – –
Silvares & Pereira51	2012	Children & adolescents	Enuresis	Child Behavior Checklist (CBCL)	–	–
De Souza etal.52	2013	Adolescents	Anxiety	The Clinical Global Impression The Pediatric Anxiety Rating Scale The Screen for Child Anxiety Related Emotional Disorders (SCARED) The Children's Global Assessment Scale Children's Depression Inventory (CDI) Youth Quality of Life Instrument-Research Version (YQOL-R) SNAP-IV	– – – – – – –	– – – – – – –
Sbardelotto & Donelli53	2014	Children	Without specific pathology/multiple complaints	HTP Fables Test	– –	– –
Corgosinho etal.54	2015	Adolescents	Sleep disorder	Binge Eating Scale (BES)	EV	–
				Bulimic Investigatory Test of Edinburgh (BITE)	EV	–
				Body Shape Questionnaire (BSQ)	EV	–
				Beck Depression Inventory (BDI)	EV	–
				Profile of Mood States (POMS)	EV	–
				SF-36	EV	–
Fidelix et al.55	2015	Adolescents	Eating disorder	Eating Attitudes Test (EAT-26)	EV	–
				Bulimic Investigatory Test of Edinburgh (BITE)	EV	–
				Binge Eating Scale (BES)	EV	–
				The State-Trait Anxiety Inventory for Children (STAI-CH)	EV	–
				Beck Depression Inventory (BDI)	EV	–
				Body Shape Questionnaire (BSQ)	EV	–
				SF-36	EV	–
Gastaud etal.56	2015	Children	Without specific pathology/multiple complaints	Child Psychotherapy Q-Set (CPQ) The Rorschach Test	– –	– –
Masquio etal.57	2015	Adolescents	Eating disorder	Beck Depression Inventory (BDI)	EV	–
				State-Trait Anxiety Inventory	EV	–
				Body Shape Questionnaire (BSQ)	EV	–
				Binge Eating Scale (BES)	EV	–
				Bulimic Investigatory Test of Edinburgh (BITE)	EV	–
Carvalho et al.58	2016	Children	Without specific pathology/multiple complaints	Child Psychotherapy Q-Set (CPQ)	–	–

• - = evidence of validity or reliability was not described for the Brazilian population; ADHD = attention deficit/hyperactivity disorder; CV = content validity; EV = evidence of validity was mentioned, but studies did not detail which validity parameter was used; HTP = The House-Tree-Person Test; OCD = obsessive-compulsive disorder; SF-36 = 36-Item Short Form Health Survey; SNAP-IV = Swanson, Nolan and Pelham Questionnaire.

Discussion and considerations

Studies that seek to evaluate the results of psychotherapeutic interventions with children and adolescents are extremely relevant for indicating the possibilities, benefits and limits of each modality of intervention for different pathologies or psychological structures. That type of study, which aims to evaluate results in the health field, should seek the most accurate, reliable and cautious way to measure the phenomenon observed. The decision regarding the preferred treatment for a particular pathology should be based on the most accurate data available. Therefore, information on the validity of the instruments used for this purpose is extremely important. Considering that this systematic review was performed in five databases in September 2017 without time limitations, the outcome - that only 28 studies focusing on psychotherapeutic intervention with children and adolescents in Brazil used some type of instrument or assessment to measure their own result - indicates a significant lack of research into mental health in this population. These findings reinforce the fact that the field of quantitative measurement of psychotherapy effects remains insufficiently explored (and perhaps inadequately valued) by Brazilian researchers.⁵⁹ Instruments are an essential tool for research designs that aim for external validity and generally applicable results.⁶⁰ Qualitative research designs, which are widely employed in the psychotherapy research field, do not contribute to the generalization of the results of interventions or evaluations performed.

The scarcity of research on psychotherapeutic interventions can also obscure the identification of variables that promote positive and negative treatment outcomes and that may contribute substantially to the development of evidence-based and effective practices and techniques.⁶¹ Moreover, this gap restricts the offer of psychological interventions in places where only treatments that show evidence of effectiveness are available; for example, the difficulty in demonstrating the efficacy of psychotherapy limits its adaptation into public health systems. This is one of the reasons why health psychology demands suitable measurements of psychological phenomena.⁶² Socially vulnerable adults and children are at risk of developing mental disorders due to financial difficulty, inappropriate housing, dysfunctional relationships and social prejudice. This is why it is so important that they have access to mental health services that offer evidence-based interventions.⁶³ For this reason, the development and employment of instruments that maximize the validity potential of research results can expand the impact of evidence-based psychotherapy and help to open new paths and fronts for law projects, public policies and/or investment of public resources in services and treatments focused on mental health.

It is worth noting that half of the studies reviewed did not use specific evaluations of the process or outcome in psychotherapy or global assessments of patient functioning; they focused on the evaluation circumscribed to the specific symptom or complaint that motivated the search for psychotherapy. One explanation for this may be the limited instruments available to the Brazilian population that evaluate both processes and outcomes in psychotherapy with evidence of validity and that are aimed at youth groups. Another explanation is the lack of research investment in the field of psychotherapy for children and adolescents, as indicated by the small number of studies found. However, it is important to note that psychotherapy results that evaluate only the relief of specific symptoms do not take into account other variables related to psychotherapy, such as comorbidities or contextual variables. The psychotherapeutic process is an extremely complex phenomenon with innumerable interacting variables, including patient features (social and economic lifestyle, comorbidities, set of varied psychopathological frameworks), therapist profile (personally and professionally), and institutional characteristics of where treatment takes place (quality of services and resources, professional support).⁶⁴ Thus, instruments used in psychotherapy research must take into account - and measure - not only symptom relief but also overall improvement of patient functioning.⁶⁵

Another relevant issue is the lack of characterization and description of the instruments used in the studies: few describe evidence of validity or reliability of its measures. Evidence of validity for use with certain populations provides assurance that the instrument is capable of measuring what it is intended to measure and has adequate psychometric properties. The process of adapting an instrument to another culture is not only limited to the translation process but should follow a rigorous method that guarantees the validity of the instrument content and respects the cultural richness of each environment.⁶⁶ Although publication guidelines may impose a reduction in the size of manuscripts, thereby compressing the details on instruments used, information on the measures that support the conclusion of the study is essential for analyzing the quality of the process conducted by researchers.

Given the fact that 64.2% of the 28 studies only assessed patient perception of the psychotherapeutic process, it is necessary to discuss the limitations of selfreport measures and the restrictions of the participants’ self-assessment about his/her own behaviors, feelings and thoughts. There is no absolute correspondence between what is reported and what is actually experienced. In this way, using more than one instrument - or more than one evaluated perspective - reduces self-reporting biases and allows the triangulation of data, establishing a more thorough method for psychological evaluations.⁶⁷ Triangulating data among measurements that assess the perspective of the patient, close relatives, teachers, patient's friends and the therapist could make research designs substantially more robust.⁶⁴ The wide use of instruments answered only by patients is also a limiting factor when the participants are pre-verbal or nonliterate children. Because self-assessment through psychometric measures is very difficult or impossible in these cases, these patients need the evaluation of an external observer who can help to check the treatment progress. Considering that the evaluation of treatment effectiveness in pre-verbal children depends almost exclusively on their guardians’ or therapist's assessment, we found a surprising dearth of studies that use other perspectives to evaluate treatment outcomes.

Limitations

The objective of the present review was to offer an overview of how psychotherapeutic intervention studies on children and adolescents have been methodologically developed with a particular emphasis on the measures used and their description by researchers. The limitations of this study are due to language and the number of databases involved; both limitations may have excluded papers that would have fit this systematic review's inclusion criterion. As a suggestion, other studies should widen their searches to other countries to identify whether the lack of data on child and adolescent psychotherapy is only a Brazilian issue or if it is part of a larger dilemma. Another limitation of this review is that it does not cover whether the studies that were described as providing validity and reliability evidence for their instruments (and those that were not) actually used valid and reliable instruments for the Brazilian context and target population. Although this was not the objective of the present review and would exceed our initial scope, we suggest future reviews on this topic.

Implications

There is an ample scope for the development of studies in this area, as demonstrated by the limited number of studies in the field of child and adolescent psychotherapy that use instruments that provide evidence of validity to evaluate results and that take place in Brazil. The American Psychological Association has encouraged the development of standardized measures to help therapists evaluate the diagnostic process and therapeutic progress.⁶⁸ To qualify mental health interventions and increase their applicability to different populations, we suggest that research should focus on the development of instruments to assess these interventions and that the benefits of such research should be shared beyond academic boundaries.

Acknowledgements

Coordenação de Aperfeiçoamento de Pessoal de Ensino Superior/Suporte à Pós-Graduação de Instituições de Ensino Particulares (CAPES/PROSUP).

References

• ¹
  Patel V, Flisher AJ, Hetrik S, McGorry P. Mental health of young people: a global public-health challenge. Lancet. 2007;9569:1302-13.
• ²
  Klasen H, Crombag AC. What works where? A systematic review of child and adolescent mental health interventions for low and middle income countries. Soc Psychiatry Psychiatr Epidemiol. 2013;48:595-11.
• ³
  Kieling C, Baker-Henningham H, Belfer M, Conti G, Ertem I, Omigbodun O, et al. Child and adolescent mental health worldwide: evidence for action. Lancet. 2011;378:1515-25.
• ⁴
  World Health Organization. Mental health policy and service guidance package: Child and adolescent mental health policies and plans. Geneva: WHO; 2005.
• ⁵
  Pheula GF, Isolan LR. Psicoterapia baseada em evidências em crianças e adolescentes. Rev Psiquiatr Clin. 2007;34:74-83.
• ⁶
  Coster WJ. Making the best match: selecting outcome measures for clinical trials and outcome studies. Am J Occup Ther. 2013;67:162-70.
• ⁷
  Gadotti, IC, Vieira, ER, Magee, DJ. Importance and clarification of measurement properties in rehabilitation. Rev Bras Fisioter. 2006;10:137-46.
• ⁸
  Gudmundsson E. Guidelines for translating and adapting psychological instruments. Nord Psychol. 2009;61:29-45.
• ⁹
  Hambleton RK. Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In: Hambleton RK, Merenda PF, Spielberger CD, editors. Adapting educational and psychological tests for cross-cultural assessment. New Jersey: Lawrence Erlbaum; 2005. p. 3-38.
• ¹⁰
  Borsa JC, Damásio BF, Bandeira DR. Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paideia. 2012;22:423-32.
• ¹¹
  Gjersing L, Caplehorn JRM, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical. BMC Med Res Methodol. 2010;10:13.
• ¹²
  Pasquali L. Testes referentes a construto: teoria e modelo de construção. In: Pasquali L. editor. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010. p. 165-98.
• ¹³
  Brasil, Conselho Federal de Psicologia. Resolução CFP n° 002/2003. Brasília: CFP; 2003. https://site.cfp.org.br/wp-content/uploads/2012/05/resoluxo022003.pdf
  » https://site.cfp.org.br/wp-content/uploads/2012/05/resoluxo022003.pdf
• ¹⁴
  Baptista MN. Questões sobre avaliação de processos psicoterápicos. Psicol Pesq. 2010;4:109-17.
• ¹⁵
  Chaves G, Lopes SMP, Perez GH, Watanabe DM, Romano BW. Instrumentos de avaliação de resultados em psicoterapia individual com pacientes adultos acometidos por doenças orgânicas: uma revisão sistemática. Rev SBPH. 2016;19:57-75.
• ¹⁶
  Chorpita BF, Daleiden EL, Ebesutani C, Young J, Becker KD, Nakamura BJ, et al. Evidence-based treatments for children and adolescents: an updated review of indicators of efficacy and effectiveness. Clin Psychol Sci Prac. 2011;18:154-72.
• ¹⁷
  Leonardi JL, Meyer SB. Prática baseada em evidências em psicologia e a história da busca pelas provas empíricas da eficácia das psicoterapias. Psicol Cienc Prof. 2015;35:1-18.
• ¹⁸
  De-la-Torre-Ugarte MC, Takahashi RF, Bertolozzi MR. Revisão sistemática: noções gerais. Rev Esc Enferm USP. 2011;45:1260-66.
• ¹⁹
  Sampaio RF, Mancini MC. Estudos de revisão sistemática: um guia para síntese criteriosa da evidência científica. Braz J Phys Ther. 2007;11:83-9.
• ²⁰
  Costa AB, Zoltowski APC, Koller SH, Teixeira MAP. Construção de uma escala para avaliar a qualidade metodológica de revisões sistemáticas. Cien Saude Colet. 2015;20:2441-52.
• ²¹
  Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. BMJ. 2009;6:339-66.
• ²²
  Kazdin, AE. Understanding change: from description to explanation in child and adolescent psychotherapy research. J Sch Psychol. 2000;38:337-48.
• ²³
  Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington: University of Vermont Research Center for Children, Youth, & Families; 2001.
• ²⁴
  Shaffer D, Gould MS, Brasic J, Ambrosini P, Fisher P, Bird H, et al. A Children's Global Assessment Scale (CGAS). Arch Gen Psychiatry. 1983;40:1228-31.
• ²⁵
  Ciconelli RM, Ferraz MB, Santos W, Meinão I, Quaresma MR. Tradução para a língua portuguesa e validação do questionário genérico de avaliação de qualidade de vida SF-36 (Brasil SF-36). Rev Bras Reumatol Engl Ed. 1999;39:143-50.
• ²⁶
  Guy W. ECDEU Assessment manual for psychopharmacology, Revised. Rockville: National Institute of Mental Health; 1976.
• ²⁷
  Schneider C. The development of the Child Psychotherapy Q-Set [dissertation]. Berkeley: University of California; 2004.
• ²⁸
  McNair DM, Loor M, Droppleman LF. Manual for the Profile of Mood States. San Diego: Educational and Industrial Testing Service (EdITS); 1971.
• ²⁹
  Goodman R, Ford T, Richards H, Gatward R, Meltzer H. The development and well-being assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry. 2000;41:645-55.
• ³⁰
  Edwards TC, Huebner CE, Connell FA, Patrick DL. Adolescent quality of life, Part I: conceptual and measurement model. J Adolesc. 2002;25:275-86.
• ³¹
  Golfeto JH. A psicossomática de um caso clínico infantil. Rev ABPAPAL. 1991;13:68-74.
• ³²
  Jordy CF, Gorodscy RC. The hyperactive child and the body: a clinical study on the origin of hyperactivity in children. Arq Neuropsiquiatr. 1996;54:628-36.
• ³³
  D'Affonseca SM, Williams LCA. Clubinho: intervenção psicoterapêutica com crianças vítimas de violência física intrafamiliar. Temas Desenvolv. 2003;67:33-43.
• ³⁴
  Asbahr FR, Castillo AR, Ito LM, Latorre MR, Moreira MN, Lotufo-Neto F. Group cognitive-behavioral therapy versus sertraline for the treatment of children and adolescents with obsessive-compulsive disorder. J Am Acad Child Adolesc Psychiatry. 2005;44:1128-36.
• ³⁵
  Sauaia NML, Araújo CA. Psicoterapia de orientação junguiana com foco corporal para grupos de crianças vítimas de violência: avaliando habilidades da resiliência. Temas Desenvolv. 2005;14:30-8.
• ³⁶
  Carnier J, Lofrano MC, Prado WL, Caranti DA, de Piano A, Tock L, et al. Hormonal alteration in obese adolescents with eating disorder: effects of multidisciplinary therapy. Horm Res Paediatr. 2008;70:79-84.
• ³⁷
  Coelho LSG. Vicarious learning in toilet training through an animated film: case study of behavioral play therapy. Psicol Cienc Prof. 2008;28:846-61.
• ³⁸
  Deakin EK, Graeff RL. O teste do Rorschach na avaliação dos resultados da psicoterapia com crianças. Publ CEAPIA. 2008;17:74-84.
• ³⁹
  Fernandes LFB, Silveira LR, Miyazaki MC, Domingos NAM, Luiz AMAG, Micheletto MRD. Eventos aversivos e depressão na adolescência: relato de caso. Rev Bras Ter Cogn. 2008;4:65-72.
• ⁴⁰
  Habigzang LF, Hatzenberger R, Corte FD, Stroeher F, Koller S. Avaliação de um modelo de intervenção psicológica para meninas vítimas de abuso sexual. Psic Teor Pesq. 2008;24:67-75.
• ⁴¹
  Deakin EK, Nunes MLT. Child psychotherapy dropout. Rev Psiquiatr Rio Gd Sul. 2009;31:145-51.
• ⁴²
  Habigzang LF, Stroeher FH, Hatzenberger R, Cunha RC, Ramos MS, Koller SH. Cognitive behavioral group therapy for sexually abused girls. Rev Saude Publica. 2009;43:70-8.
• ⁴³
  Lofrano-Prado MC, Antunes HKM, do Prado WL, de Piano A, Caranti DA, Tock L, et al. Quality of life in Brazilian obese adolescents: effects of a long-term multidisciplinary lifestyle therapy. Health Qual Life Outcomes. 2009;7:61.
• ⁴⁴
  Fonseca RP, Pacheco JTB. Análise funcional do comportamento na avaliação e terapia com crianças. Rev Bras Ter Comp Cogn. 2010;12:1-19.
• ⁴⁵
  Gomide PIC. Abuso, negligência e parricídio: um estudo de caso. Temas Psicol. 2010;18:219-30.
• ⁴⁶
  Telles JCCP, Sei MB, Arruda SLS. Comunicação silenciosa mãebebê na visão winnicottiana: reflexões teórico-clínicas. Aletheia. 2010;33:109-22.
• ⁴⁷
  Turkiewicz G, Pinzon V, Lock J, Fleitlich-Bilyk B. Feasibility, acceptability, and effectiveness of family-based treatment for adolescent anorexia nervosa: an observational study conducted in Brazil. Rev Bras Psiquiatr. 2010;32:169-72.
• ⁴⁸
  Bolsson JZ, Benetti SPC. Angústia infantil: um estudo de caso clínico. Aletheia. 2011;34:61-80.
• ⁴⁹
  Ramires VRR, Godinho LR. The mentalization-based treatment of children who were maltreated. Psicol Estud. 2011;16:61-70.
• ⁵⁰
  Steibel D, Hallberg AE, Sanchotene B, Campezatto PM, Silva MR, Nunes MLT. A latência na atualidade: considerações sobre crianças encaminhadas para psicoterapia. Aletheia. 2011;36:51-68.
• ⁵¹
  Silvares EFM, Pereira RF. Adherence in health and psychotherapy: conceptualization and application in nocturnal enuresis. Psicol USP. 2012;23:539-58.
• ⁵²
  De Souza MA, Salum GA, Jarros RB, Isolan LR, Davis R, Knijnik DZ, et al. Cognitive-behavioral group therapy for youths with anxiety disorders in the community: effectiveness in low and middle income countries. Behav Cogn Psychother. 2013;41:255-64.
• ⁵³
  Sbardelotto FC, Donelli TMS. Entre bruxas e lobos: o uso dos contos de fadas na psicoterapia de grupo com crianças. Contextos Clinic. 2014;7:37-48.
• ⁵⁴
  Corgosinho FC, Ackel-D'Elia C, Tufik S, Dâmaso AR, de Piano A, Sanches PL, et al. Beneficial effects of a multifaceted 1-year lifestyle intervention on metabolic abnormalities in obese adolescents with and without sleep-disordered breathing. Metab Syndr Relat Disord. 2015;13:110-18.
• ⁵⁵
  Fidelix YL, Júnior JCF, Lofrano-Prado MC, Guerra RLF, Cardel M, Prado WL. Multidisciplinary intervention in obese adolescents: predictors of dropout. Einstein (Sao Paulo). 2015;13:388-94.
• ⁵⁶
  Gastaud MB, Carvalho C, Goodman G, Ramires VRR. Assessing levels of similarity to a "psychodynamic prototype" in psychodynamic psychotherapy with children: a case study approach (preliminary findings). Trends Psychiatry Psychother. 2015;37:161-65.
• ⁵⁷
  Masquio DC, de Piano A, Campos RM, Sanches PL, Carnier J, Corgosinho FC, et al. The role of multicomponent therapy in the metabolic syndrome, inflammation and cardiovascular risk in obese adolescents. Br J Nutr. 2015;113:1920-30.
• ⁵⁸
  Carvalho C, Godinho LR, Ramires VRR. O processo psicoterapêutico de uma criança: análise baseada no Child Psychotherapy Q-Set. Temas Psicol. 2016;24:1153-67.
• ⁵⁹
  Abreu JRP, Piccinini W, Cacilhas A, Trahtman CE, Thorman NJ. Psicoterapia no Brasil: duas décadas através de publicações psiquiátricas. Rev Bras Psicoter. 2000;2:89-104.
• ⁶⁰
  Fonagy P. The research agenda: the vital need for empirical research in child psychotherapy. J Child Psychother. 2003;29:129-36.
• ⁶¹
  Peuker AC, Habigzang LF, Koller SH, Araujo LB. Avaliação de processo e resultado em psicoterapias: uma revisão. Psicol Estud. 2009;14:439-45.
• ⁶²
  Gorayeb R. Psicologia da saúde no Brasil. Psic Teor Pesq. 2010;26:115-22.
• ⁶³
  Santiago CD, Kaltman S, Miranda J. Poverty and mental health: how do low-income adults and children fare in psychotherapy? J Clin Psychol. 2013;69:115-26.
• ⁶⁴
  Roth A, Fonagy P. What works for whom? A critical review of psychotherapy research. 2nd ed. London: Guilford Press; 2005.
• ⁶⁵
  Hibbs ED. Evaluating empirically based psychotherapy research for children and adolescents. Eur Child Adolesc Psychiatry. 2001;10:3-11.
• ⁶⁶
  Delgado-Rico E, Carretero-Dios H, Ruch W. Content validity evidences in test development: an applied perspective. Int J Clin Health Psychol. 2012;12:449-59.
• ⁶⁷
  Kohlsdorf M, Junior ADL. O autorrelato na pesquisa em psicologia da saúde: desafios metodológicos. Psicol Argum. 2009;27:131-39.
• ⁶⁸
  American Psychological Association. Evidence-based practice in psychology. Am Psychol. 2006;61:271-85.

Publication Dates

• Publication in this collection
  Oct-Dec 2018

History

• Received
  20 June 2017
• Accepted
  28 Mar 2018