Internal construct validity of a Brazilian version of the McMaster Family Assessment Device

Validade interna de construto de uma versão brasileira do McMaster Family Assessment Device

ABSTRACT

Objective

To assess the internal validity of a Brazilian version of the McMaster Family Assessment Device (FAD).

Methods

The questionnaire was applied to a sample of 80 adults of both sexes. An exploratory factor analysis was conducted after previous analysis of communalities and global suitability of the method to determine questions that did not share a significant percentage of variance with the defined factors. Varimax rotation method with Kaiser’s normalization was used to minimize the number of questions that had high factor loads.

Results

The Brazilian version maintained 28 questions distributed in four new dimensions containing 11, 5, 7, and 5 questions, respectively. To obtain those four factors or dimensions, we selected those with eigenvalue greater than two, which explained about 50% of the data variability.

Conclusion

The Brazilian version under examination showed different behaviors from those in the original instrument. A review of the instrument questions and dimensions is clearly needed. Further research with larger samples is required to examine the rating scale and its suitability to the Brazilian context.

Family functioning; questionnaire; cross-cultural adaptation; validity

RESUMO

Objetivo

Proceder estudo de validação interna de uma versão brasileira do McMaster Family Assessment Device – FAD.

Métodos

O questionário foi aplicado em uma amostra de 80 indivíduos adultos de ambos os sexos. Foi realizada análise fatorial exploratória após análise prévia de adequabilidade global do método e análise de comunalidades para averiguar perguntas que não compartilhavam percentual significativo de variância com os fatores definidos. Para minimizar a quantidade de perguntas que apresentavam elevadas cargas, foi utilizado o método de rotação Varimax com normalização de Kaiser.

Resultados

Na versão brasileira em estudo foram mantidas 28 perguntas em quatro novas dimensões contendo 11, 5, 7 e 5 perguntas. Para obtenção de quatro fatores ou dimensões, foram selecionados aqueles com autovalores superiores a dois, o que explicou cerca de 50% da variabilidade dos dados.

Conclusão

A versão brasileira em estudo apresentou comportamento diferente do instrumento original. Observou-se a necessidade de revisar as perguntas e as dimensões do instrumento e também a realização de mais pesquisas com amostras maiores para observar o comportamento da escala e sua adequabilidade para utilização no contexto brasileiro.

Funcionamento familiar; questionário; adaptação transcultural; validade

INTRODUCTION

The need to understand intra-family relationships led to the creation of different measurement tools. One of these tools is the McMaster Approach to Families proposed by Epstein et al. in 1983¹. The instrument is based on the following assumptions: all family members are interrelated and cannot be understood in isolation from the rest of the system; family functioning cannot be understood by investigating only one member or family group; family organization and structure are important factors to determine the behavior of family members and, transactional patterns shape family behavior².

One of the tools necessary for the assessment of family functioning is a self-administered questionnaire termed Family Assessment Device (FAD), which was prepared in English and adapted into multiple languages, in several countries³-¹⁵. It has already undergone the preliminary stages of a cross-cultural Brazilian Portuguese adaptation process to be used in Brazil¹⁶. However, the construct validity has not been performed so far. Validity testing allows to check whether a particular instrument is actually measuring what it is intended to measure, which is given by the capability to reflect a trait or theoretical construct, thus validating a body of theory¹⁷.

The aim of this study was to examine the internal construct validity of a Brazilian version¹⁶ of the Family Assessment Device (FAD) proposed by Epstein et al.¹.

METHODS

Participants and procedure

An epidemiological cross-sectional study was conducted with the approval of the Ethics Committee of the University of Southern Santa Catarina (protocol number CAAE 38240114.0.0000.5369).

The Brazilian version of the self-report instrument was applied to a community non-probabilistic sample of 80 subjects with a mean age of 33.9 years (SD = 12.8 years), of whom 52.5% were female. Educational attainment was 14.1 years in average (SD = 4.12 years). For more methodological details see Traebert et al.¹⁶.

Statistical analysis

Data were entered into a Microsoft Excel 2013 spreadsheet (Microsoft Corporation, USA) and subsequently exported to the SPSS software version 18.0 (IBM Corporation, USA) where they were analyzed.

Exploratory factor analysis (EFA) was performed in order to verify whether the questions of the Brazilian version behave similarly to the questions posed by the FAD original instrument.

Previous analyses of overall suitability of the method were performed by means of Kaiser-Meyer-Olkin (KMO) and Bartlett’s test of sphericity, as well as Pearson’s correlation coefficient analysis between each pair of questions.

For the definition of the factors or dimensions, we used the declivity diagram criterion of components or eigenvalue (scree plot) to identify the optimal number of factors that could be extracted before the amount of unique variance started to dominate the structure of the common variance, i.e., very similar values that made the graph curve tend to be horizontal¹⁸. The graph is made by plotting the latent root values or eigenvalue on the Y axis and the number of factors in the X axis, according to the order of extraction. So, the point from which the graph becomes horizontal (greater variability of the slope of the tangents to the graph) reflects an indication of the maximum number of factors to be extracted¹⁹. For that purpose, we used the principal component analysis (PCA) method to minimize the correlation between factors, formed by the highest percentage of the variance of the scores shared by the original questions. This method allowed to reduce data, define levels of observations by factors, and check the validity of previously established constructs.

Communalities analysis, which represent the shared total variance of each question on all factors, starting from the eigenvalues greater than 1²⁰, allowed to ascertain what question did not share significant percentage of variance with the defined factors. Varimax with Kaiser’s normalization was used to minimize the number of questions that had high factor loads²⁰.

RESULTS

Overall suitability analysis

Preliminary tests required to examine EFA suitability identified a poor result in the KMO test (0.56), very close to the lower limit of acceptability. Subsequently, the communalities analysis revealed that almost half of the questions of the original instrument (47%) presented an inadequate explanatory power, which would require to be eliminated. Those questions referred mainly to the first three dimensions of the Brazilian version of the FAD.

The same analysis was performed by grouping the table using Varimax rotation with Kaiser normalization²⁰ for correlation values greater than 0.4, which resulted in a different grouping as proposed by the original FAD, leaving out 25 questions because they were not significant. A careful analysis indicated that several questions in the original dimensions had very little correlation and, therefore, could be eliminated. The deleted questions were the following: questions 2 and 4 of the original five, referring to the Problem Solving dimension; questions 7, 10, and 11 of the original six, referring to the Communication dimension; questions 12, 13, 14, 16, 17, 18, and 19 of the original eight, referring to the Roles dimension; questions 21 and 23 of the original six, referring to the Affective Responsiveness dimension; questions 31 and 32 of the original seven, referring to the Affective Involvement dimension; questions 35, 37, 38, 39, and 41 of the original nine, referring to the Behavior Control dimension; and questions 45 and 46 of the original 12, referring to the General Functioning dimension.

Based on this analysis, the dimensions were remodeled. With the same initial parameters, however, with four factors or dimensions, a preliminary solution was found for the lack of correlation between the questions. The four factors accounted for 39% of data variability. Thus, the questions with high correlation (0.50 to 0.84) between them were grouped within each new dimension.

Following the tool remodeling, the suitability tests were applied again. The KMO test scored 0.74, indicating a good fit between data. Bartlett’s test of sphericity resulted in a significance level of less than 0.001, which also meant adequacy of the data to support the EFA. Matrix significance, represented by the significance values of Pearson’s correlation coefficient, showed 85% of coefficients lower than 0.33. In the new proposal, all communalities presented a maximum value of 1.

Defining factors

When using the principal component analysis (PCA), eigenvalue expresses the total variance that each factor can explain from the variability. Diagram analysis of the declivity components/eigenvalues (scree plot) revealed that on the point referring to four factors, the graph slope varied substantially (Figure 1).

Figure 1
Graph showing the relationship between component number and eigenvalue.

Table 1 shows the factor loadings for each variable or question in relation to the extracted components. Since it is important that the same variable does not contribute to several factors, a rotation was applied between the observed variables and extracted factors by using a Varimax model. Thus, dimension 1 consisted of 11 questions, dimension 2 consisted of 5 questions, dimension 3 consisted of 7 questions, and dimension 4 consisted of 5 questions.

Table 1
Matrix of rotated factor loadings with four dimensions

Questions and respective number in the original instrument	Component
	Dimension 1	Dimension 2	Dimension 3	Dimension 4
51. We are able to make decisions about how to solve problems (Somos capazes de tomar decisões sobre como resolver problemas)	0.4
3. We resolve most emotional upsets that come up (Resolvemos a maioria dos distúrbios emocionais que surgem)	0.5
30. We show interest in each other when we can get something out of it personally (Demonstramos interesse para com os outros quando podemos tirar algum proveito pessoal)	0.5
44. We cannot talk to each other about the sadness we feel (Não podemos nos abrir com os outros sobre a tristeza que sentimos)	0.5
29. We get involved with each other only when something interests us (Nos envolvemos com os outros somente quando temos algum interesse)	0.6
1. We usually act on our decisions regarding problems (Costumamos agir de acordo com as decisões que tomamos em relação a problemas)	0.6
52. We don’t get along well together (Não nos damos muito bem juntos)	0.6
50. Making decisions is a problem for our Family (Tomar decisões é um problema para a nossa família)	0.6
53. We confide in each other (Temos confiança uns nos outros)	0.7
42. Planning family activities is dificult because we misunderstand each other (Planejar as atividades familiares é difícil, porque não nos entendemos)	0.7
48. There are lots of bad feelings in the Family (Há muitos sentimentos ruins na família)	0.7
24. We express tenderness (Expressamos nossa ternura)		0.6
25. We cry openly (Choramos abertamente)		0.6
22. We do not show our love for each other (Não demonstramos o nosso amor uns pelos outros)		0.7
26. If someone is in trouble, the others become too involved (Se alguém está com problemas, os outros se envolvem demais)		0.7
6. When someone is upset the others know why (Quando alguém fica chateado, os outros sabem por quê)		0.8
20. We are reluctant to show our affection for each other (Temos resistência em demonstrar nosso afeto um pelo outro)			0.4
33. We don’t know what to do when an emergency comes up (Não sabemos o que fazer quando surge uma emergência)			0.4
49. We feel accepted for what we are (Nos sentimos aceitos por aquilo que somos)			0.5
9. We are frank with each other (Somos francos uns com os outros)			0.5
15. We have trouble meeting our bills (Temos dificuldade em pagar nossas contas)			0.5
8. People come right out and say things instead of hinting at them (As pessoas vêm e falam abertamente em vez de insinuar as coisas)			0.5
28. We are too self-centered (Somos muito egocêntricos)			0.7
43. In times of crisis we can turn to each other for support (Em tempos de crise, podemos recorrer uns aos outros para ter apoio)				0.4
34. You can easily get away with breaking the rules (Você pode facilmente escapar quebrando as regras)				0.6
5. We try to think of diferent ways to solve problems (Tentamos resolver os problemas de diversas formas)				0.6
36. We have no clear expectations about toilet habits (Não temos expectativas claras sobre os hábitos de higiene)				0.7
40. Anything goes in our Family (Vale tudo na nossa família)				0.8

• Extraction method: principal component analysis.
• Rotation method: Varimax with Kaiser normalization.

DISCUSSION

Analysis of the internal validation of the Brazilian version containing four different dimensions, as proposed by Traebert et al.¹⁶, presented a better solution, both in the number of dimensions and in the number of variables or questions in each of them, as compared to the original assessment tool.

In the proposed Brazilian version, the 28 remaining questions of the original instrument were ordered by the PCA: the new dimension 1 was composed of 11 questions, of which 7 were derived from the original dimension of the FAD General Functioning, 2 questions (18.2%) from the Solving Problem dimension, and 2 questions (18.2%) from the Affective Involvement dimension. For that reason, it might be suggested that such a dimension could represent the family overall functioning and, therefore, could hold such a title. The new dimension 2 was composed of 5 questions as follows: 3 questions (60%) from the original Affective Responsiveness dimension, 1 question (20%) of the Affective Involvement dimension, and 1 question (20%) of the Communication dimension. This way, dimension 2 should hold the title of Affective Responsiveness. The new dimension 3 was created by reordering 7 questions as follows: 2 questions (28.5%) from the original Communication dimension, 1 question (14.3%) from the Affective Responsiveness dimension, 1 question (14.3%) from the Affective Involvement dimension, 1 question (14.3%) from the Behavior Control dimension, 1 question (14.3%) from the General Functioning dimension, and 1 question (14.3%) from the Roles dimension. Finally, the new dimension 4 was composed for 5 questions as follows: 3 questions (60.0%) from the original Behavior Control dimension, 1 question (20.0%) from the Family Functioning, and 1 question (20.0%) from the Problem Solving dimension. Dimension 4 could maintain the designation Behavior Control.

These results are in line with those on FAD cross-cultural adaptations in other countries. In many studies, the results were inconclusive, showing variable values, very different from the original tool. In conducting EFA analysis of the data that resulted in the FAD, Ridenour et al.²¹,²² concluded that the original dimensions did not form different components; therefore, the subscales of the instrument should be only two, General Functioning and Behavior Control.

FAD versions in different countries with a similar number of factors or dimensions than the Brazilian version include the Chinese⁷ version studied by PCA, and the Spanish³ and French¹¹ versions studied by EFA, both with three dimensions. The Dutch⁶ version supported the original FAD, whereas the Italian⁵ and Icelandic⁴ versions have seven and eight factors, respectively.

It is important to consider this study as a possible solution or at least a proposal to be taken into account in further studies about the cross-cultural adaptation of the assessment tool to the Brazilian reality. In this context, Pires et al.²³ have indicated another Brazilian version of FAD, which they stated revealed a good applicability, easy understanding by the population, and good measurement of the construct of interest.

Among the limitations of this study, we must mention the relatively small sample, which might have influenced the results of EFA analysis. However, there are indications that better results will be obtained in factor analysis with great sample heterogeneity, so as to represent the target population²⁴. This author has analyzed sample size in several surveys, and he has detected disagreement regarding the minimum number of samples per variable required to obtain a reliable factor analysis. The fact that a discriminatory analysis between groups with and without a history of family or psychiatric problems was not carried out because and only a sample of community was used instead, may be cited as another limitation of this study.

Findings from this study indicate the need to review the questions and dimensions of the assessment tool. Further research with a larger sample is required to analyze the rating scale and its suitability for the Brazilian context.

CONCLUSIONS

It could be concluded that the Brazilian version under examination showed different behaviors from those in the original instrument. A review of the instrument questions and dimensions is clearly needed. Further research with larger samples is required to examine the rating scale and its suitability to the Brazilian context.

ACKNOWLEDGMENT

We thank Programa de Suporte à Pós-Graduação de Instituições de Ensino Particulares/Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Prosup/Capes), Brazil for granting doctoral scholarship to the author ET.

REFERENCES

• ¹
  Epstein NB, Baldwin LM, Bishop DS. The McMaster Family Assessment Device. J Marital Fam Ther. 1983;9:171-80.
• ²
  Miller IW, Ryan CE, Keitner GI, Bishop DS, Epstein NB. The McMaster Approach to Families: theory, assessment, treatment and research. J Fam Ther. 2000;22:168-89.
• ³
  Barroilhet SPA, Cano-Prous A, Cervera-Enguix S, Forjaz MJ, Guillen-Grima F. A Spanish version of the Family Assessment Device. Soc Psychiatry Psychiatr Epidemiol. 2009;44(12):1051-65.
• ⁴
  Juliusdottir GM, Olafsdottir H. An Icelandic version of McMaster Family Assessment Device (FAD). Res Social Work Pract. 2014;25(7):815-27.
• ⁵
  Roncone RL, Muiere E, Impallomeni M, Matteucci M, Giacomelli R, Tonietti G, et al. The Italian version of the Family Assessment Device. Soc Psychiatry Psychiatr Epidemiol. 1998;33(9):451-61.
• ⁶
  Wenniger WFM, Hageman WJJM, Arrindell WA. Cross-national validity of dimentions of family functioning: first experiences with the Dutch version of the McMaster Family Assessment Device (FAD). Person Individ Diff. 1993;14(6):769-81.
• ⁷
  Shek DTL. The General Functioning Scale of the Family Assessment Device: does it work with Chinese adolescents? J Clin Psychol. 2001;57(12):1503-16.
• ⁸
  Saeki T, Asukai N, Miyake Y, Miguchi M, Yamawaki S. Characteristics of family functioning in patients with endogenous monopolar depression. Hiroshima J Med Sci. 2002;51(2):55-62.
• ⁹
  Diareme S, Tsiantis J, Kolaitis G, Ferentinos S, Tsalamanios E, Paliokosta E, et al. Emotional and behavioural difficulties in children of parents with multiple sclerosis: a controlled study in Greece. Eur Child Adolesc Psychiatry. 2006;15(6):309-18.
• ¹⁰
  Kazarian SS. Cultural appropriateness of the Family Assessment Device (FAD) in the case of ethnic Armenian adolescents in Lebanon. Int J Soc Psychiatry. 2010;56(3):230-8.
• ¹¹
  Speranza M, Guénolé F, Revah-Levy A, Egler PJ, Negadi F, Falissard B, et al. The French version of the Family Assessment Device. Can J Psychiatry. 2012;57(9):570-7.
• ¹²
  Byles J, Byrne C, Boyle MH, Offord DR. Ontario Child Health Study: reliability and validity of the general functioning subscale of the McMaster Family Assessment Device. Fam Process. 1988;27(1):97-104.
• ¹³
  Keitner GI, Fodor J, Ryan CE, Miller IW, Bishop DS, Epstein NB. A cross-cultural study of major depression and family functioning. Can J Psychiatry. 1991;36(4):254-9.
• ¹⁴
  Al-Krenawi A, Graham JR, Slonim-Nevo V. Mental health aspects of Arab-Israeli adolescents from polygamous versus monogamous families. J Soc Psychol. 2002;142(4):446-60.
• ¹⁵
  Kazarian SS. Family functioning, cultural orientation, and psychological well-being among university students in Lebanon. J Soc Psychol. 2005;145(2):141-52.
• ¹⁶
  Traebert E, Santos K, Carvalho LM, Silva J, Traebert J. Preliminary stages of cross-cultural adaptation of the Brazilian Portuguese version of McMaster Family Assessment Device. J Bras Psiq. 2016;65(3):201-8.
• ¹⁷
  Reichenheim ME, Moraes CL. Operationalizing the cross-cultural adaptation of epidemiological measurement instruments Rev Saude Publica. 2007;41(4):665-73.
• ¹⁸
  Hair Jr JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. Bookman, Porto Alegre/RS, 2009.
• ¹⁹
  Favero LP, Belfiore P, Silva FL, Chan BL. Análise de dados: modelagem multivariada para tomada de decisões. Elsevier, Rio de Janeiro/RJ, 2005.
• ²⁰
  Kaiser HF, Caffrey J. Alpha factor analysis. Psychometrika. 1965;30:1-14.
• ²¹
  Ridenour TA, Daley JG, Reich W. Factor analyses of the family assessment device. Fam Process. 1999;38(4):497-510.
• ²²
  Ridenour TA, Daley JG, Reich W. Further evidence that the family assessment device should be reorganized: response to Miller and colleagues. Fam Process. 2000;39(3):375-80.
• ²³
  Pires T, Assis SG, Avanci JQ, Pesce RP. Cross-Cultural adaptation of the General Functioning Scale of the Family. Rev Saude Publica. 2016:50.
• ²⁴
  Laros JA. O uso da Análise Fatorial: algumas diretrizes para pesquisadores. In: Pasquali L (editor). Análise fatorial para pesquisadores. LabPAM Saber e Tecnologia, Brasília/DF, 2012.

Publication Dates

• Publication in this collection
  Oct-Dec 2016

History

• Received
  6 Dec 2016
• Accepted
  15 Dec 2016