VALIDITY AND RELIABILITY OF THE BRAZILIAN VERSION OF THE INSTRUMENT THE ASSESSMENT STRATEGIES IN FAMILIES-EFFECTIVENESS (ASF-E)

VALIDEZ Y CONFIABILIDAD DE LA VERSIÓN BRASILEÑA DEL INSTRUMENTO THE STRATEGIES IN FAMILIES-EFFECTIVENESS ASF-E/BRASIL

ABSTRACT

Objective:

to test the psychometric validity of an instrument to assess family effectiveness strategies and its reliability for use with Brazilian families.

Method:

this is a methodological study, content validation, construct, criterion, which used exploratory and Confirmatory Factor Analysis and reliability. The sample consisted of 100 families. Data were collected from October 2017 to March 2018.

Results:

the percentage of agreement among family health experts was 100%, with a perfect Content Validity Index and reliability (1.00). For the target audience (pre-test), 89.3% of agreement and Content Validity Index of 0.89 were obtained. Construct validity had a total explained variance of 73.8%. The concurrent criterion validity presented a correlation coefficient (ρ) of 0.75 (p = 0.004). Reliability had a Cronbach’s α coefficient of 0.82 for the full scale and a range of 0.62 to 0.82 in the instrument’s goals.

Conclusion:

the instrument proved to be valid and reliable to assess the effectiveness of the functioning of Brazilian families.

DESCRIPTORS:
Family nursing; Family; Self efficacy; Transcultural nursing; Reproducibility of tests; Psychometry

RESUMEN

Objetivo:

probar la validez psicométrica del instrumento para evaluar las estrategias de efectividad familiar y su confiabilidad para su uso con familias brasileñas.

Método:

estudio metodológico, validación de contenido, constructo, criterio, mediante análisis factorial exploratorio y confirmatorio y confiabilidad. La muestra estuvo formada por 100 familias. Los datos se recopilaron desde octubre de 2017 hasta marzo de 2018.

Resultados:

el porcentaje de concordancia entre los especialistas en Salud de la Familia fue del 100%, con un perfecto Índice de Validez de Contenido y confiabilidad (1,00). Para la población objetivo (pre-test), se obtuvo 89,3% de acuerdo y un Índice de Validez de Contenido de 0,89. La validez de constructo tuvo una varianza explicada total del 73,8%. La validez del criterio concurrente presentó un coeficiente de correlación (ρ) de 0,75 (p = 0,004). La confiabilidad tuvo un coeficiente α de Cronbach de 0.82 para la escala completa y un rango de 0.62 a 0.82 en las metas del instrumento.

Conclusión:

el instrumento demostró ser válido y confiable para evaluar la efectividad del funcionamiento de las familias brasileñas.

DESCRIPTORES:
Enfermería de la familia; Familia; Autoeficacia; Enfermería transcultural; Reproducibilidad de los resultados; Psicometría

RESUMO

Objetivo:

testar a validade psicométrica do instrumento de avaliação das estratégias de efetividade familiar e a confiabilidade para uso com famílias brasileiras.

Método:

estudo metodológico, de validação de conteúdo, construto, critério, pela Análise Fatorial Exploratória e confirmatória e confiabilidade. A amostra foi composta por 100 famílias. Os dados foram coletados no período de outubro de 2017 a março de 2018.

Resultados:

o percentual de concordância entre os especialistas em Saúde da Família foi de 100%, com Índice de Validade de Conteúdo e confiabilidade perfeitos (1,00). Para a população-alvo (pré-teste), obtiveram-se 89,3% de concordância e Índice de Validade de Conteúdo de 0,89. A validade de construto apresentou variância total explicada de 73,8%. A validade do critério concorrente apresentou coeficiente de correlação (ρ) de 0,75 (p=0,004). A confiabilidade apresentou coeficiente α de Cronbach de 0,82 para a escala total e variação de 0,62 a 0,82 nas metas do instrumento.

Conclusão:

o instrumento mostrou-se válido e confiável para avaliar a efetividade do funcionamento das famílias brasileiras.

DESCRITORES:
Enfermagem familiar; Família; Autoeficácia; Enfermagem transcultural; Reprodutibilidade dos testes; Psicometria

INTRODUCTION

The family is a unit with structure and organization that interacts with the environment. It is a system composed of subsystems, shaped by the feeling of belonging and the commonality of responsibilities, which have distinct relationships with family members1. In this context, family nurses have, as a challenge, the development of skills to perform culturally sensitive care, taking into account the relationships of members in the cultural and contextual environment of families in the community, in addition to using systemic thinking and hypotheses to allow multiple understanding of the family, expanding the focus of nursing assessments and interventions2.

International recommendations, from an entity composed of specialists in family health, pointed to the need for nurses to have instruments to assess family functionality or the level of family health based on the systemic approach2^-3. To know the instruments used by nurses to assess Brazilian families’ health, the literature was explored and the lack of instruments developed by nursing, validated and supported by a consistent theoretical model was identified4. It was also evidenced that, in Brazil, one of the instruments available to assess family functionality with a systemic approach is the Family Environment Scale5 and there is no record of use of The Strategies in Families-Effectiveness (ASF-E), which is based on a consistent theoretical model, developed by an English-speaking nurse, tested and validated in the United States6. Other ASF-E tests and validations were developed with families in Mexico7, in Colombia8, in Germany and Switzerland9 and in Finland10.

The ASF-E, free access, was developed to assess the effectiveness of family functioning or its level of health, supported by the Framework of Systemic Organization1. Through its items, it can offer the family an opportunity for reflection, allowing members to explore their organizational capacity. To measure how the family organizes itself to function effectively as a system and to respond to the demands of each member and the environment, it is necessary to take into account the factors that can interfere with this gear. The theoretical model of systemic organization defines that, in order to achieve balance in the family system, four dimensions that compose it are called: stability, growth, control and spirituality. To respond to system requirements, functioning is maintained by the values and beliefs developed in family processes in relation to coherence, system maintenance, individuation and system change strategies1.

When considering the importance of nurses assessing the family functionality and the lack of instruments developed by nursing to do so, the dissemination of this study can contribute to nurses’ practice as a strategy for promoting family health. Thus, it was understood that the ASF-E/Brazil can favor assessment and development of intervention strategies in promoting health for families. Thus, the process of cross-cultural adaptation and the validity test began, following the rigorous methodological process, supported by references used by nursing and internationally acknowledged11. The aim of this study was to test the psychometric validity of an instrument for assessing family effectiveness strategies, ASF-E/Brazil, by assessing content, construct and criteria validity as well as reliability for use with Brazilian families.

METHOD

This is a psychometric study to test the ASF-E/Brazil content validity, construct, criteria and reliability for use with Brazilian families. The study was authorized by the author of the instrument and developed with families assisted in four health units, with the Family Health Strategy of the Unified Health System (SUS - Sistema Único de Saúde), located in southern Brazil. Therefore, the methodological steps of the Guidelines for the Process of Cross-Cultural Adaptation of Self-Report Measures were followed12^-14.

Content validity can be considered cross-cultural adaptation and included: 1) translation; 2) translation synthesis; 3) back-translation; 4) synthesis of back translations; 5) back-translation synthesis assessment; 6) assessment by an expert committee; and 7) pre-test with the target audience. Translation into Brazilian Portuguese was performed by four official translators, and two translators performed back-translation in two independent versions in English, but they had no contact with the original version of the instrument and did not participate in the initial translations into Brazilian Portuguese. The family health expert committee was composed of five nurses from different regions of Brazil. This committee judged the relevance and comprehensiveness of all items in the instrument and assessed equivalence in four distinct areas: semantic, idiomatic, conceptual and experimental discrepancies. At the end of this step, it was possible to obtain the consensus version to be tested with the families among the experts. To do so, we calculated the percentage of agreement and Content Validity Index, which consists of the proportion of judges who are in agreement with certain items. The Kappa coefficient assessed the ratio of the proportion of times experts agreed with the item with the maximum proportion they could agree to determine the instrument reliability as satisfactory. This committee also participated in face validity with a documentalist librarian and a Bachelor of Arts, who assessed three different versions of the instrument (font size, typeface, with items presented in a framed and unframed table). At the end, a pre-test was carried out with a non-probabilistic sample of 15 individuals to assess the target audience’s perception about the facilities and difficulties in understanding the instrument’s items. Individuals were approached until there were no answers reporting incomprehension, testing the initial reliability with 30 individuals. Regarding the number of participants in the pre-test, the orientation was followed that, in order to carry out pilot studies, it is not necessary for the number of participants to exceed 10% of the desired sample15. This percentage is considered satisfactory for this assessment.

This study complied with the national standards of ethics in research involving human subjects of the Brazilian National Health Council (Conselho Nacional de Saúde), and received approval from an Institutional Review Board.

Participants were recruited based on the indication of community health workers in the coverage area of the health units in Pelotas, southern Brazil. The sample size, to test validity and reliability, was defined following the recommendation of at least ten participants per instrument item (n=200). In all, 100 households were selected by convenience sampling, with two individuals each, which allowed the application of Exploratory Factor Analysis15^-16. The criteria for inclusion in the study were: a) being over 18 years of age; b) reading in Brazilian Portuguese; c) having two family members to answer the instrument; and d) belonging to the coverage area of the health units. Exclusion criteria were: a) not being able to communicate verbally to answer the questions and b) having answered the instruments in the pre-test. Among the families approached, around 10.0% did not meet the inclusion criterion “possibility of participation of two family members”.

Families were visited by the researcher, followed by community health workers, nurses, physicians and/or nursing students. On this occasion, the objectives of the study, the data collection instruments and the Informed Consent Form (ICF) were presented. After participants agreed to participate in the study, the ICF was read and signed in duplicate, one for the researcher and one for each participant. Visits lasted about one hour, as some were routine visits by the team, and the time for the families to complete the ASF-E/Brazil, which was self-administered, was 20 minutes. Members completed the ASF-E individually in the same environment. The ASF-E items were read only to those who asked the researcher. On that occasion, participants followed the reading and indicated the option that best represented their family, individually. Data collection took place between 2017 and 2018.

For data collection, two instruments were used, in addition to the sociodemographic questionnaire, developed and applied by the researcher, to characterize the family profile (with information on sex, age, marital status, number of children, years of education and origin of the main source of family income).

The first was the ASF-E/Brazil, which is a nominal, self-applicable scale, and has 20 items that measure the effectiveness of family functioning. Each item has three response alternatives (one, two or three points), and alternative three, medium, alternative two, and low, alternative one, are considered to be highly effective in family functioning. The total value of the instrument is 60 points; the low effectiveness of family functioning corresponds to a score between 20 and 33 points; for the intermediate level, 34-47 points; for the high level, the result between 48 and 60 points.

The theoretical model of systemic organization is based on the operationalization of 20 items corresponding to the system dimensions in which the processes are found (coherence, individuation, system maintenance and system change). System goals are classified as Stability = System Coherence and Maintenance; Growth = Individuation and System Change; Control = System Maintenance and System Change; Spirituality = Coherence and Individuation. The ASF-E/Brazil dimensions and corresponding items are: Coherence - 1, 10, 15, 18; Individuation - 3, five, 7, 12; System Change - 4, 8, 13, 17, 19; System Maintenance - 2, 6, 9, 11, 14, 16, 20.

The ASF-E, developed in the United States, had its reliability obtained by internal consistency, in which Cronbach’s α ranged from 0.60 to 0.80. Factor analysis, with four factors, carried eigenvalues, ranging from 1.01 to 1.486. In the ASF-E reliability tests developed with families in Mexico, internal consistency ranged from 0.57 to 0.807; in Colombia, it was from 0.60 to 0.808; in Germany and Switzerland, reliability was 0.80 and 0.82, respectively9, and in Finland, reliability was 0.8510.

The second instrument, the Family Environment Scale (FES)5, adapted and validated in Brazil17, was used for concurrent criterion validity analysis because it theoretically assesses a similar construct.

Sociodemographic characteristics were descriptively analyzed for each question, and the values observed in the sample (n) and the percentages were expressed (%).

The ASF-E/Brazil validity was verified through content, construct and criterion validity tests. The ASF-E/Brazil content validity was assessed through the degree of agreement of family health experts and individuals in the pre-test. For this, the measures of percentage of agreement (agreement (%) = number of participants who agreed/total number of participants x 100), Content Validity Index (CVI)18 and the Kappa coefficient were used. For CVI, a Likert-type scale was used, with scores from one to five, in which: one = not clear; two = unclear; three = clear; four = very clear and five = totally clear. The equation used to calculate was CVI = number of valid answers (“3”, “4”, “5”)/total number of answers13^,19^-20, considering for this study as an acceptable rate of agreement values above 80%. The reliability analysis of the instrument’s evaluators (family health experts and individuals from the pre-test) was also performed, based on the intraclass correlation coefficient (ICC). This coefficient was obtained by means, based on consistency, using the 95% confidence interval.

Construct validity determines the extent to which the instrument measures the theoretical concepts or characteristics it is supposed to measure, linking the instrument with its theoretical framework and extracting concepts operationalized through logical and statistical methods21. For this purpose, Exploratory Factor Analysis (EFA) was performed, using the principal components extraction method, with Varimax orthogonal rotation16. The number of components was determined by the eigenvalue criterion, and only factors with eigenvalues > 1.0 were considered to have at least 70% explained variance22. The verification of the sample adequacy to the factor analysis was assessed by the Kaiser-Meyer-Olkin (KMO) test, adopting a value greater than 0.80 as the adjustment criterion for the factor analysis model adjustment23. The extraction criterion adopted for the selection of items in each factor in the rotated factor matrix was that the item load was > 0.3024, ensuring that the item loading was selected based on the conceptually adequate model25.

To assess the ASF-E/Brazil concurrent criterion validity, Spearman’s correlation coefficient (ρ) (p<0.05) was examined between the total scores of the two instruments, ASF-E/Brazil and FES. Spearman’s correlation (ρ) was chosen based on the indication in the normality test (Kolmogorov-Smirnov) that the data were non-parametric.

In order to test the applicability of the four-factor model proposed by the author of the ASF-E in its original version6, Confirmatory Factor Analysis (CFA) was used with the Analysis of Moment Structures (AMOS) and Statistical Package for the Social software Sciences (v.21, SPSS Inc, Chicago, IL), using the Maximum Likelihood (ML) method). Comparative Fit Index (CFI), Tucker-Lewis Index (TLI) and Standardized Root Mean Square Residual (SRMR) were used, which refers to the root mean square of approximation errors. As criteria for satisfactory adjustment of the model to the data, CFI greater than 0.90 and Root Mean Squared Error of Approximation (RMSEA) were adopted to assess the ideal upper bound (optimal upper bound) whose values are <0.05 and <0.08. The chi-square test (X2) was used, which indicates the magnitude of the discrepancy between the observed and modeled covariance matrix25.

Reliability was assessed using the internal consistency measure and Cronbach’s Alpha (α) coefficient. The latter reflects the degree of covariance between the instrument items and values below 0.7 and close to 0.6, which are considered satisfactory26. In addition to Cronbach’s α, ICC, calculated as mentioned above, and Spearman’s correlation (p) (p<0.05) were used as complements to verify the instrument reliability (ASF-E/Brazil). Cronbach’s α and ICC were calculated for the total instrument, separately, for each factor of EFA and also for each dimension and goal of the instrument (ASF-E/Brazil), after the completion of EFA and the formation of the new grouping of items27. Spearman’s coefficient (p) (p<0.05) was calculated by correlating the total scale with the values of the dimensions formed from EFA.

RESULTS

The sample consisted of 60% female individuals, 70.5% aged between 18 and 60 years and married or in a stable relationship. There was a predominance of families with children and more than half of the sample had more than 11 years of education (Table 1).

Table 1 -
Sample characterization. Pelotas, RS, Brazil, 2018. (n=200)

Variable	Category	(n) *	(%)
Sex	Female	130	65.0
	Male	70	35.0
Age	18 to 60	141	70.5
	> 61	59	29.5
Marital status	Married or in stable relationship	122	61.0
	Single	45	22.5
	Divorced or legally separated	19	9.5
	Widowed	14	7.0
Children	No	39	19.5
	Yes	161	80.5
Years of education	Did not attend school	04	2.0
	5 to 9 (Elementary level)	86	48.0
	10 to 12 (High school level)	52	26.0
	15 to 16 (Higher education level)	48	24.0
Familiar income†	3 to 6 wages	129	64.5
	> 3 wages	71	35.5

• ^*
  (n= 200) = total number of participants ^†Brazilian minimum wage in the amount of R$ 937.00 (about US$171,00).

Content validity

The assessment of the percentage of agreement among family health experts was 100% and, for the other measures, i.e., CVI, Kappa coefficient and ICC, perfect agreement and reliability were obtained, with a value of 1.00. Of the 20 items that make up the instrument, with three alternatives each, totaling 60 assessed alternatives, nine had some term changed, in agreement, by family health experts, to ensure semantic, idiomatic, conceptual, cultural and experiential equivalence. The amended terms were approved by the author of the instrument as they did not change the meaning. For instance, in item number one, the consensus of translations was Nossa família é feliz de maneira geral and this was changed by experts to Nossa família é feliz de um modo geral.

In the instrument’s pre-test, with 15 individuals from the target population, in the assessment of conceptual and semantic equivalence and in understanding the ASF-E items for face and content validity, 89.3% of agreement was obtained, CVI of 0.89, Kappa coefficient of 0.80 and ICC (95% CI) of 0.99 (0.997 - 0.999).

Construct validity

The EFA of the 20 items initially resulted in seven factors, three of which consisted of a very small number of items with no conceptual meaning. The next decision was to force the items, aiming to obtain a smaller number of factors, according to the Varimax orthogonal rotation, by which solutions of three to six factors were explored for conceptual adjustment. The four-factor solution was adequate, since the resulting factors corresponded to the dimensions of the established structural process. Based on the analysis, it was not necessary to exclude items, as all factor loadings of the items were greater than 0.40; similarly, the registration of crossed items in several factors did not occur. The presence of significant correlation and a correlation coefficient greater than 0.30 between items of the same factor also allowed the items to remain in the instrument.

The final factor structure was organized into four factors with eigenvalues ranging from 7.7 to 1.5, which explained 73.8% of the total data variance. Cronbach’s α ranged from 0.80 to 0.54 and the ICC (95% CI) from 0.81 (0.77 - 0.84) to 0.53 (0.37 - 0.64) (Table 2).

Table 2 -
Structure of Exploratory Analysis Factors. Pelotas, RS, Brazil, 2018. (n=200)

Item	Factor 1	Factor 2	Factor 3	Factor 4
	SM*/C†	SC‡/I§	SC‡	I§
Item 11 (SM) *	0.67
Item 15 (C)†	0.66
Item 1 (C)†	0.64
Item 9 (SM)*	0.59
Item 20 (SC)‡ → (SM) *	0.58
Item 18 (C)†	0.54
Item 10 (C)†	0.52
Item 14 (SM) *	0.52
Item 6 (SM) *	0.50
Item 16 (SM) *	0.47
Item 2 (SM)*	0.46
Item 3 (I)§		0.77
Item 8 (SC)‡		0.59
Item 12 (I)§		0.56
Item 19 (SM)* → (SC)‡		0.53
Item 17 (SC)‡			0.65
Item 4 (SC)‡			0.63
Item 13 (SC)‡			0.50
Item 7 (I)§				0.73
Item 5 (C)† → (I)§				0.66
Eigenvalues	7.8	3.6	1.85	1.6
(%)¶ explained variance	38.8	17.9	9.2	7.9

• ^*
  SM = System Maintenance; ^‡SC = System Change; ^†C = Coherence; ^§I = Individuation; ^¶ (%) = Percentage.

Factor one, with an eigenvalue of 7.8, accounted for 38.8% of the data variance and expressed the system stability goal or the combination of system maintenance and coherence. The system maintenance dimension was composed of seven items in which six were originally coded as system maintenance. The item coded as system change (item 20) became part of the system maintenance factor. The coherence dimension was made up of four items, all originally coded in this category.

Factor two, with an eigenvalue of 3.6, explained 17.9% of the data variance and was designated as the growth goal or the combination of system change and individuation. In this factor, three originally coded items were included, such as individuation (3 and 12) and system change (eight). Item 19, initially coded as system maintenance, became part of the factor, being recoded for system change.

Factor three, with an eigenvalue of 1.85, representing 9.2% of the data variance, was defined as a change in the system, and items four, 13 and 17 contained in this dimension were grouped as expected. Factor four, with an eigenvalue of 1.6 and explaining 7.9% of the data variance, was characterized as individuation. Item seven was originally weighted for this dimension, while item five, determined for consistency, was restructured for individuation.

Concurrent Criterion Validity

Concurrent criterion validity presented a Spearman’s correlation coefficient (ρ) of 0.75 (p=0.004) between the total scores of the FES and the ASF-E/Brazil.

Confirmatory Factor Analysis

CFA showed that the four-factor model with 20 items presents fit indicators with measures of comparison of fit of CFA, and CFI and TLI showed good modeling, with results of 0.982 and 0.975, respectively. The SRMR residue was small, i.e., 0.005. Population surveys verified as RMSEA were slightly above, but within the tolerance area, 0.073 and 0.092, respectively. The chi-square test value was statistically significant at 5%, possibly due to the high sensitivity of the index to the sample size. This result is not characterized as a limitation for the validation of two models, but also for the adjustment indicators of the empirical data model. All saturations are in the range 0-1 (Figure 1).

Figure 1 -
Structural equation model of Confirmatory Factor Analysis of the ASF-E/Brazil. Pelotas, RS, Brazil, 2018.

In CFA, the four dimensions, system maintenance, system change, individuation and coherence, which configure the effectiveness strategies of family functioning, are the latent variables, composed of 20 items, which are the observed variables. By analyzing Figure 1, it can be seen that the factor weights between the observed and latent variables were weak and ranged from 0.22 to 0.44.

Regarding the magnitude of the observed variables and exogenous variables, in the System Change dimension, item 17 had the greatest influence (0.44); in the individuation dimension, items three and seven had the same result, with a magnitude of 0.44. In the dimension of coherence, the item with the greatest magnitude was item 15, with an influence of 0.42, and also in the dimension of system maintenance, item nine had the greatest influence (0.25). Therefore, items three, seven and 17 were the ones that most influenced family functioning as a whole.

The prediction errors of each dimension, i.e., the correlation between the dimensions showed that all latent variables are significantly intercorrelated (consistent with the theory), except for the individuality dimension, showing a negative correlation with the coherence dimension (-0.14). Among the correlated variables, the highest correlation was between system coherence and maintenance and individuation dimensions (0.68) (Figure 1).

ASF-E/Brazil reliability

The ASF-E/Brazil reliability was assessed using consistency analysis, based on Cronbach’s α and obtaining 0.82for the scale as a whole and range from 0.59 to 0.70 for the instrument dimensions, separately, after EFA, with the rearrangement and sum of the items in each dimension. The results were consistent and most goals achieved high coefficients, ranging from 0.62 to 0.82 (Tables 3 and 4). The behavior of ICC (95% CI) was similar to that which occurred with Cronbach’s α, with 0.81 (0.77 - 0.85) for the total scale and a variation of 0.58 (0.45 - 0, 69) to 0.71 (0.65 - 0.77) for the dimensions. For the goals, reliability was high, with the highest coefficient of 0.81 (0.77 - 0.84) for stability and the lowest of 0.59 (0.50 - 0.67) for growth.

When correlating the total scale with each dimension, positive correlations were observed and the highest coefficient occurred for system maintenance (ρ = 0.84), confirming that the increase in the level of effectiveness of family functioning resulted mainly from increases obtained in the system maintenance dimension.

Table 3 -
Reliability of ASF-E/Brazil dimensions, considering the rearrangement and sum of items in each dimension after Exploratory Factor Analysis. Pelotas, RS, Brazil, 2018. (n=200)

Dimension and Items	α*	ICC (95%)†	Spearman (ρ) ‡
Coherence (1, 10, 15, 18)	0.70	0.67 (0.54 - 0.71)	0.77
Individuation (3, 5, 7, 12)	0.59	0.58 (0.45 - 0.69)	0.66
System Maintenance (2, 6, 9, 11, 14, 16, 20)	0.74	0.71 (0.65 - 0.77)	0.84
System Change (4, 8, 13, 17, 19)	0.62	0.60 (0.49 - 0.71)	0.63

• ^*
  α=Cronbach’s Alpha; ^†ICC=Intraclass Correlation Coefficient and ^‡Spearman’s correlation coefficient (ρ) p < 0.0001.

Likewise, there was also a positive correlation between the total scale and all goals, with a higher coefficient for control (ρ=0.93) (Table 4).

Table 4 -
Reliability of the ASF-E/Brazil goals. Pelotas, RS, Brazil, 2018. (n=200)

Goals	Dimensions and Items	α*	ICC (95%)†	Spearman (ρ) ‡
Stability	SM§ (2, 6, 9, 11, 14, 16, 20)	0.82	0.81 (0.77 - 0.84)	0.89
	Cǁ (1, 10, 15, 18)
Growth	SC¶ (4, 8, 13, 17, 19)	0.62	0.59 (0.50 - 0.67)	0.81
	I** (3, 5, 7, 12)
Control	SC¶ (4, 8, 13, 17, 19)	0.72	0.70 (0.64 - 0.76)	0.93
	SM§(2, 6, 9, 11, 14, 16, 20)
Spirituality	Cǁ (1, 10, 15, 18)	0.65	0.64 (0.56 - 0.71)	0.87
	I** (3, 5, 7, 12)
α* total	0.82
ICC (95%)† total	0,81 (0,77 - 0,85)

• ^*
  α = Cronbach’s Alpha; ^†ICC = Intraclass Correlation Coefficient; ^‡Spearman’s correlation coefficient (ρ), p < 0.0001; ^§SM = System maintenance; ^¶SC = System Change; ^ǁC = Coherence; ^**I = Individuation.

DISCUSSION

This is the first psychometric analysis study to test the ASF-E/Brazil reliability and validity for use with Brazilian families and the results obtained confirmed the validity and reliability. Content validity, carried out by an expert committee, contributed to the semantic, idiomatic, conceptual and experiential assessment. Adequacy of the Brazilian Portuguese version to the construct is evidenced by the agreement of experts, considering that the minimum agreement of 80% among judges can serve as a decision criterion on the relevance of the item to the factor that theoretically refers to28. Thus, it can be said that the alteration of items, carried out by this committee, aimed to make them more understandable from an experiential, conceptual and cultural point of view. This is due to the fact that everyone has experience with families and knowledge of systems thinking in approaching families, which is part of the theoretical foundations that support the ASF-E/Brazil construction and its construct. Therefore, given the cultural diversity of Brazilian families, terms and expressions that approximate Brazilian sociocultural reality and behavior were adapted.

The validity explored with the AFE allowed the permanence of all items in the ASF-E/Brazil. In the factor analysis, the dimension with the lowest number of items was individuation, a fact similar to what happened in the Finnish study9.

Factor analysis showed the reallocation of some items to other scales (dimensions) of the ASF-E/Brazil. In factor one, item 20 (somos livres para ser nós mesmos - three; alguns de nós gostariam de mais liberdade - two; temos de fazer o que nos é dito - one) was part of the system change and became part of the system maintenance dimension. In this dimension, strategies for decision-making, the establishment of rituals and traditions, and the definition of roles and norms are addressed in order to maintain family harmony, providing a sense of security and autonomy1. This demonstrated that individuals, in Brazilian culture, have freedom and autonomy in the family and keep the system working healthily.

In factor two, item 19 (Nossa família tem uma maneira tradicional de comemorar eventos - three; não prestamos muita atenção a eventos especiais - two; nossa família não tem nada para comemorar - one) integrated the system maintenance dimension and became part of the system change dimension, which concerns the incorporation of new events6. Cultural factors may have caused this factor to change, as Brazilian families need to be part of and belong to a social network that supports and fosters the growth/increase of family’s health.

In factor four, item five (Nossa família é mais apoiadora do que a maioria. Temos orgulho de fazer parte dela - three; Nossa família é como a maioria das outras. Nós temos nossos altos e baixos - two; Nossa familiar não é muito boa. A maioria das famílias parece se harmonizar melhor do que a nossa - one) made up the coherence dimension and became part of individuation. Coherence considers that harmonious relationships between family members can provide a sense of unity and family belonging through internalization of respect, love, concern for others and sharing of values and beliefs that allow for the creation of emotional connections necessary for the system to survive. Regarding individuation, it is related to personal identity, which includes roles and responsibilities in which talents are reinforced and initiatives and knowledge allow the incorporation of knowledge to have behaviors against personal/family and environmental pressures. It is represented by attitudes taken by family members to achieve particular interests and give meaning to life¹. In the results related to Brazilian families, their cultural aspects can be explained by respect for individuation, as it is possible to maintain a harmonious relationship with family members with respect to personal identity.

From CFA, it can be stated that the most significant items, in each dimension, are related to the theoretical model. In terms of coherence, the most significant item was 15, referring to opinion, demonstrating that, in general, due to the bond of the participants that comes from nuclear families (52%), extended families (37%) and single-parent families (11%), Brazilian families seem to accept the different ways of thinking of their members, which allows healthy relationships. In system change, item 17, related to decision-making, indicates that Brazilian families are, in general, free to adopt independent attitudes. In the system maintenance dimension, the most significant was item nine, related to solution to problems, which indicates that Brazilian families find support in their members in difficult times. Finally, in relation to individuation, the most significant were items three and seven, related to help and participation, reaffirming the cultural characteristics of Brazilian families who generally enjoy collaborating and participating in community activities.

The concurrent criterion validity presented a Spearman’s correlation coefficient (ρ) of 0.75 (p = 0.004) between the total scores of the FES and the ASF-E/Brazil, and these values indicate a high relationship between the two instruments in relation to to the measured construct26^-28.

The ASF-E/Brazil reliability test showed an overall internal consistency of 0.82, considered good, similar to the coefficients obtained in studies carried out in the United States6, Mexico7, Finland, Germany and Switzerland9^-10, with Cronbach’s α between 0.74 and 0.85 and higher than the study carried out in Colombia8.

Reliability was satisfactory for both isolated dimensions (system maintenance, seven items = 0.74; coherence, four items = 0.70) and the items were theoretically interpretable, which demonstrates that there was no methodological problem. On the contrary, the issue is related to the family proximity inherent in Brazilian culture and the fact that families found unity and a sense of belonging in the joint management of the family operation, i.e., they found coherence while maintaining the system.

Despite the condition in which system change and individuation were found, merged into factor two (System change (SC)/Individuation (I)), it was possible to separate them into factor three = system change, with Cronbach’s α of 0.57, and factor four = individuation, with Cronbach’s α of 0.54. For system maintenance and coherence, as there was a lack of clear distinction between these dimensions, both included factor one, with a Cronbach’s α of 0.80. In this sense, the values obtained for ICC were close to Cronbach’s α, confirming the resulting behavior of EFA.

From the grouping of items into the four EFA factors, the results obtained for the analysis of internal consistency were 0.54 for individuation (factor four), 0.57 for system change (factor three), 0.70 for consistency and 0.74 for system maintenance (both included factor one). The low values of Cronbach’s α of factors three and four may be related to the reduced number of items that these factors carried in the EFA, results similar to those obtained in a Finnish study10.

In the analysis of the internal consistency of the subscales (goals), coefficients ranged from 0.62 to 0.82. Based on international parameters and studies that assessed the reliability of this instrument in different contexts and cultures, it is considered that the reliability levels found reinforce the reliability of the ASF-E/Brazil for use with Brazilian families.

By using Spearman’s correlation coefficient, positive correlations support the understanding of the conceptual equivalence of the scale and, therefore, aspects of validity and reliability. The tests resulted in a useful instrument with subscales that express the dimensions of stability, control, growth and spirituality, with results that have acceptable reliability and conceptual validity. In this context, the ASF-E/Brazil, cross-culturally adapted to Brazilian Portuguese and its culture, should be considered of sufficient quality to be used in research in which the four goals of family functioning are relevant.

The limitation of this study is related to participants’ characteristics in the validation study, as they were all adults, living in an urban area in southern Brazil, so the results cannot be generalized to all Brazilian families. The implications for nursing practice with families are related to the evidence of proof of ASF-E/Brazil validity and reliability to assess the effectiveness of family functioning, to maintain, promote, restore and strengthen the health of Brazilian families living in Brazil or abroad, and also by the availability of the instrument developed by nursing, freely accessible and supported by the Framework of Systemic Organization, to support the achievement of objectives of health that families want.

The ASF-E/Brazil is shown in Chart 1 (supplementary material). Additional information about the Systemic Organization theoretical model1 can be accessed in the doctoral thesis and on the website of the author of the original instrument1

Chart 1
Instrument of Assessment Strategies in Families-Effectiveness ASF-E/Brazil. Pelotas, RS, Brazil, 2018.

Por favor, responda a estas perguntas sobre sua família. “Família” aqui significa todas as pessoas que você considera como sendo sua família: todos os membros, parentes e até mesmo amigos que são como membros da família. Família são as pessoas que você sente emocionalmente próximas ou com as quais você fica chateado. Elas podem viver em sua casa ou viver em outro lugar, mas têm laços estreitos com você. Com a sua família em mente, por favor, siga estas instruções:
De cada grupo de três afirmações, escolha a que é mais parecida com a sua família, marcando um X nessa alternativa. Se houver duas que se encaixam, escolha a melhor.
#1	C	Nossa família é feliz de um modo geral.	Há raiva ou tristeza em nossa família.	As pessoas, em nossa família, não demonstram muito seus sentimentos.
		3	1	2
#2	SM	Nós nunca concordamos sobre quem, na família, deve ter o direito de fazer o quê.	Às vezes, um membro da nossa família é descuidado e/ou não tem consideração.	Geralmente, não pensamos apenas em nós mesmos, mas nos outros membros da família também.
		1	2	3
#3	I	Se há trabalho a ser feito na comunidade, às vezes, nós ajudamos, caso isso traga algum benefício para a nossa família.	Se há trabalho a ser feito na comunidade, normalmente, não participamos porque não nos sentimos parte dela.	Se há trabalho a ser feito na comunidade, gostamos de contribuir.
		2	1	3
#4	SC	Não estamos felizes porque estamos presos em um bairro onde não queremos estar.	Nós gostamos do nosso bairro porque as pessoas, ao nosso redor, são amigáveis.	Nosso bairro é um lugar ruim e temos de nos proteger das coisas que acontecem nele.
		2	3	1
#5	I	Nossa família é mais apoiadora do que a maioria. Temos orgulho de fazer parte dela.	Nossa família é como a maioria das outras. Nós temos nossos altos e baixos.	Nossa família não é muito boa. A maioria das famílias parece harmonizar-se melhor do que a nossa.
		3	2	1
#6	SM	As pessoas da nossa família sentem que ninguém as entende.	As pessoas da nossa família, às vezes, sentem-se mal compreendidas.	Sabemos quando há algo de errado com um de nós.
		1	2	3
#7	I	Participamos de atividades que a comunidade, a igreja ou a escola oferece porque gostamos de fazer parte do que está acontecendo.	Não temos tempo, nem energia, para nos envolver nas atividades realizadas pela comunidade, pela igreja ou pela escola.	Às vezes, participamos de atividades na comunidade, igreja ou escola, se alguém nos convidar.
		3	1	2
#8	SC	Seria fácil pedir ajuda a nossos amigos, pois eles se preocupam com a gente.	Pedir ajuda em uma organização comunitária ou uma igreja seria mais fácil do que pedir aos nossos amigos.	Se precisássemos de ajuda, não teríamos para onde ir.
		3	2	1
#9	SM	Um problema sério em nossa família causa desentendimento e dor.	Se temos um problema sério, todos nós pensamos sobre isso de forma muito diferente.	Se temos um problema sério, todos nós enfrentamos juntos.
		1	2	3
#10	C	Alguns membros de nossa família fazem todo o trabalho e outros não fazem nada.	Todos, na família, ajudam tanto quanto possível para que o trabalho seja concluído.	Alguns membros da família poderiam ajudar mais.
		1	3	2
#11	SM	Com frequência, alguém discorda da forma como as decisões são tomadas.	Nossa família tem uma boa maneira de tomar decisões.	Não conseguimos tomar decisões sem ferir os sentimentos de alguém.
		2	3	1
#12	I	Não confiamos em organizações como as escolas ou o local de trabalho. Geralmente, ficamos quietos e fazemos nossas próprias coisas.	Se temos problemas com organizações, como as escolas ou o local de trabalho, lutamos por nossos direitos.	Se temos problemas com organizações, como as escolas ou o local de trabalho, geralmente, conversamos sobre isso com as pessoas envolvidas.
		1	2	3
#13	SC	A maioria dos nossos amigos não entende nossos problemas se contamos a eles.	Podemos compartilhar problemas com nossos amigos, pois eles são prestativos.	Não falamos sobre problemas familiares com amigos.
		1	3	2
#14	SM	Nossa família estaria melhor se não fosse por alguns problemas que tivemos.	Problemas causaram muita infelicidade em nossa família.	Nossa família aprendeu lições a partir de problemas que tivemos.
		2	1	3
#15	C	Nossa família é de opinião.	Mesmo as opiniões muito incomuns são respeitadas.	Não podemos dizer o que realmente pensamos.
		2	3	1
#16	SM	Na nossa família, não nos importamos, de verdade, uns com os outros.	Em nossa família, sentimo-nos mais próximos de alguns do que de outros.	Sentimo-nos muito próximos em nossa família.
		1	2	3
#17	SC	Nunca decidimos nada sozinhos.	Em nossa família, tomamos decisões independentes.	Pedimos a aprovação da família para tomar decisões pessoais.
		1	3	2
#18	C	Nós preferimos ficar longe da família.	Basicamente, nossa família nos faz sentir seguros e felizes.	Há alguns momentos em que estar com os membros da família faz com que nos sintamos bem e, em outros, preferimos não estar com eles.
		1	3	2
#19	SC	Não prestamos muita atenção a eventos especiais.	Nossa família não tem nada para comemorar.	Nossa família tem uma maneira tradicional de comemorar eventos.
		2	1	3
#20	SM	Somos livres para ser nós mesmos.	Alguns de nós gostariam de mais liberdade.	Temos de fazer o que nos é dito.
		3	2	1

• C = Coherence/Coerência; I= Individuation/Individuação; SC= System Change/Mudança no sistema; SM= System Maintenance/Manutenção do Sistema.

CONCLUSION

The results of the psychometric tests of the ASF-E/Brazil, to assess validity of content, construct, criteria and reliability for use with Brazilian families, presented relevant information that demonstrates validity and reliability. The ASF-E/Brazil can contribute to nurses’ work with families and/or professionals interested in an instrument that can reliably assess the effectiveness of family functioning in different contexts.

ACKNOWLEDGMENT

We are thankful to CAPES, for granting the doctoral scholarship to the first author, and to CNPQ for granting the research productivity scholarship to the second author.

REFERENCES

• 1. Friedemann M-L. The framework of systemic organization: a conceptual approach to families and nursing. Thousand Oaks, CA(US): SAGE Publications, Inc.; 1995.
• 2. International Family Nursing Association (IFNA). Position statement on advanced practice competencies for family nursing [Internet]. 2017 [cited 2020 Nov 12]. Available from: https://internationalfamilynursing.org/wordpress/wp-content/uploads/2017/05/APC-Print-friendly-PDF-document-in-color-no-photos-English-language.pdf
  » https://internationalfamilynursing.org/wordpress/wp-content/uploads/2017/05/APC-Print-friendly-PDF-document-in-color-no-photos-English-language.pdf
• 3. International Family Nursing Association (IFNA). Position statement on generalist competencies for family nursing practice [Internet]. 2015 [cited 2020 Nov 12]. Available from: https://internationalfamilynursing.org/wordpress/wp-content/uploads/2015/07/GC-Complete-PDF-document-in-color-with-photos-English-language.pdf
  » https://internationalfamilynursing.org/wordpress/wp-content/uploads/2015/07/GC-Complete-PDF-document-in-color-with-photos-English-language.pdf
• 4. Lise F, Schwartz E, Spagnolo LM, Lange C, Dall'Agnol J. Assessment and intervention strategies for families of children and adolescents: Integrative literature review. Rev Pesq Cuid Fundam Online [Internet]. 2020 Jan-Dec [cited 2020 Nov 12];12:984-91. Available from: https://doi.org/10.9789/2175-5361.rpcfo.v12.7145
  » https://doi.org/10.9789/2175-5361.rpcfo.v12.7145
• 5. Vianna VPT, Silva EA, Souza-Formigoni MLO. Versão em português da Family Environment Scale: aplicação e validação. Rev Saúde Pública [Internet]. 2007 Jun [cited 2020 Nov 12];41(3):419-26. Available from: https://doi.org/10.1590/S0034-89102007000300014
  » https://doi.org/10.1590/S0034-89102007000300014
• 6. Friedemann ML. An instrument to evaluate effectiveness in family functioning. West J Nurs Res [Internet]. 1991 Apr 1 [cited 2020 Nov 12];13(2):220-36. Available from: https://doi.org/10.1177/019394599101300205
  » https://doi.org/10.1177/019394599101300205
• 7. Chávez M, Friedemann ML, Alcorta GA. Evaluación de la escala de efectividad en el funcionamiento familiar. Desarrollo Cientif Enferm. 2000; 8(1):12-18.
• 8. Bustamante EG, Ávila IC, Montes CED, Luis A, Cogollo Z. Validez y confiabilidad de la versión en español de la escala de evaluación de la funcionalidad familiar (ASF-E-20). Av Enferm [Internet]. 2013 Jul-Dec [cited 2020 Nov 12];31(2):21-9. Available from: http://www.scielo.org.co/pdf/aven/v31n2/v31n2a03.pdf
  » http://www.scielo.org.co/pdf/aven/v31n2/v31n2a03.pdf
• 9. Köhlen C, Friedemann M-L. Überprüfung eines familien-the assessment-instruments auf der Grundlage der Theorie des systemischen Gleichgewichts. Pflege [Internet]. 2006 Feb [cited 2020 Nov 12];19(1):23-32. Available from: https://doi.org/10.1024/1012-5302.19.1.23
  » https://doi.org/10.1024/1012-5302.19.1.23
• 10. Åstedt-Kurki P, Friedemann ML, Paavilainen E, Tammentie T, Paunonen-Ilmonen M. Assessment of strategies in families tested by Finnish families. Int J Nurs Studies [Internet]. 2001 Feb 1 [cited 2020 Nov 12];38(1):17-24. Available from: https://doi.org/10.1016/S0020-7489(00)00056-0
  » https://doi.org/10.1016/S0020-7489(00)00056-0
• 11. Oliveira F de, Kuznier TP, Souza CC, Chianca TCM. Aspectos teóricos e metodológicos para adaptação cutural e validação de instrumentos na enfermagem. Texto Contexto Enferm [Internet]. 2018 [cited 2020 Nov 12];27(2):e4900016. Available Available from: https://doi.org/10.1590/0104-070720180004900016
  » from: https://doi.org/10.1590/0104-070720180004900016
• 12. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) [Internet]. 2000 Dec 15 [cited 2020 Nov 12];25(24):3186-91. Available Available from: https://doi.org/10.1097/00007632-200012150-00014
  » from: https://doi.org/10.1097/00007632-200012150-00014
• 13. Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health [Internet]. 1997 Jun [cited 2020 Nov 12];20(3):269-74. Available from: https://doi.org/10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G
  » https://doi.org/10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G
• 14. Gjersing L, Caplehorn JRM, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol [Internet]. 2010 Feb 10 [cited 2020 Nov 12];10:13. Available from: https://doi.org/10.1186/1471-2288-10-13
  » https://doi.org/10.1186/1471-2288-10-13
• 15. Canhota C. Qual a importância do estudo piloto? In: Silva EE, org. Investigação passo a passo: perguntas e respostas para investigação clínica. Lisboa (PT): APMCG; 2008. p. 69-72.
• 16. Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6th ed. Porto Alegre, RS(BR): Bookman, 2009.
• 17. Moos RH, Moos BS. Family environment scale manual. 3rd ed. Palo Alto, CA(US): Consulting Psychologists Press; 1994.
• 18. Tilden VP, Nelson CA, May BA. The IPR inventory: development and psychometric characteristics. Nurs Res [Internet]. 1990 Nov-Dec [cited 2020 Nov 12];39(6):337-43. Available from: https://doi.org/10.1097/00006199-199011000-00004
  » https://doi.org/10.1097/00006199-199011000-00004
• 19. Hyrkäs K, Appelqvist-Schmidlechner K, Oksa L. Validating an instrument for clinical supervision using an expert panel. Int J Nurs Stud [Internet]. 2003 Aug [cited 2020 Nov 12];40(6):619-25. Available from: https://doi.org/10.1016/S0020-7489(03)00036-1
  » https://doi.org/10.1016/S0020-7489(03)00036-1
• 20. Wynd CA, Schmidt B, Schaefer MA. Two quantitative approaches for estimating content validity. West J Nurs Res [Internet]. 2003 Aug 1 [cited 2020 Nov 12];25(5):508-18. Available from: https://doi.org/10.1177/0193945903252998
  » https://doi.org/10.1177/0193945903252998
• 21. Burns N, Grove SK. The practice of nursing research appraisal, synthesis, and generation of evidence. 6th ed. St. Louis, MO(US): Saunders; 2009.
• 22. Jolliffe IT. Graphical representation of data using principal components. In: Jolliffe IT. Principal component analysis. New York, NY(US): Springer; 2002. p 78-110.
• 23. Kaiser HF. An index of factorial simplicity. Psychometrika [Internet]. 1974 Mar [cited 2020 Nov 12];39(1):31-6. Available from: https://doi.org/10.1007/BF02291575
  » https://doi.org/10.1007/BF02291575
• 24. Coleta MFD. A medida da satisfação conjugal: adaptação de uma escala. Psico. 1989;18(2):90-112.
• 25. Kline RB. Principles and practice of structural equation modeling. New York, NY(US): The Guilford Press; 2011.
• 26. Bonett DG, Wrigh TA. Cronbach's alpha reliability: interval estimation, hypothesis testing, and sample size planning. J Organ Behav [Internet]. 2015 Jan [cited 2020 Nov 12];36(1):3-15. Available from: https://doi.org/10.1002/job.1960
  » https://doi.org/10.1002/job.1960
• 27. Waltz CF, Strickland OL, Lenz ER. Measurement in nursing and health research. 4th ed. Philadelphia, PA(US): Springer Publishing Company; 2010.
• 28. Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. Petrópolis, RJ(BR): Editora Vozes Limitada; 2017.

NOTES

• ORIGIN OF THE ARTICLE

  Article extracted from the thesis - Adaptação transcultural e validação do instrumento The Assessment of Strategies in Family-Effectiveness ASF-E para uso com famílias brasileiras, presented to the Graduate Program in Nursing at the Universidade Federal de Pelotas in 2020.

• FUNDING INFORMATION

  This study was financed by the Coordination for the Improvement of Higher Education Personnel - Brazil (CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior) - Sandwich Abroad Doctoral Program Process -88881.189659/2018-01.

• APPROVAL OF ETHICS COMMITTEE IN RESEARCH

  Approved by the Ethics Committee in Research the Universidade Federal de Pelotas, under Opinion 2.088.369/2017 and CAAE (Certificado de Apresentação para Apreciação Ética - Certificate of Presentation for Ethical Consideration) 68095317.3.0000.5316.

Edited by

EDITORS

Associated Editors: Gisele Cristina Manfrini, Natália Gonçalves, Monica Motta Lino. Editor-in-chief: Roberta Costa.

Publication Dates

• Publication in this collection
  09 Feb 2022
• Date of issue
  2022

History

• Received
  23 Nov 2020
• Accepted
  24 Aug 2021