Acessibilidade / Reportar erro

Validity and accuracy of the Guedes Tool for the evaluation of informal social support for older adults

Abstract

The objective of the present study was to confirm the validity of the Guedes Tool, an instrument for assessing informal social support (ISS) for older adults, and to estimate its accuracy. Confirmatory factor analysis (CFA) was performed with the CFI and RMSEA indexes and the chi-square relationship (X2) with degrees of freedom (gl). For the analysis of accuracy, the area under the ROC curve, sensitivity ​​(S) and specificity (SP) values, positive predictive value (PV+), negative predictive value (PV-) and the Younder’s J Index (J) were verified to confirm the best cut-off point. Data collection was carried out with older adults from Natal and metropolitan region in 2018. The inclusion criteria were: be 60 years old or older, with preserved cognitive levels. Two hundred and six older adults participated in the CFA study and 197 participated in the accuracy analysis. The estimates of the indexes evaluated in the CFA were: X2/gl =1.33, RMSEA=0.04 (95% CI 0.025-0.054) and CFI=0.91. The ROC curve obtained an area of ​​0.78 (CI95: 0.72-0.85; p<0.001) for the determination of older adults with low ISS. The highest value J was 0.44 for a score ≤34, with an S value of 59.76% and an SP value of 84.96%. The instrument presented a well-adjusted model with four dimensions, according to CFA criteria. It had a good area under the ROC curve and good to moderate S and SP values for the cutoff value of 34 points or less, for the diagnosis of insufficient ISS. Good PV+ and PV- indicators confirmed the desirable levels of accuracy of the tool.

Keywords:
Social Support; Health of the Elderly; Validation Study; Factor Analysis; Statistical; Data Accuracy

Resumo

Os objetivos foram confirmar a validade do Guedes Tool, instrumento para avaliação do apoio social informal (ASI) para idosos, e estimar sua acurácia. Foi realizada análise fatorial confirmatória (AFC) com avaliação dos índices CFI, RMSEA e relação do qui-quadrado (X2 ) com graus de liberdade (gl). Para análise de acurácia foi verificada a área sob a curva ROC, valores de sensibilidade (S), especificidade (E), valor preditivo positivo (VP+), valor preditivo negativo (VP-) e o Índice J de Younder (J) para confirmar o melhor ponto de corte. A coleta de dados ocorreu com idosos de Natal e região metropolitana em 2018. Os critérios de inclusão foram: ter 60 anos ou mais, com níveis cognitivos preservados. Duzentos e seis idosos participaram do estudo de AFC e 197 participaram da análise de acurácia. As estimativas dos índices avaliados na AFC foram: X2/gl=1,33, RMSEA=0,04 (IC95% 0,025-0,054) e CFI=0,91. A curva ROC obteve área de 0,78 (IC95: 0,72-0,85; p<0,001) para a determinação de idosos com baixo ASI. O J de maior valor foi de 0,44 para pontuação ≤34, o qual tem uma S de 59,76% e E de 84,96%. O instrumento apresentou um modelo bem ajustado com quatro dimensões, segundo critérios da AFC. Apresentou boa área sob a curva ROC e boa E e S moderada para o valor de corte de 34 pontos ou menos para o diagnóstico do ASI insuficiente. Os bons indicadores de VP+ e VP- reforçaram sua acurácia desejável.

Palavras-chave:
Apoio Social; Saúde do Idoso; Estudos de Validação; Análise Fatorial; Acurácia de Dados

INTRODUCTION

The demographic profile of Brazil has been undergoing a transformation into a mainly urban society, with fewer children and a new structure for Brazilian families11 Lopes JM. Transição demográfica e epidemiológica. In: Lopes JM, Guedes MBOG. Fisioterapia na atenção primária: manual de prática profissional baseado em evidência. São Paulo: Atheneu; 2019. p. 21-6..22 Almeida-Filho N, Rouquayrol MZ. Introdução à Epidemiologia. 4ª ed. Rio de Janeiro: Guanabara Koogan; 2014.. Nowadays a growing contingent of people aged 60 or over22 Almeida-Filho N, Rouquayrol MZ. Introdução à Epidemiologia. 4ª ed. Rio de Janeiro: Guanabara Koogan; 2014. has led to a high prevalence of chronic diseases or conditions33 Guedes MBOG, Lopes JM. Abordagem familiar e do apoio social como estratégia de suporte ao cuidado na Atenção Primária. In: Lopes JM, Guedes MBOG. Fisioterapia na atenção primária: manual de prática profissional baseado em evidência. São Paulo: Atheneu; 2019. p: 79-90.. Thus, with new demands for the care of older adults, more complex models of care are required, which take into account the assessment of social determinants as a support strategy for the management of comprehensive care for this segment of the population44 Marmot M, Wilkinson RG. Social determinants of health. 2ª ed. New York: Oxford University Press; 2011.,55 Rethorn ZD, Cook C, Reneker JC. Social determinants of health: If you aren’t measuring them, you aren’t seeing the bg picture. J Orthop Sports Phys Ther. 2019;49(12):872-4. .

Formal and informal social support are important determinants for healthy aging, representing the set of resources provided by other individuals, and reflecting the totality of the relationships that a person has at their disposal44 Marmot M, Wilkinson RG. Social determinants of health. 2ª ed. New York: Oxford University Press; 2011.,66 Maia CML, Castro FV, Fonseca AMG, Fernández MIR. Redes de apoio social e de suporte social e envelhecimento ativo. INFAD Rev Psicol. 2016;1(1):293-303.. Insufficient social support is associated with several negative physical and mental health outcomes77 Aragão EIS, Portugal FB, Campos MR, Lopes CS, Fortes SLCL. Distintos padrões de apoio social percebido e sua associação com doenças físicas (hipertensão, diabetes) ou mentais no contexto da atenção primária. Ciência Saúde Colet. 2017;22(7):2367-74.. This social characteristic represents an intermediary between people’s behavior and their living conditions55 Rethorn ZD, Cook C, Reneker JC. Social determinants of health: If you aren’t measuring them, you aren’t seeing the bg picture. J Orthop Sports Phys Ther. 2019;49(12):872-4., and is possibly the most significant external condition that the health professional deals with.

Informal support networks simultaneously include family, friends, neighbors, spiritual counselors, and social groups such as clubs, associations, churches, and play an important role in supporting older adults from a social, emotional and instrumental point of view, allowing them to resolve many health problems without the intervention of official or professional institutions. Older adults tend to resort to formal support only when the structures of informal support have been exhausted88 Guedes MBOG, Lima KC, Caldas CP, Veras RP. Apoio social e o cuidado integral à saúde do idoso. Physis. 2017;27(4):1185-204..

Considering the importance of informal social support in a more complex analysis of the health-disease process of older adults, there is a need to assess this type of support with suitable, accurate instruments, so that errors of judgment are avoided. Therefore, a new instrument with effective psychometric indicators, which corresponds to the socio-cultural characteristics of the older Brazilian population and which assesses informal social support, is necessary for an accurate assessment of these aspects of social support for this population segment. There is no instrument in the literature that specifically evaluates such aspects for older adults Brazilian population88 Guedes MBOG, Lima KC, Caldas CP, Veras RP. Apoio social e o cuidado integral à saúde do idoso. Physis. 2017;27(4):1185-204..

Understanding the validation of instruments as a process, confirmatory factor analysis (CFA) is a step used to test hypotheses regarding certain constructs99 American Educational Research Association (AERA); American Psychological Association (APA); National Council on Measurement in Education (NCME). The standards for educational and psychological testing. New York: American Educational Research Association; 2014.. In this case, the researcher, guided by previous theory, tests to what extent certain variables are representative of a certain concept or dimensions1010 Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. São Paulo: Bookman; 2009..

Accuracy analysis is another decisive tool in the questionnaire validation process, as it defines parameters for diagnosis/prognosis and proposes more reliable reference measures to identify true cases or exclude false cases (sensitivity and specificity, for example), thus making the diagnostic assessment process more assertive1111 Medronho RA, Bloch KV, Luiz RR, Werneck GL. Epidemiologia. 2ª ed. São Paulo: Atheneu; 2009. p. 389-402.. Other steps in the validity process for this instrument have been previously developed, and have demonstrated good indicators of content validity and in the response process of the target population1212 Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 1. Rev Bras Geriatr Gerontol. 2018;21(6):671-80., as well as internal validity of the factors and items chosen from the exploratory factor analysis1313 Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 2. Rev Bras Geriatr Gerontol. 2019;22(2):1-11..

In view of the above, the present study aims to verify the psychometric quality of the Guedes Tool, an instrument for assessing informal social support for older adults and to estimate its diagnostic accuracy, based on CFA and analysis of the ROC curve, of the indicators of sensitivity, specificity, and negative and positive predictive value.

METHOD

The present study is an evaluation of psychometric and diagnostic indicators, with an observational design. It is of the transversal and analytical type and adopted a quantitative approach. The study was performed from October to December 2018, as part of the project entitled “Construction and Validation of a Social Support Scale for Older Adults”. This project included the stages of CFA and accuracy analysis.

The project was approved by the Research Ethics Committee of the Onofre Lopes University Hospital, under opinion number 1,644,533. The present study complies with the recommendations of National Health Council Resolutions nº 196/96 and nº 466/2012.

The target population of the study was composed of older Brazilian adults residing in the community. The source population was older adults living in the city of Natal, Rio Grande do Norte, Brazil.

As a sample, a minimum total of 200 participants was established. To calculate the sample for CFA, the total number of respondents followed a minimum proportion of ten respondents for each variable included in the instrument, which had 20 items, following criteria guided by Hair et al.1010 Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. São Paulo: Bookman; 2009..

For the design of the diagnostic accuracy of the Guedes Tool, a type I (α) error of 5%, type II (β) error (test power) of 20%, and a minimum area under the curve of 0.7 were taken into account, along with a null hypothesis area of 0.50 and a negative/positive ratio of up to 3:11414 Obuchowski NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res. 1998;7:371-92.. Thus, a minimum of 88 participants was required, with at least 22 positive cases and 66 negative cases.

The inclusion criteria of the study participants were an age of 60 (sixty) years old or older, with preserved cognitive levels (no report of a clinical diagnosis of cognitive deficit), capable of answering the proposed questions. These criteria were valid for both the CFA and accuracy stages. Only those who answered all the questions in the Guedes Tool were included in the CFA, while for the accuracy analysis only those who responded to the two questionnaires in their entirety, including the Social Support Scale (MOS-SSS), were included.

For both analyzes, data collection always took place in person, with a population aged 60 or over, from the city of Natal and its metropolitan region. Collection took place in an association for older adults, a public consultation center, a municipal park and through home visits in Natal, Rio Grande do Norte and metropolitan region (Santa Cruz, Macaíba, São Gonçalo do Amarante). All the participants signed an Informed Consent Form (ICF) before the interview.

The instruments were applied sequentially to each respondent, with the Guedes Tool being applied first. The average time for application of the two questionnaires was 25 minutes per respondent. Four interviewers, who were undergraduate students, were previously trained by one of the researchers who designed the instrument.

CFA was performed using the statistical program M PLUS Version 7®. This analysis was carried out to demonstrate how different indicators of informal social support can be reduced to represent four dimensions or factors, namely: x1, x2, x3 and x4. The technique of dimensional reduction by main factors was used, and oblique type goemin rotation was applied. Seven iteration points per dimension were used. All variables were treated as dichotomous categorical variables. Delta parameterization and the WLSMV estimator (weighted least square with diagonal weight matrix with standard errors and mean- and variance-adjusted chi-square test statistics) were used.

For validity purposes, the incremental index Comparative Fit Index (CFI) and the absolute indexes, Root Mean Square Error of Approximation (RMSEA), as well as the relationship between the chi-square value (X2) and degrees of freedom (df), were used. The following reference values were considered: CFI> 0.90; RMSEA<0.05; X2/df<3.01010 Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. São Paulo: Bookman; 2009..1515 Thompson B. Exploratory and confirmatory factor analysis: understanding concepts and applications. Appl Psychol Meas. 2007;31(3):245-8.,.

To assess accuracy, the data were analyzed descriptively and inferentially. The descriptive analysis obtained the summary and dispersion measures of the studied variables. Inferential analysis established the diagnostic validity of the Guedes Tool. The complete instrument with its items and the respective values assigned to them is shown in Table 1. The answers to this instrument were dichotomous, assigning the total value of the item when the answer was “yes”, and zero when the answer was “no”, for each question asked.

Table 1
Guedes Tool Questionnaire. Point values assigned to the items and the instrument. Natal, Rio Grande do Norte, Brazil.

Based on the responses of the respondents, the ROC (Receiver Operator Characteristics) curve, a diagnostic/prognostic validation technique, was developed, with the outcome of a low social support result of the dichotomous variable of the Social Support Scale (MOS-SSS). Values below 52 points were considered to be low social support for this instrument, which was taken as the gold standard1616 Zanini DS, Peixoto EM, Nakano TC. Escala de Apoio Social (MOS-SSS): proposta de normatização com referência nos itens. Trends Psychol. 2018;26(1):387-99.. The results of the gold standard test and the Guedes Tool were interpreted independently, without knowledge of the results of the other.

The area under the ROC curve was also calculated, as well as its 95% confidence interval (95% CI) and the associated probability (p-value). In addition, the accuracy of the Guedes Tool was obtained from the sensitivity (S) and specificity (SP) values, the positive predictive value (PV +) and the negative predictive value (PV-)1717 Boussuyt PMM. Interpreting Diagnostic Test Accuracy Studies. Semin Hematol. 2008;45(3):189-95..

To identify the best cutoff point for the Guedes Tool, Younder’s J Index (J) was used, which summarizes the diagnostic test performance for the data of the study participants, and which also served for dimensioning of the accuracy sample. This statistic is obtained as follows: J = sensitivity + specificity-1, where values range from 0 to 11818 Schisterman NJ, Perkins AL, Howard B. Optimal Cut-point and Its corresponding youden index to discriminate individuals using pooled blood samples. Epidemiology. 2005;16:73-81.. In all inferential analysis strategies, a significance level of ≤5% was adopted in an attempt to minimize the type I error. The analyzes were also stratified by gender and age group, in order to estimate different accuracy for these social conditions.

RESULTS

Two hundred and six older adults (112 from an older adults association, 42 from home visits, 30 from a public consultation center, and 22 from the Natal municipal park) participated in the study, with ages varying between 60-99 years and an average of 69.80 (± 7.63) years, and of whom 145 (70.7%) were women. The most common schooling level among participants was elementary (49.5%), followed by the illiterate (24.4%) and those with a high school education (20.5%). Of this total, nine older adults did not answer a question on the MOS-SSS social support scale and were excluded from the accuracy analysis, which therefore had a total of 197 respondents. The MOS-SSS instrument identified 82 (41.62%) older adults with low social support.

The estimates of the indexes evaluated in the CFA were as follows: X2/ gl=1.33, RMSEA=0.04 (95% CI 0.025-0.054) and CFI=0.91. As shown in Figure 1, the analysis of the ROC curve revealed a good area, of 0.78 (CI95: 0.72-0.85; p<0.001), for the determination of older adults with low social support. The Younder’s J Index with the highest value was 0.44 for the cutoff ≤34 in the Guedes Tool, which has a sensitivity of 59.76% and specificity of 84.96% (Table 2). However, other cut off points can be used depending on the purpose of their application, as shown in Table 2.

Figure 1
Analysis of the ROC curve area to determine the low social support of the Guedes Tool, with the MOS-SSS Social Support Scale as a reference. Natal, Rio Grande do Norte, Brazil, 2018.
Table 2
Cutoff points of the Guedes Tool and its parameters of sensitivity (Sen), specificity (Spec), and positive (PV +) and negative (PV-) predictive values for older adults. Natal, Rio Grande do Norte, Brazil, 2018.

When stratifying the ROC curve according to gender, it was observed that the area under the curve barely changed in comparison with the area of the total population evaluated, being 0.79 (95% CI: 0.68-0.91) for men and 0.79 (95% CI: 0.72-0.87) for women. The best cutoff point for men would be ≤40 with a Younder’s J Index of 0.50 (S=78.12%; SP=72.00%). For women, the best cutoff point is ≤34, with a Younder’s J Index of 0.46 (S=62.00%; SP=84.09%).

When we stratified the sample by age group, we identified an area under the ROC curve of 0.74 (95%CI: 0.64-0.84) for older adults aged up to 69 years (cut≤34; J=0.44; S=83.33%; SP=60.71%), 0.85 (95%CI:0.76-0.94) for those between 70 and 79 years (cut-off≤30; J=0.61; S=64.52%; SP=97.22%) and 0.72 (95%CI: 0.45-0.98) for ages over 80 years (cut-off≤36; J=0.51; S=66.67%; SP=85.71%).

DISCUSSION

The steps covered in this research were based on the previous stages of the construction of this instrument, among them, the proposition of items based on literature review88 Guedes MBOG, Lima KC, Caldas CP, Veras RP. Apoio social e o cuidado integral à saúde do idoso. Physis. 2017;27(4):1185-204., content-based validity, validity based on the response process1717 Boussuyt PMM. Interpreting Diagnostic Test Accuracy Studies. Semin Hematol. 2008;45(3):189-95. and exploratory factor analysis1818 Schisterman NJ, Perkins AL, Howard B. Optimal Cut-point and Its corresponding youden index to discriminate individuals using pooled blood samples. Epidemiology. 2005;16:73-81..

All the estimates of the absolute and incremental indexes demonstrated the adequate fit of the model with 20 items and four dimensions. The RMSEA obtained was within the desirable range. This absolute indicator assesses how well the model fits the population, while the goodness of fit values for this criterion indicate that the model has a good fit, and correct internal structure1010 Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. São Paulo: Bookman; 2009..

Regarding the second absolute indicator estimated, the chi-square analyzed in isolation demonstrated excellent results for this study. Variance, which depends on the sample size, was stable, and represents another important absolute indicator in determining the quality of fit of the model1111 Medronho RA, Bloch KV, Luiz RR, Werneck GL. Epidemiologia. 2ª ed. São Paulo: Atheneu; 2009. p. 389-402.,1616 Zanini DS, Peixoto EM, Nakano TC. Escala de Apoio Social (MOS-SSS): proposta de normatização com referência nos itens. Trends Psychol. 2018;26(1):387-99..

The CFI assessed is an incremental indicator that measures the relative improvement of the model in relation to a standard model. This standard model is typically considered to have a variance between variables equal to zero, with the closer to 1 (one) the CFI, the better the quality of the adjustment. The CFI value obtained for this instrument (0.91) was adequate.

The latent dimensions confirmed in the present study are the subject of discussions in scientific literature, and should therefore be justified in the present discussion, with a theoretical model provided for each one. Our decision to choose such domains is also based on previous studies regarding this instrument1212 Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 1. Rev Bras Geriatr Gerontol. 2018;21(6):671-80.,1313 Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 2. Rev Bras Geriatr Gerontol. 2019;22(2):1-11..

Studies have pointed out the importance of the composition and extension of the social network of individuals, such as having someone close on whom one can count, whether a family member, friend or neighbor1919 Brito TRP, Nunes DP, Duarte YAO, Lebrão ML. Redes sociais e funcionalidade em pessoas idosas: evidências do estudo Saúde, Bem-Estar e Envelhecimento (SABE). Rev Bras Epidemiol. 2018;21(2):1-15.

20 Sant´Ana LAJ, D´Elboux MJ. Comparação da rede de suporte social e a expectativa para o cuidado entre idosos em diferentes arranjos domiciliares. Rev Bras Geriatr Gerontol. 2019;22(3):1-12.

21 Gouveia OMR, Matos AD, Schouten MJ. Redes sociais e qualidade de vida dos idosos: uma revisão e análise crítica da literatura. Rev Bras Geriatr Gerontol. 2016;19(6):1030-40.
-2222 Corrêa CS, Queiroz BL, Fazito D. Relação entre tamanho e estrutura da rede de apoio e o tempo individual dedicado à atenção ao idoso na cidade de São Paulo, 2000. Rev Bras Est Pop. 2016;33(1):75-97., as well as how these structures are arranged and thus, how they can influence the physical and mental health of older adults2323 Rabelo DF, Neri A. Tipos de configuração familiar e condições de saúde física e psicológica em idosos. Cad Saúde Pública. 2015;31(4):874-84.. This was the first dimension presented in the instrument, and the score attributed to its items has a greater weight than the score of the items in the other dimensions. The justification for this greater weighting of items can be verified in a previous study of this instrument1313 Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 2. Rev Bras Geriatr Gerontol. 2019;22(2):1-11..

The solicitude and support provided during the performance of activities, addressed in the dimension of instrumental support and availability, is important for the maintenance of adequate informal social support, and can be a fundamental tool in the promotion of self-care for older adults and, consequently, a support mechanism for such individuals and public and private health services2424 Almeida L, Bastos PRHO. Autocuidado do Idoso: revisão sistemática da literatura. Rev Espacios. 2017;38(28):3-13..

Informal social support, when considered as a two way street, can generate a feeling of appreciation for both the older adults and the other social actors involved2525 Figueiredo MLG. Participação social e redes sociais pessoais de idosos. Dissertação de Mestrado. Coimbra: Instituto Superior Miguel Torga; 2016.,2626 Canesqui AM, Barsaglini RA. Social support and health: standpoints from the social and human sciences. Ciênc Saúde Colet. 2012;17(5):1103-14.. The perception of adequate social involvement throughout life, on the other hand, may be associated with a better quality of life for older adults, with positive repercussions on their health and independence2727 Pinto FNFR, Oliveira DC. Capacidade funcional e envolvimento social em idosos: há relação? Rev Bras Ciênc Envelh Hum. 2015;12(1):56-68.. These aspects were addressed in the reciprocity and longitudinality dimension.

Emotional support and social participation are important requirements for the informal social support of older adults. The feeling of positive insertion in a social context can generate important feelings of welcome and appreciation for older adults and those who participate in their social network2525 Figueiredo MLG. Participação social e redes sociais pessoais de idosos. Dissertação de Mestrado. Coimbra: Instituto Superior Miguel Torga; 2016.. Insufficient levels in these aspects of informal social support can lead to significant psychological damage to older adults2020 Sant´Ana LAJ, D´Elboux MJ. Comparação da rede de suporte social e a expectativa para o cuidado entre idosos em diferentes arranjos domiciliares. Rev Bras Geriatr Gerontol. 2019;22(3):1-12.,2828 Possatto JM, Rabelo DF. Condições de saúde psicológica, capacidade funcional e suporte social de idosos. Rev Kairós. 2017;20(2):45-58..

Regarding the analysis of accuracy, the choice of the social support scale (MOS-SSS) as the gold standard is justified, as this instrument has good psychometric construct validity indicators for the Brazilian population2929 Griep RA, Chor D, Faerstein E, Werneck GL, Lopes CS. Validade de constructo de escala de apoio social do Medical Outcomes Study adaptada para o português no Estudo Pró-Saúde. Cad Saúde Pública. 2005;21(3):703-14.. The instrument also underwent recent standardization of its cutoff points1414 Obuchowski NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res. 1998;7:371-92.. Although in both the studies cited, the sample was not exclusively limited to older adults, this instrument was originally designed for an older population, and contains items relevant for the assessment of social support among older adults3030 Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med. 1991;38:705-14..

Considering the results of the total sample of this research, a score of 34 points or less for considering the informal social support for older adults as insufficient provided good specificity and moderate sensitivity, as well as good indicators for the positive and negative predictive values.1717 Boussuyt PMM. Interpreting Diagnostic Test Accuracy Studies. Semin Hematol. 2008;45(3):189-95.. In addition, the area on the ROC curve obtained was desirable, demonstrating an acceptable degree of accuracy at this cutoff point for this population1111 Medronho RA, Bloch KV, Luiz RR, Werneck GL. Epidemiologia. 2ª ed. São Paulo: Atheneu; 2009. p. 389-402..

Other cutoff points may be applicable, depending on the objective and priorities of the observer. If the option is for a higher sensitivity or if the priority is for greater specificity, this reference can be adjusted, according to the results obtained in this study.

When stratifying the sample for both sex and age groups, the ROC curve did not undergo major changes. Thus, the use of this cutoff point for the overall older population is correct. However, in specific research cases, the observer may select, for example, cutoff points to diagnose insufficient informal social support for men (40 points or less) or women (34 points or less), as the patterns of perception of social support may differ between men and women77 Aragão EIS, Portugal FB, Campos MR, Lopes CS, Fortes SLCL. Distintos padrões de apoio social percebido e sua associação com doenças físicas (hipertensão, diabetes) ou mentais no contexto da atenção primária. Ciência Saúde Colet. 2017;22(7):2367-74..

When stratifying for age groups, in all groups the area of the ROC curve was desirable. However, the group of older adults aged 80 and over did not exhibit statistically significant indicators, and their cut-off point should be used with caution. We therefore suggest that the value obtained for the overall sample be considered eligible, regardless of the age group. Thus, we consider that the accuracy of this instrument does not depend on the stratification of age for the older adult group.

This instrument was designed with the diagnosis of insufficient informal social support for the older population in mind. However, the positive accuracy indicators related to the MOS-SSS scale for the diagnosis of social support for the overall population, emphasizes that the application of the Guedes Tool for other populations cannot be ruled out. Its broad approach in other dimensions which are common to several vulnerable groups, not just older adults, should also be considered

The sample was representative for both stages of this study (confirmatory factor analysis and accuracy). Data collection in various environments minimized errors in relation to selection bias. A considerable number of participants of both sexes, with different ages and varying levels of education, resulted in a desirable heterogeneity of the sample, considering that the instrument is being developed for the overall older population. The prior calibration of the interviewers reduced the chances of errors relating to the observer during the interviews (information bias)1111 Medronho RA, Bloch KV, Luiz RR, Werneck GL. Epidemiologia. 2ª ed. São Paulo: Atheneu; 2009. p. 389-402..

Considering the aforementioned aspects, this instrument is an important tool for epidemiological screening to diagnose insufficient informal social support among older adults, and can be applied in various health services and levels of health care, especially in primary care, to provide a more complex assessment perspective that contemplates the demands of an expanded concept in health44 Marmot M, Wilkinson RG. Social determinants of health. 2ª ed. New York: Oxford University Press; 2011.,55 Rethorn ZD, Cook C, Reneker JC. Social determinants of health: If you aren’t measuring them, you aren’t seeing the bg picture. J Orthop Sports Phys Ther. 2019;49(12):872-4..

Regarding the limitations of the study, it should be mentioned that, despite having respondents from a range of environments, the sample was convenience based and concentrated in only one region of Brazil. For a country of continental size, with a population with a wide range of socio-cultural and economic aspects, studies in other locations are important. Previous steps in the development of this instrument have taken a broader approach, with respondents from the five regions of Brazil1313 Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 2. Rev Bras Geriatr Gerontol. 2019;22(2):1-11..

Another relevant question refers to the failure to carry out a direct assessment of the cognitive capacity of the respondents, which increases the risk of older adults with cognitive deficits answering the questionnaire, impairing the analysis and interpretation of the data, as this exclusion criterion was assessed from the self-reporting of the interviewees.

The “information” dimension for assessing social support is discussed in scientific literature1616 Zanini DS, Peixoto EM, Nakano TC. Escala de Apoio Social (MOS-SSS): proposta de normatização com referência nos itens. Trends Psychol. 2018;26(1):387-99. and was not addressed in this instrument. Addressing all the aspects of a complex construct such as social support, is a great challenge, and could lead to the creation of an excessively extensive instrument. Therefore, in professional practice, the evaluator should apply care in relation this aspect when evaluating older adults.

Considering the obtaining of evidence of the validity of instruments as a process of constant improvement, it is important to emphasize that other steps, in different population groups should be carried out, such as, for example, cross-cultural adaptations to other languages, and confirmatory factor analysis for other vulnerable groups.

CONCLUSIONS

The Guedes Tool instrument presented a well-adjusted model with four dimensions, according to criteria obtained by confirmatory factor analysis. This indicates an internal structure with items and dimensions of good psychometric quality.

The Guedes Tool exhibited a good area under the ROC curve and good specificity and moderate sensitivity for the cutoff value of 34 points or less for the diagnosis of insufficient informal social support, in relation to the total sample of older adults evaluated. The good indicators of positive and negative predictive values for this reference cut-off point reinforce the desirable accuracy of the instrument. Other steps for improving the process of obtaining evidence of validity are important.

  • No funding was received in relation to the present study.

REFERENCES

  • 1
    Lopes JM. Transição demográfica e epidemiológica. In: Lopes JM, Guedes MBOG. Fisioterapia na atenção primária: manual de prática profissional baseado em evidência. São Paulo: Atheneu; 2019. p. 21-6.
  • 2
    Almeida-Filho N, Rouquayrol MZ. Introdução à Epidemiologia. 4ª ed. Rio de Janeiro: Guanabara Koogan; 2014.
  • 3
    Guedes MBOG, Lopes JM. Abordagem familiar e do apoio social como estratégia de suporte ao cuidado na Atenção Primária. In: Lopes JM, Guedes MBOG. Fisioterapia na atenção primária: manual de prática profissional baseado em evidência. São Paulo: Atheneu; 2019. p: 79-90.
  • 4
    Marmot M, Wilkinson RG. Social determinants of health. 2ª ed. New York: Oxford University Press; 2011.
  • 5
    Rethorn ZD, Cook C, Reneker JC. Social determinants of health: If you aren’t measuring them, you aren’t seeing the bg picture. J Orthop Sports Phys Ther. 2019;49(12):872-4.
  • 6
    Maia CML, Castro FV, Fonseca AMG, Fernández MIR. Redes de apoio social e de suporte social e envelhecimento ativo. INFAD Rev Psicol. 2016;1(1):293-303.
  • 7
    Aragão EIS, Portugal FB, Campos MR, Lopes CS, Fortes SLCL. Distintos padrões de apoio social percebido e sua associação com doenças físicas (hipertensão, diabetes) ou mentais no contexto da atenção primária. Ciência Saúde Colet. 2017;22(7):2367-74.
  • 8
    Guedes MBOG, Lima KC, Caldas CP, Veras RP. Apoio social e o cuidado integral à saúde do idoso. Physis. 2017;27(4):1185-204.
  • 9
    American Educational Research Association (AERA); American Psychological Association (APA); National Council on Measurement in Education (NCME). The standards for educational and psychological testing. New York: American Educational Research Association; 2014.
  • 10
    Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. São Paulo: Bookman; 2009.
  • 11
    Medronho RA, Bloch KV, Luiz RR, Werneck GL. Epidemiologia. 2ª ed. São Paulo: Atheneu; 2009. p. 389-402.
  • 12
    Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 1. Rev Bras Geriatr Gerontol. 2018;21(6):671-80.
  • 13
    Guedes MBOG, Lima KC, Lima AL, Guedes TSR. Validation of a questionnaire for the evaluation of informal social support for the elderly: section 2. Rev Bras Geriatr Gerontol. 2019;22(2):1-11.
  • 14
    Obuchowski NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res. 1998;7:371-92.
  • 15
    Thompson B. Exploratory and confirmatory factor analysis: understanding concepts and applications. Appl Psychol Meas. 2007;31(3):245-8.
  • 16
    Zanini DS, Peixoto EM, Nakano TC. Escala de Apoio Social (MOS-SSS): proposta de normatização com referência nos itens. Trends Psychol. 2018;26(1):387-99.
  • 17
    Boussuyt PMM. Interpreting Diagnostic Test Accuracy Studies. Semin Hematol. 2008;45(3):189-95.
  • 18
    Schisterman NJ, Perkins AL, Howard B. Optimal Cut-point and Its corresponding youden index to discriminate individuals using pooled blood samples. Epidemiology. 2005;16:73-81.
  • 19
    Brito TRP, Nunes DP, Duarte YAO, Lebrão ML. Redes sociais e funcionalidade em pessoas idosas: evidências do estudo Saúde, Bem-Estar e Envelhecimento (SABE). Rev Bras Epidemiol. 2018;21(2):1-15.
  • 20
    Sant´Ana LAJ, D´Elboux MJ. Comparação da rede de suporte social e a expectativa para o cuidado entre idosos em diferentes arranjos domiciliares. Rev Bras Geriatr Gerontol. 2019;22(3):1-12.
  • 21
    Gouveia OMR, Matos AD, Schouten MJ. Redes sociais e qualidade de vida dos idosos: uma revisão e análise crítica da literatura. Rev Bras Geriatr Gerontol. 2016;19(6):1030-40.
  • 22
    Corrêa CS, Queiroz BL, Fazito D. Relação entre tamanho e estrutura da rede de apoio e o tempo individual dedicado à atenção ao idoso na cidade de São Paulo, 2000. Rev Bras Est Pop. 2016;33(1):75-97.
  • 23
    Rabelo DF, Neri A. Tipos de configuração familiar e condições de saúde física e psicológica em idosos. Cad Saúde Pública. 2015;31(4):874-84.
  • 24
    Almeida L, Bastos PRHO. Autocuidado do Idoso: revisão sistemática da literatura. Rev Espacios. 2017;38(28):3-13.
  • 25
    Figueiredo MLG. Participação social e redes sociais pessoais de idosos. Dissertação de Mestrado. Coimbra: Instituto Superior Miguel Torga; 2016.
  • 26
    Canesqui AM, Barsaglini RA. Social support and health: standpoints from the social and human sciences. Ciênc Saúde Colet. 2012;17(5):1103-14.
  • 27
    Pinto FNFR, Oliveira DC. Capacidade funcional e envolvimento social em idosos: há relação? Rev Bras Ciênc Envelh Hum. 2015;12(1):56-68.
  • 28
    Possatto JM, Rabelo DF. Condições de saúde psicológica, capacidade funcional e suporte social de idosos. Rev Kairós. 2017;20(2):45-58.
  • 29
    Griep RA, Chor D, Faerstein E, Werneck GL, Lopes CS. Validade de constructo de escala de apoio social do Medical Outcomes Study adaptada para o português no Estudo Pró-Saúde. Cad Saúde Pública. 2005;21(3):703-14.
  • 30
    Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med. 1991;38:705-14.

Edited by

Edited by: Ana Carolina Lima Cavaletti

Publication Dates

  • Publication in this collection
    18 Sept 2020
  • Date of issue
    2020

History

  • Received
    01 Mar 2020
  • Accepted
    10 Aug 2020
Universidade do Estado do Rio Janeiro Rua São Francisco Xavier, 524 - Bloco F, 20559-900 Rio de Janeiro - RJ Brasil, Tel.: (55 21) 2334-0168 - Rio de Janeiro - RJ - Brazil
E-mail: revistabgg@gmail.com