Validation of a questionnaire for the evaluation of informal social support for the elderly: section 1

Abstract

Objectives:

to construct and carry out content (CV) and response process (RPV) validation for a questionnaire to assess informal social support for the elderly.

Method:

a descriptive, observational, quantitative study was performed between January and December 2016 in the city of Natal (Rio Grande do Norte) and other locations in Brazil. The inclusion criteria were: proven experience in the area of social support (for experts) or 60 years of age or older and with preserved cognitive status (for the elderly). The CV stage evaluated the relevance of the items according to the general Content Validity Index (CVI) and per item as well as the assembly of the panel based on the observations of the experts. In the RPV stage, the understanding of the items by the target audience was evaluated.

Results:

the CV stage included a total of 40 interviewees. The overall CVI was 0.88 and only one item had a CVI considered poor. In the RPV stage 41 people were interviewed.

Conclusion:

the questionnaire exhibited good relevance for the proposed items and the observations of the interviewees allowed an approximation of the language used in the instrument to the language of the elderly.

Keywoods:
Social Support; Health of the Elderly; Surveys and Questionnaires; Validation Studies

Resumo

Objetivos:

construir e desenvolver a validação de conteúdo (VC) e de processo baseado em respostas (VPR) de um questionário para avaliação do apoio social informal para idosos.

Método:

trata-se de uma pesquisa descritiva, observacional, do tipo quantitativa. Teve execução entre janeiro e dezembro de 2016, no município de Natal, Rio Grande do Norte, e em outras localidades do Brasil. Os critérios de inclusão foram: experiência comprovada na área de apoio social (para especialistas) ou ter 60 anos ou mais e ter capacidade cognitiva preservada (para os idosos). Na etapa de VC, avaliou-se a relevância dos itens segundo Índice de Validade de Conteúdo (IVC) geral e por item e montagem de painel com as observações feitas pelos especialistas. Na etapa de VPR, avaliou-se o entendimento referente aos itens pelo público alvo.

Resultados:

na VC, obteve-se um total de 40 entrevistados. O IVC geral foi de 0,88 e apenas um item teve IVC considerado pobre. Na VPR, obteve-se um total de 41 entrevistados.

Conclusão:

o questionário demonstrou uma boa relevância dos itens propostos e as observações feitas pelos entrevistados permitiram uma aproximação da linguagem utilizada no instrumento à linguagem usada pelos idosos.

Palavras-chaves:
Apoio Social; Saúde do Idoso; Inquéritos e Questionários; Estudos de Validação

INTRODUCTION

Social relations, when inadequate, constitute an obvious health risk, comparable to others proven to be harmful, such as smoking, high blood pressure, obesity and sedentary lifestyle, and may have clinical implications for people's health¹. Social support (SS) seems to have a broad impact in many aspects of people's lives, especially in populations that are vulnerable in social, psychological and health terms, as is the case of the elderly².

SS composes the social and individual coping resources in which people base their responses to everyday needs and stressful situations³. An individual’s assessment of their SS as successful has been related to several positive outcomes in physical and mental health, influencing how stressful situations are perceived, emotional and psychological well-being and even longevity^3,4,5.

The differentiation of social support relationships according to their content, process and development can directly impact the adaptation of individuals to their social environment. From this perspective, SS must be evaluated by distinguishing its types in accordance with the relationships that give rise to it⁶. In scientific literature it is common to observe a differentiation between Formal Social Support (FSS), which includes state services, social security, and diverse organizations, such as church groups and health professionals, among others, and Informal Social Support (ISS), which includes the network of relatives, friends and neighbors, for example^7,8. Isolating and breaking links within the ISS network could increase the vulnerability of individuals to illness and become a barrier to achieving active aging. ISS, therefore, is the main source of support for many elderly people and, on many occasions, the only one^9,10.

In light of the above and other assumptions regarding scientific evidence in the evaluation process¹¹, the importance of using instruments constructed and validated for the analysis of ISS is clear, due to the scarcity of resources in literature for the evaluation of this specific construct³. Most such instruments deal with Formal Social Support or do not separate the constructs from one another among the elderly population^12,13,14.

Within the logic of the systematization of the evaluation process, it is important that such instruments follow the methodological rigor of the stages of the validation process^15,16. The careful evaluation of ISS can become excessively complex, abstract and subjective, if validated evaluation tools are not used in the research process. There are multiple observation variables, which can cause difficulty for exclusively subjective observations.

Validity refers to the ability of an instrument to accurately measure the phenomenon being studied, or in other words, to what extent it can actually assess its objective¹⁷. For the construction and validation of a questionnaire, the steps of "content-based validity" and "response process validity " are required, which can be carried out by qualitative and quantitative procedures, according to the proposed methodology^15,17-19. When validation is considered as a process, the publication of steps already carried out for a certain instrument foments academic discussion and consequently, the creation of new researches that corroborate its validity. Other steps to improve the instrument, such as Exploratory and Confirmatory Factor analysis, accuracy analysis and cross-cultural adaptations, are necessary and will be dealt with in later sections, as some have already been performed by the research group responsible.

Content-based validity assesses the degree to which each element of an instrument of measurement is relevant and representative of a specific construct with a particular purpose for evaluation, and is usually developed through a comprehensive literature review, consulting experts by interviews, group discussion (focus groups) or panel building, followed by assessment by judges^18,20.

Response process validity encompasses the analysis of interviews with lay people potentially related to the study population and the subsequent evaluation of judges. This step ensures the correction of sentences and an approximation of the language used in the instrument to the language frequently used by the target population, thus discarding terms that were not clear for this audience^15,17,20. Unfortunately, most authors do not describe these steps in detail. Therefore, the purpose of this section is to analyze the content and response process validity of a questionnaire that seeks to evaluate Informal Social Support for the Elderly.

METHOD

Research Characteristics

A descriptive, observational, quantitative type research was carried out. The present study took place between January and October 2016, as part of the project "Construction and Validation of a Social Support Scale for the Elderly". This research project comprised two stages of the process of the construction and validation of the questionnaire: content and response process validity^15,18.

Data Collection

The first stage referred to content-based validity, where the judges selected the items after a thorough review of the literature for the preparation of the first version of the instrument¹⁶. Subsequently, the relevance of the items was judged by experts. The initial construction of the questions concerning the first model of the questionnaire was therefore carried out by two academic judges from the area of Social Support, with the choice of each item always consensual, following a review of literature. The questionnaire was then issued to specialists via the internet, using the following inclusion criteria: authors of scientific articles in the area, professors of Psychology and Social Services courses (those who train professionals working close to the evaluation and management of social support among the elderly) and professionals who worked directly with the theme.

The sample at this stage was selected for convenience and included 40 experts. Contact was made following the creation of an e-mail database of the authors responsible for the publication of articles available in the virtual computer network and contact with professors available on the websites of their respective institutions of higher education or the personal invitation of professionals in the area. The invitation was simultaneously sent to all experts through the SurveyMonkey^® research tool. The letter of invitation contained the following words: "Social support is an important determinant for the health of a range of elderly people. This study aims to construct and validate a questionnaire for the evaluation of Informal Social Support for the elderly. This is the initial stage of the research, in which researchers and experts participate in the theme of Informal Social Support, to evaluate the relevance of the proposed items and possible changes or the addition of new items. Your participation is voluntary and will be given through the answers made in this questionnaire. If you feel comfortable answering the questions, you will be making a significant contribution to our research." All participants accepted the free and informed consent form.

The second stage of response process validity was also carried out with a convenience sample, totaling 41 people (target population), in face-to-face interviews, selected in a long-term institution, an association for the elderly and a public health system clinic, all located in the city of Natal, Rio Grande do Norte (RN).

The inclusion criteria were: 60 years old or older and with cognitive ability maintained (no clinical diagnosis of cognitive deficit). All participants signed the informed consent form. In addition to ISS questions, respondents were asked about their age, gender, schooling, and income. At this stage, participants were asked whether they understood each item or not, were asked to repeat the questions, and encouraged to suggest changes. In addition, the non-verbal reactions of respondents were observed during the question process (e.g. facial expressions of doubt or discomfort) along with long response times^15,18,21. The maintenance, alteration or exclusion of the items occurred by the consensus decision of the judges after the construction of a panel with the information provided by the interviewees ^17,22.

Data Analysis

The questionnaire addressed to the experts asked for answers regarding level of schooling, as well as multiple choice and open questions relating to the proposed items with the following words: "1- Classify each question below according to degree of relevance for the evaluation of Social Support for the elderly" (responses in this question followed the Likert Scale structure: 1- Irrelevant, 2- Not Very Relevant, 3- Relevant, 4- Very Relevant, 5- Extremely Relevant); 2- "If you would like to suggest the inclusion of an item, do so in the text box below"; "3- If you would like to suggest changes to an item, please state the corresponding number and how you would like it changed in the text box below”¹⁷.

In order to determine the degree of relevance of the items and the overall relevance of the instrument, the Content Validity Index (CVI) was used to measure the degree of relevance of each item, based on the answers of the judges (responses 3, 4 or 5 were considered suitable), where: CVI by item = number of suitable judgments (answers 3+4+5)/total number of judgments. Items with a CVI greater than or equal to 0.8 were maintained²³. The overall CVI indicates the degree of relevance of the mean of the entire instrument, where Overall CVI = Overall Mean of all Suitable Responses/total number of judgments. The maintenance, alteration or exclusion of the items was based on the consensus decision of the judges after the construction of a panel with the information provided by the interviewees and the analysis of the CVI of the items^20,22,23.

Ethical Aspects

This project was approved by the Ethics Research Committee of the Hospital Universitário Onofre Lopes, under approval number 1.644.533 and CAAE 54608616.8.0000.5292. The present study complied with Resolutions nº 196/96 and nº 466/2012, of the National Health Council (CNS).

RESULTS

In terms of descriptive analysis, in the content-based validity stage with experts, 90% of the interviewees were doctors, 7.5% were masters and 2.5% were specialists. The response process validity phase involving the elderly included a total of 41 interviewees, 17 (41.5%) of whom were males, 24 (58.5%) of whom were women, and the mean age of whom was 70.87 (±8.01). The majority (39.0%) had completed primary education, followed by secondary education (26.8%), and higher education (22.0%), those with no education (7.3%) and those with a post-graduate qualification (4.9%). The average income was: 2439.02 reais (± 2792.86). Table 1 shows the results of the overall CVI and per item.

Table 1
First version of items proposed by judges following a review of literature and the Content Validity Index (CVI) by item and overall, according to the judgement of relevance by experts. Natal, Rio Grande do Norte, 2018.

Item	Content Value Index by item
1. Can sir/madam rely on many people that are close to him/her?	0.92
2. Does sir/madam live with many people?	0.82
3. Does sir/madam have close friends?	0.90
4. Does sir/madam have a close relative that lives nearby?	0.90
5. Does sir/madam have a friend that lives nearby?	0.92
6. Does sir/madam have a neighbor present?	0.88
7. Does sir/madam often pay visits other people?	0.88
8. Does sir/madam often have visitors?	0.90
9. Does sir/madam have anyone to talk to?	0.92
10. Does sir/madam have someone to help with the housework?	0.92
11. Does sir/madam have someone to help them leave the house when they need it?	0.92
12. Does sir/madam have someone to help when they are bedridden or sick?	0.95
13. Does sir/madam have someone to help when they have financial difficulties?	0.95
14. Does sir/madam take part in discussions about family decisions?	0.90
15. Does sir/madam take part in discussions about decisions among friends?	0.82
16. Does sir/madam take part in discussions about community decisions?	0.82
17. Does sir/madam listen to the problems of others when asked?	0.90
18. Does sir/madam comfort others when asked?	0.88
19. Does sir/madam share their leisure time with someone?	0.95
20.Is social contact with others enduring?	0.88
21.Was the help that sir received from others in the last 30 days adequate?	0.82
22. Did sir receive adequate help from other people when young?	0.65*
CVI - Overall	0.88

• *Value below reference value of 0.823.

Chart 1 provides suggestions for changes and the additions of new items made by experts. Chart 2 shows a comparison of before and after changes in the items after such suggestions. Chart 3 shows a comparison of before and after changes in the items after the suggestions of the elderly. Chart 4 presents the proposal of the questionnaire after validity steps based on the content and process of responses.

Chart 1
Observations for amendments and suggestions for the inclusion of items of specialists accepted by judges. Natal, Rio Grande do Norte, 2018.

Observations of specialists accepted
Restriction to Informal Social Support in the header
Use of the singular to facilitate understanding
Reduction of the number of words in some sentences
Use "you" instead of "sir/madam"
Suggestions for the inclusion of items by specialists accepted
When you are sad or miss someone do you have anyone to talk to?
Do you have close relatives who help you look after yourself when you need it?

Chart 2
Items with and without alterations, following specific suggestions of specialists accepted by judges. Natal, Rio Grande do Norte, 2018.

Unaltered item	Reformulated Item
Did sir/madam receive adequate help from other people when young?	When you were young, did you receive adequate help from other people?
Does sir/madam take part in discussions about family decisions?	Do you take part in family decisions?
Does sir/madam participate in discussions about decisions among friends?	Do you take part in decisions among friends?
Does sir/madam listen to the problems of others when asked?	Do you listen to the problems of others when asked?
Does sir/madam often have visitors?	Do you often have visitors?
Does sir/madam have a friend that lives nearby?	Do you have a friend you see often?
Was the help that sir received from others in the last 30 days adequate?	Was the help you received in the last 30 days satisfactory?
Does sir/madam have a neighbor present?	Do you have a neighbours you can rely on when required?
Does sir/madam have a relative that lives nearby?	Do you have a relative you can rely on and who lives nearby?
Is social contact with other people enduring?	Is your social contact with other people enduring?

Chart 3
Items with and without alterations following suggestions of target audience accepted by judges. Natal, Rio Grande do Norte, 2018.

Unaltered item	Reformulated Item
Do you have a relative that you can rely on and who lives nearby?	Do you have a family member that you can rely on and who lives nearby?
Do you often pay visits?	Do you often visit other people?
Do you have someone who can help you leave the house when you need it?	Do you have someone who can help you leave the house in case you need it?
Do you have someone to help when you’re bedridden or sick?	Do you have someone to help in case you’re bedridden or sick?
Do you have someone to help when you have financial difficulties?	Do you have someone to help in case you have financial difficulties?
Do you take part in family decisions?	Have you taken part in a family decision?
Do you listen to the problems of others when asked?	Do you help others when asked?
Do you comfort others when asked?	Do you console others when they’re sad?
Is your social contact with others enduring?	Is your social contact with others permanent?
Was the help that you had or would have had in the last 30 days satisfactory?	Was the help that you had or would have had in the last 30 days satisfactory, or would it have been?

Chart 4
Informal Social Support Questionnaire for the Elderly following content and response process evaluation stages. Natal, Rio Grande do Norte, 2018.

Informal Social Support is all emotional, financial, material or daily support for activities of daily living, received by a network of people (relatives, friends, neighbors, among others). INDICATE THE OPTION THAT YOU BELIEVE TO BE MOST SUITABLE FOR EACH ITEM REFERRING TO INFORMAL SOCIAL SUPPORT AMONG THE ELDERLY
Item/Response alternatives	YES	NO
1.Can you rely on people close to you?
2. Do you live with many people?
3. Do you have a friend you see often?
4. Do you have a family member you can rely on and who lives nearby?
5. Do you have a friend that lives nearby?
6. Do you have a neighbor who you can rely on in case you need it?
7. Do you often visit other people?
8. Do you often have visitors?
9. Do you have someone you can talk to?
10. Do you have someone to help with the housework?
11. Do you have someone to help you leave the house in case you need to?
12. Do you have someone to help you in case you’re bedridden or sick?
13. Do you have someone to help you in case of financial difficulty?
14. Have you taken part in a family decision?
15. Do you take part in decisions among friends?
16. Have you taken part in a community decision?
17. Do you help other people when asked?
18. Do you console other people when they’re sad?
19. Do you share your leisure time with anyone?
20. Is your social contact with other people permanent?
Item/Response alternatives	YES	NO
21. Was the help that you had or would have had in the last 30 days satisfactory, or would it have been?
22. Have you received adequate help from other people throughout your life?
23. When you are sad or miss someone do you have anyone to talk to?
24. Do you have a family member who helps you look after yourself in case you need it?

DISCUSSION

Some points for reflection concerning the construction and validation of instruments should be highlighted. It is important to understand this process as a collection of resources and procedures that are organized as a process and not as a finished product. The stages of the validation process should complement each other and the level of evidence obtained by the instrument to measure a given construct or phenomenon should be constantly evolving^15,21,24,25.

In terms of descriptive analysis, a high degree of education and training was observed among the expert participants regarding the content-based validity sample, which could reduce response errors caused by a failure to understand the questions or the purpose of the study. All levels of education were represented among the respondents of the response process validity stage, from "uneducated" to formal postgraduate education, with a quantitative balance between men and women. In addition, data collection was performed in both a long-term institution and in public places. These aspects reinforce the applicability of the questionnaire whether in an institutionalized or non-institutionalized population.

Content-based validity was measured using a panel of experts¹⁷. This tool was integrated into the research as part of a preliminary phase, contributing to the establishing of the bases for research. This phase was important to support the eradication of bias and false evidence about the construct studied^17,20,26.

The general observations made by the experts regarding the instrument resulted in a reduction in the length of sentences, making them more objective, which could reduce the final application time of the questionnaire¹⁸. The language of the sentences was also improved, which could strengthen the link between the respondent and the process of applying the instrument and thus increase adherence, leading to all the questions being answered, and also facilitate the understanding of the interviewee, reducing response bias²⁷. It was suggested that the questionnaire was restricted to the object of study of "Informal Social Support", which was not clear in the first version and was also accepted by the judges.

The experts also suggested the inclusion of some items. Accepted suggestions covered important questions that include variables that directly influence perceived informal social support and are presented as important dimensions for the evaluation of the construct studied. These are: "emotional support and social participation" and instrumental support or availability^28-30.

Regarding the CVI, the overall index was satisfactory, demonstrating the general relevance of the items initially included in the instrument²³. Item twenty-two, "Did you receive adequate help from other people when you were young?", was the only question with an unsatisfactory performance according to the item based CVI²³ and was restructured in a way that facilitated understanding and gave the idea of perceived informal social support throughout life. It was therefore restructured as follows: "Have you received adequate help from other people throughout your life?”.

On some occasions, experts can be an authority on the subject studied, but when suggesting issues pertinent to the topic, may do so in a language of debatable comprehension for the target audience. The response process validity stage is therefore important to bring the language used in the instrument closer to the target audience's usual language, making it potentially more comprehensible^15,21.

The collection of direct and indirect indicators, such as the answers about the understanding of each question and the reaction of the interviewees to each question, was important in this stage of interviews with the elderly²¹. The documentation of the reactions and perceptions about the understanding of the questions by the target population, with the subsequent creation of the panel for this population, allowed the specificities of this group to be captured, which was only possible through face-to-face interviews.

The choice of the format of answers for the elderly, in terms of the understanding or not of each question, was dichotomous instead of using, for example, a Likert type scale. This format of responses could facilitate the interviewee's understanding and reduce the amount of random responses¹⁸. Regarding the analysis of the responses of the target population, in terms of the understanding of each item, the analysis was specific by item, with the aim of maximizing the understanding of the sentence according to the objective of evaluation proposed by the question and reducing the possibility of "does not apply” type answers”²². The specificities of the studied population were therefore taken into account in the analysis of the acceptance or rejection of the proposed changes.

The questions used a language close to that most commonly spoken by the elderly, so that they still maintained the basic principles of the cultured norm of the Portuguese language and that understanding was potentialized¹⁸. For example, in the question "do you listen to the problems of others when asked?", many elderly persons understood "listen to the problems of others" as something negative, so the solution found was to rephrase the question "do you help other people when asked?”.

The phrase "in case of" was used to refine some questions, in order to reduce the possibility of "not applicable" answers. For example, the question "do you have someone to help when you have financial difficulties?" includes the possibility that some people have never had financial difficulties. The phrase was therefore rephrased as "do you have someone to help you in case of financial difficulty?" A similar situation occurred in the sentence that was rephrased as "do you have someone to help you leave the house in case you need it?”

At times, relatively simple terms created confusion for the respondents. For example, many elderly persons confused "have visitors" (receber visitas in Portuguese) with "pay visits to" (realizar visitas in Portuguese), so the term was restructured to "do you often visit?". The term "enduring" was unusual for this audience and was changed to "permanent". The same occurred with the term "comfort", which was modified to "console”.

In general, the construct of Informal Social Support was familiar and easy to understand for the elderly persons interviewed, mainly because the questions were common and had few technical or scientific terms. However, some reactions of surprise or delays in responding or negative responses of understanding or difficulty in repeating the question that had just been read were noted, as well as suggestions of alterations by the interviewees, guided the reformulation of questions which frequently presented problems.

Limitations of the study included the fact that the answers of the experts were entirely collected in online format, which made it difficult for the responding professionals to explain their concerns about the items, which may favor the emergence of response bias in some cases. In addition, the respondents were not asked for their locality. The sample used in the face-to-face interviews with the target audience was from only one city. Ideally, this should be taken from different cities and regions of Brazil, in order to include the desired cultural variety among the interviewees. It is worth highlighting the importance of other validation steps for this instrument, such as Factor Analysis, accuracy and reproducibility analysis, among others, which will be considered in later sections to reduce the length of each article and ensure that each stage receives the proper attention.

CONCLUSIONS

The content-based validity stage identified the effective relevance of the proposed items through positive indicators of both the overall and item content validity index. The suggestions of the specialists led to the inclusion of two items and the improvement of the other initially proposed questions. The response process validity allowed an approximation of the language used in the instrument to that employed by the elderly. A better understanding of the issues by the target population can result in more reliable responses and more effective implementation of the instrument, and other steps of the validation process are necessary.

REFERENCES

• ¹
  Andrade GRB, Vaitsman J. Apoio social e redes: conectando solidariedade e saúde. Ciênc Saúde Colet. 2002;7(4):925-34.
• ²
  Johnson ER, Carson TL, Affuso O, Hardy CM, Baskin ML. Relationship between social support and body mass index among overweight and obese african american women in the rural deep south, 2011-2013. Prev Chronic Dis. 2014;11(224):1-9.
• ³
  Gonçalves TR, Pawlowski J, Bandeira DR, Piccinini CA. Avaliação de apoio social em estudos brasileiros: aspectos conceituais e instrumentos. Ciênc Saúde Colet. 2011;16(3):1755-69.
• ⁴
  Melchiorre MG, Chiatti C, Lamura G, Torres-Gonzales F, Stankunas M, Linder J, et al. Social support, socio-economic status, health and abuse among older people in seven european countries. PLoS ONE. 2013;8(1):548-56.
• ⁵
  Boen H, Dalgard OS, Bjertness E. The importance of social support in the associations between psychological distress and somatic health problems and socio-economic factors among older adults living at home: a cross sectional study. BMC Geriatr. 2012;12(27):1-12.
• ⁶
  Ornelas J. Suporte social: origens, conceitos e áreas de investigacão. Anál Psicol. 1994;2-3(12):333-9.
• ⁷
  Martins RML. A relevância do apoio social na velhice. Rev Millenium. 2005;(31):128-34.
• ⁸
  Chatters LM, Taylor RJ, Jackson JS. Sizeand compositions of the informal helper networks of elderly blacks. J Gerontol. 1985;40(5):605-14.
• ⁹
  Maia CML, Castro FV, Fonseca AMG, Fernández MIR. Redes de apoio social e de suporte social e envelhecimento ativo. INFAD Rev Psicol. 2016;1(1):293-303.
• ¹⁰
  Neri AL, Vieira LAM. Envolvimento social e suporte social percebido na velhice. Rev Bras Geriatr Gerontol. 2013;16(3):419-32.
• ¹¹
  Salmond SS. Evaluating the reliability and validity of measurement instruments. J Orthop Nurs. 2008;27(1):28-30.
• ¹²
  Squassoni CE, Matsukura TS, Panúncio-Pinto MP. Versão brasileira do Social Support Appraisals: estudos de confiabilidade e validade. Rev Ter Ocup. 2016;27(1):1-11.
• ¹³
  Domingues MAR. Mapa Mínimo de Relações do Idoso: análise de reprodutibilidade. Rev Kairos. 2011;14(6):153-66.
• ¹⁴
  Griep RA, Chor D, Faerstein E, Werneck GL, Lopes CS. Validade de constructo de escala de apoio social do Medical Outcomes Study adaptada para o português no Estudo Pró-Saúde. Cad Saúde Pública. 2005;21(3):703-14.
• ¹⁵
  American Educational Research Association. American Psychological Association (APA). National Council on Measurement in Education (NCME). The standards for educational and psychological testing. New York: AERA; 2014.
• ¹⁶
  Guedes MBOG, Lima KC, Caldas CP, Veras RP. Apoio social e o cuidado integral à saúde do idoso. Physis. 2017;27(4):1185-204.
• ¹⁷
  Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Colet. 2011;16(7):3061-8.
• ¹⁸
  Vieira S. Como elaborar questionários. São Paulo: Atlas; 2009.
• ¹⁹
  Pasquali L. Validade dos testes psicológicos: será possível reencontrar o caminho? Psicol Teor Pesqui. 2007;23(esp):99-107.
• ²⁰
  Pinheiro JQ, Farias TM, Abe-Lima JY. Painel de especialistas e estratégia multimétodos: reflexões, exemplos, perspectivas. Psico. 2013;44(2):184-92.
• ²¹
  Padilha JL, Benítez I. Validity evidence based on response processes. Psicothema. 2014;26(1):136-44.
• ²²
  Gibbs G. Análise de dados qualitativos. Porto Alegre: Art Med; 2009.
• ²³
  Terwee CB, Prinsen CAC, Chiarotto A, De Vet HCW, Bouter LM, Alonso J, et al. COSMIN methodology for assessing the content validity of PROMs . User manual version 1.0. Amsterdan: [publisher unknown]; 2018.
• ²⁴
  Rios J, Welles C. Validity evidence based on internal structure. Psicothema. 2014;26(1):108-16.
• ²⁵
  Sireci S, Padilla JL. Validating assessments: introduction to the special section. Psicothema. 2014;26(1):97-9.
• ²⁶
  Davidson M, Keating J. Patient-reported outcome measures (PROMs): how should I interpret reports of measurement properties? a practical guide for clinicians and researchers who are not biostatisticians. Br J Sports Med. 2014;48:792-6.
• ²⁷
  Amatuzzi MLL, Barreto MCC, Julio Litvoc J, Leme LEG. Linguagem metodológica. Act Ortop Bras. 2006;14(2):108-12.
• ²⁸
  Rath T, Panigrahi D. Instrumental social support for the rural elderly: study of a rural block of a costal district of Odisha. Int J Community Med Public Health. 2017;4(7):2320-6.
• ²⁹
  Shumaker SC, Susan KF, Debra KM, Misook LC. Psychometric properties of the Multidimensional Scale of Perceived Social Support in Patients With Heart Failure. J Nurs Meas. 2017;25(1): 90-102.
• ³⁰
  García-Martín MA, Hombrados-Mendieta I, Gómez-Jacinto L. A Multidimensional approach to social support: the Questionnaire on the Frequency of and Satisfaction with Social Support (QFSSS). An Psicol. 2016;32(2):501-15.

• Funding:

  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - (CAPES) - Brazil - Code 001. Scholarship for doctorate.

Publication Dates

• Publication in this collection
  Nov-Dec 2018

History

• Received
  24 June 2018
• Reviewed
  21 Sept 2018
• Accepted
  28 Sept 2018