Content validation of the Nutritional Attention Assessment Instrument in Primary Health Care

ABSTRACT

Objective

Construct and validate the content of an instrument to evaluate the organization of nutritional care in Primary Health Care in Brazil.

Methods

This is a cross-sectional and validation study. The construction of the Nutritional Attention Assessment Instrument was based on literature review and review of official documents. The instrument was submitted to content validation using the Delphi technique in two rounds.

Results

The validation panel consisted of 29 specialists from the five Brazilian Macroregions: 13 higher education teachers/investigators, seven food and nutrition managers, four Primary Health Care professionals and five professionals knowledgeable of the subject, most of them (89.7%) working in the public service with experience of over six years. The validated instrument contains 68 questions and 10 indicators: (1) Nutritionist performance; (2) Support to the Nutritional Attention actions: infrastructure and permanent education; (3) Intersectoriality; (4) Social Control; (5) Food and Nutrition Surveillance; (6) Individual Nutritional Attention; (7) Nutritional Attention for Groups; (8) Nutritional Attention focused on prenatal care; (9) Nutritional Attention focused on postpartum/breastfeeding and (10) Nutritional Attention focused on child health.

Conclusion

The proposed instrument is an advance which enables the establishment of evaluation processes for Nutritional Attention. It is expected that this instrument be widely used to evaluate the Nutritional Attention given in the municipalities, states and at the federal level. The results obtained with future applications may contribute to support management qualification of food and nutrition policies.

Keywords
Evaluation studies as topic; Primary health care; Nutrition programs and policies; Nutrition policy; Nutritionists

RESUMO

Objetivo

Construir e validar o conteúdo de um instrumento para avaliar a organização da atenção nutricional na Atenção Primária à Saúde no Brasil.

Métodos

Trata-se de um estudo transversal e de validação. A construção do Instrumento de Avaliação da Atenção Nutricional foi baseada em revisão de literatura e em consulta a documentos oficiais. O instrumento foi submetido à validação de conteúdo utilizando técnica Delphi em duas rodadas.

Resultados

O painel para validação foi formado por 29 especialistas das cinco Macrorregiões Brasileiras: 13 docentes/pesquisadores do ensino superior, sete gestores de alimentação e de nutrição, quatro profissionais da Atenção Primária à Saúde e cinco alinhados ao tema, a maioria (89,7%) atuante em serviços públicos, com experiência acima de seis anos. O instrumento validado contém 68 questões e 10 indicadores: (1) Atuação do nutricionista; (2) Apoio às ações de Atenção Nutricional: infraestrutura e educação permanente; (3) Intersetorialidade; (4) Controle Social; (5) Vigilância Alimentar e Nutricional; (6) Atenção Nutricional Individual; (7) Atenção Nutricional para grupos; (8) Atenção Nutricional voltada ao pré-natal; (9) Atenção Nutricional voltada ao puerpério/aleitamento e (10) Atenção Nutricional à saúde da criança.

Conclusão

O instrumento proposto avança ao viabilizar o estabelecimento de processos avaliativos da Atenção Nutricional. Espera-se que ele seja amplamente utilizado para avaliar a atenção nutricional ofertada em municípios, em estados e na esfera federal. Os resultados obtidos com futuras aplicações poderão contribuir para respaldar a qualificação da gestão das políticas de alimentação e de nutrição.

Palavras-chave
Estudos de avaliação como assunto; Atenção primária à saúde; Programas e políticas de nutrição e alimentação; Política nutricional; Nutricionistas

INTRODUCTION

The major cause of death worldwide is due to Chronic Noncommunicable Diseases (NCD) [¹,²]. Between 1990 and 2017, a 40% increase in premature deaths and NCD-related disabilities occurred worldwide [²]. In Brazil, such diseases also represent the major cause of mortality [³].

Inadequate diet is one of the main risk factors for these diseases [¹]. In 2017, 11 million deaths were attributed to these diseases, representing an increase of 18.9%, when compared to the year 2007 [⁴,⁵].

Worldwide, in 2015, obesity took its toll of approximately 4 million people deaths and removed more than 100 million people from productive activities [⁶,⁷]. In Brazil, in 2019, 20.3% adults were obese [⁸]. Regarding children between five and nine years of age, 16.6% were obese [⁹].

Children with obesity at two years of age are 75% more likely to become obese at 35 years of age and Breastfeeding (BF) can protect them from NCD [¹⁰,¹¹]. This is what a recent meta-analysis has demonstrated revealing a negative association between BF, childhood and adulthood obesity [¹²,¹³]. Such findings enhance the importance of Nutritional Attention (NA) in Primary Health Care (PHC) in prenatal and puerperium, as a strategy to implement maternal and child care and encourage BF, aiming at the prevention of NCD and obesity [¹⁴,¹⁵].

In international review studies, the need for NA in PHC is evident, due to its role in encouraging the improvement of food consumption, glycemic control and nutritional status [¹⁶]. Furthermore, it has a significant impact on health outcomes related to obesity, malnutrition and NCD, such as reducing the risk of developing diabetes, besides generating savings for the health systems [¹⁷,¹⁸]. Thus, it is necessary to strengthen and qualify NA in this area [¹⁶,¹⁷].

In Brazil, although it is part of the PHC program agenda, NA occurs in a fragmented and not universal way [¹⁹]. The evaluation seems neglected, even though it is essential to identify gaps in its implementation. The need for investigations that describe and evaluate the actions and programs of Food and Nutrition (F&N) which is underway in PHC has already been pointed out [²⁰]. Furthermore, the Política Nacional de Alimentação e Nutrição (PNAN, National Food and Nutrition Policy) suggests assessment of F&N actions and the Política Nacional de Atenção Básica (PNAB, National Primary Care Policy) highlights these actions as relevant to the Brazilian population [²¹,²²].

Despite the recognition of the role of NA in PHC, no records of its evaluation were found in major journals, which is a knowledge gap. Likewise, there are no studies on an NC assessment instrument in this area of care that includes a questionnaire and indicators. The purpose of this work was to build and validate the content of an instrument for the assessment of the NA organization in PHC, named Instrumento de Avaliação da Atenção Nutricional (IAAN, Nutritional Attention Assessment Instrument).

METHODS

The construction of the IAAN was based on the Brazilian definition of NA, which is associated with the “care related to F&N aimed at the promotion and protection of health, prevention, diagnosis and treatment of diseases”; it makes up the comprehensive care in the Rede de Atenção à Saúde (RAS, Health Care Network) of the Sistema Único de Saúde (SUS, Unified Health System) and is intended for individuals, families and communities [²¹].

The development of the IAAN was carried out in five stages, shown in Chart 1: stage 1 was an extensive bibliographic review to identify the F&N actions foreseen for PHC in Brazil [²³]. In stage 2, an online review of the contents of each IAAN indicator was performed by investigators in the area of Alimentação e Nutrição em Saúde Coletiva (ANSC, Food and Nutrition in Public Health) [²⁴]. In stage 3, the IAAN Consultation Instrument (CI) was developed. In stage 4, the CI and the IAAN were submitted to an online pilot study [²⁵]. From stage 4, the IAAN composition was obtained, based on the recommendations and guidelines of the NA in the PHC. Finally, in stage 5, the instrument’s content validation process was carried out, resulting in consensus among specialists, through the application of the Delphi technique [²⁶].

Chart 1
Development stages of the Nutritional Attention Assessment Instrument in Primary Health Care, according to the Delphi Technique. Brazil, 2017.

1. Literature review IAAN (version 1)	2. IAAN Review (version 2)	3. Preparation of the IAAN Consultation Instrument	4. Consultation Instrument/Final version of Pilot Study	5. IAAN Content Validation
Database (2010-2107) 716 scientific articles Official documents: 22 documents	Researcher’s evaluation in the UNIFESP area (4 participants)	Assertions for assessing issues and indicators on clarity, conciseness, relevance and sufficiency	Sending the CI to researchers in the ANSC area of the State of São Paulo (14 were invited and 5 participants)	Sending CI to nutritionists selected for IAAN content validation
IAAN: 133 questions, 123 of evaluation, distributed in 18 blocks	IAAN: 53 questions, 43 of evaluation, distributed in 10 indicators	CI: 106 assertions	CI: 112 assertions; IAAN: 56 questions, 46 of evaluation, distributed in 10 indicators	IAAN Validated (Final version)

• Note: CI: Consultation Instrument; IAAN: Instrumento de Avaliação da Atenção Nutricional; UNIFESP: Universidade Federal de São Paulo.

Instrumento de Avaliação da Atenção Nutricional content validation was performed, using the Delphi technique, a method based on the approach to consensus through an experts’ panel judgment on a problem [²⁶]. This technique is widely used in health studies, such as, for example, building consensus on curriculum, training, professional skills, clinical practice and interventions and also for validating models of action and development of indicators [²⁷*³⁴].

Content validation is fundamental for any evaluation instrument, as it demonstrates that the choice and format of the items consistently and adequately represent the content that is intended to be measured, in addition to being a quality criterion of that content [³⁵,³⁶]. Expert judgment is used as a tool for content validation [³⁷]. The process consists of the successive submission of the instrument to a group of experts, in a sufficient number of rounds, to identify, elucidate and improve issues in order to reach consensus in the end [³⁸].

The main characteristics of this method are anonymity, which reduces the influence of factors such as academic or professional status, feedback of the group responses, and the possibility of using electronic means for consultation, facilitating the participation of experts from different locations, reducing costs and allowing all experts to express their opinion without interference of the other participants [³⁹,⁴⁰].

To compose the experts’ panel, nutritionists from all Brazilian macro-regions were invited. The selection of these professionals is justified on account of their specific skills in promoting improvement of the actions of Nutritional Attention with the teams from the Núcleo Ampliado de Saúde da Família e Atenção Básica (eNASF-AB, Expanded Family Health and Primary Health Care, from the Estratégia Saúde da Família (FHS, Family Health Strategy and from PHC [³⁴,⁴¹].

Although, in the PNAN, the strategic role of multiprofessional teams is reported, the nutritionist is the reference professional in setting the matrix of F&N actions at PHC [⁴¹]. Hence the importance of developing the eNASF-AB, FHS and PHC qualifications, by promoting permanent education [⁴²]. Furthermore, the nutritionist reveals greater knowledge and self-efficacy to implement the recommendations of the Dietary Guidelines for the Brazilian Population [⁴³].

The inclusion criteria in the panel of experts were: being a nutritionist, not having participated in the previous stages of the study, working in PHC units or in the management of the municipal/state/federal NA (Group 1); or be a nutritionist, working in teaching and/or investigation in the area of ANSC (Group 2). Therefore, experts from the core of F&N actions at PHC, with qualification and practical and/or scientific experience; they support the assessment of the instrument and give credibility to the validation [⁴⁰,⁴⁴].

The Conselhos Regionais de Nutricionistas (CRN, Regional Councils of Nutritionists) and the Conselho Federal de Nutricionistas (CFN, Federal Council of Nutritionists) were requested to disseminate the study on social media and to indicate participants (Group 1). We searched for members of the extinct Collaborating Centers in governmental and academic F&N and state F&N coordinators. Authors of papers published in the annals of an event of interest were also consulted. In total, 268 nutritionists were invited, 205 PHC professionals and 63 professors and investigators.

The invitation was made by e-mail, containing the Free and Informed Consent Form (FICF), the instructions for the filling out of the questionnaire and the link to the QuestionPro^® [⁴⁵] software, which was used to build the CI, composed by the IAAN and by assertions and the questionnaire to describe the experts.

Experts were asked to assess the clarity and relevance of the questions and indicators [⁴⁶]. In addition, the CI sheet had room for suggestions or comments [⁴⁷].

The degree of agreement of the experts on the clarity and relevance of the items was verified using Likert-type scale with four points of possible answers: 4- Full agreement, 3- Inclined to agree, 2- Inclined to disagree, 1- Full disagreement [⁴⁸-⁵⁰]. For all answer options, except option 4, justification was requested.

The criteria for assuming that an item reached consensus and had its content validated were: (1) Average score on the Likert-type scale: 1 to 1.99 points = negative perception about the evaluated item, requiring adjustments; 2 to 2.99 points = perceived need for adjustments in the evaluated item, but with a positive general perception of the item; 3 to 4 points = positive perception of the evaluated item; (2) Frequency distribution of responses, at least 51% of specialists with responses between 3 and 4 on the scale; (3) Interquartile difference: <1.0, if = 1, frequency distribution of responses between “full agreement” and “inclined to agree” >60% and (4) Standard Deviation (SD): <1.5 [⁵¹,⁵²]. To check the consensus criteria, the Microsoft Excel® 2010 program was used. The suggestions or comments, as well as the justifications, were evaluated for later adjustment of the IAAN, before submission to the Second Delphi Round.

For the Second Round, new emails were sent to the experts, containing: feedback report on the main results of the First Delphi Round and the profile of the participants; the theoretical foundation on the justifications provided in the First Round, to reiterate concepts and other constituent elements of the IAAN that, until then, were not clear to them and also as feedback to the suggestions presented [⁴⁰]. The CI only with the items which consensus was not reached in the First Round. The same criteria as in the First Round were used in the Second Round. Since consensus was reached with the Second Round, later rounds were disregarded. After reviewing the results of this round, a report on the outcomes of the validation process was delivered to the panel of experts.

This work was approved by the Research Ethics Committee of Universidade Federal de São Paulo (Federal University of São Paulo), under the number 1.661.045.

RESULTS

The process of construction and validation of the instrument took approximately 15 months, from the bibliographic review for the construction of the IAAN to the validation, with the completion of the Second Delphi Round. Figure 1 illustrates how the Delphi rounds were conducted.

Figure 1
Content validation process of the Nutritional Attention Assessment Instrument in Primary Health Care, according to the Delphi Technique, Brazil, 2017.

A total of 41 experts from the five Brazilian macro-regions participated in the First Delphi Round study, and 29 in the Second Round. The participants in the Second Round (n=29) who also participated in the First Round worked with NA in the execution of actions, management, research or teaching; of these, 26 self-assessed as having adequate experience in “Nutritional Attention at PHC”, and 15 others claimed experience of 6 to 10 years.

Most investigators (44.8%) worked in colleges or universities; followed by nutritionists, active in the management of F&N actions, in one of the three government levels (local, state, federal: 24.1%). As to their institutions, most investigators (89.7%) worked in the public service.

In the First Round, which lasted 22 days, 95% of the instrument’s items had their content validated.

Only Indicator 1: Performance of the nutritionist, did not reach consensus and was modified and submitted for consideration in the Second Round. The Indicator’s wording was changed to “Performance of the nutritionist: training, knowledge, relationship with the team, workload and work process”. Three questions were added and two were reworded. In the first question we included two more aspects (type of labor agreement and workload), in addition to the nutritionist’s institutional link. The format of the answer was changed from alternatives to a table which columns represent the aspects mentioned.

The second question, which required an evaluation of the nutritionist’s performance, was reworded adding the aspects to be considered for the assessment: professional training, skills and performance knowledge, relationship and involvement with the work team, workload and work process. In addition, this became a filter question, with objective answer options: sufficient, insufficient, does not know. In future applications, should the respondent not be a nutritionist, when answering “insufficient”, he/she should select the reasons within 11 response options, prepared according to the experts’ suggestions. If the interviewee is a nutritionist, he/she should answer the self-assessment question and, if he/she considers his/her performance insufficient, he/she must justify it.

In addition to these adjustments, justifications were clustered by theme affinity, choosing those that best represented the experts’ opinion. Thus, theoretical foundations were developed to reiterate concepts and other constituent elements of the IAAN, hitherto insufficiently clear. This process also fulfilled the role of feedback for the suggestions presented.

In the Second Round, which lasted 35 days, Indicator 1 and its relevant adjusted questions were sent to the expert panel. The results of this Second Round revealed an increase in the experts’ agreement rate, from 70.7% in the First Round to 96.6% in the Second, towards a consensus regarding the clarity and conciseness of Indicator 1 (Table 1).

Table 1
Content validation of the Nutritional Attention Assessment Instrument in the Primary Health Care, according to consensus criteria in two rounds, Brazil, 2017.

Rounds
				Suficiente in its composition												Clarity and Conciseness
Indicators	N of items			Median			SD			%			Q3-Q1			Median			SD			%			Q3-Q1
	1st	2nd		1st	2nd		1st	2nd		1st	2nd		1st	2nd		1st	2nd		1st	2nd		1st	2nd		1st	2nd
1. Nutricionists’ performance	2	5		2.8	3,8		0.9	0.5		65.8	96.5		1.0	0.0		3.2	3,9		1.0	0.4		70.7	96.6		2.0	0.0
2. Support for nutritional attention actions	6	-		3.5	-		0.6	-		95.1	-		1.0	-		3.7	-		0.5	-		97.6	-		0.0	-
3. Inter-sectoriality	3	-		3.4	-		0.7	-		87.8	-		1.0	-		3.5	-		0.8	-		87.8	-		1.0	-
4. Social Control	2	-		3.7	-		0.5	-		100.0	-		1.0	-		3.8	-		0.5	-		97.6	-		0.0	-
5. Food and Nutrition Surveillance	7	-		3.7	-		0.5	-		100.0	-		1.0	-		3.8	-		0.6	-		97.6	-		0.0	-
6. Individual nutritional attention	9	-		3.6	-		0.6	-		97.6	-		1.0	-		3.8	-		0.6	-		97.6	-		0.0	-
7. Nutritional attention for groups	8	-		3.8	-		0.5	-		97.6	-		0.0	-		3.8	-		0.4	-		97.6	-		0.0	-
8. Nutritional attention focused on prenatal care	3	-		3.8	-		0.4	-		100.0	-		0.0	-		3.9	-		0.5	-		97.6	-		0.0	-
9. Postpartum nutritional attention and breastfeeding	4	-		3.8	-		0.5	-		97.6	-		0.0	-		3.9	-		0.3	-		100.0	-		0.0	-
10. Nutritional attention for children’s health	2	-		3.8	-		0.5	-		97.6	-		0.0	-		3.8	-		0.5	-		92.7	-		0.0	-

• Note: %: Agreement rate; Q1-Q3: Interquartile difference; SD: Standard Deviation.

As for the sufficiency of questions in the composition of Indicator 1, after adding three more questions, the agreement rate related to sufficiency boosted from 65.8% in the First Round to 96.5% in the Second.

At the end of the Second Round, the content of the IAAN was validated, presenting 68 questions, 19 of them on general interview data (n=4), characterization of the respondent (n=6) and the health unit (n=9), in addition to the 49 questions that allow the assessment of NA through 10 indicators, as can be seen in data repository [⁵³]. Figure 2 presents the modalities and instructions for application of the instrument.

Figure 2
Modalities and instructions for the application of the Nutritional Attention Assessment Instrument in Primary Health Care, Brazil, 2017.

DISCUSSION

The organization of Nutritional Attention in Health Services, the monitoring and evaluation of F&N actions, are foreseen in the PNAN, in order to identify the effectiveness of the actions and support their improvement [²¹]. The Ministério da Saúde (Brazilian Ministry of Health) has proposed guidelines to subsidize NA in PHC, which make up the IAAN. However, a number of them do not, per se, translate into an assessment tool. This is actually the proposal of this IAAN: to make the situational diagnosis of NA at PHC feasible, as well as to facilitate the performance of periodic evaluations, by compiling and translating, in the form of indicators and validated questions, these guidelines content, and can act as a compass that shows the way for resource allocation.

The IAAN content was validated in just two Delphi rounds, which demonstrates the congruence of the proposed instrument. There are no globally accepted requirements or guidelines that specify the number of rounds or the definition of consensus in the Delphi technique [⁵⁴]. In this study there were two rounds. To define consensus, a set of criteria was used, which provided a reliable evaluation method [⁵¹,⁵⁵]. The content validation of the IAAN conveys quality to the evaluation processes to be developed using this instrument and this is the contribution of this study to the advancement of knowledge, since, until then, similar works have not been reported in major journals.

The presence of experts with different insertions in the NA at PHC favored the generation of more dense responses. The qualification of the experts supported the analysis of the instrument and gave credibility to the validation. The participation of all Brazilian macro-regions enabled the analysis of the instrument by professionals working in different SUS settings. In addition, the number of experts was greater than that observed in studies of this nature [⁵⁶,⁵⁷].

The extensive literature review for the construction of the IAAN allowed to ascertain the data concerning the topic in question [³⁷,⁵⁸]. This stage ensured that the instrument was based on official recommendations, both national and international, and the strictness employed was reflected in almost the entire validation of the instrument, already in the First Round.

One of the difficulties of this study was to raise awareness among specialists to participate in the panel. However, the number of participants exceeded that observed in other surveys [⁵⁶,⁵⁷,⁵⁹]. To minimize losses, reminders were sent out and deadlines were extended, even though the Delphi technique does not require a representative sample of participants [⁴⁰,⁵¹,⁶⁰]. The reduction in the number of experts between the rounds was not a factor of concern, since this difference does not affect quality or reliability, because we are dealing with a panel [⁵¹]. Another difficulty in the process was an error in the QuestionPro^® software, which prevented the recording of some answers; this inconvenience was remedied by sending the questions directly through these participants’ email.

In relation to other assessment instruments, the Programa Previne Brasil (PPB, Previne Brasil Program) is PHC’s new cost financing model [⁶¹]. For 2020, seven indicators were defined for performance payment, covering strategic actions in Prenatal Care, Women’s Health, Child Health and NCD (Hypertension and Diabetes Mellitus) [⁶²]. According to Massuda [⁶³], the PPB seems to limit the principle of universality, enhancing the distortions in the financing of SUS and health social inequalities in Brazil. Although the PPB crosses the boundaries of IAAN, it does not develop them, so that the instrument proposed in the present study is more comprehensive and turns to the indicators for assessing NA.

The official evaluation strategy previously adopted, the Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica à Saúde (PMAQ, National Program for the Improvement of Access and Quality of Primary Health Care), also did not focus on F&N actions, even if it allowed to obtain some specific information about these initiatives [⁶⁴,⁶⁵].

The Primary Care Assessment Tool (PCATool), used in national and international surveys, is similar to the IAAN. However, regarding F&N actions, it is restricted to inquiring about the provision of nutritional orientation [⁶⁶]. Other instruments adapted to Brazil for PHC, such as the European Task Force on Patient Evaluation of General Practice Care (EUROPEP) and the Patient-Doctor Relationship Questionnaire (PDRQ-9), are used to assess users’ satisfaction with health care and the doctor-patient relationship, respectively [⁶⁷,⁶⁸].

The Questionário de Avaliação da Qualidade de Serviços de Atenção Básica à Saúde (QualiAB, Questionnaire for the Evaluation of the Quality of Primary Health Care Services), developed to evaluate services in the State of São Paulo, was relevant to the construction of the IAAN regarding the questions format. However, QualiAB did not focus on the NA assessment, despite investigating elements of this core of actions [⁵⁶].

The present study has limitations, such as the extensive size of the proposed instrument. However, because it is divided into indicators, sections can be selected for application, depending on the purpose of their use, whether in management or research. One aspect that could be included in the IAAN would be scores for the results of each indicator. Nevertheless, this is an unprecedented study, in which the content of the only evaluation instrument with an expanded focus on the Organization of Nutritional Attention at PHC in Brazil was developed and validated.

CONCLUSION

Instrumento de Avaliação da Atenção Nutricional advances by enabling the establishment of evaluation processes for Nutritional Attention. It is expected that this instrument be widely used to identify and evaluate the NA offered to municipalities, states and at the federal level. The results obtained with future applications may contribute to support the qualification of F&N policies management.

ACKNOWLEDGMENTS

To Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes, Coordination for the Improvement of Higher Education Personnel); Conselhos Regionais de Nutricionistas (CRN, Regional Councils of Nutritionists); Universidade Federal de São Paulo researchers, who collaborated with the revision of the Instrument; the professionals who participated in the pilot study and the Brazilian experts, who contributed with the validation process.

• Support: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, Coordination for the Improvement of Higher Education Personnel) (Process n. 33009015).
• Article elaborated from the dissertation by A.B. PACITO-ALMEIDA, entitled “Desenvolvimento e validação de conteúdo de Instrumento de Avaliação da Organização da Atenção Nutricional na Atenção Primária à Saúde”. Universidade Federal de São Paulo; 2017.

How to cite this article

• Pacito-Almeida AB, Santana ABN, Zangirolani LTO, Medeiros MAT. Content validation of the Nutritional Attention Assessment Instrument in Primary Health Care. Rev Nutr. 2020;33:e200065. https://doi.org/10.1590/1678-9865202033e200065

REFERÊNCIAS

• ¹
  World Health Organization. Noncommunicable diseases. Geneva: Organization; 2018 [cited 2020 Mar 11]. Available from: http://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases
  » http://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases
• ²
  Institute for Health Metrics and Evaluation. Findings from the Global Burden of Disease Study 2017. Seattle: Institute; 2018.
• ³
  World Health Organization. Noncommunicable diseases country profiles 2018. Geneva: Organization; 2018.
• ⁴
  GBD 2017 Diet Collaborators. Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2019;393(10184):1958-72. https://doi.org/10.1016/S0140-6736(19)30041-8
  » https://doi.org/10.1016/S0140-6736(19)30041-8
• ⁵
  GBD 2017 Risk Factor Collaborators. Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1923-94. https://doi.org/10.1016/S0140-6736(18)32225-6
  » https://doi.org/10.1016/S0140-6736(18)32225-6
• ⁶
  Jardim PCBV. Overweight, the cardiovascular risk of the century. Arq Bras Cardiol. 2019:113(2)185-7.
• ⁷
  Afshin A, Forouzanfar MH, Reitsma MB, Sur P, Estep K, Lee A, et al Health effects of overweight and obesity in 195 countries over 25 years. N Engl J Med. 2017;377(1):13-27. https://doi.org/10.1056/NEJMoa1614362
  » https://doi.org/10.1056/NEJMoa1614362
• ⁸
  Ministério da Saúde (Brasil). Vigitel Brasil 2019: vigilância de fatores de risco e proteção para doenças crônicas por inquérito telefônico: estimativas sobre frequência e distribuição sociodemográfica de fatores de risco e proteção para doenças crônicas nas capitais dos 26 estados. Brasília: Ministério; 2020.
• ⁹
  Instituto Brasileiro de Geografia e Estatística. Pesquisa de Orçamentos Familiares 2008-2009: antropometria e estado nutricional de crianças, adolescentes e adultos no Brasil. Rio de Janeiro: Instituto; 2010.
• ¹⁰
  Ward ZJ, Long MW, Resch SC, Giles CM, Cradock AL, Gortmaker SL. Simulation of growth trajectories of childhood obesity into adulthood. N Engl J Med. 2017;377(22):2145-53. https://doi.org/10.1056/NEJMoa1703860
  » https://doi.org/10.1056/NEJMoa1703860
• ¹¹
  Horta BL, Victora CG, França GVA, Hartwig FP, Ong KK, Rolfe EDL, et al Breastfeeding moderates FTO related adiposity: a birth cohort study with 30 years of follow-up. Nature. 2018;8(1):1-11. https://doi.org/10.1038/s41598-018-20939-4
  » https://doi.org/10.1038/s41598-018-20939-4
• ¹²
  Victora CG, Bahl R, Barros AJD, França GVA, Horton S, Krasevec J, et al Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect. Lancet. 2016;387(10017):475-90. https://doi.org/10.1016/S0140-6736(15)01024-7
  » https://doi.org/10.1016/S0140-6736(15)01024-7
• ¹³
  Horta BL, Mola CL, Victora CG. Long-term consequences of breastfeeding on cholesterol, obesity, systolic blood pressure and type 2 diabetes: a systematic review and meta-analysis. Acta Paediatr Int J Paediatr. 2015;104(467):30-7. https://doi.org/10.1111/apa.13133
  » https://doi.org/10.1111/apa.13133
• ¹⁴
  Laporte-Pinfildi ASC, Medeiros MAT. Nutritional care during prenatal and postpartum periods: a report of experiences in a city on São Paulo’s coast. Rev Nutr. 2016;29(6):947-61. https://doi.org/10.1590/1678-98652016000600016
  » https://doi.org/10.1590/1678-98652016000600016
• ¹⁵
  Laporte-Pinfildi ASC, Zangirolani LTO, Spina N, Martins PA, Medeiros MAT. The nutritional care during the prenatal and postpartum: perceptions of managers of Primary Health Care. Rev Nutr. 2016;29(1):109-23. https://doi.org/10.1590/1678-98652016000100011
  » https://doi.org/10.1590/1678-98652016000100011
• ¹⁶
  Mitchell LJ, Ball LE, Ross LJ, Barnes KA, Williams LT. Effectiveness of dietetic consultations in primary health care: a systematic review of randomized controlled trials. J Acad Nutr Diet. 2017;117(12):1941-62. https://doi.org/10.1016/j.jand.2017.06.364
  » https://doi.org/10.1016/j.jand.2017.06.364
• ¹⁷
  Howatson A, Wall CR, Turner-Benny P. The contribution of dietitians to the primary health care workforce. J Prim Health Care. 2015;7(4):324-32. https://doi.org/10.1071/hc15324
  » https://doi.org/10.1071/hc15324
• ¹⁸
  Dietitians of Canada. Dietitians in primary health care. Ottawa: Dietitians of Canada; 2017 [cited 2017 Aug 29]. Available from: https://www.dietitians.ca/DietitiansOfCanada/media/Documents/Resources/DieittiansInPrimaryHealthCare-bilingual.pdf?ext=.pdf
  » https://www.dietitians.ca/DietitiansOfCanada/media/Documents/Resources/DieittiansInPrimaryHealthCare-bilingual.pdf?ext=.pdf
• ¹⁹
  Recine E, Carvalho MF, Leão M. O papel do nutricionista na atenção primária à saúde. Brasília: Conselho Federal de Nutricionistas; 2015 [citado 3 jun 2017]. Disponível em: https://www.cfn.org.br/wp-content/uploads/2015/11/livreto-atencao_primaria_a_saude-2015.pdf
  » https://www.cfn.org.br/wp-content/uploads/2015/11/livreto-atencao_primaria_a_saude-2015.pdf
• ²⁰
  Pereira TN, Monteiro RA, Santos LMP. Food and nutrition in primary care in Brazil. Gac Sanit. 2018;32(3):297-303. https://doi.org/10.1016/j.gaceta.2017.08.004
  » https://doi.org/10.1016/j.gaceta.2017.08.004
• ²¹
  Ministério da Saúde (Brasil). Política Nacional de Alimentação e Nutrição. 2nd. ed. Brasília: Ministério; 2012.
• ²²
  Ministério da Saúde (Brasil). Portaria n. 2.436, de 21 de setembro de 2017. Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes para a organização da Atenção Básica, no âmbito do Sistema Único de Saúde (SUS). Brasília: Ministério; 2017 [citado 2020 Jun 29]. Disponível em: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2017/prt2436_22_09_2017.html
  » https://bvsms.saude.gov.br/bvs/saudelegis/gm/2017/prt2436_22_09_2017.html
• ²³
  Oliveira JTC, Gabriel CG, Machado ML, Réos MF, Soar C, Venske DKR. Government-subsidized restaurants as promoters of the realization of the human right to adequate food: proposal of an evaluation model. Rev Nutr. 2019;32:e180193. https://doi.org/10.1590/1678-9865201932e180193
  » https://doi.org/10.1590/1678-9865201932e180193
• ²⁴
  Pessini J, Gerage AM, Meurer ST, Barbosa AR, Lopes ACS, Benedetti TRB. Cross-cultural equivalence and psychometric proprieties of the social support scale for healthy eating habits. Rev Nutr. 2016;29(6):797-807. https://doi.org/10.1590/1678-98652016000600005
  » https://doi.org/10.1590/1678-98652016000600005
• ²⁵
  Pereira RDM, Alvim NAT. Delphi technique in dialogue with nurses on acupuncture as a proposed nursing intervention. Esc Anna Nery. 2015;19(1):174-80. https://doi.org/10.5935/1414-8145.20150024
  » https://doi.org/10.5935/1414-8145.20150024
• ²⁶
  Hohmann E, Brand JC, Rossi MJ, Lubowitz JH. Expert opinion is necessary: delphi panel methodology facilitates a scientific approach to consensus. Arthroscopy. 2018; 34(2):349-51. https://doi.org/10.1016/j.arthro.2017.11.022
  » https://doi.org/10.1016/j.arthro.2017.11.022
• ²⁷
  Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159-70. https://doi.org/10.1007/s11136-018-1829-0
  » https://doi.org/10.1007/s11136-018-1829-0
• ²⁸
  Jokiniemi K, Meretoja R, Pietilä AM. Constructing content validity of clinical nurse specialist core competencies: exploratory sequential mixed-method study. Scand J Caring Sci. 2018;32(4):1428-36. https://doi.org/10.1111/scs.12588
  » https://doi.org/10.1111/scs.12588
• ²⁹
  Roller-Wirnsberger R, Masud T, Vassallo M, Zöbl M, Reiter R, Van Den Noortgate N, et al European postgraduate curriculum in geriatric medicine developed using an international modified Delphi technique. Age Ageing. 2019;48(2):291-9. https://doi.org/10.1093/ageing/afy173
  » https://doi.org/10.1093/ageing/afy173
• ³⁰
  Hughes R, Begley A, Yeatman H. Consensus on the core functions of the public health nutrition workforce in Australia. Nutr Diet. 2016;73(1):103-11. https://doi.org/10.1111/1747-0080.12168
  » https://doi.org/10.1111/1747-0080.12168
• ³¹
  Desroches S, Lapointe A, Deschenes SM, Bissonnette-Maheux V, Gravel K, Thirsk J, et al Dietitians’ perspectives on interventions to enhance adherence to dietary advice for chronic diseases in adults. Can J Diet Pract Res. 2015;76(3):103-8. http://doi.org/10.3148/cjdpr-2015-009
  » https://doi.org/10.3148/cjdpr-2015-009
• ³²
  Williams KE, Sansoni J, Morris D, Thompson C. A Delphi study to develop indicators of cancer patient experience for quality improvement. Support Care Cancer. 2018;26(1):129-38. https://doi.org/10.1007/s00520-017-3823-4
  » https://doi.org/10.1007/s00520-017-3823-4
• ³³
  Vitorino SAS, Cruz MM, Barros DC. Modeling of food and nutrition surveillance in primary health care. Rev Nutr. 2017;30(1):109-26. https://doi.org/10.1590/1678-98652017000100011
  » https://doi.org/10.1590/1678-98652017000100011
• ³⁴
  Jaime PC, Tramontt CR, Maia TM, Gabe KT, Reis LC. Content validity of an educational workshop based on the Dietary Guidelines for the Brazilian Population. Rev Nutr. 2018;31(6):593-602. https://doi.org/10.1590/1678-98652018000600008
  » https://doi.org/10.1590/1678-98652018000600008
• ³⁵
  Primo CC, Henrique LR, Bertazo QS, Resende FZ, Leite FMC, Brandão MAG. Validation of the “Interactive Breastfeeding Scale”: theoretical and empirical analysis. Esc Anna Nery. 2020;24(1):e20190207. https://doi.org/10.1590/2177-9465-ean-2019-0207
  » https://doi.org/10.1590/2177-9465-ean-2019-0207
• ³⁶
  Silva AB, Piccoli AB, Pellanda LC. Knowledge and food practices questionnaire: construction and validation. J Pediatr. 2020 [in press]. https://doi.org/10.1016/j.jped.2019.11.006
  » https://doi.org/10.1016/j.jped.2019.11.006
• ³⁷
  Coluci MZO, Alexandre NMC, Milani D. Construction of measurement instruments in the area of health. Ciênc Saúde Coletiva. 2015;20(3):925-36. https://doi.org/10.1590/1413-81232015203.04332013
  » https://doi.org/10.1590/1413-81232015203.04332013
• ³⁸
  Dolan C, Glynn R, Lawlor B. A systematic review and Delphi study to ascertain common risk factors for T2DM and dementia and brain-related complications of diabetes. Can J Diabetes. 2020;44(7):628-35. https://doi.org/10.1016/j.jcjd.2020.01.004
  » https://doi.org/10.1016/j.jcjd.2020.01.004
• ³⁹
  Massaroli A, Martini JG, Lino MM, Spenassato D, Massaroli R. The delphi method as a methodological framework for research in nursing. Texto Contexto Enferm. 2017;26(4):e1110017. https://doi.org/10.1590/0104-07072017001110017
  » https://doi.org/10.1590/0104-07072017001110017
• ⁴⁰
  Fernández-Ávila DG, Rojas MX, Rosselli D. The Delphi method in rheumatology research: are we doing it right? Rev Colomb Reumatol. 2020 [in press]. https://doi.org/10.1016/j.rcreue.2019.04.007
  » https://doi.org/10.1016/j.rcreue.2019.04.007
• ⁴¹
  Ministério da Saúde (Brasil). Contribuições dos Núcleos de Apoio à Saúde da Família para a Atenção Nutricional. Brasília: Ministério; 2017.
• ⁴²
  Fittipaldi ALM, Barros DC, Romano VF. Matrix support in food and nutrition actions: the perspective of professionals of the Family Health Strategy. Physis. 2017;27(3):793-811. https://doi.org/10.1590/S0103-73312017000300021
  » https://doi.org/10.1590/S0103-73312017000300021
• ⁴³
  Reis LC, Jaime PC. Knowledge and perceived selfefficacy and collective efficacy of health professionals to implement the Brazilian Dietary Guidelines in Primary Health Care. Demetra. 2019;14:e39140. https://doi.org/10.12957/demetra.2019.39140
  » https://doi.org/10.12957/demetra.2019.39140
• ⁴⁴
  Campos GWS. Saúde pública e saúde coletiva: campo e núcleo de saberes e práticas. Ciênc Saúde Coletiva. 2000;5(2):219-30. https://doi.org/10.1590/S1413-81232000000200002.
  » https://doi.org/10.1590/S1413-81232000000200002
• ⁴⁵
  QuestionPro Brasil. Logiciels informatiques. São Francisco: QuestionPro; 2016 [cited 2016 Nov 20]. Available from: https://www.questionpro.com/pt-br
  » https://www.questionpro.com/pt-br
• ⁴⁶
  Leite SS, Áfio ACE, Carvalho LV, Silva JM, Almeida PC, Pagliuca LMF. Construction and validation of an Educational Content Validation Instrument in Health. Rev Bras Enferm. 2018;71(Suppl 4):1635-41. https://doi.org/10.1590/0034-7167-2017-0648
  » https://doi.org/10.1590/0034-7167-2017-0648
• ⁴⁷
  Holanda FL, Marra CC, Cunha ICKO. Professional competence of nurses in emergency services: evidence of content validity. Rev Bras Enferm. 2019;72(Suppl1):66-73. https://doi.org/10.1590/0034-7167-2017-0518
  » https://doi.org/10.1590/0034-7167-2017-0518
• ⁴⁸
  Linstone HA, Turoff M. The Delphi method: techniques and applications. Boston: Addison-Wesley Educational Publishers Inc; 2002 [cited 2017 Aug 29]. Available from: https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf
  » https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf
• ⁴⁹
  Likert R. A technique for the measturement of attittudes [thesis]. New York: New York University; 1932.
• ⁵⁰
  Nadler JT, Weston R, Voyles EC. Stuck in the middle: the use and interpretation of mid-points in items on questionnaires. J Gen Psychol. 2015;142(2):71-89. https://doi.org/10.1080/00221309.2014.994590
  » https://doi.org/10.1080/00221309.2014.994590
• ⁵¹
  Giannarou L, Zervas E. Using Delphi technique to build consensus in practice. Int J Bus Sci Appl Manage. 2014;9(2):65-82.
• ⁵²
  Camargo RGM, Caivano S, Bandoni DH, Domene SMA. Healthy eating at school: consensus among experts. Rev Nutr. 2016;29(6):809-19. https://doi.org/10.1590/1678-98652016000600006
  » https://doi.org/10.1590/1678-98652016000600006
• ⁵³
  Pacito-Almeida AB, Santana ABN, Medeiros MAT. Nutritional Attention Assessment Instrument_Iaan [dataset]. Figshare; 2020 [cited 2020 Oct 13]. https://doi.org/10.6084/m9.figshare.13087370
  » https://doi.org/10.6084/m9.figshare.13087370
• ⁵⁴
  Veziari Y, Kumar S, Leach M. The development of a survey instrument to measure the barriers to the conduct and application of research in complementary and alternative medicine: A Delphi study. Bmc Complement Altern Med. 2018 [cited 2020 Mar 16];18(1):335. Available from: https://bmccomplementalternmed.biomedcentral.com/articles/10.1186/s12906-018-2352-0
  » https://bmccomplementalternmed.biomedcentral.com/articles/10.1186/s12906-018-2352-0
• ⁵⁵
  Lang M, Plessis E. Sensory processing disorder: perceptions on the clinical role of advanced psychiatric nurses. Health SA Gesondheid. 2019[cited 2020 Mar 11];24:1197. Available from: http://www.ncbi.nlm.nih.gov/pubmed/31934431
  » http://www.ncbi.nlm.nih.gov/pubmed/31934431
• ⁵⁶
  Zarili TFT. Avaliação de serviços de atenção básica: atualização e validação do instrumento QualiAB. Universidade Estadual Paulista: Botucatu; 2015.
• ⁵⁷
  Braga RD, Lucena FN, Ribeiro-Rotta RF. A multiprofessional information model for Brazilian primary care: defining a consensus model towards an interoperable electronic health record. Int J Med Inform. 2016;90:48-57.
• ⁵⁸
  Bolarinwa O. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195-201. https://doi.org/10.4103/1117-1936.173959
  » https://doi.org/10.4103/1117-1936.173959
• ⁵⁹
  Jokiniemi K, Haatainen K, Meretoja R, Pietilä AM. The future of the clinical nurse specialist role in Finland. J Nurs Scholarsh. 2015;47(1):78-86.
• ⁶⁰
  Lee J, Lee SH, Chang GT. Expert consensus on the development of a health-related questionnaire for the pediatric field of Korean medicine: a Delphi study. Bmc Complement Med Ther. 2020;20(1):1-13.
• ⁶¹
  Ministério da Saúde (Brasil). Portaria no 2.979, de 12 de novembro de 2019. Institui o Programa Previne Brasil, que estabelece novo modelo de financiamento de custeio da Atenção Primária à Saúde no âmbito do Sistema Único de Saúde, por meio da alteração da Portaria de Consolidação nº 6/GM/MS, de 28 de setembro de 2017. Brasília: Diário Oficial da União, 2020 [citado 29 jun 2020]. Disponível em: http://www.in.gov.br/en/web/dou/-/portaria-n-2.979-de-12-de-novembro-de-2019-227652180
  » http://www.in.gov.br/en/web/dou/-/portaria-n-2.979-de-12-de-novembro-de-2019-227652180
• ⁶²
  Ministério da Saúde (Brasil). Nota Técnica no 5/2020-DESF/SAPS/MS. Brasília: Diário Oficial da União, 2020 [citado 29 jun 2020]. Disponível em: https://egestorab.saude.gov.br/image/?file=20200204_N_SEIMS-0013327270-NotaTecnicaIndicadores_3604088260565235807.pdf
  » https://egestorab.saude.gov.br/image/?file=20200204_N_SEIMS-0013327270-NotaTecnicaIndicadores_3604088260565235807.pdf
• ⁶³
  Massuda A. Primary health care financing changes in the Brazilian Health System: advance ou setback? Ciênc Saúde Coletiva. 2020;25(4):1181-8. http://doi.org/10.1590/1413-81232020254.01022020
  » https://doi.org/10.1590/1413-81232020254.01022020
• ⁶⁴
  Silva FMVA. Alimentação e Nutrição na Atenção Básica à Saúde: estudo a partir da avaliação externa do Programa de Melhoria do Acesso e da Qualidade na Atenção Básica: PMAQ [dissertation]. Salvador: Universidade Federal da Bahia; 2019.
• ⁶⁵
  Ministério da Saúde (Brasil). Instrumento de Avaliação Externa para as Equipes de Atenção Básica (Saúde da Família e Equipe Parametrizada): Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ). Brasília: Ministério; 2013.
• ⁶⁶
  Ministério da Saúde (Brasil). Manual do Instrumento de Avaliação da Atenção Primária à Saúde: PCATool-Brasil: 2020. Brasília: Ministério; 2020.
• ⁶⁷
  Roque H, Veloso A, Ferreira PL. Portuguese version of the EUROPEP questionnaire: contributions to the psychometric validation. Rev Saúde Pública. 2016;50:61. https://doi.org/10.1590/S1518-8787.2016050006259
  » https://doi.org/10.1590/S1518-8787.2016050006259
• ⁶⁸
  Wollmann L, Hauser L, Mengue SS, Agostinho MR, Roman R, Van Der Feltz-Cornelis CM, et al Cross-cultural adaptation of the Patient- Doctor Relationship Questionnaire (PDRQ-9) in Brazil. Rev Saúde Pública. 2018;52:71. https://doi.org/10.11606/s1518-8787.2018052000380
  » https://doi.org/10.11606/s1518-8787.2018052000380

Publication Dates

• Publication in this collection
  27 Nov 2020
• Date of issue
  2020

History

• Received
  03 Apr 2020
• Reviewed
  09 Sept 2020
• Accepted
  07 Oct 2020