Diet Quality Index associated with Digital Food Guide: update and validation

O Índice de Qualidade da Dieta associado ao Guia Alimentar Digital: atualização e validação

Índice de Calidad de la Dieta asociado a la Guía Digital de Alimentos: actualización y validación

Abstracts

Monitoring and assessing dietary intake may favor the promotion of healthy choices and the indexes that assess the quality of the diet contribute to this purpose. The Diet Quality Index associated with the Digital Food Guide (DQI-DFG) was developed from guidelines of the School of Public Health at Harvard University, adjusted for the Brazilian food habits. Based on new studies on prevention of chronic non-communicable diseases, the objective of this study was to improve and validate the DQI-DFG for a second version. The following psychometric properties were used: (a) content validation: consensus on healthy eating and DQI among experts; (b) construct validation: enhancement of a reference diet to determine recommended serving sizes and consumption ranges; correlation between each group score and energy value; comparative analyses between menu assessment by experts and DQI-DGF results to confirm the score criterion; (c) reliability: agreement between the result shown by the evaluators and the result presented by DQI-DFG; correlation between each food group score and total DQI score to understand the relationship between these variables. The energetic values do not influence the DQI score. There is a high correlation between the score attributed by experts and the result of DQI-DFG (r = 0.78 until r = 0.97). The Index components have a balanced score of influence in its final result (r = 0.49 until r = 0.10). Consensus result performed among experts legitimates the concepts that justify DQI-DFG. There was an agreement between menu quality evaluation by experts and the results shown by DQI-DFG.

Keywords:
Food Guide; Index; Food and Nutritional Education; Psychometrics

O monitoramento e a avaliação da ingestão alimentar podem favorecer a promoção de escolhas saudáveis, e os índices que avaliam a qualidade da dieta contribuem para esse objetivo. Foi desenvolvido o Índice de Qualidade da Dieta associado ao Guia Alimentar Digital (IQD-GAD), com base nas diretrizes produzidas pela Faculdade de Saúde Pública da Universidade de Harvard, ajustadas para hábitos alimentares brasileiros. Com base em novos estudos sobre a prevenção das doenças crônicas não transmissíveis, o estudo teve como objetivo melhorar e validar uma segunda versão do IQD-GAD. Foram utilizadas as seguintes propriedades psicométricas: (a) validação de conteúdo: consenso de especialistas sobre alimentação saudável e IQD; (b) validação do construto: aprimoramento de uma dieta de referência para determinar as porções recomendadas e faixas de consumo; correlação entre a pontuação de cada grupo e o valor energético; análises comparativas entre a avaliação do cardápio por especialistas e os resultados do IQD-GAD para confirmar o critério de pontuação; (c) confiabilidade: concordância entre o resultado mostrado pelos avaliadores e o resultado apresentado pelo IQD-GAD; correlação entre a pontuação de cada grupo alimentar e o IQD total, para compreender a relação entre as variáveis. Os valores energéticos não influenciam a pontuação do IQD. Há uma forte correlação entre a pontuação atribuída pelos especialistas e o resultado do IQD-GAD (r = 0,78 até r = 0,97). Os componentes do Índice apresentam uma pontuação equilibrada de influência sobre o resultado (r = 0,49 até r = 0,10). O resultado do consenso de especialistas legitima os conceitos que justificam o IQD-GAD. Houve concordância entre a avaliação da qualidade da dieta pelos especialistas e os resultados apresentados pelo IQD-GAD.

Palavras-chave:
Guias Alimentares; Índice; Educação Alimentar e Nutricional; Psicometria

Supervisar y evaluar la ingesta de alimentos podría favorecer la promoción en la elección de alternativas saludables, por este motivo, son importantes los índices que evalúan la calidad de la dieta. El Índice de Calidad de la Dieta, asociado a la Guía Digital de Alimentos (ICD-GDA), se desarrolló a partir de las directrices de la Escuela Pública de Salud de la Universidad de Harvard, ajustadas a los hábitos alimentarios brasileños. Basado en los nuevos estudios sobre la prevención de enfermedades crónicas no transmisibles, el objetivo de este estudio fue mejorar y validar la segunda versión del ICD-GDA. Se utilizaron las siguientes propiedades psicométricas: (a) validación del contenido: consenso sobre comer sano y el índice de calidad de la dieta entre expertos; (b) validación del constructo: mejora de la dieta de referencia para determinar el tamaño de las porciones recomendadas y rangos de consumo; correlación entre la puntuación de cada grupo y valor de la energía; análisis comparativos entre la evaluación del menú por parte de expertos y resultados del ICD-GDA para confirmar los criterios de puntuación; (c) fiabilidad: concordancia entre el resultado mostrado por los evaluadores y el resultado presentado por ICD-GDA; correlación entre la puntuación de cada grupo de comida y la puntuación total del ICD para comprender la relación entre estas variables. Los valores energéticos no influencian la puntuación del ICD. Existe una alta correlación entre la puntuación atribuida por expertos y el resultado del ICD-GDA (r = 0,78 hasta r = 0,97). Los componentes del índice poseen una puntuación equilibrada de influencia en su resultado final (r = 0,49 hasta r = 0,10). El resultado de consenso entre expertos legitima los conceptos que justifican el ICD-GDA. Existe acuerdo entre la evaluación por parte de expertos de un menú de calidad y los resultados mostrados por el ICD-GDA.

Palabras-clave:
Guías Alimentarias; Índice; Educación Alimentaria y Nutricional; Psicometría

Introduction

Brazil experiences, as other countries, the effects of the double burden of diseases caused by eating errors: nutritional deficiencies in addition to obesity and comorbidities ^1,2,3; while 2.7% of the infant population shows low weight, obesity already reaches 14.8% of the population ⁴. Several studies bring evidence of relationships between consumption of specific food groups, as well as their degree of processing and the occurrence of chronic non-communicable diseases. These findings subsidize the national and international dietary guidelines for the prevention of chronic non-communicable diseases ⁵.

Food guides provide guidelines to assist citizens, the productive sector, health professionals, and public administrators in promoting adequate nutrition ⁶. However, adjusting eating habits requires a multi-factor long-term approach, and monitoring and evaluating dietary intake may promote public policies that favor progress towards healthy choices ⁵.

Since 1980, indexes or other types of measures have been developed and their association with the guides has been tried ⁷. Diet quality indexes were developed for food intake monitoring and are also used as a tool for food and nutrition education ^8,9,10. The Health Eating Index (HEI) ⁹ is updated at every revision of the Dietary Guidelines for Americans¹¹. Thus, it incorporates advances regarding the relationship between diet and health and enhances its psychometric properties. Its second version ¹² was adapted for the use in Brazil and considered the first version of the Food Guide for the Brazilian Population as a reference for its calculations ¹³.

The Digital Food Guide (DFG) was developed in the years of 2011-2012 as a technological tool that could evaluate the quality of the diet. The creation of the Diet Quality Index (DQI) associated with DFG (DQI-DFG) is the result of studies on nutritional consensus about patterns of food intake compatible with health promotion. Since its launch in 2009, it has contributed to promoting choices in key food groups for good nutrition. In the construction of this index, foods are classified into food groups according to their similarity in nutritional composition, as well as the evidences about the implications of their regular intake on health ¹⁴. Based on the criteria established in the HEI-2005 ¹², in addition to the guidelines from the Department of Nutrition of the Harvard School of Public Health (United States) ^15,16,17, adjustments were carried out for the valuation of the Brazilian population food habits, and thus, the DQI-DFG was developed (DQI-DFG-2013) ¹⁴, instead of the indicators that derive from the guidelines of the North American Food Guide.

New findings on the effect of food on health motivate the revision of the criteria for the organization of the moderation components and the adequacy of DQI-DFG. The present study aims to improve and validate the (DQI-DFG-2019).

Methods

To improve and validate the DQI-DFG-2019, its psychometric properties for content validation, construct validation, and reliability assessment were studied this way: (i) Content validation - Consensus from an expert panel about healthy eating and DQI; (ii) Construct validation - Reference diet improvement for determining the portions and ranges of intake of each food group; Correlation between the components of the DQI-DFG and the energy value (Pearson’s linear correlation coefficient); Analysis of agreement between the experts' opinion about the quality of menus and the result of the DQI-DFG (frequency, Pearson’s linear correlation coefficient); (iii) Reliability - Reliability based on the evaluators (intraclass correlation coefficient, Student’s t-test, signal test, and Model of analysis of variance in blocks); Correlation between the components of the DQI-DFG and the total score (Pearson’s linear correlation coefficient).

Database construction

The DQI-DFG-2019 was applied to menus originated from secondary data of the cross-sectional population-based survey named Health Survey in Campinas - IS-ACamp 2008. Food intake was estimated by 24-hour recalls (24hR), collected on different days of the week and months of the year, by a team of trained and supervised interviewers. The criteria for sampling and randomizing the study have been published ¹⁸. The sample of this study consisted of 664 adults, aged 19-60 years, of both sexes, who do not practice professional physical exercise, are not in pregnancy and lactation period, and with no presence of disease that demands nutritional intervention. This sample comes from Brazilian individuals, from the southeast region, living in the city of Campinas, State of São Paulo. Data from the 24hR were processed by Nutrition Data System for Research software, 2007 version (http://www.ncc.umn.edu/products/) and exported to the Microsoft Access software (https://products.office.com/) for the calculation of the DQI-DFG-2019.

Content validation

Consensus from an expert panel about healthy eating and DQI

Likert is a scale for perception assessment by indicating agreement or disagreement about what is being measured ¹⁹. Thus, assertions related to the topic were developed and the experts attributed agreement or disagreement. The Delphi technique is adopted to establish consensus among experts ²⁰. The frequency of agreement or disagreement of each assertion was presented to the experts in order to confirm the consensus on the subject. In this way, the Likert scale ¹⁹ and the Delphi technique ²⁰ were combined to perform index content validation.

Assertions contained in the analysis instrument followed the same theoretical background used for the construction of the DQI-DFG-2013 ¹⁴. These assertions and their respective statistical analyses are contained in the article Consensus Between Specialists in Healthy Eating and Diet Quality Index²¹.

Construct validation

Reference diet for determining the portion sizes and ranges of intake of each food group

The reference diet (RD) developed in the DQI-DFG-201314 was enhanced through the attribution of criteria to define the portion size for each of the food groups and the corresponding intake range. The RD was calculated using the Brazilian Table of Food Composition (TACO) ²² and the Database of Nutrients from the U.S. Department of Agriculture (USDA) ²³. The nutritional quotas were determined through the reference values for nutritional requirements proposed by the Institute of Medicine (United States) ^{24,25,26,27,28,29} for macronutrients, vitamins and minerals, linoleic acid, and linolenic acid; for saturated, polyunsaturated, and monounsaturated fats, we employed the guidelines from the Arquivos Brasileiros de Cardiologia³⁰. This topic includes the application of nutritional concepts derived from a dietary technique, which will be confirmed through the statistical analysis presented in items Correlation between the Components of the DQI-DFG and the Energy Value and Analysis of Agreement between the Experts’ Opinion about the Quality of Menus and the Result of the DQI-DFG from the construct validation.

Correlation between the components of the DQI-DFG and the energy value

This analysis was applied to ensure the independence between the score in the index components and the energy value of the diet and to avoid the overestimation of the final score in high energy diets.

Analysis of agreement between the experts’ opinion about the quality of menus and the result of the DQI-DFG

The DQI-DFG is organized into moderation components and adequacy components. Score and weight criteria were assigned according to the ranges of portions intake, according to the relevance of each food group. Blind evaluation by experts in HEI and DQI was submitted to analysis of agreement with the final result generated by the DQI-DFG-2019.

a) Selection of experts

A search in the Web of Science database, with keywords “healthy eating index” and “diet quality index”, added with Brazil as the address, was used to identify authors of articles published between June 2005 and June 2015. Through the Lattes platform, provided by the Brazilian National Research Council (CNPq), we selected graduated dietitian experts.

b) Data collection with the experts

The experts received an email with the instructions and the link to access the survey instrument, hosted in the tool Question Pro (https://www.questionpro.com/pt-br/).

c) Analysis instrument

A random selection of ten different menus representatives of each tertile of the DQI-DFG score was sent to experts. In blind analysis, the experts were asked to consider: occurrence of different food groups, concentrated sources of sugar and fat, derived products with high degree of processing and characteristics of preparation associated with risk or protection to classify each menu as “good” “regular”, or “bad”, assigning grades from 0 (very bad) to 10 (very good). The foods and recipes contained in the menus were accompanied by household measure with the objective of assisting in qualitative analysis, without requiring nutritional calculation.

Reliability

Reliability based on the evaluators

In order to assess the reproducibility of the DQI-DFG-2019, three undergraduate students and five nutritionists from the research group received a list of ten different menus chosen randomly. The result of each evaluator for the processed menus was compared with the result of the DQI-DFG-2019 (inter-rater evaluation).

Nutrabem Pro

The Nutrabem Pro (https://nutrabem.unifesp.br/) was the software used to calculate the menus. The Nutrabem Pro software database was built with data from the TACO ²² and from the USDA Food Composition Database ²³; this tool is free of charge and operates the DQI-DFG-2019 application.

Correlation between the components of the DQI-DFG and the total score

This analysis was employed to determine which component exerts greater influence on the score of the Index.

Ethical aspects

This study was approved by the Ethics Research Committee of the São Paulo Federal University - opinion n. 707.010, and by the system CEP/CONEP - CAAE n. 28924514.6.0000.5505.

Results and discussion

The theoretical framework and the conception that support the DQI-DFG were examined in the validation and reliability study, as shown below.

Content validation

Consensus on healthy eating and diet quality index between experts

The result of the consensus between the experts ²¹ ratifies the division of the food groups into moderation components and adequacy components (Table 1). Foods classified among the moderation components bring risk to health when consumed in excess. However, they can integrate the menu without compromising the quality of nutrition for possible satisfaction of cultural or sensory values, as ultra-processed foods. The groups that comprise the adequacy components are those with important foods to meet the nutritional needs ¹⁴. It is the opinion of the experts that composing a DQI using food groups enables to emphasize foods and not nutrients ²¹, similarly to the precepts of the Food Guide for the Brazilian Population⁶:

Table 1
List of foods for each food group that compose the Diet Quality Index associated with Digital Food Guide (DQI-DFG-2019). Brazil, 2019.

Food groups	Examples of foods
Moderation components
Sugars and sweets	Sugar (vanilla, coconut, confectioner, demerara, brown, corn or refined sugar), processed acai berry, chocolate milk, sweetener/industrialized sweetener, tonic water, carob, taffy, cereal bar, sugar based energy drink, chocolate flavored milk drink, cookies (with and without filling), industrialized cake (simple and with filling), syrup, hominy, cappuccino, tea ready to drink, chocolate, flan, sweetened corn flakes of several flavors, canned fruit in syrup, canned fruit, candied fruit, gelatin, jelly fruit, chewing gum, currant drink, chocolate milk, condensed milk, marshmallows, honey, local recipes of sweets and desserts, chikki, molasses, artificial juice, soda, ice cream, soy-based juice, industrialized juice, Nougat - in all versions: traditional and diet/light
Dairy drinks and sweetened yogurts	Fruit-flavored yogurt - in all versions: traditional and diet/light
Bovine and pork meat	Chuck steak, lamb chops, pork chop, tripe, lamb, beef, lamb, beef jerky, veal, rabbit, ribs, sirloin steak, carpaccio, veal, rabbit, topside, hump, liver, filet mignon, flank, boar, pork knee, tongue, loin, rump skirt, molejjas, black pudding, muscle, panceta [streaky bacon], pastrami, knuckle, pig feet, pork ham, top sirloin, brisket, oxtail, kidney, pork backbone
Processed meats	Ham, turkey blanquette, coppa, hamburger (beef, pepperoni) sausage (Calabrese sausage, pork, large pork sausage, Portuguese, Tuscan), pork loin and smoked pork loin, mortadella, pepperoni, roast beef, salami, hot-dog sausage, steak, tender
Cereals, grains, potatoes and flours	Cornstarch, arrowroot, rice (arboreal, type 1, oriental, parboiled, basmati, jasmine), cereal bar based on refined cereal, potato, rice and corn powder, flour), starch (potato and cassava), cornflakes, cornmeal, pasta, puff pastry, bread (baguette, Beirut, small roll, yam, ciabatta, sliced, roll, Italian, hot-dog/hamburger, milk, corn, egg, kneaded, Syrian, Swedish bread, wrap), cheese bread, tapioca, toast, wheat
Crackers and snacks	Crackers (water crackers, water and sesame crackers, saltine, salty crackers), cassava flour crackers, industrialized snacks (bacon snacks, potato chips, onion snacks, corn snacks)
Processed fats	Bacon, lard, whipped cream ready for consumption, vegetable cream, mayonnaise, margarine, sauce ready for consumption, pastes and pâtés - in all versions: traditional and light
Adequacy components
Poultry, fish and eggs	Haddock, anchovies, herring, tuna, ostrich, cod, dogfish, shrimp, crab, carp, chester, chicken heart, croaker, corimba, golden, swordsish, chicken, haddock, kani kama, lobster, herring, halibut, squid, shellfish, hake, grouper, mussel, gizzard, namorado, turkey, hake, white hake, whiting, neck, pintado, pirarucu, octopus, porquinho, oyster, egg (quail, chicken, duck), duck, sea bass, saint peter, salmon, sardines, crab, tambaqui, tilapia, trout
Cereals and whole grains	Amaranth, rice, oats, cereal bar based on whole cereal, whole oat biscuit, quinoa, bran (oat, wheat), flour (rye, flaxseed, quinoa, whole wheat), sesame seeds, granola, linseed, popcorn, cereal mixture, whole wheat bread (sliced bread, bread roll), popcorn, seeds (sesame, sunflower, chia), whole-wheat toast
Tubers and roots	Yams, manioc starch, yams, manioc, pine
Fruits	Avocado, pineapple, abiu, acai, acerola, coconut water, plum, red plum, blackberry, atemóia, avocado, bacuri, banana, buriti, hog plum, cajá-manga, cashew, persimmon, star fruit, cherry, coconut, ciriguela, cupuaçu, damascus, figs, raspberries, sweetsop, breadfruit, grapefruit, guava, guanabana, jaboticaba, jackfruit, jambo, kiwi, orange, lychee, lemon, apple, macaúba, papaya, mango, quince, passion fruit, watermelon, melon, tangerine, blueberry, strawberry, nectarine, loquat, pequi, pear, peach, pine cones, dragon fruit, pitanga, pomegranate, sapoti, date, tamarind, tangerine, tucuman, umbu, grape
Vegetables	Pumpkin, zucchini, artichoke, garlic, leeks, asparagus, olives, eggplant, beets, broccoli, sprouts (alfava, bamboo), onions, carrots, chayote, mushrooms, Brussels sprouts, cauliflower, endive, jambu, eggplant, gherkin, tomato sauce, turnip greens, hearts of palm, cucumber, bell pepper, okra, radishes, pumpkin seed, tomatoes, string beans
Leafy vegetables	Chard, watercress, celery, lettuce, alfava, kelp, seaweed, catalonha, chicory, collards, endive, spinach, mustard leaf, radicchio, cabbage, arugula, parsley, milkweed, taro
Legumes	Bean sprouts, soy cream, peas, soy flour, fava beans, chickpeas, soya milk, lentils, natto, soy protein, soybean curd cheese, soy, tofu
Oilseeds	Almond, peanut, hazelnut, chestnut of baru, Brazilian nut, cashew nuts, milk of almonds, macadamia nuts, pecan walnut, walnuts, pistachios, lupine
Milks and yogurts	Milk (skimmed, semi-skimmed or whole), plain yogurt (low fat, semi-skimmed or whole)
Cheeses	Brie, camembert, burrata cheese, cheddar, shanklish, chèvre or goat’s milk, rennet, cottage, cream cheese, cream of ricotta cheese, emmental, fetta, fused, gorgonzola, gouda, grana padano, gruyere, half cure, minas frescal, standard Minas cheese, mozzarella, buffalo milk mozzarella, parmesan, pingo de leite, port salut, queijo prato, provolone, quark, cottage cheese, ricotta, reino, roquefort, tilsit with kümmel, trança
Oils and fats	Palm oil, olive oil, extra virgin olive oil, vegetable oil (peanut, babassu, canola, coconut, toasted sesame, sunflower, flaxseed, corn, palm, pequi, soy), butter
Milk cream	Milk cream - in all versions: traditional and light

(i) Sugars and sweets: sugar-rich foods like sodas, chocolates, ice cream, or cookies can bring risks to health when there is excessive intake; therefore, to be part of a healthy eating, they should not be of frequent intake, and in small portion sizes ^31,32. Studies on sugar dependence and sweet taste dependence show that excessive intake of sugary foods or only sweet tasting, such as diet and light products, stimulates the pleasure derived from food and the amount of food intake ³³. Therefore, diet and light foods also compose this group.

(ii) Meats - beef, pork, and processed meat: they are important sources of protein and vitamin B12, but their intake is associated with the risk of developing colorectal cancer ³⁴; therefore, considering the risk caused by excess intake of these foods, quotas for ingestion must be established.

(iii) Refined cereals: considering their higher glycemic index and association with obesity, type 2 diabetes, metabolic syndrome, and cardiovascular diseases, foods from these groups should have their intake reduced ³⁵; for the same reason, potatoes are classified into this group ^36,37.

(iv) Processed fats: foods with high concentration of saturated fat and sodium, when consumed in excess are associated with the development of cardiovascular diseases, especially on diets high in refined carbohydrates ⁶. For this reason, they are among the moderation components.

(v) Poultry, fish, and eggs: the classification into a distinct group in relation to other meats relies in the understanding of healthy eating, since the intake of red meat and processed meat is associated with the risk of developing colorectal cancer ³⁴.

(vi) Whole cereals, tubers and roots: they are sources of dietary fiber, bioactive compounds, and have low glycemic index ³⁵.

(vii) Fruits and vegetables: intake of these fresh foods, sources of micronutrients and bioactive compounds, is associated to lower risk for developing cardiovascular diseases and to the prevention of the most chronic diseases ^6,38.

(viii) Legumes and oilseeds: beans are part of the Brazilian staple foods ⁶ and are sources of proteins, fibers, vitamins, monounsaturated fatty acids, and minerals ¹⁷. Although oilseeds also provide these nutrients, there is no indication that supports the need for their daily intake; they should be part of a healthy diet in order to vary the food repertoire.

(ix) Milk and dairy products: dairy products present high concentrations of nutrients, especially calcium, which justifies its inclusion in the repertoire of healthy food choices ⁶.

(x) Oils and fats: they provide essential fatty acids for biochemical processes related to various systems, in addition to being a protection factor for cardiovascular diseases ^17,38.

(xi) Alcoholic beverage: its consumption is not evaluated by the DQI-DFG-2019. Alcohol is the most used drug in the world and is responsible for 3.2% of deaths in Brazil - a social and public health problem ³⁹. Although moderate intake of wine is recognized as part of the strategies for the prevention of cardiovascular diseases, alcohol is among the risk factors for cancer ⁴⁰.

Construct validation

Improvement of the reference diet for determining portion sizes and ranges of intake of each food group

We developed an RD (2,000Kcal) consisting of typical foods that preserve aspects of culture and preferences of the Brazilian population. The dietary calculation of this RD met the quotas of nutrients and energy density so they are consistent with the reduction of chronic diseases and of the risk of nutritional deficiencies:

(i) Breakfast: 0.5 small unit (151g) of papaya, 8 leveled tablespoons (30g) of skimmed cow’s milk powder, 2 slices (57g) of whole wheat sliced bread, and 4 leveled tablespoons (30g) of creamy curd cheese;

(ii) Mid-morning snack: 1 large unit (110g) of yellow cavendish banana with 1 tablespoon shallow (7g) of flaked oats;

(iii) Lunch: 2 saucers full (50g) of lettuce, 1 medium unit (80g) of salad tomato, 1 teaspoon (2g) of flaxseed, 2 tablespoons (8mL) of extra virgin olive oil, 80g of raw brown rice, 40g of beans, 1 small unit (100g) of raw chicken breast, 0.5 small unit (30g) of raw carrot, 1 medium unit (110g) of orange. Seasonings: 2 teaspoons (2g) of refined salt, 1 medium unit (70g) of onion, 10 cloves (20g) of garlic, 4 teaspoons (8mL) of soy oil.

(iv) Afternoon snack: 1 unit (160g) skimmed natural yogurt with 70g of Palmer mango and 2 dessert spoons (16g) of traditional granola;

(v) Dinner: soup: 2 tablespoons (30g) of chickpeas, 2 saucers (40g) of kale, 2 medium pieces (50g) of pumpkin, 2 medium pieces (130g) of yams, 1 unit (50g) of chicken egg, 1.5 teaspoon (1.5g) of refined salt, 2 teaspoons (3mL) of extra virgin olive oil, 2 teaspoons (3mL) of soy oil and 1 small slice (100mL) of watermelon;

(vi) Night snack: 11 units (3-4g) of Brazilian nuts, 0.5 tablespoon (3-5g) of salted roasted peanuts, and 2 tea cups (180mL) of fennel tea.

The portion sizes for each food group represented the core value for the definition of the adequate intake ranges. Table 2 shows the corresponding energy according to the food groups and the proposed intake ranges.

Table 2
Energy correspondent (Kcal), intake interval (portion/1,000Kcal) and score, according to food groups of the update from the Diet Quality Index associated with Digital Food Guide (DQI-DFG-2019). Brazil, 2019.

Food groups	EC	II	Score according to the intake interval
			Below	Inside	Above
Moderation components
Sugar and sweets	155	0.0-0.5	-	0.5	≤ 1.00: 0.5 + (0.5 - portion)
Dairy drinks and sweetened yogurts	120				> 1.00: 0.00
Meats: beef, pork and processed meat
Bovine and pork meat	100	0.0-0.5	-	0.5	≤ 1.00: 0.5 + (0.5 - portion)
Processed meats	50				> 1.00: 0.00
Refined cereals
Cereal, grains, potatoes and flours	140	0.0-1.0	-	0.5	≤ 2.00: 0.5 + (0.5 x (1 - portion)
Crackers and snacks	60				> 2.00: 0.00
Processed fats	90	0.0-0.5	-	0.5	≤ 1.00: 0.5 + (0.5 - portion)
Adequacy components					> 1.00: 0.00
Poultry, fish and eggs	140	0.5-1.0	(portion x 1.25)/0.5	1.25	1.25
Whole cereals, tubers and roots
Cereal and whole grains	140	2.0-3.0	(portion x 0.5)/2.0	0.50	≤ 6: 0.5 + (0.5 x (3-portion)/3)
Tubers and roots	120				> 6: 0.00
Fruits	65	1.5-3.0	(portion x 1.50)/1.5	1.50	≤ 6: 1.50 + (1.50 x (3-portion)/3)
					> 6: 0.00
Vegetables
Vegetables	20	2.0-3.0	(portion x 1.50)/2.0	1.50	1.50
Leafy vegetables	10
Legumes and oilseeds					1.50
Legumes	150	1.0-1.5	(portion x 1.5)	1.50
Oilseeds	50
Milk and dairy products
Milk and yogurts	120	1.0-1.5	portion	1.00	≤ 3: 0.5 + (0.5 x (1.5-portion)/1.5)
Cheeses	80				> 3: 0.00
Oils and fats
Oils and fats	72	1.25-1.75	(portion x 0.75)/1.25	0.75	0.75
Cream	144

• EC: energy correspondent; II: intake interval.
• Note: add the score of each food group and apply the following formula to generate values between 0 and 100 points: total score x 10.

a) Moderation components

In the DQI-DFG-2013 construct, the score attributed to these components was dichotomous (within the intake range they receive the maximum points - 5 or 10; when the range was exceeded, the score was set to zero) ¹³. In this improvement, when the portion is within the intake range, the score is fixed at 0.5 points; if the portion is up to twice the upper limit of the intake range the score will be proportionally decreasing; above twice the value these food groups will not receive points. Thus, we protect the diet from excessive refined carbohydrates, high glycemic index foods, and saturated fats ^15,16.

Twice the upper limit of the intake range: if the range is 0.0-0.5 portions, twice the upper limit is 1.0; to 0.4 portions of sugars and sweets is attributed 0.5 point; if 0.8 portions, proportional descending score (0.2 points); if 1.4 portions, 0 points.

b) Adequacy components

In the DQI-DFG-2013 construct, the score was proportionally increasing up to reaching the minimum value of the intake range; within or above the intake range, each food group received its maximum score (5 or 10 points) ¹⁴. In this improvement, portions below and within the intake range remain as in the DQI-DFG-2013. However, if the portion is above the intake range, the score will be different according to the food group: for Whole cereals, tubers, and roots, Fruits, and Milk and dairy products, if the portion is up to twice the upper limit of the intake range the score will be proportionally decreasing; and above twice the upper limit of the intake range, these food groups will not receive points. For the groups Poultry, fish, and eggs, Vegetables, Legumes and oilseeds, and Oils and fats, even with portion values above the intake range, the score remains fixed.

Twice the upper limit of the intake range: if the range is 1.5-3.0 portions, twice the upper limit is 6.0. To 1.3 portions of Fruit is assigned a proportionally increasing score (1.3 points); if 2.0 portions, 1.5 points; if 5.0 portions, proportionally decreasing score (0.5 points); if 6.4 portions, 0 points.

Consuming fruits, legumes, vegetables, whole cereals, poultry, fish, eggs, milk and dairy products, vegetable oils and oilseeds regularly is compatible with a healthy diet, as it ensures the supply of nutrients in order to meet the nutritional needs for adults ¹⁴. However, high intake of Whole cereals, tubers, and roots and Fruits can be associated to the development of obesity, type 2 diabetes, metabolic syndrome, and cardiovascular diseases, due to increasing the glycemic index of the whole diet ⁴¹. High intake of Milk and dairy products can exceed the recommendation of 10% for saturated fats ²⁷, favoring the development of cardiovascular diseases ⁴².

DQI generates values between 0 and 100 points: < 40 points, low quality diet; 40-70 points, intermediate quality diet; > 70 points, good quality diet. These cut-off points were defined according to the qualitative characteristic of the study’s menus and can be adjusted according to the criteria defined by other researchers to be adapted to the different patterns of consumption.

Correlation between the components of the DQI-DFG-2019 and the energy value

The organization of the food groups into moderation or adequacy components is adopted by the HEI-2005 ¹² and HEI-2010 ^9,10. Considering that food intake is positively correlated with energy intake, the calculation of the number of portions was proportional to 1,000Kcal, similarly to the procedure adopted in the DQI-DFG-2013 ¹⁴. The data in Table 3 show that Refined cereals is the group that resulted in the higher correlation with energy, however very weak (r = 0.24), suggesting that all the Index components are independent from the energy value of the menu. This result is similar to that observed for the application of the HEI-2010 (r = -0.21 for the component Fruit) ^9,10. The seven components that showed significant correlation with the energy resulted in negative coefficients, except for Refined cereals - the same result presented in the HEI-2010 ^9,10.

Table 3
Pearson’s linear correlation coefficient between the score of each component, the energy value and the total score of the Diet Quality Index associated with Digital Food Guide (DQI-DFG-2019). Brazil, 2019.

	SS	ME	RC	PF	PFE	WC	FR	VT	LO	MD	OF	TS	En
SS	1.00
ME	-0.04	1.00
RC	-0.13 *	-0.09 **	1.00
GF	0.00	-0.04	0.13 *	1.00
PFE	-0.03	0.40 *	-0.07	-0.02	1.00
WC	0.02	0.09 **	0.16 *	0.10 *	-0.06	1.00
FR	0.02	0.07 **	0.12 *	0.02	-0.06	0.15 *	1.00
VT	0.03	0.16 *	0.03	0.03	0.04	0.04	0.21 *	1.00
LO	0.20 *	-0.08 **	0.02	0.10 *	-0.06	-0.03	-0.12 *	-0.10 *	1.00
MD	-0.05	0.04	-0.06	0.03	0.07 **	0.11 *	0.04	0.03	-0.08 **	1.00
OF	0.28 *	0.05	-0.24 *	0.13 *	0.11 *	-0.07	-0.04	0.06	0.23 *	-0.04	1.00
TS	0.25 *	0.40 *	0.10 *	0.25 *	0.48 *	0.18 *	0.48 *	0.49 *	0.29 *	0.32 *	0.33 *	1.00
En	-0.10 *	-0.16 *	0.24 *	-0.11 *	-0.02	-0.07	-0.10 *	-0.22 *	-0.01	-0.02	-0.10 *	-0.19 *	1.00

• En: energy; FR: Fruit; LO: Legumes e oilseeds; MD: Milk and dairy products; ME: Meats: beef, pork and processed meat; OF: Oils and fats; PF: Processed fats; PFE: Poultry, fish and eggs; RC: Refined cereals; SS: Sugar and sweets; TS: total score; VT: Vegetables; WC: Whole cereals, tubers and roots.
• * p < 0.01;
• ** p < 0.05.

Analysis of agreement between the experts’ opinion about the quality of menus and the result of the DQI-DFG-2019

a) Profile of the experts

Of the 67 experts selected and invited into the study, 26 (39%) agreed to participate, with 73% of them being affiliated with public institutions, regionally distributed as follows: 7% from the Northeast, 27% from the Central, 47% from the Southeast, and 19% from the South region of the country. All the researchers have worked on the theme of Healthy Eating for at least six years (46% for 6-10 years, 35% for 11-20 years, and 19% for 20-30 years); 35% of them classified their own experience on the subject as “proper qualification” and 65% as “highly qualified”. Regarding the theme Diet Quality Index, 27% report having experience above 10 years, 50% between five and nine years, 23% between two and four years; 27% classify their condition as “highly qualified”, 65% understand that they have “proper qualification”, and 7% declare themselves with “inappropriate qualification”. These data show homogeneity between the research participants regarding knowledge on the subjects under study.

b) Analysis of the menus through the classifications “good”, “regular” and “bad”

For the ten menus evaluated by the 26 experts, 84.6% of the classifications attributed agreed with the result presented by the DQI-DFG-2019. All the analyses resulted in high agreement, except for menu 5 (54% agreement).

c) Analysis of the menus through grades (0 to 10 points)

Table 4 shows the coefficients of the association between the score assigned by each expert in the ten menus and the result of the DQI-DFG-2019. The results obtained allow to claim that there is strong correlation in all situations studied.

Table 4
Pearson’s linear correlation coefficients calculated between the score attributed by the experts on the menus and the result obtained by the Diet Quality Index associated with Digital Food Guide (DQI-DFG-2019). Brazil, 2019.

Experts	Coefficient	Confidence interval
1	0.91	0.64	0.98
2	0.78	0.31	0.95
3	0.91	0.65	0.98
4	0.90	0.62	0.98
5	0.92	0.68	0.98
6	0.94	0.74	0.99
7	0.82	0.40	0.96
8	0.90	0.61	0.98
9	0.84	0.45	0.96
10	0.94	0.75	0.99
11	0.91	0.64	0.98
12	0.91	0.67	0.98
13	0.90	0.61	0.98
14	0.91	0.66	0.98
15	0.78	0.29	0.94
16	0.91	0.66	0.98
17	0.91	0.65	0.98
18	0.94	0.75	0.99
19	0.97	0.87	0.99
20	0.94	0.76	0.99
21	0.91	0.64	0.98
22	0.93	0.74	0.98
23	0.87	0.53	0.97
24	0.93	0.72	0.98
25	0.88	0.57	0.97
26	0.85	0.47	0.96

Reliability

Reliability based on the evaluators

To determine if the Index produces the same score in each application and in identical situations there are some statistical tools available. As the result of the DQI-DFG, by definition, is similar for similar diets, when collected and registered in the same way, sources of error are attributed to the respondent, to the data collection, or to their processing. For this reason, we opted for reliability from the evaluators through preset menus for all of them. The agreement analysis between the result of the DQI-DFG-2019 and the result of each evaluator allows to conclude that they all agree with the result of the DQI-DFG-2019 (Table 5), and that the evaluators agree between themselves (correlation coefficient of 0.99 [CI: 0.98-1.00]; analysis of variance: p = 0.678).

Table 5
Analysis of agreement between the results presented by the Diet Quality Index associated with Digital Food Guide (DQI-DFG-2019) and the evaluators from the descriptive analysis, intraclasses correlation coefficient (ICC), test t (t) and signal test (ST). Brazil, 2019.

Evaluators	Average	SD	Minimum	Maximum	ICC			t	ST
					R	95%CI		p-value	p-value
1	50.98	24.30	14.80	76.43	0.98	0.93	1.00	0.36	1.00
2	54.11	25.18	15.15	85.95	0.99	0.96	1.00	0.09	0.76
3	52.80	24.42	14.70	83.75	1.00	0.98	1.00	0.57	0.88
4	52.50	24.21	15.00	82.40	0.99	0.98	1.00	0.89	1.00
5	51.98	24.25	15.35	80.90	1.00	0.99	1.00	0.56	1.00
6	52.13	24.28	15.00	78.60	1.00	0.98	1.00	0.75	0.82
7	52.64	24.85	14.80	81.10	1.00	0.98	1.00	0.73	0.94
8	52.82	25.24	14.85	82.50	0.99	0.98	1.00	0.60	0.80
DQI-DFG-2019	52.38	24.15	14.60	83.20	-	-	-	-	-

• 95%CI: 95% confidence interval; SD: standard deviation.
• Note: p > 0.05.

Correlation between the components of the DQI-DFG and the total score

Since the diet may not satisfy the reference standard for all food groups, we conducted analysis of the correlation between each of the Index components and the total score to know which component has greater influence on the total score of the Index. Moderate and high correlations indicate greater influence on the final result. Therefore, the Index will present better results when it has proportional and similar correlations. This study presents correlation ranged from r = 0.10 for Refined cereals and r = 0.49 for Vegetables (Table 3), that is, weak or very weak correlations. Although the groups of Vegetables and Fruits are the groups with the highest correlation in comparison to the others (r = 0.49 and r = 0.48, respectively), both values are classified as weak correlation. This is a better result than that found in HEI-2010 ^9,10, where seven components presented moderate correlations.

According to Cronbach ⁴³, internal consistency analysis evaluates the degree of association between the internal components of the instrument through the means of the correlations between all possible combinations. Several authors, such as those responsible for the publication of the HEI ^8,9,10,12, use Cronbach’s alpha coefficient as a way to assess reliability through the internal consistency of the instrument while acknowledging its limitations ¹⁰. However, this analysis presents weakness if applied to the DQI-DFG, since the internal components of this Index do not correlate. Applying this coefficient would be an assessment referring to the response pattern of the diets studied and not to a characteristic of the instrument, because the value of alpha will suffer modifications depending on the pattern of diets to which the Index is applied ⁴⁴.

Limitations of the DQI-DFG

The limitations of this study are focused on the construction of the Index, that is, for its content validation. There are some concepts of dietary technique that need to be analyzed and contemplated in the Index in order to preserve the nutritional value of foods. It is possible to highlight some points: presence of fruit juices (There is loss of fibers and nutrients during preparation, and satiety is lower ⁶), preparation technique, sources of organic food and division of meals throughout the day.

Final considerations

The DQI-DFG showed consistent validity and reliability characteristics according to the criteria adopted in this study. This Index is the first to submit the criteria established for its development to a consultation with experts in the field, which provided authentic results about the diet quality. Construct validation based on a reference diet is another element that differentiates the development of this index, either from national or international ones. Other dimensions of the DQI-DFG that also express the original contribution of this study include the identification of Legumes as an adequacy component, the non-inclusion of alcohol to obtain the total score, the composition of the components of analysis focused on food groups and not on nutrients, and the attribution of decreasing points when there is excess intake of Cereals, tubers and roots, Fruits, and Milk and dairy products. This assessment is based on the agreement between the experts’ opinion and the results generated by the Index, both in relation to the classification of the diet into categories and to the result of the total score. The reproducibility of the DQI-DFG was confirmed through its application by different evaluators.

Therefore, the results of this work contributed to the construction of the components of the DQI-DFG-2019 through parameters supported by scientific evidence, by the experts’ opinion, and by a diet analysis model that promotes the valuation of regional eating habits, considering that the simplicity of the moderation and adequacy components allow the classification of diverse eating patterns.

Maintaining a regular pattern with regard to healthy eating promotes the conditions for protection against chronic non-communicable diseases and deficiency diseases. Understanding diet quality through monitoring and evaluating food intake is a strategy that can promote healthy choices and assist in nutritional care. This Index assesses diet quality to facilitate the monitoring and evaluation of food intake, as well as to know individuals dietary patterns.

Thus, it is possible to apply this Index as a tool for food intake studies. Knowing the dietary patterns of the population promotes the development of health care and intervention actions as well as the definition of public policies. This Index can also be used as a food and nutrition education tool. When the individual can explore the characteristics of their diet through an indicator and composition in food groups, it is possible to generate autonomy and freedom of food choice. Thus, this tool can be suggested as a complement to the guidelines of the current Food Guide for the Brazilian Population.

Acknowledgments

We would like to thank you for giving away the data from ISA-Camp 2008.

References

• ¹
  Conde WL, Monteiro CA. Nutrition transition and double burden of undernutrition and excess of weight in Brazil. Am J Clin Nutr 2014; 100:1617S-22S.
• ²
  Rivera JA, Pedraza LS, Martorell R, Gil A. Introduction to the double burden of undernutrition and excess weight in Latin America. Am J Clin Nutr 2014; 100:1613S-6S.
• ³
  World Cancer Research Fund; American Institute for Cancer Research. Diet, nutrition, physical activity and cancer: a global perspective. Continuous update project expert report. Washington DC: American Institute for Cancer Research; 2018.
• ⁴
  Ministério da Saúde. Pesquisa de Orçamentos Familiares 2008-2009. Avaliação nutricional da disponibilidade de alimentos no Brasil. Brasília: Ministério da Saúde; 2010.
• ⁵
  GBD 2017 Diet Collaborators. Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2019; 393:1958-72.
• ⁶
  Departamento de Atenção Básica, Secretaria de Atenção à Saúde, Ministério da Saúde. Guia alimentar para a população brasileira. Brasília: Ministério da Saúde; 2014.
• ⁷
  Dixon LB. Updating the healthy eating index to reflect current dietary guidance. J Am Diet Assoc 2008; 108:1837-42.
• ⁸
  Kennedy ET, Ohls J, Carlson S, Fleming K. The healthy eating index: design and applications. J Am Diet Assoc 1995; 95:1103-8.
• ⁹
  Guenther PM, Casavale KO, Reedy J, Kirkpatrick S, Hiza HAB, Kuczynski KJ, et al. Update of the healthy eating index: HEI-2010. J Acad Nutr Diet 2013; 113:569-80.
• ¹⁰
  Guenther PM, Kirkpatrick S, Reedy J, Krebs-Smith SM, Buckman DW, Casavale KO, et al. The Healthy Eating Index-2010 is a valid and reliable measure of diet quality according to the 2010 Dietary Guidelines for Americans. J Nutr 2014; 144:399-407.
• ¹¹
  U.S. Department of Agriculture; U.S. Department of Health and Human Services. Dietary guidelines for Americans 2010. 7th Ed. Washington DC: U.S. Government Printing Office; 2010.
• ¹²
  Guenther PM, Reedy J, Krebs-Smith SM. Development and evaluation of the Healthy Eating Index-2005: technical report. Washington DC: Center for Nutrition Policy and Promotion; 2007.
• ¹³
  Previdelli AN, Ferreira SRG, Fisberg RM, Marchioni DM. Índice de Qualidade da Dieta revisado para população brasileira. Rev Saúde Pública 2011; 45:794-8.
• ¹⁴
  Caivano S, Domene SMA. Diet quality index for healthy food choices. Rev Nutr 2013; 26:693-9.
• ¹⁵
  Harvard School of Public Health. Food pyramid and plates: what should you really eat? http://www.hsph.harvard.edu/nutritionsource/pyramid-full-story/ (accessed on Aug/2012).
  » http://www.hsph.harvard.edu/nutritionsource/pyramid-full-story/
• ¹⁶
  Harvard School of Public Health. Harvard researchers launch Healthy Eating Plate. https://www.hsph.harvard.edu/news/press-releases/healthy-eating-plate/ (accessed on Aug/2012).
  » https://www.hsph.harvard.edu/news/press-releases/healthy-eating-plate/
• ¹⁷
  Willett W. Eat, drink, and be healthy: the Harvard Medical School guide to healthy eating. New York: Simon and Schuster; 2011.
• ¹⁸
  Alves MCGP. Plano de amostragem. In: Barros MBA, César CLG, Carandina L, Goldbaum M, organizadores. As dimensões da saúde: inquérito populacional em Campinas. São Paulo: Editora Hucitec; 2008. p. 46-55.
• ¹⁹
  Rowe G, Wright G. The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 1999; 15:353-75.
• ²⁰
  Wakita T, Ueshima N, Noguchi H. Psychological distance between categories in the Likert Scale: comparing different numbers of options. Educ Psychol Meas 2012; 72:533-46.
• ²¹
  Caivano S, Domene SMA. Consensus among experts on healthy eating and diet quality index. Ciênc Saúde Colet 2018. http://www.cienciaesaudecoletiva.com.br/artigos/consensus-among-experts-on-healthy-eating-and-diet-quality-index/16995?id=16995
  » http://www.cienciaesaudecoletiva.com.br/artigos/consensus-among-experts-on-healthy-eating-and-diet-quality-index/16995?id=16995
• ²²
  Núcleo de Estudos e Pesquisas em Alimentação. TACO - Tabela Brasileira de Composição de Alimentos. 4a Ed. Campinas: Universidade Estadual de Campinas; 2011.
• ²³
  U.S. Department of Agriculture. National Nutrient Database for Standard Reference, release 26. https://ndb.nal.usda.gov/ndb/search/list (accessed on Jun/2011).
  » https://ndb.nal.usda.gov/ndb/search/list
• ²⁴
  Institute of Medicine. Dietary reference intakes for thiamin, riboflavin, niacin, vitamin B6, folate, vitamin B12, pantothenic acid, biotin, and choline. Washington DC: The National Academies Press; 2000.
• ²⁵
  Institute of Medicine. Dietary reference intakes for vitamin C, vitamin E, selenium, and carotenoids. Washington DC: The National Academies Press; 2000.
• ²⁶
  Institute of Medicine. Dietary reference intakes for vitamin A, vitamin K, arsenic, boron, chromium, copper, iodine, iron, manganese, molybdenum, nickel, silicon, vanadium, and zinc. Washington DC: The National Academies Press; 2001.
• ²⁷
  Institute of Medicine. Dietary reference intakes for energy, carbohydrate, fiber, fat, fatty acids, cholesterol, protein, and amino acids (macronutrients). Washington DC: The National Academies Press; 2002.
• ²⁸
  Institute of Medicine. Dietary reference intakes for water, potassium, sodium, chloride, and sulfate. Washington DC: The National Academies Press; 2005.
• ²⁹
  Institute of Medicine. Dietary reference intakes for calcium and vitamin D. Washington DC: The National Academies Press; 2010.
• ³⁰
  Sociedade Brasileira de Cardiologia. IV diretriz brasileira sobre dislipidemias e prevenção da aterosclerose. Arq Bras Cardiol 2007; 88:2-19.
• ³¹
  Martins APB, Levy RB, Claro RM, Moubarac JC, Monteiro CA. Increased contribution of ultra-processed food products in the Brazilian diet (1987-2009). Rev Saúde Pública 2013; 47:656-65.
• ³²
  Parylak SL, Koob GF, Zorrilla EP. The dark side of food addiction. Physiol Behav 2011; 104:149-56.
• ³³
  Lustig RH, Schmidt LA, Brindis CD. The toxic truth about sugar. Health Policy 2012; 482:7-9.
• ³⁴
  American Institute for Cancer Research. Food, nutrition, physical activity and prevention of cancer: a global perspective. Washington DC: World Cancer Research Fund/American Institute for Cancer Research; 2007.
• ³⁵
  Katz DL, Meller S. Can we say what diet is best for health? Annu Rev Public Health 2014; 35:83-103.
• ³⁶
  Bach-Faig A, Berry EM, Lairon D, Requant J, Trichopoulou A, Dernini S, et al. Mediterranean diet pyramid today. Science and cultural updates. Public Health Nutr 2011; 14:2274-84.
• ³⁷
  Muraki I, Rimm EB, Willett WC, Manson JE, Hu FB, Sun Q. Potato consumption and risk of type 2 diabetes: results from three prospective cohort studies. Diabetes Care 2016; 39:376-84.
• ³⁸
  Bonaccio M, Iacoviello L, de Gaetano G. The Mediterranean diet: the reasons for a success. Thromb Res 2012; 129:401-4.
• ³⁹
  Laranjeira R, Pinsky I, Zaleski M, Caetano R. I levantamento nacional sobre os padrões de consumo de álcool na população brasileira. Brasília: Secretaria Nacional Antidrogas; 2007.
• ⁴⁰
  Brenner H, Kloor M, Pox CP. Colorectal cancer. Lancet 2014; 383:1490-502.
• ⁴¹
  Santos FL, Esteves SS, da Costa Pereira A, Yancy Jr. WS, Nunes JP. Systematic review and meta-analysis of clinical trials of the effects of low carbohydrate diets on cardiovascular risk factors. Obes Rev 2012; 13:1048-66.
• ⁴²
  Siri-Tarino P, Sun Q, Hu FB, Krauss RM. Saturated fat, carbohydrate, and cardiovascular disease. Am J Clin Nutr 2010; 91:502-9.
• ⁴³
  Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16:297-334.
• ⁴⁴
  Streiner DL. Being inconsistent about consistency: when coefficient alpha does and doesn't matter. J Pers Assess 2003; 80:217-22.

Publication Dates

• Publication in this collection
  16 Sept 2019
• Date of issue
  2019

History

• Received
  05 Mar 2019
• Reviewed
  15 May 2019
• Accepted
  31 May 2019