Development of a quantitative health inspection instrument in food and nutrition services, Brazil

The study aimed to develop a Quantitative Health Inspection Instrument (IQIS) large-sized Brazilian food and nutrition services. The inspection technology based on the Potential Risk Assessment Model (MARP) and the Brazilian Health Legislation was used. Twelve dimen-sions, 41 modules, and 57 risk control (critical/ non-critical) indicators were structured on a scale of 0-5, totalling 1,512 indices with closed-ended response coding. The IQIS was validated with the Kappa Coefficient, with excellent agreement for the attributes of clarity and relevance (k = 0.82 and k = 0.92) and good agreement for applicability (k = 0.78). The Kruskal-Wallis test showed no statistically significant difference between the assessments (p = 0.423), the Intraclass Correlation Coefficient was satisfactory (ICC = 0.53), and Cronbach’s Alpha ( α = 0.71) was acceptable. The final result made it possible to classify the service as having an unacceptable health risk. IQIS is considered to have validated content, be reliable and reproducible to assess the hygienic-sanitary conditions, being a technological innovation for food and nutrition services and sanitary, allowing a detailed and rigorous inspection.


introduction
Foodborne diseases (FBD) are common and underreported in Brazil. The World Health Organization (WHO) estimates that one-third of the population will suffer from FBD annually, but only a small part will make the notification 1-3 . According to the Centers for Disease Control and Prevention (CDC), 48 million people fall ill each year from FBD in the U.S. In 2014, the CDC reported 864 outbreaks, resulting in 13,246 diseases, 712 hospitalizations and 21 deaths 4 . In Brazil, 11,241 outbreaks were recorded in the period 2000-2015. Of these, 218,507 people became ill, and 2,121,110 were exposed. Fifteen percent of the outbreaks were related to food consumed in restaurants and bakeries, and 8.2% with housing and work 5 . Food contamination can occur throughout the production chain, and risk management is a significant health control measure, which is essential to avoid it 6,7 . A considerable proportion of food outbreaks arise from the association between the consumption of food contaminated by improper handling and conservation or distribution 8,9 .

regulation of health risk in food services
The oldest technical health control regulation is the Codex Alimentarius, established by the United Nations Food and Agriculture Organization and the WHO 10 . In 1993, the Food Hygiene Committee of the Codex published a guide for the application of the Hazard Analysis and Critical Control Point (HACCP) system, whose application in Brazil is based on Ordinance Nº 1.428/1993 11,12 of the Ministry of Health. In Brazil, the publication of RDC Nº 216/2004 is a health control milestone, establishing minimal conditions for food and nutrition services, but without an inspection roadmap 13  The health inspection technology is highlighted as an instrument of risk management, assessing compliance with health legislation 7 throughout the food chain. The usual health inspection routines evaluate the services using dichotomous variables with compliant and non-compliant indicators, which show the level of compliance with the legislation, without judging the criticality of the analyzed items, such as the Roadmap for the Assessment of Hygienic-Sanitary Conditions in Food Services 15 . The Integrated Handbook for Prevention and Control of Foodborne Diseases 7 , published by the Ministry of Health, provides for an inspection roadmap that defines a criterion for classifying indicators based on the relevance of individual risk regarding product quantity and safety and workers' safety. The School Feeding Good Practices Checklist 16 considers the consequence potential of each indicator. On May 10, 2013, the Ministry of Health approved Ordinance Nº 817/2013, with the national guidelines for the elaboration and implementation of the Food Service Categorization Pilot Project 17 , implemented in the host cities of the 2014 FIFA World Cup, providing for the classification of the criticality of each indicator, health and safety consequences, level of association concerning outbreak-associated flaws, and weighting to establish values 18 .

technological innovation for large food services
Health surveillance has three sets of practices, with varying risk notions depending on the strategy. Health promotion actions aimed at group education aim at increasing the quality of health of the population and are unrelated to a specific risk factor. Risk or harm prevention actions act on specific factors, based on epidemiological risk, to reduce or eliminate new occurrences. Health protection actions seek to strengthen defenses. They address risk as possible harmful events to health 19 . Considering its dynamic nature, it is necessary to search for new technologies that consider the complex and cross-cutting nature of the processes and manage health risks 20 . All the health inspection guidelines found in the literature 15,16,18,21 are based on the probabilistic risk concept. The notion of risk concept, of great relevance in the area of health surveillance, was thus proposed 22 to cover the complexity of a risk concept.
The potential risk has two essential characteristics that differentiate it from the concept of classic risk, relating to the possibility and not to the probability of occurrence of the unexpected. A classical risk evaluation is based on events that have occurred, while the potential risk builds on those that are occurring and the effects that may or may not occur. The potential risk can be quantified and classified into levels of acceptability, and its operationalization enables the monitoring and comparison of several objects under health surveillance control. In this context, the Potential Risk Assessment Model (MARP) was developed to gauge risk and its classification in the space of acceptability 19 . It classifies the potential risk into acceptable, tolerable and unacceptable through mathematical formalism. MARP's application is particularized from the risk control indicators, based on a defined acceptability scale classified into critical and non-critical 22 categories.
This study aimed at developing the IQIS for large food services based on the MARP and Brazilian legislation. The use of the potential risk concept evidenced advances in health risk management in related areas such as hemodialysis 20 and radiodiagnosis 22 services. Its application in food and nutrition services is a technological innovation, enabling risk anticipation and health protection.

Methods
This study was developed from October 2015 to May 2016, with the following steps: i) Elaboration of the instrument by sector health professionals; ii) IQIS evaluation concerning content validity through submission to the Expert Committee, as well as external validity and reliability for the performance of Pretest and Test, applied in large food and nutrition service in São Sebastião do Passé, Bahia, Brazil, which prepares 2,541 daily meals for workers: 276 breakfasts, 1,393 lunches, 243 dinners and 629 snacks. The Research Ethics Committee of the Bahia School of Medicine and Public Health approved the research. Figure  1 summarizes the IQIS development and validation process.

elaboration of the iQiS
This stage counted on a group of seven professionals, of which two are doctors in public health and risk management researcher, two are ergonomists and managers in workers' health, two are nutritionist experts in public health and experience in collective nutrition, and one is a health surveillance expert working in the area of risk management. The literature review allowed the analysis of the existing instruments and methods, as well as the theoretical basis for the decision and elaboration of a new instrument. Following the identification of primary sources, review of secondary sources, critical reading and summary of the literature was performed using keywords or descriptors: risk, health risk, epi-demiological risk, potential risk, health surveillance, health inspection, food health inspection, health inspection roadmap, health inspection in restaurants roadmap, industrial kitchen health inspection roadmap, health inspection validation, questionnaires, evaluation instruments, and elaboration and validation of instrument 21,[23][24][25][26][27][28] .
The IQIS is based on the MARP, whose mathematical formalism is detailed by Navarro 22 , applied to food and nutrition services. This model proposes underlying processes according to the natural flow of the inspector. The main activities that are a potential risk for each of these processes are defined, as are the risk control indicators for each of these activities. Finally, each indicator is classified as critical (I C ) and non-critical (I NC ) and associated with an interval scale from zero to 5, where zero evidences nonexistent or inadequate risk control and five is excellent risk control, with the following classification: 0 -nonexistent or inadequate; 1 -insufficient; 2 -reasonable; 3good; 4 -very good and 5 -excellent.
The coding of closed-ended answers for each index of the scale was developed to reduce the subjectivity of the evaluator, with six possibilities for each indicator. The food production flow was used as a criterion to define the realms, which were broken down into activities, areas or equipment that generate potential risk, called modules. These developed into risk control indicators and were associated with indices with quantitative variables, coded in closed-ended responses. Fourteen eight-hour meetings were held with professionals, who employed RDC N o 216/2004. The requirements to meet the resolution were considered in index 3 of each indicator, and the lower and upper grades were defined from the experience and literature researched.
A panel was developed with the professionals to classify indicators into critical and non-critical; these professionals used their knowledge for this task and, later, the table of the association of risk factors for the occurrence of outbreaks was elaborated by National Health Surveillance Agency specialists 18 . The panel was developed from three 8-hour meetings, in which each member expressed his opinion on the indicator in question, achieving the group's consensus.

iQiS validation
Many validity and reliability methods are mentioned in the research literature 29-32 and, concerning methodological conduction, we chose the definitions that establish validity as how much a test measures what we wish to gauge and what reliability is related to the accuracy and precision of the measurement procedure 33 .
A Judges' Committee was established to validate the content and consisted of three nutritionists, with average professional experience in the areas of food safety and health surveillance of 17 years; one worked as a teacher and two at the Regional Council of Nutritionists. One had a master's degree, and two had specialization in health surveillance. The first version of the IQIS was submitted to the Judges' Committee, along with the electronic spreadsheet for content evaluation, regarding the attributes of applicability, clarity, and  relevance. Applicability was evaluated on a scale with four alternatives: 1 -fully agree, 2 -partially agree, 3 -no opinion, and 4 -disagree. Clarity was evaluated on a scale with three alternatives: 1 -high, 2 -fair and 3 -low, to verify whether the indicators were written intelligibly. Relevance was assessed on a scale with three alternatives: 1 -relevant, 2 -not relevant, and 3 -not applicable, noting if the items reflected the concepts involved and if they were adequate to achieve the proposed objectives. An eight-character code was created for each set of realm, module, indicator and index to organize the database generated in this stage. Experts gathered for two days in a quiet and uninterrupted environment, answered the 1,344 evaluations and were instructed not to exchange information with each other. The Kappa coefficient was used to analyze the results, considering k > 0.80-1.00 as excellent concordance; k > 0.60-0.79, good concordance; k > 0.40-0.59, moderate concordance; k > 0.20-0.39, poor concordance and k > 0-0.19, no concordance 34 . All results with a Kappa Coefficient above 0.60 were accepted, i.e. showing good concordance.
The Pretest checked whether all items were understandable to members of the target population. To that end, we invited five nutritionists, with a mean age of 42 years and an average of 14 years of experience in this area, all with specialization in health surveillance or related areas. The IQIS was applied on the same day, by direct observation for all the risk control indicators, except for the documentary ones, answered by interviews with the sector's managers, which was conducted over eight hours during the food service's administrative hours. The evaluators received the printed instrument and were instructed to assess the modules at the same time, as well as not to exchange information among themselves, to avoid influencing the answers. We used interobserver frequency statistical tests in the pre-test analysis, discussing the results in an interactive process between researchers and nutritionists to clarify controversial points. All items with a percentage of concordance less than 80% were analyzed to eliminate or adjust ambiguous indicators or indices or that carried value judgments. We assessed whether the concepts were drafted in a way that was comprehensible to what was expected to be measured and adequate for the proposed objectives. The documents and their modification proposals were analyzed and accepted, generating the second version of IQIS.
The test mainly aimed to evaluate the psychometric characteristics of the instrument. To this end, we invited seven nutritionists with a mean age of 38 years and an average of 11 years' food production experience. All of them had specialization in health surveillance or related areas, and applied the second version of IQIS in the same food and nutrition service of the pre-test, following the same previous conditions. The data were collected with the printed forms imported to the SPSS software version 24.0 to perform the analysis 35 .

Statistical analysis
Data statistical treatment was performed through descriptive and exploratory analyses to investigate the accuracy of entries, the distribution of missing cases and the distribution of frequencies. The non-parametric analysis of variance was used to compare the percentages of the adequacy of hygienic-sanitary conditions of the food and nutrition service, employing the Kruskal-Wallis test and considering the significance level of 5% in all analyses 36 . The Intraclass Correlation Coefficient was applied to verify the reliability and usability of the instrument in other contexts using the same Test data, considering that in the case of ICC ≥ 0.75, reliability should be considered excellent; an ICC of 0.40-0.75 shows satisfactory reliability; and an ICC < 0.40 37 reflects poor reliability. Finally, the Cronbach Alpha coefficient was calculated for each realm to verify the internal reliability, where α ≥ 0.70 values were considered acceptable and α ≥ 0.80 38,39 were highly reliable values.

results
The IQIS was defined in its first version based on the requirements of RDC N o 216/2004 13 and the production process flow. It contained 12 realms and 41 modules, spread into 76 risk control indicators and 1,344 indices associated with quantitative variables. Also, indicators and indices were associated with each realm and module to assess the respective health risk, resulting in 224 different verification items. Each realm-module-indicator-index combination features an IQIS item. Box 1 shows an example of an IQIS Verification Item.
The panel of experts initially ranked 42% of the 76 indicators as critical. After the Pre-test and evaluation of the concordance rate, 57 indicators were redefined and evaluated in light of the risk factors table for the event of outbreak 18 . The Kap-pa Coefficient was used to validate the content. The IQIS items were evaluated for applicability, clarity and relevance, and results showed that the instrument has a high interobserver concordance, as per Table 1.
The applicability attribute obtained 78% of the option I fully agree in the response of the judges. Clarity was evaluated as high, obtaining 85% of the answers. The relevance attribute achieved the option relevant with 91% of the answers. The IQIS was considered validated with an excellent agreement for the attributes of clarity and relevance, with k = 0.82 and k = 0.92, respectively, and good agreement for the applicability attribute, with k = 0.78. Suggestions to change the index texts (30%) were analyzed and accepted and gave rise to the second version of IQIS, adjusted to the opinion of the Committee. The results of the application were analyzed as to the rate of agreement between them. Of the 224 responses of each evaluator, 115 had an agreement of less than 80%, which were reevaluated by the researchers and adjusted, giving rise to the second version. Following content validation, the IQIS was restructured with 12 realms, 41 modules, 57 indicators (30 critical and 27 non-critical), 252 items and 1,512 indices.
The Kruskal-Wallis test was applied to the data resulting from the evaluation of the seven nutritionists in the Test and showed no statistically significant difference (p = 0,423) between the evaluations obtained for both the instrument as a whole and its realms. The items were evaluated according to the risk for the maintenance of hygienic-sanitary quality, classified in the 12 realms, as shown in Table 2.
When calculating the proportion of adequate hygienic-sanitary conditions of the food service and of each realm, we observed that the IQIS instrument allowed us to identify the items that required correction in order to comply with the legal requirements, since there was agreement on the answers indicated. There was no statistically significant difference between the evaluations of nutritionists by the Kruskal-Wallis test, showing that the instrument is reliable and reproducible. Using the same database, the statistical analyses of the Cronbach Alpha and the Intraclass Cor-Box 1. Example of an IQIS Verification Item.

Food handlers
Cooking Hand hygiene 0 Handling workers do not perform hand washing and antisepsis (hygienization) or do not have exclusive lavatory, strategically placed against the food preparation flow. 1 Handling workers do not use antiseptic odorless liquid soap or odorless liquid soap and antiseptic or non-recycled paper towels or other hygienic and safe hand drying system or dispose of in a paper collector without rational odoon or does not perform frequent, adequate hygiene. 2 Proper hand hygiene, but no poster is available for guidance on hand washing and antisepsis. 3 Performing hand hygiene in an exclusive washbasin, strategically placed against the food preparation flow, with adequate frequency (when arriving at work, before and after handling food, after service interruption, after touching contaminated materials, after using restrooms), and when necessary, with antiseptic odorless liquid soap or odorless liquid soap and antiseptic product, using non-recycled paper towels or other hygienic and safe hand drying system, and disposing of in a non-hand-activA A posteror. Poster is affixed and available for guidance on hand washing and antisepsis. 4 Same previous condition without hand-activations (faucet and paper towel holder). 5 Same previous condition, with frequent, systematized hand washing every 60 minutes.
relation Coefficient by realm were performed to verify IQIS' reliability, which evidenced acceptable reliability. The Kruskal-Wallis test showed that there was no statistically significant difference between the evaluations (p = 0.423), the Intraclass Correlation Coefficient was satisfactory (ICC = 0.53), and the Cronbach's Alpha was considered acceptable (α = 0.71). The Test's database was used, and the mathematical formalism of the MARP was applied to each realm of the IQIS to evaluate the potential risk range of the food and nutrition service. The rates of compliance were analyzed to evaluate its effectiveness, in which the responses indicated in indices 3, 4 and 5 were "compliant". Table 3 shows the classification of each realm vis-à-vis the potential risk variation range, as acceptable, tolerable or unacceptable. The results of the application of the IQIS showed that the food and nutrition service evaluated had a mean compliance rate of 35% in the evaluation of the risk control indicators and the potential risk classified in the range of risk variation was deemed unacceptable.

Discussion
In the current Brazilian social and economic context, health control measures require risk management of hygienic-sanitary conditions of food and nutrition services as an essential factor to reduce the incidence of FBDs. Thus, the preparation and validation of a quantitative instrument for inspection of large food and nutrition services, based on the MARP, support prevention and control measures. The IQIS was developed and tested for its validity to corroborate this, with consistent results that allow the immediate application. Its elaboration considered the RDC Nº 216/2004 13 in full, which contributes to standardized terminology, besides ensuring that all national legal requirements be considered. Similar studies, such as the Food Service Categorization Assessment List 18 and the Best Practices in School Food (BPAE) 16 , used methods to prioritize items in RDC Nº 216/2004 13 , which were more relevant to health risk control, as well as developed their respective risk classification methods at the end of the application of the lists. Despite the different methodologies for classification of risk    control indicators as critical and non-critical, a similarity between the IQIS critical indicators and the Food Service Categorization Checklist is observed 18 . The MARP applied to food and nutrition services resembles HACCP in its purpose of ensuring food security, but evidences different methodological aspects, since it is used to analyze potential hazards of operations, based on the concept of probabilistic risk. On the other hand, the MARP works with the potential risk, regarding a possible health problem, without necessarily describing the problem and its probability. It is a concept that expresses the value judgment about the potential exposure to a possible risk 19 , and this is an advance in food safety assurance.
Although it is usual to use the health inspection instrument to collect nonconformities in food and nutrition services, it is necessary to use methods to classify the risk control indicators, defining their respective criticalities, for the adequate health risk management. Table 3 data analysis showed that the proportion of nonconformities in each realm is not always equivalent to their level of criticality. We observed that, concerning realms VII. Raw materials, Ingredients, and packaging; VIII. Food preparation; and X. Exposure to the consumption of the prepared food, while more than 60% of their items were evaluated with appropriate hygienic-sanitary conditions, they were classified with a potentially unacceptable risk and jeopardized the whole system. The MARP facilitates the comparison of the potential risks evaluated, guiding crucial stages of risk management, as well as implementing corrective measures in cases of failure. A similar study evaluates the risk classification of food and nutrition services as a reliable strategy for risk communication and food security promotion, significantly contributing to reduced foodborne diseases 40 .
The validation of quantitative instruments, with a view of risk management for food safety control in collective food and nutrition, is useful and gains substantial significance by supporting various professionals in the field of research and practice, enabling an accurate view of the most critical issues, without losing sight of the other items required by law. IQIS has been shown to be a viable application instrument, with a reduced level of interference of the evaluator due to the objective response options, appropriate for a detailed and thorough inspection, which takes time to complete in large food and nutrition services. It should be noted that this time difference to implement the IQIS and dichotomous instruments is not significant, which reinforces its applicability by private nutritionists, as well as by regulatory bodies linked to health surveillance.
The lack of validation studies involving quantitative evaluation tools for food and nutrition services -which include all the items required in RDC N o 216/2004 13 -and of a gold standard appear as hindrances to the design of this study, which led to the combination of several methodological strategies to ensure the validation of the IQIS. Among the main limitations of this research is that sample size was restricted by the need of the group of evaluators to apply the IQIS concurrently, since the reality of food production is dynamic and changes its hygienic-sanitary conditions in the respective sectors as per the production flow throughout the day.

conclusions
The results of the IQIS evidence content validity and it can be used with good reliability and reproducibility by nutritionists to evaluate the hygienic-sanitary conditions of the large food and nutrition services and to manage the priority risks, supporting the adoption of best practices of food handling and favoring the prevention of diseases. The contributions evidenced the relevance of developing the IQIS, taking into account the need for nutritionists to have a specific technological innovation for the segment of collective feeding that meets the requirements of RDC Nº 216/2004 13 and allows health risk management, covering all its complexity. Similar work will generate knowledge and tools to ensure health risk management in food and nutrition services, thus contributing to the provision of safer food with a lower probability of foodborne diseases. An essential benefit of the work is the appropriate management of resources in the hierarchy of improvements, fundamental in the current socioeconomic context of crisis and scant resources. We recommend that the technical application of IQIS be extended to other food and nutrition services, as well as its possible be considered for use by regulatory agencies such as Health Surveillance. As a perspective for future studies, we suggest the preparation of instructional material for a better understanding of the instrument by professionals employing the IQIS and a more objective and appropriate version for application in realities with reduced availability of resources or health emergency situations.