Services on Demand
On-line version ISSN 1678-4405
Braz. J. Microbiol. vol.31 n.1 São Paulo Jan./Mar. 2000
Martine W. Reij*; Michiel van Schothorst
Wageningen University, Agrotechnology and Food Sciences, Laboratory of Food Microbiology,
Although numerous papers on Microbiological Risk Assessment (MRA) of food products have been published, a number of issues related to it remain unresolved. This paper explains the role of Microbiological Risk Assessment in the context of Risk Analysis as outlined by Codex Alimentarius. It reviews some representative work in the area, with particular emphasis on the objectives, outputs and conclusions of the studies, and on how researchers propose using the resulting Risk Estimate for decision making. Several problems and sources of confusion concerning MRA are identified, such as terminology, the application of Risk Estimates to establish Food Safety Objectives and microbiological criteria for foods, lack of data, and the difference between Risk Assessment and the Hazard Analysis and Critical Control Point system (HACCP). In the context of Codex Alimentarius, MRA was developed as a tool for decision making and priority setting by government risk managers. However, elements of Risk Assessment can be used for other purposes. A transparent description of a Risk Assessment study is useful for Risk Communication. Industrial food safety managers can compare the effect of various hypothetical production scenarios using estimates of the level and the probability of a pathogen in the product at the time that it is consumed. This limited form of Risk Assessment could better be called Safety Assessment, and can be used as a tool for food product and process development.
Key words: Risk assessment, food microbiology, microbial ecology, Codex Alimentarius
Assessing risks is part of our daily life. When we cross the street, we decide whether or not it is safe to walk. This decision is based on what we learned as children about hazards on the street. Cars, buses, trucks, and cyclists are all hazards to the pedestrian, but some can hurt us more than others. Every time we start to cross a street, we estimate the likelihood that a vehicle will hit us and the extent of damage it could do. If the estimated risk is small enough, we start to walk, because we have decided that the risk, although not zero, is acceptable and that the benefit (i.e. reaching the other side of the street) outweighs the risk.
Microbiological Risk Assessment of foods is essentially the same process. Like the pedestrian, the risk assessor uses his expertise (based on factual information or experience) to estimate the risk of harm consequent to an action (eating a certain food). This estimate can be used to make a thoughtful and balanced decision. The decision may be to accept the risk and inform the consumer, or to implement actions to reduce the risk to a tolerable level (zero risk does not exist). Such decisions are mainly taken by government risk managers. Although certain aspects of risk assessment can also be used by food safety managers in industry, they will normally assess product safety rather than risks.
The aim of this publication is to consider the state of the art of government Microbiological Risk Assessment as described by the Codex Alimentarius (6, 8), to consider some of the objectives and limitations and to help the reader to identify texts of interest.
Risk Assessment as part of Risk Analysis
Microbiological Risk Assessment has become an important subject of discussion, research and general interest because of the WTO/SPS (World Trade Organisation / Sanitary and Phyto-Sanitary) agreement. This agreement specifies that decisions on whether a food can be considered safe and fit for international trade have to be based on science and particularly on risk assessment. The WTO relies on the Codex Alimentarius to define the safety of food and to specify how Risk Analysis, and particularly Risk Assessment, should be performed. Thus, we will use the Codex definitions of Risk Analysis, Risk Assessment, Risk Management and Risk Communication as the basis for this discussion (see Table 1).
According to the Joint FAO/WHO Consultation on Risk Management (9), the risk manager takes the first step in Risk Analysis (Figure 1). It is the definition of the subject of risk assessment (Hazard Identification and Risk Assessment Policy, Fig 1 at 1) e.g.: "what is the risk of consuming low numbers of Listeria monocytogenes in a ready-to-eat meat product?". Then the risk assessor performs a Hazard Characterization and an Exposure Assessment to characterise that risk by analysing scientific information objectively, systematically and transparently (Fig 1 at 2-4).
Once the estimate is made (Risk Characterization), the risk managers task is to evaluate it (Risk Evaluation, Fig 1 at 5), taking into account not only scientific, but also social, ethical and economic considerations. Then the risk manager decides which management actions, if any, are required (Fig 1 at 6-7). The risk estimate should thus be presented in a form that is appropriate for this kind of decision making such as: "the risk for a sensitive consumer to get seriously ill with listeriosis by eating 100 gram of a ready-to-eat meat product is less than 1 in a million".
Protocols for Risk Assessment were originally developed to manage chemical hazards. Many tools and reviews on the subject were published before it was applied to risks associated with microorganisms. Various review articles have described the methodology of Microbiological Risk Assessment (13, 14, 15, 18, 19).
The literature on Microbiological Risk Assessment presents various frameworks (10, 13, 16, 20, 21, 26). In all of them, the risk assessor first identifies the hazard(s) of concern and then describes their impact on the population and finalises with a risk characterization.
We will take the principles of Microbiological Risk Assessment as described by Codex Alimentarius (6) and the FAO/WHO report on Risk Management (9) as the basis for this paper. According to the latter report, it is the risk manager who selects the hazard (Fig. 1 at 1), while the risk assessor describes the behaviour and other important characteristics of the selected hazard (Fig. 1 at 2). Subsequently, the risk assessor determines the level of exposure to the hazard either by analysing products or by describing the complete route from the raw materials, transport, processing, and storage to consumption (Fig. 1 at 3). This is called a Dynamic Flow Tree model (16), Process Risk Model (7), Pathogen-Product pathway or Farm-to-Fork model. It allows to estimate the various levels of the hazard in various situations / circumstances and the probability that the population is exposed to them. Finally, the risk assessor combines the exposure data with data on the dose-response relationship and severity of the effects, in a final risk estimate (Risk Characterization, Fig. 1 at 4). This estimates the probability and the severity of illness due to a particular pathogen in a particular food, in a specific group of consumers.
Presentation of risk estimate
Mathematical models are often used to estimate risk when there are several input parameters. Traditionally, single values or point estimates are assumed for input parameters such as initial count and storage temperature. The value can be based on the mean value, on a conservative estimation, or on a worst-case scenario. However, single value parameters cannot take into account the natural variability of the input parameters. This can be done by replacing single values with probability distributions (26). By using Monte Carlo simulation, the various probability distributions can be combined, resulting in a probability density function with attendant uncertainties as shown in Fig. 2. Monte Carlo simulation thus incorporates the effect of a range of possible input parameters into the estimated risk.
In other studies a relative risk is presented to allow comparison of various pathways and control options (2, 22). Alternatively, parameters can be compared using the Spearman rank correlation, such as shown in Fig. 3 (7), also called a tornado chart. A large correlation, either negative or positive, indicates that the input parameter is an important factor determining the risk.
|Figure 3. Spearman rank correlation between the estimated probability of illness and the fifteen most important predictive factors determining the probability of illness caused by E. coli O157:H7 in ground beef hamburgers as predicted by the Process Risk Model as presented by Cassin et al. (7). A positive value of the rank correlation indicates that the predictive factor is positively related with the probability of infection, while a negative factor correlates to a decrease in probability. It should be noted that the rank correlations depend on the way the predictive factors are defined. Copyright (1998), with permission from Elsevier Science.|
Attemps have also been made to convert a risk estimate into costs to society, because it allows different types of risk to be compared. McNab (20) presented a framework by which direct and indirect costs to society due to food poisoning are included.
Examples of Risk Assessment studies
We have chosen the following studies as examples of different ways of presenting the risk estimate, and of the decision-making that is based on it. The Codex procedure is meant to be used by government risk assessors and managers, the selection of risk assessments has been done with this in mind.
Todd (22) published one of the first quantitative estimates of the probability of illness, based on a statistical evaluation of the whole pathogen/product pathway. He considered various hazards associated with cracked eggs, but found Salmonella to be the only hazard of concern. After using mean values to calculate a point estimate, he estimated that the probability of illness from eating cracked eggs is 1 in 3800, although the probability is higher for certain populations. Expressed as a relative probability; cracked eggs are 3 to 93 times more likely to cause illness than uncracked eggs. The same study suggests eight options to manage the use of cracked eggs and discusses their effectiveness and feasibility without making a quantitative comparison.
Van der Logt et al. (24) used a Monte Carlo simulation to estimate the probability of infection with the tapeworm Taenia saginata due to consumption of beef. They considered the probability of infection, the severity of the symptoms and the uncertainties. The authors compared the number of infections per year due to beef consumption with and without routine beef inspection. They conclude that the severity of infection is limited and that routine beef inspection has a negligible impact on risk reduction; thus, they question the value of routine beef inspections for Taenia saginata.
Whiting and Buchanan (27) developed a Monte Carlo simulation model to calculate the probability of illness due to Salmonella enteritidis in whole liquid eggs, using four scenarios for pasteurisation and storage. The authors indicate that their model can directly relate food processing operations to the probability of illness, thus facilitating decision making. In a production process, risk assessment can be used to define quantitatively various options for critical control points. The authors do not discuss the possible use of their results by government risk managers.
Van Schothorst (25) used market surveillance data from Germany on the presence and levels of the microorganism in a wide variety of foods to determine the risk from consuming small numbers of Listeria monocytogenes. He linked these data with the frequency of listeriosis and chose a worst-case scenario; i.e. that all cases were provoked by a ready-to-eat food (smoked fish). He estimated the risk of contracting listeriosis for a sensitive consumer eating a food with <100 CFU / g Listeria monocytogenes to be <1 in 100,000. Buchanan et al. (4) used the same data but more sophisticated calculation techniques to estimate the chance to be <1 in 1,000,000 and composed a dose-response relationship using the surveillance data.
Baker et al. (2) studied the prevalence and the risks of Salmonella enteritidis in eggs and egg products in the United States to establish the risk of foodborne illness and to identify potential risk reduction strategies and data needs. They suggested a best-case scenario in which the eggs are directly after laying cooled down to a temperature of 7.2°C and kept below that temperature during transport and processing. They estimated that the risk would be reduced by 8 % (no attendant uncertainties given). The authors indicate that any alternative scenario should be complemented with a cost-effectiveness study and cost-benefit analysis before choosing the optimal strategy.
Cassin et al. (7) analysed the pathway of the virulent strain E. coli O157:H7 from the farm to consumption in hamburger patties. The authors presentation of results focuses on informing the government risk manager. They identify the most important factors affecting risk and compare three hypothetical mitigation strategies. The authors state that their model can be used to allocate resources to risk reduction strategies that are immediately feasible.
Bemrah et al. (3) assessed the probability of acquiring listeriosis from a serving of soft cheese made from raw milk. They related incidence of listeriosis cases and deaths according to different consumption patterns, and demonstrated the efficiency of various management options. The authors stress that the data are of poor quality, resulting in considerable uncertainty.
McElroy et al. (17) estimated the risk of acquiring emetic disease from consuming Chinese-style rice contaminated with Bacillus cereus. They used four temperature scenarios and applied several conservative assumptions. The authors do not discuss the use of their results for other than scientific purposes.
Use of Risk Assessment
The list of examples shows that risk assessors have struggled with the techniques and have published a limited number of risk estimates. Only some of these risk assessment studies are aimed at government risk managers. According to the Codex definition, Risk Assessment is a scientific evaluation of the probability of occurrence and severity of adverse health effects, including attendant uncertainties. Once an estimate is made, government risk managers have to decide whether control measures are necessary, which options are available and which control measures can or should be taken to prevent or reduce foodborne illness. For example, they may decide that various control measures can be taken to reduce the risk of salmonellosis caused by eggs. Examples are: eradication of Salmonella at the farm, pasteurisation of all eggs before marketing, cooling of shell-eggs directly after laying, informing cooks and consumers about the risks of serving or eating soft or undercooked eggs.
The question is: How can a scientifically established Risk Estimate help decision making? First, a Risk Estimate may help risk managers to set priorities. It is most important to regulate foods that are likely to make someone seriously ill. An evaluation of the different scenarios presented in the risk assessment may also help in choosing the most effective intervention. In this respect it is useful if a risk assessment study presents not only the current situation, but also scenarios that describe the effect of feasible control measures. This could help identify the most critical control points.
Problems with Risk Assessment
Terminology. A major problem with Risk Assessment is confusion about terminology. Many publications titled "Risk Analysis" deal with something that does not correspond to the Codex definition. For example, the paper "Risk Analysis of a thermal sterilisation process" (1) is an estimation of the probability that spores of Cl. botulinum will survive processing. ILSI (13) mentioned "analysis" as a part of the Risk Assessment process and a study describing methods for predictive modelling, to be used in exposure assessment, was called Risk Assessment (23). McNab (19) identified three models for performing Risk Assessment. In our opinion, the term Risk Analysis, with its three components Risk Assessment, Risk Management and Risk Communication, should be reserved for the total process (Fig. 1).
Furthermore, the term "risk" is often confused with "probability". A risk is the product of the probability of illness due to a particular microorganism in a certain food, and the severity of illness, in a specific group of consumers. Numerous articles use the term "risk" when they mean "hazard" (1, 22, 27).
Finally, the term Risk Assessment has been applied to activities that determine the risk of illness but that should not fall under the term Risk Assessment as defined by Codex Alimentarius because these activities are not meant to provide government risk managers with scientific information. Many articles in literature are not clear about how they use the term "Risk Assessment" or what they intend to do with the result (17). This is most probably due to the fact that MRA is still under development.
Apart from the terminology, the presentation of risk estimate studies is very important. So far risk estimates have been presented in different ways, which makes comparisons difficult or impossible. If risk estimates were presented in a uniform way, they would be a more effective risk communication tool.
What is an "Acceptable Risk"? Even if a risk estimate indicates that there is a chance of one in a million that an average person will get ill from a food containing a certain microorganism, this does not mean that such a probability is "acceptable". Acceptability is not only based on scientific data, but also on social, ethical and economic considerations (15), and thus part of Risk Management and not of Risk Assessment. At present, acceptable levels for microbes (we prefer the expression "tolerable levels") are still based on a "historic" performance of a product or based on the feasibility of adhering to a certain level. Although a risk estimate might indicate that the level of Salmonella contamination in chicken would be below 5% if certain control options were taken, such options are often not feasible and thus a higher level of contamination and thus risk would have to be accepted.
The level of pathogens in foods that is considered acceptable for public health protection is currently termed as a Food Safety Objective (FSO) At this moment, the way to use risk estimates to set Food Safety Objectives has not been defined. However, in certain cases the Food Safety Objective can easily be related to a risk estimate. For instance, if there is an estimated chance of <1 in 108 that a food containing <100 Listeria monocytogenes per gram will provoke listeriosis (4), and if this chance is considered acceptable (tolerable), than the FSO for this organism could be set at <100/g (at the moment of consumption).
Microbiological criteria are different from Food Safety Objectives. Their establishment is described in the Codex document Principles for the Establishment and Application of Microbiological Criteria for Foods (5). These criteria include the microorganism of concern, the analytical methods for detection and/or quantification, a sampling plan and the size of the analytical unit and finally, the appropriate microbiological limits for the food at a specified point of a food chain. They are meant to be used for inspecting consignments of products, and as such determine an acceptable level of a hazard, when there is not an adequate record of production / manufacturers performance. Food Safety Objectives are safety guidelines for producers and are not designed for testing purposes. Food Safety Objectives could be translated into microbiological criteria, but a method for doing this has still to be developed and agreed.
Risk Assessment is of limited use for setting microbiological criteria for many processed foods. However, in the Listeria example above, the level of < 100 Listeria monocytogenes per gram in the food when it is eaten could be used as a microbiological criterion as well as a FSO. The seriousness of a risk and its likelihood of occurrence should determine the number of samples taken. The ICMSF has worked out nine "cases" or sampling schemes for pathogens based on three levels of risk seriousness, and three levels of likelihood of occurrence (11).
Availability of data. In microbiological Risk Assessment, many risk factors are not well understood. All the risk assessment studies mentioned emphasise the limitations of the available data. Consequently, many assumptions are made when estimating risk. For example, there are few data on the "normal prevalence" of pathogens in raw materials in many regions of the world. Although the effect of most processing treatments on bacterial load is known, there is little information on the recontamination rate of a processed food with pathogens, and the level of such a contamination. Of the Risk Assessment publications mentioned, only one framework incorporates the probability of recontamination (20). Finally, real exposure data, i.e. the number of pathogens ingested, and the frequency of illness, by the various categories of consumers are rarely available, nor are dose-response relationships for vulnerable groups (3, 4, 14, 21).
Lack of data is a limitation of Microbiological Risk Assessment of food, but it is inconceivable that all the data claimed by some authors to be necessary will ever be available. Hence Microbiological Risk Assessment must be able to provide answers with data available, supplemented by expert opinion.
Risk Assessment and HACCP. The relation between Risk Assessment and the Hazard Analysis and Critical Control Point (HACCP) system has been the source of much confusion. HACCP is a food safety management tool applied in a production or catering environment, used to continuously control hazards and thus, to reduce risks. Control measures are put into place at Critical Control Points in the production process to prevent or eliminate a food safety hazard or reduce it to an acceptable level. Risk Assessment, on the other hand, is a scientific process of compiling and analysing information objectively, systematically and transparently to estimate risk. A HACCP study is done for a particular product on a particular processing line, sold and used under a specific set of conditions. Government Risk Assessments cover all similar products on the market.
It is sometimes thought that Risk Assessment is part of a HACCP study, maybe because the first activity in HACCP is called "hazard analysis". In HACCP this includes identifying potential hazards and determining which are significant, i.e. those that need to be controlled. In Risk Assessment the first activity is called "hazard identification". This is one of the reasons for confusion.
To determine the significance of potential hazards, the HACCP study team assesses the probability of contamination, survival and growth of the pathogen in the food during and after processing, as well as in the production environment. This part of the HACCP study is similar to the Product Pathogen pathway analysis that is used in Risk Assessment, however, the aim and output are different. In HACCP, it is done to introduce control measures at Critical Control Points to prevent, eliminate or reduce hazards. In Risk Assessment, it is done to assess exposure. In government Risk Assessments the input is drawn from many production lines to reduce the level of uncertainty attached to a risk estimate. In HACCP, the input is product and production line specific. After implementation of the HACCP plan, a "residual" level of a hazard can remain and this is the input for exposure assessments in Risk Assessment.
A full Risk Assessment as defined by Codex can be useful when acceptable levels (or Food Safety Objectives) have not been established, and when dealing with a production line that does not reduce pathogens (i.e. when HACCP is not fully effective). Examples are production lines for sprouts, seafood that is consumed raw, or frozen chicken. In such cases products on the market may contain pathogens at levels that warrant a risk assessment. The risk assessor may estimate the effectiveness of changes in control measures or the introduction of new control measures in terms of reduction of estimated illnesses. The results of such a Risk Assessment might help the HACCP-team to determine CCPs or the Critical Limits at CCPs. This is one reason that publications on Risk Assessment mention its usefulness for HACCP (12). This limited form of Risk Assessment could better be called Safety Assessment, and can be used as a tool for product and process development.
This approach (first Risk Assessment, then HACCP) is not normally taken in an industrial setting. CCPs and Critical Limits are determined on the basis of previous experience. Such experience includes both incidents that initiated corrective actions and the safety record for the particular product and processing line. For new products, the experiences with similar products, or challenge and storage tests, are used.
It is the authors opinion, however, that one of the main problems in the processed food industry is preventing recontamination. Cross-contamination and recontamination have been responsible for a large portion of foodborne illnesses (28). On the other hand, recontamination rates and levels are difficult to estimate. This increases the uncertainties attendant to a risk estimate to such an extent that the outcome may become meaningless for decision making. So far, only one Risk Assessment framework has addressed the problems associated with recontamination (20).
Risk Assessment is a tool to determine priorities in Risk Management, but does not determine whether a risk is tolerable. Judgements of acceptability depend on social, ethical and economic considerations and should be determined in discussions with all stakeholders. Thus, it is crucial to present the Risk Estimate and attendant uncertainties in a uniform and transparent way.
Other stakeholders will profit from transparent and unbiased information on food safety. If consumers understand the information, they will be able to compare microbiological risks with risks that are easier to imagine and thus, to view them in perspective. Hathaway (10) stresses that Codex and national governments should actively combat peoples unrealistic expectations concerning the possibility of zero risk and the effectiveness of regulatory action.
In international trade, transparent communication on risks is essential. In fact, unacceptably high risk to consumers is the only non-tariff barrier allowed by the WTO/SPS. The purpose of Risk Assessment in international trade is to establish objective criteria for food import and export. In cases of dispute, international bodies such as the WTO need objective and scientific information to judge the correctness of the criteria used.
The authors thank Dr. Susan Jongeneel and Mrs. Gilma Chitarra for their help in preparation of this manuscript. Dr. T.A. Roberts, Dr. A. Lammerding and Dr. Y. Motarjemi are thanked for their critical remarks.
Avaliação de risco microbiológico em alimentos: avaliação crítica
Embora inúmeros artigos já tenham sido publicados sobre Avaliação de Risco Microbiológico em alimentos, vários aspectos relacionados ao assunto continuam incompletos. O presente artigo explica o papel da Avaliação de Risco Microbiológico no contexto da Análise de Risco, segundo o Codex Alimentarius. Este trabalho revê alguns artigos representativos na área, com particular ênfase nos objetivos, resultados e conclusões dos estudos, e em como os pesquisadores usam os resultados do Risco Estimado nas tomadas de decisão. Vários problemas e fontes de confusão relacionados à Avaliação de Risco Microbiológico são identificados, tais como a terminologia e a aplicação do Risco Estimado no estabelecimento dos Objetivos da Segurança dos Alimentos e dos Critérios Microbiológicos para Alimentos. A falta de dados para a avaliação dos riscos e a diferença entre Avaliação de Risco Microbiológico e sistema HACCP são também discutidas. No contexto do Codex Alimentarius, a Avaliação de Risco Microbiológico foi desenvolvida como uma ferramenta para tomada de decisões e estabelecimento de prioridades pelos gerenciadores governamentais de risco. No entanto, elementos da Avaliação de Risco Microbiológico podem ser usados com outros objetivos. Uma descrição clara de um estudo de Avaliação de Risco é útil para a Comunicação de Risco. Gerentes industriais da área de segurança dos alimentos podem comparar o efeito de vários cenários hipotéticos de produção, utilizando estimativas do nível e da probabilidade de um patógeno estar presente em um alimento no momento de seu consumo. A denominação mais correta para essa forma limitada de Avaliação de Risco seria Avaliação de Segurança, que pode ser utilizada como uma ferramenta no desenvolvimento de um processo ou de um alimento.
Palavras-chave: avaliação de risco, microbiologia de alimentos, ecologia microbiana, Codex Alimentarius.
1. Akterian, S.G.; Fernandez, P.S.; Hendrickx, M.E.; Tobback, P.P.; Periago, P.M.; Martinez, A.; Risk analysis of the thermal sterilization process. Analysis of factors affecting the thermal resistance of microorganisms. Int. J. Food Microbiol. 47: 51-57, 1999.
2. Baker, A.R.; Ebel, E.D.; Hogue, A.T; McDowell, R.M.; Morales, R.A.; Schlosser, W.D.; Whiting, R.; Salmonella enteritidis risk assessment; Shell eggs and egg products. USDA FSIS Washington, 1998, http://www.fsis.usda.gov/OPHS/risk/index.htm.
3. Bemrah, N.; Sanaa, M.; Cassin, M.H.; Griffiths, M.W.; Cerf, O.; Quantitative risk assessment of human Listeriosis from consumption of soft cheese made from raw milk. Prev. Vet. Med. 37: 129-145, 1998.
4. Buchanan, R.L.; Damert, W.G.; Whiting, R.C.; van Schothorst, M; The use of epidemiologic and food survey data to estimate a purposefully conservative relationship for Listeria monocytogenes levels and incidence of listeriosis. J. Food Prot. 60: 918-922, 1997.
5. CAC (Codex Alimentarius Commission). Joint FAO/WHO Food Standards Programme. Codex Committee on Food Hygiene. Food Hygiene, supplement to Volume 1B-1997. Principles for the establishment and application of micriobiological criteria for foods. CAC/GL 21-1997.
6. CAC (Codex Alimentarius Commission). Joint FAO/WHO Food Standards Programme. Codex Committee on Food Hygiene. Food Hygiene. Draft Principles and guidelines for the conduct of Microbiological Risk Assessment. ALINORM 99/13A, Appendix II, -1998.
7. Cassin, M.H.; Lammerding,A.M.; Todd, E.D.; Ross, W.; McColl, R.S.; Quantitative risk assessment for Escherichia coli O157:H7 in ground beef hamburgers. Int. J. Food. Microbiol. 41: 21-44,1998.
8. FAO/WHO. Application of Risk Analysis to Food Standards Issues, Report of the Joint FAO/WHO Expert Consultation. WHO, Geneva, Switzerland, 13-17 March 1995.
9. FAO/WHO. Risk Management and Food Safety, Report of the Joint FAO/WHO Consultation. FAO Food and Nutrition Paper 65. WHO, Rome, Italy, 27-31 January 1997.
10. Hathaway, S.C. Development of food safety risk assessment guidelines for foods of animal origin in international trade. J. Food Prot. 60: 1432-1438, 1997.
11. ICMSF. Microorganisms in foods 2: Sampling for microbiological analysis - principles and specific applications. Second edition, University of Toronto Press, 1986.
12. ICMSF. Potential application of risk assessment techniques to microbiological issues related to international trade in food and food products. J. Food Prot. 61: 1075-1086, 1998.
13. ILSI - Risk Science institute pathogen risk assessment working group. A conceptual framework to assess the risks of human disease following exposure to pathogens. Risk analysis. 16: 841-848, 1996.
14. Jaykus, L.-A. The application of quantitative risk assessment to microbial food safety risks. Crit. Rev. Microbiol. 22: 279-293, 1996.
15. Lammerding, A.M. An overview of microbial food safety risk assessment. J. Food Prot. 11: 1420-1425, 1997.
16. Marks, H.M.; Coleman, M.E.; Lin, C.-T.; Roberts, T.; Topics in Microbial risk assessment: Dynamic flow tree process. Risk Analysis 18: 309-328, 1998.
17. McElroy, D.M; Jaykus, L.-A.; Foegeding, P.M.; A quantitative risk assessment for Bacillus cereus emetic disease associated with the consumption of chinese-style rice. J. Food Safety. 19: 209-229, 1999.
18. McKone, T.E. Overview of the risk analysis approach and terminology: the merging of science, judgement and values. Food Control 7: 69-76, 1996.
19. McNab, W.B. A literature review linking microbial risk assessment, predictive microbiology, and dose-response modelling. Dairy, Food and Environmental sanitation 17: 405-416, 1997.
20. McNab, W.B. A general framework illustrating an approach to quantitative microbial food safety risk assessment. J. Food Prot. 61:1216-1228, 1998.
21. Notermans, S.; Mead, G.C.; Incorporation of elements of quantitative risk analysis in the HACCP system. Int. J. Food Microbiol. 30: 157-173, 1996.
22. Todd, E.C.D. Risk assessment of use of cracked eggs in Canada. Int. J. Food Microbiol. 30: 125-136, 1996.
23. Van Gerwen, S.J.C.; Zwietering, M.H.; Growth and inactivation models to be used in quantitative risk assessments. J. Food Prot. 61: 1541-1549, 1998.
24. Van der Logt, P.B.; Hathaway, S.C.; Vose, D.J.; Risk assessment model for human infection with the cestode Taenia saginata. J. Food Prot. 9: 1110-1119, 1997.
25. Van Schothorst, M. Sampling plans for Listeria monocytogenes. Food Control. 7: 203-208, 1996.
26. Vose, D.J. The application of quantitative risk assessment to microbial food safety. J. Food Prot. 61: 640-648, 1998.
27. Whiting, R.C.; Buchanan, R.L.; Development of a quantitative risk assessment model for Salmonella enteritidis in pasteurised liquid eggs. Int. J. Food Microbiol. 36: 111-125, 1997.
28. WHO Surveillance Programme. Sixth report of WHO surveillance programme for control of foodborne infections and intoxications in Europe. FAO/WHO Collaborating Centre for Research and Training in Food Hygiene and Zoonoses, Berlin, 1995.
* Corresponding author. Mailing address: Wageningen University, Agrotechnology and Food Sciences, Laboratory of Food Microbiology, European Chair in Food Safety Microbiology, P.O. Box 8129, 6700, EV, Wageningen, The Netherlands. Fax: (+31) 317-484893. E-mail: Martine.Reij@micro.fdsci.wag-ur.nl