Monte Carlo simulation for the evaluation of measurement uncertainty of pharmaceutical certified reference materials

ARTICLE

Monte Carlo simulation for the evaluation of measurement uncertainty of pharmaceutical certified reference materials

Werickson F. C. Rocha^*; Raquel Nogueira

Instituto Nacional de Metrologia, Qualidade e Tecnologia, (Inmetro), Divisão de Metrologia Química (Dquim), 25250-020 Xerém-RJ, Brazil

ABSTRACT

The supplemental Guide to the Expression of Uncertainty Measurement (2008), which deals with the propagation of distributions, encourages the use of the Monte Carlo simulation (MCS) for estimating the uncertainty of measurands. This paper describes the application of this method to estimate the measurement uncertainty of active pharmaceutical ingredient (API) mass fractions of two certified reference materials (CRMs): metronidazole and captopril. The Monte Carlo results complied with the GUM results in terms of the critical value d of 0.005 for metronidazole and 0.05 for captopril. Therefore, the Monte Carlo method validated the GUM through the expression of the API mass fraction with at least two decimal places.

Keywords: Monte Carlo simulation, evaluation of measurement uncertainty, certified reference materials, active pharmaceutical ingredients

RESUMO

O suplemento do Guia para a Expressão da Incerteza de Medição (GUM) publicado em 2008, o qual trata da propagação de distribuições, encoraja o uso do método de Monte Carlo (MC) para a estimativa de incerteza de medição. Este artigo descreve a aplicação deste método para estimativa da incerteza de medição do teor de ingredientes ativos farmacêuticos em dois novos materiais de referência certificados (MRC): metronidazol e captopril. Os resultados obtidos pelo método de Monte Carlo e pelo método tradicional (GUM) mostraram concordância considerando um valor crítico d de 0,005 para metronidazol e 0,05 para captopril. Deste modo, o método de Monte Carlo validou os resultados obtidos pelo método tradicional (GUM) para a expressão do teor dos fármacos com no mínimo duas casas decimais.

Introduction

Metrological activities are fundamental to ensure the quality of scientific and industrial activities. Measurement results must be valid, comparable, and reproducible, and their uncertainties are the quantitative expression of their quality. In accordance with the ISO/IEC 17025:2005 standard,¹ all calibration or testing laboratories must have and apply procedures to evaluate uncertainty in measurements.

In order to establish an international consensus for the estimation of measurement uncertainties, the International Standardization Organization (ISO) has developed and published the Guide to the Expression of Uncertainty in Measurement (GUM),² which has been widely accepted and followed.^3-5 The GUM is based on sound theoretical principles. In addition, it supports a fully consistent and transferable estimation of measurement uncertainty and traceability to the International System of Units (SI). However, this approach exhibits some important limitations that are mainly derived from the use of the law of propagation of uncertainty and from the application of the central limit theorem.

To avoid these limitations, the working group one of the joint committee for guides in metrology (JCGM-WG1) promotes the use of the GUM and prepares supplemental guides. The supplemental guide² "The propagation of distributions using a Monte Carlo method" (2008) considers the propagation of distributions for the general probabilistic basis of uncertainty evaluation through the direct use of the probability density function (PDF) of the input quantities, rather than just the input quantities means and standard uncertainties. It also recognizes the Monte Carlo method as the most efficient numerical implementation for the propagation of distributions.^6-8

Validation of the law of propagation of uncertainty using the Monte Carlo simulation

In order to compare the results of uncertainties estimated by the ISO GUM method and the Monte Carlo simulation (MCS), the Bureau International des Poids et Mesures (BIPM)⁹ recommends a procedure based on the number of significant decimal places expressed in the calculation, which is summarized in Figure 1. The comparison procedure consists of determining whether the coverage intervals obtained by the law of propagation of uncertainty and by the MCS agree to a defined degree of approximation.^9-16 This degree of approximation is assessed (i) in terms of the endpoints of the coverage intervals, and (ii) whether this degree of approximation corresponds to the given endpoints by expressing the standard uncertainty in what is regarded as a meaningful number of significant decimal places.

Certified reference materials (CRMs) of active pharmaceutical ingredients (APIs)

According to the ISO Guide 30,¹⁸ a certified reference material (CRM) is a reference material characterized by a metrologically valid procedure for one or more specified properties, which is accompanied by a certificate that provides the value of the specified property, its associated uncertainty, and a statement of metrological traceability.

CRMs are important tools to enable and safeguard reliable analytical measurements and are typically used for calibration, method validation purposes, and method performance verification in the routine of laboratories accredited by ISO/IEC 17025:2005.¹ Their use also ensures the metrological traceability of measurement results to the SI.

While the use of CRMs is already well established in several fields, it was only in 2008 that the first CRM of an active pharmaceutical ingredient, dextromethorphan hydrobromide, appeared in the United States Pharmacopeial Convention catalogue, which was followed by three other CRMs in the next year. In 2010, the National Institute of Metrology, Standardization, and Industrial Quality (Inmetro, Brazil) made the captopril CRM available on the market. Currently, Inmetro is developing the metronidazole CRM.

Metronidazole is a nitroimidazole antibiotic medication particularly used for anaerobic bacteria and protozoa.^19-22 Captopril, on the other hand, is an angiotensin-converting enzyme inhibitor used for the treatment of hypertension and some types of congestive heart failure.^23-25 Their structural formulas are shown in Figure 2.

Due to the large use of these APIs, laboratories are required to ensure the reliability of their analysis through the use of CRMs in several steps of the quality control of pharmaceutical raw materials and finished products, including assay procedures to determine the API content, and related substances tests to ensure the API purity, in order to avoid lack of pharmaceutical efficacy and serious harm to patients due to API mass fractions outside the specifications, as well as unexpected side effects and health risks due to the presence of unexpected impurities and contaminants.

The aim of this paper is to apply the Monte Carlo method to perform the propagation of distributions and determine the measurement uncertainties of the API contents for the CRMs of metronidazole and captopril, in order to validate the uncertainties estimated through the GUM method.

Experimental

Instrumentation

A high performance liquid chromatography (HPLC) system (Shimadzu, Kyoto, Japan), consisting of a LC-20AT quaternary pump, a DGU-20A₃/DGU-20A₅ on-line degasser, a SIL-20A/20AC auto-sampler, a SPD-20A photodiode array detector, and a CBM-20A/20A interface, was used for the determination of related substances (organic impurities). All sample solutions were prepared gravimetrically using an analytical balance (Shimadzu), model AUW 220D, with resolution of 0.01 mg.

Reference material certification

The experimental procedures used for certification of metronidazole and captopril reference materials were previously described^26-28 and were based on the ISO Guides 34:2009 and 35:2006.^29,30

Initially, the material was characterized, i.e., the mass fractions of organic, inorganic, and volatile impurities were determined. The analyses of organic impurities were carried out by high-performance chromatography with diode array detection (HPLC-DAD) and the analytical methods were previously validated.^27,28

For metronidazole, the method was based on the related substances test of the European Pharmacopoeia V monograph,³¹ and the following experimental conditions were used: reversed-phase column C18, 25 cm × 4.6 mm i.d., 5 µm, Nucleosil (Macherey Nagel, Dueren, Germany), pre-column C18, 4.6 mm i.d., 5 µm (Varian, Inc.; Palo Alto, USA), mobile phase 1.36 g L^-1 potassium dihydrogen phosphate-methanol (70:30, v/v), flow-rate 1.0 mL min^-1, UV detection at 315 nm.

For captopril, the method was taken from the Brazilian Pharmacopeia IV monograph,³² and the experimental conditions were as follows: reversed-phase column C18, 25 cm × 4.6 mm i.d., 5 µm, supelcosil (Supelco, Bellefonte, PA, United States), pre-column C18, 4.6 mm i.d., 5 µm (Varian, Inc.; Palo Alto, USA), mobile phase methanol-phosphoric acid 0.11% (55:45, v/v), flow-rate 1.0 mL min^-1, UV detection at 220 nm.

The content of organic impurities was determined according to equations 4 and 5.

where w_org is the mass fraction of each organic impurity expressed in g per 100 g; A_imp is the chromatogram peak area of each organic impurity (concentrated solution); A_{dil analyte} is the chromatogram peak area of the API (diluted solution); DF is the dilution factor from the concentrated to the diluted solution of API; and ΣA_imp is the sum of the chromatogram peak areas of all organic impurities (concentrated solution).

where w_{org tot} is the total mass fraction of organic impurities.

The inorganic impurities were determined by residue on ignition (sulphated ash, 600 ºC until constant weight), while the volatiles were determined by loss on drying (105 ºC per 2 h for metronidazole; 60 ºC per 3 h under vacuum for captopril), according to the Brazilian Pharmacopeia IV.³²

The between-bottle homogeneity was evaluated using at least 12 candidate CRM flasks selected at random and analyzed. For the short-term stability study, at least 8 CRM flasks selected at random were exposed simultaneously to the temperature of 50 ºC, and on pre-determined days one flask was taken out of the oven and individually analyzed (classical design). For the long-term stability study, 12 flasks samples were kept at the recommended storage conditions (25 ºC) and analyzed four times throughout the year of certification studies. All HPLC analyses were performed in triplicate.

Finally, the API mass fractions (w_API) were calculated using the mass balance approach (equation 6)

where w_API is the API mass fraction of each CRM expressed in g per 100 g, and Σw_org,i, Σw_inorg,i, Σw_vol,i are the sums of mass fractions of organic, inorganic, and volatiles, respectively, also expressed in g per 100 g.

The API mass fraction determined according to the mass balance approach was cross-checked by differential scanning calorimetry (DSC).

Uncertainty estimation of API mass fraction

The uncertainties of API mass fractions of the two certified reference materials were estimated using both the GUM method and the Monte Carlo simulation.

There were previous descriptions of the estimation of uncertainties according to the GUM method.^26-28 Initially, the combined standard uncertainties of the contents of organic impurities (u_org), inorganic impurities (u_inorg), and volatiles (u_vol) were determined by the traditional method.^2,5 The uncertainties due to between-bottle (in)homogeneity (u_bb) and long-term stability (u_lts) studies were estimated according to ISO Guide 35:2006.³⁰ The combined standard uncertainty of each CRM (u_CRM) was calculated according to the law of propagation of uncertainty, which consists of "the square root of the total variance obtained by combining all the uncertainty components".⁵ Considering that the uncertainty due to short-term stability (u_sts) was significantly smaller than u_lts, it was not taken into account for u_CRM calculation, as allowed by the Guide 35:2006.³⁰ Finally the expanded uncertainty of the certified property value (U_CRM) was obtained by multiplying u_CRM by the coverage factor (k = 2).

The Monte Carlo simulation was performed using the Crystal Ball standard edition program.^33,34 This software contains a fully integrated Microsoft Office Excel add-in program that allows users to carry out the simulation in spreadsheets. The steps carried out to perform the simulation will be detailed and discussed in the following section.

Results and Discussion

Identification of uncertainty sources

The identification of all relevant uncertainty sources for a complex analytical procedure is best done by drafting a cause and effect diagram, sometimes known as an Ishikawa or ‘fishbone' diagram.³⁵ The main factor influencing the property value of the two pharmaceutical CRMs are represented by the main branches of the diagram in Figure 3. Factors that are more detailed were added to each branch, without neglecting to consider each step in the analytical procedure.

Application of the GUM method

The uncertainty results of both API mass fractions estimated according to the GUM method can be seen in Table 2.

Application of the Monte Carlo method

For the Monte Carlo simulation, the variables of equation 6 (mass balance) and the results of the homogeneity and stability studies were taken into account. First, it was necessary to define the types of the probability density functions (PDF) of each of these input parameters.

For the total organic impurities content (equation 5), a normal probability distribution, featured by its mean value and standard deviation, was considered (Table 1). In order to confirm the reproducibility of results, the simulation was carried out by using the data obtained in three different days of analysis. The uncertainty source due to the dilution factor DF (Figure 3) was not directly used in the simulation, since it was already taken into account in the calculation of the total organic impurity content (equations 4 and 5).

For both inorganic impurities and volatiles, the main source of uncertainty was related to the repeatability of results; nevertheless, the smaller contribution of the analytical balance uncertainty was also included. Asymmetric triangular distribution was chosen, which was featured by its mean value (mode), -SD/√n (min), and +SD/√n (max), where SD is the standard deviation and n is the number of measurements (Table 1). Triangular distribution was considered the most appropriate in this case, considering the limited data sample.

Equation 6 does not include the results of the homogeneity and stability studies; nevertheless, they are taken into account for the uncertainty estimation of the CRM property value (u_CRM) in both the GUM and the Monte Carlo methods, as previously shown in Figure 3. Here, a symmetric triangular distribution was chosen, which was featured by the mean value (mode) (considered as zero), -u_bb or -u_lts (min), and +u_bb or +u_lts (max) (Table 1).

In order to perform the Monte Carlo simulation, the input parameters and their corresponding PDF types and features were added to Excel spreadsheets. Using Crystal Ball software, the cells containing the mean values of each input parameter were associated with the corresponding PDF types and were shown in green. During this step, the other PDF features (standard deviation for normal distribution, min and max for triangular distribution) were also included. Finally, it was necessary to define the forecast parameter, which in our case was the API mass fraction (g per 100 g). This cell, which was marked by the Crystal Ball in blue and consisted of equation 6, was slightly modified to also subtract the means of the homogeneity and stability studies (considered as zero). The Crystal Ball simulation was then carried out three times for each CRM, considering the availability of three-day results for organic impurity analysis.

Due to its stochastic nature (and in contrast to the GUM), the Monte Carlo method yields (slightly) different results when applied repeatedly to the same problem (unless the same random numbers are reused). This property is certainly an issue for metrologists. One of the ways to compensate for this variability is related to the adjustment of the number of trials (the number of evaluations of the measurement model), which should be large enough to ensure the reliability of the Monte Carlo results (API mass fraction, expanded uncertainty, and coverage interval end-points). Each simulation was carried out using 10⁵ random trials, and it was observed that simulations using 10⁶ trials were not able to significantly improve the results, even if a much longer time was demanded for the computer simulation. Therefore, 10⁵ random trials were considered sufficient.

In this study, the expanded uncertainty at a confidence level of 95.50% was considered, based on the coverage factor of 1.96, as recommended in the literature.⁵ Equation 7 was used to calculate for expanded uncertainty (U):

where UL and LL are the upper and lower limits, respectively, obtained in the Monte Carlo simulation and expressed in g per 100 g.

Table 2 shows the comparison between the results obtained by the Monte Carlo and GUM methods. Note that the values of |d_max| and |d_min| are smaller than the critical value (δ) 0.005 for metronidazole and 0.05 for captopril, but are larger than the critical values (δ) 0.0005 and 0.005 for metronidazole and captopril, respectively. This means that, for a 95.50% confidence level, the Monte Carlo simulation results are consistent with the GUM results when the API content of metronidazole is expressed with up to three decimal places, and up to two decimal places for captopril.

Table 3 shows the Monte Carlo results expressed as recommended by the BIPM,⁹ that is, the uncertainties were expressed using two significant digits (one significant digit would also be allowed), while the certified property value was expressed with the same number of decimal places as its uncertainty. It can be seen that this way of expressing was consistent with the conclusions of the previous paragraph.

Figure 4 shows the histograms from the Monte Carlo simulation of metronidazole and captopril CRMs. The resultant Gaussian shaped probability density functions obtained by the MCS are in accordance with the uncertainty budget shown in Table 1, which indicated that two main uncertainty sources were predominant: uncertainty of organic impurities content (u_org) (normal distribution) and uncertainty of volatiles content (u_vol) (triangular distribution).

Conclusions

Uncertainty estimation is a critical step in the certification of reference materials. In this paper, two internationally recognized methods, namely the ISO GUM approach (traditional method) and the Monte Carlo simulation, were used to estimate the uncertainty of mass fractions of metronidazole and captopril in two new pharmaceutical certified reference materials.

The study showed the agreement between the uncertainties calculated by the GUM and Monte Carlo methods when the metronidazole mass fraction was expressed with up to three decimal places, e.g., (99.625 ± 0.030) g per 100 g, and the captopril mass fraction was expressed with up to two decimal places, e.g., (99.56 ± 0.15) g per 100 g. This is in accordance with the BIPM recommendation to express the uncertainties with two (or one) significant digits and the certified property values with the same number of decimal places as their uncertainties.

Considering the agreement between both methods, the use of the Monte Carlo method for uncertainty estimation should be encouraged, since the MCS does not require the evaluation of partial derivatives (as the GUM method usually does), good quality and easy-to-use software are available to perform the simulation, and the risks of unreliable uncertainty estimation can be reduced through the use of the MCS, particularly in cases of complicated measurement models.

Submitted: August 15, 2011

Published online: December 20, 2011

• ¹
  ABNT NBR ISO/IEC 17025; Requisitos Gerais para Competência de Laboratórios de Ensaio e Calibração, ABNT: Rio de Janeiro, Brasil, 2005.
• ²
  ISO/IEC 98-3; Guide to the Expression of Uncertainty in Measurement, ISO: Geneva, 2008.
• 3. Ellison, S. L. R.; Barwick, V. J.; Analyst 1998, 123, 1387.
• 4. Taylor, B. N.; Kuyatt, C. E.; NIST Technical Note 1297, NIST: Gaithersburg, MD, 1994.
• ⁵
  Eurachem/Citac Guide; Quantifying Uncertainty in Analytical Measurement, 2^nd ed., 2000.
• 6. Cox, M.; Harris, P.; Siebert, B. R. L.; Meas. Tech. 2003, 46, 824.
• 7. Robert, C. P.; Casella, G.; Monte Carlo Statistical Methods, Springer: New York, 1999.
• 8. Rubenstein, R. Y.; Simulation and the Monte Carlo Method, Wiley: New York, 1981.
• ⁹
  ISO/IEC 98-3, Suppl. 1:2008; Propagation of Distributions using a Monte Carlo Method, ISO: Geneva, 2008.
• 10. Cordero, R. R.; Roth, P.; Metrologia 2005, 42, 39.
• 11. Cordero, R. R.; Molimard, J.; Opt. Commun. 2007, 275, 144.
• 12. Ripley, B. D.; Stochastic Simulation, John Wiley & Sons: New York, 1987.
• 13. Miranda, M.; Dorro, B. V.; Opt. Laser Eng. 2011, 49, 439.
• 14. Oliveira, S. P.; Rocha, A. C.; Filho, J. T.; Couto, P. R. G.; Measurement 2009, 42, 1497.
• 15. Papadopoulos, C. E.; Yeung, H.; Flow Meas. Instrum. 2001, 12, 291.
• 16. Gilks, W. R.; Richardson, S.; Spiegelhalter, D. J.; Markov Chain Monte Carlo in Practice, Chapman and Hall: London, 1996.
• 17. Oliveira, E. C.; Aguiar, P. F.; Quim. Nova 2009, 32, 1571.
• ¹⁸
  ISO Guide 30; Terms and Definitions Used in Connection with Reference Materials, ISO: Geneva, 2008.
• 19. Zeleny, R.; Harbeck, S.; Schimmel, H.; J. Chromatogr., A 2009, 1216, 249.
• 20. Simbar, M.; Azarbad, Z.; Moiab, F.; Phytomedicine 2008, 15, 1025.
• 21. Hu, M.; Maroo, S.; Cloud, J.; Gastroenterology 2008, 134, 670.
• 22. Sagan, C.; Salvador, A.; Dubreuil, D.; J. Pharm. Biomed. Anal. 2005, 38, 298.
• 23. Hernández-Ledesma, B.; Martín-Álvarez, P. J.; Pueyo, E.; J. Agric. Food. Chem. 2003, 51, 4175.
• 24. Opie, L. H.; Cardiovasc. Drug Ther. 1987, 1, 111.
• 25. Rabkin, S. W.; Redston, M.; Can. J. Physiol. Pharmacol. 1989, 67, 857.
• 26. Nogueira, R.; Rego, E. C. P.; Sousa, M. V. B.; Wollinger, W.; Silva, T. E.; Moreira, G. F.; Barin, J. S.; Laporta, L. V.; Mesko, M. F.; Bittencourt, C. F.; Rodrigues, J. M.; Cunha, V. S.; Braz. J. Pharm. Sci. 2011, 47, 339.
• 27. Nogueira, R.; Wollinger, W.; Silva T. E.; Oliveira, L. M.; Rego, E. C. P.; Moreira, G. F.; Barin, J. S.; Laporta, L. V.; Mesko, M. F.; Bittencourt, C. F.; Rodrigues, J. M.; Cunha, V. S.; Braz. J. Pharm. Sci. 2011, 47, 351.
• 28. Nogueira, R.; Rocha, W. F. C.; Silva, T. E.; Rego, E. C. P.; Moreira, G. F.; Wollinger, W.; Rodrigues, J. M.; J. Braz. Chem. Soc. 2012, 23, 435.
• ²⁹
  ISO Guide 34; General Requirements for the Competence of Reference Material Producers, ISO: Geneva, 2009.
• ³⁰
  ISO Guide 35; Reference Materials: General and Statistical Principles for Certification, ISO: Geneva, 2006.
• 31. European Pharmacopoeia, 5^th ed., EDQM: Strasbourg, 2005.
• 32. Farmacopeia Brasileira, 4a. ed., Atheneu: São Paulo, Brasil, 2002.
• 33. Gonzalez, A. G.; Herrador, M. A.; Asuero, A. G.; Accred. Qual. Assur. 2005, 10, 149.
• 34. Crystal Ball 2002, standard edition, Decisioneering Inc.; http://www.crystallball.com accessed in July 2011.
• 35. Ellison, S. L. R.; Barwick, V. J.; Accred. Qual. Assur. 1998, 3, 101.

Publication Dates

• Publication in this collection
  04 Apr 2012
• Date of issue
  Mar 2012

History

• Received
  15 Aug 2011
• Accepted
  20 Dec 2011