Expected Benefits and Challenges of Using Economic Evaluations to Make Decisions About the Content of Newborn Screening Programs in Vietnam: A Scoping Review of the Literature

Abstract

Screening newborns for genetic and other diseases is one of the most effective ways to improve health and reduce disease in a population. In developed countries, newborn screening has been a cornerstone of public health for decades. In many developing countries, however, newborn screening is still in its infancy. Many countries still lack screening programs. When a program is available, it generally lacks well-defined criteria on which decision-makers can justify the choice of diseases screened for and the methods used. One of the reasons put forward to understand this observation is the fact that little consideration is given by decision-makers to economic evaluations as a pillar of decision-making, as is the case in industrialized countries. This article provides a brief description of the challenges of using economic evaluation of newborn screening in developing countries. This will be illustrated by the example of the national newborn screening program in Vietnam.

Keywords:
Newborn screening; Economic evaluation; Health technology assessment; Low- and middle-income countries; Health policy making

Introduction

Newborn screening (NBS) is a public health intervention that has been operating in many national post-natal screening programs around the world since the 1960s [1]. It aims at detecting a range of potentially preventable life-threatening conditions before the appearance of symptoms. Typically, the conditions for which screening is offered benefit from early interventions, i.e. with treatment often preventing physical and mental disability, and even death for some of those conditions. NBS programs in high-income countries are based on dried blood spot (DBS) specimens tested using various methodologies, including tandem mass spectrometry [2]. Moreover, NBS programs usually include short-term (initial investigation and diagnosis, counseling) and may also include long-term (periodic monitoring) follow-up activities, diagnosis, treatment/management, and an evaluation of the program that must be institutionalized and sustained within public health systems.

The number of conditions screened by NBS programs varies around the world, with some countries providing no newborn screening, while others screen for more than 50 conditions [3-5]. The situation in each country might change in the near future, as the technology, which allows screening for a vast number of conditions, is becoming more readily available in various clinical settings. Yet, the decision to add or remove conditions to newborn screening programs should only be made after a careful assessment of the evidence about benefits and risks, as well as organizational, financial, social, and ethical issues.

Initiation of NBS programs in many developing countries has been slow compared to high-income countries, mainly because these programs compete with other health priorities, such as infectious disease control, immunization, and malnutrition. While all countries face challenges in implementing NBS, developing countries face additional challenges related to poor economies, unstable governments, unique cultures, geographic extremes, and different public health priorities. These factors influence the government prioritization, public acceptance, and health practitioner cooperation/involvement [6-7]. Furthermore, adding a condition to a program is often arbitrary.

Economic evaluations (EEs) are a key element to support stakeholders evaluate the options in a non-arbitrarily decision-making process leading to the implementation of a new program or the expansion of an already existing NBS program, by providing data on the expected outcomes of different screening strategies in terms of both their costs and health consequences. EEs contribute to evidence-based decision-making by helping the public health community identify, measure, and compare options regarding their impact, scalability, and sustainability to optimize population health.

To date, no studies have discussed how EE could become a relevant tool to help low and middle-income countries (LMICs) to define the content of their national NBS programs. The question is relevant when we consider that NBS programs in LMICs are obviously more constrained by funding issues than in high-income countries. There are also, in LMICs, less technical and human resources able to provide data on the expected cost and effectiveness outcomes from a screening program.

This article is a reflection on the relevance of using EEs in developing countries in order to define their NBS program. Conclusions will be illustrated with the case of Vietnam.

EE as a Tool for a Rational Decision-Making Process

Health economic evaluations aim at providing information on the relative efficiency of options for intervention. EE emerged as a method susceptible to help healthcare decision-makers make defendable choices under conditions of uncertainty, conflicting objectives, and resource constraints [8]. To support this objective, the field of EE has become increasingly institutionalized with the development and adoption of national methodological guidelines and the creation of health technology assessment (HTA) agencies around the world, whose process lies mainly on a standardized conception of EE [9-10]. Indeed, the trend toward evidence-based decision-making reinforced the need to base resource allocation decisions on rational criteria making consensus among economic evaluators. Over the years, the field of EE has evolved with greater recognition and more resources, new research challenges, and more sophisticated techniques and methodologies with the introduction of simulation approaches, and new components such as cost-effectiveness acceptability curve [11]. Economic evidence is now an important factor taken into consideration in the formulation of health technology policies, practices, and reimbursement decisions [9].

EE approaches are categorized into three main types: cost-benefit, cost-effectiveness, and cost-utility analyses (CUA). In Cost benefit-analyses (CBA), costs and health benefits are evaluated in a common monetary unit. CBAs can be used to compare programs with different health outcomes, as these outcomes are measured in a common unit [11-12]. In Cost-effectiveness analyses (CEA),costs are opposed to health outcomes measured in natural health outcome units, such as glycemia, the number of life years saved, the number of cases averted, or Disability-Adjusted Life Years (DALY). DALYs are an indicator that expresses the health gap in a population compared to a theoretical maximum, which is a life expectancy of 82.5 years without disability [13]. DALYs are a main health outcome indicator used in EE performed in LMICs. In CUA, costs are opposed to quality-adjusted life-year (QALY). QALY is a mathematical measurement of health outcomes consisting of the quantity of life (in years) multiplied by a desirability score for this life, ranging from 0 (death hypothesized to be the least desirable state) to 1 (perfect health hypothesized to be the most desirable state) [11-12].

Furthermore, a budget impact analysis, which is an economic assessment that estimates the financial consequences of adopting a new intervention, is usually performed in addition to cost-effectiveness analysis. A budget impact analysis evaluates whether the high-value intervention is affordable for a payer, generally the public health insurance scheme [14].

The results of cost-effectiveness or cost-utility analysis are described in terms of a ratio of incremental costs per unit of incremental health benefit, the incremental cost-effectiveness ratio (ICER) between two options. If an intervention is less costly and more effective than the other option, it is dominant and considered cost-effective. This approach is complemented with a cost-effectiveness acceptability curve analysis, which takes into account the payer's willingness-to-invest money in an alternative that is more effective but also most costly [15].

The place of EE in the decision-making process to judge the value of a new intervention in HICs

EEs have been developed to help decision-makers achieve efficiency in health care. EE of health interventions provides decision-makers with a useful tool that permits the comparison of competing technologies in terms of the benefits they provide and the resources required to reach these benefits [11].

Indeed, a health care system relies on technologies to help answer health needs. However, the development and adoption of these technologies are costly. In rich countries, EEs are now widely used to prioritize interventions that lead to the most effective and efficient use of available resources in the health care system. Nearly every new health technology goes now through predefined phases of assessment to prove its added value. This assessment is usually undertaken by specialized HTA agencies. A country's HTA agency is mandated to provide guidance regarding which interventions and technologies should be offered and for which purposes taking into consideration the country's resource constraints [16]. HTA's processes are multidisciplinary as they deal with medical, social, ethical, and economic implications of the development, diffusion, and use of health technology.

One notes that the way HTA agencies proceed in HICs allows aiming at two major objectives. The first objective is to support interventions that have added value to the diagnostic or therapeutic arsenal offered to the population in terms of effectiveness and efficiency. The second objective is to support innovation. The fact that in Canada for example, a new intervention whose expected additional cost for the health care system is < 50 000$/QALY is acceptable, i.e., considered cost-effective, expresses the fact that new interventions that lead to an increased cost may be acceptable. In other words, the relevant health policy goal is not simply to head off rising costs, but also to promote healthcare innovations, provided they contribute to an increase in the health of the population [17].

Economic Evaluations and Newborn Screening Programs

Providing cost-effective data on a neonatal screening test or modality can present challenges. First, there is usually uncertainty on the true prevalence and the natural history of a disorder, particularly when the disease is rare, which is the case of most of the diseases screened for. Rarity hinders the constitution of samples big enough to study a disease, particularly when its outcomes have a high penetrance variance or when migration impacts the prevalence [18]. Moreover, prospective natural history studies typically begin after a diagnosis is made. Understanding pre-diagnostic natural history tends to rely on parent recall, review of medical records, or observations of disease progression in siblings. These methods might be flawed especially if symptoms and signs are light, non-specific, and draw little attention. Milder cases or those with atypical late-onset manifestations tend therefore to be less well characterized [19].

Another challenge is related to the choice of the outcome, as the various candidate conditions to be included in NBS programs differ in the physiological functions affected. Health benefits of the NBS expressed as natural clinical outcome units, such as glycemia or blood pressure, may be easy to quantify, but they may limit the comparability between conditions screened, as different conditions might affect different functions. For this reason, the use of QALY has become the standard outcome in economic evaluation studies, as this single indicator encompasses two outcomes considered particularly relevant: life expectancy and the (usually social) desirability (or utility) for this life (Whitehead & Ali, 2010). The target population, the infants, cannot or are a non-optimal source of information, for building and validating a QALY questionnaire [20-22]. Nevertheless, with the recent development of two validated instruments, there has been some breakthrough in this field. The Quality of life Instrument (IQI) is a valid instrument to measure utilities in infants up to 1 year of age [23], and the health-related quality of life utility measure for pre-school children (HuPS) is a valid instrument for children 2 to 5 years of age [24-25]. One notes that measuring the Health-Related Quality of Life outcome in less than 5 years old children brings its unique difficulties, because young children are unable to complete questionnaires by themselves, these must be completed by a proxy.

When cost-utility studies cannot be easily undertaken, such as in the case of newborn children, Cost-Effectiveness Acceptability Curve (CEAC) on a common outcome, such as the life expectancy without disability, can be used instead. CEAC enables a payer to determine, over a range of willingness to pay (WTP) thresholds, the probability that a new medical intervention is cost-effective compared to its appropriate comparator (e.g. usual care). This, like cost-utility approaches, allows labeling interventions that might be more effective than the current care, but also more costly, to be considered as cost-effective. In other words, CEAC, as CUA, allows the health care system to support innovation [26]. However, CEAC is limited by uncertain data, which is more frequent when dealing with rare diseases. The use of CEAC for the evaluation of NBS programs is therefore limited.

The place of EE in the decision-making process to evaluate the value of a new intervention in LMICs

Unlike health systems in many HICs with formal HTA frameworks that incorporate EEs, the health systems in LMICs tend to have poorly formalized the process of the evaluation of new interventions [27]. Reasons are numerous and not only related to the lack of a critical mass of expertise for conducting EE studies, or to a lack of interest in rational decision-making on the part of decision makers. For example, a survey done in Southeast Asia showed that most countries have constrains challenging their ability to carry out work similar to what is done in countries with well-established HTA agencies, such as poor access to local data on costs, to clinical information and health outcomes in the country. This access is necessary for a HTA Agency to fulfill its mandate [28].

LMICs face therefore a number of limitations in conducting EE studies, as well as in inciting decision makers to consider EE studies as part of their policy-making process. A key challenge is the difficulty in obtaining data and the limited capacity to conduct EE studies [29-30]. In addition to a lack of trained researchers with analytic skills and experience in conducting EE studies, most LMICs also lack of methodological guidelines, they have deficient institutionalized research environment, and little awareness of the benefit of evidence-based policymaking among researchers and decision-makers [31-34].

Yet, in LMICs, having an HTA agency to support a rational decision-making process on what to offer to the population, has been considered particularly relevant considering the limited availability of healthcare resources [35]. Moreover, while there is an increase in the number of EE studies undertaken in LMICs year after year, their quality remains questionable [29-30]. Systematic reviews of EEs undertaken in LMICs have highlighted a lack of consistency in methodological and reporting standards due to an absence of common guidelines [36].

As an answer to these problems, some tools have been produced, such as the Guide to Economic Analysis and Research (GEAR) [37]. This free Guide is intended to help evaluators overcome the challenges that researchers in LMICs face when they plan to conduct high-quality economic evaluations to answer questions asked by decision-makers. The tool has been devised to tackle both technical and contextual difficulties focusing on the short-term but with the long-term aspects in mind. For each of the technical and context-specific issues, short-term advice is provided in the form of currently available solutions to problems, based on the latest evidence and examples of best practices. For the longer term, GEAR identifies and encourages further research in key areas in which it is known that there is insufficient empirical research [37]. The GEAR is a reliable aid for researchers in LMICs to overcome the challenges faced in conducting and using health EEs. Although GEAR is a priceless help, it cannot be considered a satisfactory substitute for national HTA agencies, because it doesn't prescribe a consensual national approach to EEs, that would make health interventions evaluated the same way as interventions in other sectors, allowing for intersectoral comparisons. For example, discounting rates are not prescribed, making it difficult to compare investments in health with other sectors. Indeed, because of the lack of national guidelines, researchers individually make choices from a broad range of non-similar guidelines available in the world. They decide arbitrarily which guidelines or sometimes just which methodological approach to applying to their studies. The variability in the quality of data produced that results from this fact, affects the estimation of the effectiveness and cost of data considered by decision-makers.

In addition, considering the limited healthcare budget that characterizes LMICs, one can wonder if a methodology of EE that, as in HICs, has also as an objective to support the adoption of effective new and costly health innovations is relevant. When their basic health needs remain unanswered and when inequities in health remain flagrant, this methodology may not be the most relevant. Because of this, an EE methodology that focuses more on the opportunity costs of interventions to meet basic needs in a closed (non-increasing) budget, might be considered more relevant in LMICs. It is tempting to point out that if resources were more carefully targeted towards the priority health needs of the population, more benefits could probably be obtained from existing levels of expenditure.

EE for NBS Programs in LMICs

NBS programs to identify severe congenital disorders are a major public health success, saving lives and preventing disability in thousands of infants each year [38]. In LMICs, NBS and other forms of infant screening compete with other health priorities, i.e., control of infectious diseases, immunization, and malnutrition. The objective of answering the basic needs of the entire population is quite often too resource-demanding to become challenged by an intervention that can be lifesaving but that will benefit a small number of children.

Because of budget constraints and the extend of unmet basic health needs in the population, one may wonder if the main EE approach should not, therefore, be a cost/DALY approach. Cost/DALY approach uses DALYs as the health outcome indicator. DALYs is a composite indicator that combines premature mortality (mortality that occurs in < 92 years of age) and a disability score established internationally. DALYs are used to measure the burden of diseases in a population. It can for example allow determining the percentage of the burden (premature mortality and disability) that a health condition brings into a population. Measuring the DALY changes thanks to intervention, for example, the screening for a newborn condition allows estimating the global impact on the total burden of disease in a territory or country. DALYs are therefore particularly suited to measure opportunity costs at the population level. For LMICs particularly concerned with the difficulty to meet unanswered basic health needs, DALY provides a piece of priceless information that CEA and CUA cannot provide.

However, data needed to judge the relevance of an NBS program in LMICs, as in HICs, cannot come down to just cost/DALY studies. A policy-making process is based on more than just cost-effectiveness data. In many LMICs, a lack of explicit policy-making criteria for NBS programs is the main reason for the lack of transparency in the decision-making process that has been deplored in the literature [39]. Building a consensus on transparent and standardized criteria on what interventions to base the decision on what to offer to the population is probably the starting point of a rational offer of health care services that each country should consider.

Indeed, an HTA agency should help LMICs reduce resource waste, inefficiencies, and inappropriate investments in health systems. An HTA agency can help decision-makers address issues relating to both affordability and equity when allocating resources [40-42]. Above all, HTA would allow defining which country-based suitable methodological approach to use in order to address national preoccupations in the offer of health services.

Effectively, NBS programs in many developing countries are nonexistent or have been defined by an ad hoc committee consisting of clinician experts in neonatology (Table 1). This illustrates the need to replace a decision-making process based on experts’ opinion, by an institution as a HTA agency to support the rational identification of needs and the most effective and efficient way to answer those needs, taking into account the specificity of the country.

Table 1.
Newborn screening and criteria considered by HTAs in some developed and developing countries.

Country	Number of conditions screened (national panel)	HTA agency	Criteria				References
			The severity of the disease a	Availability of an effective treatment	Prevalence of the condition	Cost-effectiveness data
France	6	HAS1	Yes	Yes	Yes	Yes	[68-70]
Germany	21	IQWiG2	Yes	Yes	Yes	Optional	[69-70]
Sweden	26	TLV3	Yes	Yes	Yes	Yes	[69-70]
United Kingdom	9	NICE4	Yes	Yes	Yes	Yes	[70-71]
Italy	48	AIFA5	Yes	Yes	Yes	Yes	[69-70]
Netherland	25	ZIN6	Yes	Yes	Yes	Yes	[69-70]
Poland	29	AOTMiT7	Yes	Yes	Yes	Yes, mandatory by law	[69-70]
Spain	7	RedETS/ISCIII or ICP8	Yes	Yes	Yes	Yes (not mandatory)	[69-70]
United States	29	No national HTA but the American College of Medical Genetics (ACMG) to outline a process of standardization of outcomes and guidelines for state NBS program	Yes	Yes	Yes	Yes	[72]
Developing country
Philippine	6	The country has an HTA agency, but this was not involved in the decisions on the NBS program	-	-	-	-	[73-74]
Thailand	2	The country has an HTA agency, but this was not involved in the decisions on the NBS program	-	-	-	-	[6,75]
Malaysia	2	The country has an HTA agency, but this was not involved in the decisions on the NBS program	-	-	-	-	[76]
Laos	0	No HTA agency	-	-	-	-	[55]
Cambodia	0	No HTA agency	-	-	-	-	[54-55]

• ^a
  severity is considered as part of clinical benefit assessment, taking into account symptoms, possible consequences, including physical or cognitive handicap, and disease progression in terms of mortality and morbidity
• ¹
  The Haute Autorité de santé or French National Authority for Health
• ²
  Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen or The independent Institute for Quality and Efficiency in Health Care
• ³
  Tandvårds- och läkemedelsförmånsverket or The Dental and Pharmaceutical Benefits Agency
• ⁴
  National Institute for Health and Care Excellence
• ⁵
  Agenzia Italiana del Farmaco or The Italian Medicines Agency
• ⁶
  Zorginstituut Nederland or The National Health Care Institute
• ⁷
  The Agency for Health Technology Assessment and Tariff System
• ⁸
  Red de Agencias de Evaluación de Tecnologías Sanitarias y Prestaciones del Sistema Nacional de Salud (RedETS) and the Interministerial Committee for Pricing (ICP)

An Illustration of the Role of EE in the Development of NBS Programs in LMICs: the Case of Vietnam

Newborn screening programs have been around since the 1960s in Western countries. Over time, due to technological advances, the scope of conditions included in the programs has expanded. Vietnam implemented its core NBS program in 1998 with a regional project called RAS/6/032 sponsored by the International Atomic Energy Institute. The program was established at the National Children's Institute. In 2006, NBS became national in scope. Since 2015, the NBS program is offered nationwide to every child born in a public health care facility. The program includes 3 diseases: G6PD enzyme deficiency, congenital hypothyroidism (CH), and congenital adrenal hyperplasia (CAH) [43]. However, in reality, it is still offered to a minority of children. According to the Department of Population and Family Planning of the ministry of health, the percentage of newborns undergoing newborn screening in 2018 represented 38.5% of births [44]. Moreover, Vietnam counts for 3.6% of children who were not born in medical facilities and whose parents are not offered the possibility to get screened for their neonates [45]. The reasons given for the low screening rate include the limited amount of central funding for this activity which explains why in some hospitals screening tests are available; the lack of interest by some hospitals in implementing a postnatal screening program; poor professional knowledge by the medical staff on how to screen newborns: there is a lack of human resources to provide competent training on screening. In addition, many hospitals operate in old facilities and equipment [46]. On the patient side, it has been shown that many women often ask to be discharged as early as possible after giving birth, mainly for economic reasons, before blood can be taken for the screening test. Finally, it is known that many parents refuse the test because they fear their children will get hurt [46].

Congenital hypothyroidism has been reported to affect 1 in 2500 to 1 in 5000 newborns in Vietnam [47-48], which is comparable to the world global prevalence of this condition [49-50]. The occurrence of G6PD deficiency in Vietnam is influenced by ethnicity. The highest prevalence (0.4-9.1%) is found in the Kinh population, who constitutes around 85% of the Vietnamese [51]. There is currently no available data on the prevalence of congenital adrenal hyperplasia in Vietnam. This is surprising given that the condition is included in the country's newborn screening program. It is important to note that the choice of the three diseases included in Vietnam's initial newborn screening program was reportedly based on the clinician's opinions. There is no formal information on whether economic, social, and ethical issues were considered [53].

The importance of experts’ opinion based on clinical experience, in the choice of conditions to be screened for is also present in other Southeast countries, as Cambodia and Laos [54-55]. In other countries, such as Singapore [56-57], Thailand [58] and the Philippine [59], the decision to include a particular condition tend to be based on strong evidence demonstrating the effectiveness of screening and the potential benefits to affected individuals and families

Changes brought by the technological diagnostic development in the field of neonatal screening, a new understanding of screenable conditions, and new possibilities of treatment, have fueled the expansion of NBS programs around the world [60-61]. However, Vietnam as many other LMICs is not taking part in this trend, but the government has set a specific target according to which, by 2030, 90% of newborns should be screened for at least 5 of the most common congenital diseases in the country: GPD deficiency; congenital hypothyroidism; congenital adrenal hyperplasia, galactose metabolism disorder, and phenylketonuria. However, other than their prevalence, why these conditions were chosen is unclear. No document describes the decision-making process. There is no available information on whether expected effectiveness and the cost-effectiveness data were taken into consideration when the decision to expand the program was taken.

Yet, Vietnam, has the capacity to apply a Health Technology Assessment Agency process to decide, based on an open and structured process, what kind of NBS program the country could offer to the population. In 2013, the Ministry of Health (MoH) of Vietnam decided to establish and operate a Health Strategy and Policy Institute (HSPI) to foster efficient cooperation between policymakers, research agencies, and other stakeholders. HSPI is responsible for evaluating studies and providing scientific evidence that the MoH can use to build and modify health strategies. HSPI is a Vietnamese version of an HTA agency. Moreover, HSPI is mandated to develop collaborations with international partners in the field of study on health policy and the healthcare system. It can therefore look for foreign support in its mandate to evaluate interventions. One notes, the HSPI has a Department of Health Economics that, among its responsibilities, has to promote the production of relevant economic data on interventions that could be offered to the population [53-62].

However, there is no indication so far that recommendations made by HSPI have been considered in regulatory and reimbursement decisions. Many studies have been undertaken by HSPI. Reports are categorized into eight subgroups: community healthcare, health financing, health system, human resource for health, medical sociology and HIV/AIDS, population, executive management, provision of health services, and public health. None of the studies were about a screening program or a topic in the field of perinatology. In addition, up to now, the institute has not produced any guidelines, rules, or procedure documents on outcome measurement and social and ethical issues, to support those who plan to produce economic data on health interventions.

Unfortunately, there is no real substitute to the HSPI in the country. As a consequence, as in many other LMICs, Vietnam has few explicit formal frameworks and guidelines for pricing and reimbursement processes [53-63] that are needed to make EE studies. Again, decisions are mainly taken based on experts' opinion and regulatory requirements [53].

The Vietnamese example illustrates the relevance of creating a formalized structure and processes to support stakeholders who have to decide which services should be offered by their national health care system. An HTA agency is the key pillar of a rational, transparent, and socially acceptable decision-making process by a public servant who is expected to be accountable to the general population. Yet, an HTA agency cannot be an institution that tries to mimic functional HTA agencies elsewhere in the world. HTA agencies have the responsibility to evaluate the relevance of intervention regarding the specific context of the country. This can only be done, if its preliminary work focuses on drawing an open societal consensus in the country on how to proceed to evaluate an intervention, hence on what methodology and the standardized process should be applied [64-65]. It appears that this step has not yet been taken in Vietnam. Taking this step would make the Institute better equipped to help the Vietnamese health care system meet the needs of the population more effectively and efficiently.

HTA agencies should be an essential component of well-functioning health systems in LMICs where there are budget constraints and poorly defined processes regarding how to prioritize healthcare needs [66]. However, implementing and operating an HTA agency faces challenges because of the lack of local data, limited technical expertise, stability of budget, and weak or non-existent local institutions with the capacity to conduct HTA [66]. These considerations are limiting the perspective of a structured NBS program decision process in Vietnam as well as in many LMICs.

One of the main aims of HTA is to inform decision-makers in relation to policy development. However, in LMIC a number of problems can prevent HTA from fulfilling its role. The most common barriers to implementation that have been identified in the Asia-Pacific region include independence in the policy-making process, poor quality assessment of policy-making, limited dissemination of research, and high respect for expert opinions [67]. Implementing an influential HTA agency is therefore challenging.

Conclusion

This article advocates for the establishment of health technology assessment agencies in LMICs to respond more effectively and efficiently to the health needs of their populations. The challenges are numerous but surmountable. The economic growth experienced by many developing countries, the fact that these countries have a growing body of well-trained young professionals who can conduct epidemiological and economic studies, and an interconnected world that offers opportunities for methodological and conceptual support for generating scientific data, suggest that the barriers to rational decision-making processes often described in the literature can be overcome. The absence of the basic structure of a functional Technology Assessment Agency, the pillar of rational decision-making, found in Vietnam and many other countries, is less and less justified. As a systematic approach and solid tool to promote a more efficient and sustainable healthcare system, an operational national HTA agency could provide specifically for its country, the multi-dimensional evidence needed on effectiveness, safety, economic implications, ethical, social, cultural, and legal issues, for which economic evaluation is an important and unique part of the optimizing decision-making process. Supporting the emergence of HTA and supporting those who will work to give it its specificity is probably the best approach to help LMICs to offer health care services that are more appropriate to their populations.

The example of the NBS program in Vietnam shows that unless these conditions are fulfilled, one should not expect the health care system to be able to define rationally which diseases should be screened for and how, in the country. Until then, it is expected that NBS implementation and expansion will unfortunately not be based on a formalized, structured, and transparent decision-process.

References

• 1. Guthrie R. Blood Screening for Phenylketonuria. JAMA 1961;178(8):863. doi:10.1001/jama.1961.03040470079019.
  » https://doi.org/10.1001/jama.1961.03040470079019
• 2. Zakaria R, Allen KJ, Koplin JJ, Roche P, Greaves RF. Advantages and challenges of dried blood spot analysis by mass spectrometry across the total testing process. EJIFCC 2016;27(4):288-317.
• 3. Sahai I, Erbe RW. Neonatal Screening. In: Pyeritz RE, Korf BR, Grody WWBTE (eds). Perinatal and reproductive genetics 7th ed. Academic Press; 2022:57-86. doi:10.1016/B978-0-12-815236-2.00004-7.
  » https://doi.org/10.1016/B978-0-12-815236-2.00004-7
• 4. Therrell BL, Padilla CD, Loeber JG, et al. Current status of newborn screening worldwide: 2015. Semin Perinatol 2015;39(3):171-187. doi:10.1053/j.semperi.2015.03.002.
  » https://doi.org/10.1053/j.semperi.2015.03.002
• 5. Cornel MC, Rigter T, Jansen ME, Henneman L. Neonatal and carrier screening for rare diseases: How innovation challenges screening criteria worldwide. J Community Genet 2021;12(2):257-265. doi:10.1007/s12687-020-00488-y.
  » https://doi.org/10.1007/s12687-020-00488-y
• 6. Padilla CD, Therrell BL. Newborn screening in the Asia Pacific region. J Inherit Metab Dis 2007;30(4):490-506. doi:10.1007/s10545-007-0687-7.
  » https://doi.org/10.1007/s10545-007-0687-7
• 7. Padilla CD. Towards universal newborn screening in developing countries: obstacles and the way forward. Ann Acad Med Singap 2008;37(12):4-6.
• 8. Weinstein MC. Recent developments in decision-analytic modelling for economic evaluation. Pharmacoeconomics 2006;24(11):1043-1053. doi:10.2165/00019053-200624110-00002.
  » https://doi.org/10.2165/00019053-200624110-00002
• 9. Cadth. Guidelines for the Economic Evaluation of Health Technologies Canada. https://www.cadth.ca/guidelines-economic-evaluation-health-technologies-canada-0 Published May 13, 2021. Accessed June 15, 2022.
  » https://www.cadth.ca/guidelines-economic-evaluation-health-technologies-canada-0
• 10. Sharma D, Aggarwal AK, Downey LE, Prinja S. National Healthcare Economic Evaluation Guidelines: A cross-country comparison. Pharmacoecon Open 2021;5(3):349-364. doi:10.1007/s41669-020-00250-7.
  » https://doi.org/10.1007/s41669-020-00250-7
• 11. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJSG. Methods for the economic evaluation of health care programmes Oxford, UK: Oxford University Press; 2007.
• 12. Zwahlen M. Prevention Effectiveness: A Guide to Decision Analysis and Economic Evaluation, 2nd Edn.: Haddix A, Teutsch SM, Corso PS. Int J Epidemiol 2003;32(6):1125-1126. doi:10.1093/ije/dyg302.
  » https://doi.org/10.1093/ije/dyg302
• 13. Fox-Rushby JA, Hanson K. Calculating and presenting disability adjusted life years (DALYs) in cost-effectiveness analysis. Health Policy Plan 2001;16(3):326-331. doi:10.1093/heapol/16.3.326.
  » https://doi.org/10.1093/heapol/16.3.326
• 14. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes Oxford, UK: Oxford University Press; 2015.
• 15. Zechmeister-Koss I, Stanak M, Wolf S. The status of health economic evaluation within decision making in Austria. Wiener Medizinische Wochenschrift 2019;169(11):271-283. doi:10.1007/s10354-019-0689-8.
  » https://doi.org/10.1007/s10354-019-0689-8
• 16. Wang BCM. The rise of health economics and outcomes research in healthcare decision-making. Leuk Res 2013;37(3):238-239. doi:10.1016/j.leukres.2012.10.024.
  » https://doi.org/10.1016/j.leukres.2012.10.024
• 17. Ciani O, Jommi C. The role of health technology assessment bodies in shaping drug development. Drug Des Devel Ther 2014;8:2273-2281. doi:10.2147/DDDT.S49935.
  » https://doi.org/10.2147/DDDT.S49935
• 18. Auvin S, Irwin J, Abi-Aad P, Battersby A. The problem of rarity: Estimation of prevalence in rare disease. Value Health 2018;21(5):501-507. doi:10.1016/j.jval.2018.03.002.
  » https://doi.org/10.1016/j.jval.2018.03.002
• 19. Bailey DB, Gehtland LM, Lewis MA, et al. Early Check: Translational science at the intersection of public health and newborn screening. BMC Pediatr 2019;19(1):238. doi:10.1186/s12887-019-1606-4.
  » https://doi.org/10.1186/s12887-019-1606-4
• 20. Griebsch I, Coast J, Brown J. Quality-adjusted life-years lack quality in pediatric care: A critical review of published cost-utility studies in child health. Pediatrics 2005;115(5):e600-e614. doi:10.1542/peds.2004-2127.
  » https://doi.org/10.1542/peds.2004-2127
• 21. Rowen D, Rivero-Arias O, Devlin N, Ratcliffe J. Review of valuation methods of preference-based measures of health for economic evaluation in child and adolescent populations: Where are we now and where are we going? Pharmacoeconomics 2020;38(4):325-340. doi:10.1007/s40273-019-00873-7.
  » https://doi.org/10.1007/s40273-019-00873-7
• 22. Germain N, Aballéa S, Toumi M. Measuring the health-related quality of life in young children: How far have we come? J Mark Access Health Policy 2019;7(1):1618661. doi:10.1080/20016689.2019.1618661.
  » https://doi.org/10.1080/20016689.2019.1618661
• 23. Jabrayilov R, Vermeulen KM, Detzel P, Dainelli L, van Asselt ADI, Krabbe PFM. Valuing health status in the first year of life: The infant health-related quality of life instrument. Value in Health 2019;22(6):721-727. doi:10.1016/j.jval.2018.12.009.
  » https://doi.org/10.1016/j.jval.2018.12.009
• 24. Poder TG, Guertin JR, Touré M, et al. Canadian French translation and linguistic validation of the health-related quality of life utility measure for pre-school children. Expert Rev Pharmacoecon Outcomes Res 2021;21(6):1195-1201. doi:10.1080/14737167.2021.1895754.
  » https://doi.org/10.1080/14737167.2021.1895754
• 25. Furlong W, Rae C, Feeny D, et al. Generic health-related quality of life utility measure for preschool children (health utilities preschool): Design, development, and properties. Value Health 2023;26(2):251-260. doi:10.1016/j.jval.2022.07.015.
  » https://doi.org/10.1016/j.jval.2022.07.015
• 26. Groot Koerkamp B, Hunink MG, Stijnen T, Hammitt JK, Kuntz KM, Weinstein MC. Limitations of acceptability curves for presenting uncertainty in cost-effectiveness analysis. Med Decis Making 2007;27(2):101-111. doi:10.1177/0272989X06297394.
  » https://doi.org/10.1177/0272989X06297394
• 27. Daccache C, Rizk R, Dahham J, Evers SMAA, Hiligsmann M, Karam R. Economic evaluation guidelines in low- and middle-income countries: a systematic review. Int J Technol Assess Health Care 2022;38(1):e1. doi:10.1017/S0266462321000659.
  » https://doi.org/10.1017/S0266462321000659
• 28. Sharma M, Teerawattananon Y, Dabak SV, et al. A landscape analysis of health technology assessment capacity in the Association of South-East Asian Nations region. Health Res Policy Syst 2021;19(1):19. doi:10.1186/s12961-020-00647-0.
  » https://doi.org/10.1186/s12961-020-00647-0
• 29. Griffiths UK, Legood R, Pitt C. Comparison of economic evaluation methods across low-income, middle-income and high-income countries: What are the differences and why? Health Econ 2016;25(S1):29-41. doi:10.1002/hec.3312.
  » https://doi.org/10.1002/hec.3312
• 30. Pitt C, Goodman C, Hanson K. Economic evaluation in global perspective: a bibliometric analysis of the recent literature. Health Econ 2016;25(S1):9-28. doi:10.1002/hec.3305.
  » https://doi.org/10.1002/hec.3305
• 31. Al-Aqeel SA. State of health economic evaluation research in Saudi Arabia: a review. Clinicoecon Outcomes Res 2012;4:177-184. doi:10.2147/CEOR.S31087.
  » https://doi.org/10.2147/CEOR.S31087
• 32. Haghparast-Bidgoli H, Kiadaliri AA, Skordis-Worrall J. Do economic evaluation studies inform effective healthcare resource allocation in Iran? A critical review of the literature. Cost Eff Resour Alloc 2014;12:15. doi:10.1186/1478-7547-12-15.
  » https://doi.org/10.1186/1478-7547-12-15
• 33. Musuuza JS, Singer ME, Mandalakas AM, Katamba A. Key actors' perspectives on cost-effectiveness analysis in Uganda: a cross-sectional survey. BMC Health Serv Res 2014;14:539. doi:10.1186/s12913-014-0539-8.
  » https://doi.org/10.1186/s12913-014-0539-8
• 34. Prinja S, Chauhan AS, Angell B, Gupta I, Jan S. A systematic review of the state of economic evaluation for health care in India. Appl Health Econ Health Policy 2015;13(6):595-613. doi:10.1007/s40258-015-0201-6.
  » https://doi.org/10.1007/s40258-015-0201-6
• 35. Alkhaldi M, Al Basuoni A, Matos M, Tanner M, Ahmed S. Health Technology Assessment in High, Middle, and low-income countries: new systematic and interdisciplinary approach for sound informed-policy making: research protocole. Risk Manag Healthc Policy 2021;14:2757-2770. doi:10.2147/RMHP.S310215.
  » https://doi.org/10.2147/RMHP.S310215
• 36. Santatiwongchai B, Chantarastapornchit V, Wilkinson T, et al. Methodological variation in economic evaluations conducted in low- and middle-income countries: information for reference case development. PLoS One 2015;10(5):e0123853. doi:10.1371/journal.pone.0123853.
  » https://doi.org/10.1371/journal.pone.0123853
• 37. Adeagbo CU, Rattanavipapong W, Guinness L, Teerawattananon Y. The Development of the Guide to Economic Analysis and Research (GEAR) online resource for low- and middle-income countries’ health economics practitioners: a commentary. Value Health 2018;21(5):569-572. doi:10.1016/j.jval.2017.10.003.
  » https://doi.org/10.1016/j.jval.2017.10.003
• 38. Grosse SD, Rogowski WH, Ross LF, Cornel MC, Dondorp WJ, Khoury MJ. Population screening for genetic disorders in the 21st Century: evidence, economics, and ethics. Public Health Genomics 2010;13(2):106-115. doi:10.1159/000226594.
  » https://doi.org/10.1159/000226594
• 39. Diaby V, Campbell K, Goeree R. Multi-criteria decision analysis (MCDA) in health care: a bibliometric analysis. Oper Res Health Care 2013;2(1):20-24. doi:10.1016/j.orhc.2013.03.001.
  » https://doi.org/10.1016/j.orhc.2013.03.001
• 40. Banta D. The development of health technology assessment. Health Policy 2003;63(2):121-132. doi:10.1016/s0168-8510(02)00059-3.
  » https://doi.org/10.1016/s0168-8510(02)00059-3
• 41. Ritrovato M, Faggiano FC, Tedesco G, Derrico P. Decision-oriented health technology assessment: one step forward in supporting the decision-making process in hospitals. Value Health 2015;18(4):505-511. doi:10.1016/j.jval.2015.02.002.
  » https://doi.org/10.1016/j.jval.2015.02.002
• 42. Developing national institutional capacity for evidence-informed policy-making for health. East Mediterr Health J 2021;27(3):314-315. doi:10.26719/2021.27.3.314.
  » https://doi.org/10.26719/2021.27.3.314
• 43. Bionet. Giới Thiệu Về Sàng lọc sơ sinh. http://sanglocsosinh.vn/gioi-thieu/gioi-thieu-ve-sang-loc-so-sinh-165.html Published 2016. Accessed August 30, 2021.
  » http://sanglocsosinh.vn/gioi-thieu/gioi-thieu-ve-sang-loc-so-sinh-165.html
• 44. Trang M. 90% trẻ sơ sinh được tầm soát ít nhất 5 bệnh bẩm sinh. http://dansohcm.gov.vn/tin-tuc/tin-tuc-chuyen-nghanh/5327/90-tre-sinh-duoc-tam-soat-nhat-5-benh-bam-sinh/ Published 2018. Accessed June 15, 2022.
  » http://dansohcm.gov.vn/tin-tuc/tin-tuc-chuyen-nghanh/5327/90-tre-sinh-duoc-tam-soat-nhat-5-benh-bam-sinh/
• 45. UNICEF. Việt Nam SDGCW-Chăm Sóc Trước Sinh. https://www.unicef.org/vietnam/media/8776/file/Ch%C4%83m%20s%C3%B3c%20tr%C6%B0%E1%BB%9Bc%20sinh.pdf Published 2021. Accessed June 15, 2022.
  » https://www.unicef.org/vietnam/media/8776/file/Ch%C4%83m%20s%C3%B3c%20tr%C6%B0%E1%BB%9Bc%20sinh.pdf
• 46. VŨ K. Sàng lọc trước sinh và sơ sinh: Còn những khó khăn từ nhiều phía. SỨC KHỎE. http://hanoimoi.com.vn/tin-tuc/Suc-khoe/556189/sang-loc-truoc-sinh-va-so-sinh-con-nhung-kho-khan-tu-nhieu-phia Published 2012. Accessed June 15, 2022.
  » http://hanoimoi.com.vn/tin-tuc/Suc-khoe/556189/sang-loc-truoc-sinh-va-so-sinh-con-nhung-kho-khan-tu-nhieu-phia
• 47. Phạm Thị Hạnh. Phát hiện sớm suy giáp bẩm sinh, tránh nguy biến. Suc Khoe Va Doi Song. https://suckhoedoisong.vn/phat-hien-som-suy-giap-bam-sinh-tranh-nguy-bien-169147852.htm Published August 28, 2018. Accessed March 7, 2023.
  » https://suckhoedoisong.vn/phat-hien-som-suy-giap-bam-sinh-tranh-nguy-bien-169147852.htm
• 48. International Atomic Energy Agency. Screening of Newborns for Congenital Hypothyroidism Guidance for Developing Programmes. https://www-pub.iaea.org/MTCD/publications/PDF/Pub1234_web.pdf Published 2005. Accessed March 7, 2023.
  » https://www-pub.iaea.org/MTCD/publications/PDF/Pub1234_web.pdf
• 49. Mengreli C, Kanaka-Gantenbein C, Girginoudis P, et al. Screening for congenital hypothyroidism: the significance of threshold limit in false-negative results. J Clin Endocrinol Metab 2010;95(9):4283-4290. doi:10.1210/jc.2010-0057.
  » https://doi.org/10.1210/jc.2010-0057
• 50. Pearce MS, Korada M, Day J, et al. Increasing incidence, but lack of seasonality, of elevated TSH levels, on newborn screening, in the North of England. J Thyroid Res 2010;2010:101948. doi:10.4061/2010/101948.
  » https://doi.org/10.4061/2010/101948
• 51. Thị Thúy CV. Xét nghiệm G6PD giúp phát hiện bệnh thiếu men G6PD. MEDLATEC. https://medlatec.vn/tin-tuc/xet-nghiem-g6pd-giup-phat-hien-benh-thieu-men-g6pd-s58-n16531#:~:text=T%E1%BB%B7%20l%E1%BB%87%20m%E1%BA%AFc%20b%E1%BB%87nh%20n%C3%A0y,s%E1%BB%91%20hay%20d%C3%A2n%20t%E1%BB%99c%20Kinh Published December 14, 2019. Accessed March 12, 2023.
  » https://medlatec.vn/tin-tuc/xet-nghiem-g6pd-giup-phat-hien-benh-thieu-men-g6pd-s58-n16531#:~:text=T%E1%BB%B7%20l%E1%BB%87%20m%E1%BA%AFc%20b%E1%BB%87nh%20n%C3%A0y,s%E1%BB%91%20hay%20d%C3%A2n%20t%E1%BB%99c%20Kinh
• 52. Văn Quốc Vũ H, Ngọc DT. The prevalenceof G6PD deficiency and some related factors in newborn at can tho obstetrics and gynecology hospital 2021-2022. Vietnam Med J 2022;516(1):31-35. doi:10.51298/vmj.v516i1.2938.
  » https://doi.org/10.51298/vmj.v516i1.2938
• 53. Lee HY, Nguyen TT, Park S, Hoang VM, Kim WH. Health technology assessment development in Vietnam: a qualitative study of current progress, barriers, facilitators, and future stategies. Int J Environ Res Public Health 2021;18(16):8846. doi:10.3390/ijerph18168846.
  » https://doi.org/10.3390/ijerph18168846
• 54. WHO. Western Pacific Program. Wild Salmon Center 2015;(2017):19-20.
• 55. Sivalal S. Health technology assessment in the Asia Pacific region. Int J Technol Assess Health Care 2009;25(S1):196-201. doi:10.1017/S0266462309090631.
  » https://doi.org/10.1017/S0266462309090631
• 56. Pwee KH. Health technology assessment in Singapore. Int J Technol Assess Health Care 2009;25(S1):234-240. doi:10.1017/S0266462309090692.
  » https://doi.org/10.1017/S0266462309090692
• 57. Viswambaram A, Brooks-Rooney C. Health technology assessment in Singapore: shining a light in the darkness. Value in Health 2017;20(9):A662. doi:10.1016/j.jval.2017.08.1593.
  » https://doi.org/10.1016/j.jval.2017.08.1593
• 58. Leelahavarong P, Doungthipsirikul S, Kumluang S, et al. Health Technology Assessment in Thailand: institutionalization and contribution to healthcare decision making: review of literature. Int J Technol Assess Health Care 2019;35(6):467-473. doi:10.1017/S0266462319000321.
  » https://doi.org/10.1017/S0266462319000321
• 59. Padilla CD, Therrell Jr BL, Alcausin MMLB, et al. Successful implementation of expanded newborn screening in the Philippines using tandem mass spectrometry. Int J Neonatal Screen 2022;8(1):8. doi:10.3390/ijns8010008.
  » https://doi.org/10.3390/ijns8010008
• 60. Moyer VA, Calonge N, Teutsch SM, Botkin JR. Expanding newborn screening: process, policy, and priorities. Hastings Cent Rep 2008;38(3):32-39. doi:10.1353/hcr.0.0011.
  » https://doi.org/10.1353/hcr.0.0011
• 61. Therrell BLJ. U.S. newborn screening policy dilemmas for the twenty-first century. Mol Genet Metab 2001;74(1-2):64-74. doi:10.1006/mgme.2001.3238.
  » https://doi.org/10.1006/mgme.2001.3238
• 62. Tantivess S, Chalkidou K, Tritasavit N, Teerawattananon Y. Health Technology Assessment capacity development in low- and middle-income countries: Experiences from the international units of HITAP and NICE. F1000Res 2017;6:2119. doi:10.12688/f1000research.13180.1.
  » https://doi.org/10.12688/f1000research.13180.1
• 63. Dankó D. Health technology assessment in middle-income countries: recommendations for a balanced assessment system. J Mark Access Health Policy 2014;2. doi:10.3402/jmahp.v2.23181.
  » https://doi.org/10.3402/jmahp.v2.23181
• 64. Millar R, Morton A, Bufali MV, et al. Assessing the performance of health technology assessment (HTA) agencies: developing a multi-country, multi-stakeholder, and multi-dimensional framework to explore mechanisms of impact. Cost Effectiveness and Resource Allocation 2021;19(1):37. doi:10.1186/s12962-021-00290-8.
  » https://doi.org/10.1186/s12962-021-00290-8
• 65. Schlander M. HTA agencies need evidence-informed deliberative processes comment on “use of evidence-informed deliberative processes by health technology assessment agencies around the globe”. Int J Health Policy Manag 2021;10(3):158-161. doi:10.34172/ijhpm.2020.22.
  » https://doi.org/10.34172/ijhpm.2020.22
• 66. Chalkidou K, Levine R, Dillon A. Helping poorer countries make locally informed health decisions. BMJ 2010;341:c3651. doi:10.1136/bmj.c3651.
  » https://doi.org/10.1136/bmj.c3651
• 67. Murunga VI, Oronje RN, Bates I, Tagoe N, Pulford J. Review of published evidence on knowledge translation capacity, practice and support among researchers and research institutions in low- and middle-income countries. Health Res Policy Syst 2020;18(1):16. doi:10.1186/s12961-019-0524-0.
  » https://doi.org/10.1186/s12961-019-0524-0
• 68. HAS. Évaluation a Priori de l ’ Extension Du Dépistage Néonatal à Une Ou Plusieurs Erreurs Innées Du Métabolisme Par La Technique de Spectrométrie de Masse En Tandem En Population Générale En France. https://www.has-sante.fr/jcms/c_2866458/fr/evaluation-a-priori-de-l-extension-du-depistage-neonatal-a-une-ou-plusieurs-erreurs-innees-du-metabolisme-par-spectrometrie-de-masse-en-tandem-volet-2 Published February 3, 2020. Accessed March 12, 2023.
  » https://www.has-sante.fr/jcms/c_2866458/fr/evaluation-a-priori-de-l-extension-du-depistage-neonatal-a-une-ou-plusieurs-erreurs-innees-du-metabolisme-par-spectrometrie-de-masse-en-tandem-volet-2
• 69. Angelis A, Lange A, Kanavos P. Using health technology assessment to assess the value of new medicines: results of a systematic review and expert consultation across eight European countries. Eur J Health Econ 2018;19(1):123-152. doi:10.1007/s10198-017-0871-0.
  » https://doi.org/10.1007/s10198-017-0871-0
• 70. Wilsdon T, Saada R, Ferguson M. A Landscape Assessment of Newborn Screening in Europe. Technology Networks. https://www.technologynetworks.com/diagnostics/blog/a-landscape-assessment-of-newborn-screening-in-europe-359964 Published April 4, 2022. Accessed March 12, 2023.
  » https://www.technologynetworks.com/diagnostics/blog/a-landscape-assessment-of-newborn-screening-in-europe-359964
• 71. Downing M, Pollitt R. Newborn bloodspot screening in the UK - past, present and future. Ann Clin Biochem 2008;45(1):11-17. doi:10.1258/acb.2007.007127.
  » https://doi.org/10.1258/acb.2007.007127
• 72. American College of Medical Genetics Newborn Screening Expert Group. Newborn screening: toward a uniform screening panel and system--executive summary. Pediatrics 2006;117(5 Pt 2):S296-S307. doi:10.1542/peds.2005-2633I.
  » https://doi.org/10.1542/peds.2005-2633I
• 73. Padilla CD. Newborn screening in the Philippines. Southeast Asian J Trop Med Public Health 2003;34 Suppl 3:87-88.
• 74. Padilla CD, Therrell BL Jr, Alcausin MMLB, et al. Successful Implementation of Newborn Screening for Hemoglobin Disorders in the Philippines. Int J Neonatal Screen 2021;7(2):30. doi:10.3390/ijns7020030.
  » https://doi.org/10.3390/ijns7020030
• 75. Charoensiriwatana W, Janejai N, Boonwanich W, Krasao P, Chaisomchit S, Waiyasilp S. Neonatal screening program in Thailand. Southeast Asian J Trop Med Public Health 2003;34 Suppl 3:94-100.
• 76. Leong YH, Gan CY, Tan MAF, Majid MIA. Present status and future concerns of expanded newborn screening in malaysia: sustainability, challenges and perspectives. Malays J Med Sci 2014;21(2):63-67.

• Funding

  This article received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Publication Dates

• Publication in this collection
  08 May 2023
• Date of issue
  2023

History

• Received
  06 Oct 2022
• Accepted
  04 Apr 2023