Decision-making in the midst of uncertainty: appraising expensive

Decisions need to be made about which services or technologies should be prioritized for provision in the NHS in England .The analysis focuses specifically on the National Institute for Health and Care Excellence (NICE), and on how they appraise expensive medicines. This analysis takes a sociological perspective on decision-making in relation to uncertainty and how uncertainties are managed, drawing on evidence from a scoping study and an ethnographic study. Uncer-tainties were central to these rationing decisions. Three types of layers of uncertainty -epistemic, procedural and interpersonal – were shown to be salient. Another form of uncertainty was associated with the complexity of the science and that included the level of technicality of the information provided. The analysis highlighted the salience of uncertainties associated with interpersonal relations and the relations between the committees and the drug industry, clinical and patient experts. A key element in these relationships was trust. Decision makers adopted a mixture of formal and informal, collective and individual strategies in making decisions and a need to exercise pragmatism within a more formal institutional framework. The paper concludes by considering more recent policy developments in relation to appraising expensive medicines.


Background context
Health care in England is predominantly provided by the National Health Service (NHS) which is a universal system free at the point of access funded by the government, primarily from public taxation.The public and patients can obtain the medicines that they need by purchasing them directly over-the-counter from pharmacists or on prescription via clinicians although the extensive system of exemptions allows nearly 90% to be dispensed in the community free of charge.The National Health Service, like many other publicly funded health systems, has limited resources so decisions need to be made about which services or technologies should be prioritised or in a more negative sense rationed.In its early days NHS was characterised by approach which could be described as implicit rationing 1 where decisions about the allocation of resources were made predominantly by clinicians and the criteria for decision-making was not transparent and did not involve the public.Implicit rationing strategies 1 , included denial e.g.turning away non -urgent patients or patients requiring elective surgery being encouraged to use the private sector; selection e.g.only accept patients with greatest likelihood of benefit from care; deflection e.g.patients directed to other service; deterrence e.g.make access to a service difficult; delay e.g.long waiting lists/times;dilutione.g.everyone gets less which is reflected in the postcode lottery and termination e.g.early hospital discharge.In the 1990s there was shift in policy towards a more explicit form of rationing 1 with an emphasis on transparency and public involvement as the NHS was seen as increasingly inefficient and doctors' authority and expertise was increasingly being contested.
This shift in approach to rationing or priority setting was reflected in the introduction of National Institute for Health and Care Excellence (NICE).The introduction of NICE into the NHS medicines evaluative context was part of a broader approach which was the implementation of evidence-based medicine (EBM) into the NHS, particularly as an attempt to eradicate inefficient use of resources and regional and local disparities in care i.e. the so-called post code lottery.The post code lottery was where there were marked local variationa in the availability and accessibility of expensive medicines and health technologies 2 .This process began in the early 1990s under the Conservative government but was inherent in health policies of the New Labour government after 1997 3 .However, the emergence of NICE can be traced indirectly to 1984 when a limited list of medicines was introduced into the NHS which met with strong opposition from doctors (concerned about the challenges to their clinical autonomy) and industry, so eventually a diluted selected list was put in place.
Further conflicts were involved over the availability of the drug Viagra through the NHS, which led the government to introduce NICE as a measure beyond the selected list to review data and make evidence based pharmacoeconomic recommendations 4,5 .The development of NICE since 1999 created another level of evaluation in the NHS 5 .Though NICE has no power to license drugs, it does however have a very significant role in appraising drugs on the basis of cost-effectiveness for use within the NHS, particularly where there is the potential for significant impact on NHS resources 4,5 .This may take the form of economic appraisals of technology (analysed in terms of costs per quality adjusted life years (QA-LYs)) or in the form of clinical guidelines that may advocate a particular treatment (which may also be justified in terms of QALYs when advocating a drug treatment) 2 .NICE formally make decisions based on evaluations of cost-effectiveness.In other words, NICE evaluates whether improvements in QALYs (including overall survival) are justified by the expenditure required 5 .A recommendation is made on the basis of the cost effectiveness threshold (the incremental cost-effectiveness-ICER must be £20,000-£30,000 orless) 3 .There are now higher thresholds for patients in end of life situations (£50,000 per QALY) and for very rare diseases (£100, 000 -£300,000 per QALY 6 .
The introduction of NICE created a 'fourth hurdle' in the regulation of pharmaceuticals in England NICE and cost effectiveness evaluation was added to the assessment of quality, efficacy and safety by other medicines regulatory bodies 7 .Thus, it could be characterised as adopting a role as an explicit rationing agency to ensure consistent equitable patient access to drugs across the entire NHS and the efficient use of public finances by regulating NHS consumption of new and expensive drugs by cost effectiveness criteria.This was to be based on rigorous appraisals of scientific evidence and NICE was to manage uncertainty through a calculative and evidence-based approach.
NICE, however, has experienced problems since it was set up, not least in terms of maintaining its legitimacy 1,8 .It has been well docu-mented 1,9 how on some occasions NICE decision-making has been undermined or bypassed by the government in response to pressures from the public or some sections of the public and the drug industry which has been fuelled or amplified by the media.There are related concerns about the extent to which NICE has experienced regulatory capture where a government agency regulating the drug industry represents the interests of the drug industry rather than the public 4 .The risk of regulatory capture has been enhanced by introduction of single as opposed to multiple technological appraisals which rely more on the drug industry for data and modelling.
Decision-making about technological appraisals by NICE emphasise the dominant influence of the technical criteria of cost-effectiveness although in some cases social values tended to receive some explicit recognition in the decision making such as in the treatment for younger children.The attempt to explicitly incorporate social and ethical values was shaped by an approach described as 'accountability for reasonableness' which emphasised the conditions of transparency, relevance and revisability 10 .Hence, NICE introduced further prioritisation criteria based on social value judgements which included: severity of underlying condition, unmet need, significant innovation, wider social benefit, disadvantaged populations and children 4 .Thus, there was a possible tension for NICE in that they were attempting to explicitly take social values and factors into account, but at the same time trying to enhance transparency and standardisation.This approach is evident also in Scotland which has its own Scottish Medicines Consortium 11 and has a different procedure to NICE but has recently introduced PACE which aims to give patients and clinicians a stronger voice in the assessment of orphan and end of life medicines 11 This paper focuses on how decisions are made at the national level about recommendations for the provision of expensive medicines by NICE in England.The analysis presented here focuses specifically on evidence about how NICE appraise expensive medicines.This analysis takes a sociological perspective, as opposed to a clinical and economic approach.The focus here is on the social influences which are more implicit in the decision-making in relation to uncertainty and how they are managed using evidence primarily from a scoping study using documentary analysis and an ethnographic study [12][13][14] .

Uncertainty and decision making about appraising medicines
Uncertainty has been broadly defined as 'the normal determinant or unsettled quality of a statement or knowledge claim' 15 but its meaning varies according to the disciplinary context in which uncertainty is being explored 16 .The different forms of uncertainty faced by clinicians in decision making are well documented 17 but the types of uncertainty which manifest themselves in rationing or priority setting decisions are less recognised.
A scoping analysis 18 carried out on NICE documents relating to technological evaluations identified at least three distinct layers or forms of uncertainty which might be manifest with in NICE appraisals.At the most basic level isthe domain of epistemic uncertainty -relating to the effectiveness of certain methods of investigation to provide knowledge about conditions and their treatment.At least two socially constructed processes may be applied in dealing with this layer of uncertainty -confidence in the system of bio-medical knowledge and especially in the approach of randomised controlled trials 19 and confidence in the publication system of medical journals to differentiate reliable (published) studies from biased (non-published/drug industry) data 20 .
The second layer of uncertainty is procedural -involving the various possible methods and approaches to considering/modelling effectiveness, the vast amounts of evidence which could be considered, and the contestable and conflicting outputs associated with these.Such complexity might exclude any notion of comprehensive rationality and rather encourages strategies of 'muddling through 12,21 .
The third layer of uncertainty is interpersonal or relational -regarding the competency and motives of those providing evidence and/or recommendations within the process.Uncertainty may exist due to conflicting perspectives and relative expertise 22 of an array of contributors, but also due to the interests that certain individuals (eg.expert patients, drug industry representatives) have in the outcome of the decision-making process.
Drawing on this conceptual framework identifying the three layers of uncertainty, an ethnographic study 13 was carried out to explore the decision-making process and more specifically the various ways in which different forms of uncertainty -epistemic, procedural, relational Calnan M and others were perceived, presented and tackled within these drug appraisals (Single Technological Appraisals STAs). .It became evident early in the data collection that the management of uncertainties spontaneously emerged as a key issue for participants.Uncertainty was a key issue for the committee members, but what of the importance and relevance of the three key aspects of uncertainty that were emphasized in the conceptual framework which was derived from the previous documentary analysis.The evidence showed that epistemic, procedural and interpersonal uncertainty had relevance for decision-making based on these ethnographic data.
A significant source of epistemic uncertainty was the quality of the evidence base as the published evidence was dated and based on trials carried out with different aims and were not necessarily suited to NICE's criteria.Therewas also the maturity of the data in that some of these trials were carried out over a relative short period and the epidemiologists in particular on the committee were looking for evidence of longer-term outcomes.In addition, quality of evidence base was also seen to be problematic in relation to certain social groups, such as younger children, where there was also a lack of evidence about trials for drugs for the treatment of younger people but also where social or cultural values also came into play.There was also uncertainty about the publication process and committee members reported that the evidence review groups showed the flaws in some of the published papers suggesting that the kind of 'gold standard' of publication might not be a trustworthy source or indicator of strong evidence.
In terms of these different types of uncertainty, there was also evidence that one tended to flow into another.
There was also another area which was important in terms of its relationship with uncertainty and that was complexity.This refers not only to the sheer volume of the evidence but also to the complexity of the evidence.One source of this complexity was the drug industry who might be responsible for overcomplicating the modelling and the presentation of the data as it was perceived as a deliberate strategy by the drug manufacturers to mystify or complicate the analysis to conceal maybe some of the uncertainties in the messages becoming evident.However, it is suggested that this had a cost for the industry, as the committee members become more conservative in their decision making and their willingness to recommend, the go ahead of that particular drug.
Interpersonal or relational uncertainty was evident in a number of contexts such as in the level and nature of trust in the drug industry and the ambivalence about both the clinical expert's testimonies and patient's testimonies.Thus, there is lack of trust in the line that the drug industries take in terms of interpretation of particular data.Clinical experts tend to support the recommendation of a drug as it can both enhance their clinical specialism and the quality of patient care that they can provide.They can be influential but there was concern expressed by committee members that the clinical experts had been compromised and had conflicts of interest because they were also associated with the company which was manufacturing this drug.
What of patient engagement and voice?Patient organisations usually select a patient representative to articulate why this particular drug was so important and needed to be provided.There was some suggestion that patient's voice tended to be neglected or underrepresented.However, while patient representatives can have a powerful emotional voice and, as with the clinical expert, there was also criticism of how the patient voice tended to adopt, quite understandably, a narrower view than the role of the committee member.The former focused on the specific need for treatment of their condition whereas the latter needed to see the bigger picture, in terms of if this drug was to be recommended, what drug or technology would not be resourced on the NHS, so committee members had to think about those rationing decisions in this broader context.This had implications for the 'patient voice' or the 'patient influence' and the dual roles the committee took.So the committee can either be rational and take the position that "it's too limited in focus, too emotional" or they could actually adopt what might be called a more 'human approach' and move away from the harsh reality of explicit rationing decisions 1 .Once again, as with clinical experts there were concerns expressed by members about conflicts of interest and the link between some of the patients and the drug industry although patient representatives along with other representatives do have to declare if they have conflicts of interest.It is perhaps understandablewhy patient support groups and the drug industry might come together in an alliance in that they may share the same interests -they both might have wanted the drug into the health system as quickly as possible, but for different reasons.
The ethnography also explored the strategies adopted for addressing and managing uncer-tainty although not all the uncertainties are addressed, and some are ignored.This approach relates to what is called bounded rationality, which involves an understanding that there are some uncertainties we are not going to be addressed and which are marginalized and placed on the periphery, with the focus being on the best available evidence.There are statistical techniques 23 for quantifying and attempting to estimate uncertainties which are used in the analysis presented in these meetings although not all uncertainties are resolved by these techniques.The Chair of the committee was crucial in steering the committee towards a decision and enabled the bypassing of uncertainties which was described as 'the fudge factor' .
One strategy used to manage these uncertainties is a referral for more evidence because it is seen as possibly the easy option.Requesting more evidence is seen to be justified when there was uncertainty about whether the drug manufacturers "presented all the evidence available".Alternatively, a referral is made to other specialist assessment groups as the analysis of the drug manufacturers may not be trusted.But there were also informal individual strategies for dealing with uncertainties: one was to invoke a subjective 'gut' feeling and being pragmatic was a common stance adopted by committee members which circumscribed a number of personalised strategies.
There was evidence of a form of negotiation between committee members and the drug industry.From some of industry's representatives' point of view, the appraisal process is seen in terms of a trial and error strategy whereby the initial submission was turned down and the resubmission ended up with the required outcomes.This was an approach that committee members also engaged with to elicit more information from the drug industry.
The complexity of the information presented sometimes constrained discussions and led to a lack of participation.Those people who did not have any training in statistics or health economics tended to be excluded from the discussion and so there was evidence of trust in experts by some committee members because of the complexity of the science and leading to the exclusion of those with that lack of expertise.However, a number of different types of trust were prevalent which were acquiescent trust, conditional trust and explicit distrust in some contexts 12 and this shed light on the extent to which NICE might have experienced regulatory capture, corporate bias or being dominated by the interests of the drug industry 4 .Certainly, there was little evidence of acquiescent trust adopted by committee members in their relationships with drug manufacturers, suggesting a type of resistance to regulatory capture, although the procedure of relying on data provided by the drug industry for modelling and evaluation suggests the procedure might be skewed towards the interests of the drug industry.Zinn 24 has described a number of different strategies which are used in decision-making to address forms of uncertainty as 'in between' strategies -including emotion and intuition -which fall between rational (calculative and probabilistic) and non-rational strategies (belief, hope, faith and avoidance).Trust is, as was confirmed in the ethnographic study, another 'in between' strategy which is seen as an especially vital means of bridging over uncertainty 25,26 through judging the reliability of individuals' intentions, integrity and competences.
In the 1970's, the 27 rationing process was characterised as "Muddling Through Elegantly", in that it was messy and nonlinear.The evidence from the ethnography suggested that the decision-making process was not linear but appears to be negotiated.There appears to be adoption of a more pragmatic rationality involved in this decision-making process which suggested that the decision -making process nowadays might be better characterised as a form of navigation rather than muddling through.Decision makers adopted a mixture of formal and informal, collective and individual, strategies in making decisions and a need to exercise pragmatism within a more formal institutional framework.This pragmatic rationality was also evident in a study in the Netherlands 28 which explored if and how societal weighing rationalitywas incorporated in health care coverage decisions.This study showed that these decisions required understanding different types of knowledge and combining them into one decision, while also adhering not only to substantive requirements, but also processual ones.
This body of evidence, therefore, suggests that the rationality of economics is not sufficient to carry out rationing decisions effectively and although the formal 1 discourse associated with NICE technological appraisals and other systems is framed in terms of cost effectiveness, the evidence suggest that this only presents a partial picture of how decisions were made.The evidence from the ethnographic study also threw light on the relationship between science and trust in medicine suggesting that trust shapes relation-Calnan M ships between science and medicine far more than science directs trust in medicines 12 .

Heightened uncertainty and recent policy developments
The field work for this ethnographic study was carried out between 2012 and 2014 and since then has been significant developments which have implications for the management of uncertainty.One development has involved anincrease into the market of new, more complex,medicines for cancer and orphan diseases 29 which are typically of high price and uncertain value with some being launched based on limited evidence of effectiveness from trials or the evidence base is less robust than in the past.This is evident both in the US 30 and the UK 31 and this policy might also reflect the cultural capital associated with some diseases.Epistemic uncertainty is particularly relevant here although the evidence from the ethnographic study of decision making showed that this epistemic uncertainty can lead into other forms of uncertainty particularly interpersonal uncertainty.This development has led to number of policy developments aimed at managing these greater uncertainties as it can lead to greater risks for the healthcare payer.
The launch of the Cancer Drugs Fund (CDF) in 2010 gave the government in England the opportunity to build political capital 32 as it was to be funding cancer drugs which NICE appraisals had rejected 33 although it might also be described as a policy of deregulation.Its impact was limited and it was revamped in 2016 introducing different criteria and for a recommendation for use within the CDF, a Managed Access Agreement needed to be agreed between the drug company and NHS England.The purpose of the 'managed access' period is to resolve significant remaining clinical uncertainty after consideration by NICE, with the CDF Managed Access Agreement consisting of two key components -a data collection arrangement and a CDF Commercial Agreement.The data collection arrangement aims to set out the data that need to be collected in order to address the key areas of uncertainty.Drugs or treatments that are expensive and do not have a significant benefit over existing treatments are unlikely to be approved by NICE for use in the NHS.However, there is some debate about the need to recognise the costs of carrying out more research to reduce decision uncertainty against its benefits.Other initiatives were introduced to deal with the risks of uncertainties in clinical and cost effectiveness for example Managed Entry Agreements (MEAs).MEAs are arrangements being introduced by NICE between the drug industry and healthcare payers that allow for coverage of new medicines while managing uncertainty around their financial impact or performance 34 .Patient access schemes have also been introduced which are pricing agreements proposed by the drug industry to enable patients to gain access to high costs drugs.Clearly then with these developments and the changing landscape of the marketplace the management of uncertainty will continue to be a key element in the appraisal of expensive medicines.NICE has been reviewing its policy towards appraisal which includes how NICE assesses uncertainty in its appraisals, for example, uncertainty about the effectiveness of a new drug or its costs 6 .
From a broader policy point of view there are the implications for England, and the UK of leaving the European Union (EU) in 2020.There is still uncertainty about what the exact outcome of leaving the EU will look like and these have been complicated by the social and economic consequences of the COVID-19 pandemic.In an analysis of the impact of Brexit on the supply and access to medicines 35 , it was concluded that if the UK were to cut ties with Europe fully -i.e.fragmentation of the current centralised regulatory and authorisation structures (the EMA and the MHRA) -and restrict free movement of the sector's talented EU workforce, the UK will no longer be an attractive place to do business.It was argued that this situation could mean disruption in supply chains with patients facing long delays in accessing new vaccines and innovative medicines if the 'hard' Brexit option is adopted although the number of genuinely innovative drugs, especially for cancer, appears to be minimal.The report favoured the 'soft' Brexit option in line with a Swiss model where a close relationship would exist between the UK and EMA although it is doubtful now if that approach will materialise.Recent concern has been expressed about the possible threat to the NHS from the US 'for profit' health corporate companies and also the costs of having to pay for more expensive medicines from the US as a result of trade deals 1 .
Finally, what are the implications for further research in the light of the evidence presented in this paper?There is a clear need for some international research comparing how decisions about the allocation of resources are made in countries with different health systems and associated socio-political values and different levels of re-sources.More specifically,from an international comparative point of view, the focus might be on whether these decision-making processesin relation to the management of uncertainties found in England are also evident and relevant to comparable health systems with priority setting agencies.The evidence from comparative research in Europe 36 showed that while the HTA agencies in the countries under study assessed similar types of evidence, the specific criteria/ endpoints used, their level of provision and requirement, and how they are incorporated (e.g.explicitly vs. implicitly) varies across countries but with uncertainty about their relative importance.This analysis needs to be extended globally to less well-resourced health systems such as to CONITEC in Brazil and similar agencies in other South American countries 1,[37][38][39] where contemporary phenomena have pressured the health systems, such as in the judicialization of access to medicines 40,41 .Evidence from documen-tary analysis of decision making by CONITEC showed that the evidence used in recommendation reports and those considered to be mandatory were very different, suggesting problems in decision-making processes.The authors conclude that there is a need to take a broader look at the factors that influence the type of evidence used in decision-making processes in order to contribute to the development of better practices and policies 42 .
Trust in health systems may also influence the efficiency of medicines appraisal and pharmaceutical policy 43 .The uncertainties associated with the recent demands of the COVID-19 pandemic have added even more pressure on to health systems 44 and the rational use of medicines has been jeopardized such as in the rhetoric about the benefits of certain drugs exemplified by the case of hydroxychloroquine 45 .It is a new scenario that can impact on decision-making in public policies and must be assessed in the future.