Decision-making in the midst of uncertainty: appraising expensive medicines in England

Tomada de decisão em meio à incerteza: avaliando medicamentos de alto preço na Inglaterra

Abstract

Decisions need to be made about which services or technologies should be prioritized for provision in the NHS in England .The analysis focuses specifically on the National Institute for Health and Care Excellence (NICE), and on how they appraise expensive medicines. This analysis takes a sociological perspective on decision-making in relation to uncertainty and how uncertainties are managed, drawing on evidence from a scoping study and an ethnographic study. Uncertainties were central to these rationing decisions. Three types of layers of uncertainty -epistemic, procedural and interpersonal - were shown to be salient. Another form of uncertainty was associated with the complexity of the science and that included the level of technicality of the information provided. The analysis highlighted the salience of uncertainties associated with interpersonal relations and the relations between the committees and the drug industry, clinical and patient experts. A key element in these relationships was trust. Decision makers adopted a mixture of formal and informal, collective and individual strategies in making decisions and a need to exercise pragmatism within a more formal institutional framework. The paper concludes by considering more recent policy developments in relation to appraising expensive medicines.

Key words:
Technology assessment; Evidence-based medicine; Medical sociology; Uncertainty

Resumo

O Serviço Nacional de Saúde Inglês precisa tomar decisões sobre quais serviços ou tecnologias devem ser priorizados. A análise se concentra no Instituto Nacional de Excelência em Saúde e Cuidados (NICE) e em como avalia medicamentos de alto preço. Essa análise adota uma perspectiva sociológica na tomada de decisões em relação à incerteza e como elas são gerenciadas, com base em um estudo etnográfico e um estudo de escopo. As incertezas foram centrais para essas decisões. Três tipos de camadas de incerteza - sistêmica, processual e interpessoal - mostraram-se salientes. Outra forma de incerteza estava associada à complexidade da ciência e que incluía o nível de tecnicidade das informações fornecidas. A análise enfatizou particularmente a importância das incertezas associadas às relações interpessoais e às relações entre os comitês e os fabricantes de medicamentos, especialistas clínicos e de pacientes. Um elemento-chave nesses relacionamentos foi a confiança. Os tomadores de decisão adotaram uma mistura de estratégias formais e informais, coletivas e individuais na tomada de decisões e a necessidade de exercer o pragmatismo dentro de uma estrutura institucional formal. O texto finaliza considerando o desenvolvimento mais recente de políticas relacionadas à avaliação de medicamentos caros.

Palavras-chave:
Avaliação de tecnologias; Medicina baseada em evidências; Sociologia médica; Incerteza

Background context

Health care in England is predominantly provided by the National Health Service (NHS) which is a universal system free at the point of access funded by the government, primarily from public taxation. The public and patients can obtain the medicines that they need by purchasing them directly over-the-counter from pharmacists or on prescription via clinicians although the extensive system of exemptions allows nearly 90% to be dispensed in the community free of charge. The National Health Service, like many other publicly funded health systems, has limited resources so decisions need to be made about which services or technologies should be prioritised or in a more negative sense rationed. In its early days NHS was characterised by approach which could be described as implicit rationing¹ where decisions about the allocation of resources were made predominantly by clinicians and the criteria for decision-making was not transparent and did not involve the public. Implicit rationing strategies¹, included denial e.g. turning away non -urgent patients or patients requiring elective surgery being encouraged to use the private sector; selection e.g. only accept patients with greatest likelihood of benefit from care; deflection e.g. patients directed to other service; deterrence e.g.make access to a service difficult; delay e.g.long waiting lists/times;dilutione.g. everyone gets less which is reflected in the postcode lottery and termination e.g. early hospital discharge. In the 1990s there was shift in policy towards a more explicit form of rationing¹ with an emphasis on transparency and public involvement as the NHS was seen as increasingly inefficient and doctors’ authority and expertise was increasingly being contested.

This shift in approach to rationing or priority setting was reflected in the introduction of National Institute for Health and Care Excellence (NICE). The introduction of NICE into the NHS medicines evaluative context was part of a broader approach which was the implementation of evidence-based medicine (EBM) into the NHS, particularly as an attempt to eradicate inefficient use of resources and regional and local disparities in care i.e. the so-called post code lottery. The post code lottery was where there were marked local variationa in the availability and accessibility of expensive medicines and health technologies². This process began in the early 1990s under the Conservative government but was inherent in health policies of the New Labour government after 1997³. However, the emergence of NICE can be traced indirectly to 1984 when a limited list of medicines was introduced into the NHS which met with strong opposition from doctors (concerned about the challenges to their clinical autonomy) and industry, so eventually a diluted selected list was put in place.

Further conflicts were involved over the availability of the drug Viagra through the NHS, which led the government to introduce NICE as a measure beyond the selected list to review data and make evidence based pharmacoeconomic recommendations^4,5. The development of NICE since 1999 created another level of evaluation in the NHS⁵. Though NICE has no power to license drugs, it does however have a very significant role in appraising drugs on the basis of cost-effectiveness for use within the NHS, particularly where there is the potential for significant impact on NHS resources^4,5. This may take the form of economic appraisals of technology (analysed in terms of costs per quality adjusted life years (QALYs)) or in the form of clinical guidelines that may advocate a particular treatment (which may also be justified in terms of QALYs when advocating a drug treatment)². NICE formally make decisions based on evaluations of cost-effectiveness. In other words, NICE evaluates whether improvements in QALYs (including overall survival) are justified by the expenditure required⁵. A recommendation is made on the basis of the cost effectiveness threshold (the incremental cost-effectiveness- ICER must be £20,000-£30,000 orless)³. There are now higher thresholds for patients in end of life situations (£50,000 per QALY) and for very rare diseases (£100, 000 -£300,000 per QALY⁶.

The introduction of NICE created a ‘fourth hurdle’ in the regulation of pharmaceuticals in England NICE and cost effectiveness evaluation was added to the assessment of quality, efficacy and safety by other medicines regulatory bodies⁷. Thus, it could be characterised as adopting a role as an explicit rationing agency to ensure consistent equitable patient access to drugs across the entire NHS and the efficient use of public finances by regulating NHS consumption of new and expensive drugs by cost effectiveness criteria. This was to be based on rigorous appraisals of scientific evidence and NICE was to manage uncertainty through a calculative and evidence-based approach.

NICE, however, has experienced problems since it was set up, not least in terms of maintaining its legitimacy^1,8. It has been well documented^1,9 how on some occasions NICE decision-making has been undermined or bypassed by the government in response to pressures from the public or some sections of the public and the drug industry which has been fuelled or amplified by the media. There are related concerns about the extent to which NICE has experienced regulatory capture where a government agency regulating the drug industry represents the interests of the drug industry rather than the public⁴. The risk of regulatory capture has been enhanced by introduction of single as opposed to multiple technological appraisals which rely more on the drug industry for data and modelling.

Decision-making about technological appraisals by NICE emphasise the dominant influence of the technical criteria of cost-effectiveness although in some cases social values tended to receive some explicit recognition in the decision making such as in the treatment for younger children. The attempt to explicitly incorporate social and ethical values was shaped by an approach described as ‘accountability for reasonableness’ which emphasised the conditions of transparency, relevance and revisability¹⁰. Hence, NICE introduced further prioritisation criteria based on social value judgements which included: severity of underlying condition, unmet need, significant innovation, wider social benefit, disadvantaged populations and children⁴. Thus, there was a possible tension for NICE in that they were attempting to explicitly take social values and factors into account, but at the same time trying to enhance transparency and standardisation. This approach is evident also in Scotland which has its own Scottish Medicines Consortium¹¹ and has a different procedure to NICE but has recently introduced PACE which aims to give patients and clinicians a stronger voice in the assessment of orphan and end of life medicines¹¹

This paper focuses on how decisions are made at the national level about recommendations for the provision of expensive medicines by NICE in England. The analysis presented here focuses specifically on evidence about how NICE appraise expensive medicines. This analysis takes a sociological perspective, as opposed to a clinical and economic approach. The focus here is on the social influences which are more implicit in the decision-making in relation to uncertainty and how they are managed using evidence primarily from a scoping study using documentary analysis and an ethnographic study^12-14.

Uncertainty and decision making about appraising medicines

Uncertainty has been broadly defined as ‘the normal determinant or unsettled quality of a statement or knowledge claim’¹⁵ but its meaning varies according to the disciplinary context in which uncertainty is being explored¹⁶. The different forms of uncertainty faced by clinicians in decision making are well documented¹⁷ but the types of uncertainty which manifest themselves in rationing or priority setting decisions are less recognised.

A scoping analysis¹⁸ carried out on NICE documents relating to technological evaluations identified at least three distinct layers or forms of uncertainty which might be manifest with in NICE appraisals. At the most basic level isthe domain of epistemic uncertainty - relating to the effectiveness of certain methods of investigation to provide knowledge about conditions and their treatment. At least two socially constructed processes may be applied in dealing with this layer of uncertainty - confidence in the system of bio-medical knowledge and especially in the approach of randomised controlled trials¹⁹ and confidence in the publication system of medical journals to differentiate reliable (published) studies from biased (non-published/drug industry) data²⁰.

The second layer of uncertainty is procedural - involving the various possible methods and approaches to considering/modelling effectiveness, the vast amounts of evidence which could be considered, and the contestable and conflicting outputs associated with these. Such complexity might exclude any notion of comprehensive rationality and rather encourages strategies of ‘muddling through^12,21.

The third layer of uncertainty is interpersonal or relational - regarding the competency and motives of those providing evidence and/or recommendations within the process. Uncertainty may exist due to conflicting perspectives and relative expertise²² of an array of contributors, but also due to the interests that certain individuals (eg. expert patients, drug industry representatives) have in the outcome of the decision-making process.

Drawing on this conceptual framework identifying the three layers of uncertainty, an ethnographic study¹³ was carried out to explore the decision-making process and more specifically the various ways in which different forms of uncertainty - epistemic, procedural, relational and others were perceived, presented and tackled within these drug appraisals (Single Technological Appraisals STAs).^.It became evident early in the data collection that the management of uncertainties spontaneously emerged as a key issue for participants. Uncertainty was a key issue for the committee members, but what of the importance and relevance of the three key aspects of uncertainty that were emphasized in the conceptual framework which was derived from the previous documentary analysis. The evidence showed that epistemic, procedural and interpersonal uncertainty had relevance for decision-making based on these ethnographic data.

A significant source of epistemic uncertainty was the quality of the evidence base as the published evidence was dated and based on trials carried out with different aims and were not necessarily suited to NICE’s criteria. Therewas also the maturity of the data in that some of these trials were carried out over a relative short period and the epidemiologists in particular on the committee were looking for evidence of longer-term outcomes. In addition, the quality of evidence base was also seen to be problematic in relation to certain social groups, such as younger children, where there was also a lack of evidence about trials for drugs for the treatment of younger people but also where social or cultural values also came into play. There was also uncertainty about the publication process and committee members reported that the evidence review groups showed the flaws in some of the published papers suggesting that the kind of ‘gold standard’ of publication might not be a trustworthy source or indicator of strong evidence.

In terms of these different types of uncertainty, there was also evidence that one tended to flow into another.

There was also another area which was important in terms of its relationship with uncertainty and that was complexity. This refers not only to the sheer volume of the evidence but also to the complexity of the evidence. One source of this complexity was the drug industry who might be responsible for overcomplicating the modelling and the presentation of the data as it was perceived as a deliberate strategy by the drug manufacturers to mystify or complicate the analysis to conceal maybe some of the uncertainties in the messages becoming evident. However, it is suggested that this had a cost for the industry, as the committee members become more conservative in their decision making and their willingness to recommend, the go ahead of that particular drug.

Interpersonal or relational uncertainty was evident in a number of contexts such as in the level and nature of trust in the drug industry and the ambivalence about both the clinical expert’s testimonies and patient’s testimonies. Thus, there is lack of trust in the line that the drug industries take in terms of interpretation of particular data. Clinical experts tend to support the recommendation of a drug as it can both enhance their clinical specialism and the quality of patient care that they can provide. They can be influential but there was concern expressed by committee members that the clinical experts had been compromised and had conflicts of interest because they were also associated with the company which was manufacturing this drug.

What of patient engagement and voice? Patient organisations usually select a patient representative to articulate why this particular drug was so important and needed to be provided. There was some suggestion that patient’s voice tended to be neglected or underrepresented. However, while patient representatives can have a powerful emotional voice and, as with the clinical expert, there was also criticism of how the patient voice tended to adopt, quite understandably, a narrower view than the role of the committee member. The former focused on the specific need for treatment of their condition whereas the latter needed to see the bigger picture, in terms of if this drug was to be recommended, what drug or technology would not be resourced on the NHS, so committee members had to think about those rationing decisions in this broader context.This had implications for the ‘patient voice’ or the ‘patient influence’ and the dual roles the committee took. So the committee can either be rational and take the position that “it’s too limited in focus, too emotional” or they could actually adopt what might be called a more ‘human approach’ and move away from the harsh reality of explicit rationing decisions¹. Once again, as with clinical experts there were concerns expressed by members about conflicts of interest and the link between some of the patients and the drug industry although patient representatives along with other representatives do have to declare if they have conflicts of interest. It is perhaps understandablewhy patient support groups and the drug industry might come together in an alliance in that they may share the same interests - they both might have wanted the drug into the health system as quickly as possible, but for different reasons.

The ethnography also explored the strategies adopted for addressing and managing uncertainty although not all the uncertainties are addressed, and some are ignored. This approach relates to what is called bounded rationality, which involves an understanding that there are some uncertainties we are not going to be addressed and which are marginalized and placed on the periphery, with the focus being on the best available evidence. There are statistical techniques²³ for quantifying and attempting to estimate uncertainties which are used in the analysis presented in these meetings although not all uncertainties are resolved by these techniques. The Chair of the committee was crucial in steering the committee towards a decision and enabled the bypassing of uncertainties which was described as ‘the fudge factor’.

One strategy used to manage these uncertainties is a referral for more evidence because it is seen as possibly the easy option. Requesting more evidence is seen to be justified when there was uncertainty about whether the drug manufacturers “presented all the evidence available”. Alternatively, a referral is made to other specialist assessment groups as the analysis of the drug manufacturers may not be trusted. But there were also informal individual strategies for dealing with uncertainties: one was to invoke a subjective ‘gut’ feeling and being pragmatic was a common stance adopted by committee members which circumscribed a number of personalised strategies.

There was evidence of a form of negotiation between committee members and the drug industry. From some of industry’s representatives’ point of view, the appraisal process is seen in terms of a trial and error strategy whereby the initial submission was turned down and the resubmission ended up with the required outcomes. This was an approach that committee members also engaged with to elicit more information from the drug industry.

The complexity of the information presented sometimes constrained discussions and led to a lack of participation. Those people who did not have any training in statistics or health economics tended to be excluded from the discussion and so there was evidence of trust in experts by some committee members because of the complexity of the science and leading to the exclusion of those with that lack of expertise. However, a number of different types of trust were prevalent which were acquiescent trust, conditional trust and explicit distrust in some contexts¹² and this shed light on the extent to which NICE might have experienced regulatory capture, corporate bias or being dominated by the interests of the drug industry⁴. Certainly, there was little evidence of acquiescent trust adopted by committee members in their relationships with drug manufacturers, suggesting a type of resistance to regulatory capture, although the procedure of relying on data provided by the drug industry for modelling and evaluation suggests the procedure might be skewed towards the interests of the drug industry. Zinn²⁴ has described a number of different strategies which are used in decision-making to address forms of uncertainty as ‘in between’ strategies - including emotion and intuition - which fall between rational (calculative and probabilistic) and non-rational strategies (belief, hope, faith and avoidance). Trust is, as was confirmed in the ethnographic study, another ‘in between’ strategy which is seen as an especially vital means of bridging over uncertainty^25,26 through judging the reliability of individuals’ intentions, integrity and competences.

In the 1970’s, the²⁷ rationing process was characterised as “Muddling Through Elegantly”, in that it was messy and nonlinear. The evidence from the ethnography suggested that the decision-making process was not linear but appears to be negotiated. There appears to be adoption of a more pragmatic rationality involved in this decision-making process which suggested that the decision -making process nowadays might be better characterised as a form of navigation rather than muddling through. Decision makers adopted a mixture of formal and informal, collective and individual, strategies in making decisions and a need to exercise pragmatism within a more formal institutional framework. This pragmatic rationality was also evident in a study in the Netherlands²⁸ which explored if and how societal weighing rationalitywas incorporated in health care coverage decisions. This study showed that these decisions required understanding different types of knowledge and combining them into one decision, while also adhering not only to substantive requirements, but also processual ones.

This body of evidence, therefore, suggests that the rationality of economics is not sufficient to carry out rationing decisions effectively and although the formal¹ discourse associated with NICE technological appraisals and other systems is framed in terms of cost effectiveness, the evidence suggest that this only presents a partial picture of how decisions were made. The evidence from the ethnographic study also threw light on the relationship between science and trust in medicine suggesting that trust shapes relationships between science and medicine far more than science directs trust in medicines¹².

Heightened uncertainty and recent policy developments

The field work for this ethnographic study was carried out between 2012 and 2014 and since then has been significant developments which have implications for the management of uncertainty. One development has involved anincrease into the market of new, more complex,medicines for cancer and orphan diseases²⁹ which are typically of high price and uncertain value with some being launched based on limited evidence of effectiveness from trials or the evidence base is less robust than in the past. This is evident both in the US³⁰ and the UK³¹ and this policy might also reflect the cultural capital associated with some diseases. Epistemic uncertainty is particularly relevant here although the evidence from the ethnographic study of decision making showed that this epistemic uncertainty can lead into other forms of uncertainty particularly interpersonal uncertainty. This development has led to number of policy developments aimed at managing these greater uncertainties as it can lead to greater risks for the healthcare payer.

The launch of the Cancer Drugs Fund (CDF) in 2010 gave the government in England the opportunity to build political capital³² as it was to be funding cancer drugs which NICE appraisals had rejected³³ although it might also be described as a policy of deregulation. Its impact was limited and it was revamped in 2016 introducing different criteria and for a recommendation for use within the CDF, a Managed Access Agreement needed to be agreed between the drug company and NHS England. The purpose of the ‘managed access’ period is to resolve significant remaining clinical uncertainty after consideration by NICE, with the CDF Managed Access Agreement consisting of two key components - a data collection arrangement and a CDF Commercial Agreement. The data collection arrangement aims to set out the data that need to be collected in order to address the key areas of uncertainty. Drugs or treatments that are expensive and do not have a significant benefit over existing treatments are unlikely to be approved by NICE for use in the NHS. However, there is some debate about the need to recognise the costs of carrying out more research to reduce decision uncertainty against its benefits. Other initiatives were introduced to deal with the risks of uncertainties in clinical and cost effectiveness for example Managed Entry Agreements (MEAs). MEAs are arrangements being introduced by NICE between the drug industry and healthcare payers that allow for coverage of new medicines while managing uncertainty around their financial impact or performance³⁴. Patient access schemes have also been introduced which are pricing agreements proposed by the drug industry to enable patients to gain access to high costs drugs. Clearly then with these developments and the changing landscape of the marketplace the management of uncertainty will continue to be a key element in the appraisal of expensive medicines. NICE has been reviewing its policy towards appraisal which includes how NICE assesses uncertainty in its appraisals, for example, uncertainty about the effectiveness of a new drug or its costs⁶.

From a broader policy point of view there are the implications for England, and the UK of leaving the European Union (EU) in 2020. There is still uncertainty about what the exact outcome of leaving the EU will look like and these have been complicated by the social and economic consequences of the COVID-19 pandemic. In an analysis of the impact of Brexit on the supply and access to medicines³⁵, it was concluded that if the UK were to cut ties with Europe fully - i.e. fragmentation of the current centralised regulatory and authorisation structures (the EMA and the MHRA) - and restrict free movement of the sector’s talented EU workforce, the UK will no longer be an attractive place to do business. It was argued that this situation could mean disruption in supply chains with patients facing long delays in accessing new vaccines and innovative medicines if the ‘hard’ Brexit option is adopted although the number of genuinely innovative drugs, especially for cancer, appears to be minimal. The report favoured the ‘soft’ Brexit option in line with a Swiss model where a close relationship would exist between the UK and EMA although it is doubtful now if that approach will materialise. Recent concern has been expressed about the possible threat to the NHS from the US ‘for profit’ health corporate companies and also the costs of having to pay for more expensive medicines from the US as a result of trade deals¹.

Finally, what are the implications for further research in the light of the evidence presented in this paper? There is a clear need for some international research comparing how decisions about the allocation of resources are made in countries with different health systems and associated socio-political values and different levels of resources. More specifically,from an international comparative point of view, the focus might be on whether these decision-making processesin relation to the management of uncertainties found in England are also evident and relevant to comparable health systems with priority setting agencies. The evidence from comparative research in Europe³⁶ showed that while the HTA agencies in the countries under study assessed similar types of evidence, the specific criteria/endpoints used, their level of provision and requirement, and how they are incorporated (e.g. explicitly vs. implicitly) varies across countries but with uncertainty about their relative importance. This analysis needs to be extended globally to less well-resourced health systems such as to CONITEC in Brazil and similar agencies in other South American countries^1,37-39 where contemporary phenomena have pressured the health systems, such as in the judicialization of access to medicines^40,41. Evidence from documentary analysis of decision making by CONITEC showed that the evidence used in recommendation reports and those considered to be mandatory were very different, suggesting problems in decision-making processes. The authors conclude that there is a need to take a broader look at the factors that influence the type of evidence used in decision-making processes in order to contribute to the development of better practices and policies⁴².

Trust in health systems may also influence the efficiency of medicines appraisal and pharmaceutical policy⁴³. The uncertainties associated with the recent demands of the COVID-19 pandemic have added even more pressure on to health systems⁴⁴ and the rational use of medicines has been jeopardized such as in the rhetoric about the benefits of certain drugs exemplified by the case of hydroxychloroquine⁴⁵. It is a new scenario that can impact on decision-making in public policies and must be assessed in the future.

Acknowledgements

Thanks go to Dr P Brown for his contribution to both the scoping and ethnographic studies, to Dr F Hashem for her contribution to the ethnographic study, to Professor Leite and the reviewers for their constructive comments on an earlier draft. This text is based on a plenary lecture given at a workshop at the University of Santa Catarina, Florianópolis, Brazil in August, 2018.

References

• ¹
  Calnan M. Health Policy, Power and Politics: Sociological Insights. United Kingdom: Bingley Emerald; 2020.
• ²
  Graley C, May K and McCoy D, Postcode Lotteries in Public Health - The NHS Health Checks Programme in North West London. BMC 2011; 11:738.
• ³
  Harrison S, Checkland K. Evidence Based Practice in UK Health Policy. In: Gabe J, Calnan M. The New Sociology of the Health Service. London: Routledge; 2009. p. 99-121.
• ⁴
  Abraham J. The pharmaceutical industry, the state and the NHS. In: Gabe J, Calnan M. The New Sociology of the Health Service. London: Routledge; 2009. p. 99-120.
• ⁵
  Rawlins M, Barnett D, Stevens A. Pharmacoeconomics: NICE's approach to decision-making. BR J Clin Pharmacol 2010; 70(3):346-349.
• ⁶
  Collins B, Access to new medicines in the English NHS. London: Kings Fund; 2020. [cited 2020 Apr 29]. Available from: https://www.kingsfund.org.uk/publications/access-new-medicines-english-nhs
• ⁷
  Timmins N, Rawlins SM, Appelby J. A Terrible Beauty. A short history of NICE. Health Intervention and Technology Assessment Program (HITAP) [serial on the Internet]. 2016 [cited 2020 Mayo 12]. Available from: http://www.idsihealth.org/wp-content/uploads /2016/02/A-TERRIBLE-BEAUTY_resize.pdf
  » http://www.idsihealth.org/wp-content/uploads /2016/02/A-TERRIBLE-BEAUTY_resize.pdf
• ⁸
  Brown P, Calnan M. Political accountability of explicit rationing: legitimacy problems faced by NICE. J Health Serv Res Policy 2010; 15(2):1-2
• ⁹
  Crinson I. The politics of regulation within the 'Modernized' NHS: the case of beta interferon and the 'Cost-Effective' treatment of multiple sclerosis. Critical Social Policy 2014; 24(1):30-34.
• ¹⁰
  Daniels N, Sabin JE. Accountability for reasonableness: an update. Br Medical J 2008; 337: a1850.
• ¹¹
  SScottish Medicines Consortium (SMC). Patient and Doctor Engagement (PACE). Glasgow: SMC; 2016. [cited 2020 Apr 25]. Available from: https://www.scottishmedicines.org.uk
• ¹²
  Brown P, Hashem F, Calnan M. Trust, regulatory processes and NICE decision-making: Appraising cost-effective models through appraising people and systems. Social Studies of Science 2015; 46(1):87-111.
• ¹³
  Calnan M, Hashem F, Brown P. 'Still Elegantly Muddling Through? NICE anduncertaintyabout the Rationing of Expensive Medicines in England'. Int J Health Serv 2017; 47(3):571-594.
• ¹⁴
  Hashem F, Calnan M, Brown P. Decision-making in NICE single technological appraisals (STAs): How does NICE incorporate patient perspectives? Health Expectations 2018; 21(1):128-137.
• ¹⁵
  Moreira T. Health care rationing in an age of uncertainty: a conceptual model. Soc Sci Med 2011; 72(8):1333-1341.
• ¹⁶
  Calnan M. Clinical uncertainty: is it a problem in the doctor-patient relationship? Sociol Health Ill 1984; 1:74-85.
• ¹⁷
  Brown P, Calnan M. NICE technology appraisals: working with multiple levels of uncertainty and the potential for bias. Medicine, Health Care and Philosophy 2013; 16(2):281-293.
• ¹⁸
  Busfield J. Pills, Power, People: Sociological Understandings of the Pharmaceutical Industry. Sociology 2006; 40(2):297-314.
• ¹⁹
  Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med 2005; 2(5): e138.
• ²⁰
  Lindblom CE. Still muddling, not yet through. Public Administration Review 1979; 39(6):517-526.
• ²¹
  Britten N. Medicines and Society: Patients, Professionals and the Dominance of Pharmaceuticals. Basingstoke: Palgrave Macmillan; 2008.
• ²²
  Calnan M. Decisions of Value: Going Backstage. Int J Health Policy Manag 2018; 7(11):1067-1069.
• ²³
  Claxton K, Sculpher M Drummond M. A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet 2002; 360(9334):711-715.
• ²⁴
  Zinn J. Heading into the unknown: everyday strategies for managing risk and uncertainty. Health, Risk and Society 2008; 10(5):439-450.
• ²⁵
  Möllering G. The Nature of Trust: From Georg Simmel to a Theory of Expectation, Interpretation to and Suspension. Sociology 2001; 35:403-420.
• ²⁶
  Brown P, Calnan M. Braving a faceless new world? Conceptualising trust in the pharmaceutical industry and its products.Health: An Interdisciplinary Journal for the Social Study of Health, Illness and Medicine 2012; 16(1):57-75.
• ²⁷
  Mechanic D. Muddling through elegantly: finding the proper balance in rationing. Health Affairs 1997; 16:83-92.
• ²⁸
  Kleinhout-Vliek T, Bont A, Bert Boer B. Necessity under construction - societal weighing rationality in the appraisal of health care technologies. Health Econ Policy Law 2020; 16(4):457-472.
• ²⁹
  Cohen D. Cancer drugs: high price, uncertain value. BMJ 2017; 359:j4543.
• ³⁰
  Luzatto L, Hyry HI, Schieppati A, Costa E, Simoens S, Schaefer F, Roos JCP, Merlini G, Kääriäinen H, Garattini S, Hollak CE, Remuzzi G; Second Workshop on Orphan Drugs participants. Outrageous prices of orphan drugs: a call for collaboration. Lancet 2018; 392(10149):791-794.
• ³¹
  Kantarjian HM, Fojo T, Mathisen M, Zwelling LA. Cancer drugs in the United States: Justum Pretium--the just price [published correction appears in J Clin Oncol. 2015 Oct 20; 33(30):3523]. J Clin Oncol 2013; 31(28):3600-3604.
• ³²
  Lancet. Editorial, New 50 millionpound cancer fund already intellectually bankrupt. Lancet 2010; 376:389.
• ³³
  Haycox A. Why Cancer? PharmacoEconomics 2016; 34:625-627.
• ³⁴
  Grimm SE, Strong M, Brennan A,Wailoo AJ. The HTA Risk Analysis Chart: Visualising the Need for and Potential Value of Managed Entry Agreements in Health Technology Assessment. PharmacoEconomics 2017; 35(12):1287-1296.
• ³⁵
  Angelis A, Lange A, Kanavos P. Using health technology assessment to assess the value of new medicines: results of a systematic review and expert consultation across eight European countries. Eur J Health Econ 2018; 19:123-152.
• ³⁶
  Yuba T, Novaes H, Coelho de Soárez P. Challenges to decision-making processes in the national HTA agency in Brazil:operational procedures, evidence use andrecommendations. Health Policy Systems 2018; 16:40.
• ³⁷
  Fahy N, Hervey T, Greer S, Jarman H, Stuckler D, Galsworthy M, McKee M. How will Brexit affect health services in the UK? An updated evaluation. The Lancet 2019; 393(10174).
• ³⁸
  Calnan M, Douglass T. Chapter 16 - The evaluation of new medicines. In: Greve B. Handbook of the Social Policy Evaluation. Denmark: Edward Elgar; 2017. p. 300-322.
• ³⁹
  Balthussen R, Jansen MP, Mikkelsen E, Tromp N, Hontelez J, Bijlmakers L, Wilt GJV der. Priority setting for universal health coverage: we need evidence-informed deliberative processes, not just more evidence on cost-effectiveness. Int J Health Policy Manage 2016; 5(11):615-618.
• ⁴⁰
  Santana RS, Lupatini EO, Leite SN. The regulation and adoption of health technologies under Brazil's Unified Health System: barriers to access to medicines for diseases of poverty? Cien Saude Colet 2017; 22(5):1417-1428.
• ⁴¹
  Vargas-Pelaz C, Rover MRM, Soares L, Blat CR, ManteTeewisse AK, Rossi FQ, Restrepo LG, Latorre MC, Lpez JJ, Burgin MT, Silva C, Leite SN, Faria MR. Judicialization of access to medicines in four Latin American countries: a comparative qualitative analysis. Int J Eq Health 2019; 18:1-10.
• ⁴²
  Vargas-Pelaz CM, Rover MRM, Leite SN, Farias MR . Right to health, essential medicines, and lawsuits for access to medicines: a scoping study. Soc Sci Med 2014; 121:48-55.
• ⁴³
  Leite SN, Calnan M. Pharmaceutical policy as a right and as a service: reflections on population trust. Revista Brasileira de Farmácia Hospitalar e Serviços de Saúde 2019; 9:e093.001.
• ⁴⁴
  Calnan M. Health policy and controlling Covid-19 in England: sociological insights. Emerald Open Res 2020; 2:40.
• ⁴⁵
  Skipper CP, Pastick KA, Engen NW, Bangdiwala AS, Abassi M, Lofgren SM, Williams DA, Okafor EC, Pullen MF, Nicol MR, Nascene AA, Hullsiek KH, Cheng MP, Luke D, Lother SA, MacKenzie LJ, Drobot G, Kelly LE, Schwartz IS, Zarychanski R, McDonald EG, Lee TC, Rajasingham R, Boulware DR. Hydroxychloroquine in Nonhospitalized Adults With Early COVID-19 A Randomized Trial. Ann Internal Med 2020; 173(8):623-631.

Chief editors:

Romeu Gomes, Antônio Augusto Moura da Silva

Publication Dates

• Publication in this collection
  26 Nov 2021
• Date of issue
  Nov 2021

History

• Received
  15 May 2020
• Accepted
  09 Dec 2020
• Published
  11 Dec 2020