Services on Demand
- Cited by Google
- Similars in SciELO
- Similars in Google
Print version ISSN 0080-6234
Rev. esc. enferm. USP vol.45 no.1 São Paulo Mar. 2011
Validación de intervenciones de enfermería en personas con diabetes mellitus
Carla Regina de Souza TeixeiraI; Tânia Alves Canata BeckerII; Rachel CitroIII; Maria Lúcia ZanettiIV; Camila Aparecida Pinheiro LandimV
Professor, University of São Paulo at Ribeirão Preto, General
and Specialized Nursing Department. Ribeirão Preto, SP, Brazil. firstname.lastname@example.org
IIRN. Master Student, University of São Paulo at Ribeirão Preto, Fundamental Nursing Program. Ribeirão Preto, SP, Brazil. email@example.com
IIIRN. Master Student, University of São Paulo at Ribeirão Preto, Fundamental Nursing Program. Ribeirão Preto, SP, Brazil. firstname.lastname@example.org
IVRN. Associate Professor, University of São Paulo at Ribeirão Preto, General and Specialized Nursing Department. Ribeirão Preto, SP, Brazil. email@example.com
VRN. Doctoral Student, University of São Paulo at Ribeirão Preto, Fundamental Nursing Program. Ribeirão Preto, SP, Brazil. firstname.lastname@example.org
This descriptive, exploratory study aimed to validate nursing interventions proposed by the Nursing Interventions Classification for impaired skin integrity, deficient knowledge, ineffective therapeutic regimen management for predominant nursing diagnoses in people with diabetes. Participants were 21 specialist nurses in diabetes mellitus in Brazil, in 2007. Interventions were analyzed according to Fehring's evaluation model. The nursing interventions with the highest weighted average were care to injuries: closed drainage and circulatory precautions for the impaired skin integrity nursing diagnosis, teaching: disease process and teaching: prescribed medication for deficient knowledge and teaching: disease process and teaching: prescribed diet for ineffective therapeutic regimen management. Among the 1005 activities of nursing, 51% were validated as very characteristic by experts. Other studies should be carried out to expand the validation of the nursing interventions to people with diabetes mellitus in Brazil, searching for scientific evidences for care to these clients
Key words: Diabetes mellitus. Nursing diagnosis. Nursing care.
Estudio descriptivo exploratorio que objetivó validar las intervenciones de enfermería propuestas por la Nursing Interventions Classification para los diagnósticos de enfermería: Integridad de la piel perjudicada, Conocimiento deficiente y Control ineficaz del régimen terapéutico, predominantes en personas con diabetes. Participaron 21 enfermeros especialistas en diabetes mellitus en Brasil, en 2007. Las intervenciones de enfermería que obtuvieron la mayor media ponderada fueron cuidado con lesiones: drenaje cerrado, precauciones circulatorias para el diagnóstico de enfermería Integridad de la piel perjudicada, enseñanza: proceso de la enfermedad y enseñanza: medicación prescripta para el Conocimiento deficiente, y enseñanza: proceso de la enfermedad y enseñanza: dieta prescripta para Control ineficaz del régimen terapéutico. Entre las 1005 actividades de enfermería, el 51% fueron validadas como muy características por los especialistas. Se cree que deben ser efectuados otros estudios para ampliar la validación de las intervenciones de enfermería a personas con diabetes mellitus en Brasil, buscando evidencias científicas para el cuidado de dichos pacientes.
Descriptores: Diabetes mellitus. Diagnóstico de enfermería. Atención de enfermería.
Diabetes mellitus is currently considered an important health problem in terms of its prevalence, incidence and relationship to premature mortality, as well as due to the costs involved to control and treat its complications. Better understanding of the causes and reasons that lead to DM complications has enabled professionals to provide a more appropriate treatment and consequently lead to increased life expectancy and a higher rate of survival for people with diabetes mellitus(1).
Additionally, the complexity of the treatment of diabetes in a daily routine seeks to achieve normal levels of blood glucose in order to avoid acute complications and to have a satisfactory adaptation of lifestyle, which requires the multi-professional health team to be qualified in order to meet the nursing goal of implementing an integrated approach to all the body systems(2).
Therefore, the nursing process is a variation of the scientific rationale that helps nurses to organize, systematize and conceptualize nursing practice. Organization is a sequence of stages or components necessary to achieve this objective. The nursing process currently includes, in its most familiar form, five interrelated stages: data collection, nursing diagnosis, planning, implementation, and evaluation(3). Through this method, nurses are encouraged to continually assess care delivery, reviewing their practices and reflecting on the best way to implement them(4).
The nursing diagnosis is a result of critical thinking because it encompasses decisions about nursing situations, seeking results for which nurses are responsible. We stress the importance of accuracy in data collection as an essential step for establishing the nursing diagnosis(3). In Brazil, some authors identified various nursing diagnoses in people with diabetes mellitus(5-7); however, in this study, we opted for the following: Impaired skin integrity, Knowledge deficit, and Ineffective management of therapeutic regimen, in line with the majority of studies on diabetes mellitus.
With the expansion of nursing diagnoses and classification systems, the need to recover and also classify nursing interventions emerges in order to establish a channel of communication between the nurse and the other members of the health team or within the nursing team itself(8).
In this context, the Nursing Interventions Classification (NIC) is a taxonomy that includes actions recommended in nursing care. It is designed to improve clinical documentation, communication of care, integration of data in computer systems, use of data by nurses from various outpatient clinics and health facilities, data sources for research, indicators of productivity, evaluation of competence, payment for services, and curricular structure. The recommendation to implement studies using this taxonomy will contribute to the generation of new knowledge and hypotheses, contributing to the advancement of Brazilian nursing(9).
This study proposes to validate nursing interventions according to NIC for the following nursing diagnoses: Impaired skin integrity, Knowledge deficit, and Ineffective management of therapeutic regimen, which are predominant in people with diabetes mellitus.
This is a descriptive and exploratory study. The population was composed of 21 nurse-experts in diabetes mellitus in Brazil. Aiming to homogenize the sample, the following inclusion criteria were used: being a nurse and working with individuals with diabetes for at least three years and having clinical experience of at least one year, and reading and signing a consent form to participate in the study.
Two questionnaires were developed, one containing socio-demographic data (age, gender, profession, time of work in the institution) and data from the institution (whether it is a public or private facility, number of consultations delivered to people with diabetes per month, service flow), and another containing the scale of nursing interventions proposed by NIC for the diagnoses under study(9).
The second questionnaire had a five-point Likert scale. A numerical value was attributed to each variation of points: (0) not, (0.25) little, (0.5) somewhat, (0.75) very, and (1) very much characteristic. The number corresponds to the degree of importance of each action and the possibility of suggesting new actions and/or relevant observations(10).
Four experts with knowledge in diabetes mellitus assessed the instrument. For data collection, a letter of invitation containing the study's objectives and nature, the data collection instrument and a free and informed consent form, were sent by mail in sealed and stamped envelopes to facilitate participants returning the documents.
Data collection was carried out between August 2007 and June 2008 through three strategies. First, a letter was sent to 38 institutions registered in the Brazilian Society of Diabetes asking their authorization to carry out the survey. After various telephone contacts with the 38 institutions, 12 directors agreed to participate in the study but only six nurses returned the data collection instruments.
Secondly, authorization to collect data was obtained from a Program of Hypertension and Diabetes of the City Health Department; five nurses from primary health care and four from hospital care returned the instrument. Finally, given the difficulty of obtaining answered instruments, the Lattes Platform from the National Council for Scientific and Technological Development (CNpQ) was searched using the descriptors: nursing diagnosis and diabetes mellitus. A total of 28 experts were identified. Of these, six returned the instrument. The final sample was composed of 21 nurse-experts in diabetes.
Most (83%) of the 21 nurses were women, aged from 30 to 39 years (50%); 27% had attended postgraduate education programs and 27% specialization courses. Time since graduation of 10 years or more (66%) predominated and 55% of the nurses worked in public institutions, in outpatient clinics. The participants were not able to report the number of patients with diabetes cared for in the facilities for which they worked. In terms of offering education in DM, 67% reported they do not organize educational groups in DM.
For each answer option in the second data collection instrument, a weighted average was computed for each nursing activity. Then, the sum of weighted averages was divided by the number of nursing activities in each nursing intervention and the number of participants. Interventions with weight > 0.80 were considered essential nursing interventions. Nursing activities with weight between 0.50 and 0.79 were considered complementary.
Data were analyzed and processed through the Statistical Package for Social Sciences for Windows version 14.0. The project was approved by the Research Ethics Committee at the University of São Paulo at Ribeirão Paulo, College of Nursing (protocol nº 0607/2005).
Tables 1, 2 and 3 presents the distribution of weighted averages of answers of individuals for each nursing intervention of the nursing diagnoses: Impaired skin integrity, Knowledge deficit, and Ineffective management of therapeutic regimen.
According to NIC, 11 nursing interventions and 213 nursing activities are listed for the nursing diagnosis Impaired skin integrity. Two interventions obtained the highest weighted averages and were considered essential: wound care: closed drainage and circulatory precautions, and three interventions that obtained weighted averages between 0.50 and 0.79 were considered complementary: amputation care, exercise promotion and medication management.
A total of 12 nursing interventions and 307 nursing activities are listed for the nursing diagnosis Knowledge deficit. The interventions that obtained the highest weighted averages and thus were considered essential were: teaching: disease process and teaching: prescribed medication. The three complementary interventions, that is, those that obtained weighted averages between 0.50 and 0.79 were: health system guidance, health education, and patient rights protection.
Finally, 20 nursing interventions and 485 nursing activities are listed under the nursing diagnosis Ineffective management of therapeutic regimen. The two interventions that obtained the highest weighted averages were: teaching: disease process and teaching: prescribed diet. Those that obtained weighted averages between 0.50 and 0.79 were the following complementary interventions: financial resource assistance, risk identification and health system guidance.
Hence, 1,005 nursing activities were analyzed by the 21 individuals who participated in the study, totaling 21,105 answers. Of these answers, 10,763 (51%) were considered as very much characteristic, 7,598 (36%) as very characteristic, 1,900 (9%) somewhat characteristic, 633 (3%) as little characteristic, and 211 (1%) not characteristic. No nursing activities were added by the participants in the three analyzed nursing diagnoses.
The Nursing Classification Interventions (NIC) is more recent than the NANDA-I International System of Nursing Diagnosis. Therefore, the methods used to validate nursing diagnoses have also been used to validate nursing interventions.
One study proposed a model to validate nursing diagnoses called Differential Diagnostic Validation (DDV), in addition to having significantly changed the two existent models(10) Diagnostic Content Validation (DCV) and Clinical Diagnostic Validation (CDV)(11).
In the Diagnostic Content Validation (DCV) model, the characteristics with averages equal to or higher than 0.80 are considered leading provisory indicators and those with an average between 0.50 and 0.79 are considered secondary provisory indicators. Finally, the total DCV score is obtained summing up the individual scores and dividing them by the total number of the diagnosis defining characteristics, discarding those characteristics with averages equal to or below 0.50(10-11).
The nursing activities with averages equal to or higher than 0.80 were considered in this study as essential or relevant interventions for the studied diagnoses and therefore should be implemented for people with diabetes; interventions with averages between 0.50 and 0.79 were considered complementary. Those with averages equal to or below 0.50 were considered not essential for the studied diagnoses. This study revealed that 51% of the nurses considered most of the interventions essential for caring for individuals with diabetes mellitus.
We stress that the individual needs of each patient should be taken into account in nursing care and that nursing interventions can be implemented according to the nurses' assessment. Hence, it is expected that this study's results will aid in the analysis of activities performed in DM care.
Two essential interventions were identified for the nursing diagnosis Impaired skin integrity, wound care: closed drainage and circulatory precautions.
A study was carried out to identify the characteristics of ulcers in people with peripheral vasculopathy with the diagnosis Impaired skin integrity related to the interruption of arterial and/or venous blood flow, hospitalized in the vascular ward of the Dante Pazzanese Institute of Cardiology of São Paulo, Brazil. This study found that ulcers present particular and distinct characteristics because they are either arterial, venous, diabetic or hypertensive, hence, nurses should recognize them to reach an accurate diagnosis and adopt an appropriate regimen for healing the wounds(12). This study raises the importance of the nursing intervention circulatory precautions for individuals with diabetes, because these are individuals who need rapid diagnosis and appropriate treatment so that chronic complications, occasionally severe or even fatal, are avoided.
Another study investigated 403 individuals with diabetes (31% with type 1 diabetes mellitus and 69% type 2 diabetes mellitus) cared for in the outpatient facility of a university hospital and found that detection of skin wounds was high among people with diabetes. The results show that 81% of the studied individuals presented various types of dermatoses, and 1,198 skin disorders were found corresponding to three to four skin manifestations per individual. The study concluded that careful skin assessment should be associated with outpatient follow-up of people with diabetes in order to provide appropriate treatment of the diagnosed dermatoses(13). Hence, we emphasize that the actions of the health team should be integrated and maintain multi-professional consensus while skin assessment of people with diabetes should be carefully carried out by nurses.
The complementary intervention amputation care was also found for the nursing diagnosis Impaired skin integrity, in which people with diabetes present a 15 times increased risk of being subject to amputation of lower limbs than those who do not have diabetes(14); due to Diabetic Foot, one of the most severe and prevalent chronic complications of diabetes mellitus.
Therefore, nurses need to inform people with diabetes that amputation goes beyond the biological dimension. Nurses, as members of the multi-disciplinary team, have a special role, which is to educate people. Patient understanding the amputation process is essential for health professionals to perform their activities more effectively, which enables care to be focused on the patients' individuality and particularities concerning their lives' experiences(15).
A descriptive study carried out in two primary health care units in Ribeirão Preto, SP, Brazil characterized 101 people with type 2 diabetes mellitus to identify factors that triggered the condition of diabetic foot. In the feet assessment, 73% presented poor hygiene and inappropriate nail cutting, 63% had dry and scaly skin, 52% altered nails, 49% calluses and/or cracks, and 33% flat feet. The risk classification identified that 7.9% presented plantar insensitivity and 6.9% presented plantar insensitivity and foot deformity. The study concluded that such disorders could be prevented through basic education and interventions for foot care, in which external factors identified in this study could be resolved and monitored with actions of low complexity(16).
Nurses should take the responsibility to track and monitor risk factors for Diabetic Foot during the care process and instruct people with DM concerning the risks and, with the involvement of the entire health team, determine the basic interventions and plan educational activities for self-care and maintenance of a good metabolic control in order to avoid severe complications(16). Hence, this intervention is closely related to the nursing intervention of the diagnosis Ineffective management of therapeutic regimen, considered complementary for identification of risk.
An essential step in identifying risk factors for Diabetic Foot is podiatric assessment, which consists of inspecting the skin, its structural and circulatory aspects, and tactile pressure sensitivity, in addition to hygienic conditions and characteristics of shoes. These actions, when performed by professionals at the primary care health level, contribute to reducing the risk of morbidities and complications in the feet of people with diabetics(17).
In relation to interventions for the nursing diagnoses Knowledge deficit and Ineffective management of therapeutic regimen, the following essential interventions were identified: teaching: disease process, teaching: prescribed diet and teaching: prescribed medication focused on providing guidance concerning the disease, medication and diet. Hence, there is a predominance of interventions in the literature related to dietary planning, physical exercise and the appropriate use of medication(18).
We stress that the nursing intervention teaching: disease process was identified as essential for both nursing diagnoses previously reported. It acknowledges education as a fundamental strategy in DM treatment, aiming to improve the knowledge of people with diabetes and to enable them to intervene, themselves, in order to achieve the desired metabolic control. Therefore, the metabolic control of individuals with diabetes is obtained through education, glucose monitoring, nutritional therapy, regular exercise, pharmacological therapeutic schemes, information on the prevention and treatment of chronic and acute complications, and the reinforcement of treatment objectives(18).
Due to the chronic nature of diabetes and the importance of glycemic control to preventing complications and subsequent problems, knowledge acquired through education is a relevant aspect in the treatment(18).
A study developed in an educational center in the interior of São Paulo, Brazil to verify the implementation of a program to care for individuals with diabetes used the Staged Diabetes Management (SDM) protocol and to characterize the activities developed by the multi-professional health team verified that the use of the protocol contributed to improve the metabolic control of people with diabetes, which consequently improved medication coverage for diabetes mellitus, hypertension, and knowledge concerning the disease and also reduced the signs and symptoms reported by people with DM(19).
Relating the essential nursing intervention teaching: prescribed diet with the results reported earlier, the study revealed that the objective of nutritional care for people with diabetes is to evaluate the nutritional state and define needs, establishing attainable goals in the diet plan and metabolic control and also adjust eating based on the response and metabolic control of the individual with diabetes(19).
Another study conducted in the interior of São Paulo evaluated the impact of an educational program that showed that people improved their knowledge concerning their disease with a significant increase regarding general subjects of diabetes mellitus such as: concept, physiopathology and treatment; other subjects that stood out were exercise and diet(20).
It is noteworthy that 67% of the individuals in this study do not organize educational groups in the studied facility. In contrast, the literature indicates that group intervention is a valuable strategy for achieving educational objectives, promoting greater interaction and group cohesion(21).
Finally, for the nursing diagnoses Knowledge deficit, and Ineffective management of therapeutic regimen, complementary interventions were identified in relation to health system guidance, patient rights protection, and financial resources assistance, while the nursing intervention health system guidance was found in both nursing diagnoses previously reported.
There is a science currently used as a field of action in health, in which human beings are considered subjects, not only objects. This science is known as bioethics, ethics applied to life, and it is presented as a search for responsible behavior of people who should decide upon types of treatment, research or positions in relation to humanity(22).
Bioethics considers autonomy one of its principles or references for the analysis of dilemmas and/or value conflicts that emerge in human relationships. An action is considered autonomous only when it is subject to an individual's informed consent. In health, integrality of care is damaged when the person who seeks the health care service is not considered a subject, a holder of rights and life options(22).
Nursing, as part of the health system, should ensure that people who seek out the health system have a guarantee of care under its responsibility in a equitable way, since everyone has the right to information, regardless of his/her condition(22).
Health professionals should have the competence to enable people who seek health services to know increasingly more about their bodies, diseases and relationship with their social environment, enabling people to play an active role in the face of their problems. This would imply the development of a more effective health care and more successful treatment(22).
This study revealed that most of the nursing interventions for the nursing diagnoses Impaired skin integrity, Knowledge deficit and Ineffective management of therapeutic regimen were considered essential by 51% of the studied nurses. Thus, they should be implemented for individuals with diabetes mellitus.
Two nursing interventions obtained the highest weighted averages and were considered essential for the diagnosis Impaired skin integrity: wound care: closed drainage and circulatory precautions; three interventions obtained weighted averages between 0.50 and 0.79 and thus were considered complementary: amputation care, exercise promotion, and medication management.
In relation to the nursing diagnosis Knowledge deficit, the interventions that obtained the highest averages and thus were considered essential were: teaching: disease process and teaching: prescribed medication. The tree complementary interventions, that is, those whose weighted averages were between 0.50 and 0.79, were: health system guidance, health education, and patient rights protection.
Finally, for the nursing diagnosis Ineffective management of therapeutic regimen the two essential interventions were: teaching: disease process and teaching: prescribed diet and the complementary interventions: financial resource assistance, risk identification, and health system guidance.
The nursing interventions evidenced by this study indicate the need for health education. The professional practice of nurses included in the multi-professional team, when centered on the scope of education to enable people with diabetes mellitus to efficaciously practice self-care, can minimize the onset of chronic complications.
Some difficulties such as: length of the data collection instrument, identification of nurse-experts in diabetes mellitus coupled with the need to contact participants through mail, by telephone or e-mail, and also the delayed return of instruments, represented limitations to this study.
We consider the fact that for one to deliver nursing care to people with diabetes mellitus, the activities included in the interventions of the three studied nursing diagnoses are interrelated and need further studies to identify them in clinical practice, verifying their applicability, especially in the nurses' educational actions.
We believe that other studies should be conducted to expand the validation of nursing interventions to people with diabetes mellitus in Brazil in the search of scientific evidence to provide integral care to this clientele.
1. Franco LJ, Rocha JSY. O aumento das hospitalizações por diabetes mellitus na região de Ribeirão Preto, no período 1988-97. Diabetes Clínica. 2002;6(2):108. [ Links ]
2. Mancini MC. Obstáculos diagnósticos e desafios terapêuticos no paciente obeso. Arq Bras Endocrinol Metabol. 2001;45 (6):584-608. [ Links ]
3. Alfaro-Lefevre R. Aplicação do processo de enfermagem: um guia passo a passo. 4ª ed. Porto Alegre: Artes Médicas Sul; 2000. [ Links ]
4. Pellison F, Nagumo MM, Cunha ES, Melo LL. Aplicação prática do processo de enfermagem a uma adolescente portadora de doença crônica. Rev Esc Enferm USP. 2007;41(3):513-7. [ Links ]
5. Doenges ME, Moorhouse MF. Application of nursing process and nursing diagnosis: an interactive text for diagnostic reasoning. 5th ed. Philadelphia: Davis; 2003. [ Links ]
6. Becker TAC, Teixeira CRS, Zanetti ML. Diagnósticos de enfermagem em pacientes diabéticos em uso de insulina. Rev Bras Enferm. 2008;61(6):847-52. [ Links ]
7. Volpato MP, Cruz DALM. Diagnósticos de enfermagem de pacientes internadas em unidade médico-cirúrgica. Acta Paul Enferm. 2007;20(2):119-24. [ Links ]
8. Barreto PL, Oliveira FM, Silva RCP. Utilização do processo de enfermagem em diabéticos nas Unidades de Saúde de Coronel Fabriciano, Minas Gerais. RBPS Rev Bras Promoção Saúde. 2007;20(1): 53-9. [ Links ]
9. Mccloskey JC, Bulechek GM. Classificação das Intervenções de Enfermagem (NIC). 4ª ed. Porto Alegre: Artes Médicas; 2008. [ Links ]
10. Fehring RJ. Validating diagnostic labels: standardized methodology. In: Hurley ME. Classification of nursing diagnosis: proceedings of the sixth conference. St. Louis: Mosby; 1986. p. 183-90. [ Links ]
11. Creason NS. Clinical validation of nursing diagnoses. Int J Nurs Terminol Classif. 2004;15(4):123-32. [ Links ]
12. Bersusa AAS, Lages JS. Integridade da pele prejudicada: identificando e diferenciando uma úlcera arterial e uma venosa. Ciênc Cuidado Saúde. 2004;3(1):81-92. [ Links ]
13. Foss NT, Polon DP, Takada MH, Foss-Freitas MC, Foss MC. Dermatoses em pacientes com diabetes mellitus. Rev Saúde Pública. 2005;39(4):677-82. [ Links ]
14. Most RS, Sinnock P. The epidemiology of lower extremity amputation in diabetic individuals. Diabetes Care. 1983;6 (1):87-91. [ Links ]
15. Chini GCO, Boemer MR. A amputação na percepção de quem a vivencia: um estudo sob a ótica fenomenológica. Rev Lat Am Enferm. 2007;15(2):330-6. [ Links ]
16. Ochoa-Vigo K, Torquato MTCG, Silvério IAS, Queiroz FA, De La Torre Ugarte-Guanilo MC, Pace AE. Caracterização de pessoas com diabetes em unidades de atenção primária e secundária em relação a fatores desencadeantes do pé diabético. Acta Paul Enferm. 2006;19(3):296-303. [ Links ]
17. Milman MHSA, Leme CBM, Borelli DT, Kater FR, Baccili ECDC, Rocha RCM, et al. Pé diabético: avaliação da evolução e custo hospitalar de pacientes internados no Conjunto Hospitalar de Sorocaba. Arq Bras Endocrinol Metabol. 2001;45(5):447-51. [ Links ]
18. American Diabetes Association. Recomendações para os portadores de diabetes mellitus. Diabetes Clínica. 2001;5(4):258-62. [ Links ]
19. Zanetti ML, Otero LM, Freitas MCF, Santos MA, Guimarães FPM, Couri CEB, et al. Atendimento ao paciente diabético utilizando o protocolo Staged Diabetes Management: relato de experiência. RBPS Rev Bras Promoção Saúde. 2006;19 (4):253-60. [ Links ]
20. Otero LM, Zanetti ML, Ogrizio MD. Knowledge of diabetic patients about their disease before and after implementing a diabetes education program. Rev Lat Am Enferm. 2008; 16 (2):231-7. [ Links ]
21. Santos MA, Péres DS, Zanetti ML, Otero LM. Grupo operativo como estratégia para a atenção integral ao diabético. Rev Enferm UERJ. 2007;15(2):242-7. [ Links ]
22. Zoboli ELCP, Fortes PAC. Bioética e atenção básica: um perfil dos problemas éticos vividos por enfermeiros e médicos do Programa Saúde da Família, São Paulo, Brasil. Cad Saúde Pública. 2004;20(6):1690-9. [ Links ]