Content valid ation of quality indicators for nursing care evaluation

Abstracts

O objetivo do estudo é submeter dez Indicadores de Qualidade do Cuidado de Enfermagem na Prevenção de Eventos Adversos à validação de conteúdo. Participaram nove experts, que responderam a três formulários. Os resultados apontaram a validade dos indicadores, porém com reformulações. Resultaram do processo doze indicadores: Identificação do leito do paciente; Identificação de risco para queda do leito; Identificação de acessos venosos periféricos; Verificação de lesões cutâneas pós-infiltrativas; Identificação de equipos para infusão venosa; Identificação de frascos de soro e controle da velocidade de infusão; Identificação de sondas gástricas; Fixação da sonda vesical de demora e posicionamento da bolsa coletora de diurese; Checagem dos procedimentos na prescrição de enfermagem; Controle de sinais vitais; Checagem dos procedimentos de enfermagem na prescrição médica e Elaboração da prescrição diária e completa pelo enfermeiro. A partir dos resultados acredita-se no procedimento de validação de conteúdo como imprescindível para o desenvolvimento de medidas avaliativas.

Cuidados de enfermagem; Garantia da qualidade dos cuidados de saúde; Indicadores de qualidade em assistência à saúde; Estudos de validação

ORIGINAL ARTICLE

Content validation of quality indicators for nursing care evaluation^*

Validación de contenido de indicadores de calidad para evaluación del cuidado de enfermería

Dagmar Willamowius Vituri^I; Laura Misue Matsuda^II

^IMaster in Nursing. Nurse in the Control and Quality Supervision for Nursing Service at Hospital Universitário Regional do Norte do Paraná. Londrina, PR, Brazil. [e-mail]

^IINurse. Doctor Professor of the Nursing Department at Maringá State University. Maringá, PR, Brazil. [e-mail]

^{Correspondence addressed to}

ABSTRACT

The objective of this study is to perform the content validation of ten Nursing Care Quality Indicators in Adverse Event Prevention. Nine experts took part in this study and answered three forms. The results appointed the validity of the indicators, but with reformulations. The process yielded twelve indicators: identification of the patient's bed; identification of the risk of falling out of bed; Identification of peripheral venous accesses; Verification of skin lesions after infiltrations; Identification of venous infusion equipment; Identification of serum bottles and infusion speed control; Identification of peptic probes; Fixation of vesical delay probe and positioning of the urine collection bag; Checking of Procedures in Nursing Prescription; Control of Vital Signs; Checking of Nursing Procedures in Medical Prescription and Nursing elaboration of daily and complete prescription. The results confirm that the content validation procedure is indispensable for the development of evaluative measures.

Key words: Nursing care. Quality assurance, health care. Quality indicators, health care. Validation studies.

RESUMEN

El objetivo del estudio es someter a validación de contenido a diez Indicadores de Calidad del Cuidado de Enfermería en la Prevención de Eventos Adversos. Participaron nueve experts, que respondieron a tres formularios. Los resultados apuntaron la validad de los indicadores, sin embargo con reformulaciones. Resultaron del proceso doce indicadores: Identificación de la cama del paciente; Identificación de riesgo para caídas de la cama; Identificación de accesos venosos periféricos; Verificación de lesiones cutáneas después de infiltraciones; Identificación de equipos para infusión venosa; Identificación de frascos de suero y control de la velocidad de infusión; Identificación de sondas gástricas; Fijación de la sonda vesical de demora y posición de la bolsa colectora de diuresis; Verificación de los procedimientos en la prescripción de enfermería; Control de señales vitales; Verificación de los procedimientos de enfermería en la prescripción médica y Elaboración de la prescripción diaria y completa por el enfermero. A partir de los resultados se piensa en el procedimiento de validación de contenido como imprescindible para el desarrollo de medidas de evaluación.

Descriptores: Atención de enfermería. Garantía de la calidad de atención de salud. Indicadores de calidad de la atención de salud. Estudios de validación.

INTRODUCTION

In the context of our globalized world, where science, technology, and information are within everyone's reach, occupations, and more specifically nursing, face the need to improve their knowledge in order to guarantee a satisfactory level of quality of care.

Health quality has become a compulsory feature and it is modernity's trademark; however, the systematization of the whole set of practices and processes must take place so that quality is achieved⁽¹⁾.

The Ministry of Health defines quality as the degree of compliance with established standards of norms and protocols that organize actions and practices, as well as all current scientific technical knowledge⁽²⁾.

The path toward care quality improvement is part of the daily routine of healthcare professionals and it is a legal demand in several countries. Hence, care quality control, grounded in indicators that according to literature can be used as health assessment tools⁽³⁾, becomes a must.

When incorporated as managerial dispositives, and thus routinely applied as measurement instruments, quality indicators allow nurses to self-govern workers; this action represents an empowerment in their administrative practices⁽⁴⁾.

One of the examples of the employment of indicators in the search for quality improvement, and a subsequent humanization process, was described by a study on professional satisfaction in nursing. The application of the professional satisfaction indicator allowed authors to conclude that, throughout the years, countless nurses have offered resistance to labor innovation and also do not act based in principles, indicators, and quality standards as a way of complying with personal and professional demands. Undoubtedly, this behavior renders any planned and systematized nursing action impossible, and can result in far-reaching damage to clients, their family members, other professionals, the health institution, and so on and so forth⁽⁵⁾.

An indicator can be defined as a measurement unit for an activity; however, it is not a direct measure of quality. Indicators signal discrepancies toward the determined desired standard, and act as an alert that identifies and draws attention to the necessary review of key care elements⁽⁶⁾.

In accordance with, and reinforcing the above-mentioned principles related to the determination of an indicator as a measurement, authors define it as a means to certify, estimate, value, control, and self-regulate results of a given process⁽⁷⁾.

The application of quality indicators indispensably demands that the employed measurements are secure, because when changes are not effectively captured and monitored, the supposedly positive incentive can become a wicked incentive, thus generating discouragement, malfunction, and crisis⁽⁸⁾.

As measurements, and consequently indicators, have the characteristic of describing phenomena so that they can be statistically analyzed, it is vital that this set of information to be validated (that is, the measurement variables) represent the phenomenon of interest⁽⁹⁾.

The determination of the true validity of measurements is reached by means of the empirical research, grounded in a systematic exam of conceptual abstractions and performed through a process of observation and estimation of responses, aimed at identifying and explaining a phenomenon of interest⁽¹⁰⁾.

Validity is deemed by scholars on the subject to be a crucial factor in the choice for and/or application of a measurement or measurement instrument. There is a mutual consensus among authors that validity is calculated by the extension or degree in which the measurement, or the datum, is a representation of the concept the instrument is set to estimate; that is, the capacity it has of capturing or revealing any given phenomenon⁽¹¹⁾.

One of the measures of validity, content validity, which is the object of this study, is understood as being the determination of representativity and extension with which each measurement item appropriately confirms the domain of interest and the dimension of each item, within the scope of what was set to be measured in a determined investigated phenomenon⁽¹²⁾.

The content validation strategy will be applicable only when the universe of behaviors to be measured⁽¹³⁾ is clearly delimited.

In order to assign any content validity test, it must constitute a representative sample of a finite universe of behaviors – the domain⁽¹³⁾.

The content validation instrument proposed in this investigation is comprised of quality indicators grounded in the conception of adverse events. The referred instrument is based on some assessment issues that were extracted from a study developed in 1999⁽¹⁴⁾ in the same institution.

An adverse event is defined as a non-intentional lesion resulting in temporary or permanent incapacity, lengthening of hospital stay, or even death resulting from the rendered care⁽¹⁵⁾.

Bearing in mind the need to safeguard quality care processes and consequently to free patients of any eventual risk, the following question arises: does a given instrument made of nursing basic care quality indicators aimed at the prevention of adverse event have content and applicability validity in such a way that it becomes a rational and objective managerial tool to assess the nursing quality care offered to adult patients admitted into a medical-surgical unit of a university hospital in the Northern region of the State of Paraná?

It is worth remembering that the present research is part of the Masters Degree thesis titled Development and validation of nursing care quality assessment instruments; the work was developed in order to optimize nursing care quality assessment processes, as a way to contribute to the qualification of practices aimed at the prevention of adverse events.

From the results achieved by the application of the proposed instrument and validated by the content validation strategy, it is definitely possible to perfect managerial profiles to such an extent that a continuous nursing care quality improvement is consolidated, since results obtained with the employment of a validated instrument can serve as quality drivers, which allow for elaboration and rearrangement of goals, aiming at the highest quality possible.

The objective of this investigation is to validate the content of a nursing care assessment instrument comprised of Nursing Care Quality Indicators for the Prevention of Adverse Events.

METHODOLOGICAL REFERENCE

The content validation methodological procedures begin with the development of a clinical performance measurement and the construction of the instrument, which must be implemented in four steps⁽¹¹⁾:

1. Choose the care aspect to be submitted to assessment, based on three criteria: the importance of the care activity to be measured; the improvement potential it presents; and the degree of control that professional care executives have on the mechanisms that will perform the desired improvement. From this point on, the instrument's items are built, and they must express the behavioral representation of what is being measured; that is, the tasks people will execute and that will undergo assessment.

2. Select performance indicators within each area based on the certification of the strength of the scientific evidence. This step foresees the determination of a name for the indicator, and this term must describe the major care activity or event to be analyzed⁽¹⁶⁾.

3. Build a trustworthy and valid measurement grounded on the determination of the concept to be measured, the target population, the need of additional data collection to explain the indicator's variations, the data source and the details on the collection process, and finally the determination of the measurement scoring.

4. Test the scientific strength of the measure so that it is possible to confirm how clear and pertinent the instrument's items are. In this sense, built items must be assessed in their relation to the opinion of judges or experts, who are not representative samples of the population for which the instrument was built⁽¹³⁾.

As soon as the above-mentioned steps are completed, two methods for the analysis of data originated from the validation strategy are recommended; in the present study they are listed in the 5^th step and are described below⁽¹²⁾.

5. Analysis of data achieved by the content validation strategy:

5.1. Reliability index or interrater agreement (IRA): it assesses the agreement of experts concerning the representativity and clearness of the studied content. A more conservative approach considers only the items in which judges reached 100% agreement. A less conservative approach considers the items that reached at least 80% agreement.

5.2. Content Validity Index (CVI): it assesses the agreement of experts concerning the representativity of the measure in relation to the studied content. This method considers as valid all items and the instrument as a whole, whenever they achieve a CVI of 0.80 % ⁽¹²⁾.

METHOD

This is a methodological, applied, qualitative-based research, developed between November 2006 and June 2007 in a medical-surgical unit of a university hospital for adult males, containing 74 beds, and located in Northern Paraná.

The study's first phase took place in November 2007 and was comprised of the selection of care aspects to be measured, focusing on the problems indicated by the methodology applied by the institution since 1999⁽¹⁴⁾. The determination of care aspects to be measured was founded on the recommendation of high frequency problems that affect a great number of patients, thus establishing a risk situation⁽⁶⁾. Problems related to ethical and legal aspects of care documentation were also taken into account, as well as those problems liable to being minimized or solved with permanent and continuous education measures^(6,14).

In the next phase, the researcher developed 10 quality indicators comprised of 32 certification items, based on 15 investigative questions extracted from the methodology applied in the institution⁽¹⁴⁾; the questions carefully made to comply with all determined steps of the adopted reference points^(11,14,16).

After the development of indicators and certification items, and grounded in the need to build a trustworthy and valid measure⁽¹¹⁾, an Operation Manual for each one of the proposed indicators was elaborated⁽¹⁷⁾. One example can be observed in ^{Annex 1}, which elucidates indicator number 3.

For each of the proposed indicators – namely descriptors - the Operation Manual will consider: the scientific reference that was the foundation of the descriptor's determined standard; the indicator's typology specification (process or result); the numerator and the denominator, including their calculation; the data source; the criteria for the assessment qualification; the sample to undergo conformity analysis; assessment periodicity; and the indicator's assessment items recorded in the proposed instrument. In addition to this information, the Manual also presents the Ideal Conformity Index (ICI); that is, the expected adjustment percentage regarding the determined standard – or measure scoring⁽¹⁷⁾.

As a way of allowing experts to evaluate both indicators and certification items, three other assessment instruments were developed, grounded in the study⁽⁹⁾: the first was aimed at assessing the Operational Manual content; the second at assessing each one of the 32 certification items; and the last, the set of certification items that constitute each one of the ten indicators. Experts applied assessment criteria adjusted from an indicator validation study performed to assess the practices related to infection prevention processes associated with vesical catheters (inserted into ^{Annex 2}).

The objective of the instrument assessment was to verify to what degree the representativity and the extension of each measure item corroborates the studied domain, as well as the dimension of each item, aiming at measuring the nursing care quality for the prevention of adverse events.

Following the necessary methodological procedures for the content validation of the proposed instrument, the scientific strength of the measure, or the so-called item theoretical analysis, was tested⁽¹³⁾.

In this phase, the sample was composed of researching professors who developed work in administrative nursing, quality assurance and methodological research, selected from a search in the Lattes Platform on the National Council of Scientific and Technologic Development (CNPq) website, resulting in an intentional sample of 12 nursing doctors from several regions of Brazil. Of these, nine consented to participate in the study.

Following the experts' confirmation and considering the Delphi technique recommendation, a group communication framework method for the analysis of complex problems⁽¹⁸⁾, the material to be validated was individually forwarded by mail (courier) to each judge. They were instructed to forward the received papers containing their opinion the material by means of a sealed envelope. This process was carried out from January to April 2007.

Following the experts' analysis and the return of the material, responses were manually tabbed and all comments and suggestions were organized in tables for analysis.

Assessment items, indicators, and consequently the instrument were reformatted in accordance with the experts' opinions. Following a pilot study of the reformulated instrument was carried out in order to certify its applicability.

Selection of subjects for the pilot study took place using the systematic sampling method, which delimited the drawing of five students taking the 6^th period of the graduate nursing course at the local State University, and who are trainees in the Advising Service of Nursing Assistance Quality Control of the researched hospital.

The pilot study was carried out by means of the application of the reformatted instrument in five nursing wards containing six beds each, in the same unit where the remainder of the investigation was developed, under the supervision of the researcher.

The research project was submitted for the approval to the Ethics Committee of the State University of Londrina and approved on 18 October 2006 under the Legal Opinion number 246/06.

Microsoft^® Excel^®2002 program was used to tabulate the data. Now, the results will be presented by a descriptive statistic oriented program in the format of tables comprised of percentage numbers.

RESULTS

The experts' opinion included, besides the analysis and judgment of the indicators proposed by the Operational Manual, the content of each assessment item and the attributes of the contents of nursing care quality indicator assessment items for the prevention of adverse events.

The content validation procedure allowed for the capture of the experts' opinions concerning the studied domain, and subsidized the reformulation of proposed items and indicators, as well as the application of statistical calculations to determine both the Content Validity Index (CVI) and the Reliability Test (IRA) shown in Tables 1 and 2 below.

The values shown in Table 1 indicate the percentage of experts who judged the set of items that composed each indicator as valid measure representatives concerning the studied content. All observed items scored higher than the CVI minimum standard of 80% determined by the adopted literature; therefore, it is possible to affirm that all items were considered as valid in relation to their content.

The achieved values displayed in Table 2 consider the Reliability Analysis, or Interrater Agreement.

When CVI values in Table 1 are compared with IRA percentage in Table 2 it is possible to conclude that the proposed indicators, under assessment measure conditions, are representative of the Nursing Assistance Quality construction; however, seven indicators presented serious clearness and representativity problems regarding this same construction.

Data from Table 2 show that indicators numbered 1, 9, and 10 had 100% reliability percentages, that is, they are clear and representative of the studied content.

Indicator 8 achieved 80% reliability, which corroborates experts' comments. They pointed out that for the items related to nursing procedures and missed medications it was not clear if justifications for the non-performance should be reported in the space indicated by nursing notes on the corresponding prescriptions.

As for indicators 3, 4, and 6, the reliability percentage reached 50%. The problems displayed by indicator 3 are related to the presence of two variables in the item that assessed the issue of absence of identification and improper identification of peripheral venous accesses, and also the unnecessary presence of the item in the assessment of venous access through intracath, phlebotomy, and Central Catheter of Peripheral Insertion (CCIP); this item did not present any relation to the indicator's objective.

Indicator 4 showed a 50% reliability percentage and confirmed problems concerning the lack of the descriptor's patient indicator determination, those who less than 24 hours prior to the assessment had had venous infusion, but who did not have venous access at the time of the active search. These patients presented risk of post-infiltration skin lesions, and in this case they were undergoing assessment. Hence, the descriptor should be reformulated.

Indicator 6 displayed a 50% reliability percentage, and indicated a similar problem to indicator 3 regarding the presence of two variables in only one assessment item, those being the lack of identification and the improper identification of fluid flasks.

Indicator 5 received a 33% reliability percentage due to the same problems identified in indicators 3 and 6, that is, the assessment of two variables in the same certification item, lack of identification and improper identification of venous infusion equipment, and also due to the need to readjust the item description, which should replace the expression dated but expired for adequate but expired identification.

Indicator 2 was not considered reliable concerning clearness and representativity of the studied content (0%) due to problems with the descriptor, which was not clear on how the risk of bed collapse would be determined, whether by examining the records or the patient's physical status.

Regarding indicator 7 (0%), the problem rested on the fact that it suggested the assessment of two distinct variables (gastric tubes and intravenous lines), to which prevention procedures for adverse events are distinct.

The opinion of one of the experts suggested that indicator 7 - Identification of tubes - should be subdivided. As a result, two other indicators were developed: identification of gastric tubes, which kept the descriptor's characteristics and assessment criteria; and the indicator of delayed intravenous line fixation and positioning of the urine-collection bag, to which a subsidy from a study developed in 2005⁽⁹⁾ to validate assessment indicators of practices related to the prevention of catheter-associated urinary infection was applied. In this case, the assessment criteria had already been validated.

Data analysis of Table 2 makes possible the identification of a series of problems related to clearness and representativity of assessment items of the majority of proposed indicators concerning the studied content, thus confirming all comments and suggestions presented by the experts' opinions. This was made possible by the option of applying the IRA assessment - the conservative approach presented in the theoretical reference - that is, the division of the amount of items that presented 100% experts' agreement by the total amount of items.

In case the less conservative approach (80%) was taken into account – and it is indicated when the number of judges exceeds five, such as in this study - results of the IRA calculations would not confirm the problems indicated by the judges' opinion. This is explained by the fact that the problems pointed out by the experts when framing the indicators did not render them unfeasible assessment measures; however, there is still a need for further review in order to improve the selected variables toward the nursing quality assurance aimed at the prevention of adverse events.

All comments and suggestions received from the experts were taken into consideration, and the items and indicators were reformatted in order to comply with the judges' guidelines.

After the reformulation of the instrument, the pilot study was carried out. As there were no doubts left on the understanding of items and application of the instrument, it was deemed as valid concerning content and applicability toward the assessment of basic nursing care quality for the prevention of adverse events. The validated instrument was named Active Search Report and was finally comprised of 12 quality indicators, listed in Chart 1, and composed of 49 certification items⁽¹⁷⁾.

After the validation proceedings, the institution which served as a basis for the study adopted the Active Search Report through its Quality Control Advising Group, aimed at the prevention of adverse events.

FINAL CONSIDERATIONS

The assessment of nursing care quality through indicators can be applied to reinforce the natural desires of healthcare professionals towards improving care processes; at the same time, it functions as a way to assess the quality of such care.

Assessment processes, however, must employ valid information, since the application of validated measures is the only way to prevent the risk of promoting healthcare teams' unsafe behavior and consequently the discouragement of nursing professionals.

Based on previous premises, this study was carried out inspired by the general objective of validating a nursing care assessment instrument comprised of quality indicators by means of the content validation strategy.

The proposed instrument was considered valid as per its content and applicability to a population of adult patients hospitalized in a medical-surgical clinic of a public university hospital.

After analysis and opinion by experts, the content validation strategy's results showed that from ten initial indicators, comprised of 32 certification items, 12 indicators and 49 certification items were achieved.

Experts' suggestions mostly tackled issues such as the adjustment of the name of some indicators by adding verbs that could infer action; readjustment of indicators' calculation (numerator and denominator) by replacing the average of events observed by the total number; and the subdivision of indicators that measured more than one variable.

As per indicator 12 (verification of complete daily orders by nurse), several comments and suggestions were presented by experts in order to reinforce the importance of the daily basis on the indicator's descriptor, and also the need of complying with the real care demands displayed by the patient.

The indicators proposed by this study do not encompass all relevant aspects for preventing adverse events, nor do they include all care issues that deserve attention, but they are directed at the more urgent needs for intervention, aimed at improving the quality assurance practices in the studied institution, whose reality certainly does not differ from many others all around the country.

The challenge of validating nursing care quality indicators by means of the content validation strategy was worthwhile, since we believe that valid measurement instruments make possible the assessment of care quality patterns, thus directing all necessary changes toward the nursing team work process, aiming at achieving the best possible quality in the context of local realities.

It is worth highlighting, as a limitation of this study, the non-insertion of an indicator related to the interactive aspect between nursing teams and patients. The decision not to include this aspect is justified by the more prominent problems detected in the basic care process, which demanded them to be top priority for interventions.

Another limitation concerns the preciseness of indicator 12 for the quality assessment of the orders carried out by the nurse. The structure of this indicator, along with the data collection methodology, does not allow for a detailed analysis subsidized by individual criteria for the assessment of care provided towards the real needs of patients. For this reason, whenever a detailed analysis is an imperative, it is recommended that the instrument developed in 1999⁽¹⁴⁾ be applied, since it is directed exactly to this end.

The relevance of other studies toward the determination of the instrument's reliability should be stressed, making way for the determination of the coherence degree with which the instrument measures the studied attribute.

REFERENCES

1. Almeida HMS. Programa de Qualidade do Governo Federal aplicado à Saúde. Rev Adm Saúde. 2001;3(12):5-10.

2. Brasil. Ministério da Saúde. Avaliação para melhoria da qualidade da estratégia saúde da família: documento técnico. Brasília; 2006.

3. Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developing and applying quality indicators in primary care. BMJ. 2003;326(7393):816-9.

4. Ermel RC, Fracolli LA. Processo de trabalho de gerência: uma revisão da literatura. Rev Esc Enferm USP. 2003;37(2):89-96.

5. Matsuda LM, Évora YDM. Gestão da equipe de enfermagem da uma UTI: a satisfação profissional em foco. Ciên Cuid Saúde. 2003;2(1):11-8.

6. Idvall E, Rooke L, Hamrin E. Quality indicators in clinical nursing: a review of the literature. J Adv Nurs. 1997;25(1):6-17.

7. Ramírez A, Pérez M. Indicadores de calidad en enfermaría: eventos accidentals de los pacientes. Enfermería. 2003;38 (124):15-21.

8. Pringle M, Wilson T, Grol R. Measuring "goodness" in individuals and healthcare systems. BMJ. 2002;325(7366):704-7.

9. Fernandes MVL. Indicadores de avaliação de práticas de controle e prevenção de infecção do trato urinário associada a cateter: construção e validação [dissertação]. Ribeirão Preto: Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo; 2005.

10. Wynd CA, Schimidt B, Schaefer MA. Two quantitative approaches for estimating content validity. West J Nurs Res. 2003;25(5):508-18.

11. Mcglynn EA, Asch SM. Developing a clinical performance measure. Am J Prev Med. 1998;14(3):14-21.

12. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003;27(2):94-111.

13. Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. Petrópolis: Vozes; 2003.

14. Haddad MCL, Lima SM, Freitas IS, Vituri DW. Avaliação da qualidade da assistência de enfermagem prestada em um Hospital de Ensino. Rev Olho Mágico. 1999;5(19):27-8.

15. Torres RM, Castro CGSO. Gerenciamento de eventos adversos relacionados a medicamentos em hospitais. Rev Eletrônica Adm Hosp [periódico na Internet]. 2007 [citado 2007 ago. 8];3(1). Disponível em: http://www.saocamilo-rj.br/REAH/pdf/gerenciamento_eventos.pdf

16. Bohomol E. Padrões para avaliação da qualidade da assistência de enfermagem. In: D'Innocenzo M, coordenador. Indicadores, auditorias, certificações: ferramentas de qualidade para gestão em saúde. São Paulo: Martinari; 2006. p. 73-85.

17. Vituri DW. Desenvolvimento e validação de um instrumento para avaliação da qualidade do cuidado de enfermagem, Maringá-PR [dissertação]. Maringá: Universidade Estadual de Maringá; 2007.

18. Konow I, Pérez G. Método DELPHI. Métodos y técnicas de investigación prospectiva para la toma de decisiones [texto na Internet]. Santiago: FUNTURO; 1990. [citado 2007 jul. 29]. Disponível em: http://www.geocities.com/Pentagon/Quarters/7578/pros01.html

Received: 02/27/2008

Approved: 09/03/2008

ANNEX 1 

ANNEX 2 

Correspondence addressed to:

Dagmar Willamowius Viteri

Av Robert Koch, 60 - Operária

CEP 86038-350 - Londrina, PR, Brazil

*

Extracted from the thesis "Desenvolvimento e validação de um instrumento para avaliação da qualidade do cuidado de enfermagem", Nursing Masters Program at Maringá State University, 2007.

Publication Dates

• Publication in this collection
  30 June 2009
• Date of issue
  June 2009

History

• Accepted
  03 Sept 2008
• Received
  27 Feb 2008