Development and content validity of the new version of a patient classification instrument

Abstracts

Este estudo teve como propósito reconstruir o instrumento de classificação de pacientes, proposto por Perroca, e avaliar a validade de conteúdo da nova versão. A apreciação do instrumento foi realizada por um grupo de especialistas, constituídos por dez enfermeiros, mediante aplicação da técnica Delphi. A coleta de dados ocorreu no período de março de 2008 a fevereiro de 2009, por meio de questionários estruturados, enviados por correio eletrônico. A nova versão passou a ser constituída por nove áreas de cuidados. Houve concordância >90% em relação à estrutura do instrumento e de 80 a 96% nas áreas de cuidados. O refinamento de instrumentos de classificação é de fundamental importância para geração de dados válidos e confiáveis, embasando a tomada de decisão gerencial relativa ao planejamento da assistência e mensuração de carga de trabalho da equipe de enfermagem.

Pacientes Internados; Carga de Trabalho; Estudos de Validação; Avaliação em Enfermagem

Este estudio tuvo como propósito reconstruir el instrumento de clasificación de pacientes propuesto por Perroca y evaluar la validez de contenido de la nueva versión. La apreciación del instrumento fue realizada por un grupo de expertos constituidos por diez enfermeros mediante aplicación de la Técnica Delphi. La recolección de datos ocurrió en el período de marzo de 2008 a febrero de 2009 a través de cuestionarios estructurados enviados por correo electrónico. La nueva versión pasó a ser constituida por nueve áreas de cuidados. Hubo concordancia > 90% en relación a la estructura del instrumento y de 80 a 96% en las áreas de cuidados. El refinamiento de instrumentos de clasificación es de fundamental importancia para generación de datos válidos y confiables en los cuales se puede basar la toma de decisiones de administración relativa a la planificación de la asistencia y mensuración de carga de trabajo del equipo de enfermería.

Pacientes Internos; Carga de Trabajo; Estudios de Validación; Evaluación en Enfermería

ORIGINAL ARTICLES

Development and content validity of the new version of a patient classification instrument¹

Marcia Galan Perroca

RN, Ph.D. in Nursing, Adjunct Professor, Faculdade de Medicina de São José do Rio Preto, SP, Brazil. E-mail: [e-mail]

^{Corresponding Author}

ABSTRACT

The purpose of this study was to reconstruct Perroca's patient classification instrument; and to assess the content validity of the new version. A group of experts comprising ten nurses appraised the instrument, using the Delphi technique. Data collection took place from March 2008 to February 2009 by means of structured questionnaires sent by electronic mail. The new version contained nine nursing care areas. Agreement levels on the instrument structure amounted to > 90%, and ranged between 80 and 96% in the nursing care areas. The refinement of classification instruments is of great importance to generate valid and reliable data, supporting management decision making on nursing care planning as well as nursing workload measurement.

Descriptors: Inpatients/Classification; Workload; Validation Studies; Nursing Assessment.

Introduction

In recent years, the introduction of the evidence-based practice model and the importance of outcomes achievement in nursing services seems to have lead to the increasing use of management tools. In response to this demand, researchers have attempted to develop specific instruments, cross-culturally adapt or improve existing ones, offering users valid and reliable scales.

Validity and reliability are essential criteria to assess the quality of an instrument. The validity concept relates to the extent to which it shows to be appropriate to measure what it is supposed to measure, in other words, the goal for which it is being used. Reliability refers to its degree of precision, that is, when its measures manage to precisely reflect the actual measures of the attribute under analysis⁽¹⁾.

Patient classification systems (PCS) have been put in practice in different countries for many decades, with a view to continuous monitoring of the nursing team workload by identifying patients' care complexity. The variations in care needs these instruments capture permit quanti and qualitative staff adjustments, so as to achieve a balanced relation between workload and available human capital. This enhances the quality and cost-effectiveness of care delivery. Knowledge on patients' care needs also allows for effective care and discharge planning, patient distribution across units, care quality assessment and strengthens arguments on additional nursing staff needs^(2-3).

In Brazil, the Federal Nursing Council (COFEN), patient classification according to Resolution 293/04⁽⁴⁾; recommends patient classification according to the PCS as one of the variables to support staff decision-making. It seems, however, that the application of this instrument is being slowly incorporated into daily nursing practice.

The classification instrument

At the end of the 1990's, Perroca⁽⁵⁾ developed an instrument to identify patients' nursing care needs and, consequently, measure the team workload. The original instrument^(5-6) is based on individual nursing care needs and serves for use with adult patients. It comprises 13 care areas: Mental State and Level of Consciousness, Oxygenation, Vital Signs, Nutrition and Hydration, Motility, Locomotion, Personal Hygiene, Eliminations, Therapy, Health Education, Behavior, Communication and Skin Integrity. Scores for each of these indicators range from 1 (lowest level of nursing care) to 5 (highest level of care complexity). The minimum score is 13 and the maximum 65 points. Through the instrument, patients can be classified in one of the four care categories: Minimal (13-26 points), Intermediary (27-39 points), Semi-Intensive (40-52 points) and Intensive (53-65 points).

Psychometric property assessment research on this instrument⁽⁶⁾ has given evidence of reliability and validity. Its use at different hospital institutions aroused some nurses' comments that it did not adequately estimate the patient's care category. To examine this question, the following studies were carried out: comparison with an international instrument⁽⁷⁾; surveying users' opinion on the structure, applicability and reliability of the produced data⁽⁸⁾ and analysis of most relevant care indicators inter and intra-care categories⁽⁸⁾.

These research results evidenced a moderate level of agreement among instruments in different care categories⁽⁷⁾. Participating users showed their satisfaction with the proposed classification instrument, except for a certain trend to underestimate the care category the patient belonged to⁽⁸⁾. They also suggested adding other factors that influence nursing teams' workload⁽⁸⁾.

Considering these results, a review of the instrument was started to update its contents and renew its structure, so as to achieve conformity with advances in health sciences and incorporation of new nursing care and management practice trends in the last decade. This paper presents the initial phases of the process to develop and validate the new version of a PCS and aims to: 1. Reconstruct Perroca's patient classification instrument^(5-6) and 2. Assess the content validity of the new version.

Method

Research Design

The new version of the instrument was developed in two phases. To produce the items (care areas), the following were taken into account: users' opinion on the original instrument⁽⁸⁾; research results on how the indicators contributed to patient classification in different care categories⁽⁸⁾; literature review and visits to hospital institutions abroad. The reference framework was related to basic concepts of PCS^(2,9); nursing workload^(10-11); patient care needs^(12-13); instrument development⁽¹⁴⁾ and analysis of international instruments^(3,15-17).

Subsequently, content validity, i.e. checking the relevance of the proposed items and their representativeness to adequately capture the concept one intends to measure⁽¹⁾; was assessed through the application of the Delphi Technique. This technique aims to reach a consensus among experts' opinions on a topic through a series of structured questionnaires, called phases. Answers to each questionnaire are taken into account to formulate subsequent ones. Thus, each phase is constructed on the responses from the previous phase and the process continues until agreement is reached among the participants (consensus)⁽¹⁸⁾.

The selection criteria to compose the expert panel included a minimum five years of experience in direct patient care and/or nursing management, teaching and research. Hence, ten nurses were included in the panel, five teachers and five working in teaching and care. Experts were female, with a mean age of 48.9±4.9 years, mean professional experience 23.8±8.3 years; eight held a Ph.D. and two a Master's degree in nursing. Regarding their professional trajectory, the predominant work areas were nursing service management, medical-surgical nursing, intensive care unit, gynecology and obstetrics and pediatrics and neonatology.

Data collection procedure

Data were collected between March 2008 and February 2009, after a favorable opinion from the Institutional Review Board (Protocol No 3262/2006) and the experts' consent. Care areas for which expert answers reached agreement levels of 80% or higher were considered validated. The consensus criterion was previously defined.

The questionnaires, forwarded by electronic mail, comprised a five-point Likert scale, a first part with ten assertions on the instrument structure and a second with seven assertions on each care area. The assertions referred to relevance, clarity, objectivity, care complexity level and applicability of the instrument. Score five was for "I totally agree" and one for "I totally disagree". When the assertion took the negative form (assertions 5 and 6 in the structure assessment and assertions 4 and 5 in the care area assessment), the score was inverted, according to literature recommendations⁽¹⁾; which means that one referred to total agreement and five to total disagreement. Experts were also asked to mark whether the care areas should be maintained, excluded or modified. Open questions in the questionnaire allowed participants to transmit other considerations on the structure and care areas. Together with the first questionnaire, the experts were asked to complete data on personal and professional characteristics.

Data analysis

Minitab Statistical Software (MINITAB) version 12.22 was used for statistical analysis. Descriptive data were presented as percentages, means and standard deviation. The Likert scale was considered as an ordinal measurement level and median and quartiles were calculated (Q1 and Q3). Subjective data were grouped, categorized and related according to the study objective.

Results

Instrument reconstruction

In the proposed new version, some care areas from the original instrument were excluded or their contents merged, while others were added to permit more accurate measurement of patients' care complexity and the nursing resources they used. Thus, the following care areas were considered: 1- Care Process Planning and Coordination; 2- Investigation and Monitoring; 3- Personal Hygiene and Eliminations; 4- Nutrition and Hydration; 5- Locomotion or Activity; 6- Therapeutics; 7- Emotional Support and 8- Health Education. Each area is scored according to four gradations, with increasing care complexity. Score intervals are as follows: 8-11 points (Minimal Care), 12-18 (Intermediary Care), 19-25 (Semi-Intensive Care) and 26-32 (Intensive Care).

To elaborate the score intervals, it was considered that the patient would continue in a given category if obtaining the minimum score for one gradation in all care areas until about 40% of the care areas in the subsequent gradation.

Expert assessment

The Delphi Technique was applied in three phases to reach a consensus on the structure and contents of the new version of the classification instrument.

Delphi Phase 1

Answers to the first questionnaire showed that, regarding the instrument structure, agreement levels exceeded 80%, except for assertion 2 on clear statements (74% - Median -Md 4.0) and assertion 8 on the observation of growing care complexity levels in care area gradations (78% - Md 4.0). The lowest agreement level was for assertions 5 (extension) and 6 (instrument complexity), each corresponding to 38% - Md 2.0 (Table1).

Agreement levels in the different care areas ranged from 82% (Md 4.5 - 5.0) to 96% (Md 5.0). For assertion 4 (extension), agreement percentages ranged between 34% (Md 2.0) and 46% (Md 2.0) and, for assertion 5 (instrument complexity), between 38% (Md 2.0) and 44% (Md 2.0) (Table 2).

Although the experts agreed on maintaining the care areas, however, suggestions were made to modify the structure and contents of some of them regarding the quantification of activities, ordering of gradations and use of more appropriate terminologies. It was argued that pressure ulcer prevention measures would be better classified as care instead of monitoring, and also that the care area Investigation and Monitoring contained many variables, hampering its assessment. The experts suggested separating variables in this area in two domains or categories.

Delphi Phase 2

The construction of the second questionnaire was based on the experts' evaluations. The care area Investigation and Monitoring was opened up, giving rise to a new area called Skin Integrity. Thus, the instrument started to include nine care areas and patient classification scores were reviewed, observing the same criteria mentioned above. Terms that indicated subjectivity were removed and unclear statements were explained better. Some care area gradations were associated or altered to disclose the increasing care complexity.

Answers to the second questionnaire showed that, regarding the instrument structure, agreement levels ranged from 80% (Md 4.0) to 94% (Md 5.0). As for the statements, the lowest percentage (80%) was related to the clarity aspect (Md 4.0) (Table 1). The care area Care Process Planning and Coordination (CA1) showed agreement levels below 80% on the following aspects: clear statements 70% (Md 4.0), establishment of a common language among professionals 78% (Md 4.5) and representativeness of more time-consuming activities 76% (Md 4.0). Agreement on the representativeness of time-consuming activities corresponded to 80% (Md 4.0) regarding Skin Integrity (CA4). Agreement levels on the other care areas ranged from 88% (Md 5.0) to 98% (Md 5.0) and were considered validated (Table 3). Some of the experts appointed that the position change aspect was being scored twice in the care areas Locomotion and Activity and Skin Integrity.

Delphi Phase 3

Thus, version 3 of the instrument was elaborated, incorporating the changes the experts requested. Care Process Planning and Coordination was thoroughly restructured with a view to greater clarity and concision. In the Skin Integrity area, some terminologies were altered and the technical complexity of dressings and some preventive measures were added. Position change was included in the contents of the Skin Integrity care area. Scores were changed to cover situations in which patients do not receive any score in some care areas.

Agreement levels on Care Process Planning and Coordination (CA1) ranged from 80% (Md 4.0) to 96% (Md 5.0). Some judges disagreed from the non-inclusion of some administrative activities nurses perform in this care area. Agreement levels on Care for Skin and Mucosa ranged from 92% (Md 4.0) to 96% (Md 5.0) (Table 4). Due to space constraints, the final version of the instrument cannot be presented in this paper, but is available to anyone interested.

Discussion

Refining an instrument is an effortful and long process. The concept under analysis (construct) needs to be clearly defined. Regarding the PCS, the construct comprises the patients' care needs. As they are countless and multidimensional, they cannot be represented as a whole. The goal of the instrument is not to offer a comprehensive list of all care activities nursing performs, as that would make it extensive and exhaustive; instead, the goal is to contain the representative dimensions of care delivery that most influence the nursing team workload.

Thus, structural design aspects, such as the extent and the way items are presented, become fundamental. Efforts were made to make the instrument shorter in comparison with the original version, without compromising the collected information. Literature has recommended some characteristics of classification instruments to enhance their use for decision-making purposes and their acceptability. Simplicity, clarity, objectivity and not consuming much time for completion have been emphasized as criteria⁽⁹⁾. In this study, the experts' evaluation evidenced that it seemed to comply with these requisites.

In this new version of the instrument, the number of care areas was reduced to nine, in comparison with 13 in the original instrument. Some areas were maintained, contents were merged and, also, other areas were added to picture the patients' true needs more reliably. Many aspects not included in the previous instrument and which consume the nursing team's time were included, such as care planning, help during diagnostic and therapeutic tests, clinical nursing assessment, urgency and emergency care, pressure ulcer prevention measures, among others.

In a previous study⁽⁸⁾; users reported perceiving that the instrument seemed not to show patient complexity as they observed it. This means that the nurses' clinical judgment and objective assessment through the instrument did not seem to be in line. Hence, in the new structure, besides alterations in the care areas' contents, the score standardization was revised. The previously used method of equal class intervals among the different categories was replaced by different intervals, as patients with more than 40% of characteristics from the subsequent gradation should also belong to the same care category.

Another important alteration was the exclusion of the companion's participation in some care areas (nutrition and hydration, motility, locomotion, personal hygiene, eliminations). Despite acknowledging relatives/companions' important contribution to patients' recovery, they cannot be obliged to assume the responsibility for the care act, which will always be a nursing responsibility. As their presence influences the number of care hours, considering that relatives need nursing orientation and supervision, this aspect was considered in the care areas Emotional Support and Health Education. Thus, four gradations were included in the instrument to direct care complexity.

Nursing care systemization (NCS - in Portuguese SAE) is fundamental to equip and humanize nursing actions. Their accomplishment and documentation, in all phases, demands time and influences the workload. Some studies^(19-20) have attempted to measure the time spent on this activity, with different results. Although accomplished at a teaching hospital, it should be highlighted that different methods were used (estimated and observed time), different process phases were measured, some with a standardized language and computer system, which makes it difficult to compare the findings.

In response to the need to put in practice NCS and cover its influence on nurses' workload, the care area Care Process Planning and Coordination was incorporated into the new version. To elaborate its contents, aspects related to patient information access were taken into account, as well as communication with nursing professionals and other areas, use of resources for care practice and documentation. International studies have shown that documentation activities represent 13%⁽²¹⁾ and 18-27%⁽²²⁾ of the time nurses spend.

Administrative activities were not taken into account, however, for two reasons: nurse supervisors or coordinators are responsible for most of these activities, and not clinical nurses; classification instruments are structured based on the care concept, that is, they measure care delivered when the patient and/or family is present (direct care) and care delivered when they area not present, but which serves to prepare or complete care (indirect care)⁽²³⁾. Direct and indirect care have represented about 80% of nursing activities, approximately 50% of which refers to indirect care activities⁽²⁴⁾.

International literature displays different workload concepts, which consequently influence its measurement form. Some authors consider that it only comprises direct and indirect care delivery⁽²⁵⁾; and others that it should also included non-patient related activities, such as unit and nursing team management⁽¹¹⁾.

The fundamental goal of classification instruments needs to be highlighted, which is to produce information about the patient for care administration purposes, reflecting the service offered to the patient/family. Incorporating other variables not specifically related to the patient, such as operating elements at the unit, into the instrument structure is not recommended, as this can compromise data precision⁽²³⁾.

Conclusion

This paper discussed the initial phases in the development and validation of the new version of a PCS. The goal was to create an instrument that would permit identifying the quantity of nursing resources patients demand and, consequently, monitor the nursing team's workload. This re-construction was complex, as it involves a range of factors intervening in the nursing work process, which still need more extensive and in-depth research.

The instrument demonstrated content validity in the experts' opinion. It still needs further testing, however, to assess other psychometric properties, including internal consistency, inter-rater reliability and construct validity. The author hopes the new version can more reliably represent the nursing team's activity and time spent in the care process.

Received: May. 24^th 2010

Accepted: Dec. 10^th 2010

1 Supported by Faculdade de Medicina de São José do Rio Preto, SP, Brazil (Researcher Funding Scholarship).

• 1. Polit DF, Beck CT, Hungler BP. Fundamentos de Pesquisa em Enfermagem: métodos, avaliação e utilização. 5. ed. Porto Alegre: Artmed; 2004.
• 2. Giovannetti P. Understanding patient classification systems. J Nurs Admin. 1979;9(2):4-9.
• 3. Malloch K, Conovaloff A. Patient classifications systems, Part 1: The third generation. J Nurs Adm. 1999;29(7-8):49-56.
• ⁴
  Conselho Federal de Enfermagem. Resolução n.293/04. Fixa e estabelece parâmetros para dimensionamento do quadro de profissionais de enfermagem nas unidades assistenciais das instituições de saúde e assemelhados [legislação na Internet]. São Paulo: COREN-SP; 2004. [acesso 10 junho 2009]. Disponível em: http://corensp.org.br/072005/legislacoes/legislacoes_busca.php?leg_id=10105&texto=293
• 5. Perroca MG, Gaidzinski RR. Sistema de Classificação de Pacientes: construção e validação de um instrumento. Rev Esc Enferm USP. 1998;32(2):153-68.
• 6. Perroca MG. Instrumento de Classificação de Pacientes de Perroca: validação clínica [tese]. São Paulo: Escola de Enfermagem, Universidade de São Paulo; 2000.
• 7. Perroca MG, Ek AC. Assessing patient's care requirements: a comparison of instruments. Scand J Caring Sci. 2007;21(3):390-6.
• 8. Perroca MG. Instrumento para classificação de pacientes: opinião de usuários e análise de indicadores de cuidado. Rev Esc Enferm USP. 2008;42(4):656-64.
• 9. De Groot HA. Patient classification system evaluation: part two, system selection and implementation. J Nurs Adm. 1989;19(7):24-30.
• 10. Hughes M. Nursing workload: an unquantifiable entity. J Nurs Manage 1999;7:317-22.
• 11. O'Brien-Pallas L, Irvine D, Peereboom E. Measuring nursing workload: understanding the variability. Nurs Econ. 1997;15:171-82.
• 12. Brennan CW, Daly BJ. Patient acuity: a concept analysis. J Adv Nurs. 2009;65(5):1114-26.
• 13. Fagerstron L, Bergbom Engberg I. measuring the unmeaasurable: a cring science perpective on patient clasification. J Nurs Manage. 1998;6:165-72.
• 14. Clark LA, Watson D. Constructing Validity: Basic Issues in Objective Scale Development. Psychol Assess. 1995;7(3):309-19.
• 15. Rauhala A, Fagersrtrom L. Determining optimal nursing intensity: the Rafaela method. J Adv Nurs. 2004;45(4):351-9.
• 16. Levenstam A-K, Bergbom I. Changes in patients´need of nursing care refleted in the Zebra system. J Nurs Manag.2002;10:191-9.
• 17. Nordestedt A. BEAKTA - Användarhandbok (em sueco, manual do usuário). Lund: BEAKTA AB; 1997.
• 18. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008-15.
• 19. Minuceli JS, Utilização do tempo por enfermeiras clínicas clínicas em unidades de internação [Monografia]. São José do Rio Preto: Faculdade de Medicina de São José do Rio Preto; 2007.
• 20. Rezende PO, Gaidzinski RR. Tempo despendido no sistema de assistencia de enfermagem após implementacao de sistema padronizado de linguagem. Rev Esc Enferm USP. 2008;42(1):152-9.
• 21. Upenieks V. Working sampling:assessing nursing efficiency. Nurs Manage. 1998;29(4):27-9.
• 22. Urden LD, Roode JL. Work sampling: a decision-making tool for determining resources and work redesign. J Nurs Adm. 1997;27(9):34-41.
• 23. Van Slyck A. Patient classification systems: not a Proxy fro nurse "busyness". Nurs Adm Q. 2000;24(4):60-5.
• 24. Chaboyer W, Wallis M, Duffield C, Courtney M, Seaton P, Holzhauser K, Schluter J, Bost N. A comparison of activities undertaken by enrolled and registerd nurses on medical wards in Australia: an observational study. Int J Nurs Stud. 2008;45(9):1274-84.
• 25. Fagerström L, Raino A, Rauhala A, Nojonen K. Validation of a new method for patient classification, the Oulu Patient Classification. J Adv Nurs. 2000;31:481-90.

Publication Dates

• Publication in this collection
  11 Mar 2011
• Date of issue
  Feb 2011

History

• Accepted
  10 Dec 2010
• Received
  24 May 2010