Instrument to assess the nursing care product: development and content validation ¹

Abstract

Objectives:

develop and validate the content of a tool about nursing care production.

Method:

the data were collected between 2011 and 2013, based on focus groups, the application of semistructured questionnaires (prototype test) and the Delphi technique. The focus groups were used to produce the instrument items and held at three hospitals in the interior of the State of São Paulo, involving 20 nurses. A panel of 10 experts evaluated the instrument.

Results:

after two phases of the Delphi technique, the tool consisted of eight items. The content validity index of the scale corresponded to ≥0.9 and the content validity of the items ranged between 0.8 and 1.0, indicating the maintenance of the structure and content. The assertion on the applicability in daily nursing practice showed a content validity index of the scale equal to 0.8.

Conclusion:

this study permitted the development and content validation of scale on nursing care production, equipping the nurses in their management practice.

Descriptors:
Process Assessment (Health Care); Validation Studies; Nursing Assessment; Professional Practice

Resumo

Objetivos:

desenvolver e validar o conteúdo de um instrumento sobre produção do cuidado de enfermagem.

Método:

a coleta de dados ocorreu entre 2011 e 2013, a partir de grupos focais, aplicação de questionários semiestruturados (teste do protótipo) e técnica Delphi. Os grupos focais foram utilizados para geração de itens do instrumento e realizados em três hospitais do interior do Estado de São Paulo, com a participação de 20 enfermeiros. A apreciação do instrumento foi conduzida por um painel de 10 especialistas.

Resultados:

após duas fases da técnica Delphi, o instrumento passou a ser constituído por oito itens. O índice de validade do conteúdo da escala foi de ≥0,9 e a validade dos conteúdos dos itens apresentou variação de 0,8 a 1,0, indicando a manutenção da estrutura e do conteúdo. A afirmativa referente à aplicabilidade na prática diária do enfermeiro apresentou índice de validade do conteúdo da escala de 0,8.

Conclusão:

este estudo possibilitou desenvolver e validar o conteúdo de uma escala sobre produção do cuidado de enfermagem, instrumentalizando os enfermeiros em sua prática gerencial.

Descritores:
Avaliação de Processos (Cuidados de Saúde); Estudos de Validação; Avaliação em Enfermagem; Prática Profissional

Resumen

Objetivos:

desarrollar y validar el contenido de un instrumento sobre producción del cuidado de enfermería.

Método:

los datos fueron recolectados entre 2011 y 2013, a partir de grupos focales, aplicación de cuestionarios semiestructurados (prueba del prototipo) y técnica Delphi. Los grupos focales fueron utilizados para generar ítems del instrumento y organizados en tres hospitales del interior del Estado de São Paulo, con la participación de 20 enfermeros. La apreciación del instrumento fue conducida por un panel de 10 especialistas.

Resultados:

tras dos fases de la técnica Delphi, el instrumento pasó a ser constituido por ocho ítems. El índice de validez de contenido de la escala fue ≥0,9 y la validez de los contenidos mostró variación de 0,8 a 1,0, indicando la manutención de la estructura y del contenido. La afirmativa respecto a la aplicabilidad en la práctica diaria del enfermero mostró índice de validez del contenido de la escala de 0,8.

Conclusión:

este estudio permitió desarrollar y validar el contenido de una escala sobre producción del cuidado de enfermería, instrumentalizando los enfermeros en su práctica gerencial.

Descriptores:
Evaluación de Proceso (Atención de Salud); Estudios de Validación; Evaluación en Enfermería; Práctica Profesional

Introduction

The nursing activities can be understood as a care production system ¹. The conception of this product involves interaction among human beings, dynamic, non-linear and emerging processes, besides the ability to self-organize and adapt to the needs of the patient/family member, the team and the institution, characterized as a complex adaptive system ².

Nurses are responsible for managing care through expressive and instrumental direct and indirect care actions. This practice implies staff dimensioning, leadership and training of the team, resource projection and provision, care coordination and execution, as well as the planning and assessment of the interventions made ³. The service ability and complexity of the services provided in hospital organizations influence the activities performed. The incorporation of quality improvement and cost control processes modifies the organization of nursing work and can limit the time spent on direct care delivery to patients^(4-5). Thus, the managers have faced severe problems around the world, related to insufficient human capital, an excessive work burden, lack of qualification, absenteeism and professional evasion, which influence the quality of care and may result in care errors and/or omission ⁶.

To guarantee the delivery of this service and grant further visibility to the profession, an accelerated expansion of studies is ongoing that emphasizes the standardization and classification of patient-centered nursing activities, including diagnoses, interventions and desired outcomes ⁷. In parallel, the safety movement in health care highlights the relation between the burden of the nursing professionals and the care results obtained ⁸. Based on these dimensions, another research line considers the performance of this service as the ability to obtain the resources and use them sustainably to produce care and discusses aspects to measure and assess its contribution in organizations ⁹.

Although the increasing use of management tools in nursing services drives the researchers to develop specific, valid and reliable tools ¹⁰, however, no discussions on tools to assess the care product were found, neither in the Brazilian nor in the international literature. In that sense, this research departs from the following problem: can a tool be developed that contains activities and/or situations of nursing practice to assess the product at the end of the work shift? By exploring this knowledge, the efficiency and efficacy of the nursing activities can be measured, contributing to decision making and the improvement of processes. Thus, in this study, the aim was to develop and validate the content of a tool about nursing care production.

Method

The study was developed in three phases: production of items and development of the tool, test of the prototype and content validation.

Phase 1: production of items and development of the tool

The focus group technique was used to produce the items of the tool to assess the production of nursing care ¹¹. This phase was undertaken at three large hospitals in the interior of the State of São Paulo, with a view to joining subjects from distinct realities. The data were collected between October 2011 and July 2012. Four focus groups were constituted involving 20 nurses and the meetings took between one hour and a half and two hours at most. Further details on this phase have been described in another study.

The discussions were conducted by one of the researchers (moderator) and guided by the following questions: What activities do you assess at the end of the workday to consider your shift was excellent? In what situations is your shift good? When do you consider your shift regular? and What was your shift like to be considered bad?

The discourse was recorded and filmed, with the participants' consent, and then transcribed and investigated using thematic content analysis ¹². The theoretical framework was based on the concept of complex adaptive system ²and care production¹.

To construct the prototype tool, reference frameworks on nursing care management^(3,13) and

quality process in health^(6,8) were used, among others. The grading was based on the measures used in the classification of nursing outcomes ¹⁴and a patient classification tool ¹⁰.

Phase 2: prototype test

Intentional sampling was used to select nine nurses to assess the prototype, representing six medical and surgical wards, one pediatric ward and two adult and pediatric intensive care services at one of the hospitals, in the different work shifts.

After they agreed to participate, the subjects received individual orientations about the composition of the prototype and its use. The participants were expected to apply it at the end of at least two shifts. The nurses received a questionnaire, which initially contained the respondent's profile and seven proposals, considering the relevance, clarity, simplicity and applicability of the prototype. To assess each proposal, a five-point Likert scale was used, ranging from one (I completely disagree) to five (I completely agree), besides space for the subjects to add other comments.

The data were collected in March 2013. Seventeen assessments were obtained, as one of the participants went on leave.

Phase 3: content validation

This type of validity indicates whether the instrument items are appropriate to measure the specific construct and whether they adequately address their domain, being necessarily judgment-based ¹⁵. Therefore, the Delphi technique was applied, which aims to transform the experts' opinions on a theme into a group consensus, through the validation of structured questionnaires, conducted in phases or cycles. Based on the answers to each questionnaire, the subsequent phase is reformulated and the process continues until a pre-established agreement is obtained among the participants ¹⁶.

To compose the expert panel (judges), 15 nurses were eligible ¹⁷, who had at least ten years of professional experience, acted as teachers, preferably as leaders of research groups in the management area, certified by the Brazilian Scientific and Technological Development Council (CNPq) and nurse managers from Brazilian hospitals. The experts were invited electronically, addressing information on the research objectives, method, ethical aspects and access link. The questionnaires were constructed online, using the software Google Drive(online panel). The data were collected between May and December 2013, totaling 10 judges.

The questionnaire contained a pretext with a short explanation letter about the study objectives, followed by the consent form. Access was only permitted after consent had been obtained. Then, demographic and professional data were collected to determine the respondents' profile.

The first part consisted of items corresponding to the nursing care production, assessed individually through eight assertions, involving: pertinence of contents, understanding among the professionals, extent and complexity of the item, clarity of the statements, grading, applicability in daily nursing practice and contribution to management decision making. The second part permitted assessing the structure of the tool (nine assertions), considering the abovementioned aspects and whether they represented the most expressive dimensions of this production. What the content of each item is concerned, the experts were asked to indicate the maintenance, addition, modification of the grades, elimination, merger or others, justifying their decision or any other considerations. The questionnaire was organized in a Likert-type format, similar to what was done in the prototype analysis.

Score five was attributed to the answers "I completely agree" and one to "I completely disagree". In case of assertions formulated as negatives (assertions 4 and 5 in the care production items; and 5 and 6 in the assessment of the tool structure), the score was inverted.

The consensus criterion in this study was previously determined, considering the content validity index of the items (IVC-I) equal to 0.8 or higher and, for the structure (scale), the content validity index of the scale (IVC-E) equal to 0.9 or higher^(13,15) was considered. For each assertion, IVC-I and IVC-E were calculated by adding up the answers "4" - I agreeand "5" -I completely agree, using the total number of judges as the denominator ¹⁷. For a new phase (cycle of questions), as a form of feedback to the judges, a report was included in the questionnaire that described the results and the items that were revised, eliminated or validated, highlighting what should be reassessed.

The statistical analysis of the data corresponding to the analysis of the prototype (Phase 2) and content validation (Phase 3) was undertaken using Microsoft Office Excel 2007. The Likert scale was considered as the ordinal measuring level and median and quartiles were calculated (Q1 and Q3). The subjective data were grouped and analyzed according to the research objective.

Approval for the research was obtained from the Institutional Review Board - Process 0050/12.

Results

Phase 1: production of items and development of the tool

In the content analysis of the discourse, obtained through the groups, four thematic categories emerged: planning, intervention and assessment of care; dimensioning and qualification of the nursing team; resources needed to deliver care and multiprofessional interaction.

Based on the categorization of the themes, subcategories were delimited, constituting items to compose the prototype: 1. nursing care planning, 2. patient/family care, 3. care needs, 4. nurse supervision and care transition, 5. interaction and multidisciplinary activity, 6. resources needed and support services, 7. problems and emergency situations, 8. staff dimensioning according to workload and 9. qualification and professional development.

Based on the guiding questions, item grades were generated (from one till four), showing the increasing intensity concerning the best nursing care product. Thus, at the end of the work shift, the nurse could assess all items at one of the four grades, considering the option that best approached the reality experienced. The individual item scores are added up and lead to a classification according to the following intervals: 9-12 points (bad), 13-21 (regular), 22-30 (good) and 31-36 (excellent).

To elaborate the score intervals, it was considered that the classification of the nursing care product remains in a given category if the minimum score for one grade is obtained on all items and up to about 40% for the higher grade ¹⁰.

Phase 2: prototype test

Nine nurses participated in the prototype test ( Table 1), predominantly women (n=7), with a mean age of 32 years (Standard deviation sd=5.0) - ranging between 27 and 40 years, and mean professional experience of seven years (sd=3.0) - ranging between 2 and 14 years. Concerning the education, six subjects had concluded specialization courses in the activity area or related areas. The lowest percentage of agreement (50% - Md 3.0) was found for the assertion related to the applicability of the tool in daily nursing practice.

Table 1
Nurses' opinion about the prototype tool. Campinas, SP, Brazil, 2014

Assertion	%	Md (Q1-Q3)*
Covers most expressive items	89	4.0 (4.0-5.0)
Addresses pertinent factors in each item	100	4.0 (4.0-4.0)
Presents clear statements	89	4.0 (4.0-5.0)
Permits common language among the professionals	88	4.0 (4.0-5.0)
Is complex	33	2.0 (2.0-4.0)
Is very long	40	2.0 (2.0-4.0)
Can be introduced in daily nursing practice	50	3.0 (2.0-4.0)

• *Md: median; Q1-Q3: quartiles

Phase 3: content validation

The panel consisted of 10 female experts, with a mean age of 57.7 years (sd=8.3), 33.7 years of professional experience (sd=7.2). Eight were Ph.D.'s in nursing, one held an M.Sc; and one was a hospital administration specialist. Seven nurses taught, one was a faculty member and nurse coordinator, one was a manager and one a nursing service management consultations, including different institutions in seven Brazilian cities in the South and Southeast.

The Delphi technique was applied in two phases to reach the pre-established consensus. The answers to the first questionnaire showed an agreement index in the different items (IVC-I) of nursing care production ranging between 0.6 (Md 4.0) and 1.0 (Md 4.0-5.0). Assertion 4 (extent) showed an IVC-I ranging between 0.0 (Md 2.0) and 0.4 (Md 2.0) and assertion 5 (complexity of the tool) ranged between 0.0 (Md 2.0) and 0.3 (Md 2.0), representing the lowest agreement levels ( Table 2).

Although the judges mostly agreed to maintain the indicators, they suggested several modifications in the structure and content of some items. Thus, concepts were revised, terms were removed that indicated subjectivity and statements that were unclear were explained better.

The item "nurse supervision and care transition" was (re)named "monitoring and care transfer", evidencing the educational aspect of the supervision and the systematization of information when the shift is transferred. It was argued that the support services can be considered as resources. Therefore, the title of this item was altered to "resources needed to deliver care", correlating the time the team consumes to deliver care. The indicator "problems and/or emergency situations" was excluded because it is considered redundant in view of the changes made in the items "resources needed to deliver care" and "nursing staff dimensioning".

The title of the tool - classification of the nursing work process - was also changed, aiming to further clarify the research problem. The order in which the items were presented was also modified. The tool was now identified as Assessment of the Nursing Care Process ( Avaliação do Processo de Cuidar de Enfermagem- APROCE) and included eight items or indicators with revised scores.

As regards the second questionnaire (Delphi 2), the answers showed a variation in IVC-I between 0.8 (Md 4.0) and 1.0 (Md 4.0-5.0) for the different items. Agreement with assertion 4 (extent) ranged from 0.0 (Md 2.0) to 0.4 (Md 2.0), and with assertion 5 (complexity) between 0.1 (Md 2.0) and 0.3 (Md 2.0) ( Table 3).

Table 2
Experts' opinion on nursing care production items in the Delphi 1 phase. Campinas, SP, Brazil, 2014

• *Score ranges from 1 to 5: the higher the score, the higher the agreement
• †IVC-I: content validity index of items
• ‡Md: median
• §Q1-Q3: quartiles

Table 3
Experts' opinion on nursing care production items in the Delphi 2 phase. Campinas, SP, Brazil, 2014

• *Score ranges from 1 to 5: the higher the score, the higher the agreement
• †IVC-I: content validity index of items
• ‡Md: median
• §Q1-Q3: quartiles

In view of the experts' contributions, the following changes were made: inversion of grade "3" by "4" in the item "nursing care planning"; exclusion of the term that indicated subjectivity in grade "4" - almost all- for the item "interaction and multidisciplinary action" and an increasing time interval was adopted in response to the requests/calls in "patient and/or family care".

As regards the structure of the instrument (scale), the IVC-E was higher than 0.9 in the first and second phase of the Delphi technique. Scores below the established level were highlighted for assertion 4, about the clarity of the statements (0.7 - Md 4.0) (phase 1) and for assertion 8, which refers to the applicability in daily nursing practice (0.8 - Md 4.0), in phases 1 and 2 ( Table 4).

Table 4
Experts' opinion on the structure of the tool in the phases Delphi 1 and 2. Campinas, SP, Brazil, 2014

Assertion*	IVC-E†	Delphi 1 Md‡	Q1-Q3§	Delphi 2	IVC-E†	Md‡	Q1-Q3§
AF1 - addresses more expressive areas	1.0	4.0	4.0-5.0	1.0	4.0	4.0-4.7
AF2 - item content	1.0	4.0	4.0-5.0	1.0	4.0	4.0-5.0
AF3 - shared language among professionals	0.9	4.0	4.0-4.0	1.0	4.0	4.0-4.7
AF4 - clear statements	0.7	4.0	2.5-4.0	1.0	4.0	4.0-4.0
AF5 - very long	0.4	2.0	2.0-4.0	0.1	2.0	2.0-20
AF6 - complex	0.4	2.0	2.0-4.0	0.2	2.0	2.0-2.0
AF7 - increasing intensity in grades	1.0	4.5	4.0-5.0	0.9	4.0	4.0-4.7
AF8 - can be introduced in daily practice	0.8	4.0	4.0-4.7	0.8	4.0	4.0-4.0
AF9 - permits management decision	0.9	4.0	4.0-4.7	1.0	5.0	4.0-5.0

• *Score ranges from 1 to 5: the higher the score, the higher the agreement
• †IVC-I: content validity index of items
• ‡Md: median
• §Q1-Q3: quartiles

The title of the tool was again changed to Assessment of Care Product in Nursing (APROCENF), in view of the experts' validation, so as to further approach the theoretical framework adopted in the research. And, in this second phase, they indicated the maintenance of the tool structure and content and suggested applying the scale at regular instead of daily intervals. Thus, in its final form, the tool consisted of eight items (described earlier), with scores ranging between 8 and 32 points.

Discussion

In this study, the development and content validation of a tool to assess the product of care in nursing is described.

To build a measure, items need to be produced that are relevant to the research question and target public. Besides the literature review, interviews are the most used source to obtain qualitative data. In particular terms, the focus group allows the participants

to explore what they have in common instead of

presenting individual viewpoints ¹⁸. Therefore, the use of these groups at different institutions allowed for the identification of several aspects that influence the care production and are common in the different realities the nurses experience.

These aspects were analyzed from the complex system perspective, centered on the best nursing practices^(2,19). The items produced were ordered in view of the application of this science to the traditional management concepts: planning, organization, direction, coordination and control ²⁰. The understanding of the factors inherent in the care system expands the managers' view on the processes, which permits forecasting their behavior and determining measures to mitigate and/or solve problems^(20-21).

A study demonstrates that the nursing work environment and actions affect the patients' perception of care quality. The nurses' clinical competency, the relations among professionals, the appropriate composition of the team, a more autonomous practice with management support, besides patient-centered care, can contribute to a more positive experience of this care consumer ²².

The grades of each item (from one to four), indicating an increasing level of intensity in terms of the best care product, represented the main challenge in the development of this tool. The discourse obtained through the guiding questions supported these grades, but the measures to classify nursing outcomes ¹⁴, initially used in this construction, did not delimit the scale answers objective and clearly. Therefore, the most important changes in its content were made in the first phase of the Delphi technique.

To male the tool more objective, some values were mentioned in each items to better understand and distinguish its grades. No studies were found that evidenced the consumption of the nursing team's time to provide for the necessary resources and, hence, estimates values were used, related to the distribution of nursing time and to associated activities ²³.

Similarly, the nursing team's response time to the requests/calls, corresponding to the item "patient/ family care", was based on international literature on the assessment on nursing care not delivered/delayed ²⁴.

As a possibility to approach theory and care practice, the preliminary assessment of the prototype tool was chosen, involving clinical nurses who are directly active in patient care. The pretest evidenced the users' understanding of the items produced and appointed no substantial changes in the format or content. Nevertheless, the feasibility of daily applications represents a reflection on the researchers' initial proposal, as it supports the experts' considerations (IVC-E corresponding to 0.8 in Delphi 1 and 2) and represents an important aspect to be tested.

The daily completion of the tool would help the clinical nurse to assess the product delivered at the end of the work shift. Nevertheless, some experts considered the scale too long to be included in the work routine and suggested its periodical application, at regular intervals. Thus, the analysis of each item would be based on direct observation (supervision), on the survey of data through care records and interviews with patients/relatives and professionals.

To quantify the content validity of this tool, the IVC was chosen as the index of inter-rater agreement or estimated consensus because of its easy understanding and because of the distinction among the information on the items (IVC-I) and the scale (IVC-E). Thus, each item could be highlighted and refined and the scale could be made more accurate^(15,17).

The results demonstrated the validity of the tool's content, but should be further analyzed to assess the other psychometric properties. Further research is also needed to investigate the different possible applications so as not to consume much nursing time to complete it and produce results for inter and extra-institutional comparison.

Conclusion

This study permitted a better understanding of nursing care production from the perspective of nurses working at hospital institutions and permitted developing and validating the content of a tool to measure this

product. Its application permits both the identification of the main dimensions involved in care and the measuring of their influence, contributing to management decision making and effective care management.

Acknowledgements

To the Research Group Health and Nursing Service Management (GESTSAÚDE).

References

• ¹
  Backes DS, Silva DM, Siqueira HH, Erdmann AL. O produto do serviço de enfermagem na perspectiva da gerência da qualidade. Rev Gaúcha de Enferm. 2007;28(2):163-70.
• ²
  Chaffee MW, Mcneill MM. A model of nursing as a complex adaptive system. Nurs Outlook. 2007;55:232-41.
• ³
  Santos JLG, Pestana AL, Guerrero P, Schlindwein B, Horner Meirelles H, Erdmann AL. Práticas de enfermeiros na gerência do cuidado em enfermagem e saúde: revisão integrativa. Rev Bras Enferm. 2013;66(2):257-63.
• ⁴
  Giordani JN, Bisogno SBC, Silva LAA. Percepção dos enfermeiros frente às atividades gerenciais na assistência ao usuário. Acta Paul Enferm. 2012;25(4):511-6.
• ⁵
  Camelo SHH. Professional competences of nurse to work in Intensive Care Units: an integrative review. Rev. Latino-Am. Enfermagem. 2012;20(1):192-200.
• ⁶
  Aiken LH, Sermeus W, Van den K, Sloane DM, Busse R, McKee M, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patient in 12 countries in Europe and the United States. BMJ [Internet]. 2012[cited 2014 ago 21];344:1-14. Available from: http://www.bmj.com/content/344/bmj.e1717
  » http://www.bmj.com/content/344/bmj.e1717
• ⁷
  Furuya RK, Nakamura FRY, Gastaldi AB, Rossi LA. Sistemas de classificação de enfermagem e sua aplicação na assistência: revisão integrativa de literatura. Rev Gaúcha Enferm. 2011;32(1):167-75.
• ⁸
  Ball JE, Murrells T, Rafferty AM, Morrow E, Griffiths P. Care left undone during nursing shifts: associations with workload and perceived quality of care. BMJ Qual Saf. 2014;23:116-25.
• ⁹
  Dubois CA, D'Amour D, Pomey MP, Girard F, Brault I. Conceptualizing performance of nursing care as a prerequisite for better measurement: a systematic and interpretive review. BMC Nursing [Internet]. 2013. [acesso 22 ago 2014];12(7):1-20. Disponível em: http://www.biomedcentral.com/1472-6955/12/7
  » http://www.biomedcentral.com/1472-6955/12/7
• ¹⁰
  Perroca MG. Development and content validity of the new version of a patient classification instrument. Rev. Latino-Am. Enfermagem. 2011;19(1):58-66.
• ¹¹
  Doody O, Slevin E, Taggart L. Focus group interviews in nursing research: part 1. Br J Nurs. 2013;22(1):16-9.
• ¹²
  Bardin L. Análise de conteúdo. São Paulo: Edições 70; 2011.
• ¹³
  Santos JLG, Prochnow AG, Silva DC, Silva RM, Leite JL, Erdmann AL. Prazer e sofrimento no exercício gerencial do enfermeiro no contexto hospitalar. Esc Anna Nery. 2013;17(1):97-103.
• ¹⁴
  Moorhead S, Johson M, Maas ML, Swanson E. Classificação de resultados de enfermagem (NOC). Rio de Janeiro (RJ): Elsevier; 2010.
• ¹⁵
  Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health. 2006;29:489-97.
• ¹⁶
  Hasson F. Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008-15.
• ¹⁷
  Polit DF, Beck CT, Owen SV. Focus on research methods is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459-67.
• ¹⁸
  Anne-Christine Rat A, Pouchot J, Guillemin F, Baumann M, Retel-Rude N, Spitz E, et al. Content of quality-of-life instruments is affected by itemgeneration methods. Int J Qual Health Care. 2007;19(6):390-8.
• ¹⁹
  Kempfer SS, Birolo IVB, Meirelles BHS, Erdmann AL. Reflexão sobre um modelo de sistema organizacional de cuidado de enfermagem centrado nas melhores práticas. Rev Gaúcha Enferm. 2010;31(3):562-6.
• ²⁰
  Clancy TR. Planning - what we can learn from complex systems science. J Nurs Adm. 2007;37(10):436-9.
• ²¹
  Clancy TR. Organizing - new ways to harness complexity. J Nurs Adm. 2007;37(12):534-6.
• ²²
  Kieft RAMM, Brouwer BBJM, Francke AL, Delnoij DMJ. How nurses and their work environment affect patient experiences of the quality of care: a qualitative study. BMC Health Serv Res. 2014;14:249-59.
• ²³
  Westbrook JI, Duffield C, Li L, Creswick NJ. How much time do nurses have for patients? a longitudinal study quantifying hospital nurses' patterns of task time distribution and interactions with health professionals. BMC Health Serv Res. 2011;11:319-31.
• ²⁴
  Kalisch BJ, Williams RA. Development and psychometric testing of a tool to measure missed nursing care. J Nurs Adm. 2009;39(5):211-9.

• 1
  Paper extracted from doctoral dissertation "Carga de trabalho e sua influência sobre os resultados do processo de cuidar - título inicial", presented to Faculdade de Medicina de São José do Rio Preto, São José do Rio Preto, SP, Brazil. Supported by Fundação de Apoio à Pesquisa e Extensão de São José do Rio Preto (FAPERP), Brazil, process #149/2015.

Publication Dates

• Publication in this collection
  July-Aug 2015

History

• Received
  16 Oct 2014
• Accepted
  01 Mar 2015