Knowledge of the care of wounded patients: evidence of validity of an instrument

Conocimiento acerca del cuidado de la persona com heridas: evidencias de validez del instrumento

ABSTRACT

Objective:

to present evidence of validity of an instrument aimed at evaluating the knowledge of nursing students about the care of the wounded patient, according to the difficulty and discrimination indexes of the items.

Method:

methodological study conducted in a nursing higher education institution with 117 undergraduate students and 38 professionals from a research group with experience in the area of wounds. For data collection, a questionnaire with 10 multiple choice questions was applied before and after classes on wounds.

Results:

most of the questions presented low level of difficulty and inefficient discrimination index, requiring a revision of the instrument. After two review stages, the difficulty and discrimination indexes of the instruments improved.

Conclusion:

an instrument with better evidence of validity was obtained. However, it still requires refinement for later revalidation in the same population.

Descriptors:
Nursing Assessment; Nursing Care; Validation Studies; Wounds and Injuries; Knowledge

RESUMEN

Objetivo:

identificar las evidencias de validez según los índices de dificultad y discriminación de los ítems de un instrumento con el fin de evaluar el conocimiento de académicos de enfermería en el cuidado de la persona lesionada.

Método:

se trata de un estudio metodológico desarrollado en una institución de enseñanza superior de enfermería realizado entre 117 estudiantes de graduación y 38 profesionales con experiencia en el área de heridas pertenecientes a un grupo de investigación. Se recogieron los datos mediante un cuestionario de 10 preguntas de opción múltiple, aplicado antes y después de las clases sobre heridas.

Resultados:

la mayoría de las cuestiones presentó un nivel bajo de dificultad y poder discriminativo ineficiente, haciéndose necesaria la revisión del instrumento. Tras dos etapas de revisión, el cuestionario demostró una mejoría de los índices de dificultad y discriminación.

Conclusión:

se logró un instrumento con mejor evidencia de validez, aunque tuvo que ser perfeccionado para su posterior revalidación en la misma población.

Descriptores:
Evaluación en Enfermería; Cuidados de Enfermería; Estudios de Validez; Heridas y Lesiones; Conocimiento

RESUMO

Objetivo:

Identificar as evidências de validade, segundo os índices de dificuldade e discriminação dos itens, de um instrumento para avaliação do conhecimento de acadêmicos de enfermagem no cuidado à pessoa com lesão.

Método:

Estudo metodológico desenvolvido em instituição de ensino superior de enfermagem com 117 estudantes de graduação e 38 profissionais componentes de grupo de pesquisa com experiência na área de feridas. Para coleta de dados, utilizou-se um questionário com 10 questões de múltipla escolha, aplicado antes e após as aulas de feridas.

Resultados:

A maioria das questões apresentou baixo nível de dificuldade e poder discriminativo ineficiente, sendo necessária a revisão do instrumento. Após duas etapas de revisão, o questionário apresentou melhora dos índices de dificuldade e discriminação.

Conclusão:

Obteve-se um instrumento com melhores evidências de validade, embora com necessidade de refinamento para posterior revalidação na mesma população.

Descritores:
Avaliação em Enfermagem; Cuidados de Enfermagem; Estudos de Validação; Ferimentos e Lesões; Conhecimento

INTRODUCTION

Cutaneous lesions or wounds are common in health services and are considered a public health problem in Brazil and in the world^(1-2). They have a significant economic impact on health systems and cause suffering and social isolation to the affected individual, also generating absenteeism and unemployment⁽³⁾.

The treatment can be long and complex, and it requires multi-professional work, protocols, specific knowledge, technical skills, articulation between the levels of care of the Unified Health System (SUS) and the active participation of the wounded patients and their relatives, all within a comprehensive perspective of care⁽⁴⁾.

Nurses have a prominent role in this process;they must evaluate the wounded patient, prescribe the most appropriate care and execute, guide and supervise nursing staff during dressing changes^(5-6).

The nurses’ level of knowledge and evidence-based practices are essential factors for an adequate care, both for the prevention and for treatment of wounds⁽⁵⁾. This should begin to be addressed during training in undergraduate courses⁽⁷⁾, and it is important to evaluate the knowledge of nursing students to identify deficiencies in undergraduate teaching and to propose improvements regarding this topic.

Several studies have demonstrated the lack of knowledge of nursing undergraduate students in important characteristics of the process of evaluation and treatment of wounded patients^(8-9). Studies also point that the undergraduate teaching offered does not prepare students for caring for the sepatients^(9-10).

To evaluate this knowledge, the instruments used must be reliable, based on scientific evidence, objective, valid and reflect various achievement levels^(11-12). An instrument is valid when its construction and applicability allow a true measurement of what they are intended to measure⁽¹³⁾. Through the analysis of the items of an instrument it is possible to verify if they properly evaluate the attribute to be measured.

An item analysis assesses the reliability and validity of a test by examining respondent’s performance regarding each question and applying statistical analyses to determine whether the item should be kept, reviewed or discarded from the test⁽¹²⁾.

Common item analysis parameters include the difficulty index, which reflects the percentage of correct answers to total responses; and the discrimination index, also known as the point biserial correlation, which identifies discrimination between students with different levels of achievement^(12,14).

These indexes can be used as validity criteria, according to the Classical Test Theory (CTT), an important psychometric theory used in the evaluation of the quality of psychological and educational tests^(15-16). These indexes are adopted as validity criteria in the present study.

This study is justified by the need to verify the ability of the instrument to measure knowledge regarding care of wounded patients. The instrument can later be used to evaluate this competence in undergraduate nursing students and nurses.

OBJECTIVE

To present evidence of validity of an instrument aimed at evaluating knowledge about the care of the person with cutaneous lesion,according to the difficulty and discrimination indexes of the items and with nursing academics and professionals with expertise in the subject.

METHOD

Ethical aspects

The research was developed following the guidelines of Resolution 466/12⁽¹⁷⁾. It obtained a favorable opinion from the Research Ethics Committee of the University Hospital Onofre Lopes, the Certificate of Presentation for Ethical Assessment (CAAE) No. 0002.0.294.000-10 and signature of the Informed Consent Term (TCLE) by study participants.

Design, area and period

Methodological study, developed at the Federal University of Rio Grande do Norte (UFRN), from April 2017 to April 2018.

Population and sample, inclusion and exclusion criteria

The target population was composed of students enrolled in the discipline Symptomology and Nursing Techniques, in the semesters 2017.1, 2017.2 and 2018.1, and professionals from the Research Group Incubator of Nursing Procedures (GPIPE) of UFRN, which works on the theme of wounds. The inclusion criterion for the selection of the students was being present in the theoretical and practical classes on the theme of wounds. The exclusion criterion was failing to answer the instrument before the theoretical classes or after the practical class. All students accepted to participate in the study, totaling 117 students (34 from class 1 in semester 2017.1, 43 from class 2 in semester 2017.2 and 40 from class 3 in semester 2018.1).

This discipline was selected because it addresses the basic content regarding skin wound care; therefore, it provided the ideal and opportune moment to test the instrument and to assess the students’ knowledge before and after the theoretical and practical classes on wound care.

The inclusion criteria for the participants from the Research Group were being a professional and participating in the UFRN GPIPE. The exclusion criterion was failing to answer the instrument in the time provided for consultation (15 days). Of the 45 professionals participating in the research group, 38 accepted to participate in the study.

Study protocol

The instrument used was the questionnaire⁽¹⁸⁾ “Nursing care for people with skin lesions”, composed of 10 multiple choice questions with 05 alternatives each divided in four domains: Patient and wound assessment (Questions 1, 2 and 3); Choice of dressing (Questions 4, 7 and 8); Dressing Technique (Questions 5 and 6) and Recording and orientation (Questions 9 and 10). The instrument was applied to the students in two moments, before the theoretical classes on wound care and after the practical class.

After applying the questionnaire in classes 1 and 2 and identifying the difficulty and discrimination indexes, it was found that the instrument needed to be reviewed. The review process occurred in two stages, each step followed by the application of the instrument with the undergraduate students to ascertain the results obtained. The first review was carried out by the researchers of the present study. The modified instrument was reapplied in class 2 (3rd collection in this class), as an evaluation activity referring to the wounds module, and sent to the GPIPE research group for professionals to respond to the instrument and suggest improvements. After analyzing and accepting the suggestions from the research group members, a second revision phase was conducted, resulting in the final version of the instrument, which was submitted to class 3 before and after wound classes.

Analysis of results and statistics

The difficulty and discrimination indexes, according to CTT, were used to investigate the validity of the instrument.

The difficulty index is the proportion of respondents who answered the item correctly.The classification proposed by Garret, as quoted by Almeida and Freire⁽¹⁹⁾, was adopted as parameter.The classification is as follows: difficulty index> 0.75 (Low difficulty level), from 0.25 to 0.75 (Medium difficulty level) and <0.25 (High difficulty level).

The Kelley’s method was used to calculate the discrimination index based on the difference between the scores of the most proficient participants (27% of the respondents with the highest scores) and the scores of the least proficient ones (27% of the respondents with the lowest scores)⁽¹²⁾. The classification adopted for each item was discrimination index<0.20 (inefficient, discard or totally review item), between 0.20 and 0.29 (needs revision), between 0.30 and 0.39 (acceptable, does not require revision) and ≥ 0.40 (satisfactory, should remain in the test), according to Ebel, as quoted by Maia⁽¹⁵⁾.

The Cronbach’sα coefficient was verified, with α> 0.70 being adopted as the ideal. The α coefficient, proposed by Cronbach (1951), is the most used method to measure reliability⁽²⁰⁾.

The data were organized in an electronic data sheet in Microsoft Excel and then exported to the statistical software SPSS 21.0. After coding and tabulation, the data were analyzed through descriptive statistics, distribution of the categorical variables of the profiles of the students from the three semesters and of the professionals in absolute and relative numbers and inferential analysis of the scalar variables of the questionnaire through the Kolmogorov-Smirnov Normality Test. The variables did not follow normality; therefore, the Sign Test was used to compare the scores of the ten questions before and after the classes given, and p ≤ 0.05 was adopted as statistical significance.

RESULTS

A total of 117 nursing students participated in the study. Most of the students were female (78.6%) and the age ranged from 18 to 48 years, with a mean of 22.5 years. The majority (76.9%) reported they had acquired some knowledge about wounds in the pathology discipline. However, 85.5% stated that they did not study this content, 53% reported they did not have the opportunity to care for or to observe the care of wounded patients and 88% did not participate in training or extracurricular courses on the subject.

In the stage conducted with professionals, 38 members of the research group participated. Most of them were female (89.5%) and the age ranged from 25 to 63 years, with a mean of 34.3 years. The majority (86.9%) reported they had access to content on wounds in the discipline Symptomology and Nursing Techniques, 73.7% said that they had deepened their knowledge on the subject and all the professionals surveyed had had the opportunity to care for people with cutaneous lesions. The majority (57.9%) participated in a theoretical course on wounds, were nurses (89.5%), had a doctorate degree or were pursuing one (68.4%), did not have a specialization in the area of wounds, and half of them worked with care and the other half in teaching.

Table 1 compares the two moments of application of the instrument in classes 1 and 2. Before the classes on wounds, six questions were classified with low level of difficulty and, after the classes, this number increased to nine, with only one question with medium difficulty. In addition, after the classes, eight questions of the instrument required a review of the discrimination index and questions 6, 7, 9 and 10 did not show significant differences in the Sign Test due to the high level of equal responses in the two moments of application of the questionnaire.

Table 1
Distribution of questions and domains according to difficulty and discrimination indexes before and after classes on wounds in classes 1 and 2, Natal, Rio Grande do Norte, Brazil, 2018

Knowledge questionnaire Domains/Questions	Before knowledge Classes 1 and 2		After knowledge Classes 1 and 2		Sign test			p value
	DIFI	DI	DIFI	DI	N	E	P
Wound assessment
Q1	0.69 (M)	0.57 (S)	0.94 (B)	0.10 (I)	02	54	21	< 0.001
Q2	0.81 (B)	0.38 (A)	1.00 (B)	0.00 (I)	00	62	15	< 0.001
Q3	0.79(B)	0.43 (S)	0.92 (B)	0.19 (I)	03	61	15	< 0.001
Choice of dressing
Q4	0.75 (M)	0.62 (S)	0.88 (B)	0.24 (R)	04	59	14	0.031
Q7	0.90 (B)	0.29 (R)	0.95 (B)	0.14 (I)	03	67	07	0.344
Q8	0.17 (AT)	0.10 (I)	0.64 (M)	0.67 (S)	03	35	39	< 0.001
Dressing technique
Q5	0.57 (M)	0.62 (S)	0.83 (B)	0.29 (R)	07	43	27	< 0.001
Q6	0.81 (B)	0.48 (S)	0.87 (B)	0.38 (A)	03	66	08	0.227
Recording and orientation
Q9	0.90 (B)	0.33 (A)	0.94 (B)	0.24 (R)	03	68	06	0.508
Q10	0.87 (B)	0.38 (A)	0.94 (B)	0.14 (I)	03	66	08	0.227
Total	0.72 (M)	0.42 (S)	0.89 (B)	0.24 (R)	06	17	54	< 0.001
Cronbach's Alpha	0.515		0.040		-

• Note: DIFI (Difficulty Index) DI (Discrimination Index), N (Negative), E (Equal), P (Positive). Difficulty Index: > 0.75 L (Low), from 0.25 to 0,75 M (Medium) and< 0.25 H (High) and Discrimination Index:< 0.20 I (Inefficient, discard or totally review), from 0.20 a 0.9 R (Review), from 0.30 to 0.39 A (Acceptable, does not require revision) ≥ 0.40 S (Satisfactory, keep item).

The Cronbach’s alpha coefficient showed that the instrument did not present good reliability, and the internal consistency was worse after the acquired knowledge.

Chart 1 presents the results obtained in class 2 after the classes on wounds and points out the changes made in the instrument in order to improve the difficulty and discrimination indexes.

Chart 1
Distribution of the questions and domains according to difficulty and discrimination indexes after classes on wounds in class 2 and alterations in the instrument, Natal, Rio Grande do Norte, Brazil, 2018

2nd application - After theoretical and practical classes Domains/Questions Index (DIFI ;DI)	Changes made to the questionnaire(18)
Wound assessment	The answer "all alternatives are correct" was replaced(21).
Q1 (0.95L; 0.009I)
Q2 (1.00L;0.00I)	The answer "all alternatives are correct" was replaced (21).
Q3 (0.95L;0.18I)	Distractors were improved to be similar to the correct alternative(21).
Choice of dressing	The answer "all alternatives are correct" was replaced(21).
Q4 (0.88L;0.18I)
Q7 (0.95L;0.09I)	It was renamed as Q5 so that questions about the same domain are in sequence in the instrument(22). The answer "all alternatives are correct" was replaced(21), the terms "any" and "only" were removed from the alternatives(21)and the distractors were improved to be similar to the correct alternative(21).
Q8 (0.55M;0.91S)	It was renamed as Q6 so that questions about the same domain are in sequence in the instrument(22). The correct alternative and one of the distractors were altered so that all alternatives are similar(21).
Dressing technique	It was renamed as Q7 so that questions about the same domain are in sequence in the instrument(22). Distractors were improved to be similar to the correct alternative(21).
Q5 (0.90L; 0.18I)
Q6 (0.79L; 0.64S)	It was renamed as Q6 so that questions about the same domain are in sequence in the instrument(22). Distractors were improved to be similar to the correct alternative(21).
Recording and orientation	The option that considered all the alternatives as correct was replaced(21).
Q9 (0.98L; 0.00I)
Q10 (0.98L; 0.00I)	Distractors were improved(21).
Total (0.89L; 0.23R)	The content of the instrument was not modified, but all the questions were reviewed.

• Note: DIFI (Difficulty Index) DI (Discrimination Index); Difficulty Index: > 0.75 L (Low), from 0.25 to 0,75 M (Medium) and <0.25 H (High) and Discrimination Index:< 0.20 I (Inefficient, discard or totally review), from 0.20 a 0.9 R (Review), from 0.30 to 0.39 A (Acceptable, does not require revision) ≥ 0.40 S (Satisfactory, keep item).

It was verified that only Q8 presented medium difficulty and no question was considered difficult. As to the discrimination index, eight questions presented inefficient discrimination, and the removal or total review of these questions is recommended.

Table 2 shows the results obtained after the first review of questions and reapplication in class 2 for the evaluation of the wounds module (3rd collection in this class).

Table 2
Distribution of questions and domains according to difficulty and discrimination indexes after reapplication in class 2, Natal, Rio Grande do Norte, Brazil, 2018

Knowledge questionnaire Questions/Domains	3rd application Class 2
	DIFI	DI
Wound assessment
Q1	0.40(M)	0.73(S)
Q2	0.95(B)	0.18(I)
Q3	0.88(B)	0.18(I)
Choice of dressing
Q4	0.76(B)	0.45(S)
Q7	0.26(M)	0.64(S)
Q8	0.67(M)	0.55(S)
Dressing technique
Q5	0.62(M)	0.73(S)
Q6	0.79(B)	0.18(I)
Recording and orientation
Q9	0.83(B)	0.18(I)
Q10	0.86(B)	0.00(I)
Total	0.70(M)	0.38(A)

• Note: DIFI (Difficulty Index) DI (Discrimination Index); Difficulty Index: > 0.75 L (Low), from 0.25 to 0,75 M (Medium) and <0.25 H (High) and Discrimination Index:< 0.20 I (Inefficient, discard or totally review), from 0.20 a 0.9 R (Review), from 0.30 to 0.39 A (Acceptable, does not require revision) ≥ 0.40 S (Satisfactory, keep item).

After making the changes in the instrument, the number of questions with medium difficulty increased from one to four questions in the same group of respondents (Group 2). Regarding the discrimination index, the number of questions with satisfactory index increased from two to a total of five; however, none presented high difficulty and some still needed to be revised.

Table 3 shows the results obtained after the first review of the questions and application in the research group.

Table 3
Distribution of questions and domains according to difficulty and discrimination index after application in the research group, Natal, Rio Grande do Norte, Brazil, 2018

Knowledge questionnaire Questions/Domains	Research Group
	DIFI	DI
Wound assessment
Q1	0.39 (M)	0.20(R)
Q2	1.00(B)	0.00 (I)
Q3	0.84(B)	0.30(A)
Choice of dressing
Q4	0.68(M)	0.50(S)
Q7	0.29(M)	0.60(S)
Q8	0.92(B)	0.00(I)
Dressing technique
Q5	0.47(M)	0.90(S)
Q6	0.92(B)	0.10 (I)
Recording and orientation
Q9	0.74(M)	0.20 (R)
Q10	0.68(M)	0.70 (S)
Total	0.69(M)	0.35 (A)

• Note: DIFI (Difficulty Index) DI (Discrimination Index); Difficulty Index: > 0.75 L (Low), from 0.25 to 0,75 M (Medium) and <0.25 H (High) and Discrimination Index:< 0.20 I (Inefficient, discard or totally review), from 0.20 a 0.9 R (Review), from 0.30 to 0.39 A (Acceptable, does not require revision) ≥ 0.40 S (Satisfactory, keep item).

There is a predominance of questions of medium difficulty (06 questions) and satisfactory discrimination index (04 questions). However, none of the questions presented a high level of difficulty and some still required revision.

Given the need to review some questions in order to increase the degree of difficulty and the discrimination index, the already modified instrument received new modifications suggested by the research group, as shown in Chart 2.

Chart 2
Suggestions of alterations made by the research group and procedures adopted and justification, Natal, Rio Grande do Norte, Brazil, 2018

Question	Suggestion from professionals (n)	Procedure adopted/Justification
Q1	Remove "No" from question (3).	Removed/Requesting a wrong answer results in lack of clarity(21).
	Clearer alternatives with a single correct alternative (8).	Terms were added to the alternatives, providing greater clarity and a single correct answer(21).
Q2	Remove "No" from question(7).	Removed/ Requesting a wrong answer results in lack of clarity(21).
	Improve clarity and specificity in the question and in the alternatives(3).	The term "skin wound" was replaced by "skin wound bed", making the question more clear and specific.
Q3	Standardize the number of signs/symptoms in the alternatives (1).	Done, making the alternatives more similar(21).
	Specify the type of exudate in all alternatives (2).	Done/The type of necrosis was also specified, making the alternatives more clear, complete and similar (21).
Q4	Remove the term "except" (3).	Removed/Requesting a wrong answer results in lack of clarity (21).
	Clearer alternatives with a single correct alternative (3).	Terms were added to the alternatives, providing greater clarity and a single correct answer (21).
Q5	Remove the initial verbs from the alternatives (3).	Removed. It allows grammatical agreement and agreement of meaning between the question and the alternatives, avoiding the excess of verbs(21).
	Review the alternatives so that there is only one correct alternative(2).	Two alternatives that were correct were altered, providing a single true alternative (21).
Q6	Specify the type of debridement(2).	Not done, as this was not the purpose of the question.
	Put all the debriders into a single alternative (3).	One more debrider was added in each alternative so that all are similar(21). All of them were not put into a single alternative to increase the difficulty of the question.
Q7	Replace the term curative with dressing when referring to the product used in the wound bed (1).	Replaced. The term curative is broad, referring not only the product/dressing, but to the set of care given to a wound(23).
	Use technical terms such as "perilesional tissue" and "lesional", instead of terms such as "toward the inside of the wound" (5).	Used. The use of the different terms in nursing records makes it difficult to retrieve information, impairing the measurement of results in nursing practice, as well as the possibility of constructing evidence (24).
	Use the term "preferably" when indicating the use of saline to clean the lesion (1).	Used. Several studies recommend different cleaning solutions; however, saline solution (0.9%) is the preferred solution for wound cleaning and it can be used safely(25).
Q8	Remove the term "independent"(2).	Removed, because it can induce the right answer(21).
	Use term "exudate" instead of "secretion" (3).	Used. The use of the different terms in nursing records makes it difficult to retrieve information, impairing the measurement of results in nursing practice, as well as the possibility of constructing evidence (24).
Q9	Review the alternatives so that there is only one correct alternative(4).	All alternatives were altered, providing greater clarity and a single correct answer(21).
Q10	Review the alternatives so that there is only one correct alternative (4).	Terms were added to the alternatives so that only one can be considered correct(21).

Table 4 compares the two moments of application of the final version of the instrument in class 3. There is a better distribution of the questions regarding the level of difficulty, since five questions presented medium level of difficulty, three had a low level of difficulty and two presented high difficulty in the first moment. The number of questions with inefficient discrimination index was also reduced. In addition, questions 6 and 7 started to present a significant difference in the Sign Test, along questions 1, 2 and 8.

Table 4
Distribution of questions and domains of the final version according to difficulty and discrimination indexes before and after classes on wounds in class 3, Natal, Rio Grande do Norte, Brazil, 2018

Knowledge questionnaire Domains/Questions	Before knowledge Class 3		After knowledge Class 3		Sign test			p value
	DIFI	DI	DIFI	DI	N	E	P
Wound assessment
Q1	0.58 (M)	0.64 (S)	0.93 (B)	0.09 (I)	00	26	14	< 0.001
Q2	0.15 (AT)	0.45 (S)	0.63 (M)	0.36 (A)	01	19	20	< 0.001
Q3	0.50 (M)	0.55 (S)	0.70 (M)	0.64 (S)	03	26	11	0.057
Choice of dressing
Q4	0.78 (B)	0.45 (S)	0.93 (B)	0.18 (I)	02	30	08	0.109
Q7	0.73 (M)	0.18 (I)	0.95 (B)	0.09 (I)	01	29	10	0.012
Q8	0.28 (M)	0.73 (S)	0.65 (M)	0.64 (S)	05	15	20	0.004
Dressing technique
Q5	0.33 (M)	0.27 (R)	0.48 (M)	1.00 (S)	04	26	10	0.180
Q6	0.10 (AT)	0.18 (I)	0.50 (M)	0.82 (S)	01	22	17	< 0.001
Recording and orientation
Q9	0.80 (B)	0.27 (R)	0.93 (B)	0.18 (I)	01	33	06	0.125
Q10	0.95 (B)	0.18 (I)	1.00 (B)	0.00 (I)	00	38	02	0.500
Total	0.52 (M)	0.39 (A)	0.77 (B)	0.40 (S)	01	04	35	< 0.001
Cronbach's alpha	0.405		0.517		-

• Note: DIFI (Difficulty Index) DI (Discrimination Index), N (Negative), E (Equal), P (Positive). Difficulty Index: > 0.75 L (Low), from 0.25 to 0,75 M (Medium) and <0.25 H (High) and Discrimination Index:< 0.20 I (Inefficient, discard or totally review), from 0.20 a 0.9 R (Review), from 0.30 to 0.39 A (Acceptable, does not require revision) ≥ 0.40 S (Satisfactory, keep item).

The Cronbach’s alpha coefficient showed an increase in the reliability and internal consistency of the instrument; however, it was still below the ideal value.

DISCUSSION

The results obtained with the application of the instrument in classes 1 and 2 demonstrated a high number of questions with low level of difficulty. This result leads us to believe that the way the questions were elaborated contributed to the fact that they were easy. One example is the fact that five questions of the instrument had the alternative “all alternatives are correct”, and in four of them this was the option that should be selected by the respondents, as well as the presence of terms like “any” and “only”, which can induce the correct answer and mislead the respondent only by the presence of those terms, which should be avoided⁽²¹⁾.

Studies point to a decrease in the reliability of questions that use the alternative “all alternatives are correct”⁽²¹⁾ as a response, corroborating the data of the present research, which showed extremely low reliability and internal consistency in the application of the initial instrument, especially after the classes.

Regarding the criterion of balance of the level of difficulty of the questions, it is recommended that a test presents 50% of medium difficulty questions, since this level of difficulty allows the maximum differentiation of the individual performances, 25% of low difficulty questions, with the objective of differentiating the individuals with low performance and 25% of high difficulty questions, for the differentiation of the most skilled subjects, according to Garret, as cited by Almeida and Freire⁽¹⁹⁾.

In none of the moments of application of the instrument in classes 1 and 2 this balance was observed. There was a predominance of questions of low difficulty, results that are divergent from similar studies^(12,26).

Regarding the discrimination index, eight questions of the initial instrument⁽¹⁸⁾ required revision after the classes and five of them had inefficient discrimination indexes (eight if we consider only the results of class 2). These questions should be removed from the instrument or totally reviewed.

As the instrument was previously constructed and validated in terms of its content⁽¹⁸⁾ and only presented 10 items, it would not be appropriate to remove questions, with the risk of obtaining a very small instrument that did not include all content and domains of the issue addressed. Therefore, the questions were reviewed.

This way, all “all alternatives are correct” options were replaced, the words “only” and “any” were removed and the distractors of various questions were altered.

Distractors, as the name suggests, are incorrect answers that have the function of attracting those who do not know and choose the answer that seems right at sight. The distractors must be similar to the correct alternative in terms of content and grammatical structure, being more effective distractors and providing a better discriminative power⁽²¹⁾.

In addition, the questions were reorganized so that items on the same domain were in sequence in the instrument. Therefore, the questions were organized in their respective domains⁽²²⁾ and the correct alternatives were redistributed in varied positions to avoid response biases⁽²¹⁾.

Question 8, concerning the Choice of Dressing domain, was the only question of high difficulty in the first moment and of medium difficulty in the second. It is a very objective and specific question related to the debriders used in the topical treatment of wounds and it had alternatives very similar in content, complexity and extension of text.

As in the present research, this was also the domain with the largest number of incorrect answers in other studies performed with nursing students. This may be related to the continuous development of new products for the treatment of wounds, which is making health professionals feel insecure about which option is the most appropriate for each type of injury and requiring constant updating of students and professionals⁽⁸⁾.

Question 6, in the dressing technique domain, question 7, in the choice of dressing domain and both questions in the recording and orientation domain did not present a significant difference when comparing the two application moments of the instrument in classes 1 and 2, due to a high number of correct answers in the first moment, indicating the need to review these questions in order to allow a better assessment of these areas.

Knowledge about how to perform the dressing procedure improves the students’ performance when caring for the person with skin wound, promoting safe and efficient care⁽²⁷⁾. Therefore, it is important to have an adequate evaluation in this item.

Regarding the domain recording and orientation, a study⁽²⁸⁾ reports the importance of documenting and registering the follow-up of the individual with a wound. This study also highlights the importance of continuing educational work involving health professionals, patients, relatives and caregivers. Therefore, it is extremely important that the instrument used to evaluate the knowledge about the care of the wounded patient includes questions that can in fact measure the knowledge of the nursing student in this subject.

Studies^(10,29-30) show that nursing students and nurses have limited knowledge and skills regarding the care of individuals with cutaneous lesions. For this reason, wounds that could be treated early develop into a chronic condition that is difficult to heal, resulting in increased treatment costs and impairments to the quality of life of the individuals affected.

The identification of these limitations allows reviewing fragile points, providing support to improve teaching and guarantee a better-quality nursing care, with a satisfactory evolution of wounds resulting in closure or reduction and in a better quality of life for these individuals.

After the first review of the instrument and reapplication in class 2, there was an increase in the number of questions with median difficulty, with a better balance of the difficulty of the questions, according to Garret, cited by Almeida and Freire⁽¹⁹⁾. However, none of the questions presented a high degree of difficulty. The same occurred when applying the instrument to the group of professionals.

There was also an increase in the number of questions with satisfactory discrimination index. However, five questions still presented inefficient discrimination, requiring continuity of the revision process.

After the presentation of the instrument to the professionals, one of the most common suggestion was the removal of the terms “no” and “except” from the questions. Negative phrases easily result in lack of clarity and requesting an incorrect answer is something that should be avoided⁽²¹⁾.

In addition, some questions of the modified instrument presented more than one correct alternative and required greater clarity and specificity. An objective question must present only one correct alternative and must be direct, clear and precise⁽²¹⁾.

After implementing the changes and applying the final version of the instrument to class 3, the instrument obtained a better distribution of questions regarding the level of difficulty in the first moment, in agreement with Garret, cited by Almeida and Freire⁽¹⁹⁾. The number of questions with inefficient discrimination indexes also decreased, indicating an improvement in the quality and validity of the instrument.

Questions 6 and 7 started to present significant differences, along with questions 1, 2 and 8, indicating the increase of knowledge in these issues and the ability of the instrument to measure this attribute.

In addition, there was an improvement in the reliability and internal consistency of the questionnaire after implementing the changes. However, the value obtained was still below ideal, which reinforces the need for more changes in the instrument studied. It is worth mentioning that, even if an instrument is considered valid and reliable, it can still be improved in future studies, since there are no definitive and perfect instruments, especially when it comes to educational evaluation⁽¹¹⁾.

Limitations of the study

One of the limitations of this research is the lack of other studies in the literature that contain validated instruments to evaluate nursing undergraduates’ knowledge about the care of wounded patients, or studies on item analysis, making it difficult to make comparisons. Another limitation is the sample composed by students from only one educational institution and professionals who are not specialists in the area, despite their experience in the subject. It would be relevant to apply the instrument to students from other educational institutions in the country and to specialists in future studies.

Contributions to the area of nursing, health or public policy

By presenting evidences of validity of the instrument in the target population, the research allowed the improvement of the quality of the instrument and the identification of the need for adjustments for later revalidation in the same population. The use of the validated instrument may contribute to the professional training of nurses, to the identification of gaps that should be overcome in the teaching-learning process and to the improvement of the care provided to the individual with a wound.

CONCLUSION

The investigation of validity of the instrument, carried out through the analysis of the difficulty and discrimination indexes of the items, demonstrated the need to review most of the questions in order to increase the degree of difficulty and the discrimination index of the instrument, improving its quality and ability to measure knowledge.

After the two review stages, it was possible to increase the degree of difficulty and the discrimination index of some questions, which resulted in an instrument with better evidence of validity. However, it still needs refinement for later revalidation in the same population.

The process of analysis of the questionnaire and the stages of review allowed a discussion about the items’ difficulty and discrimination and about the process of construction of multiple-choice questions suitable for the evaluation of knowledge. These subjects had not yet been discussed in the area of health and in nursing, despite their importance for the construction of effective and objective evaluation instruments.

There are also few studies that apply validated instruments to evaluate the knowledge of nursing students in the area of wounds, adopting indexes for measuring quality.

Further studies are recommended in order to improve this instrument. It is also important to apply it to the target audience in other educational institutions.

Thus, the instrument may favor the evaluation process and the professional training of nurses, identifying gaps that must be overcome in the teaching-learning process and prompting new studies to improve the instrument.

REFERENCES

• ¹
  Azevedo IC, Costa RKS, Ferreira Jr MA. Perfil da produção científica da enfermagem nacional sobre feridas. Rev Cubana Enferm [Internet]. 2018 [cited 2018 Aug 4];34(1). Available from: http://www.revenfermeria.sld.cu/index.php/enf/article/view/1440/339
  » http://www.revenfermeria.sld.cu/index.php/enf/article/view/1440/339
• ²
  Barros MPL, Ferreria PJO, Maniva SCFJ, Holanda RE. Caracterização de feridas crônicas de um grupo de pacientes acompanhados no domicílio. Rev Interdisc. 2016 [cited 2019 Jan 10];9(3):1-11. Available from: https://revistainterdisciplinar.uninovafapi.edu.br/index.php/revinter/article/view/926
  » https://revistainterdisciplinar.uninovafapi.edu.br/index.php/revinter/article/view/926
• ³
  Nogueira GA, Oliveira BGRB, Santana RF, Cavalcanti ACD. Nursing diagnoses in patients with chronic venous ulcer: observational study. Rev Eletr Enf. 2015;17(2):333-9. doi: https://doi.org/10.5216/ree.v17i2.28782
  » https://doi.org/10.5216/ree.v17i2.28782
• ⁴
  Liberato SMD, Araújo RO, Souza AJG, Marconato AMP, Costa IKF, Torres GV. Adesão ao tratamento de pessoas com úlceras venosas atendidas na atenção primária à saúde. Aquichan. 2017;(2):128-39. doi: http://dx.doi.org/10.5294/aqui.2017.17.2.2
  » http://dx.doi.org/10.5294/aqui.2017.17.2.2
• ⁵
  Zarchi K, Latif S, Haugaard VB, Hjalager IRC, Jemec GB. Significant differences in nurses' knowledge of basic wound management: implications for treatment. Acta Derm Venereol. 2014;94(4):403-7. doi: 10.2340/00015555-1770
  » https://doi.org/10.2340/00015555-1770
• ⁶
  Santos EI, Oliveira JGAD, Ramos RS, Silva ACSS, Belém LS, Silva AL. Facilidades e dificuldades à autonomia profissional de enfermeiros no cuidado de pessoas com feridas: estudo de representações sociais. Estima. 2017;15(1):3-9. doi: http://dx.doi.org/10.5327/Z1806-3144201700010002
  » http://dx.doi.org/10.5327/Z1806-3144201700010002
• ⁷
  Lopes CM, Andrade EMLR, Luz MHBA. Conhecimento de graduandos de enfermagem sobre úlcera por pressão. Enferm Foco 2015;6(1/4):24-30. doi: https://doi.org/10.21675/2357-707X.2015.v6.n1/4
  » https://doi.org/10.21675/2357-707X.2015.v6.n1/4
• ⁸
  Baratieri T, Sangaleti CT, Trincaus MR. Conhecimento de acadêmicos de enfermagem sobre avaliação e tratamento de feridas. Rev Enferm Atenção Saúde [Internet]. 2015 [cited 2018 Aug 8];4(1):2-15. Available from: http://seer.uftm.edu.br/revistaeletronica/index.php/enfer/article/view/1259
  » http://seer.uftm.edu.br/revistaeletronica/index.php/enfer/article/view/1259
• ⁹
  Kielo E, Salminem L, Stolt M. Graduating student nurses' and student podiatrists' wound care competence - an integrative literature review. Nurse Educ Pract. 2018;29:1-718. doi: 10.1016/j.nepr.2017.11.002
  » https://doi.org/10.1016/j.nepr.2017.11.002
• ¹⁰
  Faria GBG, Prado TN, Lima EFA, Rogenski NMB, Borghardt AT, Massaroni L. Knowledge and practice of nurses on the care of wounds. Rev Enferm UFPE. 2016 [cited 2018 Aug 4];10(12):4532-8. Available from: https://periodicos.ufpe.br/revistas/revistaenfermagem/article/view/11520
  » https://periodicos.ufpe.br/revistas/revistaenfermagem/article/view/11520
• ¹¹
  Salerno MR, Lopes MHI, Mattiello R, Icaza ES, Antonello IC. Validação de instrumento de avaliação de disciplinas de escola médica no Brasil. Rev Bras Educ Med. 2016;40(4):644-52. doi: 10.1590/1981-52712015v40n4e01702015
  » https://doi.org/10.1590/1981-52712015v40n4e01702015
• ¹²
  Kheyami D, Jaradat A, Al-Shibani T, Ali FA. Item analysis of multiple choice questions at the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos Univ Med J. 2018;18(1):e68-e74. doi: 10.18295/squmj.2018.18.01.011.
  » https://doi.org/10.18295/squmj.2018.18.01.011
• ¹³
  Silva RLDT, Ishida EE, Cordoni Jr L, Marcon SS. Construction and validation of an instrument to evaluate assistance to the person with hypertension in primary care. Texto Contexto Enferm. 2017;26(4):e0500014. doi: 10.1590/0104-07072017000500014
  » https://doi.org/10.1590/0104-07072017000500014
• ¹⁴
  Mahjabeen W, Alam S, Hassan U, Zafar T, Butt R, Konain S, et al. Difficulty index, discrimination index and distractor efficiency in multiple choice questions. Ann Pak Inst Med Sci [Internet]. 2017 [cited 2018 Aug 4];13(4):310-5. Available from: https://www.apims.net/index.php/apims/article/view/9
  » https://www.apims.net/index.php/apims/article/view/9
• ¹⁵
  Maia JL. Uso da teoria clássica dos testes - TCT e da teoria de resposta ao item - TRI na avaliação da qualidade métrica de testes de seleção [Tese] [Internet]. Ceará: Universidade Federal do Ceará; 2009 [cited 2018 Aug 4]. Available from: http://www.repositorio.ufc.br/handle/riufc/3235
  » http://www.repositorio.ufc.br/handle/riufc/3235
• ¹⁶
  Pasquali L. Psicometria: teoria dos testes na psicometria e na educação. 5ª ed. Petrópolis: Vozes; 2013.
• ¹⁷
  Brasil. Ministério da Saúde. Resolução nº 466, de 12 de dezembro de 2012. Aprova as diretrizes e normas regulamentadoras de pesquisas envolvendo seres humanos e revoga as Resoluções CNS nºs. 196/96, 303/2000 e 404/2008 [Internet]. Brasília; 2012 [cited 2018 Aug 8]. Available from: http://bvsms.saude.gov.br/bvs/saudelegis/cns/2013/res0466_12_12_2012.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/cns/2013/res0466_12_12_2012.html
• ¹⁸
  Costa RKS, Torres GV, Salvetti MG, Azevedo IC, Costa MAT. Validity of instruments used in nursing care for people with skin lesions. Acta Paul Enferm. 2014;27(5):447-57. doi: 10.1590/1982-0194201400074
  » https://doi.org/10.1590/1982-0194201400074
• ¹⁹
  Almeida LS, Freire T. Metodologia da investigação em psicologia e educação. 5ª ed. Braga: Psiquilíbrios Edições; 2017.
• ²⁰
  Bem AB, Lanzer EA, Tambosi Filho E, Sanchez OP, Bernardi Jr P. Validade e confiabilidade de instrumento de avaliação da docência sob a ótica dos modelos de equação estrutural. Avaliação (Campinas; Sorocaba). 2011;16(2):375-401. doi: 10.1590/S1414-40772011000200008
  » https://doi.org/10.1590/S1414-40772011000200008
• ²¹
  Haladyna TM, Downing SM, Rodriguez MC. A review of multiple choice item writing guidelines for classroom assessment. Appl Meas Educ. 2002;15(3):309-34. doi: 10.1207/S15324818AME1503_5
  » https://doi.org/10.1207/S15324818AME1503_5
• ²²
  Coluci MZO, Alexandre NMC, Milani D. Construção de instrumentos de medida na área da saúde. Ciênc Saúde Colet. 2015;20(3):925-36. doi: 10.1590/1413-81232015203.04332013
  » https://doi.org/10.1590/1413-81232015203.04332013
• ²³
  Brasil. Ministério da Saúde, Secretaria de Políticas de Saúde, Departamento de Atenção Básica. Manual de condutas para úlceras neurotróficas e traumáticas [Internet]. Brasília; 2002 [cited 2018 Aug 4]. Available from: http://bvsms.saude.gov.br/bvs/publicacoes/manual_feridas_final.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/manual_feridas_final.pdf
• ²⁴
  Gomes DC, Cubas MR, Pleis LE, Shmeil MAH, Peluci APVD. Terms used by nurses in the documentation of patient progress. Rev Gaúcha Enferm. 2016;37(1):e53927. doi: 10.1590/1983-1447.2016.01.53927
  » https://doi.org/10.1590/1983-1447.2016.01.53927
• ²⁵
  Santos E, Queirós P, Cardoso D, Cunha M, Apóstolo J. A eficácia das soluções de limpeza para o tratamento de feridas: uma revisão sistemática. Rev Enf Ref. 2016;IV(9):133-44. doi: 10.12707/RIV16011
  » https://doi.org/10.12707/RIV16011
• ²⁶
  Menon AR, Kannambra PN. Item analysis to identify quality multiple choice questions. NJLM. 2017;6(2):7-10. doi: 10.7860/NJLM/2017/25690:2215
  » https://doi.org/10.7860/NJLM/2017/25690:2215
• ²⁷
  Costa RKS, Torres GV, Salvetti MG, Azevedo IC, Costa MAT. Instrument for evaluating care given by undergraduate nursing students to people with wounds. Rev Esc Enferm USP. 2015;49(2):315-23. doi: 10.1590/S0080-623420150000200018
  » https://doi.org/10.1590/S0080-623420150000200018
• ²⁸
  Dantas DV, Torres GV, Dantas RAN. Assistência aos portadores de feridas: caracterização dos protocolos existentes no Brasil. Cien Cuid Saúde 2011;10(2):366-72. doi: 10.4025/cienccuidsaude.v10i2.8572
  » https://doi.org/10.4025/cienccuidsaude.v10i2.8572
• ²⁹
  Costa RKS, Azevedo IC, Torres GV, Costa MAT, Salvetti MG. Graduandos de enfermagem: conhecimento sobre o cuidado à pessoa com lesão cutânea. Rev Enferm UFPI. 2016;5(1):10-6. doi: 10.26694/reufpi.v5i1.5016
  » https://doi.org/10.26694/reufpi.v5i1.5016
• ³⁰
  MattiaI BJ, KlebaI ME, Prado ML. Nursing training and professional practice: an integrative review of literature. Rev Bras Enferm. 2018;71(4):2039-49. doi: 10.1590/0034-7167-2016-0504
  » https://doi.org/10.1590/0034-7167-2016-0504

Publication Dates

• Publication in this collection
  21 Oct 2019
• Date of issue
  Nov-Dec 2019

History

• Received
  08 Aug 2018
• Accepted
  10 Apr 2019