Content analysis of a technology to the nursing diagnostic reasoning

Objective: to undergo a content analysis of a metacognitive strategy of indirect assessment in the pre-encounter with the client. Method: methodological study. Agreement and inter-rater reliability index for the criteria: relevance, adequacy, clarity, conciseness and accuracy of a technology to the nursing diagnosis reasoning for novices through an electronic form. The sample consisted of 13 raters. Data were analyzed using descriptive statistics. Results: high agreement and inter-rater reliability for 85 items related to data collection stage and the strategy description. Only fi ve items did not reach the validation criteria and must be rewritten. Conclusion: indirect assessment of the pre-encounter is relevant to the diagnostic reasoning process, being possible to develop competencies and diagnostic skills in the novice through strategies, proposals on innovative technology in the form of a diagram.


INTRODUCTION
The process of nursing diagnosis reasoning has been the subject of research by different conceptual and methodological approaches (1)(2)(3)(4) .Being a complex process involving the interpretation of the responses in relation to health and the problems that compete with it, are by nature subjective impressions of the diagnostician, sustained by knowledge, experience, personal beliefs and theories.In this sense, the diagnostic reasoning when fully conducted in a non-structured manner is prone to higher risk of scatter or misjudgement.
Diagnostic language of nursing practice are classifications that provide a structure for the organization of the content concerning the diagnosis as phenomena of interest to nursing subject because they provide resources to think about existing knowledge and, if necessary, the reorientation (5) .
In order to achieve this organized thinking, we highlight the importance of using clinical data collection tools that provide formalized structures to guide the specific types of data to be collected and processed, driven to a context and circumstance.The structures mentioned organize the clinical content of the diagnoses; however, they do not include a strategic plan concerning the process of handling clinical data by the diagnostician.
Thus, the authors of this study are interested in the validation of a technology for analysis of nursing reasoning directed to novices in this task.The technology firstly built in a master's dissertation has been going through refining and validation processes by experts.Grounded in a cognitive and metacognitive framework, the technology -presented in the form of a diagram -is temporally cut by the synchronous encounter event of the diagnostician with the client (pre-encounter, encounter and post-encounter), on the stage of collecting and processing data for diagnostic strategies contained in mental, emotional and behavioral operations that define and describe the strategies in tactical and operational level (6) .
The strategies are conceptualized as: plans concerning the actions, techniques and procedures that rely on subjects in the process of diagnosis, but that must have their characteristics recognized by the subjects.Therefore, considering the epistemological influence of metacognition in control and monitoring of cognitive properties in order to qualify the diagnostic strategy, so the subjects should know, even partially, the purposes, circumstances, reasons and mode of use of a given strategy.So, they are central to the process of learning and development of the nursing diagnosis for novices, the aspects concerning awareness, monitoring and regulation of knowledge and beliefs, recognizing affective aspects and their influences to the development of diagnostic skills (7)(8) .
Mental, affective and behavioral operations involve specific actions resulting from the planning and that are developed in the situated and contextualized diagnostic task.In the validation technology, there are some actions proposed to the novice such as leading the diagnostic strategy, however, regulation and implementation in the individual situation depend on semantic, procedural and clinical knowledge that are obtained by training and professional practice.
In general, it is understood that the phase of pre-encounter has received less attention in the diagnosis process when compared to encounter (direct interaction event with the subject of diagnosis) and post-encounter with the subject of the nursing diagnosis.In it are contained actions that are sometimes understood only as preparatory or management character, such as the shift change, and reading patients charts and other written reports, from the participation in rounds.

OBJECTIVE
This study aims to submit a metacognitive strategy of indirect assessment in the pre-encounter of the nursing diagnosis reasoning process to content analysis.

METHOD
Methodological development study for content validation by raters, using analytical procedures for reaching reliability.
A Delphi application was planned to incorporate rounds depending on the raters agreement index, regarding the elements concerning the content to be validated, considering the high values of agreement, additional rounds were not necessary.
Data collection from raters was developed by applying an electronic form via the internet.Invitations and access to the form, the use of Consent Form (CF) and other contacts made during the study and the feedback were given through individual or institutional e-mail, allowing the selection of participants from various regions of Brazil.The contacts were made individually, without the creation of messaging group via email, ensuring anonymity and confidentiality.
The search and the selection of raters were held in the Plataforma Lattes of the website of the National Council for Scientific and Technological Development (CNPq), in the directories of nursing research groups focusing on registered diagnostic reasoning, from their research lines and their Lattes Curriculum*.
From the initial selection, seventy seven raters were invited, twenty two agreed to participate, of which thirteen completed the data collection instrument.
The criteria for selection of raters were adapted from the scoring system proposed by Fehring (9) , specially developed to meet the needs of the study, to include nurses, Brazilian, professors or assistants, with expertise in the process of teaching and learning of nursing diagnosis, clinical assessment or diagnostic reasoning, with a score between 5 and 12 points.The raters were stratified according to score considering the degree of experience, the third group (most experienced) 12 and 11 points; the second group had 10 to 7 points; and the first group (less experienced) 5 points.
The sample size was estimated by applying a binomial statistical test.It was adopted 90% confidence interval, the expected agreement of the raters with the adequacy of the item in 0.90 (ideal) and an acceptable sampling error of 15%.For the adopted parameters, the minimum sample size required would be 11 participants (10) , which was overcome with 13 participants.
The form for data collection was provided to participants through Google Drive ® , containing 10 items regarding the relevance, adequacy, clarity, conciseness and accuracy of data collection stage in the pre-encounter and the indirect assessment strategy of the client, 80 items in relation to the criteria of adequacy, clarity, conciseness and accuracy of mental, behavioral and affective operations in relation to the purposes, circumstances, reasons and explanations to be validated in the proposed strategy.
A simple Likert scale measurement was used, where each participant established the number that best expressed their opinion in relation to statements about the description of the instrument in relation to established criteria.
The scores were established in order of increasing values, 1-6 according to the level of agreement or disagreement, such as: 1 -strongly disagree/I am totally against; 2 -disagree/I am against; 3 -Slightly disagree/I am slightly against; 4 -agree with restrictions/I am in favor with restrictions; 5 -agree/I am in favor; 6 -strongly agree/I am fully in favor.
The possibility of indecision was deleted, removing the central tendency of the responses.Each question allowed only one value to be assigned independently and as mandatory in the form.There was also a box for notes from raters, if they thought it was necessary.
Data collection occurred in the period from June to September 2013, it was organized in spreadsheets in the Microsoft Excel 2010 ® , which were estimated the percentage of agreement to the criteria of relevance to the diagnosis process.To measure the agreement of the content validity criteria of the phase, strategy and operations were assessed through the Content Validity Index (CVI) among the raters for adequacy, clarity, conciseness and precision.The CVI was calculated by dividing the sum of the assigned agreement items (scores of 4, 5 or 6) by the total number of items evaluated.The CVI acceptable was 0.90.Inter-rater reliability was estimated by the Kappa coefficient (k) (11) .The results were submitted to descriptive statistical analysis with measures of central tendency, and presented in tables.
The notes from the raters were analyzed qualitatively to assess the content and possible modifications in accordance with the criteria: 1 -for median smaller or equal to 3, considered as non-valid and not review the content; 2 -for medians 3 and 4, considered valid, but need to review the content, taking into account the notes of the raters; 3 -for medians 5 and 6, consider as valid and review the content if only half of the raters made notes.
The study was approved by the Research Ethics Committee of the Anna Nery School of Nursing (EEAN/UFRJ) under protocol.260,304 on 30/04/2013.

RESULTS
The results are presented on the basis of relevance to the diagnosis process and content validity in relation to the criteria of adequacy, clarity, conciseness and accuracy of names to the stage of "data collection prior to the encounter with the client (pre-encounter)" and to the strategy of "Indirect assessment of the client"; Content validity of mental, behavioral and affective operations regarding the adequacy, clarity, conciseness and precision of the strategy.
Table 1 shows the profile of the sample composed by 13 participants, according to the established criteria.Participants achieved assessment score of 5-12 skill points, with a median of 10 points.The mean age was 40.6 years, ranging from 27 to 70 years.Most of them were female.Group 3 had the participation of 05 (38.4%) raters; group 2 had 07 (53.9%) raters and group 5 had 01 (7.7%) rater.
Relevance of the strategy "Indirect assessment of the client" in the nursing diagnosis process The 13 raters evaluated the strategy of "indirect assessment of the client" considering the strategy relevant to the diagnostic process of nursing in the pre-encounter phase.

Content analysis of the strategy "Indirect assessment of the client"
For the content analysis, four items were assessed in relation to the name assigned to the strategy "indirect assessment of the client".The results concerning the adequacy, clarity, conciseness and accuracy are shown in Table 3.
Considering the medians, mode and the CVI, the criteria of adequacy, clarity, conciseness and accuracy were considered valid.
In the qualitative considerations, the name assigned received two suggestions to change it by "initial or prior assessment" according to the literal version of English "initial assessment" and "indirect data obtaining on the patient", even though both experts have considered the proposed name as valid.

Content analysis of mental, behavioral and affective operations of the nursing diagnosis process in the pre-encounter
To analyze the agreement of mental operations based on the purposes, circumstances, reasons and explanations proposed by the diagram, 80 specific items were assessed.
All items obtained medians of 5 or 6, producing an overall CVI of 0.94 indicating a broad strategy validity.
The scores given by the raters were measured as CVIs to the criteria of adequacy, clarity, conciseness and accuracy, according to results presented in Table 4.
Considering that 20 items were proposed regarding purposes, circumstances, reasons and explanations for application of the strategy indirect assessment of the client, we chose to analyze each of them separately.
The validity criteria related to the median, CVI and the interrater reliability for each of the items are shown below in Table 5.
The results presented did not reach the ideal CVI of 0.90 in the following items: "It is a reliable source of clinical   Relevance of the stage "Data collection prior to the encounter with the client" in the nursing diagnosis process: When asked about the relevance of the stage "data collection prior to the encounter with the client", all 13 raters agreed that the stage is relevant to the diagnostic process in the preencounter event.

Content analysis of the stage "Data collection prior to the encounter with the client"
For content validity of the stage "Data collection prior to the encounter with the client" four items were assessed by the raters regarding the name, obtaining a high agreement to the criteria.The results are presented in Table 2.
As seen in Table 2, although the stage name has reached its validity with median and mode values, the index has not reached the CVI required for the adequacy criteria, according to the sample size of 13 raters.The inter-rater reliability coefficient was high (kappa 0.93).
In the qualitative criteria just one note was sent regarding the name of the stage, suggesting the change of the term "encounter" for "consultation", although raters have agreed with the proposed name.The two raters who scored lower values to the criteria did not make considerations to the stage name.raters were: disagrees with the statement because it is not clear how the strategy increases the effectiveness of information and shall be relative to the following information; agrees, but does not believe that the word effectiveness is appropriate; agrees, but suggests delete "information".Another rater did not agree with the item, however, did not make notes.
For the reason "It is a reliable source of clinical assessment providing information before the direct assessment" out of 04 (30.1%) raters who made notes, 03 of them did not believe in the quality and reliability of the current records, and therefore, an important source of clinical assessment and the fourth suggested deleting the word "yet".Another rater did not agree with the item, however, did not make notes.assessment providing information before the direct assessment"; "The indirect assessment increases the effectiveness and processing of collected information"; "Indirect assessment of the client keeps the diagnostician within the working memory capacity, reducing the cognitive effort", and "There are data already observed and processed in the indirect assessment of the client".
The inter-rater reliability for the above items was median (0.55 to 0.72) according to the Kappa, while all other questions had high reliability.
The analysis of the raters' notes for the following purpose "Indirect assessment increases the effectiveness of observation and processing of collected information" sent by 03 (23.1%)To "Indirect assessment of the client keeps the diagnostician within the working memory capacity, reducing the cognitive effort", 03 (23.1%) raters forwarded the following suggestions: agrees, but suggests rewording the sentence; disagrees that reduces, but believes that it facilitates; disagrees because they believe that it increases the work, since it increases the amount of information that are not relevant.A fourth rater disagreed, but did not make considerations.
As for "There are data already observed and processed in indirect assessment of client", the correlation was lower and only 03 (23.1%) raters forwarded justifications stating that the question is not clear in its proposal and therefore does not favor the understanding of the subject.

DISCUSSION
The data from the tables present evidence that support the content validation of strategies for organization of diagnostic reasoning by synchronous stages to the encounter between diagnostician and the subject under care (pre-encounter; encounter and post-encounter), as decision support for novices, since the novice in the task does not have experience regarding the cognitive skills and mental habits applied during the Nursing Process (NP).
Indirect assessment of the client applies cognitive skills of critical thinking of finding information that helps interpretation and support evidence when is related to the diagnosis, whether by the consumer of health, the family, the patient's chart, books about a particular problem of health, regarding concepts of human responses and those related to aspects of development or consumer culture, as well as the use of discernment to realize that data deserve consideration, reducing the selection of evidence to be taken into account during analysis and other mental processes (12) .
When strategies of cognitive, behavioral and affective skills are adopted in association with stages, they provide subsidies to nursing practice in the development of critical thinking in the diagnostic process, essential in decision making.
Novices to the diagnostic process are those with more difficulties in the task since they do not have experience of situations that they may face.Teaching and allowing them to develop skills required to diagnose skills, monitoring and regulation of knowledge and belief, affective recognition and their influences, made possible by the incorporation of interpretative cognition approaches, as regards metacognition (6)(7) .
The above elements of conceptual basis established in the literature as well as contributions from data processing theory incorporated into the diagram, attributed high levels of agreement to the instrument, as the results presented.
Despite the high agreement, there were some elements of fragility that have not reached the validity index.It was considered relevant to discuss the suggestion of an expert to replace the word "encounter" from the name given to the initial stage of data collection for "consultation", since it is part of the conventional structure of the diagram, being synchronized in all stages.
It should be noted that the term consultation seems to be the most widely used in nursing literature, supported by Resolution No. 358/2009 COFEN which provides the systematization of nursing care and the implementation of NP in environments where Nursing professional care occurs, corresponding to Nursing Consultation when this care is carried out in institutions providing outpatient services, households, schools and community associations (13) .Thus, there are conceptual differences between what would be the strategy (that is, nursing consultation) and the circumstance or direct interaction opportunity with the subject of diagnosis (the "encounter").In addition, the term encounter has sufficient generality to encompass the broad dimensions of the nurse approach to their client, either in nursing consultation or not.
Even though it had 100% of agreement for the relevance of the data collection stage in the pre-encounter phase, during the assessment of adequacy for the name of the data collection stage in the pre-encounter, a rater held that there is no need for a separate stage collection of data in the pre-encounter, because it is already included in the collection of primary and secondary data of current and historic models, Research, NP Collection.
This assessment reflected in the agreement to adequacy in the name of the stage, which did not reach the ideal CVI to validate the item (Table 2).Therefore, it can be considered that the disagreement regarding the adequacy criteria to the name is appropriate to the criteria of not belonging to the stage name that could have reached the validity, as the rater agreed with the other criteria of clarity, conciseness and accuracy of the name for the data collection stage in the pre-encounter.
From the analysis of Table 5, two categories had not validated items that deserve qualitative review at a later stage with Delphi or literature review, although the criteria for evaluation of the medians were valid.It was observed a higher limitation in technology regarding the reasons for carrying out the strategy "Indirect assessment" which presented 03 items, while the purpose had only 01 item.The other categories were all validated.
The indirect assessment offers the possibility of two processes, focusing on obtaining specific data through the data collection and detection of prognostic changes that facilitate the assessment.The first, through the shift change communication and the second through reading patients charts.Such characteristics of targeting features facilitate the selection of areas to be investigated and disposal of information that is not important in the situation, reducing the cognitive effort, increasing the effectiveness of observation and processing of the information collected (6) .
Actions that involve the task of planning are in the basis of the learning process and making diagnosis.The metacognitive approach (monitoring and control) when applied to diagnostic reasoning increases the chances of success of this reasoning, as it advocates that knowing the nature of the task, its purposes, the circumstances of use, the reasons/justification and methodological explanations is essential.
The study presents as a key point, the elements considered as essential in the development of a valid and reliable technology for methodological rigor with which it has been subjected, since its pre-defined stage.We suggest further studies, which can establish the psychometric validity of the technology as well as clinical validation.
Future studies, still being developed, complement the data presented here, especially as they relate to the later phases of pre-encounter with the client, which should not ignore the diagnostician, given the subjective dimensions of the subject of nursing care.
Similar to other studies in diagnostic validation, the difficulties faced by researchers permeated the shortage of materials on methods and models to be used in the validation studies; the selection criteria of raters with expertise in diagnostic reasoning, since Fhering's criteria have been criticized concerning the results of this validation stage.According to experts in the field, it is likely that much of the allocated subjects in research of this kind are closer to the proficient profile than experts (14)(15) .
Another problem is related to the sample size, since 28.6% of invitations were answered and only 16.8% participated in the study.The planned time to respond the form was also a barrier, since the vast majority of raters did not meet the deadline of 15 days, which had to be repeatedly being extended (14) .
There were advantages in relation to the use of conventional mail with the application of electronic mode with regard to cost, however, limitations on the use of spam or other junk mail may have been related to low number of respondents (16) .
The validation process is complex.Thus, studies of diagnostic validation cannot guarantee that the opinion of a small group of individuals can make sure a specific set of items will represent the actual content of an instrument or a reasoning process (15) , therefore, this is a limitation of the study.

CONCLUSION
It is observed a scarcity of studies in the development and validation studies of assistive technologies to diagnostic reasoning in nursing, especially in instruments that describe the task in the form of stages and that can be used as strategic and operational planning tools.
Given this need, the study aimed to validate a technology for performing nursing diagnosis in pre-encounter with the client, since this investigation stage has often been overlooked in the Nursing Process.
Considering the results obtained in the content and reliability analysis procedures applied in the area of metacognitive strategy of initial assessment of the diagnostic reasoning process in nursing for novices, we can highlight high values of agreement for the correlation between experts to the criteria established above on relevance, adequacy, clarity, conciseness and accuracy.Of the 90 items, the five items that did not present validity should be reviewed, since the notes of the experts did not change the core of the issues, suggesting writing adjustments or replacement of terms for better understanding of the client.
The study showed that the indirect assessment in the preencounter is relevant to the diagnostic reasoning process, being possible to develop skills and diagnostic to the novice through strategies proposed in an innovative technology resource to the task, in the form of diagram.
It is expected that the results of this study may contribute to a theoretical and practical teaching tool, easy to use in the practice fields and in the subjects of elementary nursing, in which the diagnosis is one of the priority topics of the syllabus, enabling better integration with clinical reasoning and diagnostic language; task orientation, allowing to identify the attitudes of the students in coherent strategies, correcting possible misconducts, increasing the domain of this activity and the usual procedures involved.

Table 1 -
Description of the sample profile in relation to the title, scientific production and teaching experience in the development of nursing diagnosis process, Brazil, 2013

Table 2 -
Inter-rater agreement for the criteria adequacy, clarity, conciseness and accuracy of the name given to the stage data collection prior to the encounter with the client, Brazil, 2013

Table 3 -
Inter-rater agreement for the criteria of adequacy, clarity, conciseness and accuracy of the name given to the diagnostic strategy Indirect assessment of the client, Brazil, 2013 * Content Validity Index, considering the scores 4, 5 or 6.

Table 4 -
Inter-rater agreement for the criteria of adequacy, clarity, conciseness and accuracy for Indirect assessment of the client, Brazil, 2013 * Content validity index, considering the scores 4, 5 or 6.

Table 5 -
Content analysis and reliability of mental operations related to Indirect assessment of the client in the pre-encounter, Brazil, 2013 *Md = Median values