Acessibilidade / Reportar erro

APPLICABILITY ASSESSMENT OF THE PORTUGUESE VERSION OF A COMPETENCY QUESTIONNAIRE FOR HOSPITAL NURSES

ABSTRACT

Objective:

to develop psychometric assessments of the adapted version of the Competency Evaluation Questionnaire for Brazilian nurses in the hospital context, assessing the reliability of the internal structure of this version.

Method:

a methodological study was undertaken, in which the sample consisted of 273 nurses from two public hospitals. The scale consists of 27 items, distributed in five competence domains. For the construct validity of the adapted version of the instrument, Cronbach’s Alpha was verified, Confirmatory and Exploratory Factor Analysis was applied and Cronbach’s Alpha was again calculated for the questionnaire domains, aiming to validate its new structural arrangement.

Results:

the total Cronbach’s Alpha coefficient was 0.923, while the coefficients per domain ranged from 0.397 (Management) to 0.833 (Nursing process. In the factor analysis, a Comparative Fit Index and Tucker-Lewis Index of 0.808 and 0.783 were found, respectively, demonstrating acceptable and near-ideal values. In the exploratory factor analysis, the Kaiser-Meyer-Olkin index was 0.918 and the result of Bartlett’s sphericity test was p<0.00. After developing a new structural arrangement of the scale, the internal consistency of the new domains was confirmed and ranged from 0.732 to 0.845, that is, all indices were consistent.

Conclusion:

the Brazilian Portuguese version of the Competence Evaluation Questionnaire consisted of 27 items, grouped in the domains they belong to. The criteria of the exploratory factor analysis applied were met, showing an appropriate, valid and reliability instrument.

DESCRIPTORS:
Nurses; Hospitals; Professional competence; Translating; Scales

RESUMO

Objetivo:

realizar avaliações psicométricas da versão adaptada do Competence Evaluation Questionnaire para enfermeiros brasileiros no contexto hospitalar, avaliando a confiabilidade da estrutura interna desta versão.

Método:

trata-se de um estudo metodológico, no qual a amostra constituiu-se de 273 enfermeiros pertencentes a duas instituições hospitalares públicas. O instrumento é composto por 27 itens distribuídos em cinco domínios de competências. Para validade de constructo da versão adaptada do instrumento, verificou-se Alpha de Cronbach, Análise Fatorial Confirmatória e Exploratória e calculado novamente o Alpha de Cronbach dos domínios do instrumento, a fim de validar seu novo arranjo estrutural.

Resultados:

foi verificado Alpha de Cronbach total de 0,923 e por domínios variando de 0,397 (Gerenciamento) a 0,833 (Processo de enfermagem). Na análise fatorial foram encontrados valores de Índice de Ajuste Comparativo e Índice de Tucker-Lewis, respectivamente, 0,808 e 0,783, demonstrando valores aceitáveis e próximos do ideal. Na análise fatorial exploratória, o índice de Kaiser-Meyer-Okin foi de 0,918 e o teste de esfericidade de Bartlett teve valor p<0,00, ocorrendo uma nova disposição estrutural do instrumento que posteriormente se pode confirmar a consistência interna dos novos domínios variando de 0,732 a 0,845 e, portanto, todos os valores consistentes.

Conclusão:

a versão para o português brasileiro do Questionário de Avaliação de Competências ficou composta por 27 itens agrupados aos domínios a que pertencem obedecendo aos critérios da Análise Fatorial exploratória realizada, revelando-se instrumento adequado, válido e confiável.

DESCRITORES:
Enfermeiros; Hospitais; Competência profissional; Tradução; Escalas

RESUMEN

Objetivo:

realizar evaluaciones psicométricas de la versión adaptada del Competence Evaluation Questionnaire para los enfermeros brasileños en el contexto hospitalario y evaluando la confiabilidad de la estructura interna de esta versión.

Método:

se trata de un estudio metodológico y constituido por una muestra de 273 enfermeros pertenecientes a dos (2) instituciones hospitalarias públicas. El instrumento está compuesto por 27 ítems distribuidos en cinco dominios de competencias. Para la validez del constructo de la versión adaptada del instrumento se verificó el Alpha de Cronbach, el Análisis Factorial Confirmatorio y Exploratorio y se calculó nuevamente el Alpha de Cronbach de los dominios del instrumento, a fin de validar su nuevo orden estructural.

Resultados:

se verificó el Alpha de Cronbach total de 0,923 y por dominios variando de 0,397 (Gerenciamiento) a 0,833 (Proceso de enfermería). En el análisis factorial se encontraron los valores del Índice de Ajuste Comparativo e Índice de Tucker-Lewis, respectivamente, de 0,808 y 0,783, demostrando valores aceptables y próximos de lo ideal. En el análisis factorial exploratorio, el índice de Kaiser-Meyer-Okin fue de 0,918 y el test de esfericidad de Bartlett tuvo un valor de p<0,00, ocurriendo una nueva disposición estructural del instrumento que posteriormente se pudo confirmar con la consistencia interna de los nuevos dominios variando de 0,732 a 0,845 y, por lo tanto, todos los valores consistentes.

Conclusión:

la versión para el portugués brasileño del Cuestionario de Evaluación de Competencias quedó compuesta por 27 ítems agrupados con los dominios al que pertenecen y obedeciendo a los criterios del Análisis Factorial exploratorio realizado, rebelándose como un instrumento adecuado, válido y confiable.

DESCRIPORES:
Enfermeros; Hospitales; Competencia profesional; Traducción; Escalas

INTRODUCTION

In the hospital context, both hospital managers and nurses have focused on the theme of professional competence. In their work process, the nurses take care of the most severe patients, sharing care and management activities11 Camelo SHH. Professional competences of nurse to work in Intensive Care Units: an integrative review. Rev Latino-am Enfermagem [Internet]. 2012 [cited 2017 Mar 28]; 20(1):192-200. Available from: http://goo.gl/m3KCQ0
http://goo.gl/m3KCQ0...
and needing specific skills in order to guarantee excellent care.

In this context, competences represent a set of knowledge, skills and attitudes that each worker has and that serves as a reference to build their development in the company.22 Oliveira DA, Gonçalves RS, Barbosa ACQ. Percepção dos gestores de recursos humanos em relação ao modelo de gestão por competências. Rev FSA [Internet]. 2014 [cited 2017 Mar 28]; 11(2):01-26. Available from:http://www4.fsanet.com.br/revista/index.php/fsa/article/viewFile/363/316
http://www4.fsanet.com.br/revista/index....
Thus, professional competencies involve these three axes, and knowledge is understood as what the professional knows; skills as know-how; and attitudes are defined as knowing how to act, judge, choose and decide in a specific situation.33 Camelo SHH, Angerami ELS. Professional competence: the building of concepts and strategies developed by health services and implications for nursing. Texto Contexto Enferm [Internet]. 2013 [cited 2017 Apr 10]; 22(2):552-60. Available from: http://www.scielo.br/ pdf/tce/v22n2/v22n2a34.pdf
http://www.scielo.br/ pdf/tce/v22n2/v22n...

When managing people, companies and/or managers need to provide conditions so that the organization can use individual competencies. Therefore, competency-based performance evaluation is a personnel management strategy that can help to diagnose the workers’ potential and improve their performance. To this end, it is important not only for the adoption of technical-professional knowledge, but also for the ability to mobilize individuals and the organization to face problems in daily work.22 Oliveira DA, Gonçalves RS, Barbosa ACQ. Percepção dos gestores de recursos humanos em relação ao modelo de gestão por competências. Rev FSA [Internet]. 2014 [cited 2017 Mar 28]; 11(2):01-26. Available from:http://www4.fsanet.com.br/revista/index.php/fsa/article/viewFile/363/316
http://www4.fsanet.com.br/revista/index....

The international literature reveals that researchers have been concerned with this issue and have developed instruments to evaluate the competencies of nurses through their performance in both the public health area44 Kalb KB, Cherry NM, Kauzloric J, Brender A, Green K, Miyagawa L, et al. A competency-based approach to public health nursing performance appraisal. Public Health Nurs [Internet]. 2006 [cited 2017 Mar 28]; 23(2):115-38. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16684187
http://www.ncbi.nlm.nih.gov/pubmed/16684...
and the hospital area.55 Safadi R, Jaradeh M, Bandak A, Froelicher E. Competence assessment of nursing graduates of Jordanian universities. Nurs Health Sci [Internet]. 2010 [cited 2017 Mar 28]; 12(2):147-54. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20602685
http://www.ncbi.nlm.nih.gov/pubmed/20602...
-66 Nilsson J, Johansson E, Egmar AC, Florin J, Leksell J, Lepp M, et al. Development and validation of a new tool measuring nurses self-reported professional competence- The Nurse Professional Competence (NPC) Scale. Nurse Educ Today [Internet]. 2014 [cited 2017 Mar 28]; 34(4):574-80. Available from http://www.ncbi.nlm.nih.gov/pubmed/23938092
http://www.ncbi.nlm.nih.gov/pubmed/23938...
In Brazil, researchers have identified the competencies of nurses at the hospital,77 Camelo SHH, Rocha FLR, Chaves LDP, Silva VLS, Soares MI. Competencias profesionales y estrategias organizacionales de gerentes de enfermería. Cienc Enferm [Internet]. 2016 [cited 2017 Mar 28]; 22(1):75-86. Available from: http://www.scielo.cl/pdf/cienf/v22n1/art_07.pdf
http://www.scielo.cl/pdf/cienf/v22n1/art...
-88 Paiva KCM, Junior WJS. Competências profissionais de enfermeiros e sua gestão em um hospital particular. Rev Bras Enferm [Internet]. 2012 [cited 2017 Mar 28]; 65(6):899-90. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000600003
http://www.scielo.br/scielo.php?script=s...
using qualitative approaches. At the time of data collection for this study, however, no published studies have been found that used quantitative tools, such as validated scales, to evaluate the skills of hospital nurses.

In this perspective, the Competency Evaluation Questionnaire (CEQ) is a tool built to evaluate nurses’ competencies, which used statements of evaluation instruments from public and private teaching hospitals. It is a five-point Likert scale ranging from 1 = low competence to 5 = high competence. CEQ’s items reveal universal care standards of the American Nurses’ Association (ANA), Nursing and Midwifery Council (NMC) and the International Council of Nurses (ICN), and present competency statements distributed across five domains, including management, professionalism, problem solving, nursing process and knowledge of basic nursing principles.55 Safadi R, Jaradeh M, Bandak A, Froelicher E. Competence assessment of nursing graduates of Jordanian universities. Nurs Health Sci [Internet]. 2010 [cited 2017 Mar 28]; 12(2):147-54. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20602685
http://www.ncbi.nlm.nih.gov/pubmed/20602...

Thus, in the light of the extensive literature review and discussions of professionals in the area of human resources in nursing, it was noticed that the nursing-related competencies of the CEQ were pertinent to the Brazilian cultural context. For its use, translation and cross-cultural adaptation are necessary. In addition, to proceed with the validation, psychometric evaluations are necessary to confirm its reliability for use in Brazil.

Given this premise, this study aimed to perform psychometric evaluations of the adapted version of CEQ for Brazilian nurses in the hospital context, evaluating the reliability of the internal structure of this version.

METHOD

The aim in this methodological study was to verify the psychometric properties of the Portuguese version of a competency evaluation scale.

The cross-cultural adaptation process of the CEQ started with the authorization of the authors responsible for the original instrument. Those authors granted permission to carry out the entire process of cross-cultural adaptation of the instrument, including the application of further psychometric tests. Thus, a frequent approach that was used in this study involves the following sequence: translation, back translation, technical revision of the version translated by the expert committee and pilot study.99 Guillemin F, Bombardier C, Beaton DE. Cross-cultural adaptation of health related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993; 46(12):1417-32. In addition to the validation steps of the instrument, statistical analyses were carried out to assess to what extent the instrument may, in fact, be considered valid for the context it has been adapted to. Adapting and validating an instrument are, therefore, distinct but complementary steps.1010 Borsa JC, Damásio BF, Bandeira D. Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia [Internet]. 2012 [cited 2017 Mar 28]; 22(53):423-32. Available from: http://www.scielo.br/pdf/paideia/v22n53/14.pdf
http://www.scielo.br/pdf/paideia/v22n53/...

It is highlighted that the first stage began with two bilingual translators’ translation of the original instrument from English into Portuguese. An expert committee (nursing professors, hospital nurses and two linguists) analyzed the two Portuguese versions, together with the original English version, to compare them and list the items that maintained semantic equivalence with the items of the original instrument, resulting in the Portuguese consensus version.11 Camelo SHH. Professional competences of nurse to work in Intensive Care Units: an integrative review. Rev Latino-am Enfermagem [Internet]. 2012 [cited 2017 Mar 28]; 20(1):192-200. Available from: http://goo.gl/m3KCQ0
http://goo.gl/m3KCQ0...
The third stage consisted of the back-translation of the Portuguese consensus version1 into the instrument’s original language; this stage is performed by two other bilingual translators fluent in English, independently and blinded to the original version of the instrument. The fourth stage consisted of the technical revisions of the instrument to compare this translated version to the back-translations and to the original instrument in English, resulting in the Portuguese consensus version.

This version was submitted to the evaluation of a five-member expert panel of hospital nurses, who assessed the clarity, relevance and appearance of the instrument items. After adding the suggested adjustments, the pilot study was undertaken, with the participation of 29 hospital nurses. The title adopted for the adapted version of CEQ was Questionário de Avaliação de Competências (QAC).

To proceed with the previously mentioned steps, a psychometric evaluation of the QAC was done in order to verify its reliability. Therefore, a non-random sample was used, consisting of all eligible professionals, present or scaled for work in the participating hospitals during the period of data collection.

The calculation of the sample to evaluate the psychometric properties of the QAC was based on the assumption that, for an instrument containing 27 questions, a minimum of ten professionals would be necessary for each item in order to perform the factor analysis.1111 Crocker L, Algina J. Introduction to Classical and Modern Test Theory. New York (US): CBS College Publishing; 1986. Accordingly, the criteria for inclusion in the sample were: nurses who worked in those hospitals and had direct contact with inpatients; nurses who did not have direct contact with the patient, but whose functions performed at work directly affect inpatient care (leaders, managers, supervisors); and being present during one of the work shifts during the period of data collection in the organization, which occurred between March and October 2016.

Thus, for the construct validity of the adapted version of the instrument, the internal consistency of the items was verified by means of Cronbach’s alpha. Then, Confirmatory Factor Analysis (CFA), Exploratory Factor Analysis (EFA) and again CFA were performed.1212 Yuan KH, Bentler PM. Structural equation modeling. In: C Rao, S Sinharay. ‘Handbook of Statistics: Psychometrics’. Handbook of Statistics, Elsevier; 2007. chapter 10. In the CFA, the adjustment of the proposed model was evaluated by means of the comparative fit (CFI) index and the Tucker-Lewis (TLI) index, for which values superior to 0.80 are acceptable, values greater than 0.95 being considered optimal;1313 Brown TA. Confirmatory Factor Analysis for Applied Research. Second edition. New York: The Guilford Press; 2015. and the Root Mean Square Error of Approximation (RMSEA), for which values below 0.08 are expected.1414 Pilati R, Laros JA. Modelos de equações estruturais em psicologia: conceitos e aplicações. Psic: Teor e Pesq [Internet]. 2007 [cited 2017 Mar 28]; 23(2):205-16. Available from: http://www.scielo.br/scielo.php?pid=S0102-37722007000200011&script=sci_abstract&tlng=pt
http://www.scielo.br/scielo.php?pid=S010...
Then, the factorial structure of the scale was evaluated by means of EFA. To verify the appropriateness of the factorial analysis, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) index and the Bartlett Sphericity test (p <0.05) were used to check for the presence of correlations between the variables.

Next, the extraction method was performed with Principal Components Analysis and Varimax Orthogonal Rotation. As a factor selection criterion, an eigenvalue greater than onewas used, and loading superior to 0.3 were considered significant.1515 Hair Junior JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Multivariate Data analysis. Seventh Edition. Upper Saddle River: Prentice Hall; 2010. Confirmatory factor analysis (CFA) was repeated in order to validate the factorial structure obtained in the EFA. Afterwards, Cronbach’s alpha was calculated for the domains of the instrument in order to validate its new structural arrangement. In all analyses the significance level of 5% (alpha = 0.05) was adopted.

This research received approval from the Research Ethics Committee at the University of São Paulo at Ribeirão Preto College of Nursing, being registered under CAAE: 38544414.9.0000.5393 and Opinion 243/2014. The participants signed the Free and Informed Consent Form.

RESULTS

The cross-cultural adaptation process

After translation and adaptation, the QAC was composed of 27 items, in which respondents should express their score between 1 and 5, where 1 = the competency does not apply, 2 = low competency, 3 = moderate competency, 4 = good competency and 5 = excellent competency. The adapted version of the instrument for Portuguese consisted of 27 items distributed in five nursing competency domains: management, professionalism, problem solving, nursing process and knowledge of the basic nursing principles for hospital nurses.

In terms of the sociodemographic characteristics of the participants in the pilot study, 89% were female, ranging in age from 26 to 44 years, 45% with more than 10 years of professional experience and 48% between five and 10 years of work at the hospital. For the total score, the Cronbach’s alpha of the pilot was 0.932, approaching the original instrument (0.97), and varied between 0.58 and 0.896 for the domains, in line with a recent study that, when validating an instrument called Nurses Core Competence in Palliative Care, obtained a Cronbach’s Alpha in the domains of the instrument ranging from 0.51 to 0.97.1616 Slatten K, Hatlevik O, Fagerstrom L. Validation of a new instrument for self-assessment of nurses core competencies in palliative care. Nurs Res Pract [Internet]. 2014 [cited 2017 Mar 28]:1-8. Available from: http://www.hindawi.com/journals/nrp/2014/615498/
http://www.hindawi.com/journals/nrp/2014...

Construct validity

Professional and demographic characterization of hospital nurses

As the central objective of this study was to evaluate the psychometric properties of the QAC, it was applied in a sample of 273 nurses working in two hospitals, 244 (89.4%) of whom were women, the mean age being 38.06 years. Regarding their professional training, 90 (33%) possessed only an undergraduate degree, 128 (46.9%) some type of specialization, 46 (16%) obtained a Master’s degree and only nine (3%) held a Ph.D., with 13.5 years of professional experience on average and 10.8 years of work at the institution on average.

Analysis of the QAC responses by domain

Analyzing the answers of the participating nurses, the different competencies listed in the QAC were used constantly. Thus, it could be verified that the highest occurrence of the response moderate competency was only found in item 12 Participation in scientific research and/or application of outcomes, referring to the Management domain, with 33% of the answers indicated; most of the nurses considered all other competency items in the QAC as good or excellent, revealing that, even in the face of the complex reality of the hospital environment, constant updating of technologies and new forms of work relationships with the presence of multiprofessional teams, the nurses are committed to their performance, that is, positive attitudes occur, including the search to develop countless professional competencies.

Validity and Reliability of QAC

Initially, we verified the internal consistency of the items, with a total Cronbach’s Alpha of 0.923, and domain coefficients of 0.377 (Management), 0.717 (Knowledge of basic principles), 0.746 (Professionalism), 0.808 (Problem Solving) and 0.833 Nursing process). Thus, it was observed that Cronbach’s Alpha consists of four domains of QAC.

Next, CFA was carried out, in which the CFI and TLI found were 0.808 and 0.783, respectively, showing acceptable values for the CFI index and below-acceptable coefficients for TLI; and the Root Mean Square Error of Approximation (RMSEA) was 0.086, which does not mean a good fit of the proposed model. In order to verify the validity and fidelity of the instrument, however, EFA was applied, in which the KMO index was 0.918 and the Bartlett sphericity test corresponded to x² = 3452.816 with p <0.001. In the Communalities test, it was evidenced that all items had values above 0.4, varying in descending order from 0.714 to 0.406.

Regarding factor extraction, the total variance explained had five eigenvalues greater than 1.00 with a cumulative percentage of 58% of the total variance, suggesting that the instrument could be divided into five domains similar to the original instrument, but with a different arrangement. To verify these domains, we performed the Varimax rotation, shown in Table 1.

Table 1
Matrix of factor loadings with Varimax rotation of the QAC according to the new structure. Ribeirão Preto, SP, Brazil, 2017. (n=273)

Table 1 shows factor loadings superior to 0.4. The highest loading was found for item 14, corresponding to 0.752.

Next, CFA was applied, which indicated better CFI and TLI indices, corresponding to 0.869 and 0.853, respectively. The RMSEA = 0.071 and SRMR (Standardized Root Mean Square Residual) = 0.066 bordered on appropriate values were confirmed the validity of the instrument. Finally, Cronbach’s Alpha was calculated for the new scale structure, in which the internal consistence of the new domains could be confirmed, ranging from 0.732 to 0.845, all coefficients being consistent.

Thus, in view of the new arrangement and grouping of the items, after the factor analysis, the domains were renamed. In the Management domain, items 5, 12, 21, 22, 23 and 24 were included; in the Communication domain, items 3, 4, 6, 25, 26 and 27; in the Problem-Solving domain, items 15, 17, 18, 19 and 20; in the Nursing Process domain, items 10, 11, 13, 14 and 16; and in the Professionalism domain, items 1, 2, 7, 8 and 9. The Brazilian version of the validated QAC and its respective domains is displayed in Figure 1.

Figure 1
Brazilian version of QAC after construct validity. Ribeirão Preto, SP, Brazil, 2017.

DISCUSSION

The growing concern with guaranteeing the care provided in the hospital context and the need for a staff of nurses with specific competencies were the motivating factors that led the researchers to look for an instrument that could evaluate the performance through the competencies of these professionals.

In that perspective, the CEQ was a tool with an international scope, built to evaluate nurses’ competencies. The authors of the original version of the CEQ ensured semantic validation, cultural adaptation in the English language and a pilot study with 50 nurses, in which the internal consistency of the items was calculated with Cronbach’s Alpha = 0.97.1717 Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterel [Internet]. 2004 [cited 2017 Mar 28]; 126(1 Suppl. 1):124-8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/14978648
http://www.ncbi.nlm.nih.gov/pubmed/14978...
Later, it was tested in a sample of 258 nursing graduates with diverse backgrounds, coming from rural and urban areas of the country of origin. Their competencies to work in hospitals were evaluated, comparing the sex, the academic background and the type of hospital. It is noteworthy, however, that no other psychometric tests were performed for the original instrument at this stage.1717 Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterel [Internet]. 2004 [cited 2017 Mar 28]; 126(1 Suppl. 1):124-8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/14978648
http://www.ncbi.nlm.nih.gov/pubmed/14978...

It is emphasized that this research, after the cross-cultural adaptation process, was intended to evaluate the applicability of the adapted version of the CEQ. In this sense, it should be emphasized that the cross-cultural adaptation process is a complex step that requires caution, so that the version obtained is appropriate to the construct and equivalent to the original version.1010 Borsa JC, Damásio BF, Bandeira D. Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia [Internet]. 2012 [cited 2017 Mar 28]; 22(53):423-32. Available from: http://www.scielo.br/pdf/paideia/v22n53/14.pdf
http://www.scielo.br/pdf/paideia/v22n53/...

Thus, in order to meet these criteria, the translation was performed by two independent sworn translators, who are professionals qualified to issue official public translations. In the adaptation process of the CEQ, an expert committee carried out the content validity, including semantic, cultural, idiomatic and conceptual equivalences. This process is essential to obtain an instrument with satisfactory psychometric characteristics.1818 Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self report measures. Spine. 2000; 25(24):3186-91. As previously described, the CEQ is composed of 27 items, distributed in five domains. In the adaptation process , minimal adjustments were made in the expressions of items 1, 4, 7, 9, 11, 13, 17, 19, 20, 21, 23 and 27, resulting in the adapted version called Questionário de Avaliação de Competência (QAC).

Thus, underlining the information, in order to evaluate the psychometric properties, a sample of 273 nurses working in two large hospitals was used. Considering the nurses’ answers to the QAC, items 2 and 8 stand out, which 69.2% and 72.2% of the professionals identified as excellent competencies belonging to the management and professional domains of the original instrument, corroborating the research performed by the authors of the CEQ, in which the highest averages of responses were found in these same domains.55 Safadi R, Jaradeh M, Bandak A, Froelicher E. Competence assessment of nursing graduates of Jordanian universities. Nurs Health Sci [Internet]. 2010 [cited 2017 Mar 28]; 12(2):147-54. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20602685
http://www.ncbi.nlm.nih.gov/pubmed/20602...
These responses concern the high commitment of nurses to their workplace, professional achievement in activities developed within the organizational context and commitment to the ethical guidelines of the profession, besides indicating that the management style should be generating motivation and satisfaction, favoring the nurses’ performance of these competencies.

To evaluate the psychometric properties of the QAC, however, its internal consistency and construct validity were measured. The internal consistency analysis measured by Cronbach’s Alpha was 0.923, being considered an excellent coefficient the closer it gets to one and superior to 0.80 for conclusive studies. The internal consistency analysis as a parameter to measure the reliability of the questionnaire is an important criterion for the quality of adaptation studies.1919 Vieira VA. Escalas em Marketing: métricas de resposta do consumidor e de desempenho empresarial. São Paulo: Editora Atlas S.A.; 2011.-2020 Kerwee CB, Screllingerhout JM, Vergagen AP, Koes BW, De Vet HC. Methodological quality of studies on the measurement properties of neck pain and disability questionnaires: a systematic review. J Manipulative Physiol Ther [Internet]. 2011 [cited 2017 Mar 28]; 34(4):261-72. Available from: https://www.ncbi.nlm.nih.gov/pubmed/21621728
https://www.ncbi.nlm.nih.gov/pubmed/2162...

For the construct validity of the QAC, factor analysis was used and, in the beginning, considering an existing theoretical structure, the AFC was chosen to verify if the model fit the original model. The CFA showed acceptable CFI, TLI and RMSEA below the appropriate value, which means that the original model needs adjustment. Thus, considering that the TLI and CFI are measures that compare the proposed model with the original structure, in an attempt to improve the adjustment of the model, we proceeded with the EFA. In this analysis, the KMO index close to 1 and the Barlett sphericity test with p <0.05 revealed an appropriate matrix with homogeneous correlation between the items of the instrument.2121 Pallant J. SPSS survival manual: A step by step guide to data analysis using SPSS. 6th ed. McGraw-Hill Education; 2016.

In addition to the KMO analysis and the Barlett Sphericity Test, the Communality (H²) verification was also used. The communality indicates the total amount of variance that an original variable shares with all other variables included in the analysis.19 In this study, all items presented values above 0.4, indicating that no variables should be excluded. Simultaneously, the factor extraction method was performed, which suggested the extraction of five factors with their own values superior to 1, thus contributing to explain the variance in the original variables.

Then, the factor rotation was performed and, in order to make the empirical result easily interpretable without affecting the statistical properties, choosing the Varimax orthogonal rotation to minimize the number of variables that had high loadings in each factor.2020 Kerwee CB, Screllingerhout JM, Vergagen AP, Koes BW, De Vet HC. Methodological quality of studies on the measurement properties of neck pain and disability questionnaires: a systematic review. J Manipulative Physiol Ther [Internet]. 2011 [cited 2017 Mar 28]; 34(4):261-72. Available from: https://www.ncbi.nlm.nih.gov/pubmed/21621728
https://www.ncbi.nlm.nih.gov/pubmed/2162...
Thus, to interpret the role of each variable in the definition of each factor, a factor loading > 0.45 was assumed as the minimum acceptable value. In this sense, the total variance explained signaled that the scale could be divided into five domains similar to the original instrument, with a new arrangement among the items. The new presentation revealed that the items are valid because they have a factor loading superior to 0.4 and therefore, no items should be excluded in the adaptation of the original version of the QAC.

Afterwards, CFA was performed in which the improvement of the CFI, TLI and RMSEA indices was observed, demonstrating that the new structure is more appropriate. There was a new arrangement of the domains with the items though, but none of the items was withdrawn.

In this sense, it should be emphasized that all items had theoretical affinities with their respective domains, and a new Cronbach’s Alpha was identified for the domains of the instrument, confirming an appropriate reliability index in all domains, so that the instrument was considered appropriate to measure the competencies the nurses developed in the hospital.

One limitation in this study, however, is the fact that the sample belonged to only two hospitals. Further research will provide sustainability to consolidate the validity of the instrument, and corroborate towards its strengthening as an instrument to evaluate the performance of professionals in hospital organizations.

CONCLUSION

Instruments to evaluate the competences of hospital nurses should help future professionals, managers and researchers to perform diagnoses of workers’ competences for their specific function, in order to perform performance evaluations of each professional, as well as to favor the elaboration of training and development of their human capital.

This article presented the psychometric evaluation of the adapted Portuguese version called QAC, which aims to evaluate the competencies of nurses working in hospitals. It is observed that the psychometric results found in the sample used were appropriate. Thus, the final structure of the QAC was composed of 27 items that were grouped to the domains they belonged to, obeying the EFA criteria.

It is noticed that the new arrangement of the items with the domains was a more favorable representation than the original model, emphasizing that they maintained the five competence domains, although one term was replaced for the sake of a better understanding, being denominated: Nursing process, management, communication, problem solving and professionalism.

Thus, it is expected that, in the future, the QAC can be adopted in scientific investigations in order to establish normative data on the skills of hospital nurses, evaluating the sensitivity of this instrument before and after interventions, and also assisting nurse managers in the evaluation of their employees.

REFERÊNCIAS

  • 1
    Camelo SHH. Professional competences of nurse to work in Intensive Care Units: an integrative review. Rev Latino-am Enfermagem [Internet]. 2012 [cited 2017 Mar 28]; 20(1):192-200. Available from: http://goo.gl/m3KCQ0
    » http://goo.gl/m3KCQ0
  • 2
    Oliveira DA, Gonçalves RS, Barbosa ACQ. Percepção dos gestores de recursos humanos em relação ao modelo de gestão por competências. Rev FSA [Internet]. 2014 [cited 2017 Mar 28]; 11(2):01-26. Available from:http://www4.fsanet.com.br/revista/index.php/fsa/article/viewFile/363/316
    » http://www4.fsanet.com.br/revista/index.php/fsa/article/viewFile/363/316
  • 3
    Camelo SHH, Angerami ELS. Professional competence: the building of concepts and strategies developed by health services and implications for nursing. Texto Contexto Enferm [Internet]. 2013 [cited 2017 Apr 10]; 22(2):552-60. Available from: http://www.scielo.br/ pdf/tce/v22n2/v22n2a34.pdf
    » http://www.scielo.br/ pdf/tce/v22n2/v22n2a34.pdf
  • 4
    Kalb KB, Cherry NM, Kauzloric J, Brender A, Green K, Miyagawa L, et al. A competency-based approach to public health nursing performance appraisal. Public Health Nurs [Internet]. 2006 [cited 2017 Mar 28]; 23(2):115-38. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16684187
    » http://www.ncbi.nlm.nih.gov/pubmed/16684187
  • 5
    Safadi R, Jaradeh M, Bandak A, Froelicher E. Competence assessment of nursing graduates of Jordanian universities. Nurs Health Sci [Internet]. 2010 [cited 2017 Mar 28]; 12(2):147-54. Available from: http://www.ncbi.nlm.nih.gov/pubmed/20602685
    » http://www.ncbi.nlm.nih.gov/pubmed/20602685
  • 6
    Nilsson J, Johansson E, Egmar AC, Florin J, Leksell J, Lepp M, et al. Development and validation of a new tool measuring nurses self-reported professional competence- The Nurse Professional Competence (NPC) Scale. Nurse Educ Today [Internet]. 2014 [cited 2017 Mar 28]; 34(4):574-80. Available from http://www.ncbi.nlm.nih.gov/pubmed/23938092
    » http://www.ncbi.nlm.nih.gov/pubmed/23938092
  • 7
    Camelo SHH, Rocha FLR, Chaves LDP, Silva VLS, Soares MI. Competencias profesionales y estrategias organizacionales de gerentes de enfermería. Cienc Enferm [Internet]. 2016 [cited 2017 Mar 28]; 22(1):75-86. Available from: http://www.scielo.cl/pdf/cienf/v22n1/art_07.pdf
    » http://www.scielo.cl/pdf/cienf/v22n1/art_07.pdf
  • 8
    Paiva KCM, Junior WJS. Competências profissionais de enfermeiros e sua gestão em um hospital particular. Rev Bras Enferm [Internet]. 2012 [cited 2017 Mar 28]; 65(6):899-90. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000600003
    » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672012000600003
  • 9
    Guillemin F, Bombardier C, Beaton DE. Cross-cultural adaptation of health related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993; 46(12):1417-32.
  • 10
    Borsa JC, Damásio BF, Bandeira D. Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia [Internet]. 2012 [cited 2017 Mar 28]; 22(53):423-32. Available from: http://www.scielo.br/pdf/paideia/v22n53/14.pdf
    » http://www.scielo.br/pdf/paideia/v22n53/14.pdf
  • 11
    Crocker L, Algina J. Introduction to Classical and Modern Test Theory. New York (US): CBS College Publishing; 1986.
  • 12
    Yuan KH, Bentler PM. Structural equation modeling. In: C Rao, S Sinharay. ‘Handbook of Statistics: Psychometrics’. Handbook of Statistics, Elsevier; 2007. chapter 10.
  • 13
    Brown TA. Confirmatory Factor Analysis for Applied Research. Second edition. New York: The Guilford Press; 2015.
  • 14
    Pilati R, Laros JA. Modelos de equações estruturais em psicologia: conceitos e aplicações. Psic: Teor e Pesq [Internet]. 2007 [cited 2017 Mar 28]; 23(2):205-16. Available from: http://www.scielo.br/scielo.php?pid=S0102-37722007000200011&script=sci_abstract&tlng=pt
    » http://www.scielo.br/scielo.php?pid=S0102-37722007000200011&script=sci_abstract&tlng=pt
  • 15
    Hair Junior JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Multivariate Data analysis. Seventh Edition. Upper Saddle River: Prentice Hall; 2010.
  • 16
    Slatten K, Hatlevik O, Fagerstrom L. Validation of a new instrument for self-assessment of nurses core competencies in palliative care. Nurs Res Pract [Internet]. 2014 [cited 2017 Mar 28]:1-8. Available from: http://www.hindawi.com/journals/nrp/2014/615498/
    » http://www.hindawi.com/journals/nrp/2014/615498/
  • 17
    Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterel [Internet]. 2004 [cited 2017 Mar 28]; 126(1 Suppl. 1):124-8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/14978648
    » http://www.ncbi.nlm.nih.gov/pubmed/14978648
  • 18
    Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self report measures. Spine. 2000; 25(24):3186-91.
  • 19
    Vieira VA. Escalas em Marketing: métricas de resposta do consumidor e de desempenho empresarial. São Paulo: Editora Atlas S.A.; 2011.
  • 20
    Kerwee CB, Screllingerhout JM, Vergagen AP, Koes BW, De Vet HC. Methodological quality of studies on the measurement properties of neck pain and disability questionnaires: a systematic review. J Manipulative Physiol Ther [Internet]. 2011 [cited 2017 Mar 28]; 34(4):261-72. Available from: https://www.ncbi.nlm.nih.gov/pubmed/21621728
    » https://www.ncbi.nlm.nih.gov/pubmed/21621728
  • 21
    Pallant J. SPSS survival manual: A step by step guide to data analysis using SPSS. 6th ed. McGraw-Hill Education; 2016.

Publication Dates

  • Publication in this collection
    09 Aug 2018
  • Date of issue
    2018

History

  • Received
    10 Apr 2017
  • Accepted
    08 Nov 2017
Universidade Federal de Santa Catarina, Programa de Pós Graduação em Enfermagem Campus Universitário Trindade, 88040-970 Florianópolis - Santa Catarina - Brasil, Tel.: (55 48) 3721-4915 / (55 48) 3721-9043 - Florianópolis - SC - Brazil
E-mail: textoecontexto@contato.ufsc.br