Acessibilidade / Reportar erro

Predictors of health professionals’ satisfaction with continuing education: A cross-sectional study* * This article refers to the call “Educational technologies and innovative teaching methods in the training of human resources in health”.

Abstracts

Objectives:

to verify which organizational, methodological, and resource-related characteristics of Continuing Health Education (CHE) help to best predict the professionals´ satisfaction.

Method:

a cross-sectional study with multivariate logistic regressions to predict a high mean satisfaction with different dimensions of educational actions used: Overall satisfaction, Utility, Methodology, Organization and resources, and Teaching Capacity. 25,281 satisfaction questionnaires have been analysed completed by health professionals attending 1,228 training activities in Andalusia (Spain), during the period from March 2012 to April 2015.

Results:

the characteristics that best predict a high overall satisfaction are the following: clinical session type as opposed to the workshop (Odds Ratio [OR]=2.07, p<0.001); face-to-face attendance modality (OR=3.88, p<0.001) or semi-personal-attendance (OR=2.83, p<0.001), as opposed to e-learning; and 1-2 days in duration (OR=2.38, p<0.001) as opposed to those of between 3 and 14 days. A lower number of hours (OR=0.99, p<0.001) and a lower number of professionals (OR=0.98, p<0.05) also increase the probability. Having the educational actions accredited increases the probabilities in the following dimensions: Utility (OR=1.33, p<0.05), Methodology (OR=1.5, p<0.01) and Teaching capacity (OR=1.5, p<0.01).

Conclusion:

the study provides relevant information on aspects that improve professional satisfaction, such as that e-learning activities should improve their content, teaching methods, and styles, or that face-to-face clinical sessions are the type of CHE with the greatest satisfaction.

Descriptors:
Education, Continuing; Personal Satisfaction; Education, Distance; Students, Health Occupations; Staff Development; Clinical Conference


Objetivo:

verificar quais características organizacionais, metodológicas e relacionadas a recursos da educação continuada em saúde (ECS) ajudam a predizer melhor a satisfação dos profissionais.

Método:

estudo transversal com regressões logísticas multivariadas para predizer uma alta satisfação média com diferentes dimensões das ações educativas utilizadas: Satisfação Geral, Utilidade, Metodologia, Organização e Recursos, e Capacidade de Ensino. Foram analisados 25.281 questionários de satisfação preenchidos por profissionais de saúde que participaram de 1.228 atividades de treinamento na Andaluzia (Espanha), no período de março de 2012 a abril de 2015.

Resultados:

características que melhor predizem uma alta Satisfação Geral são as seguintes: Método de sessão clínica em oposição a workshops (Odds Ratio[OR]=2,07;p<0,001); modalidade presencial (OR=3,88;p<0,001) ou semipresencial (OR=2,83;p<0,001) em oposição a e-learning; e 1-2 dias de duração (OR=2,38;p<0,001) em oposição a 3-14 dias. Um menor número de horas (OR=0,99;p<0,001) e de profissionais (OR=0,98;p<0,05) também aumenta a probabilidade. A acreditação das ações educativas aumenta as probabilidades nas dimensões: Utilidade (OR=1,33;p<0,05), Metodologia (OR=1,5;p<0,01) e Capacidade de Ensino (OR=1,5;p<0,01).

Conclusão:

o estudo fornece informações relevantes sobre aspectos que melhoram a satisfação profissional, como a opinião de que atividades e-learning precisam melhorar seu conteúdo, métodos e estilos de ensino, ou de que sessões clínicas presenciais são o tipo de ECS com maior satisfação.

Descritores:
Educação Continuada; Satisfação Pessoal; Educação a Distância; Estudantes de Ciências da Saúde; Desenvolvimento de Pessoal; Conferência Clínica


Objetivo:

verificar qué características organizativas, metodológicas y de recursos de la Educación Continua en Salud (ECS) ayudan a predecir mejor la satisfacción de los profesionales.

Método:

se utiliza un estudio transversal con regresiones logísticas multivariadas para predecir un alto promedio de satisfacción con los diferentes tipos de acciones educativas: Satisfacción general, Utilidad, Metodología, Organización y recursos y Capacidad docente. Se han analizado 25.281 cuestionarios de satisfacción respondidos por los profesionales de la salud que asistieron a 1228 actividades formativas, en Andalucía (España), durante el período de marzo a abril de 2012.

Resultados:

características que mejor predicen una alta Satisfacción general: Tipo de sesión clínica frente al taller (OddsRatio [OR]=2,07, p<0,001); modalidad de asistencia presencial (OR=3,88, p<0,001) o semipersonal (OR=2,83, p<0,001), frente al aprendizaje electrónico (e-learning); y 1-2 días de duración (OR=2,38, p<0,001) frente a 3-14 días. Una menor cantidad de horas (OR=0,99, p<0,001) y una menor cantidad de profesionales (OR=0,98, p<0,05) también aumentan la probabilidad. Acreditar las acciones educativas aumenta las probabilidades en las siguientes dimensiones: Utilidad (OR=1,33, p<0,05), Metodología (OR=1,5, p<0,01) y Capacidad docente (OR=1,5, p<0,01).

Conclusión:

el estudio aporta información relevante sobre aspectos que mejoran la satisfacción profesional, como que las actividades de aprendizaje electrónico deben mejorar sus contenidos, métodos y estilos de enseñanza o que las sesiones clínicas personales son el tipo de ECS con mayor satisfacción.

Descriptores:
Educación Continua; Satisfacción Personal; Educación a Distancia; Estudiantes del Área de la Salud; Desarrollo de Personal; Conferencia Clínica


Introduction

Continuing education is the group of educational activities designed to maintain, develop or increase knowledge, skills, and the performance of active professionals(11 Accreditation Council for Continuing Medical Education. CME Content: Definition and Examples. [Internet]. [cited 2016 Aug, 1]. Available from: http://www.accme.org/requirements/accreditation-requirements-cme-providers/policies-and-definitions/cme-content-definition-and-examples
http://www.accme.org/requirements/accred...
), with it therefore being differentiated from other stages of education, such as degree education, specialized education and post-graduate education.

This is an essential and strategic tool to improve quality in health systems(22 Muñoz-Castro FJ, Valverde-Gambero E, Villanueva-Guerrero L, Mudarra-Aceituno MJ, Vázquez-Vázquez M, Almuedo-Paz A. Evolución de la formación continuada acreditada tras la puesta en marcha de la Estrategia para la Seguridad del Paciente. Rev Calid Asist. [Internet]. 2012 Sep [Acceso 21 sep 2012];27(5):262-9. Disponible en: http://linkinghub.elsevier.com/retrieve/pii/S1134282X12000139doi:10.1016/j.cali.2012.01.008
http://linkinghub.elsevier.com/retrieve/...
), to improve results in the health of patients(33 Alford DP, Zisblatt L, Ng P, Hayes SM, Peloquin S, Hardesty I, et al. SCOPE of Pain?: An Evaluation of an Opioid Risk Evaluation and Mitigation Strategy Continuing Education Program. Pain Med. [Internet]. 2016 [cited 2020 Jan, 20];17(1):52-63. Available from: https://academic.oup.com/painmedicine/article-lookup/doi/10.1111/pme.12878doi:10.1111/pme.12878
https://academic.oup.com/painmedicine/ar...
-44 Aunión CD, Ruiz-Matas JH, Márquez AÁ, José Sánchez-Trincado Pavón M, Gil MA, Castro de la Nuez P, et al. Continuing Training Accreditation in the Organ Donation Process in Andalusia: Results From the Education and Training Unit of the Regional Transplant Organization of Andalusia. Transplant Proc. [Internet]. 2019 Nov [cited 2020 Jan, 20];51(9): 3012-4. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0041134519311807doi:10.1016/j.transproceed.2019.08.016
https://linkinghub.elsevier.com/retrieve...
), to satisfy health professionals(55 Allen LM, Palermo C, Armstrong E, Hay M. Categorising the broad impacts of continuing professional development: a scoping review. Med Educ. [Internet]. 2019 Nov [cited 2020 Jan 20];53(11):1087-99. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/medu.13922doi:10.1111/medu.13922
https://onlinelibrary.wiley.com/doi/abs/...
-66 Rouleau G, Gagnon M-P, Côté J, Payne-Gagnon J, Hudson E, Dubois C-A, et al. Effects of E-Learning in a Continuing Education Context on Nursing Care: Systematic Review of Systematic Qualitative, Quantitative, and Mixed-Studies Reviews. J Med Internet Res. [Internet]. 2019 [cited 2020 Jan, 15]; 21(10):e15118. Available from: https://www.jmir.org/2019/10/e15118/doi:10.2196/15118
https://www.jmir.org/2019/10/e15118/doi:...
), and to transfer and exchange knowledge(77 Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis. J Med Internet Res. [Internet]. 2016 Jan [cited 2020 Jan, 20];18(1):e2. Available from: http://www.jmir.org/2016/1/e2/doi:10.2196/jmir.4807
http://www.jmir.org/2016/1/e2/doi:10.219...
-88 Mundet-Tuduri X, Crespo R, Fernandez-Coll ML, Saumell M, Millan-Mata F, Cardona A, et al. Expectations and perceptions of primary healthcare professionals regarding their own continuous education in Catalonia (Spain): a qualitative study. BMC Med Educ. 2017 Nov 15;17. doi: 10.1186/s12909-017-1061-6
https://doi.org/10.1186/s12909-017-1061-...
).

Various systems of accreditation of continuing education have been developed, at the international level and addressed towards health professionals(99 American Dental Association. American Dental Association's Continuing Education Recognition Program (ADA CERP) recognition standards and procedures. [Internet]. 2017 Oct [cited 2018 Aug, 1]. Available from: http://www.ada.org/~/media/CCEPR/files/cerp_standards.pdf?la=en
http://www.ada.org/~/media/CCEPR/files/c...

10 American Medical Association. The AMA Physician's Recognition Award and credit system. [Internet]. 2017 [cited 2018 Aug, 1]. Available from: https://www.ama-assn.org/sites/default/files/media-browser/public/cme/pra-booklet_0.pdf
https://www.ama-assn.org/sites/default/f...
-1111 European Accreditation Council for Continuing Medical Education. EACCME Criteria For The Accreditation Of Live Educational Events (LEE). [Internet]. 2016 [cited 2018 Aug 1]. Available from: https://www.uems.eu/__data/assets/pdf_file/0016/40156/EACCME-2-0-CRITERIA-FOR-THE-ACCREDITATION-OF-LEE-Version-6-07-09-16.pdf
https://www.uems.eu/__data/assets/pdf_fi...
), in order to guarantee the quality level of the educational actions that are performed, by means of the periodical updating and continuous improvement of models(1212 Varetto T, Costa DC. Continuing Medical Education Committee and UEMS-EACCME. Eur J Nucl Med Mol Imaging. 2013 Feb;40(3):470-4. doi: 10.1007/s00259-012-2317-3
https://doi.org/10.1007/s00259-012-2317-...
-1313 Varetto T, Costa DC. The new UEMS-EACCME criteria for accreditation of live educational events (LEEs): another step forward to improve the quality of continuing medical education (CME) in Europe. Eur J Nucl Med Mol Imaging. 2014 Jan;41(1):191-6. doi: 10.1007/s00259-013-2590-9
https://doi.org/10.1007/s00259-013-2590-...
).

Since the year 2003, the Andalusian Agency for Healthcare Quality (Agencia de Calidad Sanitaria de Andalucía, ACSA) has implemented a model for accreditation of continuing education activities of health professionals(1414 Agencia de Calidad Sanitaria de Andalucía. Manual de estándares de actividades de formación continuada (ME 3 3_03). [Internet]. 2016 [Acceso 19 nov]. Disponible en: https://mejorao.acsa.junta-andalucia.es/acsa_formacion/html/ficheiro.asp? qq=RECURSOS&url= Manual+de+est%E1ndares+de+actividades+de+formaci%F3n+continuada+%5BME+3+3%5F03%5D%2Epdf
https://mejorao.acsa.junta-andalucia.es/...
), in consonance with the quality strategies of the Andalusian Health System(1515 Consejería de Salud, Junta de Andalucía. Estrategia de las Políticas de Formación del Sistema Sanitario Público de Andalucía [Internet]. Sevilla: Consejería de Salud; 2017 [Acceso 14 feb 2020]. Disponible en: https://www.juntadeandalucia.es/export/drupaljda/EstrategiaPolíticasFormación_v2_0.pdf
https://www.juntadeandalucia.es/export/d...
) and with the National Continuing Education Committee criteria. It is structured in quality elements related to aspects such as the need that justifies the education, the characteristics of the teaching-learning process that is to be carried out, the profile of the professionals participating in the education, the resources used in its planning and development, and the results of the educational process. The assessment of these elements determines the degree of quality of the education designed, and, with this, the accreditation or non-accreditation of the educational action in question.

In addition, within the educational activities that are performed, there are different modalities of education depending on the degree of personal attendance of the participants, “face-to-face learning”: with physical presence of the professionals, “blended learning”: it integrates the characteristics inherent to personal-attendance and non-personal-attendance without any virtual tools, “e-learning”: carried out via the new virtual communication tools; and different typologies, taking into consideration the characteristics of the educational process of educational activities(1616 Comisión de Formación Continuada de las Profesiones Sanitarias (CFC). Manual de procedimiento [Internet]. Madrid; 2013 Oct [Acceso 13 jun 2019]. Disponible en: http://canalsalut.gencat.cat/web/.content/80_Professionals/Formacio/Formacio_continuada/consell_catala_de_la_formacio_continuada_de_les_professions_sanitaries/manual_procedimientos_cfcps-nov2013.pdf
http://canalsalut.gencat.cat/web/.conten...
): “courses”: educational activities addressed towards the acquisition of knowledge, “workshops”: practical educational activities to develop skills, “clinical conferences”: periodical activities in which to exchange scientific information, opinions, and experiences on the commonalities of the daily practice.

The choice of these will depend, on one hand, on the detection of educational needs, and, on the other, on the learning objectives that may have been established and, finally, on the resources available for conducting the educational activities.

In recent years, there has been an exponential increase in education in the health area, supported by new technologies, also known as e-learning, internet-based learning, online learning, computer-assisted learning, or web-based learning(1717 Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals. BMC Med Educ. 2017 Oct 10;17(1):183. doi: 10.1186/s12909-017-1022-0
https://doi.org/10.1186/s12909-017-1022-...
). Various studies have indicated the advantages of this educational modality among health professionals, such as the flexibility of self-paced learning, adapted to the timings of the professionals, the plasticity to different styles of learning, the permanent availability of access to contents, the capacity to overcome time-related hurdles, the costs of travelling, the limitations of physical spaces for teaching, and the difficulties faced by professionals in rural environments distant from large educational centres(1717 Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals. BMC Med Educ. 2017 Oct 10;17(1):183. doi: 10.1186/s12909-017-1022-0
https://doi.org/10.1186/s12909-017-1022-...

18 Baptista RCN, Martins JCA, Pereira MFCR, Mazzo A. Students' satisfaction with simulated clinical experiences: validation of an assessment scale. Rev. Latino-Am. Enfermagem. 2014 Oct;22(5):709-15. doi: 10.1590/0104-1169.3295.2471
https://doi.org/10.1590/0104-1169.3295.2...

19 Hu A, Shewokis PA, Ting K, Fung K. Motivation in computer-assisted instruction: Motivation in CAI. Laryngoscope. [Internet]. 2016 Aug [cited 2020 Jan, 20];126:S5-13. Available from: http://doi.wiley.com/10.1002/lary.26040doi:10.1002/lary.26040
http://doi.wiley.com/10.1002/lary.26040d...
-2020 Paliadelis PS, Stupans L, Parker V, Piper D, Gillan P, Lea J, et al. The development and evaluation of online stories to enhance clinical learning experiences across health professions in rural Australia. Collegian. 2015;22(4):397-403. doi: https://doi.org/10.1016/j.colegn.2014.08.003
https://doi.org/10.1016/j.colegn.2014.08...
). In addition, international models of accreditation of continuing education consider specific standards for the modality of e-learning(2121 European Accreditation Council for Continuing Medical Education. EACCME Criteria For The Accreditation Of E-Learning Materials (ELM) [Internet]. 2016 [cited 2018 Aug, 1]. Available from: https://www.uems.eu/__data/assets/pdf_file/0017/40157/EACCME-2.0-CRITERIA-FOR-THE-ACCREDITATION-OF-ELM-Version-6-07-09-16.pdf
https://www.uems.eu/__data/assets/pdf_fi...
).

Documents such as Training Evaluation Field Guide: Demonstrating the Value of Training at Every Level(2222 United States Office of Personnel Management. Training Evaluation Field Guide. Demonstrating the Value of Training at Every Level [Internet]. 2011 [cited 2014 Sep, 11]. Available from: http://www.opm.gov/policy-data-oversight/training-and-development/reference-materials/training_evaluation.pdf
http://www.opm.gov/policy-data-oversight...
) recommend integrating various systems of evaluation into the design of educational activities, which may allow for the results achieved to be analysed. For this purpose, various models of evaluation of activities exist, the Kirkpatrick model(2323 Kirkpatrick DL, Kirkpatrick JD. Evaluación de acciones formativas: los cuatro niveles. Barcelona: Ediciones Gestión 2000; 2007.) being one of the most widely used.

The Kirkpatrick model proposes evaluation of education on 4 levels: Reaction/Satisfaction, Learning, Behaviour/Transfer, and Results/Impact. The use of this model has grown to cover the evaluation of continuing education activities, as demonstrated in numerous studies(2424 Bijani M, Rostami K, Momennasab M, Yektatalab S. Evaluating the Effectiveness of a Continuing Education Program for Prevention of Occupational Exposure to Needle Stick Injuries in Nursing Staff Based on Kirkpatrick's Model. J Natl Med Assoc. [Internet]. 2018 Oct [cited 2020 Jan, 20];110(5):459-63. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0027968417301724doi:10.1016/j.jnma.2017.11.002
https://linkinghub.elsevier.com/retrieve...
-2525 Rojo E, Maestre JM, Díaz-Mendi AR, Ansorena L, Del Moral I. Innovation in healthcare processes and patient safety using clinical simulation. Rev Calid Asist. 2016 Mar 7. doi: 10.1016/j.cali.2015.12.008
https://doi.org/10.1016/j.cali.2015.12.0...
). In order to facilitate standardized instruments for users that allow for the evaluation of given aspects related to the design and development of the educational activities carried out, ACSA created a tool named “eValúa”(2626 Agencia de Calidad Sanitaria de Andalucía. eValúa. Cuestionarios de evaluación de la formación [Internet]. 2014 [Acceso 12 mar 2018]. Disponible en: https://formacionsalud.acsa.junta-andalucia.es/agenciadecalidadsanitaria/formacionsalud/evalua/login.jsp
https://formacionsalud.acsa.junta-andalu...
), in which various online questionnaires are included and which allows for the data collected through them to be processed.

The questionnaire on the satisfaction of professionals included in eValúa is a tool that has been validated(2727 Esposito T, Muñoz-Castro FJ, Herrera-Usagre M, Periáñez-Vega M. Fiabilidad y validez para un cuestionario de satisfacción con la formación continuada en salud: el cuestionario de satisfacción del discente. FEM. 2015;18(3):197-203. doi: http://dx.doi.org/10.4321/S2014-98322015000400008
http://dx.doi.org/10.4321/S2014-98322015...
) and developed taking into account the criteria established in the ACSA accreditation programmes(2828 Agencia de Calidad Sanitaria de Andalucía. Programa Integral para la acreditación de la Formación Continuada de las profesiones sanitarias. [Internet]. [Acceso mar 2016]. Disponible en: http://formacionsalud.acsa.junta-andalucia.es/agenciadecalidadsanitaria/formacionsalud/gestor/galerias/descarga_documentacion_referencia/Manual_261108_def.pdf
http://formacionsalud.acsa.junta-andaluc...
) that is designed to provide a reply on level 1 of the Kirkpatrick evaluation model (reaction level). This questionnaire considers 23 items, grouped together in five dimensions (Utility, Methodology, Organization and resources, Teaching capacity, and Overall valuation), and other descriptive aspects of the educational activity, including the typology, the modality, and the number of professionals.

Taking some studies into account(2929 Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;(2):CD003030. doi: 10.1002/14651858.CD003030.pub2
https://doi.org/10.1002/14651858.CD00303...
), the educational modality used (face-to-face learning, blended learning, e-learning and b-learning) and the proportion of the number of professionals/teachers in the activities, have a lineal relationship with the quality of the education perceived. The main objective of this article is to verify which organizational, methodological, and resource-related characteristics of continuing health education help to best predict the professionals´ satisfaction.

Method

For this study, the Questionnaire on the Satisfaction of Professionals (QSP) has been used, with professionals who are experts in the quality of education, belonging to ACSA and to the Units of Education or management of knowledge of other entities of the Public Health System of Andalusia, having intervened in its preparation. For the preparation thereof, various questionnaires used to measure the degree of satisfaction of the participants in educational activities in the area of health were analysed and, based on this information, the dimensions were designed and the items were prepared. The psychometric properties of reliability and validity of the questionnaire have already been analysed, and improvements have been incorporated(2727 Esposito T, Muñoz-Castro FJ, Herrera-Usagre M, Periáñez-Vega M. Fiabilidad y validez para un cuestionario de satisfacción con la formación continuada en salud: el cuestionario de satisfacción del discente. FEM. 2015;18(3):197-203. doi: http://dx.doi.org/10.4321/S2014-98322015000400008
http://dx.doi.org/10.4321/S2014-98322015...
).

The study group consists of a random, non-stratified sample of educational activities addressed towards health professionals (physicians, nurses, pharmacists, and other health care professionals), in any centres in the region of Andalusia (Spain), during the period from March 2012 to April 2015, in which the eValúa QSP was used to explore the level of satisfaction of the participants with each activity (n=1,228 educational activities; N=25,281 professionals who have answered the QSP).

The evaluation of the satisfaction level must be carried out as from the day on which the educational action finalizes, at which moment the eValúa tool activates the corresponding link for the web version of the questionnaire. As from that date, the person responsible for the education or for coordinating the educational activity sends an e-mail from the application to all the professionals, with the information necessary to complete the questionnaire: the web link, the term, details on the activity, the purpose of the survey, and privacy rules. Alternatively, they may print the questionnaire out in paper format and hand it in by hand, on finalizing the education, having, subsequently, to introduce the results manually for the processing of the data.

Dependent Variables: All of the evaluation items of the QSP are shown on a scale on which 0 is the “lowest degree of satisfaction, or total disagreement” and 10 is the “highest degree of satisfaction, or total agreement”. The mean scores of all of the items contained in each one of the dimensions of the questionnaire that the professionals have to complete after receiving the education have been calculated: Overall satisfaction (2 items, Cronbach’s alpha=0.964), Utility (3 items, Cronbach’s alpha=0,913), Methodology (6 items, Cronbach’s alpha=0,95), Organization and resources (6 items, Cronbach’s alpha=0,923), and Teaching capacity (5 items, Cronbach’s alpha=0,972) (Table 3). Given that the dependent variables (DVs) of the model did not comply with part of the criteria to do a least squares regression (homoscedasticity, residual normality, and autocorrelation), it was decided to dichotomize the DVs to be able to launch five logistic regressions. Due to high mean scores in all of the DVs, all of the educational activities that are over the median have been considered as “high scores” while those that are below the median were classified as “low scores”. Therefore, the logistic regressions try to predict a “high score” using the independent variables.

Table 1
Descriptive statistics. Andalusia, Spain, 2012-2015
Table 2
Correlations between the variables included in the study. Andalusia, Spain, 2012-2015
Table 3
Logistic Regression models with high ratings of continuing health education courses. Andalusia, Spain, 2012-2015

Independent Variables: Each one of the activities included is shown with the following additional information to the evaluation: Number of professionals who have attended (continuous variable), Number of hours of duration of the educational action (continuous variable), Duration in days of the education (categorical variable: “1 to 2 days”, “3 to 14 days”, “15 to 50 days”, “More than 51 days”); Modality of the activity (categorical variable: “Face-to-face learning”, “e-learning” or “Blended learning”); Typology of the educational action (categorical variable: “course”, “workshop”, “clinical conference”); Activity accredited by ACSA (dichotomic variable: “yes” or “no”).

Firstly, a correlation analysis was performed between all of the non-categorical variables included in the study (Spearman’s rank correlation coefficient). Subsequently, five logistic regression analyses were performed with each one of the dichotomized satisfaction dimensions included in the QSP to verify which independent or explicative variables are the ones that have the most influence on there being a high level of satisfaction of professionals with the educational activity that they have received. For this, the R statistic package has been used. The Odds Ratio (OR) and its 95% confidence intervals were used. In addition to Nagelkerke pseudo-R-squared, the area under the Receiver Operating Characteristic (ROC) curve, or the capacity of the model to correctly classify the educational activities with high satisfaction (Area Under the Curve, AUC) have been included.

Results

In Table 1, the descriptive statistics are shown, both for the dependent and the independent variables. The mean number of attendees for the training activities was 27 with a high typical deviation (24) showing a wide range of training activities sizes. The mean duration was 11 hours but ranged between a minimum of 1 hour and a maximum of 301 hours, thus showing a very high typical deviation (21). Teaching capacity was the dimension that showed the highest mean score (8.61) while the Overall satisfaction indicator showed the lowest (7.89). Regarding the other characteristics of the training activities, 60% of them took place in 1 or 2 days, while another 35% took between 3 to 50 days. Close to 5% of the training activities used in this study lasted more than 51 days. The majority of those training activities were Physical (70%) while only 9% were e-learning and nearly 21% required a blended learning modality, which is to say, blending physical activities and e-learning activities. Almost half of the activities were organized as clinical sessions in a health care setting. Another 14% of the training activities were organized as workshops while 38% were specialized courses. Finally, with regard to the accreditation status, only 17% of the activities were accredited through the ACSA Accreditation Programme.

In Table 2, the correlations between the numerical variables included in the study are shown. The first thing to be highlighted is the negative correlation between the number of professionals and the mean satisfaction level in all of the dimensions of the QSP. In spite of the fact that these negative correlations are statistically significant, the correlation coefficients are very low. The existence of a low but statistically significant correlation between the number of hours of educational activity and a low level of overall satisfaction is to be highlighted. On addressing the correlations between the different dimensions of the questionnaire, two aspects stand out: firstly, satisfaction with the teaching capacity is the dimension that has the least correlation with the overall satisfaction of the educational activity. Secondly, the high correlation between the Methodology and Utility dimensions stands out. In the rest of the correlations between dimensions, high levels are maintained with coefficients of over 0.7.

In Table 3, a set of five logistic regressions are shown, one for each one of the dimensions that are valued in the questionnaire. The objective is to verify which characteristics have the most influence on predicting a high mean satisfaction level with the health education activity received by the professionals. With the exception of the Overall valuation (32.1%) and of the evaluation of Methodology (14.3%), the logistic regression models performed showed percentages of variance taken to be moderate, as well as a uniform performance in sensitivity analysis (AUC). This is the case of the models performed with the evaluation of Utility (5.5%), Organization and resources (6.2%), and Teaching capacity (8.1%).

The Clinical session or conference type of education shows ratings that are considerably higher in workshops in all categories, with the exception of Teaching capacity. The sessions show twice the probability of workshops to be highly valued overall and, specifically, with respect to the Organization and resources used, and three times with respect to the Methodology used. However, the sessions show 52% fewer probabilities of Teaching capacity being highly valued with respect to workshops (OR=0.657). The workshops are twice as likely as courses to be highly valued overall, although no statistically significant differences have appeared in the other dimensions.

In relation to the modality of the education, a clear trend is observed: the face-to-face learning and blended learning modalities are more likely to be highly rated by the professionals, in comparison with the e-learning modality, in all dimensions, except in the dimension of Organization and resources. In fact, e-learning activities show almost 4 times fewer probabilities than personal-attendance ones for being highly valued overall, and almost 3 times less than semi-personal-attendance ones. Semi-personal attendance activities are especially well valued with respect to those of the e-learning type in relation to Utility, Methodology used and Teaching capacity, whereas personal-attendance activities are better rated overall.

The duration, measurement via the numbers of hours entailed, and the time that the activity takes, measured in days, do not appear to have a significant lineal effect. Nevertheless, it is observed that those activities that are performed between 2 days and 2 weeks are 2.4 times less likely to be highly valued overall than those that only last for 1 or 2 days, irrespective of the number of hours that this may entail, or of their typology or modality.

Finally, when observing the effect of accreditation, the activities have almost 33% more probabilities to be highly valued in relation to their Utility, and 50% more in relation to their Methodology and to their Teaching capacity.

Discussion

The sample used in this study consisted of 1,228 training activities, of which almost half were clinical conferences, just over a third were specialized courses, and the rest were learning workshops. The vast majority were face-to-face activities while one third were e-learning or blended learning. The Satisfaction with the Methodology dimension presented the highest correlations with the rest of the dimensions, placing it as a key dimension. All dimensions showed very high levels of satisfaction in general, with scores above 7.8 out of 10.

The results have demonstrated that the e-learning modality, which is only supported by new information and communication technologies (ICTs), systematically shows lower satisfaction levels compared to the two other modalities (face-to-face learning and blended learning) in the majority of the dimensions studied: Utility, Methodology and Teaching capacity.

The results presented herein are consistent with those found in previous studies that showed lower professionals’ levels of satisfaction with e-learning activities with regard to the methodology used(1717 Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals. BMC Med Educ. 2017 Oct 10;17(1):183. doi: 10.1186/s12909-017-1022-0
https://doi.org/10.1186/s12909-017-1022-...
,3030 McCutcheon K, O'Halloran P, Lohan M. Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: A randomised controlled trial. Int J Nurs Stud. 2018 Mar 6;82:30-9. doi: 10.1016/j.ijnurstu.2018.02.005
https://doi.org/10.1016/j.ijnurstu.2018....
), the capacity of response by the teacher, and the suitability of the education for the educational needs of the professionals(55 Allen LM, Palermo C, Armstrong E, Hay M. Categorising the broad impacts of continuing professional development: a scoping review. Med Educ. [Internet]. 2019 Nov [cited 2020 Jan 20];53(11):1087-99. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/medu.13922doi:10.1111/medu.13922
https://onlinelibrary.wiley.com/doi/abs/...
,3030 McCutcheon K, O'Halloran P, Lohan M. Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: A randomised controlled trial. Int J Nurs Stud. 2018 Mar 6;82:30-9. doi: 10.1016/j.ijnurstu.2018.02.005
https://doi.org/10.1016/j.ijnurstu.2018....
). Nonetheless, those are advantages that were traditionally attributed both to online education and to semi-personal attendance education, or blended learning, such as flexibility(3131 Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, et al. E-learning for health professionals. Cochrane Effective Practice and Organisation of Care Group, editor. Cochrane Database Syst Rev. [Internet]. 2018 Jan 22 [cited 2020 Jan, 20]. Available from: http://doi.wiley.com/10.1002/14651858.CD011736.pub2 doi:10.1002/14651858.CD011736.pub2
http://doi.wiley.com/10.1002/14651858.CD...
), anonymity in the platforms that facilitate participation, or the greater availability to be able to combine this with personal life and work(3232 Sinclair PM, Kable A, Levett-Jones T, Booth D. The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: A systematic review. Int J Nurs Stud. [Internet]. 2016 May 1 [cited 2020 Jan, 15];57:70-81. Available from: http://www.sciencedirect.com/science/article/pii/S0020748916000122doi:10.1016/j.ijnurstu.2016.01.011
http://www.sciencedirect.com/science/art...
).

Recent studies found either low or very low evidence that e-learning education may have any effect on behavioural changes in health professionals or on patients’ outcomes(3232 Sinclair PM, Kable A, Levett-Jones T, Booth D. The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: A systematic review. Int J Nurs Stud. [Internet]. 2016 May 1 [cited 2020 Jan, 15];57:70-81. Available from: http://www.sciencedirect.com/science/article/pii/S0020748916000122doi:10.1016/j.ijnurstu.2016.01.011
http://www.sciencedirect.com/science/art...
-3333 Voutilainen A, Saaranen T, Sormunen M. Conventional vs. e-learning in nursing education: A systematic review and meta-analysis. Nurse Educ Today. 2017 Mar;50:97-103. doi: 10.1016/j.nedt.2016.12.020
https://doi.org/10.1016/j.nedt.2016.12.0...
). Nevertheless, due to the continuous development of software tools that allow for online education to be carried out in a more personalized way, such as adaptive e-learning(3434 Fontaine G, Cossette S, Maheu-Cadotte M-A, Mailhot T, Deschênes M-F, Mathieu-Dupuis G, et al. Efficacy of adaptive e-learning for health professionals and students: a systematic review and meta-analysis. BMJ Open. [Internet]. 2019 Aug 28 [cited 2020 Jan, 15];9(8). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6719835/doi:10.1136/bmjopen-2018-025252
https://www.ncbi.nlm.nih.gov/pmc/article...
), and the advantages that these courses allow for in support of educational self-management(1717 Lawn S, Zhi X, Morello A. An integrative review of e-learning in the delivery of self-management support training for health professionals. BMC Med Educ. 2017 Oct 10;17(1):183. doi: 10.1186/s12909-017-1022-0
https://doi.org/10.1186/s12909-017-1022-...
), it may not be ruled out that further investigation in the subject area and further innovation in the training programs alignment with the real needs of professionals may improve the results obtained to date(66 Rouleau G, Gagnon M-P, Côté J, Payne-Gagnon J, Hudson E, Dubois C-A, et al. Effects of E-Learning in a Continuing Education Context on Nursing Care: Systematic Review of Systematic Qualitative, Quantitative, and Mixed-Studies Reviews. J Med Internet Res. [Internet]. 2019 [cited 2020 Jan, 15]; 21(10):e15118. Available from: https://www.jmir.org/2019/10/e15118/doi:10.2196/15118
https://www.jmir.org/2019/10/e15118/doi:...
).

Another of the most important findings is related to the differences found depending on the type of education. Previous works have shown the utility and acceptability of interdisciplinary clinical sessions or conferences in the health area(3535 Maeda H, Tsujimura T, Yoshida K. Questionnaire study on the utility of autopsy case conferences related to emergency medicine practices: Medicine (Baltimore). [Internet]. 2019 Apr [cited 2020 Jan, 20]; 98(17):e15315. Available from: http://Insights.ovid.com/crossref?an=00005792-201904260-00050.
http://Insights.ovid.com/crossref?an=000...
). Their organization, which is often of the personal-attendance form in the same health centres in which the professionals perform their work, as well as a methodology focused on very specific objectives, tend to be highly-valued aspects(3636 Waldron MK, Washington SL, Montague GP. Cooperative Clinical Conferences: Nursing Student Pediatric Clinical Innovation. J Nurs Educ. 2016 Jul 1;55(7):416-9. doi: 10.3928/01484834-20160615-12
https://doi.org/10.3928/01484834-2016061...
-3737 Harrison Kelly S, Henry R, Williams S. Using Debriefing Methods in the Postclinical Conference. Am J Nurs. [Internet]. 2019 Sep [cited 2020 Jan, 18]; 119(9):56. Available from: https://journals.lww.com/ajnonline/Abstract/2019/09000/Using_Debriefing_Methods_in_the_Postclinical.34.aspxdoi:10.1097/01.NAJ.0000580280.87149.12
https://journals.lww.com/ajnonline/Abstr...
). However, in spite of the importance of clinical conferences in the improvement of interdisciplinary communication(3838 Prades J, Remue E, van Hoof E, Borras JM. Is it worth reorganising cancer services on the basis of multidisciplinary teams (MDTs)? A systematic review of the objectives and organisation of MDTs and their impact on patient outcomes. Health Policy. 2015 Apr;119(4):464-74. doi: 10.1016/j.healthpol.2014.09.006
https://doi.org/10.1016/j.healthpol.2014...
), there is a need to improve the pedagogical skills of teaching professionals(3939 Serçekus P, Baskale H. Nursing students' perceptions about clinical learning environment in Turkey. Nurse Educ Pract. 2016 Mar 1;17:134-8. doi: 10.1016/j.nepr.2015.12.008
https://doi.org/10.1016/j.nepr.2015.12.0...
). The results obtained have reconfirmed both points since clinical conferences are more likely to be highly valued as regards their methodology and form of organization, but less in relation to the capacities of the teaching professionals.

In spite of the fact that, in her study, Hall found no solid evidence on the impact of the duration of the education - whether this may be measured in hours or spread over days - in relation to the satisfaction level perceived or to the acquisition of knowledge(4040 Hall BM. How Cognitive Requirement of Prompt and Time in Course Are Correlated with Intersubjectivity within Threaded Discussions. [Internet]. 2011 [cited 2018 Apr, 6]. Available from: https://eric.ed.gov/?id=ED541076
https://eric.ed.gov/?id=ED541076...
), in the current study a slight trend has indeed been observed in the sense that the lower the number of hours, the greater the general satisfaction level, with such education being concentrated in a shorter period of time. The evidence has shown that the acquisition of knowledge by professionals in continuing education decreased slightly as the number of attendants at the course increased(4141 Slater PJ, Herbert AR, Baggio S, Donovan LA, McLarty A, Duffield J, et al. Evaluating the impact of national education in pediatric palliative care: the Quality of Care Collaborative Australia. Adv Med Educ Pract. [Internet]. 2018 Dec [cited 2020 Feb, 12];Volume 9:927-41. Available from: https://www.dovepress.com/evaluating-the-impact-of-national-education-in-pediatric-palliative-ca-peer-reviewed-article-AMEPdoi:10.2147/AMEP.S180526
https://www.dovepress.com/evaluating-the...
). The results presented here sustain this hypothesis because, as the number of professionals in educational activity increases, the overall mean satisfaction levels with the utility, the methodology, and the teaching capacity, decrease.

Finally, the possible future effect of accreditation on the satisfaction of professionals was also considered. Accreditation of continuing education has become a basic tool to guarantee the quality of contents and methodology of continuing health education in the National Health System(22 Muñoz-Castro FJ, Valverde-Gambero E, Villanueva-Guerrero L, Mudarra-Aceituno MJ, Vázquez-Vázquez M, Almuedo-Paz A. Evolución de la formación continuada acreditada tras la puesta en marcha de la Estrategia para la Seguridad del Paciente. Rev Calid Asist. [Internet]. 2012 Sep [Acceso 21 sep 2012];27(5):262-9. Disponible en: http://linkinghub.elsevier.com/retrieve/pii/S1134282X12000139doi:10.1016/j.cali.2012.01.008
http://linkinghub.elsevier.com/retrieve/...
). Despite its voluntary character, accreditation of continuing health education in the world has remained constant, after a slight decrease (7%) following the financial crisis(4242 ACCME. ACCME Data Report Growth and Evolution in Continuing Medical Education - 2016 [Internet]. Chicago: ACCME; 2017 [cited 2018 Apr, 10]. Available from: http://www.accme.org/sites/default/files/755_20170712_2016_Data_Report_Addendum.pdf
http://www.accme.org/sites/default/files...
). No studies analysing the impact of accreditation of continuing health education on the satisfaction level of professionals have been found. However, different studies in other countries have demonstrated the satisfaction level of health professionals with health accreditation processes(4343 Brinza EK, Zhu LJ, Lilly M, Manning WJ, Needleman L, Gornik HL. Accreditation is Perceived to Improve the Quality of Vascular Testing Facilities. JVU.. 2016 Jun 1;40(2):63-9. doi: https://doi.org/10.1177/154431671604000201
https://doi.org/10.1177/1544316716040002...
). This study has shown a slight trend towards greater satisfaction perceived with accredited activities as compared with non-accredited ones, in particular with respect to utility, to the methodology used, and to the teaching capacity. Nevertheless, further investigation is required in relation to the effect of the accreditation of continuing health education on clinical results, the effectiveness of learning, and the satisfaction of professionals.

Some limitations of the study are worth mentioning: firstly, the data are based on mean scores of each one of the dimensions for each one of the educational activities that have participated in the survey, instead of using data from individual replies by professionals. Some eventual limitations in relation to the scope of the results that this may entail should be considered. However, those limitations have been sufficiently compensated by another two strong points; on one hand, the size of the sample and, on the other, the fact that not only one dimension of the satisfaction level has been evaluated but rather five.

Secondly, it may be asked why a logistic regression has been used instead of an ordinary least-squares regression (OLS) for continuous dependent variables such as those of satisfaction, on a scale of 0-10. The dependent variables did not fulfil even half of the most important suppositions that they must meet for OLS regression, that is, homoscedasticity, residual normality, autocorrelation, and non-collinearity. In order not to modify the data, they were transformed in binomial distribution.

Conclusion

It can be concluded that the organizational, methodological, and resource-related characteristics that better predict the health professionals’ satisfaction with continuing education are the following, in this order: activities with a face-to-face learning modality, clinical conferences with trained teaching professionals, low number of hours, education programmes that are concentrated in short period of time, reduced number of professionals, and activities’ design based on quality criteria of accreditation models.

Lessons for the practice

  • The e-learning modality must be designed with learning objectives more adapted to the expectations of learners and of limiting the scope of the learning to knowledge rather than to the acquisition of skills.

  • While educational supports have progressed considerably in recent years, the methodological designs and the pedagogical capacities of teachers and professionals who present clinical conferences have not followed the same pace.

  • A reduced teacher-student ratio is recommended in order to facilitate learning and its applicability to employment positions.

It is recommended to base the design of the education on quality criteria of accreditation models.

  • *
    This article refers to the call “Educational technologies and innovative teaching methods in the training of human resources in health”.

Referencias

Edited by

Associate Editor: Andrea Bernardes

Publication Dates

  • Publication in this collection
    01 July 2020
  • Date of issue
    2020

History

  • Received
    27 Aug 2019
  • Accepted
    06 Apr 2020
Escola de Enfermagem de Ribeirão Preto / Universidade de São Paulo Av. Bandeirantes, 3900, 14040-902 Ribeirão Preto SP Brazil, Tel.: +55 (16) 3315-3451 / 3315-4407 - Ribeirão Preto - SP - Brazil
E-mail: rlae@eerp.usp.br