Telemedicine Effectiveness in the First Elective Orthopedic Care Compared to a Standard Face-to-face Visit

Abstract

Objective

The study aimed to compare whether the diagnoses of orthopedic diseases at telemedicine (TM) consultations are the same as those established at face-to-face visits.

Method

Primary, observational, prospective, analytical study, with subjects from the local municipal network who were referred to the orthopedics outpatient clinic from May to June 2021. Subjects underwent two assessments: a telemedicine (TM) consultation and a face-to-face (FF) visit. Two different physicians attended to the patients and established a diagnosis. The physician performing the FF visit was not aware of the previous diagnoses. We compared the diagnoses obtained at both modalities to assess the degree of similarity. In addition, we determined the time required for consultations and the degree of satisfaction of the physicians.

Results

We evaluated 43 patients and seven physicians, totaling 44 TM and 43 FF visits. The diagnostic similarity index was 81.4%. TM consultations were shorter (mean time, 4.8 minutes) than FF visits. Physicians were less satisfied with TM in the four criteria evaluated (respective scores of 79.1, 23.3, 46.6, and 37.2).

Conclusion

TM consultations have a diagnoses agreement higher than 80% compared with FF visits. On the other hand, TM consultations were faster, and physicians were less satisfied with them in comparison with FF visits.

Keywords
treatment adherence and compliance; diagnosis; orthopedics; teleconsultation

Resumo

Objetivo

O objetivo do estudo foi comparar se os diagnósticos das doenças ortopédicas realizados por teleconsulta (TC) são os mesmos dos atendimentos presenciais.

Método

Estudo primário, observacional, prospectivo, analítico, com dados colhidos de maio a junho de 2021, com participantes provenientes da rede municipal local que foram encaminhados ao ambulatório de ortopedia de referência e oferecida participação no estudo com duas avaliações: a primeira por teleatendimento e a segunda de forma presencial. Cada participante foi atendido por dois diferentes profissionais, e cada um emitiu um diagnóstico. Os profissionais do atendimento presencial não conheciam os diagnósticos prévios. Os diagnósticos emitidos foram comparados para avaliar o grau de semelhança. Ainda, foi aferido o tempo para realização dos atendimentos e o grau de satisfação do profissional participante.

Resultados

Foram avaliados 43 pacientes e 07 profissionais participaram, totalizando 44 TC e 43 atendimentos presenciais. O índice de semelhança do diagnóstico foi de 81,4%. A TC teve um tempo menor para realização (média de 4,8 minutos), que o presencial. A satisfação dos profissionais foi menor na TC nos quatro critérios avaliados, sendo, respectivamente, 79,1, 23,3, 46,6 e 37,2.

Conclusão

A TC tem concordância no diagnóstico superior a 80% em comparação ao atendimento presencial. Já a realização do teleatendimento teve menor tempo de duração e os profissionais se consideraram menos satisfeitos em relação ao atendimento presencial.

Palavras-chave
cooperação e adesão ao tratamento; diagnóstico; ortopedia; teleconsultas

Introduction

The improvement in audiovisual communication within the last 20 years, especially with the widespread use of smartphones and greater access to high-speed internet,¹ allowed the expansion of telemedicine.²,³ In 2020, the COVID-19 pandemic made social distancing a requirement and boosted health systems to implement remote care. The enactment of the Brazilian federal law number 13989 from April 15, 2020, about telemedicine (TM) during the COVID-19 crisis recognizes the importance of telehealth as an instrument for citizen protection.⁴

Studies showed that TM satisfaction rates are comparable with face-to-face (FF) visits, and patients who underwent virtual consultations are more likely to seek them again.⁵ Buvik et al.⁶ analyzed orthopedic patients undergoing TM and FF visits and revealed that 99% of them classified TM as very satisfactory or satisfactory. In addition, 86% of the subjects would prefer TM in the future. There is also an economic impact resulting from cost reduction.⁷

Remote assistance by digital means (telehealth) is growing because it is safe, efficient, and cheaper.⁷ However, health professionals and patients who are not used to this technology report limitations. These limitations include the inability to perform semiology, the lack of established commercial rules, insecurity with the accelerated implementation, and discomfort with this tool.⁸

This study aimed to evaluate the agreement in diagnoses of a first elective orthopedic consultation performed by TM compared with an FF visit. We hypothesize that the diagnostic agreement between these modalities of care is high.

Material and Methods

Primary, observational, prospective, analytical study. The research ethics committee approved this protocol.

Eligible participants were adults up to 75 years old who sought elective orthopedic care for the first visit to our institution after a referral from the municipal network. We did not include emergency, urgency, or fracture follow-up visits. After scheduling at the appointment center, the hospital's administrative team invited the subjects to the study; in case of compliance, we asked patients to fill in an informed consent form (ICF) and a demographic data form (Annex 1, supplementary material). In addition, we checked the equipment available for remote access and the patient's familiarity with digital tools for video calls. Patients had to have a cell phone capable of video calls using the Microsoft Teams application (Redmond, Washington, USA).

Each patient underwent visits in both modalities: first, a TM consultation and, at another time, an FF visit. Different orthopedists were responsible for each patient's consultations, so the second physician would not be aware of the first diagnosis. TM occurred at a conventional outpatient care room with a high-speed internet access desktop, a camera (Ecam X, HK REXSO COM TECH), a microphone, and speakers (flat3w RMS, Multilaser) for video calls using the Microsoft Teams platform (Redmond, Washington, USA). For convenience, a secretary from our institution distributed TM and FF visits to prevent the same orthopedist from seeing the same patient using both modalities.

Outcomes and data standardization

The primary outcome was the agreement between the diagnosis established at TM and FF (considered the gold standard), either in its descriptive form or as code from the 10^th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10). Secondary outcomes included consultation time and the physician's satisfaction with the modality. The principal investigator analyzed the descriptive diagnoses for relative and not absolute similarity; for instance, we considered knee pain and gonalgia, or low back pain and intervertebral disc disorder, the same diagnosis. In cases of divergence, two other researchers would evaluate the diagnosis and define this agreement.

Telemedicine

TM was not standardized. Each orthopedist performed it according to their knowledge and experience; moreover, the physician was free to request maneuvers from the patient. TM started with a link to a Microsoft Teams (Redmond, Washington, USA) video call. The lead author created this link and shared it with the patients. TM started with video recording (a platform resource allowed by the subject). During TM, the physician used a specific form (Annex 2, supplementary material), including clinical history, physical examination, special tests, complementary exams (if available), descriptive diagnosis with its ICD-10 code, prescribed treatment, and visit time, that is, the recording time. Treatments would not occur at this moment, only registered. In the end, the physician completed a satisfaction questionnaire adapted from Buvik et al.⁵ This questionnaire evaluated the following four criteria: patient cooperation for information collection, examination/assessment, ease in providing information to the patient, and treatment. Each criterion was classified per a Likert scale with five rating levels (very good, good, neither good nor bad, bad, and very bad) (Annex 3, supplementary material). The questions about evaluation and treatment included an additional criterion, “not applicable,” as an option selected by a physician unable to perform them. Scores ranged from 1 to 5. After TM, the subject was referred to an FF visit with a different orthopedist unaware of any previous findings.

Face-to-face Visit

The FF visit followed the same steps as TM, filling in forms (Annexes 2 and 3, supplementary material). This visit was conventional, with a referral for treatment, which did not affect our study.

Data protection and anonymity

We assured participants of privacy regarding their information and recordings. We did not identify them personally during data analysis using a Microsoft Excel 2013 spreadsheet.

Sample size calculation

We calculated the sample size using the G*Power 3.1 software to test the correlation between the medical diagnosis at TM and FF. Literature data provided the effectiveness of TM as a diagnostic tool based on a study published by Buvik in 2016, in addition to the research feasibility, which allowed the evaluation of the effect size as high (w = 0.50). We estimated the need for 44 participants considering a two-tailed test, 95% confidence, and 80% power.

Statistical Analysis

The descriptive analysis consisted of absolute (n) and relative (%) frequencies of qualitative variables and mean ± standard deviation values of quantitative variables. Paired Student's t-test detected differences in visit time, and the Wilcoxon test revealed differences in physicians' perceptions about FF and TM. Cohen's d calculated the effect size. The chi-square adherence test assessed the percentage of absolute diagnosis agreement between TM and FF visits. The Spearman correlation test determined the relationship between variables. The analyzes were performed with IBM SPSS statistical software version 20.0 (IBM Corp., Armonk, NY, USA), adopting p < 0.05 as statistical significance.

Results

The initial study sample consisted of 44 participants, including 27 women. The subjects were 18 to 73 years old, with an average age of 48. Thirty-four patients studied up to high school, and 16 had active jobs. One subject was excluded from the analysis as she did not undergo an FF visit due to hospitalization for COVID-19. In total, there were 43 subjects for statistical analysis (Table 1).

Table 1
Demographics of the sample (n = 43)

Variable		Mean ± Standard deviation n (%)
Gender
	Female	27 (62.8%)
	Male	16 (37.2%)
Age (years)		48.1 ± 13.3
Functional status
	Retired	10 (23.4%)
	Actively working	16 (37.3%)
	Unemployed	6 (14.0%)
	On disability	6 (14.0%)
	Other	5 (11.6%)
Reason for referral
	Referral from primary care	33 (76.7%)
	Maintenance/Follow-up at the University Hospital	7 (16.3%)
	Other	3 (7.0%)

The most frequent diagnoses were knee pain and low back pain both at TM and FF visits (Figs. 1 and 2, respectively). The diagnostic agreement was 81.4% (35 of 43 diagnoses) according to the similarity in diagnosis description, ICD-10 code, or both (X² = 16.953; p < 0.001) (Fig. 3).

Fig. 1
Frequency of diagnoses established at telemedicine consultations (n = 43).

Fig. 2
Frequency of diagnoses established at face-to-face visits (n = 43).

Fig. 3
Physicians' perception on patients' cooperation during a face-to-face visit and a telemedicine consultation. *Statistically significant difference, p < 0.05.

There was a statistically significant difference in visit time, which was shorter for TM compared with FF (14.0 ± 5.7 vs. 18.8 ± 7.1 minutes, respectively; p < 0.001; d = 0.75). The observed size effect suggests that the average difference of 4.8 minutes in visit time is relevant from a practical point of view (Fig. 4).

Fig. 4
Physicians' perception on the ability to assess/examine the patient during a face-to-face visit and a telemedicine consultation. *Statistically significant difference, p < 0.05.

Physicians' satisfaction with TM was lower in the four criteria evaluated. The first criterion, regarding patients' cooperation, scored 79.1 points out of 100 (Z = -2.282; p = 0.004). The second criterion, about the ability to assess/examine the patient, scored 23.3 (Z = -5.442; p < 0.001). The third criterion measured the ability to provide information to the patient and scored 46.6 (Z = -4.915; p < 0.001). Lastly, the fourth criterion evaluated the ability to treat patients and scored 37.2 (Z = -5.334; p < 0.001). It is noteworthy that these scores were the sum of “good” and “very good” and present statistical significance (Figs. 3–6).

Fig. 5
Physicians' perception on the ability to provide information to the patient during a face-to-face visit and a telemedicine consultation. *Statistically significant difference, p < 0.05.

Fig. 6
Physicians' perception on the ability to treat the patient during a face-to-face visit and a telemedicine consultation. *Statistically significant difference, p < 0.05.

Discussion

The evaluation of the diagnosis of orthopedic disease by TM showed an 81.4% agreement rate compared with FF visits in our series. This finding is consistent with a study from Cotrel et al.⁹ These authors observed an 83% agreement rate in the physical therapy diagnosis comparing TM and FF. Moreover, our findings suggest the effectiveness of TM compared with FF, supporting its use for patients with geographical barriers and mobility issues requiring specialized care.⁹

TM consultations were faster than FF visits in our series, differing from the literature, in which TM had a longer¹⁰ or at least similar duration.⁵ This data can assist further studies about TM implementation in reference centers for musculoskeletal conditions.

This study identified a lower degree of physicians' satisfaction with TM compared with FF visits in the four criteria evaluated. According to the Likert scale, the lowest satisfaction rates referred to patient assessment/physical examination (23.3 points) and treatment prescription (37.2 points). It is worth mentioning that physicians must feel comfortable and satisfied with any work method before its implementation.

This dissatisfaction was even lower than previously expected. The unfeasibility of a physical examination by direct contact limits remote orthopedic care. The place chosen by the patient and the free handling of the sound and video equipment (i.e., smartphones) with no fixed, stable points or a standard distance also hinder assessment. All these factors can influence the results.

This level of professional dissatisfaction is inconsistent with the literature, which demonstrated a high degree of satisfaction with TM by orthopedists and patients, with 98% of remote consultations rated as “good” or “very good.”⁵. Brennan et al.¹¹ evaluated TM for patients' assessment and treatment in an emergency department. They reported that, on average, physicians' satisfaction rate was 3.8 (ranging from 1, not very satisfied, to 5, very satisfied) regarding diagnoses and treatment in this population. The authors conclude that TM is a good technique based on patients' and physicians' assessments.

TM is potentially beneficial to patients because it eliminates geographical barriers, expanding access to healthcare. However, its adoption has been limited by issues with logistics, technology platforms, healthcare providers, physicians, and patients.⁷ The pandemic accelerated TM adoption,³ and this unscheduled speed could explain the dissatisfaction observed in our study.

Our study is a pioneer in the Portuguese language regarding TM evaluation in orthopedics. It demonstrated that its implementation is feasible. The understanding of the diagnostic agreement difference between TM and FF visits may improve by increasing evaluators' training and using an appropriate environment and fixed, well-positioned audio and video equipment. In addition, an increased familiarity with the platform can improve TM performance and satisfaction rates. A sample standardization with a similar syndromic diagnosis may increase concordance.

The limitation of this study was the lack of assessment of the degree of patient satisfaction. This evaluation could have contributed to the results. The lack of experience of the physicians was also an issue, and it must be addressed by further studies.

Conclusion

TM consultation has 81.4% diagnostic agreement and a shorter duration compared with FF visits. However, physicians reported being less satisfied with it.

• Study developed at the Orthopedics and Traumatology Service, Hospital Universitário, Universidade Federal de Juiz de Fora, Juiz de Fora, MG, Brazil
• Financial Support
  This study had no financial support from public, commercial, or non-profit sources.

References

• ¹
  Tachakra S, Wang XH, Istepanian RS, Song YH. Mobile e-health: the unwired evolution of telemedicine. Telemed J E Health 2003;9 (03):247–257
• ²
  Baker J, Stanley A. Telemedicine technology: a review of services, equipment, and other aspects. Curr Allergy Asthma Rep 2018;18(11):60
• ³
  Makhni MC, Riew GJ, Sumathipala MG. Telemedicine in orthopaedic surgery: challenges and opportunities. J Bone Joint Surg Am 2020;102(13):1109–1115
• ⁴
  Brasil. Diário oficial da união. Lei 13989 de 15 de abril de 2020. Dispõe sobre o uso da telemedicina durante a crise causada pelo coronavírus. Brasilia: DOU, 2020. (Publicado em 16/04/2020. Edição:73, Seção:1, Página 1)
• ⁵
  Buvik A, Bugge E, Knutsen G, Småbrekke A, Wilsgaard T. Quality of care for remote orthopaedic consultations using telemedicine: a randomised controlled trial. BMC Health Serv Res 2016;16(01):483
• ⁶
  Buvik A, Bugge E, Knutsen G, Småbrekke A, Wilsgaard T. Patient reported outcomes with remote orthopaedic consultations by telemedicine: A randomised controlled trial. J Telemed Telecare 2019;25(08):451–459
• ⁷
  Haider Z, Aweid B, Subramanian P, Iranpour F. Telemedicine in orthopaedics and its potential applications during COVID-19 and beyond: A systematic review. J Telemed Telecare 2022;28(06):291–403
• ⁸
  Buvik A, Bergmo TS, Bugge E, Smaabrekke A, Wilsgaard T, Olsen JA. Cost- Effectiveness of telemedicine in remote orthopedic consultations: randomized controlled trial. J Med Internet Res 2019; 21(02):e11330
• ⁹
  Cottrell MA, O’Leary SP, Swete-Kelly P, et al. Agreement between telehealth and in-person assessment of patients with chronic musculoskeletal conditions presenting to an advanced-practice physiotherapy screening clinic. Musculoskelet Sci Pract 2018;38:99–105
• ¹⁰
  Nakagawa K, Kvedar J, Yellowlees P. Retail outlets using telehealth pose significant policy questions for health care. Health Aff (Millwood) 2018;37(12):2069–2075
• ¹¹
  Brennan JA, Kealy JA, Gerardi LH, et al. Telemedicine in the emergency department: a randomized controlled trial. J Telemed Telecare 1999;5(01):18–22

Publication Dates

• Publication in this collection
  23 Oct 2023
• Date of issue
  Jul-Aug 2023

History

• Received
  04 Mar 2022
• Accepted
  18 July 2022