A n a l y s i s o f D e n t i s t r y Yo u T u b e V i d e o s R e l a t e d T o C O V I D - 1 9

This study aimed to analyze the videos available on YouTube related to dentistry and the novel coronavirus (COVID-19), as there is no such analysis in the existing literature. The terms “dental” and “COVID-19” were searched on YouTube on May 9, 2020. The top 116 English-language videos with at least 300 views were analyzed by two observers. Data was saved for each video, including target audience, source, country of origin, content, number of views, time watched, average views, duration, like/dislike ratio, and usefulness. Total video information and quality index (VIQI) scores were calculated, consisting of flow, information, accuracy, quality, and precision indices. Non-parametric tests were used for analysis. The analyzed videos were viewed 375,000 times and totaled 20 h of content. Most videos were uploaded by dentists (45.7%), originated from the United States (79.3%), and contained information targeted towards patients (48.3%). Nearly half of the videos (47.4%) were moderately useful. For the usefulness of the videos, statistically significant differences were found for all indices as well as total VIQI scores. A comparison of the indices according to the relevance of the videos showed statistically significant differences in the videos’ information and precision indices and total VIQI scores. The results of this study showed that dentistry YouTube videos related to COVID-19 had high view numbers; however, the videos were generally moderate in quality and usefulness.


Introduction
In late 2019, a novel form of coronavirus emerged in Wuhan, China (1). After the virus rapidly spread around the world, the World Health Organization (WHO) declared the disease a pandemic in March 2020 and named it COVID-19 (2). As of June 10, 2020, approximately seven million cases of COVID-19 had been reported worldwide, and the death toll had reached 400,000 (3). Common methods of COVID-19 transmission include direct contamination, coughing and sneezing droplet inhalation, and face-to-face contact (4,5).
Dental procedures involve high risk of cross-infection between patients and practitioners (6). During the COVID-19 pandemic, it has been suggested that non-urgent elective procedures be postponed (7). In emergencies, it is necessary to take effective infection control measures in countries affected by COVID-19 (8). The literature indicates that precautions taken in dentistry procedures can prevent COVID-19 infection (4). These precautions include creating pre-control triage, measuring fever with a thermometer before entering the patient clinic, arranging isolation rooms for infected patients, performing only emergency procedures during the epidemic period, complying with hand hygiene, wearing protective equipment, avoiding intraoral imaging, using a pre-procedure oxidative mouthwash, good ventilation, minimizing the use of highspeed devices, caries cleaning and periodontal treatments manually, disinfection more carefully, sufficient knowledge of dental personnel, and other tele-dentistry applications (4,6,9).
One of the most important factors in preventing COVID-19 transmission is sharing accurate information (10). Information can be published and shared without any institutional or equivalent quality control on the internet; therefore, it is entirely the uploader's responsibility to determine whether the shared information is true or false (11). Although the WHO and the Centers for Disease Control and Prevention have published a great deal of content online for this purpose, the spread of misinformation is also increasing (3). YouTube (an online video-sharing platform) is the second most popular website worldwide (12). During the Ebola and Zika virus pandemics, the millions of times that informational videos were watched on YouTube demonstrated the importance of this platform in disseminating information (13,14). The terms "coronavirus" and "COVID-19" became the most searched words on Google and YouTube in the first four months of 2020 (15). It is important for dentists to be aware of the content and quality of the COVID-19 information provided online and to create accurate and efficient data sources that can be used to counsel patients (16).
Although three recent studies about COVID-19 preventive behaviors in the context of YouTube have been conducted, to the best of our knowledge, there has been no study focusing specifically on dentistry-related YouTube videos in this area (10,17,18). The purpose of this study is to analyze YouTube videos related to dentistry and COVID-19 as a source of medical information.

Material and Methods
This study was approved by the Ministry of Health, Republic of Turkey (No: 2020-05-20T17_22_42).
Based on the Cohen's effect size (0.40 effect size), 85% power, and 0.05 error margin, 116 was calculated as the total sample size, using the G*Power software, version 3.1.9.2.
The researchers accessed YouTube on May 9, 2020. A new account was created for this study, and the terms "dental" + "COVID-19" were searched. Since the term "dental" is more inclusive compared to the term "dentistry", the term "dental" was used in the search. Only English-language videos that had at least 300 views and were uploaded after January 2020 were included in this study; videos in languages other than English were excluded (19). One of the videos included was uploaded more than once. The URLs of the first 116 results matching the study criteria in decreasing order of "relevance" were saved electronically due to daily changes. If a video offered subtitles, subtitles were used to improve understanding. All videos were reviewed and analyzed by two dentomaxillofacial radiologists with eight (M.O.) and twenty (I.P.) years of clinical experience, for interobserver reliability. For intra-observer reliability, the first observer viewed half of the videos (n=58) a second time.

Assessment of Content
The following data were saved for each video: target audience (dentists and patients); source (official institution [American Dental Association, WHO, etc.]; country of origin (United States of America, India, England, Canada, and other); main content (general information about COVID-19, infection control measures, dental emergencies, patient information, or legal/financial issues); number of views; time watched (elapsed day after video upload); average views per day; duration (in minutes); like/dislike ratio; and usefulness. To assess the value of the videos of the contents, a "usefulness score" was designed to categorize each video. The scoring was modified from the previous YouTube studies related to dentistry, by consensus of three specialist dentists at least 10 years of experience (20,21). The videos were evaluated in terms of the symptoms, transmission routes, and protection of COVID-19, dental infection control measures, patient informing, dental emergencies, tele-dentistry, and legal/financial issues. Each item was scored as one-point. The scores ranged from 0 to 8: Scores between 0 and 2 were classified as less useful, 3 to 5 were moderately useful, and scores between 6 and 8 were extremely useful. The first 58 videos were considered high-relevance (HR), and the second 58 were considered low-relevance (LR).
The video information and quality index (VIQI) was used to determine video quality (16,22). The VIQI, consisting of the contents in the Global Quality Scale (GQS), is used to evaluate the overall quality of the videos (16). The contents of the index included flow, information accuracy, quality (images, animations, interviews, subtitles, and summary), and precision (title-content compatibility) indices. Each component of the VIQI score was evaluated using a 5-point Likert scale ranging from 1 (poor quality) to 5 (high quality). The equivalents of the scores were made according to the following criteria: Score 1: poor quality, poor flow, most information missing, not useful; Score 2: generally poor quality, poor flow, very limited information; Score 3: moderate quality, moderate flow, some important information is available but others are not, somewhat useful, Score 4: good quality, good flow, most of the relevant information is available, but there are deficiencies in the topics, useful, Score 5: excellent quality, excellent flow, very useful (23).

Statistical Analysis
The NCSS 2007 software package (Kaysville, Utah, United States) was used for statistical analysis. Descriptive statistics (mean, standard deviation, median, frequency, rate, minimum, and maximum) were performed, and the distribution of the data were calculated using the Shapiro-Wilk test. The Kruskal-Wallis test was used to compare three or more groups of quantitative data that were not normally distributed, and the Mann-Whitney U test was used to compare two groups that were not normally distributed. For intra-observer reliability, the Wilcoxon test was used to compare periodic quantitative data (VIQI scores), and the McNemar test was used for qualitative data (usefulness). For inter-observer reliability, the Mann-Whitney U test was used to compare quantitative data, and the chi-squared test was used to compare qualitative data. For correlation of data, Spearman's test was performed. Significance was evaluated at the p<0.05 level.

Distribution of Data
The total duration of all analyzed videos was approximately 20 h. These 116 videos attracted a total of 375.000 views. More detail regarding the distribution of the data is provided in Table 1.
The duration of the videos ranged from 1 minute to 97 min, and the total VIQI scores ranged from 1 to 20. Video quality indices and total VIQI scores averages were moderate. The descriptive analysis of the characteristics and quality of the videos is summarized in Table 2. Table 3 presents the comparison of indices according to target audience, source, country of origin, content, usefulness, and relevance. In terms of target audience, it was found statistically significant that the flow (p=0.002), information (p=0.001), and quality (p=0.024) indices and total VIQI scores (p=0.001) of videos intending dentists were higher than these for patients. When videos were compared by source, there were statistically significant differences in the flow (p=0.012) and information (p=0.001) indices and total VIQI scores (p=0.017). The level of information received from official institutions was statistically higher than dentists (p=0.001). In terms of video content, there were statistically significant differences in flow (p=0.028), information (p=0.001), and quality (p=0.043) indices and total VIQI scores (p=0.001). The information indices and total VIQI scores of the patient informing videos were found to be statistically lower compared to infection control measures and legal-financial issues (p=0.001). It was revealed that the flow (p=0.144), information (p=0.023), and precision (p=0.009) indices and total VIQI scores (p=0.021) of HR videos were statistically higher than LR.

Correlation Analysis
The correlation analysis of the data is presented in Table  4. For the indices and total VIQI score, the correlations varied positively between weakly to extreme relationships (p=0.001).

Reliability Analysis
Regarding the usefulness of the videos and total VIQI scores; no statistically significant difference was detected for either intra-observer (p=0.135 and p=0.059) or interobserver reliability (p=0.966 and p=0.897), respectively.

Discussion
Scientific accuracy and quality of medical information on the internet are very variable. The shared information is directly published without any quality control; therefore, the accuracy of the shared information is entirely the initiative and responsibility of the uploader (11). This situation leads to a significant level of information pollution as well as useful information. In literature, there are various published articles regarding the content of YouTube videos on many topics (13,14,16).
The present study is the first analysis to focus on dentistry-related YouTube videos that discuss COVID-19.  *Usefulness scores between 0 and 2 were classified as less useful, 3 to 5 were moderately useful, 6 to 8 were extremely useful.  (14) SD: Standard deviation, VIQI: Video information and quality index.

Analysis of dental videos of COVID-19
they provided about prevention behaviors, symptoms, and disease transmission. They reported that most videos did not discuss suggested prevention behaviors, symptoms, and transmission, with most instead referring to quarantine and travel restrictions. The same authors conducted another YouTube study, this time focusing on the periodic change in videos on behaviors to alleviate COVID-19 transmission (17). According to the video analysis, which was conducted at two-month intervals, the total number of video views tripled, although the content related to COVID-19 prevention behaviors did not vary significantly. In another study, Khatri et al. (18) analyzed the contents of English-language and Mandarin-language COVID-19 videos on YouTube. English-language videos had more content related to prevalence and testing, while Mandarinlanguage videos had better content related to clinical symptoms. In our current study, patient informing, and infection control measures were the major contents. The information level and quality of 'patient informing' videos were found to be low. This result reveals the need to upload more high-quality videos to YouTube related to patient information about COVID-19. Similarly, since the number of videos about 'dental emergencies' and 'legal-financial issues' to be made during the COVID-19 pandemic is low, there is a need for more videos to be prepared for these contents. Informing both healthcare professionals and patients about these contents will be important for public health. Khatri et al. 's study also found that mean medical information and content index (MICI) scores were low for videos in both languages (6.71, and 6.28, respectively) (18). Although we did not use the MICI in our study, the total scores of the VIQI index we used were high compared to the previous study.
In a 2012 study of YouTube videos about West Nile virus, 41.8% of videos originated from news agencies (22). A similar result (41.9%) was reported in a 2014 analysis of videos about Ebola hemorrhagic fever (16). In our study, 22.4% of videos originated from news agencies. Considering that news agencies were the group with the lowest information and total video quality index in our study, the decrease in the ratio of videos produced by this group can be considered promising. It would be beneficial for official institutions that are the source of more highly informative videos to increase their focus on this powerful platform.
Previous YouTube studies have used other video quality indexes, like the modified Discern index and the MICI (16,18,24). The Discern index measures the written health information, while the MICI is used to analyze the medical information in videos related to prevalence, clinical symptoms, transmission, testing, and treatment (16,18,24). The VIQI is a more general index for determining the overall quality of a video (16). Since videos related to dentistry were examined in our study, we found it appropriate to use the VIQI, which allowed us to analyze flow, information, quality, and precision. The VIQI index queries general video quality while other indices question medical information.
Since 2012, YouTube has been using an algorithm called Dwell Time in video searches. In this algorithm, the ordering of videos is determined by watch duration. If a viewer watches a video for a long period of time, the algorithm ranks the video more highly (25). Accordingly, the first half of the videos we reviewed were deemed HR and the second half LR. When we compared the video quality indices by relevance, information, precision (i.e., appropriate title preparation), and total VIQI score, the HR group was significantly higher than the LR group. This supports the validity of the YouTube algorithm. However, there was no difference between groups in the flow or quality indices. We posit that this is related to an increase in the quality of videos overall since more people have experience with video preparation, as well as technological improvements.
Since content on the internet is ever-changing, the results of this study offer time-sensitive information, like other cross-sectional investigations. Certain video characteristics, such as number of views and like/dislike ratio, change with every view. Another limitation of the current study was its restriction to English-language video content, which meant that videos in other languages could not be evaluated. Although the "likes" on the videos give some idea of viewers' reactions, the evaluation of viewers' opinions was limited since most videos were not watched by many people. Finally, since the COVID-19 outbreak is highly emergent, there was no other study investigating the subject examined here. As such, we could not compare many of our results with similar studies. These limitations should be taken into consideration in future studies as the numbers and view counts of YouTube videos about dentistry and COVID-19 increase.
This study showed the following results regarding dental YouTube videos during the COVID-19 pandemic period: Dental YouTube videos from official institutions had higher levels of knowledge and video quality. It will be beneficial for experts, universities, and other institutions to upload sufficient duration YouTube videos with scientific content, especially during COVID-19.
The contents of the videos were found to be insufficient related to dental emergencies and legal-financial issues during the COVID-19 pandemic. There is a need for more videos to be prepared for these contents.
The authors think that videos shared on the Internet, especially in the field of health, should be published after being subjected to an institutional approval and control system. Therefore, it can be beneficial for YouTube to screen and remove the videos during the upload of low-quality videos containing unnecessary/wrong information and increase the relevance of useful videos.
The authors suggest to internet users that they should be incredibly careful while learning about health topics and prefer videos uploaded by corporate sources.