Assessment of websites on cleft lip and palate in Brazilian Portuguese

ABSTRACT Purpose: this study aimed at assessing the legibility, reliability, usability and coverage of websites on cleft lip and/or palate (CL/P) in Brazilian Portuguese and providing a global comparison. Methods: in order to evaluate the obtained data, four protocols were used, i.e., the Website Coverage Protocol, the legibility (by means of the COH METRIX PORT), the reliability (Discern Protocol), and the usability (by means of the System Usability Scale). Results: 98 websites were included with average legibility scoring of 31.6±11.7, rated as a hard readability level. Coverage average scoring was 19.9±3.79 from a total of 35 points. As for reliability, average scoring was 43.2±6.51 points, rated as acceptable. The usability scoring was 24.8±2.18 points, rated as the worst imaginable one. Conclusion: therefore, the assessed Brazilian websites on CL/P were rated difficult for legibility, requiring high-school level of education, restricted coverage and acceptable reliability, in addition to the low usability. The results were similar to those of websites from other countries, except for the legibility aspect, which was lower for Brazilian websites.


INTRODUCTION
Expectations and great changes arise for parents from pregnancy.This process of baby idealization can be disrupted when parents face a malformation diagnosis, in addition to doubts on the child development 1 .Pointed as the most frequent congenital anomaly, the cleft lip and/or palate (CL/P), is a craniofacial malformation, multifactorial etiology comprising the lip, alveolar ridge, hard and soft palate, occurring, worldwide, in one to each 700 born alive babies every year 2 .It is related to abnormalities in major structures of the oral cavity, hearing disorders, feeding problems, apart from the risk for the speech and language development and consequences for physical appearance 1,3,4 .
With the diagnosis, during the pre-natal period, the family has to deal with the "bereavement" of the idealized baby, but also starts the search for information 1 .When receiving the medical diagnosis, the Internet is the research source for the patient who is after problem-solving, people who underwent the same situation, and even other procedures from those recommended by the patient's doctor or healthcare professional.A study shows that 25% of the netizens search for websites and videos on health conditions, and 80% search for information on treatment of specific diseases 5 .This behavior is important to the extent that it empowers the population for decision-making, as well as to exercise their right of choice on health matters.However, such rights are hindered by their difficulty in selecting the most updated, scientific evidence-based information sources on the Internet 6 .
The integration of high-complexity and primary healthcare services is fundamental for investments in health-promotion actions, and telehealth and teleeducation are interesting ways for such actions to cover the greatest possible number of people, not only among the population at large 7 , but also among health professionals 8 .
Although the Internet enables more knowledge spread, facilitating the access to information by patients and professionals, there is still a great number of websites that provide data in a complex language 9 , difficult accessibility and large content variety on CL/P 10 .
Websites are analyzed in different ways and the attempt to standardize this process has been increasingly frequent in literature discussions, so that the population makes use of these materials judiciously.Readability is an important characteristic that indicates how easy a document is to read.This aspect may limit the audience that would have access to this material, in cases where there is a lower instructional level 11,12 .The quality of the content, considering everything from ethical aspects 13 to the coverage of the topics covered, added to the ease of use of the layout of each website 14 , make up relevant points to guarantee different characteristics that would facilitate the information to reach the user effectively.
Therefore, this study aimed to assess the legibility, reliability, usability and coverage of websites on CL/P in Brazilian Portuguese and provide a global comparison.

METHODS
This research study comprised the production of documental data, available and free access on the Internet, not being necessary to submit it to the Ethics Board.
The search was held by three evaluators by means of the following search engines: GoogleTM (www.google.com),BingTM (www.bing.com)e YAHOO!® (www.yahoo.com), in October 2021 and updated in October 2022, using the search keywords: 1-"palate fissure", 2-"lip fissure", 3-"lip and palate fissure", 4-"cleft lip", 5-"cleft palate", 6-"cleft lip and palate", 7-"hare lip" and 8-"wolf's jaw".Websites in Portuguese were included with available and accessible language for users, shown up to the 5th page of each database, providing information on CL/P, excluding scientific, academic websites, personal opinion sites, discussion groups and forums, in addition to videos on the subject.
About the evaluators, all were females, 21 years old, in the last year of speech-language pathology graduation.These evaluators had the task of selecting the websites (at the Google search, Bing and Yahoo search engine) and evaluating them blindly, through the Website Coverage Protocol, Discern Protocol and the System Usability Scale.In cases of conflict, consensus meetings were held together with the two experts supervising the work, speech-language pathologists, ten years graduated, with clinical experience in cleft lip and palate.
Each evaluator searched in one of the mentioned databases, individually; 143 websites were initially found in the Google search engine, 161 in the Bing search engine, and 176 in the Yahoo search engine, totaling 481.After the removal of duplicate websites, and the application of the exclusion criteria, the comparison and analysis of the obtained data were performed, resulting in the final assessment of 98 websites.The websites were arranged in a list in ascending numerical order according to their position in each search engine (order the websites were located), for the organization of the exhibition of the results of this research.
In order to assess the legibility, the COH METRIX PORT tool was used, adapted from the Coh-Metrix tool for Brazilian Portuguese 11 .The Coh-Metrix tool calculates the indexes in order to assess text cohesion, coherence and comprehension difficulty, using several levels of linguistic analysis: lexical, syntactic, discursive and conceptual.This tool is automatic from Microsoft Windows, so it was not necessary for the evaluators to carry it out in a blinded way, being generated in just one check.To do this, a text in TXT format is inserted and a series of counts are returned, such as FLESH INDEX (an intelligibility index).Its indexes range from 0 to 100, being 0 to 25 rated as a very difficult reading level, requiring higher-education level of schooling, matching specific academic fields; from 25 to 50, difficult reading level, that is, from high-school to higher education level of schooling; from 50 to 75, an easy level, that is, compatible from the 5th to the 9th grades; and from 75 to 100, a very easy reading level, understandable for those with level of schooling from the 1st to the 4th grades of elementary school 12 .
The Website Coverage Protocol was developed by the authors of this research, to analyze seven aspects by means of a scale from 1 to 5 (score 1 means the worst performance, and score 5, the best performance): definition, causes and consequences, classification, guidance to parents on development, feeding and treatment for the CL/P provided by a speech-language pathologist.First, the experts met with the evaluators to calibrate the essential topics in each of the topics that would be evaluated on the coverage of the websites.After clarifying the topics, the evaluators accessed all sections of each website, following the graduation proposed for this aspect.
The same care was taken with the Discern Protocol and System Usability Scale.Before the evaluators started analyzing and scoring the websites, a meeting was held with the experts to study the questionnaires and clarify the parameters to be used.For this training, the application of the protocols was exemplified in a site that had been excluded, to resolve doubts and standardize this judgment.
The Discern Protocol is used in research in the health area, exclusively.With it, it is possible to create an analysis and content assessment pattern of a publication.The protocol comprises 16 questions divided in three groups.Questions 1 to 8 assess the text reliability; questions 9 to 15 assess the text quality, and question 16 is the general assessment of the information.Each question is responded in a scale from 1 to 5, in which the lowest number means that the assessed aspect is not present in the text, while the highest number means that the assessed aspect is fully present in the text.By adding up each question, it is possible to rate the websites in 5 categories: very poor (scores between 16 and 26), poor (between 27 and 38), fair (between 39 and 50), good (between 51 and 62), and excellent (over 63) 13 .
The System Usability Scale (SUS) is a questionnaire with 10 questions used to assess the effectiveness, efficiency and satisfaction of users towards websites.The odd statements indicate the positive points in this category, and the even statements show usability problems.Scoring may vary from 0 to 100, and rating can be attributed as the worst imaginable from 0 to 25, poor from 25 to 39, OK from 39 to 52, good from 52 to 74, excellent from 74 to 85, and the best imaginable from 85 to 100 14 .
In order to compare the data obtained with the world panorama, a literature review was carried out in the PubMed/Medline database with the following search strategy: ("evaluability"[All Fields] OR "evaluate"[All Fields] OR "evaluated"[All Fields] OR "evaluates"[All Fields] OR "evaluating"[All Fields] OR "evaluation"[All Fields] OR "evaluation s"[All Fields] OR "evaluations"[All Fields] OR "evaluative"[All Fields] OR "evaluatively"[All Fields] OR "evaluatives"[All Fields] OR "evaluator"[All Fields] OR "evaluator s"[All Fields] OR "evaluators"[All Fields] OR ("evaluability"[All Fields] OR "evaluate"[All Fields] OR "evaluated"[All Fields] OR "evaluates"[All Fields] OR "evaluating"[All Fields] OR "evaluation"[All Fields] OR "evaluation s"[All Fields] OR "evaluations"[All Fields] OR "evaluative"[All Fields] OR "evaluatively"[All Fields] OR "evaluatives"[All Fields] OR "evaluator"[All Fields] OR "evaluator s"[All Fields] OR "evaluators"[All Fields])) AND ("website"[All Fields] OR "website s"[All Fields] OR "websites"[All Fields] OR "site"[All Fields]) AND ("cleft lip and palate"[All Fields] OR "cleft palate"[All Fields] OR "cleft lip"[All Fields]) Studies that had the objective of evaluating websites about cleft lip and palate and that had used at least one of the four parameters adopted in this study (readability, coverage, reliability and usability) were

RESULTS
Ninety-eight (98) websites on child CL/P were analyzed in Brazilian Portuguese.The term used which found the greatest number of websites was "cleft lip and/or palate".Chart 1 shows the number of websites found by keywords and search engines.included, not being mandatory to use the same instruments that were used.
The questionnaire results were tabulated and described in terms of percentage, median and standard deviation.Website assessment studies from other countries were searched in the literature and their results were compared to the ones in the current study.Statistical analysis was conducted by means of Anova Test in order to compare Discern ratings to the other parameters, using the Post-hot Tukey´s Test.Pearson´s correlation was used for the correlation Chart 1. Selection of websites by keywords used in three search engines, applying the inclusion criteria and removing duplicates Regarding the legibility, the assessed websites obtained the mean score of 31.6, rated as difficult reading level, requiring high school to higher education levels of schooling from users.The minimum achieved was 6.51, and the maximum was 57.1.

Keywords
In the information coverage on CL/P, definition, cause, consequences, classification, guidance for parents and treatment can be highlighted.Definition was the highest rated one, while CL/P treatment had the lowest rating, as described in Table 1.Some websites also inform the medical specialties which treat CL/P: pediatricians, surgeons, oral and maxillofacial surgeons and plastic surgeons, otorhinolaryngologists, dentists, speech-language pathologists, psychologists, nutritionists, physical therapists, nurses, social workers and educators.Other websites also mention geneticists, obstetricians, traumatologists as specialists who treat CL/P.On average, the usability of the websites was rated "the worst imaginable" (Table 2), with scores ranging from 19.32 to 29.67 (maximum scoring) 14 .Table 4 shows that coverage as well as usability had the highest scores for higher ranks of reliability by means of the Discern 13 .

Coverage
The websites obtained scores from 28.68 (minimum scores) to 63.34.Reliability by means of the Discern was considered fair on average (Table 3).Website W92 had the highest score in two categories assessed in this study, coverage (32.33) and reliability (63.34).To legibility, the highest score was to W44 with 57.1% and to usability was W12 with 29.67.These data were compared to website assessments on the studied theme from other four countries, described in Chart 2. Only two studies, from the USA and Switzerland, presented an evaluation of three parameters, the others presented an evaluation of one or two parameters.In general, the other countries presented a similar evaluation to the Brazilian websites.

websites included
The average scoring was 31.6rated as difficult reading level (high school to higher education in order to understand its reading) The average scoring was 19.0 from a total of 35 points.
The average scoring by means of the DISCERN was 43 points, rated as fair.
The average scoring of the websites was 24, that is, the worst imaginable. USA

DISCUSSION
There is an expressive offer of websites on CL/P in Brazilian Portuguese (Chart 1), although with different qualities in terms of legibility, reliability, usability and coverage.Thus, it is relevant that their contents be certified, with reliable information, fundamentally for a genetic disorder so frequently occurred worldwide as the CL/P.
Regarding legibility, the provision of clear, legible contents is one of the main points for any website design, with good experience of use.Most of the assessed websites evidenced a difficult level for understanding, which may limit the access to information.Thus, not only aesthetic issues should be considered, but also the way the text has been written, can make the reader lose interest in the content, takes longer to read it or even cannot understand the information clearly.Therefore, the legibility level was rated as difficult, requiring schooling compatible to high school or higher education, corroborating an American study 10 , but contradicting two others American studies 9,15 and one Swiss study 16 , which evidenced an easier reading level on CL/P.However, the influence of a significantly smaller sample of websites in the two latter studies 15,16 should be considered for comparison purposes with the current study.
Legibility is important to be considered when assessing websites; however, it cannot be an isolated parameter, since it deals with quantitative metrics, syllable and word count 12 .Content understanding goes beyond quantitative aspects.Exemplifying, a study in which parents accessed websites on CL/P with different levels of legibility, contrastingly, they could obtain similar understanding of their contents 5 .
By assessing the content coverage of 98 Brazilian websites, it was observed that information on treatment had the lowest score, followed by guidance for parents and classification of CL/P (Table 1), corroborating a study from Switzerland, which pointed content gaps on treatment, specifically on the role of orthodontistry 16 .
There was variability in Brazilian websites regarding the professionals involved in the treatment of patients with CL/P.Literature was observed to support that variability of professionals specialized in treating CL/P 16,17 .This aspect should be considered during guidance to patients/parents/guardians.Additionally, website refinement is also suggested, aiming at greater coverage on the topic of specialized professionals for patients to be referred to.
The assessment on the usability of the websites had an average scoring of 24.8, rated as "the worst imaginable" (Table 2), which corroborates an American study that found that 28.1% of websites on CL/P did not work 10 .Ensuring the usability of Internet sources is important for their contents to be explored by greater number of people from different age ranges and different levels of experience/ease towards the Information and Communication Technologies 10 .
As for the reliability (Discern protocol), most websites included in this study were considered fair (Table 3).One of the lowest scored questions was number 8, which asks if the website mentions other information sources, corroborated by the results of a Swiss study, which found that 78% of the websites did not cite other references to access 16 .Comparing to a Greek study 18 , similar results were also found, with average scoring of 50% from the total scoring of the questionnaire used.However, when compared to websites from the USA 6 the maximum scoring of the Brazilian study was lower.
Still about the reliability, the analysis by rank enabled to evidence the influence of different parameters to assess the websites, stressing the importance of a multivariate assessment, and such evaluations agreed with each other (Table 4).The diversity of the adopted analyses in studies from different countries should also be pointed out 6,9,10,15,16,18 (Chart 2), and agreement on the protocols and variables for the evaluation of the information on the Internet is deemed necessary.
The highest rated website (W92) in two categories is a recurring research source, also ranking first in the search engine (rank page), despite its limitation regarding legibility and usability, important factors for greater accessibility.
The importance of the current study lies in the assessment of the impact of quality information online for laypeople, in an attempt to verify if proper information on CL/P has been provided in order to guide patients and their caregivers.Moreover, the findings in this study may guide further research for the creation of a scientific evidence-based website, with relevant legibility and quality under ethical principles, in order to provide people with detailed information on CL/P.As a limitation of this research, it should be noted that there is still no standardization of the protocols used to evaluate websites in general, at a national and international level, which makes it difficult to strengthen and advance investigations in this regard.

Chart 2 .
Comparison of website assessment parameters on cleft lip and/or palate from different countries between all evaluations.The significance level adopted was 5%.Jamovi, version 1.2.25, was the selected program.

Table 2 .
14ability of the website information on cleft lip and/or palate14 Captions: SUS -System Usability Scale; Q -question; SD -Standard Deviation.

Table 3 .
13formation reliability of the websites on cleft lip and/or palate13 Captions: Q -question; SD -Standard Deviation.

Table 4 .
13an values, standard deviation and correlations of the Discern rating13, according to rank, legibility, coverage and usability Captions: Anova Test and Post-Hoc Tukey´s Test, in order to compare the four groups in the Discern rating; (*) <0.05 statistical significance.