Computerized protocol of orofacial myofunctional evaluation with scores : usability and validity

Purpose: To test the usability of Computerized Orofacial Myofunctional Evaluation (OMES) protocol and analyze its validity. Methods: The study was divided into three stages: the first stage, production of the computerized version of OMES. The second stage was the validation of the user’s interface, in which 100  OMES protocols of a database, filled in printed version, were transferred using the computerized instrument. Necessary changes to the system have occurred at this stage. In the third stage, usability of the OMES protocol in multimedia version, three evaluators transferred data from other 25 printed protocols from database for the computerized version, and the time to transfer the data of each protocol was computed and compared between examiners by one-way ANOVA. Moreover, these evaluators analyzed the usability of computerized protocol according to the “Ten principles of Heuristics usability” as described in the literature. Results: The computerized protocol satisfied the principles of heuristics usability, according to the evaluation of the three Speech-Language Pathology evaluators, and the average time spent by the evaluators to transpose the data of each protocol to the software ranged from 3.1±0.75 to 3.83±0.91 minutes. Conclusion: The Computerized AMIOFE protocol is valid and had its usability/functionality confirmed.


INTRODUCTION
Technological advancement and qualification of professionals enabled the construction of electronic protocols.Various health services have implemented them, or are in the implementation phase, for clinical application and scientific research.Therefore, Speech-Language Pathology and Audiology must keep up with this moment of transition and participate in it.
Electronic protocols provide better access to information, greater security and electronic exchange of data between institutions, as well as facilitate collective research, with the possibility of retrieval and cross-checking of this information (1) .
Previously, its use was limited due to the cost of the equipment, its maintenance, and the lack of skilled labor or the possible resistance of people to computers.However, it is possible to create these protocols today, increasing the rate of accuracy of records, with low cost, reduced physical space, and minimal training of personnel (2)(3)(4)(5)(6) .
These tools can facilitate administrative and financial organization of consultations; staff time in handling procedures; retrieval of patient information, knowledge, and availability of this knowledge where and when it is necessary for adequate decision-making; and, in some cases, the generation of diagnosis and therapeutic guidance (1,7,8) .
On the basis of this, we developed a computerized version of the orofacial myofunctional evaluation with scores (OMES) (9) protocol to optimize the records for clinical use and research.
Briefly, the OMES protocol was designed to provide sufficient data for detection and grading of orofacial myofunctional disorders, without being too extensive and comprehensive.Previously, it has been validated for children (9) , youth, and adults, with good sensitivity and specificity (10) .
For a software to be considered valid and for its usage to be proper, it must go through a stage known as usability (functionality) inspection, which is a way of evaluating user interfaces (11) .
Usability is defined in ISO 9241-11 as: "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use" (p. 3) (12) .
Therefore, it concerns man-machine interaction.To be easily accepted, the instrument should be user-friendly, easy to use (13) , and its validity is related to the perceived satisfaction and usefulness by users (14) .
This study aimed to determine the usability of the OMES computerized protocol and analyze its validity.

METHODS
The project was approved by the research ethics committee of Hospital das Clínicas of Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo (USP-HCFMRP) according to the HCRP protocol no.15602/2-12.Participants (evaluators) were informed about the objectives and methods of the study and were asked to sign a free and informed consent.

Production of the computerized version of orofacial myofunctional evaluation with scores
In the OMES computerized protocol, the characteristics of the original version were maintained, and consequently its psychometric properties (9) .Software was developed by an undergraduate student of the Biomedical Informatics course under the guidance of a teacher in the area.For its creation, the Java programming language, executable in the Windows operating system, was used in the following order: 1. entry in the system and selection of existing or new protocol; 2. identification data of the patient; 3. evaluation data on appearance and posture; 4. mobility assessment data; 5. data on functions; 6. data from the functional evaluation of the occlusion; 7. placeholder for final comments.

Validation of the user interface
In this pretest stage of the instrument, 100 OMES protocols, completed in hard copy and taken from the prior database of the research team, were used.All protocols were transferred to the computerized version by an undergraduate student.
Twenty-five printed protocols were randomly selected from the total and transferred to the computerized OMES by another team member.In order not to create duplicates, the records were entered into the system with a code before their identification.
Subsequently, each user listed the possible changes in the software related to its operation and/or errors detected.This information was cross-checked and discussed, and then presented, so that the necessary changes were made in the area of Biomedical Informatics, before the next step.

Usability of the multimedia version of the orofacial myofunctional evaluation with scores protocol
The corrected version of the OMES computerized protocol was tested for its validity as follows: A. Three Speech-Language therapists (mean age: 25±0.8 years), with prior training in the area of orofacial motricity and with different levels of training (from 30 to 66 months, average: 46 ±18.3 months) for the use of the OMES protocol (printed version), participated as evaluators of usability.They scanned the data independently and did not exchange information.
Data from other 25 printed protocols, different from the ones in the previous step, were transferred from the database to the computerized version.In order not to create duplicates in the system, each evaluator entered a different code into the system to identify each protocol.B. Time for the data transfer of each protocol was computed.C. The three evaluators also independently analyzed the usability of the system in accordance with the "Ten Usability Heuristics" proposed by Nielsen (11) .For each of the items described, each evaluator responded to one of the alternatives: does not satisfy (score 1), partially satisfies (score 2), and satisfies (score 3).The instrument to evaluate usability, containing the heuristics and their descriptions, is presented in Chart 1.

Data analysis
Descriptive statistics were performed for the variables involved.The examiners were compared in terms of time spent for the transfer of information by one-way analysis of variance test.

Validation of the user interface
During the pretest of the OMES computerized protocol, problems were found and changes needed were proposed.In general, the main errors found were related to the standardization of markers in the protocol; buttons that were not performing their functions correctly, or even that were missing; absence of items from the printed

Match between system and the real world
The system should speak the user's language, with words, phrases, and concepts familiar to the user, rather than system-oriented terms.Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue.Support undo and redo.

Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing.Follow platform conventions.

Error prevention
Even better than good error messages is a careful design that prevents a problem from occurring in the first place.Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible.The user should not have to remember information from one part of the dialogue to another.
Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Flexibility and efficiency of use
Accelerators -unseen by the novice user -may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users.
Allow users to tailor frequent actions.

Aesthetic and minimalist design
Dialogues should not contain information that is irrelevant or rarely needed.Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation.Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
Search: Nielsen (11) protocol; and overlapping of some data when opening an already filled protocol.Before starting the usability test step of multimedia version of the OMES protocol, adjustments were made and problems already listed were solved.Thus, the computerized protocol, as previously mentioned, followed the pattern of the printed protocol, as shown in Figures 1 and 2.

Usability of the multimedia version of the orofacial myofunctional evaluation with scores protocol
The computerized protocol complied with the usability heuristics, according to the evaluation of three Speech-Language therapists, with scores ranging from 28 to 29, in a total of 30 points.The principle evaluated with the lowest score was (5) "error prevention".The data and amounts are shown in Table 1.The average time spent by the evaluators for the transposition of data of each protocol to the software ranged from 3.1 to 3.83 minutes.The time spent by evaluator 2 was significantly higher than that by evaluator 3 (p<0.01).Statistical comparisons are presented in Table 2 and Graph 1.

DISCUSSION
In this study, heuristic usability of the OMES computerized protocol was determined with excellent results in the ratings of the three users.
A usability problem can be defined as any characteristic, observed in a given situation, which may delay, hinder, or prevent the completion of a task, annoying, embarrassing, or traumatizing the user (15) .
In the case of the computerized OMES protocol, only the item regarding the possibility of "error prevention" was rated by two evaluators as partially satisfying.In fact, the program does not inform the evaluator if, for example, they forgot to enter any data from the evaluation.Therefore, there is no lock that can prevent continuity.But, however, if a given piece of information cannot be obtained, this does will not prevent the continuity of the evaluation and registration.The average time spent for the transposition of the data was brief, no more than 3.83±0.91minutes, and the difference between two of the evaluators, although statistically significant, did not exceed 1 minute.This time does not concern the evaluation of patients with simultaneous input of data in the electronic protocol, but, as explained, only the transposition of the printed protocols to the program.
The objective regarding the computerized OMES protocol was to make it functional.Following the principles proposed by Nielsen (11) , the information appears in a natural and logical order, with a user-friendly language, as already outlined in the original protocol, facilitating its management.
Electronic protocols present many conveniences to the user and ensure improved information management and quality of research (1) .In clinical terms, the computerized version of the protocol in question will add convenience, speed, and ease of visualization of results: with just one command ("click"), you can enter the result of the evaluated item.For each category of the protocol, such as appearance, posture, mobility of the stomatognathic system components, and functions (breathing, chewing, and speaking), the software presents the sum as soon and the evaluation is completed.When the assessment is complete, the total score is informed and corresponds to the orofacial myofuncional condition of the individual evaluated.
From this, the professional can define the need for orofacial myofuncional therapy for a given patient, comparing the numerical results of their assessment to the normal parameters previously described (16,17) .
It is noteworthy that the use of the computerized OMES protocol does not eliminate the need for knowledge in the area of orofacial motricity and the need for training in evaluation.
A careful orofacial myofuncional evaluation, especially when the instrument had been tested for validity and has good levels of sensitivity and specificity, favors the correct diagnosis and proper decision on therapy (18) .
The usability of the OMES computerized protocol for the evaluation of patients is feasible and a digital database is generated with all the information.Therefore, no more data entry is necessary after the evaluation, which will reduce the time to organize these, as well as improve information quality and accuracy of records (4) .The data relating to patients and the results can be retrieved quickly, clearly, without generating doubts (3) , and with reduced costs (5) .
The need for computerization in various areas, including in health, seems increasingly indispensable, because its advances have opened many possibilities for the use of information technology in clinical and scientific research (4) .Scientific research has especially grown, both qualitatively and quantitatively (3) .
According to our knowledge, the OMES computerized protocol is the first instrument of orofacial myofunctional evaluation with an electronic version, with proven construct and criteria validity (9,10) , as well as usability heuristics, developed in the area of orofacial motricity, in a digitalized version.On the basis of our experience, we believe that it has potential to foster advances in clinical practice and in scientific research in the area.

CONCLUSION
The OMES computerized protocol had its usability/functionality confirmed and proved useful for the storage and retrieval of orofacial myofuncional evaluation data.

Figure 1 .Figure 2 .
Figure 1.Example screenshot of the orofacial myofunctional evaluation with scores computerized protocol regarding the assessment of mobility

1 .
CoDAS 2014;26(4):322-7Mean time: mean total time spent in minutes by a Speech-Language therapist for the transfer of 25 printed protocols to the computerized version Graph Average total time spent per evaluator for the typing of the protocols, in minutes, with respective standard deviations 5

Chart 1 .
Usability Heuristics Evaluation conducted by evaluators with regard to the protocol Orofacial Myofunctional Evaluation with computerized scores

Table 1 .
(11)uation of usability heuristics of the orofacial myofunctional evaluation with scores computerized protocol, according to the principles of Nielsen(11)

Table 2 .
Total time spent per evaluator for the transfer of 25 orofacial myofunctional evaluation with scores protocols from hard copies to the computerized version Means and standard deviations, in minutes; Means with different letters indicate differences in the Tukey post test Caption: p = probability in the ANOVA test; SD = standard deviation