Validation of instrument to assess software of patients’ risk classification

Methodological study, conducted in three stages: adaptation of the instrument, content validation through Delphi technique and pre-test. The results were analyzed through Content Validity Index, Overall Content Validity Index and Inter-rater Reliability.

Results:

The final version of the instrument comprises 8 characteristics and 28 sub-characteristics, being 37 general questions to computer experts and nurse and 7 specific questions to computer experts, including 1 question and excluding 3 questions of the original instrument. We obtained Overall Content Validity of 92% and Inter-rater Reliability of 100% in the second Delphi round.

Final considerations:

The instrument has content validity, allowing to assess the technical quality and functional performance of the software applied to the risk classification of patients.

Descriptors:
Classification; Emergency Medical Services; Validation Studies; Validation of Computer Programs; Healthcare Quality

Characteristics and sub-characteristics	CVI (%)^* * Content Validity Index applied to characteristics and sub-characteristics;		IRR (%)^ Inter-rater Reliability;		Overall CVI (%)^* ***** Overall Content Validity Index;
Characteristics and sub-characteristics	1st Delphi round	2st Delphi round	1st Delphi round	2st Delphi round	1st Delphi round	2st Delphi round
Functional suitability	74	96	67	100	70	93
Functional completeness	50	100
Functional correctness	89	87
Functional appropriateness	83	100
Reliability	62	90	50	100
Maturity	83	100
Fault tolerance	50	80
Recoverability	83	100
Availability	33	80
Usability	85	94	67	100	70	93
Recognition of suitability	77	92
Learnability	83	100
Operability	83	90
Accessibility	100	100
Error protection	100	100
User Interface Aesthetics	67	80
Performance efficiency	79	97	67	100
Time	100	100
Resources	50	80
Capacity	87	100
Compatibility	63	95	0	100
Interoperability	58	90
Coexistence	67	100
Safety	53	92	40	100
Confidentiality	67	80
Integrity	100	93
Non-repudiation	100	100
Accountability	0	^** ****** Questions excluded in the first Delphi round.
Authenticity	0	^** ****** Questions excluded in the first Delphi round.
Maintainability	63	88	20	100
Analyzability	67	100
Changeability	50	80
Testability	83	100
Modularity	67	80
Reusability	50	80
Portability	83	95	100	100
Adaptability and installability	83	90
Replaceability	83	100

Characteristic	Sub-characteristic	Analysis and suggestion
Functional suitability	Functional completeness Functional correctness Functional appropriateness	Rephrase the writing, standardize non-inducing verbs and replace the term "system" by "software."
Functional suitability	Functional appropriateness	Considered unsuitable by an expert.
Reliability	Fault tolerance	Question 8: replace the term "system" by "the same."
	Availability	Question 10: complete the question with the term "when necessary."
	Maturity Fault tolerance Recoverability Availability	Considered unsuitable by an expert.
Usability	Recognition of suitability Learnability Operability User Interface Aesthetics	Show the acronym AACR^* * AACR - Assessment with User Embracement and Risk Classification. , described at the beginning of each characteristic, to remind the reader of its meaning.
	User Interface Aesthetics	Replace the term "graphic design" by "software design." Add what design characteristics must be assessed in more detail.
	Recognition of suitability	Insert the key question: "Was there training to use the software?"
	Learnability	Question 18: describe in detail which data the software facilitates the output. Considered unsuitable by an expert
Performance efficiency	Resource	Add what types of external resources are required to use the software. Correction of punctuation: remove quotes and semicolons, use commas to separate the external resources.
Performance efficiency	Capacity	Considered unsuitable by an expert.
Compatibility	Interoperability	Question 30: rephrase the writing by standardizing the verbs. Question 31: inform with which databases it is possible to exchange information.
Compatibility	Coexistence	Question 32: replace the term "shared environment" by "when other screens are being simultaneously processed." Considered unsuitable by an expert.
Safety	Reliability	Add the term "individualized and reliable user and password authentication."
Safety	Accountability Authenticity	Items considered inappropriate by six experts, resulting in their removal.
Maintainability	Analyzability Changeability Testability Modularity Reusability	Rephrase the writing using the term "software," because its absence causes problems regarding clarity and transmit incomplete ideas.
Portability	Adaptability and installability	Question 43: replace the term "systems" by "operating environments."
Portability	Replaceability	Question 44: Add in the question the possibility of updating the software.

Associação Brasileira de Enfermagem SGA Norte Quadra 603 Conj. "B" - Av. L2 Norte 70830-102 Brasília, DF, Brasil, Tel.: (55 61) 3226-0653, Fax: (55 61) 3225-4473 - Brasília - DF - Brazil
E-mail: reben@abennacional.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] CORRESPONDING AUTHOR: Leila Garcia de Oliveira Pegoraro E-mail: leilapegoraro2011@gmail.com