Model of EA |
1. Which theoretical reference of EA was used?
|
|
|
Thurston e Ramaliu |
7 |
33.3 |
Leviton |
3 |
14.3 |
Wholey |
2 |
9.5 |
Grouped authors (Wholey, Rutman, Smith, Thurston and Potvin) |
1 |
4.8 |
Did not define |
8 |
38.1 |
2. Has followed model steps?
|
|
|
Yes |
11 |
52,4 |
Partly |
2 |
9,5 |
Does not apply |
8 |
38,1 |
3. Has followed proper steps for EA?
|
|
|
Yes |
19 |
90.5 |
No |
2 |
9.5 |
4. Has the EA been addressed in the introduction or literature review?
|
|
|
Yes |
16 |
76.2 |
No |
5 |
23.8 |
5. Has the program modeling been carried out and presented?
|
|
|
Yes |
17 |
81.0 |
Carried out, but not presented |
4 |
19.0 |
6. What type of model was performed?
|
|
|
Logical model by activities |
13 |
61.9 |
Logical model by objectives |
1 |
4.8 |
Logical model by activities and theoretical model |
2 |
9.5 |
Theoretical/Logical Model |
2 |
9.5 |
Does not inform |
3 |
14.3 |
Proposition of the EA |
7. What is the proposal of the EA?
|
|
|
Verify if the program is evaluable |
12 |
57.1 |
Guide the evaluation |
5 |
23.8 |
Understand the program |
4 |
19.0 |
Enable use of assessment |
2 |
9.5 |
Improve the knowledge of the stakeholders on the program |
2 |
9.5 |
Improve the program |
2 |
9.5 |
Plan a program |
1 |
4.8 |
Define objectives of the Program |
1 |
4.8 |
Update the program |
1 |
4.8 |
Verify if the program is useful |
1 |
4.8 |
Proposition of the EA |
8. Were the objectives and research question appropriate for an EA?
|
|
|
Yes |
13 |
61.9 |
No |
4 |
19.0 |
Partly |
4 |
19.0 |
9. Was there alignment between purpose/questions, procedures, results, and recommendations?
|
Yes |
13 |
61.9 |
No |
3 |
14.3 |
Partly |
5 |
23.8 |
Procedures of the EA |
10. What data collection procedures were used?
|
|
|
Documentary analysis |
19 |
90.5 |
Interview |
19 |
90.5 |
Meetings and discussion groups with stakeholders
|
9 |
42.9 |
Literature review |
5 |
23.8 |
Local visit/observation |
4 |
19.0 |
Questionnaire |
1 |
4.8 |
11. Did the author describe his/her relation with the research local? If there was a dual role (that is, a student and an employee or affiliate of the organization for which an EA is being conducted), did the author describe how it was addressed?
|
Yes, he/she described the relation and how he/she was treated. |
1 |
4.8 |
Yes, he/she described the relation, but not how he/she was treated. |
3 |
14.3 |
No |
17 |
81.0 |
12. Has the research been approved by the service/institution staff investigated?
|
|
|
Yes |
5 |
23,8 |
No |
8 |
38,0 |
Did not inform |
7 |
33.3 |
Does not apply |
1 |
4.8 |
13. Has EA involved stakeholders?
|
|
|
Yes |
6 |
28.6 |
No, they were only a source of evidence |
14 |
66.7 |
No, there was no participation |
1 |
4.8 |
14. Was the involvement sufficiently representative throughout the EA process?
|
Yes |
3 |
50.0 |
No |
3 |
50.0 |
15. Was the data analysis procedures sufficiently described and appropriate?
|
Yes |
7 |
33.3 |
No |
14 |
66.7 |
Results of the EA |
16. How have the results been presented?
|
|
|
Narrative |
21 |
100.0 |
Figure |
19 |
90.5 |
Table |
7 |
33.3 |
Board |
6 |
28.6 |
Results of the EA |
17. Was the focus of the results on the process or on the results of the EA?
|
|
|
Results |
15 |
71.4 |
Process |
6 |
28.6 |
18. Are there strategies to ensure the validity and reliability of the results?
|
|
|
Yes |
3 |
14.3 |
No |
6 |
28.6 |
Partly |
12 |
57.1 |
19. Have limitations been described?
|
|
|
Yes |
3 |
14.3 |
No |
18 |
85.7 |
20. Has it presented MAJ or matrix of indicators?
|
|
|
Yes |
6 |
28.6 |
No |
15 |
71.4 |
21. Has it brought an evaluative question?
|
|
|
Yes |
4 |
19.0 |
No |
17 |
81.0 |
Use of the EA |
22. Has the article presented strategies that could facilitate the use of EE?
|
|
|
Yes |
18 |
81,0 |
No |
3 |
19,0 |
23. Has the use of EA contributed to professional knowledge? (for example, solving a real problem).
|
Yes |
16 |
94.1 |
No |
1 |
5.9 |
24. Has the use been consistent with the purpose of EA?
|
|
|
Yes |
17 |
81.0 |
No |
1 |
4.8 |
Does not apply |
3 |
14.3 |
25. Has the work contributed to academic knowledge (academic use) about EA and/or assessment?
|
Yes |
4 |
19.0 |
No |
14 |
66.7 |
Does not apply |
3 |
14.3 |
26. Has EA presented itself as an important strategy for the assessment process of the program in question?
|
Yes |
21 |
100.0 |
No |
- |
- |