Accessibility / Report Error

Development of weCope, a mobile app for illness self-management in schizophrenia

Abstract

Objective

To describe the development of weCope, a Portuguese mobile app for people with schizophrenia.

Methods

The development of weCope followed 4 stages: I-102 people with psychotic disorders completed an online questionnaire; II-a multidisciplinary focus group was conducted among five mental health professionals; III-we developed the app; IV-9 participants used weCope during 8 weeks to assess its efficiency.

Results

weCope targets coping with voices, problem solving, goals setting and stress management, and results indicated that: weCope improved symptoms, sense of recovery and personal and social functioning; 59% of the participants were willing to download an application for illness self-management; professionals revealed some concerns about mHealth but enhanced the high importance to develop these apps; usability testing revealed that 89% considered weCope useful for illness self-management. However, the more years with mental illness, the less importance is given to a mobile application for this purpose.

Discussion

weCope was developed through a comprehensive development process and may contribute to a subjective perception of the patient’s better well-being and health condition.

Schizophrenia; illness self-management; mobile applications; technology

Introduction

Schizophrenia is the most common severe mental illness, with a reported median incidence of 15.2 per 100,000 persons and a pooled lifetime prevalence of 0.40%11. Simeone JC, Ward AJ, Rotella P, Collins J, Windisch R. An evaluation of variation in published estimates of schizophrenia prevalence from 1990-2013: a systematic literature review. BMC Psychiatry. 2015;15:193.. It is a chronic neuropsychiatric disorder among the world’s top ten causes of long-term disability22. Tandon R, Nasrallah HA, Keshavan MS. Schizophrenia, “just the facts” 4. Clinical features and conceptualization. Schizophr Res. 2009;110(1-3):1-23., characterized by specific symptoms such as delusions, hallucinations, or disorganized speech that at least one must be present for not less than one month according to DSM-522. Tandon R, Nasrallah HA, Keshavan MS. Schizophrenia, “just the facts” 4. Clinical features and conceptualization. Schizophr Res. 2009;110(1-3):1-23.,33. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Washington, DC: APA; 2013.. This disorder has a negative functional impact in many life domains of individuals with schizophrenia, including self-care impairment, illness management, social and interpersonal functioning, among others44. Lepage M, Bodnar M, Bowie CR. Neurocognition: clinical and functional outcomes in schizophrenia. Can J Psychiatry. 2014;59(1):5-12..

Currently, technology has great potential to promote the quality and effectiveness of mental health services55. Aboujaoude E, Salame W. Technology at the Service of Pediatric Mental Health: Review and Assessment. J Pediatr. 2016;171:20-4.

6. Batra S, Baker RA, Wang T, Forma F, DiBiasi F, Peters-Strickland T. Digital health technology for use in patients with serious mental illness: a systematic review of the literature. Med Devices (Auckl). 2017;10:237-51.

7. Institute of Medicine. Improving the quality of health care for mental and substance-use conditions. Washington, DC: National Academies Press. 2006.
-88. New Freedom Commission on Mental Health. Achieving the promise: Transforming mental health care in America. Final report. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2003. DHHS Pub. No. SMA-03-3832.. However, there is a lack of scientific evidence and, consequently, an absence of guidelines about the incorporation of these new trends in services that treat individuals with mental health disorders99. Ben-Zeev D, Frounfelker R, Morris SB, Corrigan PW. Predictors of self-stigma in schizophrenia: New insights using mobile technologies. J Dual Diagn. 2012;8(4):305-14.. Nevertheless, interventions based on new technologies are emerging and attracting the attention of researchers and enthusiasts in this area1010. Ben-Zeev D. Technology-based interventions for psychiatric illnesses: improving care, one patient at a time. Epidemiol Psychiatr Sci. 2014;23(4):317-21.. Mobile interventions are influencing psychologic approaches, they are even more sophisticated, offering therapeutic value for people who do not have access to it, and increasing their effectiveness for people who have face-to-face treatment1111. Lui J, Marcus D, Barry C. Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof Psychol Res Pr. 2017;48(3):199-210.,1212. Morris ME, Aguilera A. Mobile, social, and wearable computing and the evolution of psychological practice. Prof Psychol Res Pr. 2012;43(6):622.. Due their increased use, efforts towards the assessment of these devices and platforms in individuals with psychotic disorders are considerably growing1313. Ben-Zeev D, Scherer EA, Gottlieb JD, Rotondi AJ, Brunette MF, Achtyes ED, et al. mHealth for Schizophrenia: Patient Engagement With a Mobile Phone Intervention Following Hospital Discharge. JMIR Ment Health. 2016;3(3):e34.,1414. Torous J, Roux S. Patient-Driven Innovation for Mobile Mental Health Technology: Case Report of Symptom Tracking in Schizophrenia. JMIR Ment Health. 2017;4(3):e27..

The new technological approaches led to Mobile Health (mHealth) creation concept1515. Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health. 2014;14:188.. Although there is no standard definition for the term, mHealth refers to the use of mobile communication technologies in order to promote health through evidence-supported interventions1616. Kay M, Santos J, Takane M. mHealth: New horizons for health through mobile technologies. Geneve: World Health Organization; 2011. p. 66-71.,1717. World Health Organization. Global Observatory for eHealth, Survey 2009 figures, mHealth. 2009. Available from: http://www.who.int/goe/survey/2009/figures/en/index2.html.
http://www.who.int/goe/survey/2009/figur...
. Regarding mHealth applications, they include the use of mobile devices in collecting community and clinical health data, delivery of healthcare information to practitioners, researchers, and patients, real-time monitoring of patient vital signs, and direct care provision1818. Naslund J, Marsch L, McHugo G, Bartels S. Emerging mHealth and eHealth interventions for serious mental illness: a review of the literature. J Ment Health. 2015;24(5):321-32..

The transition of these tools to the mental health field was delayed and its potential to improve the lives of people with mental health problems may be jeopardize by not considering some of its limitations1919. Wykes T, Brown M. Over promised, over-sold and underperforming? – e-health in mental health. J Ment Health. 2016;25(1):1-4..

This study aims to describe the 4 stages that led to the development of weCope, a mobile application for illness self-management in schizophrenia. All stages of the project were conducted with individuals receiving psychosocial rehabilitation services and mental health professionals with experience in this area. Ethical approval was sought and granted during all stages, that we present separately in order to better describe the process of app’s development and its preliminary results.

Methods

Some authors have already proposed methodologies for developing and evaluating mobile health apps2020. Torous J, Firth J, Mueller N, Onnela JP, Baker JT. Methodology and reporting of mobile heath and smartphone application studies for schizophrenia. Harv Rev Psychiatry. 2017;25(3):146-54.,2121. Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl Behav Med. 2014;4(4):363-71., this paper follows the methodology used by Ben-Zeev et al.2222. Ben-Zeev D, Kaiser SM, Brenner CJ, Begale M, Duffecy J, Mohr DC. Development and Usability Testing of FOCUS: A Smartphone System for Self-Management of Schizophrenia. Psychiatr Rehabil J. 2013;36(4):289-96., which is a staged approach that could led to the development of mobile apps for self-management in schizophrenia.

Stage I: Users’ perspectives about mobile applications

Based on the state of the art on understanding the capacities, needs, and wants of end mHealth users, an online questionnaire was developed and applied to a sample of 102 persons with psychotic disorders (63 men, Mage = 42 years, age range: 22-66 years), mental health services users’. The questionnaire had 32 questions and was structured in three groups: sociodemographic data, technologies’ use patterns of the participants, and participant’s opinion about the development of a mobile application for support their mental illness’ management. Participants were also asked to rate some requirements’ importance related with the application development and its features, using a five-point Likert scale (ranging from 1 for not important at all to 5 for most important). An email was send to several rehabilitation services in Portugal, especially FNERDM (national organization that gathers 21 mental health institutions) and Centro Hospitalar São João, introducing the study with a link to the questionnaire posted online using Google Docs. We asked these institutions, after they had ethical authorisation, to send to their patients the questionnaire and to invite them to participate. Thus, the 102 participants do not have direct contact with researchers and fulfilled anonymously the questionnaire.

Stage II: Clinicians’ perspectives about mobile applications

A multidisciplinary Focus Group was conducted among five clinicians with different experience on schizophrenia treatment representing a range of service models (e.g., psychiatric hospitals, community-based treatment). Participants were recruited by an intentional sample among rehabilitation psychosocial practitioners. Contacts were conducted by email address to five institutions where they were performing labour activities, inviting them to collaborate, being one from each institution. A semi-structured script was developed based on literature and the concordance between coders was registered and the liability calculated by Cohen’s Kappa, obtaining a score of 0.951. Data were categorized and interpreted based on the literature.

Stage III: Mobile application prototype

A multidisciplinary team composed by engineers and mental health professionals was assembled to design the weCope prototype, considering users and professionals’ perspectives. Additionally, services users were consulted throughout the development process. The weCope app includes four modules (Figure 1): symptom monitoring, problem-solving, anxiety management and goal setting, allowing also the contact with therapists in crisis situations.

Figure 1
weCOPE main menu.

The symptom monitoring module share one of the goals of Cognitive Behavioural Therapy (the conceptual framework for the intervention) including modifying patients’ distorted beliefs about delusions and hallucinations. Thus, the main objective is to monitor the frequency, intensity and duration of psychotic symptoms, their triggering events and the conditions that maintain them (Figure 2). The problem solving module aims to identify and modify the maladaptive strategies to develop more helpful alternatives, and the user must identify the problem and describe it, presenting two possible solutions and selecting his preferred one (Figure 3). The anxiety management module purposes the reduction of factors such stress for a better control of the pathology and its symptoms – the user can explore different types of relaxation, such Jacobson Technique. In goal setting module, the user has to define the goal, deadlines, and the support that he will require, so he can have an attitude of empowerment in their recovery.

Figure 2
Symptom monitoring module example.

Figure 3
Problem solving module example.

Stage IV: weCope efficacy and usability

To test the app, 9 participants with a mean age of 38 years (±9.701), being 78% men, and with mean clinical follow-up time of 10 years were invited. These participants owned a smartphone and were willing to participate in this study. They were from ANARP, a Portuguese community rehabilitation centre, being a non-probabilistic sampling method used for a sample selected for convenience.

They use the app during 8 weeks and data collection was carried out in two moments (before and after the application use), using six evaluation instruments: Recovery Assessment Scale2323. Corrigan PW, Salzer M, Ralph RO, Sangster Y, Keck L. Examining the factor structure of recovery assessment scale. Schizophr Bull. 2004;30(4):1035-41., Empowerment Scale2424. Rogers ES, Chamberlin J, Ellison ML, Crean T. A Consumer-Constructed scale to measure empowerment among users of mental health services. Psychiatr Serv. 1997;48(8):1042-7., General Self-Efficacy Scale2525. Pais-Ribeiro J. Adaptação de uma escala de avaliação da auto-eficácia geral In: Almeida L, Ribeiro I (eds.). Avaliação Psicológica: formas e contextos. Braga: APPORT; 1995. p. 163-76., Social Support Satisfaction Scale2626. Pais Ribeiro J. Escala de Satisfação com o Suporte Social (ESSS). Análise Psicol. 1999;3(17):547-58., Personal and Social Performance Scale2727. Morosini PL, Magliano L, Brambilla L, Ugolini S, Pioli R. Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale: (SOFAS) to assess routine social functioning. Acta Psychiatr Scand. 2000;101(4):323-9. and Positive and Negative Syndrome Scale2828. Kay SR, Fiszbein A, Opler LA. The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia. Schizophr Bull. 1987;13(2):261-76.. A usability brief questionnaire was also applied, asking participants their ability to use de app, satisfaction, difficulties, etc.

The use of the app was free, based on the need of each user, and the usage data was guaranteed through an algorithm created that registers them on the server and allows to be consulted. It should be noted that if the medication of any of the participants changed, the case manager would be responsible for communicating that to the researcher.

Results

Regarding survey results, 78% of the participants have mobile phones, of which 34% were smartphones, and 59% of the participants were willing to download an app for illness self-management. The importance given to self-managing their mental health condition was high: 51% said that it was essential and 26% very important. However, the more years with mental illness, the less importance is given to a mobile application for this purpose [r(100) = -.244, p < 0.05] and the older the individuals, more difficulties in using these kind of technologies [r(100) = .209, p < 0.05]. More results are presented in Simões de Almeida et al.2929. Simões de Almeida R, Sousa T, Marques A, Queirós C. Patients’ Perspectives About the Design of a Mobile Application for Psychotic Disorders. Psychol Community Health. 2018;7(1):16-28.. These results were used in stage III to develop weCope app.

Professionals recognized advantages in the use of mobile applications for therapeutic purposes, but also acknowledged some issues. Five categories were identified: participants’ overview on the use of mobile applications in mental health intervention, participants’ experience in technology-based interventions, adherence to the mobile application in schizophrenia, professionals’ concerns and doubts related with an app for schizophrenia and potential features to consider in app development. As added value, the possibility of using messages or another form of notification to remind patients of appointments and medication, to define therapeutic tasks for users with greater resistance to adhere to activities in hospital contexts complementary to working hours. Despite motivation and interest, these health professionals reinforced the need for professional supervision, privacy and the need to take into account a possible cognitive deterioration. These results were used in next stage to develop weCope app.

Regarding intervention results (Table 1), Recovery Assessment Scale total score presents significant results and the mean also increased. Empowerment Scale also presents significant results and the mean also increased. Social Support Satisfaction Scale presented statistically significant values only in subscales “intimacy” and “satisfaction with family”, and regarding the average, it increased in all subscales, excepting the subscale “social activities”. General Self-Efficacy Scale total score showed significant results and the mean also increased. Regarding Positive and Negative Syndromes Scale we found statistically significant values only in the general psychopathology subscale, and the average decreased in all subscales. The total score of the scale do not present significant results, but the mean also decreased. Finally, in Personal and Social Performance we found significant results on the total score of the scale and the mean also increased. More than the averages, it is also important to mention individual scores; there were improvements for each subject in the means of all scales with an increase in all of them except for PANSS scale.

Table 1
Instrument scores

Concerning the usability, the majority of participants (45%) used the app two or three times a week and recognize the “Anxiety Management” and “Goal Setting” modules and the possibility to exchange messages with the therapist the modules being most used by them. The participants’ opinion about the browsing experience, visual aspect and learning regarding weCope is positive as can be seen from Table 2. Participants showed that were satisfied with the weCope system, considering the application useful for the management of the disease (89%), and 78% said they would continue to use the app.

Table 2
Participant’s opinions about weCope

When questioned in an open-ended question about what they would like to change in the application, participants said it was important to enable peer interaction chats (n = 4), to monitor mood states and other symptoms (n = 1), to use reminders for medication and appointments (n = 2) and to provide videos and other type of psychoeducational materials (n = 2).

Despite some concerns from professionals, our data indicate an interest and several reasons to engagement in these kind of technologies for mental health. Considering user involvement, initial strategies were developed to enhance app adherence however this tool is suitable for a specific clinical profile of the users: younger, recent onset of illness, better psychopathology and receiving ambulatory care.

Discussion

Technologies applied to health have been evolving rapidly in recent years, which has stimulated the change of patients’ roles to one in which they are involved in their treatment, and who have the capacity and interest to make decisions about themselves. Smartphones are widely accepted and integrated in people’s daily routines making them a tool readily accessible every time. When designing mobile applications for people with schizophrenia they should be easy to navigate and developed with the needs, characteristics and preferences of final users, involving them in the whole process. In this study we presented the process of mobile app development in which service users and mental health professionals took part in every stage.

weCope targets coping with voices, problem solving, goals setting and stress management. Exchanging messages with the therapist was one of the most used modules. This suggests that the concerns of the professionals with privacy may not be a barrier, since has the potential to reinforce the alliance between therapist and the patient. In fact, data security is the leading barrier to adopting innovative technologies. Also, the importance of comprehensive understanding of the user needs in real time, with shared-decision, is one more step toward personalized medicine.

Despite technological tools are not the best option for every person, patients at a younger age and with less time of illness will probably be much more benefited by a mobile application. Finally, it is important to not forget that American Psychiatric Association3030. American Psychiatric Association. Why Rate Mental Health Apps? 2018. Available from: https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/why-rate-mental-health-apps.
https://www.psychiatry.org/psychiatrists...
recommended some caution when using apps and suggested rating mental health apps, to help service users and professionals to make the best informed decision about whether or not to use an app. World Health Organization proposed a mHealth Evaluation, Reporting and Assessment guidelines for evaluating mobile health applications2020. Torous J, Firth J, Mueller N, Onnela JP, Baker JT. Methodology and reporting of mobile heath and smartphone application studies for schizophrenia. Harv Rev Psychiatry. 2017;25(3):146-54.. Developers should integrate the following four characteristics of high-efficacy mental health apps: high patient engagement, simple user interface and experience, transdiagnostic capabilities and have self-monitoring features3131. Chandrashekar P. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. Mhealth. 2018;4:6.. WeCope was designed considering that and was well received by the participants and demonstrated significant results at the level of perception of the participants’ better well-being and health condition. Moreover, the mobile application was evaluated as performing well in terms of responding to user needs and user experience, however conclusions have to be considered preliminary considering the small number of patients who used the tool. Being an app developed in Portuguese language, it can be used in different countries such are Brazil and some African countries (e.g. Mozambique, Angola, Cabo Verde Island, etc.). Although preliminary results show good efficacy of the weCope app, more robust future studies, with larger samples, are needed to examine whether the improvements persist over time. In addition, since there was no control group, it was not possible to determine whether the clinical improvements were related only to the use of weCope.

References

  • 1
    Simeone JC, Ward AJ, Rotella P, Collins J, Windisch R. An evaluation of variation in published estimates of schizophrenia prevalence from 1990-2013: a systematic literature review. BMC Psychiatry. 2015;15:193.
  • 2
    Tandon R, Nasrallah HA, Keshavan MS. Schizophrenia, “just the facts” 4. Clinical features and conceptualization. Schizophr Res. 2009;110(1-3):1-23.
  • 3
    American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Washington, DC: APA; 2013.
  • 4
    Lepage M, Bodnar M, Bowie CR. Neurocognition: clinical and functional outcomes in schizophrenia. Can J Psychiatry. 2014;59(1):5-12.
  • 5
    Aboujaoude E, Salame W. Technology at the Service of Pediatric Mental Health: Review and Assessment. J Pediatr. 2016;171:20-4.
  • 6
    Batra S, Baker RA, Wang T, Forma F, DiBiasi F, Peters-Strickland T. Digital health technology for use in patients with serious mental illness: a systematic review of the literature. Med Devices (Auckl). 2017;10:237-51.
  • 7
    Institute of Medicine. Improving the quality of health care for mental and substance-use conditions. Washington, DC: National Academies Press. 2006.
  • 8
    New Freedom Commission on Mental Health. Achieving the promise: Transforming mental health care in America. Final report. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2003. DHHS Pub. No. SMA-03-3832.
  • 9
    Ben-Zeev D, Frounfelker R, Morris SB, Corrigan PW. Predictors of self-stigma in schizophrenia: New insights using mobile technologies. J Dual Diagn. 2012;8(4):305-14.
  • 10
    Ben-Zeev D. Technology-based interventions for psychiatric illnesses: improving care, one patient at a time. Epidemiol Psychiatr Sci. 2014;23(4):317-21.
  • 11
    Lui J, Marcus D, Barry C. Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof Psychol Res Pr. 2017;48(3):199-210.
  • 12
    Morris ME, Aguilera A. Mobile, social, and wearable computing and the evolution of psychological practice. Prof Psychol Res Pr. 2012;43(6):622.
  • 13
    Ben-Zeev D, Scherer EA, Gottlieb JD, Rotondi AJ, Brunette MF, Achtyes ED, et al. mHealth for Schizophrenia: Patient Engagement With a Mobile Phone Intervention Following Hospital Discharge. JMIR Ment Health. 2016;3(3):e34.
  • 14
    Torous J, Roux S. Patient-Driven Innovation for Mobile Mental Health Technology: Case Report of Symptom Tracking in Schizophrenia. JMIR Ment Health. 2017;4(3):e27.
  • 15
    Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health. 2014;14:188.
  • 16
    Kay M, Santos J, Takane M. mHealth: New horizons for health through mobile technologies. Geneve: World Health Organization; 2011. p. 66-71.
  • 17
    World Health Organization. Global Observatory for eHealth, Survey 2009 figures, mHealth. 2009. Available from: http://www.who.int/goe/survey/2009/figures/en/index2.html
    » http://www.who.int/goe/survey/2009/figures/en/index2.html
  • 18
    Naslund J, Marsch L, McHugo G, Bartels S. Emerging mHealth and eHealth interventions for serious mental illness: a review of the literature. J Ment Health. 2015;24(5):321-32.
  • 19
    Wykes T, Brown M. Over promised, over-sold and underperforming? – e-health in mental health. J Ment Health. 2016;25(1):1-4.
  • 20
    Torous J, Firth J, Mueller N, Onnela JP, Baker JT. Methodology and reporting of mobile heath and smartphone application studies for schizophrenia. Harv Rev Psychiatry. 2017;25(3):146-54.
  • 21
    Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl Behav Med. 2014;4(4):363-71.
  • 22
    Ben-Zeev D, Kaiser SM, Brenner CJ, Begale M, Duffecy J, Mohr DC. Development and Usability Testing of FOCUS: A Smartphone System for Self-Management of Schizophrenia. Psychiatr Rehabil J. 2013;36(4):289-96.
  • 23
    Corrigan PW, Salzer M, Ralph RO, Sangster Y, Keck L. Examining the factor structure of recovery assessment scale. Schizophr Bull. 2004;30(4):1035-41.
  • 24
    Rogers ES, Chamberlin J, Ellison ML, Crean T. A Consumer-Constructed scale to measure empowerment among users of mental health services. Psychiatr Serv. 1997;48(8):1042-7.
  • 25
    Pais-Ribeiro J. Adaptação de uma escala de avaliação da auto-eficácia geral In: Almeida L, Ribeiro I (eds.). Avaliação Psicológica: formas e contextos. Braga: APPORT; 1995. p. 163-76.
  • 26
    Pais Ribeiro J. Escala de Satisfação com o Suporte Social (ESSS). Análise Psicol. 1999;3(17):547-58.
  • 27
    Morosini PL, Magliano L, Brambilla L, Ugolini S, Pioli R. Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale: (SOFAS) to assess routine social functioning. Acta Psychiatr Scand. 2000;101(4):323-9.
  • 28
    Kay SR, Fiszbein A, Opler LA. The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia. Schizophr Bull. 1987;13(2):261-76.
  • 29
    Simões de Almeida R, Sousa T, Marques A, Queirós C. Patients’ Perspectives About the Design of a Mobile Application for Psychotic Disorders. Psychol Community Health. 2018;7(1):16-28.
  • 30
    American Psychiatric Association. Why Rate Mental Health Apps? 2018. Available from: https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/why-rate-mental-health-apps
    » https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/why-rate-mental-health-apps
  • 31
    Chandrashekar P. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. Mhealth. 2018;4:6.

Publication Dates

  • Publication in this collection
    Jan-Feb 2019

History

  • Received
    26 Dec 2017
  • Accepted
    28 Nov 2018
Faculdade de Medicina da Universidade de São Paulo Rua Ovídio Pires de Campos, 785 , 05403-010 São Paulo SP Brasil, Tel./Fax: +55 11 2661-8011 - São Paulo - SP - Brazil
E-mail: archives@usp.br