Effectiveness of mobile applications as intervention tools in suicidal ideation: a systematic review and meta-analysis

Eficácia dos aplicativos móveis como ferramentas de intervenção na ideação suicida: uma revisão sistemática e meta-análise

Abstract

Objective

Evaluate the effectiveness of mobile applications developed for intervention in suicide risk in adults.

Method

Searches for articles were carried out in four databases (PubMed, Web of Science, Scopus and PsychINFO). For the meta-analyses, we used fixed effect modelling to assess the primary outcome. The PROSPERO register is CRD42020163876.

Results

After applying inclusion and exclusion criteria, six studies remained in the systematic review, of which four were eligible to the meta-analysis. There was no difference, or a little tendency in favor of control condition (usual treatment) comparing with applications.

Conclusion

Applications can be as effective as standard treatments in reducing suicide ideation. The results should be interpreted with caution once all studies presented at least one bias in their study design.

Keywords
Mhealth; Mobile health; Smartphone; Telemedicine; Telepsychology

Resumo

Objetivo

Avaliar a eficácia dos aplicativos desenvolvidos para intervenção no risco de suicídio em adultos.

Método

As buscas por artigos foram realizadas em quatro bancos de dados (PubMed, Web of Science, Scopus e PsychINFO). Para a meta-análise, utilizamos modelagem de efeito fixo para avaliar o desfecho primário. O registro no PROSPERO é CRD42020163876.

Resultados

Após aplicação dos critérios de inclusão e exclusão, seis estudos permaneceram na revisão sistemática, dos quais quatro foram elegíveis para a meta-análise. Não houve diferença, ou uma pequena tendência a favor da condição de controle (tratamento usual) em comparação com aplicativos.

Conclusão

Os aplicativos parecem ser tão eficazes quanto os tratamentos padrões na redução de ideação suicida. Os resultados devem ser interpretados com cautela, pois todos os estudos apresentaram viés em seu desenho.

Palavras-chave
Msaúde; Saúde móvel; Smartphone; Telemedicina; Telepsicologia

Suicide is worldwide public health issue, causing of nearly 800 thousand deaths per year. The risk of suicide is the second most common cause of death between the ages of 15 and 29 years old. Most of these young people (90%) that commit suicide are from low- and middle-income countries (World Health Organization, 2019). In low-income countries, of all individuals who present risk of suicide only 17% receive some form of treatment. This number rises to 28% in middle-income and to 56% in high-income countries (Bruffaerts et al., 2011).

In addition, there is evidence that in low-income and middle-income countries the primary barrier to search for treatment is the low perceived need for help even when the individual presents a high risk of suicide (Czyz et al., 2013). Other important barriers to treatment are low financial resources, lack of available treatment in the living region, transportation problems (Priester et al., 2016) and mental illness stigma (Fukuda et al., 2016; Knaak et al., 2017). A survey indicated that in some middle-income countries 78.8% of individuals with depression do not receive treatment and 14.1% received only pharmacotherapy treatment. Furthermore, the lowest the income the higher is the difficulties to access mental healthcare (Lopes et al., 2016).

One possible alternative to help people with middle or low-income is using Mobile Applications (MAs) that are designed to access and intervene in mental illness (De la Torre et al., 2017). Besides the help to low-income population, the use of MAs may also benefit individuals that report not having enough time to attend appointments (Bruffaerts et al., 2011; Czyz et al., 2013) and mental illness stigma (Fukuda et al., 2016; Knaak et al., 2017) as barriers to treatment.

Furthermore, there is a growing number of studies indicating that young adults prefer self-management interventions as an alternative to traditional mental health treatment (Czyz et al., 2013; Lean et al., 2019). Evidence also shows that MAs presents great effectiveness in reducing symptoms and improving functionality and quality of life after the use of self-management interventions in adults with severe mental illness (Lean et al., 2019).

Nevertheless, there are several MAs that do not present evidence of effectiveness and some of those do not even present a scientific study supporting its development. A review integrating data from the literature and virtual stores (i.e. iTunes and Google Play) found 20 MAs specifically designed for suicide prevention. However, only six MAs contained evidence-based information (De la Torre et al., 2017). This demonstrates that it has been occurring a dissemination of MAs without the proper evaluation of its usability and effectiveness (Hayes et al., 2016). In one hand, there is the possibility that these MAs benefit their users by monitoring the individuals’ health (De la Torre et al., 2017). On the other hand, it is possible that the MAs are ineffective or even harmful to their users’ health, especially since their goal is an intervention (Hayes et al., 2016).

In this perspective, it is observed that the new mobile phone technologies for suicide prevention and intervention remain understudied (De la Torre et al., 2017; Hayes et al., 2016). Therefore, the present study aims to evaluate the effectiveness of MAs developed for intervention in suicide risk in adults.

Method

This study used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement as the guideline (Page et al., 2021). The study protocol is registered in the international prospective register of systematic reviews (PROSPERO) and is accessible under the ID number CRD42020163876.

Search Strategy and Selection Criteria

The searches were conducted on December 2020 by two independent raters (NOC and EZS) in four electronic databases: PubMed, Web of Science, Scopus and PsychINFO. The search terms used in this review are selected based on the Medical Subject Headings (PubMed) and Thesaurus (PsycInfo) dictionaries. The final search string was the following: (“Suicidal Idea*” or “suicide attempt*” or “suicidal thought*” or “suicidal behavior*” or suicide or suicidal) and (mobile or app or apps or smartphone* or “smart phone*” or “machine learning” or “deep learning” or “artificial intelligence” or algorithm* or ehealth or e-health or m-health or mhealth or “health technology”).

Although the same string has been used in all previously mentioned databases, search strategies were customized for each database. In the Web of Science, PubMed and PsycInfo the full string was used in a single line without any problems. In Scopus, the string was divided and used in two different lines: 1° Line: (“Suicidal Idea*” or “suicide attempt*” or “suicidal thought*” or “suicidal behavior*” or suicide or suicidal) AND 2° Line: (mobile or app or apps or smartphone* or “smart phone*” or “machine learning” or “deep learning” or “artificial intelligence” or algorithm* or ehealth or e-health or m-health or mhealth or “health technology”)

Otherwise, the database was unable to find any results. Regarding available databases filters, in Web of Science and Scopus we use the filter “article”, in PsycInfo “journal” and in PubMed “Journal Article”. After the database searches were complete, we imported all results into Rayyan, a free web-based tool (Khabsa et al., 2016; Ouzzani et al., 2016), which compiles the searches results into a single online platform. Rayyan has a blind mode that allows the evaluation of articles by independent raters and subsequently comparing their results. In addition, the platform optimizes the recognition of duplicated papers, allows the raters to assign labels to an article justifying why they included/excluded certain article and assists in the risk of bias analysis (Ouzzani et al., 2016). After removing duplicates, the two raters began screening the title and abstract of the remaining articles. After that, the remaining articles were read in full. In both phases the following inclusion and exclusion criteria were used: Inclusion (Only studies that satisfied all these criteria were included): 1) Peer-reviewed; 2) Full papers availability; 3) Empirical research that provides evidence regarding the effectiveness of an m-health application developed for intervention in suicide risk (e.g. suicidal ideation or suicidal attempt); 4) Having a control condition (e.g. Treatment As Usual [TAU] or Waitlist); 5) Single group pre-post trial were included only in cases of an standalone application intervention (i.e. without any complementary intervention); 6) Sample of adults (18-59 years old) with suicide risk. Exclusion (Studies that met at least one of these criteria were discarded): 1) Studies regarding the protocol/trial registrations; 2) Interventions delivered exclusively via paper, face-to-face, CD-ROM, or other non-application method; 3) Interventions delivered exclusively via computer, tablet, or others handheld devices without an interface compatible with smartphones; 4) Samples composed entirely of non-suicidal individuals who self-injure; 5) Applications or websites developed exclusively as a screening tool for suicide risk without an intervention; 6) Samples composed exclusively by teenagers (< 17) or elderly (60 >).

In order to cover the largest number of articles related to the topic of the present review, the language and year of the publications were not restricted. Any disagreements between the two raters during the study selection were solved by a third senior researcher (WLM).

The data related to participants, interventions, comparisons, outcomes, and study design (PICOS) is shown in Tables 1 and 2. Three researchers (NOC, KRT and EZS) performed the data extraction independently. In the end of the extraction process, WLM solved disagreements between the three raters.

Statistical Analysis

The effect sizes for each study were calculated in R software 3.6.1 using the package “meta”, which allows the estimation of the meta-analysis Fixed and Random effects and the assessment of heterogeneity (Schwarzer, 2019). The impact of between-study heterogeneity was quantified using I2 statistic. An I2 statistic of =75% as indicating substantial levels of between-study heterogeneity (Higgins et al., 2003). The original purpose of this review was to evaluate the effectiveness of MAs developed for intervention in suicide risk (e.g. suicidal ideation or suicidal attempt). However, the included studies have only suicidal ideation as a common outcome, therefore, all quantitative analyzes are related to this outcome. The effects of MAs were presented as standardized effect sizes (Cohen’s d). An effect size of ≥ 0.80 is considered a large clinical effect, an effect size of ≥ 0.50 moderate, and ≥ 0.20 small (Cohen, 1988). Two senior researchers performed all statistical analysis (WLM and JRNV).

Risk of Bias

Three researchers (NOC, EZS and KRT) independently assessed the risk of bias and any disagreement was resolved by the senior author (WLM). The Cochrane Collaboration’s tool for assessing risk of bias was adopted to evaluate the risk of bias in the included studies (Higgins & Green, 2011). This tool assesses bias in six domains: 1) selection bias (i.e., random sequence generation and allocation concealment); 2) performance bias (i.e., blinding of participants and personnel); 3) detection bias (i.e., blinding of outcome assessment); 4) attrition bias (i.e., incomplete outcome data); 5) reporting bias (i.e., selective reporting); 6) other bias (i.e. bias due to problems not covered in the previous domains).

Results

We found 3,849 articles in the initial search, of which six articles remained after the selection process (Figure 1). We also performed a manual search through the references of the six included articles in the present review and through the references of the articles included in another systematic review and meta-analysis with similar aim (Witt et al., 2017). However, no additional articles were included in our study. Tables 1 and 2 show the data extracted from the six included papers (i.e. PICOS).

Figure 1
Flowchart

Table 1
Characteristics of the included studies

nº	Author / Country	Study Design	Treatment Length (n° of total sessions)	Sessions Frequency (minutes per use)	MAs	Control Condition	Outcome Measures2	Suicide Risk Reduction1
1	Denneson et al. (2019) / USA	RCT (Secondary Data)	12 weeks (NR)	Whenever needed (NR)	VHB	ETAU	BSS	Yes
2	O’Toole et al. (2019) / Denmark	RCT with 4-month follow-up	NR (9.4 ± 4.1)	Daily (NR)	LifeApp’tite	TAU	SSFIIR	Yes
3	Chen et al. (2018) / USA	RCT (Secondary Data)	12 weeks (86 median)	Whenever needed (109 median)	VHB	ETAU	BSS	Yes
4	Tighe et al. (2017) / Australia	RCT	6 weeks (NR)	NR	Ibobbly	Waitlist	DSI-SS	Yes
5	Bush et al. (2017) / USA	RCT	12 weeks (NR)	Whenever needed (NR)	VHB	ETAU	BSS	Yes
6	Franklin et al. (2016) / International	3 RCT with 1-month follow-up	4 weeks (NR)	Whenever needed (1-2)	TEC	Control TEC	SITBI	Yes

• Note:
• ¹
  Based in outcome measures differences between pre/post intervention;
• ²
  For suicide risk.
• BSS: Beck Scale for Suicidal Ideation; DSI-SS: Depressive Symptom Inventory Suicidality Subscale; ETAU: Enhanced Treatment as Usual; NR: Not reported; RCT: Randomized Controlled Trial; SITBI: Self-Injurious Thoughts and Behaviors Interview; SSFIIR: Suicide Status Form II–R; TAU: Treatment as Usual; TEC:Therapeutic Evaluative Conditioning; VHB: Virtual Hope Box.

Table 2
Characteristics of study participants

nº	Sample	Mean Age (SD)	Sex	Suicide Risk	Medication use (name)	Previous psychotherapy experience
1	117	47.5 (14.0)	80M; 37F	Suicidal ideation	Yes (NR)	Yes
2	129	28.7 (9.5)	45M; 84F	Suicidal ideation	NR	Yes
3	58	46.50 (13.7)	36M; 22F	Suicidal ideation	Yes (NR)	Yes
4	61	26.25 (8.13)	22M; 39F	Suicidal ideation	NR	Yes
5	118	47.5 (14.0)	81M; 37F	Suicidal ideation	Yes (NR)	Yes
6	114 [1]; 131 [2]; 163 [3]	23.02 (5.47) [1]; 22.91(4.99) [2]; 24.50 (6.61) [3]	22M; 92F [1]; 34M; 97F [2]; 65M; 98F [3]	Suicidal ideation (SITBs)	Yes (NR)	Yes

• Note: F: Female; M: Male; NR: Not reported; SITBs: Self-injurious thoughts and behavior; TAU: Treatment as Usual; VHB: Virtual Hope Box.

It was observed that most studies were conducted in the USA (n = 3), one in Denmark and another in Australia. One study recruited participants through online forums including participants from multiple countries. Regarding the methodology of the studies, all were Randomized Clinical Trial (RCT) and only two performed a follow-up evaluation. The main limitation of the included studies was related to the total length of the treatment and the session’s frequency. Even though, most studies reported the total length of the treatment (n = 5), the majority did not report the total number of sessions (n = 4) neither the total duration of each session (n = 4).

One of the studies did not clearly disclosure the total length of treatment, but considering that 9.4 (±4.1) was the mean number of sessions’ performed by the TAU+APP group, and that the authors informed that participants received daily notifications from the APP, one could estimate that the total length of the treatment was in fact two weeks (study 2). Therefore, considering all the studies included, we can estimate that the mean of the length of the interventions was eight weeks (±4.56). Due to missing data, it was not possible to estimate the mean for total number of session nor the total duration of each session.

Regarding the used MA, each study used a different one. In addition to the fact that only six studies were found, it is possible that there is no gold standard MA for suicide risk intervention. Although all studies have measured suicidal ideation, none of the outcomes measured were the same, there was also variation in the control conditions. All studies point to a suicide risk reduction after interventions, but in the studies 3 and 5 only the capability of cope with negative thoughts increased, while reason for living decreased (study 3) and no significant difference was found in the scores of suicidal ideation between groups, possibly due to a floor effect (study 5).

About study participants, the sample size ranged between 58 and 163. The sum of all samples equals 716 unique participants (62.5% female), with an age average of 28.81 years old. It is important to note that the samples from studies 1 and 3 were not accounted because they are the same participants that were included in study 5. All studies had suicidal ideation as an outcome assessed to analyze the MAs effectiveness. Only the study 6 analyzed both suicidal ideation and behavior. Although most studies (n = 4) report that the sample took medication, none of them describes the medication type or name. On the other hand, all studies investigated whether the participants had experience with psychotherapy.

Risk of Bias in Included Studies

A summary of possible bias in the selected studies is shown on Table 3. All studies showed a high risk of bias in at least one of the analyzed topics. Two studies were rated as high risk of performance bias due to non-blinding of participants and/or personnel. None of the studies were rated as high risk of selection, detection, attrition and reporting bias. However, four were rated as unclear risk of reporting bias due to non-statement of trial registration or protocol publication. Assessment of other sources of biases involved the examination of missing data bias and treatment bias. Missing data bias was rated as high risk in all studies due to 1) lack of total treatment length information; 2) lack of total number of sessions; 3) lack of sessions frequency and/or 4) mean duration of MA access. Lastly, although all studies have investigated and controlled for possible effects related to previous psychotherapy experience, treatment bias was rated as high risk in two studies due to lack of medication use report.

Table 3
Summary of possible bias in the selected studies

Study	Selection bias	Performance bias	Detection bias	Attrition bias	Reporting bias	Other bias
						Missing data bias	Treatment bias
Denneson et al. (2019) Chen et al. (2018) Bush et al. (2017)	-	+	-	-	?	+	-
O’Toole et al. (2019)	-	-	-	-	-	+	+
Tighe et al. (2017)	-	+	-	-	-	+	+
Franklin et al. (2016)	-	-	-	-	?	+	-

• Note: +: high risk of bias; –: low risk of bias; ?: unclear risk of bias.

Meta-Analysis Data

To calculate the effect size of the interventions, the post-test measures of each group (i.e. control and experimental) were evaluated. Follow-up measures were not evaluated due to incompatibility of time periods and Regression Toward The Mean Effect (Barnett et al., 2005; Davis, 1976). In addition, all studies included in this meta-analysis used a different MA, control condition and a different psychometric instruments to measure suicidal ideation. Therefore, neither tests for subgroup differences nor sensitivity analyses could be undertaken.

Although the original purpose of this review was to evaluate the effectiveness of MAs developed for intervention in suicide risk (e.g. suicidal ideation or suicidal attempt), the six included studies have only suicidal ideation as a common outcome, therefore, quantitative analyzes are related to this outcome. The study 3 did not provide enough data (i.e. groups mean and standard deviation) for inclusion in meta-analysis. Further, it is important to notice that the studies 1 and 5 reported data from the same intervention/RCT and therefore only the original study (study 5) was included in the meta-analysis. Thus, the quantitative analyzes were performed with four of the six studies included in the qualitative synthesis (Figure 2).

Figure 2
Fixed effects, Standard Mean Difference and accompanying 95% confidence interval for Mobile Applications interventions of suicidal ideation scores

The four studies reported data on suicidal ideation scores using different psychometric instruments and different intervention (i.e. MA). First, both fixed and random effect metanalysis were estimated using mean, standard deviation and sample size for each study and standard mean difference (SMD) as a measure of effect size. Since the results are nearly identical for both models, and heterogeneity measure were very low, we reported only the fixed effect model. The main findings indicated no difference or a little tendency in favor of control condition. The low heterogeneity indicated a between-studies consistency. However, there is no statistically significant difference between groups as the confidence interval for SMD approaches zero in the upper limit.

Discussion

These results should be interpreted with caution, considering the peculiarities of each of the analyzed studies. In one hand, O’Toole et al. (2019) and Bush et al. (2017) used TAU as a control condition. The Virtual Hope Box (VHB) used by Bush et al. (2017) was as effective as TAU treatment for reducing suicidal ideation. The LifeApp’tite used by O’Toole et al. (2019) was a little less effective than TAU. Therefore, both studies show evidence of the effectiveness of the used MAs. On the other hand, Tighe et al. (2017) used waiting list as a control condition, therefore the MA (i.e. Ibobbly) was not effective in reducing suicidal ideation.

Lastly, Franklin et al. (2016) used a control version of the Therapeutic Evaluative Conditioning (TEC) MA as a control condition in three experiments. In the first study (Franklin A) the experimental version of TEC was a little more effective than the control condition. Contrary, in the second study (Franklin B) the control condition was a little more effective than the TEC. The main difference between these two studies was that study 2 used more aversive images than those presented in study 1. In the third study there was no significant difference between control and experimental condition. Study 3 (Franklin C) was similar to the two previous studies, however it had a greater focus on suicidal behaviors while the previous ones had a greater focus on non-suicidal injuries.

Therefore, only four MAs developed as an intervention for suicidal ideation and with evidence of effectiveness assessed through RCT were identified in the literature. The main findings suggest that three of these MAs (i.e. LifeApp’tite, VHB and TEC) show evidence of effectiveness in reducing suicidal ideation. However, these results should be treated with caution given the high risk of bias rated in at least one domain of each included study. In addition, all MAs located in this review were tested by only one study.

This low number of RCT addressing the effectiveness of MAs developed as an intervention for suicidal ideation has been also verified by another meta-analysis with a broader objective than ours (i.e. investigate the effectiveness of digital interventions for the self-management of suicidal ideation or self-harm) that found only eight RCT. Five of those RCT were carried out with web-based interventions and another one with adolescents, therefore, these studies were not included in our review (Witt et al., 2017). Only two RCT (i.e. Franklin et al., 2016 and Tighe et al., 2017) used by Witt et al. (2017) were also located and included in our review. It is noteworthy that three years later, our meta-analysis found only two new studies that were not found by Witt et al. (2017) (i.e. Bush et al., 2017 and O’Toole et al., 2019).

Therefore, in addition to the lack of academic literature on the use of MAs as suicide interventions, it is also observed a slow growth of this field of study. Besides, only a small number of MAs developed as an intervention for suicidal ideation have evidence of effectiveness, suggesting that technology-based suicide intervention remains understudied.

These findings are in line with a previous study, which reviewed online stores (i.e. iTunes and Google Play) and academic literature in search of MAs developed for suicide prevention. It was evidenced that there were 124 MAs related to suicide in virtual stores, 20 of which were related to prevention. However, only six scientific articles were found that presented evidence-based information about a few of these MAs. Thus, it is possible to suggest that most of the MAs related to suicide in online stores does not present any scientific evidence of effectiveness (De La Torre et al., 2017). Considering that suicide is a public health problem, it is highly recommended that MAs related to prevention or intervention should be developed with the assistance of health professionals and researchers.

Regarding total treatment duration, although none of the included studies has reported full data on the structure of treatments (i.e. treatment length and sessions frequency), a recent systematic review and meta-analysis, found no evidence to suggest that the number of sessions is associated with higher treatment effectiveness via e-health cognitive behavioural therapy in reducing both suicidal ideation and behaviour (Leavey & Hawkins, 2017). A similar result was found in a meta-regression review that aimed to examine the effectiveness of psychological and psychosocial interventions for reductions in repeated self-harm (Hetrick et al., 2016). Therefore, considering that all MAs located in our review used cognitive or behavioral therapy techniques, it is possible that the same principle applies to the included studies. In other words, it is possible that increasing the length of interventions with MAs would not result in an increase in effectiveness. However, this hypothesis must be further confirmed by new RCT that assess the effectiveness of MAs developed to reduce suicidal ideation.

The main limitations of this review are related to the use of only four databases and not performing all of the analyzes (i.e. subgroup analyses) proposed in the PROSPERO protocol. However, as previously stated, these analyzes were not performed because the included studies used different interventions, control conditions and psychometric instruments to measure suicidal ideation. Another limitation was the inclusion of only RCT or single group pre-post trial in cases of a standalone APP intervention. Although this criterion is common in systematic reviews and contributes to the improvement of review evidence level, it is possible that there are a few observational studies that demonstrate evidence of effectiveness in reducing suicidal risk using MAs.

Regarding the geographical origin of the MAs and the participants in the included studies, all MAs have an English-language interface, and none have been developed in Africa or Latin America. Also, there were no Latin Americans among the participants in any of the included studies. Therefore, it is recommended the development of new MAs or the cross-cultural adaptation of the MAs identified in this review to Latin America and Africa countries. Furthermore, considering that our findings suggest that MAs can be as effective as standard treatments in reducing suicide ideation, future studies should investigate the feasibility of using the available MAs or formulating new ones for use in the public health system, specially in low and middle-income countries. Lastly, the development of public policies that recommend and allow the use of these devices as intervention tools against suicidal ideation are recommended.

• How to cite this article: Cardoso, N. O., Salvador, E. Z., Tagliapietra, K. R., Sanseverino, M. A., Vissoci, J. R. N., & Machado, W. L. (2023). Effectiveness of mobile applications as intervention tools in suicidal ideation: a systematic review and meta-analysis. Estudos de Psicologia (Campinas), 40, e210035. https://doi.org/10.1590/1982-0275202340e210035

• Support

  Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes) (Process n°001).

References

• Barnett, A. G., Van Der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: what it is and how to deal with it. International journal of Epidemiology, 34(1), 215-220. https://doi.org/10.1093/ije/dyh299
  » https://doi.org/10.1093/ije/dyh299
• Bush, N. E., Smolenski, D. J., Denneson, L. M., Williams, H. B., Thomas, E. K., & Dobscha, S. K. (2017). A virtual hope box: randomized controlled trial of a smartphone app for emotional regulation and coping with distress. Psychiatric Services, 68(4), 330-336. https://doi.org/10.1176/appi.ps.201600283
  » https://doi.org/10.1176/appi.ps.201600283
• Bruffaerts, R., Demyttenaere, K., Hwang, I., Chiu, W. T., Sampson, N., Kessler, R. C., Alonso, J., Borges, G., Girolamo, G., Graaf, R., Florescu, S., Gureje, O., Hu, C., Karam, E. G., Kawakami, N., Kostyuchenco, S., Kovess-Masfety, V., Lee, S., Levinson, D., Matschinger, H., … Nock, M. K. (2011). Treatment of suicidal people around the world. British Journal of Psychiatry, 199(1), 64-70. https://doi.org/10.1192/bjp.bp.110.084129
  » https://doi.org/10.1192/bjp.bp.110.084129
• Chen, J. I., Smolenski, D. J., Dobscha, S. K., Bush, N. E., & Denneson, L. M. (2018). Correlates of mental health smartphone application use among patients with suicidal ideation. Journal of Technology in Human Services, 36(4), 191-207. https://doi.org/10.1080/15228835.2018.1502119
  » https://doi.org/10.1080/15228835.2018.1502119
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences 2nd ed. Erlbaum Associates.
• Czyz, E. K., Horwitz, A. G., Eisenberg, D., Kramer, A., & King, C. A. (2013). College students at elevated risk for suicide. National Institutes of Health, 61(7), 398-406. https://doi.org/10.1080/07448481.2013.820731
  » https://doi.org/10.1080/07448481.2013.820731
• Davis, C. E. (1976). The effect of regression to the mean in epidemiologic and clinical studies. American Journal of Epidemiology, 104(5), 493-498. https://doi.org/10.1093/oxfordjournals.aje.a112321
  » https://doi.org/10.1093/oxfordjournals.aje.a112321
• De la Torre, I., Castillo, G., Arambarri, J., López-Coronado, M., & Franco, M. A. (2017). Mobile Apps for suicide prevention: review of virtual stores and literature. JMIR MHealth and UHealth, 5(10), e130. https://doi.org/10.2196/mhealth.8036
  » https://doi.org/10.2196/mhealth.8036
• Denneson, L. M., Smolenski, D. J., Bauer, B. W., Dobscha, S. K., & Bush, N. E. (2019). The mediating role of coping self-efficacy in hope box use and suicidal ideation severity. Archives of Suicide Research, 23(2), 234-246. https://doi.org/10.1080/13811118.2018.1456383
  » https://doi.org/10.1080/13811118.2018.1456383
• Franklin, J. C., Fox, K. R., Franklin, C. R., Kleiman, E. M., Ribeiro, J. D., Jaroszewski, A. C., Hooley, J. M., & Nock, M. K. (2016). A brief mobile app reduces nonsuicidal and suicidal self-injury: evidence from three randomized controlled trials. Journal of Consulting and Clinical Psychology, 84(6), 544-557. https://doi.org/10.1037/ccp0000093
  » https://doi.org/10.1037/ccp0000093
• Fukuda, C. C., Penso, M. A., Amparo, D. M., Almeida, B. C., & Morais, C. A. (2016). Mental health of young Brazilians: barriers to professional help-seeking. Estudos de Psicologia (Campinas), 33(2), 355-365. https://doi.org/10.1590/1982-02752016000200017
  » https://doi.org/10.1590/1982-02752016000200017
• Hayes, J. F., Maughan, D. L., & Grant-Peterkin, H. (2016). Interconnected or disconnected? Promotion of mental health and prevention of mental disorder in the digital age. British Journal of Psychiatry, 208(3), 205-207. https://doi.org/10.1192/bjp.bp.114.161067
  » https://doi.org/10.1192/bjp.bp.114.161067
• Hetrick, S. E., Robinson, J., Spittal, M. J., & Carter, G. (2016). Effective psychological and psychosocial approaches to reduce repetition of self-harm: a systematic review, meta-analysis and meta-regression. BMJ Open, 6(9), e011024. http://dx.doi.org/10.1136/bmjopen-2016-011024
  » https://doi.org/10.1136/bmjopen-2016-011024
• Higgins J. P. T., & Green S. (2011). Cochrane handbook for Systematic Reviews of Interventions Version 5.1.0. Wiley Blackwell. www.handbook.cochrane.org
  » www.handbook.cochrane.org
• Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. The British Medical Journal, 327(7414), 557-560. https://doi.org/10.1136/bmj.327.7414.557
  » https://doi.org/10.1136/bmj.327.7414.557
• Khabsa, M., Elmagarmid, A., Ilyas, I., Hammady, H., & Ouzzani, M. (2016). Learning to identify relevant studies for systematic reviews using random forest and external information. Machine Learning, 102(3), 465-482. https://doi.org/10.1007/s10994-015-5535-7
  » https://doi.org/10.1007/s10994-015-5535-7
• Knaak, S., Mantler, E., & Szeto, A. (2017). Mental illness-related stigma in healthcare: barriers to access and care and evidence-based solutions. Healthcare Management Forum, 30(2), 111-116. https://doi.org/10.1177/0840470416679413
  » https://doi.org/10.1177/0840470416679413
• Lean, M., Fornells-Ambrojo, M., Milton, A., Lloyd-Evans, B., Harrison-Stewart, B., Yesufu-Udechuku, A., Kendall, T., & Johnson, S. (2019). Self-management interventions for people with severe mental illness: systematic review and meta-analysis. British Journal of Psychiatry, 214(5), 260-268. https://doi.org/10.1192/bjp.2019.54
  » https://doi.org/10.1192/bjp.2019.54
• Leavey, K., & Hawkins, R. (2017). Is cognitive behavioural therapy effective in reducing suicidal ideation and behaviour when delivered face-to-face or via e-health? A systematic review and meta-analysis. Cognitive Behaviour Therapy, 46(5), 353-374. https://doi.org/10.1080/16506073.2017.1332095
  » https://doi.org/10.1080/16506073.2017.1332095
• Lopes, C. S., Hellwig, N., E Silva, G. D. A., & Menezes, P. R. (2016). Inequities in access to depression treatment: results of the Brazilian National Health Survey – PNS. International Journal for Equity in Health, 15(1), 1-8. https://doi.org/10.1186/s12939-016-0446-1
  » https://doi.org/10.1186/s12939-016-0446-1
• O’Toole, M. S., Arendt, M. B., & Pedersen, C. M. (2019). Testing an app-assisted treatment for suicide prevention in a randomized controlled trial: effects on suicide risk and depression. Behavior Therapy, 50(2), 421-429. https://doi.org/10.1016/j.beth.2018.07.007
  » https://doi.org/10.1016/j.beth.2018.07.007
• Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan-a web and mobile app for systematic reviews. Systematic Reviews, 5(1), 1-10. https://doi.org/10.1186/s13643-016-0384-4
  » https://doi.org/10.1186/s13643-016-0384-4
• Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., ... Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372(71), 1-9. https://doi.org/10.1136/bmj.n71
  » https://doi.org/10.1136/bmj.n71
• Priester, M. A., Browne, T., Iachini, A., Clone, S., DeHart, D., & Seay, K. D. (2016). Treatment Access barriers and disparities among individuals with co-occurring mental health and substance use disorders: an integrative literature review. Journal of Substance Abuse Treatment, 61, 47-59. https://doi.org/10.1016/j.jsat.2015.09.006
  » https://doi.org/10.1016/j.jsat.2015.09.006
• Schwarzer, G. (2019). General package for meta-analysis. The R Project for Statistical Computing https://cran.r-project.org/web/packages/meta/meta.pdf
  » https://cran.r-project.org/web/packages/meta/meta.pdf
• Tighe, J., Shand, F., Ridani, R., Mackinnon, A., De La Mata, N., & Christensen, H. (2017). Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial. BMJ Open, 7(1), e013518. http://dx.doi.org/10.1136/bmjopen-2016-013518
  » https://doi.org/10.1136/bmjopen-2016-013518
• Witt, K., Spittal, M. J., Carter, G., Pirkis, J., Hetrick, S., Currier, D., Robinson, J., & Milner, A. (2017). Effectiveness of online and mobile telephone applications (‘apps’) for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysis. BMC Psychiatry, 17(1), 297. http://dx.doi.org/10.1186/s12888-017-1458-0
  » https://doi.org/10.1186/s12888-017-1458-0
• World Health Organization (2019). Suicide in the world: global health estimates https://www.who.int/publications-detail/suicide-in-the-world
  » https://www.who.int/publications-detail/suicide-in-the-world

Edited by

Editor

Raquel Souza Lobo Guzzo, André Luiz Monezi de Andrade

Publication Dates

• Publication in this collection
  01 May 2023
• Date of issue
  2023

History

• Received
  03 Mar 2021
• Reviewed
  17 Dec 2021
• Accepted
  13 June 2022