Evaluating the Efficacy of a Parent-Implemented Autism Intervention Program in Northern Brazil

Avaliando a Eficácia de uma Intervenção ao Autismo Implementada por Pais no Norte do Brasil

Evaluando la Eficacia de una Intervención al Autismo Implementada por Padres en el Norte de Brasil

Abstract

The shortage of opportunities to access effective interventions for autism in developing countries is a major concern. Research on parent-implemented interventions is critical for the dissemination of evidence-based treatments. The present study evaluated the efficacy of a parent-implemented intervention program that is offered to low and lower middle-class families in northern Brazil. The participants were three children who were diagnosed with autism and their respective parents/caregivers. All of the parents had received training on the implementation of discrete-trial teaching. We collected data on the children's performance and evaluated the integrity of the parent-implemented intervention. The results showed improvements in performance for two of the three children whose parents implemented the intervention with a high degree of integrity. For the third parent whose child had lower performance, low attendance during the training sessions was correlated with the child's slower rate of acquisition. These findings support the potential benefits of parent-implemented intervention programs for children with autism and extend to the socio-cultural context of developing countries, such as Brazil.

Keywords:
Autism Spectrum Disorder; Applied Behavior Analysis; parent-implemented intervention; verbal behavior

Resumo

A escassez de oportunidades para acessar a intervenção efetiva para o autismo nos países em desenvolvimento é uma grande preocupação. Assim, a pesquisa sobre a intervenção implementada pelos pais é crítica para disseminar tratamentos baseados em evidência. O estudo atual avaliou a eficácia de um programa de intervenção implementado pelos pais oferecido a famílias de classes baixa e média-baixa no Norte do Brasil. Os participantes foram três crianças diagnosticadas com autismo e seus respectivos pais/cuidadores. Todos os pais haviam recebido treinamento sobre a implementação de ensino por tentativa discreta. Coletamos dados sobre o desempenho das crianças, bem como da integridade da intervenção implementada pelos pais. Os resultados mostraram aumento nos desempenhos com duas das três crianças, cujos pais implementaram tratamento com alto grau de integridade. Para o terceiro pai, o baixo comparecimento durante as sessões de treinamento foi correlacionado com a taxa de aquisição mais lenta da criança. Esses achados apoiam os benefícios potenciais dos programas de intervenção implementados pelos pais para crianças com autismo e se estendem ao contexto sociocultural de países em desenvolvimento como o Brasil.

Palavras-chave:
Transtorno do Espectro do Autismo; Análise Aplicada do Comportamento; intervenção implementada pelos pais; comportamento verbal

Resumen

La escasez de oportunidades para acceder a intervenciones efectivas para el autismo en los países en desarrollo es de gran preocupación. Así, la investigación sobre la intervención implementada por los padres es fundamental para difundir los tratamientos basados en evidencia. El presente estudio evaluó la eficacia de un programa de intervención implementado por los padres ofrecido a las familias de clase media baja y baja en el norte de Brasil. Los participantes fueron tres niños diagnosticados con autismo y sus respectivos padres/cuidadores. Todos los padres habían recibido capacitación sobre la implementación de la enseñanza de prueba discreta. Recogemos datos sobre el rendimiento de los niños, así como la integridad de la intervención implementada por los padres. Los resultados mostraron mejores resultados con los tres niños cuyos padres implementaron un tratamiento con un alto grado de integridad. Para el tercer padre, la baja asistencia durante las sesiones de entrenamiento se correlacionó con la menor tasa de adquisición de su hijo. Estos hallazgos respaldan los beneficios potenciales de los programas de intervención implementados por los padres para niños con autismo y se extienden al contexto sociocultural de países en desarrollo como Brasil.

Palabras clave:
Trastorno del Espectro del Autismo; Análisis Aplicado del Comportamiento; intervención implementada por los padres; comportamiento verbal

Behavior analysts have successfully taught a variety of applied behavior analytic techniques to parents, caregivers, and teachers of children with disabilities. These techniques include functional analysis (e.g., Moore et al., 2002; Wallace, Doney, Mintz-Resudek, & Tarbox, 2004), incidental teaching (e.g., Alpert & Kaiser, 1992; Charlop-Christy & Carpenter, 2000), and discrete-trial teaching (DTT; e.g., Eid et al., 2017; Ferreira, Silva, & Barros, 2016). For example, Eid et al. (2017) used a brief Behavioral Skills Training package to teach parents of children with Autism Spectrum Disorder (ASD) in Saudi Arabia to implement DTT. The parents' accuracy increased from an average of 20% at baseline to an average of 90% after the intervention. This improvement was also associated with improvements in the children's behavior. According to the authors, research still needs to be conducted to develop procedures that require fewer resources (e.g., time and money) and can be disseminated to areas with a relative scarcity of qualified professionals.

The advancement of research on the efficacy and efficiency of parent-implemented and caregiver-based interventions is an important topic for applied behavior analysts because it promotes the generalizability of their interventions (i.e., a behavioral change must be sustained in various settings and in the presence of multiple individuals; Baer, Wolf, & Risley, 1968). Moreover, teaching caregivers to implement components of behavioral interventions may also circumvent the need for additional resources (e.g., trained professionals) in environments where their availability is scarce (Barboza, Silva, Barros, & Higbee, 2015).

In Brazil, public healthcare systems lack the funding, personnel, and infrastructure to offer affordable, high-quality, and early intensive behavior analytic interventions to individuals who are diagnosed with autism (de Mello, Andrade, Ho, & Dias, 2013). The lack of well-trained professionals and the wide range of socioeconomic and educational levels among families make it much more difficult to ensure access to effective interventions, even when the intervention is implemented by caregivers. Evaluating the feasibility, efficiency, and efficacy of autism intervention programs under these conditions is an important focus of applied research in Brazil.

Our research group (Barboza et al., 2015) recently evaluated the efficacy of an instructional video modeling procedure to teach three parents of children with autism from northern Brazil to implement DTT. After being pretested on the implementation of DTT with a confederate, the parents watched three didactic videos, each focusing on one of the following topics: (1) "How to conduct discrete trials," (2) "Prompting procedures," and (3) "Correction procedures." Additional instructions and feedback were provided when the accuracy of the implementation was below 90% during the posttests. All three parents reached mastery criterion with minimal or no instruction, showing an average improvement of 58% from baseline scores. Training lasted an average of 5 hours 30 minutes per parent.

After completing the aforementioned study, these parents were invited to participate in the present study to assess whether the parental implementation of DTT that focuses on verbal skills that were selected from the participants' individualized educational plans affect their children's performance. We measured the procedural integrity of implementation (i.e., treatment integrity [TI]) and its impact on the acquisition of verbal skills in children who were diagnosed with ASD.

Method

Participants and Setting

The participants were three parents and their respective children: Mario (32 years old) and Manuel (7 years old); Flavia (36 years old) and Guto (4 years old); and Alice (33 years old) and Luana (5 years old). The parents and their children spoke Portuguese as their primary language. The parents had completed high school. In a screening anamnesis interview, they reported belonging to either low- or lower-middle socioeconomic classes. As mentioned above, these parents had already been trained how to implement DTT during their participation in a previous study (Barboza et al., 2015) that was conducted 2 weeks before the onset of the present study. All three children were diagnosed with ASD (certified by a formal report issued by a neuropedriatrician of the Bettina Ferro de Souza University Hospital). The children presented moderate to severe deficits in their vocal-verbal repertoires, social interactions, and academic skills (according to an evaluation based on the Verbal Behavior Milestones Assessment Program [VB-MAPP]; Sundberg, 2014). None of the children had received any behavior analytic intervention before beginning the present study. Manuel and Luana presented considerable deficits in the tact repertoire (3 and 0 points, respectively, on a 0-5 scale) and mand repertoire (2 and 0 points, respectively, on a 0-5 scale; VB-MAPP, Level 2). Luana's scores also showed deficits in listener behavior. Guto presented deficits in mands and vocal behavior (VB-MAPP, Level 1).

Data collection was planned to occur at the participants' homes every day and in a research room (5.0 m × 2.5 m) at a large public university in Brazil three times per week. Generalization sessions and follow-up sessions were always conducted at the university (with the exception of a few generalization sessions that were conducted at Manuel's home for Program 3; see below). The parents collected data using a prescribed data sheet. All of the sessions that were conducted at the university were videotaped and monitored by the experimenter. The experimenter was not present at the home sessions, and these sessions were not videotaped. To assess whether home sessions had occurred, we collected the completed data sheets and also evaluated the children's progress during the sessions that were conducted at the university (see below). We hypothesized that the participants' performance accuracy that was reported in the data sheets should be correlated with performance accuracy during the sessions that were conducted at the university. The present study was approved by the Research Ethics Committee of the Health Sciences Institute of Universidade Federal do Pará - UFPA (opinion no, 175.303 of 12/14/2012).

Response Measurement and Experimental Design

The dependent measures included the percentage of correct independent (unprompted) responses. The definition of a correct response varied depending on the specific teaching program that was being implemented (Table 1). For example, during the program entitled, "Tact of Actions Using Two or More Words," correct responses, for example, consisted of saying, "He is running" when presented with a picture of a person running. Table 1 presents a list of all of the teaching programs that were implemented during the study.

Table 1
List of Programs Performed for Guto, Manuel, and Luana

Participant		Programs
Guto	Mand using one word (e.g., "cookie")	Tact of one-component actions (e.g., "clapping")	Intraverbal - Personal Information (e.g., saying "Guto" when asked "What is your name?")
Manuel	Tact of two component actions (e.g., "riding a bike")	Mand using sentences (e.g. "I want water")	Intraverbal-Personal Information
Luana	Tact of Emotions (e.g. "she is sad")	Intraverbal PersonalInformation

We used a multiple-probe design (Horner & Baer, 1978) across programs to assess the effects of the parent-implemented intervention on children's performance.

Procedure

Baseline. We assessed the children's performance by having both parents and experimenters conduct assessment sessions on different days. Each session consisted of five trials, with no programmed consequences. Initially, three baseline sessions were conducted for each of the teaching programs. The experimenters conducted the first baseline session, either the experimenters or parents conducted the second session, and the parents conducted the third session. Additional baseline sessions were conducted for all of the programs during the baseline phase to check for data stability and tendency. Following the requirements of the multiple-probe design, baseline sessions during the intervention phase with the first teaching program were conducted for the remaining teaching programs, and so on. As part of the experimental design, the experimenters conducted additional baseline sessions for all of the remaining programs when performance reached the criterion for a targeted teaching program (Figure 1).

Figure 1
Performance accuracy (percentage of correct responding) in the Baseline (BL), Intervention, Generalization (GEN), and Follow-up phases for Manuel, Guto, and Luana. Open squares indicate the accuracy of implementation by the parents. All of the other data concern the children’s performance. Black filled circles present data from the baseline sessions. Performance during the intervention is presented by gray filled circles (five-trial sessions conducted at the university) and gray triangles (10-trial sessions conducted at home). Black circles indicate when a prompting change occurred from the vocal model to the visual model.

Intervention. Manuel and Guto were exposed to three teaching programs. Luana was exposed to only two teaching programs because her performance did not reach the acquisition criterion, so the third program was not initiated. We assigned sets of teaching programs (and targets) to each child according to the results of their initial VB-MAPP evaluation (Sundberg, 2014). Manuel's teaching programs consisted of "Tact of Actions Using Two or More Words" (Program 1), "Mand Using Sentences" (Program 2), and "Intraverbal of Personal Information" (Program 3). Guto's teaching programs were "Vocal Mand" (Program 1), "Tact of Actions" (Program 2), and "Intraverbal of Personal Information" (Program 3). Luana's teaching programs consisted of "Tact of Emotions" (Program 1) and "Intraverbal of Personal Information" (Program 2). The targeted performance could be one or more word utterances. As an example of the variation of targets and instructions, the following targets were selected in the "Intraverbal of Personal Information" program: "child's own name," "mother's name," "school name," and "age." For the first target, the following instructions were presented: "What is your name?", "How do I call you?", and "Your name is..." The following instructions were presented during generalization sessions: "You are called..." and "What name do you have?"

Following baseline, the parents began to implement Program 1 only. When the children's performance in Program 1 reached at least 80% accuracy for two consecutive sessions (with at least one session that was conducted at the university), a baseline session was conducted for all of the other teaching programs. An additional baseline session was then conducted with Program 2, after which the caregiver began to implement Program 2 while continuing to implement Program 1 for maintenance. The introduction of Program 3 followed the same pattern.

The parents were required to implement at least 10 trials of each program per day and attend supervision sessions three times per week at the university campus. During these supervision sessions, the parents' implementation of the teaching programs was video recorded. We used the videos to score TI and interobserver agreement (IOA) and assess each child's performance. When the accuracy of parent implementation was below 75%, we provided verbal instructions and role-playing with feedback, focusing on the specific components of teaching that were implemented incorrectly. Positive feedback (e.g., "very good," "excellent," "perfect," etc.) was immediately delivered contingent upon accurate implementation.

Generalization and Follow-up. To assess generalization, we conducted sessions in which the participants were presented with two to four novel exemplars per target. For example, Guto was taught to tact actions, such as "running" and "taking a shower," during training, and his performance was assessed for generalization by presenting him with new figures of people running and taking a shower. These sessions were conducted 2 days after reaching the mastery criterion. Each stimulus was presented three times, and the order in which the teaching programs were presented corresponded to the order that is outlined in Table 1 (see above). Although the consequences were the same as during the intervention, no prompts were provided. Notably, Luana's performance did not reach the accuracy criterion; therefore, generalization sessions were not conducted for her.

To assess maintenance, we conducted a follow-up session approximately 1 month after the last generalization session (Manuel and Guto) or last intervention session (Luana). During the follow-up sessions, each training target was presented twice using the same procedures as described for Intervention above, but no prompts were delivered.

Treatment Integrity and Interobserver Agreement. We assessed the TI of parent implementation only for the sessions that were conducted at the university. An implementation checklist was used to evaluate each trial of every session. Each component of a trial (i.e., getting the child's attention, presenting antecedent stimuli, prompting, implementing an error correction or delivering immediate reinforcement, and recording data) was evaluated by scoring one point for correct implementation and zero points for incorrect implementation (checklist available from the authors upon request). We assessed TI by dividing the total number of components that were implemented accurately by the total of number of components, multiplied by 100. The TI averaged 97.3% across parents. More specifically, the TI was 98.9% for Mario, 97.5% for Flavia, and 95.6% for Alice.

We evaluated IOA during 30% of the sessions that were conducted on campus for each participant. The IOA was calculated by adding the number of agreements divided by the total number of agreements plus disagreements, multiplied by 100. The IOA averaged 99% for Manuel and Guto (range, 98-100%) and 100% for Luana.

Results

Figure 1 shows each child's percentage of correct responses during Baseline (BL), Intervention, Generalization (GEN), and Follow-up for Manuel (Programs 1, 2, and 3), Guto (Programs 1, 2, and 3), and Luana (Programs 1 and 2). Black filled circles depict data from the pre-intervention sessions. Gray filled circles depict data from the intervention sessions that were conducted on campus, and gray filled triangles depict data from the home sessions.

During baseline, none of the participants performed any of the skills to be taught, with the exception of some correct responding in session 29 for Guto (Program 2; 25%). During the intervention, a clear upward trend was observed for Manuel and Guto, whereas Luana's performance showed a downward trend for Program 1 and stability below the criterion for Program 2. The mean accuracy per teaching program during the intervention was 87% for Program 1, 65% for Program 2, and 70% for Program 3 for Manuel, 39% for Program 1, 60% for Program 2, and 17% for Program 3 for Guto, and 77% for Program 1 and 18% for Program 2 for Luana. Improvements occurred at or near the point of intervention for Manuel, were somewhat delayed for Guto, and were fairly immediate for Luana, albeit slow across the two baselines (Program 1 and 2). Guto's average performance during the intervention improved slowly, as shown in Figure 1. Nonetheless, his performance eventually reached mastery criterion (80%) in all three teaching programs.

The alternation between gray filled circles and gray filled triangles during the intervention phase (Figure 1) shows that Guto's and Manuel's parents (Flavia and Mario, respectively) systematically attended the weekly supervision sessions (gray filled circles). In fact, the data showed that Manuel's and Guto's parents attended supervision sessions at a much higher rate than Luana's parents (1.75 sessions per week for Manuel's parents, 2.16 sessions per week for Guto's parents, and 0.80 sessions per week for Luana's parents). The TI data (see open squares in Figure 1) showed that the intervention that was implemented by Manuel and Guto's parents was more accurate than the intervention that was implemented by Luana's parents. The average accuracy was 96% for Manuel's parents (98%, 90%, and 100% for Programs 1, 2, and 3, respectively) and 91% for Guto's parents (84%, 96%, and 93% for Programs 1, 2, and 3, respectively). The average accuracy for Luana's parents was 85% (87% and 83% for Programs 1 and 2, respectively). Thus, both the rate of attendance and TI of the parents appeared to be correlated with the outcomes of the intervention (i.e., child performance).

Luana's mother attended the lowest number of supervision sessions when these were conducted on campus (23%). Data on Luana's performance for Program 1 showed an increasing trend during the first 10 sessions while her mother attended the sessions at the prescribed frequency (i.e., three times per week), then started to vary considerably (between 20% and 100%) as her mother started to miss sessions (see gray circles after Session 20 in Figure 1). The TI percentage scores decreased (76% on average over the last three sessions of Program 1 compared with 96% on average in previous sessions; see Figure 1, white squares). Average implementation, the accuracy for Luana's mother (Program 2) was the lowest throughout the study (81.75%; standard deviation [SD] = 8.56%). Consequently, Luana's performance (see Luana's Program 2 data in Figure 1) was less accurate and unstable (18.60% average accuracy; SD = 20.20%). At the end of Luana's intervention phase, her performance increased as her mother resumed attendance at the supervision sessions but gradually decreased again. At this point, Luana's mother seldom brought any completed data sheets from home, suggesting that the sessions were not conducted in the home setting (see Sessions 46-58 for Luana). However, because we did not manipulate TI as an independent variable, it was not possible to determine whether Luana's poor performance was a result of low TI scores.

Discussion

Overall, our data suggest that when delivered as prescribed, the parent-implemented intervention effectively taught all three children the specific targeted skills. Additionally, the intervention produced generalization to novel targets, and the acquired performance was maintained at 30-day follow-up. Most notably, the children's performance appeared to be highly correlated with the parents' participation in training and implementation of the procedures at home. Our data suggest that children can benefit from behavior analytic intervention that is implemented by their parents with the support and close supervision of trained professionals.

The present study was the first well-controlled evaluation of a parent-implemented behavior analytic intervention program for children who were diagnosed with ASD in Brazil. Our findings may encourage subsequent applied research that seeks to overcome the challenge of disseminating high-quality autism interventions in developing countries, particularly with low-income populations (c.f., Wang, 2008) and in areas where well-trained behavior analysts may be scarce. Parental implementation may be an important component of behavior-analytic intervention as a way to provide the necessary intensity of treatment, given the shortage of professionals. Further research should focus on determining the profile of families who are suitable for such an intervention strategy. We recognize that this type of intervention may not be appropriate for some of the affected families.

One limitation of the present study was that the parent-implemented intervention was evaluated only with regard to a limited number of target skills. Future studies should evaluate parents' performance under more complex circumstances, such as implementing a comprehensive intervention program.

Another limitation was that TI was only assessed on campus. Importantly, there are many differences in environmental arrangements when sessions are conducted in a family's home compared with a more controlled setting, such as a research room on a university campus. These differences include the number of children and adults who are present, the quality and availability of program materials, and the number of distracting stimuli, among others. Further research should directly evaluate parent implementation at home and assess the critical variables that determine the success of parent-implemented interventions.

Although we documented the efficacy of the parent-implemented intervention, the present study did not compare the efficacy or efficiency of this intervention with one that is fully conducted by professionals. Future studies should compare the efficacy and efficiency of parent-implemented intervention with those that are implemented solely by professionals. Some of our preliminary data in this regard (Oliveira, 2017) show that if high TI is assured, then there is little or no difference between parental and professional implementation under the circumstances described herein.

The present findings show that parent-implemented interventions can be successful under highly challenging conditions and with populations that feature a wide range of socioeconomic and educational levels, thus corroborating the findings of other studies that reported effective parent-implemented interventions with children with autism in North America (e.g., Hsieh, Wilder, & Abellon, 2011) and Asia (Wang, 2008).

• Support: Brazilian National Council for Scientific and Technological Development (CNPq).

References

• Alpert, C. L., & Kaiser, A. N. (1992). Training parents as milieu language teachers. Journal of Early Intervention, 16,31-52. doi: 10.1177/105381519201600104
  » https://doi.org/10.1177/105381519201600104
• Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1,91-97. doi: 10.1901/jaba.1968.1-91
  » https://doi.org/10.1901/jaba.1968.1-91
• Barboza, A. A., Silva, A. J. M., Barros, R. S., & Higbee, T. S. (2015). Efeitos de videomodelação instrucional sobre o desempenho de cuidadores na aplicação de programas de ensino a crianças diagnosticadas com autismo. Acta Comportamentalia, 23,405-421. Retreived from http://www.revistas.unam.mx/index.php/acom/article/view/53794/47889
  » http://www.revistas.unam.mx/index.php/acom/article/view/53794/47889
• Charlop-Christy, M. H., & Carpenter, M. H. (2000). Modified incidental teaching sessions: A procedure for parents to increase spontaneous speech in their children with autism. Journal of Positive Behavior Interventions, 2,98-112. doi: 10.1177/109830070000200203
  » https://doi.org/10.1177/109830070000200203
• de Mello, A. M. S. R., Andrade, M. A., Ho, H., & Dias, I. S. (2013). Retratos do Autismo no Brasil São Paulo, SP: Associação de Amigos do Autista.
• Eid, A. M., Aljaser, S. M., AlSaud, A. N., Asfahani, S. M., Alhaqbani, O. A., Mohtasib, R. S., Aldhalaan, H. M., & Fryling, M. (2017). Training parents in Saudi Arabia to implement discrete trial teaching with their children with autism spectrum disorder. Behavior Analysis in Practice, 10(4),402-406. doi: 10.1007/s40617-016-0167-3
  » https://doi.org/10.1007/s40617-016-0167-3
• Ferreira, L. A., Silva, A. J. M. , & Barros, R. S. (2016). Ensino de aplicação de tentativas discretas a cuidadores de crianças diagnosticadas com autismo. Perspectivas em Análise do Comportamento, 7(1),101-113. doi: 10.18761/pac.2015.034
  » https://doi.org/10.18761/pac.2015.034
• Horner, R. D., & Baer, D. M. (1978). Multiple-probe technique: A variation of the multiple baseline. Journal of Applied Behavior Analysis, 11,189-196. doi: 10.1901/jaba.1978.11-189
  » https://doi.org/10.1901/jaba.1978.11-189
• Hsieh, H. H., Wilder, D. A., & Abellon, O. E. (2011). The effects of training on caregiver implementation of incidental teaching. Journal of Applied Behavior Analysis, 44,199-203. doi: 10.1901/jaba.2011.44-199
  » https://doi.org/10.1901/jaba.2011.44-199
• Moore, J. W., Edwards, R. P., Sterling-Turner, H. E., Riley, J., Dubard, M., & McGeorge, A. (2002). Teacher acquisition of functional analysis methodology. Journal of Applied Behavior Analysis, 35,73-77. doi: 10.1901/jaba.2002.35-73
  » https://doi.org/10.1901/jaba.2002.35-73
• Oliveira, J. S. C. (2017). Parent-implemented and professional intervention to children diagnosed with ASD (Master's thesis). Retreived from http://ppgtpc.propesp.ufpa.br/ARQUIVOS/dissertacoes/Juliana%20Sequeira%20Cesar%20de%20Oliveira%202017.pdf
  » http://ppgtpc.propesp.ufpa.br/ARQUIVOS/dissertacoes/Juliana%20Sequeira%20Cesar%20de%20Oliveira%202017.pdf
• Sundberg, M. L. (2014). Verbal Behavior Milestones Assessment and Placement Assessment Program: A Language and Social Skills Assessment Program for Children with Autism or other developmental disabilities Concord, CA: AVB Press.
• Wallace, M. D., Doney, J. K., Mintz-Resudek, C. M., & Tarbox, R. S. F. (2004). Training educators to implement functional analyses. Journal Applied Behavior Analysis, 37,89-92. doi: 10.1901/jaba.2004.37-89
  » https://doi.org/10.1901/jaba.2004.37-89
• Wang, P. (2008). Effects of a parent training program on the interactive skills of parents of children with autism in China. Journal of Policy and Practice in Intellectual Disabilities, 5,96-104. doi: 10.1111/j.1741-1130.2008.00154
  » https://doi.org/10.1111/j.1741-1130.2008.00154

Publication Dates

• Publication in this collection
  13 June 2019
• Date of issue
  Apr-Jun 2019

History

• Received
  02 Feb 2018
• Reviewed
  28 Aug 2018
• Accepted
  02 Sept 2018