SciELO - Scientific Electronic Library Online

 
vol.9 issue3Short-segment fixation of thoracolumbar burst fractures using pedicle fixation at the level of the fracture and in situ bendingTomographic evaluation of odontoid parameters related to its internal fixation author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Coluna/Columna

Print version ISSN 1808-1851

Coluna/Columna vol.9 no.3 São Paulo July/Sept. 2010

http://dx.doi.org/10.1590/S1808-18512010000300012 

ARTIGO ORIGINAL ORIGINAL ARTICLE

 

Efficacy and safety of dynamic stabilization for patients with degenerative disc, spinal stenosis and low back pain: a systematic review of randomized controlled clinical trials

 

Eficácia e segurança dos sistemas de estabilização dinâmica em pacientes com doença discal degenerativa, estenose de canal e lombalgia: revisão sistemática de estudos clínicos randomizados

 

Eficacia y seguridad de los sistemas de estabilización dinámica en pacientes con enfermedad discal degenerativa, estenosis de canal y lumbalgia: revisión sistemática de estudios clínicos randomizados

 

 

Gustavo Carriço de OliveiraI; Pedro Henrique Lacombe AntoneliII

IMD; Department of Orthopaedics and Spine Surgery at Hospital Homero de Miranda Gomes - São José (SC), Brazil
IIMD; Hospital Mãe de Deus - Porto Alegre (RS), Brazil

Correspondence

 

 


ABSTRACT

STUDY DESIGN: Systematic review of the literature.
OBJECTIVE: To perform a systematic review of the literature to organize, critical appraisal and select the best evidence available about the efficacy and safety of non-fusion fixation and its potential use for patients with degenerative disc, spinal stenosis and low back pain.
SUMMARY OF BACKGROUND DATA: Recent reports have increased debate about the role of dynamic stabilization in the treatment of chronic back pain associated with lumbar disc degeneration and spinal stenosis. We conducted a systematic review of randomized trials through a more sensitivity search strategy and rigorous criteria applied for the type of studies.
METHODS: An electronic search was made in the databases of the Cochrane Central Register of Controlled Trials, Medline, Embase, and Latin American and Caribbean Health Sciences (Lilacs) extended to November 31, 2008, with no linguistic restrictions.
RESULTS: One randomized controlled trial that fulfilled the inclusion criteria described above was included in this review.
CONCLUSION: The data included in this review show that the use of non-fusion stabilization could be a suitable alternative to another therapies in well selected patients with spinal stenosis and degenerative disc disease. This review highlighted the need for continued research into the use of non-fusion stabilization in the treatment of spinal disorders. There is an urgency need to conduct randomized clinical trials. Long-term efficacy should be evaluated.

Keywords: Spine; Intervertebral disk; Spinal stenosis; Low back pain; Orthopedic procedures/methods; Randomized controlled trials


RESUMO

DESENHO DE ESTUDO: Revisão sistemática da literatura.
OBJETIVO:
Realizar revisão sistemática da literatura para organizar, avaliar e selecionar evidências a respeito do uso dos sistemas de estabilização dinâmica, sua eficácia e segurança no tratamento da doença degenerativa discal, estenose do canal lombar e dor lombar.
REVISÃO DE LITERATURA:
Publicações recentes têm aumentado o debate acerca do papel dos sistemas de estabilização dinâmica no tratamento de dor lombar crônica associada à degeneração discal lombar e estenose de canal lombar. Conduzimos a revisão sistemática de ensaios randomizados por meio de estratégia de busca apurada e seleção criteriosa aplicadas para cada tipo de estudo.
MÉTODOS: Revisão da literatura pro meio das seguintes fontes: Cochrane Central Register of Controlled Trials, Medline, Excerpta Medica database (EMBASE) e Literatura Latino-Americana e do Caribe em Ciências da Saúde (Lilacs) identificando os estudos clínicos randomizados ou não. Não houve restrição de idioma e a data limite de busca foi novembro de 2008.
RESULTADOS: Um ensaio controlado randomizado que preencheu os critérios de inclusão descritos acima foi incluído neste estudo.
CONCLUSÃO: Os dados incluídos nesta revisão mostram que o uso dos sistemas de estabilização sem fusão é alternativa aceitável para tratamento de pacientes com estenose de canal lombar e doença degenerativa discal, em pacientes bem selecionados. Esta revisão destacou a necessidade de pesquisa continuada sobre o uso dos sistemas de estabilização dinâmica para tratamento das patologias da coluna vertebral. Existe uma necessidade urgente para condução de ensaios clínicos randomizados. Eficácia em longo prazo deve ser avaliada.

Descritores: Coluna vertebral; Disco intervertebral; Estenose espinhal; Dor lombar; Procedimentos ortpédicos/métodos; Ensaio clínico controlado aleatório


RESUMEN

DISEÑO DEL ESTUDIO: revisión sistemática de la literatura.
OBJETIVO:
realizar una revisión sistemática de la literatura para organizar, evaluar y seleccionar las mejores evidencias al respecto del uso de los sistemas de estabilización dinámica, su eficacia y seguridad en el tratamiento de la enfermedad degenerativa discal, estenosis del canal lumbar y dolor lumbar a través de estrategias de búsqueda apurada y selección con criterios aplicados para cada tipo de estudio.
REVISIÓN DE LITERATURA:
publicaciones recientes han aumentado el debate acerca del papel de los sistemas de estabilización dinámica en el tratamiento del dolor lumbar crónico asociado a la degeneración discal lumbar y estenosis de canal lumbar. Fue conducida una revisión sistemática de ensayos randomizados a través de la estrategia de búsqueda apurada y selección con criterios aplicados para cada tipo de estudio.
MÉTODOS: una revisión de la literatura a través de las siguientes fuentes fue realizada: Cochrane Central Register of Controlled Trials, MEDLINE, Excerpta Medica database (EMBASE) y Literatura latino-Americana y del Caribe en Ciencias de la Salud (LILACS). La revisión pudo identificar los estudios clínicos randomizados o no. No hubo restricción de idioma y la fecha límite de búsqueda fue 31 de noviembre de 2008.
RESULTADOS: un ensayo controlado randomizado, que llenó los criterios de inclusión descritos encima, fue incluido en esto estudio.
CONCLUSIÓN: los datos incluidos en esta revisión muestran que el uso de los sistemas de estabilización sin fusión son una alternativa aceptable para el tratamiento de pacientes con estenosis de canal lumbar y enfermedad degenerativa discal en pacientes bien seleccionados. Esta revisión destaca la necesidad de investigación continuada sobre el uso de los sistemas de estabilización dinámica para tratamiento de las patologías de la columna vertebral. Existe una necesidad urgente para la conducción de ensayos clínicos randomizados. La eficacia a largo plazo debe de ser evaluada.

Descriptores: Columna vertebral; Disco intervertebral; Estenosis lumbar; Dolor de la región lumbar; Procedimientos ortopédicos/métodos; Ensayo clínico controlado aleatorio


 

 

BACKGROUND

The standard treatment for degenerative disc disease, low back pain and spinal stenosis has been the spinal fusion with rigid fixation1. However, new therapies have been arising to obtain postoperative stability intending to achieve more physiological results, without the loss of motion required by fusion surgery. Soft fixation or dynamic stabilization is procedures that can reduce the compensatory hypermotion stress associated with instability and degenerative disc disease. Dynamic stabilization devices place the posterior structures under tension and create a focal increase in lordosis, altering the mechanical loading of the motion segment by unloading the disc. These techniques are known as a motion preserving devices to provide stability to restore normal segmental kinematics and also to avoid adjacent segment degeneration2.

Techniques such as Dynesys, Bioflex, Graf Ligamentoplasty, Wallis and X Stop can theoretically achieve stabilization without bone grafing3. However, few studies evaluated the clinical efficacy of these interventions, mainly, in long-term follow-up.

Although, there is a review about this topic4, we proposed to perform a systematic review of the literature to organize, critical appraisal and select the best evidence available about the efficacy and safety of non-fusion fixation and its potential use for patients with degenerative disc disease, spinal stenosis and low back pain through a more sensitivity search strategy and rigorous criteria applied for the type of studies.

 

METHODS

Literature search

There was no language restriction. Trials were obtained from the following sources: Cochrane Central Register of Controlled Trials (Central, The Cochrane Library, issue 3, 2008), Medical Literature Analysis and Retrieval System Online (Medline; 1966-2008), Excerpta Medica database (Embase; 1980-2008) and Literatura Latino-Americana e do Caribe em Ciências da Saúde (Lilacs; 1982-2008) to identify randomized and quasi-randomized controlled clinical trials. The date of the last search was November 2008.

The databases were searched using a comprehensive search strategy for degenerative disc, spinal stenosis, low back pain and non-fusion fixation, along with Mesh and text words, including an exhaustive list of synonyms. The search strategy was adapted for each database in order to achieve more sensitivity. References in the relevant studies identified were also scrutinized for additional citations. The summary of the bibliographic search strategies for type of clinical situation and intervention of interest are showed in Appendix 1.

Data collection

Two reviewers independently screened the trials identified by the literature search, extracted the data, assessed trial quality and analyzed the results. A standard form was initially used to extract the following information: study characteristics (type of design and randomization methods), participants, interventions and outcomes.

Study selection

We included randomized controlled trials that specifically stated that the conditions under investigation were dynamic stabilization devices (Dynesis; BioFlex Graf Ligamentoplasty, X Stop, etc.) and which involved adults and/or children diagnosed with degenerative disc, spinal stenosis and/or low back pain. The diagnostic was based on computed tomography scan, magnetic resonance imaging and/or radiographic signs, physical examination and patients' history. The Control Group could be arthrodesis, total disc arthroplasty, rehabilitation programme, nonoperative therapy, and others. We also considered rigid versus semirigid dynamic instrumentation techniques.

We were interested in the following clinical outcome measurements: disability (measured by Oswestry disability index - ODI - or other tests); SF-36 questionnaire; complications (perioperative and postoperative); pain intensity measured by Visual Analogue Scale (VAS); hospitalization data; patient overall assessment; any adverse events reported, cost (as a narrative description) and, others.

Methodological quality assessment

The methodological quality of the trials included in this review was judged using the Cochrane instrument approach recommended by the Cochrane Handbook5, since scales and checklists are not a reliable method for assessing the validity of a primary study6. The randomization methods, allocation concealment, blinded assessment of outcomes, intention-to-treat analysis, sample size calculations were recorded when it was available.

 

RESULTS AND DISCUSSION

Literature search results

We performed the search up to November 2008. The search strategy in the electronic databases identified 2,347 titles. After screening by title and then abstract, we obtained full paper copies for 58 potential studies that were potentially eligible for inclusion in the review. Of these, we identified 30 studies as case series, retrospective studies and controlled clinical trials that are presented as exclusion studies and their respectively main reasons of exclusion (Appendix 2). Thus, only one randomized controlled trial7 that fulfilled the inclusion criteria described above was included in this review.

Description of included studies and methodology quality assessment

Zucherman did not report how the generation of randomization, allocation concealment were, neither if there were a blinded assessment of outcomes measured and an intention-to-treat analysis. Besides that, the drop-outs and withdrawals were also not report. Thus, Zucherman was classified as B (unclear) regarding the internal validity of the study. This study had a follow-up of two years after the procedures7.

The description of data in the results sessions found in the included studies meant that it was not possible to combine studies in a meta-analysis. Therefore, we only described the main results reported by the authors and expressed some of them in a representation of meta-analysis found in the Appendix session.

Zucherman evaluated 191 patients with neurogenic intermittent claudication with or without back pain treated with interspinous process decompression system (X Stop) or nonoperative therapies at 9 US centers from May 2000 to July 20017.. The patients in the Control Group received at least one epidural steroid injections, nonsteroidal anti-inflammatory medications, analgesics, and physical therapy as necessary. The mean ages in the X Stop group was 70.0 years and in the Control Group was 69.1 years, without a statistically difference between them. The primary outcomes measured was the Zurich Claudication Questionnaire (ZCQ) and complications assessed intraoperatively and postoperatively.

There were more complications in the X Stop group compared with the Control Group. The complications were intraoperatively and included: respiratory distress, ischemic coronary episode, pulmonary edema, wound dehiscence and swelling, hematoma and incisional pain, although there was no statistical significant difference between both groups (Appendix 3).

Regarding the symptom severity domain, 60.2% patients reported a clinically significant improvement at the 2-year follow-up compared with 18.5% in the Control Group, with a statistical significant difference. A total of 57% of the patients in the X Stop group reported clinically improvement in the physical function domain compare with 14.8% in the Control Group, with also a statistical significant difference between both groups studied. Finally, 48.4% in the treatment group satisfied all the three ZCQ criteria (symptom severity, physical function and satisfaction) compared with only 4.9% in the Control Group.

The device (X Stop) was being evaluated as a part of ongoing FDA-approved investigation protocol (IDE) or corresponding national protocol for the X Stop at the time of the study (2005). Corporate/Industry founds were received in support of this work.

We planned to perform sensitivity analysis to explore methodological heterogeneity among the different types of studies (randomized and quasi-randomized clinical trials); however, there were no sufficient number of trials included on this review that allowed us to investigate causes of heterogeneity. Furthermore, we planned to perform subgroup analysis to explore clinical heterogeneity for the following subcategories: different types of clinical situation (degenerative disc, spinal stenosis and low back pain); different types of dynamic stabilization devices (Dynesis; BioFlex, etc.); different types of control groups; age (less than 18 years old versus 19 years or older), and significant medical comorbidity. Future studies should consider analysis of subgroups to better present their results avoiding bias and misunderstands.

Good clinical research aims to reduce uncertainty in order to help to make uniform clinical decisions, however the absence or poor evidence with regards all interventions proposed to be studied in this review corroborated with the findings of El Dib8, 2007 study in which 47.8% from a total of 1,016 Cochrane reviews analyzed are found to have insufficient evidence to answer the questions around therapeutic strategies for treatment and prevention of diseases, and the authors of them did ask for further research. This does not mean that the techniques, interventions or surgical procedures evaluated by these authors are not useful for the clinical practice; contrarily it means that more studies are needed to establish its real clinical efficacy.

Although there are some studies reporting the benefits of non-fusion fixation, there is only one study found in our review with adequate design to evaluate its effectiveness available in the literature. The results of this study showed that non-lumbar fusion techniques is a feasible way to treat some of the spinal disorder, however there were some complications related to the procedure.

 

CONCLUSIONS

Implications for practice

The data included in this review showed that the use of non-fusion fixation could be a suitable alternative to other therapies in patients with degenerative disc disease, spinal stenosis and low back pain.

Implications for research

This review highlighted the need for continued research into the use of non-fusion fixation in the treatment of spinal disorders. There is an urgency need to conduct randomized clinical trials, and long-term efficacy should be evaluated.

 

REFERÊNCIAS

1. Kim KA, McDonald M, Pik JH, Khoueir P, Wang MY. Dynamic intraspinous spacer technology for posterior stabilization: case-control study on the safety, sagittal angulation, and pain outcome at 1-year follow-up evaluation. Neurosurg Focus. 2007;22(1):E7.         [ Links ]

2. Freudiger S, Dubois G, Lorrain M. Dynamic neutralization of the lumbar spine confirmed on a new lumbar spine simulator in vitro. Arch Orthop Trauma Surg. 1999;119:127-32.         [ Links ]

3. Dubois B, de Germay B, Schaerer NS, Fennema P. Dynamic neutralization: a new concept for restabilization of the spine. In: Szalski M, Gunzburg R, Pope MH (Ed.) Lumbar Segmental Instability. Philadelphia, PA: Lippincott Williams & Wilkins; 1999. p. 233-40.         [ Links ]

4. Nockels RP. Dynamic stabilization in the surgical management of painful lumbar disorders. Spine. 2005;30(16 Suppl):S68-S72.         [ Links ]

5. Alderson P, Green S, Higgins JPT (Ed.) Cochrane Reviewers' Handbook 4.2.2. In: The Cochrane Library [updated December 2003]. Chichester, United Kingdom: John Wiley & Sons; 2003.         [ Links ]

6. Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;288(11):1054-60.         [ Links ]

7. Zucherman JF, Hsu KY, Hartjen CA, Mehalic TF, Implicito DA, Martin MJ, et al. A multicenter, prospective, randomized trial evaluating the X STOP interspinous process decompression system for the treatment of neurogenic intermittent claudication: two-year follow-up results. Spine (Phila Pa 1976). 2005;30(12):1351-8.         [ Links ]

8. El Dib RP, Atallah AN, Andriolo RB. Mapping the Cochrane evidence for decision making in health care. J Eval Clin Pract. 2007;13(4):689-92.         [ Links ]

9. Beastall J, Karadimas E, Siddiqui M, Nicol M, Hughes J, Smith F, et al. The Dynesys lumbar spinal stabilization system: a preliminary report on positional magnetic resonance imaging findings. Spine (Phila Pa 1976). 2007;32(6):685-90.         [ Links ]

10. Benezech J. Mitulescu A. Retrospective patient outcome evaluation after semi-rigid stabilization without fusion for degenerative lumbar instability. Eur J Orthop Surg Traumatol. 2007;17(3):227-34.         [ Links ]

11. Cakir B, Ulmar B, Koepp H, Huch K, Puhl W, Richter M. [Posterior dynamic stabilization as an alternative for dorso-ventral fusion in spinal stenosis with degenerative instability]. Z Orthop Ihre Grenzgeb. 2003;141(4):418-24.German.         [ Links ]

12. Cakir B, Richter M, Huch K, Puhl W, Schmidt R. Dynamic stabilization of the lumbar spine. Orthopedics. 2006;29(8):716-22.         [ Links ]

13. Frymoyer JW, Hanley EN Jr, Howe J, Kuhlmann D, Matteri RE. A comparison of radiographic findings in fusion and non-fusion patients ten or more years following lumbar disc surgery. Spine (Phila Pa 1976). 1979;4(5):435-40.         [ Links ]

14. Gardner A, Pande KC. Graf ligamentoplasty: a 7-year follow-up. Eur Spine J. 2002;11 Suppl 2:S157-63.         [ Links ]

15. Gertzbein SD, Betz R, Clements D, Errico T, Hammerberg K, Robbins S, et al. Semirigid instrumentation in the management of lumbar spinal conditions combined with circumferential fusion. A multicenter study. Spine (Phila Pa 1976). 1996; 21(16):1918-25.         [ Links ]

16. Guigui P, Chopin D. [Assessment of the use of the Graf ligamentoplasty in the surgical treatment of lumbar spinal stenosis. Apropos of a series of 26 patients]. Rev Chir Orthop Reparatrice Appar Mot. 1994;80(8):681-8. French.         [ Links ]

17. Grevitt MP, Gardner AD, Spilsbury J, Shackleford IM, Baskerville R, Pursell LM, et al. The Graf stabilisation system: early results in 50 patients. Eur Spine J. 1995;4(3):169-75.         [ Links ]

18. Grob D, Benini A, Junge A, Mannion AF. Clinical experience with the Dynesys semirigid fixation system for the lumbar spine: surgical and patient-oriented outcome in 50 cases after an average of 2 years. Spine (Phila Pa 1976). 2005;30(3):324-31.         [ Links ]

19. Hadlow SV, Fagan AB, Hillier TM, Fraser RD. The Graf ligamentoplasty procedure. Comparison with posterolateral fusion in the management of low back pain. Spine (Phila Pa 1976). 1988;23(10):1172-9.         [ Links ]

20. Hashimoto T, Oha F, Shigenobu K, Kanayama M, Harada M, Ohkoshi Y, et al. Mid-term clinical results of Graf stabilization for lumbar degenerative pathologies: a minimum 2-year follow-up. Spine J. 2001;1(4):283-9.         [ Links ]

21. Kanayama M, Hashimoto T, Shigenobu K, Togawa D, Oha F. A minimum 10-year follow-up of posterior dynamic stabilization using Graf artificial ligament. Spine (Phila Pa 1976). 2007;32(18):1992-6.         [ Links ]

22. Katz JN, Lipson SJ, Chang LC, Levine SA, Fossel AH, Liang MH. Seven- to 10-year outcome of decompressive surgery for degenerative lumbar spinal stenosis. Spine (Phila Pa 1976). 1996;21(1):92-8.         [ Links ]

23. Kim YS, Zhang HY, Moon BJ, Park KW, Ji KY, Lee WC, Oh KS, Ryu GU, Kim DH. Nitinol spring rod dynamic stabilization system and Nitinol memory loops in surgical treatment for lumbar disc disorders: short-term follow up. Neurosurg Focus. 2007;22(1):E10.         [ Links ]

24. Kong DS, Kim ES, Eoh W. One-year outcome evaluation after interspinous implantation for degenerative spinal stenosis with segmental instability. J Korean Med Sci. 2007;22(2):330-5.         [ Links ]

25. Lee SE. Park SB. Jahng TA. Chung CK, Kim HJ. Clinical experience of the dynamic stabilization system for the degenerative spine disease. J Korean Neurosurg Soc. 2008;43(5):221-6.         [ Links ]

26. Markwalder TM, Dubach R, Braun M. Soft system stabilization of the lumbar spine as an alternative surgical modality to lumbar arthrodesis in the facet syndrome. Preliminary results. Acta Neurochir (Wien). 1995;134 (1-2):1-4.         [ Links ]

27. Markwalder TM, Wenger M. Dynamic stabilization of lumbar motion segments by use of Graf's ligaments: results with an average follow-up of 7.4 years in 39 highly selected, consecutive patients. Acta Neurochir (Wien). 2003; 145(3):209-14.         [ Links ]

28. Putzier M, Schneider SV, Funk J, Perka C. [Application of a dynamic pedicle screw system (DYNESYS) for lumbar segmental degenerations - comparison of clinical and radiological results for different indications]. Z Orthop Ihre Grenzgeb. 2004;142(2):166-73.German.         [ Links ]

29. Putzier M, Schneider SV, Funk JF, Tohtz SW, Perka C. The surgical treatment of the lumbar disc prolapse: nucleotomy with additional transpedicular dynamic stabilization versus nucleotomy alone. Spine (Phila Pa 1976). 2005;30(5):E109-14.         [ Links ]

30. Saxler G, Wedemeyer C, von Knoch M, Render UM, Quint U. [Follow-up study after dynamic and static stabilisation of the lumbar spine]. Z Orthop Ihre Grenzgeb. 2005;143(1):92-9. German.         [ Links ]

31. Sapkas GS, Themistocleous GS, Mavrogenis AF, Benetos IS, Metaxas N, Papagelopoulos PJ. Stabilization of the lumbar spine using the dynamic neutralization system. Orthopedics. 2007;30(10):859-65.         [ Links ]

32. Schaeren S, Broger I, Jeanneret B. Minimum four-year follow-up of spinal stenosis with degenerative spondylolisthesis treated with decompression and dynamic stabilization. Spine (Phila Pa 1976). 2008;33(18):E636-42.         [ Links ]

33. Schnake KJ, Schaeren S, Jeanneret B. Dynamic stabilization in addition to decompression for lumbar spinal stenosis with degenerative spondylolisthesis. Spine (Phila Pa 1976). 2006;31(4):442-9.         [ Links ]

34. Sénégas J, Vital JM, Pointillart V, Mangione P. Long-term actuarial survivorship analysis of an interspinous stabilization system. Eur Spine J. 2007;16(8):1279-87.         [ Links ]

35. Stoll TM, Dubois G, Schwarzenbach O. The dynamic neutralization system for the spine: a multi-center study of a novel non-fusion system. Eur Spine J. 2002; Suppl 2:S170-8.         [ Links ]

36. Welch WC, Cheng BC, Awad TE, Davis R, Maxwell JH, Delamarter R, et al. Clinical outcomes of the Dynesys dynamic neutralization system: 1-year preliminary results. Neurosurg Focus. 2007;22(1):E8.         [ Links ]

37. Würgler-Hauri CC, Kalbarczyk A, Wiesli M, Landolt H, Fandino J. Dynamic neutralization of the lumbar spine after microsurgical decompression in acquired lumbar spinal stenosis and segmental instability. Spine (Phila Pa 1976). 2008;33(3):E66-72.         [ Links ]

 

 

Correspondence:
Gustavo Carriço de Oliveira
Centro de Ortopedia e Reabilitação
Rua Dom Joaquim, 885 - Centro
Florianópolis(SC), Brasil
E-mail: gustavocoluna@yahoo.com.br

Received: March 1, 2010
Accepted: June 4, 2010

 

 

Department of Orthopaedics and Spine Surgery at Hospital Homero de Miranda Gomes - São José (SC), Brazil; Hospital Mãe de Deus - Porto Alegre (RS), Brazil
No funding was received in support of this work. No benefits in any form have been or will be received from a commercial party related directly or indirectly to the subject of this manuscript.

 

 

Appendix

 

Appendix 1 - Click to enlarge

 

 

Appendix 2 - Click to enlarge

 

 

Appendix 3 - Click to enlarge

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License