Seeking tools for image fusion between computed tomography, structural and functional magnetic resonance methods for applications in neurosurgery

Abstracts

OBJETIVO: Avaliar ferramentas de fusão de imagens geradas por tomografia, ressonância magnética estrutural e funcional. MÉTODOS: Foram realizados exames de ressonância magnética e ressonância magnética funcional com paradigmas motor e somatossensitivo em aparelho de 3.0 Tesla em voluntário com tomografia de crânio prévia. Os dados de imagem foram analisados por diferentes programas e os resultados, comparados. RESULTADOS: Determinamos um fluxograma de processos computacionais que permitiram mensurar a congruência espacial entre as modalidades. Não foi encontrada ferramenta computacional que, isoladamente, permitisse todo o conjunto de funcionalidades necessárias para atingir o objetivo. CONCLUSÃO: O processo de fusão das três modalidades mostrou-se viável com a utilização de quatro softwares de acesso gratuito (OsiriX, Register, MRIcro e FSL) e pode servir de base para a construção de um software próprio para uso como ferramenta virtual pré-neurocirúrgica.

Imagem por ressonância magnética; Tomografia computadorizada por raios X; Processamento de imagem assistida por computador; Neurocirurgia

ORIGINAL ARTICLE

Seeking tools for image fusion between computed tomography, structural and functional magnetic resonance methods for applications in neurosurgery

Liana Guerra Sanches da Rocha^I; Edson Amaro Junior^II

^IDepartment of Diagnostic Imaging, Hospital Israelita Albert Einstein – HIAE, São Paulo (SP), Brazil

^IIFaculdade de Medicina, Universidade de São Paulo – USP, São Paulo(SP), Brazil; Department of Diagnostic Imaging and Instituto do Cérebro – InCe, Hospital Israelita Albert Einstein – HIAE, São Paulo (SP), Brazil

^{Corresponding author}

ABSTRACT

OBJECTIVE: To evaluate tools for the fusion of images generated by tomography and structural and functional magnetic resonance imaging.

METHODS: Magnetic resonance and functional magnetic resonance imaging were performed while a volunteer who had previously undergone cranial tomography performed motor and somatosensory tasks in a 3-Tesla scanner. Image data were analyzed with different programs, and the results were compared.

RESULTS: We constructed a flow chart of computational processes that allowed measurement of the spatial congruence between the methods. There was no single computational tool that contained the entire set of functions necessary to achieve the goal.

CONCLUSION: The fusion of the images from the three methods proved to be feasible with the use of four free-access software programs (OsiriX, Register, MRIcro and FSL). Our results may serve as a basis for building software that will be useful as a virtual tool prior to neurosurgery.

Keywords: Magnetic resonance imaging; Tomography, x-ray computed; Image processing, computer-assisted; Neurosurgery

INTRODUCTION

Diagnostic imaging tests are an important part of the pre-surgical routine in many specialties, including neurosurgery. Technological advances in this area permit the collection of data that display anatomical structures and lesions of these structures with high spatial resolution and high contrast definition.

Computerized tomography (CT) and magnetic resonance imaging (MRI) are different methods of acquiring images that reveal distinct characteristics of the structures imaged. CT uses the physical principle of the attenuation of x-ray beams (XR)⁽¹⁾ as they pass through structures of different densities, after which the beams are captured using radiation-sensitive detectors. Thus, dense structures such as bone, which produces an extreme degree of attenuation, contrast with structures such as cerebrospinal fluid (CSF), which produce much lower attenuation. MRI uses signals from the spins of the nuclei of hydrogen protons after they are aligned in a constant magnetic field. Radio frequency (RF) waves provide energy that perturbs the alignment of the nuclei, but the nuclei realign in the magnetic field in a phenomenon called "relaxation". As the nuclei relax, they release the absorbed energy, which is detected by electromagnetic coils⁽²⁾. The rate of relaxation is examined because it varies between structures high in hydrogen (e.g., CSF) and low in hydrogen (e.g., bone).

Functional MRI (fMRI) is an indirect method of measuring neuronal activity through differences in the oxygen concentration of arterial and venous blood. This endogenous contrast is termed the Blood Oxygen Level Dependent (BOLD) signal^(3,4). The difference in the magnetic properties of oxy- and deoxyhemoglobin are detected by the MRI equipment and analyzed by statistical methods that result in estimations of the probability that a certain brain region is involved in the execution of the proposed task (paradigm).

The fusion of images from two imaging methods, such as PET and CT, SPECT and CT, is widespread. The functional information so obtained is aligned with and superimposed on the high-resolution images, allowing the surgeon to plan the procedure in more detail and avoid injuries to delicate structures. The neurosurgeon uses CT images to visualize the key points of access to the cranium and the optimal spatial resolution of MRI images to differentiate encephalic structures. Adding functional information from fMRI and the associated structural information gathered during an MRI session yields better spatial and temporal resolution than other functional techniques such as SPECT.

While seeking multi-method fusion tools, we realized that the available studies focus on merging the images of only two specific modalities, for example, the fusion of CT and MRI images for use with surgical neuronavigators⁽⁴⁾. In the present study, we propose adding images from a third method, such as fMRI, to the fusion of CT and MRI. To accomplish this addition, we looked for software on the world wide web and in the market that supports the transformation of more than one image or allows the processing of different formats.

OBJECTIVE

To evaluate tools for the fusion of images from tomography and structural and functional magnetic resonance imaging.

METHODS

MRI and fMRI exams were performed as diagnostic tests. A volunteer who had had a recent cranial CT in the same service was selected for the study and signed an informed consent form. Data were collected at the Department of Diagnostic and Preventive Medicine of the Hospital Israelita Albert Einstein (HIAE). The protocol was approved by the ethics committee of the HIAE.

CT images were obtained with an Aquilion 64 multidetector (Toshiba Medical, Japan) with 64 detector rows and a protocol for isotropic voxels (0.5- mm cuts and 0.3-mm increments, 0.75mm pitch, 240mm FOV, 120kV and 225 mAs). The images were sent to the PACS (Picture Archiving and Communication System) of Carestream Health (Canada).

The MRI images were acquired using a Magneton TIM Trio system (Siemens Medical, Germany) with a 3-Tesla main field and a 50mT/m gradient (slew rate 150mT/m/s). A dedicated head matrix cranial coil with 12 elements was used. The parameters of the structural sequence were the following: prospective acquisition correction (PACE) with sufficient sections to cover the entire encephalus in sagittal, 1.1mm thickness with 50% increment, 240mm FOV, 2300 repetition time (TR), 2.98 echo time (TE), and 900 ms inversion time (IT) in a 256x230 matrix.

For detection of the BOLD signal, a gradient echo (GRE) echo planar imaging (EPI) sequence was used. There were 32 isotropic voxels, and slices were acquired in the plane of the anterior and posterior commissures (3.5mm thickness, 10% increment, 64x64 matrix, 210mm FOV, 2000ms TR, and 30ms TE). The behavioral paradigms performed by the subject included a finger tapping motor task (the thumb and other fingers were held together and moved apart repeatedly, guided by auditory signals to MOVE and STOP, for 20 seconds followed by 20 seconds of rest) and a somatosensitive task (the finger pads were stimulated by a soft object for 20 seconds, alternating with 20 seconds of rest).

Images acquired in the MRI session were sent via internal network (PACS) to workstation 1, where the CT images were stored. Images in the Digital Imaging and Communications in Medicine (DICOM) format were selected and converted to the Analyze (Mayo Clinic, Rochester, USA) format for use in processing and image fusion programs. This conversion was performed in the MRIcro software (http://www.sph.sc.edu/comd/rorden/mricro.html).

At workstation 2, which consisted of a computer running the Macintosh operating system, statistical analysis of the fMRI data was performed with the FSL software (Analysis Group, FMRIB Center, Oxford University, UK).

Research on image fusion algorithms was performed via searches of bibliographic sources in PubMed, using the term "fusion" associated with "imaging", "MRI" and "CT". The results were technical articles on the construction and improvement of software for use in academic development.

RESULTS

The research found the following four open-access software programs that perform CT and MRI image fusion and can export the results in a format that allows interaction with other software programs. These programs were "CAT3D"^(5), "3D Slicer"⁽⁶⁾, OsiriX⁽⁷⁾ and Register (Adam Guthrie, Montreal Neurological Institute http://noodles.bic.mni.mcgill.ca/ServicesSoftwareVisualizationTools/HomePage. The CAT3D software was developed for use in image fusion; it is used in radiotherapy and was adapted for the purposes proposed herein. Modifications were made to the CAT3D software so that fMRI image formats could be accepted. We chose four reference points in the CT and MRI images as markers that allowed the generation of information that enabled automatic fusion processing. After manually marking these points, the software performed an automatic adjustment based on the mutual information algorithm. There was good congruence with the marker points and external edges of the CT/MRI images (Figure 1). We also tested fMRI image fusion based on the Analyze data format. In this format, activation maps are not shown; instead, the acquired EPI images are added to the signal fluctuation information of each voxel. Because this is a rough spatial matrix image, the fusion congruence of the fine matrix with other methods lost statistical value. Additionally, 3D imaging was not performed in this software; thus, this format only allowed the display of points in images merged between two imaging methods, i.e., CT-MRI, MRI-fMRI and CT-fMRI. It was not possible to show results of the three methods together.

The Slicer software proved to be a friendlier image fusion tool. This software features four display windows that show three orthogonal planes and a display window that allows free rotation between planes, thus allowing a complete view of any selectable section.

The fusion was performed manually using graphic resources to assist in visually adjusted correct placement of the edges. Although it should have been possible to perform automatic alignment after this step, the software presented an error on all attempts. The large number of images per volume produced by CT and MRI in relation to the software version is the likely cause of this problem.

The Register software was chosen to register the CT and MRI images. Register is an algorithm that is part of the Brain Imaging Software Toolbox of the McConnell Brain Imaging Centre (Montreal Neurological Institute, Canada) and is available at no cost on the World Wide Web. The available tutorials on the software's website provided the necessary support for learning its techniques and applicability.

To load the images in the Register software, isotropic linear reduction of the dimensions of the CT and MRI volumes was necessary because the algorithm did not support images with voxel number greater than 256x256x256. This reduction was performed with a simple algorithm in the MRIcro software. For isotropic volumes in the Analyze data format, conversion to the native format used by the Register software (.mnc) was necessary. This conversion occurred in two steps: the FSL (package used in the analysis of functional images) fslchfiletype function was first used to convert the images to the NIFTI format (.nii); this format was then converted by means of an algorithm (nii2mnc) f to the.mnc format in the Register package.

The volumes were loaded in the platform, and fusion began with the deposition of congruence points in each of the two acquisitions beginning with the right and left crystallines, the right and left internal auditory canals, the sella turcica, the central point of the occipital bone, the frontal sinus and the most prominent point of the nose. This arrangement was chosen for, and proved to be effective in, the selected type of conversion (full affine 12 parameter). Thereafter, it was possible to use additional points to reach an optimal setting.

The result of this procedure is an adjusted volume that allows the viewer to change the transparency of the images from each method for crosschecking. The whole volume was verified, and the registration of the tomography and resonance images (Figures 2 and 3) was confirmed. This new volume was saved in the format provided by the software (.mnc). Again, commands were used to return the volume to the Analyze format because this is a widely accepted image visualization software. The resulting image was the registration of one of the methods (source image) in the three-dimensional space of the other (target image). An MRI image was chosen as a source image and was spatially converted to correspond to the CT image.

The volume resulting from the fusion was called "volumeregis". The volumeregis was used with functional data analysis with FSL so that the functional activation data could be automatically registered. This registration did not occur on a template⁽⁸⁾; rather, it occurred with the individual anatomy of the subject. For this purpose, extraction of tissues was performed with the Brain Extraction Tool (BET)⁽⁹⁾. Those tissues were not part of the encephalon. Then, recordings based in linear transformations were performed as previously described.

After statistical analyses of the fMRI images and their anatomical superposition onto the volumeregis, the visualization function of the FSL package was used (FSL view) to check the laterality and format of the images and to save them for the next step.

The OsiriX software (Osirix Medical Imaging Software, USA) was chosen for multi-method visualization and the addition of fMRI data to enable the manipulation of 3D and 4D images⁽¹⁰⁾ (Figure 4). For the three-dimensional presentation of the fused images, we first loaded the CT volume and volumeregis in two software windows. Using the point-based fusion tool, the volumes were fused after they were registered with the Register software. However, when the move to 3D MIP was requested, the software presented the images in their pre-fusion volumes, in parallel, on the screen. Due to this limitation, we chose an individual presentation of the modalities for easier navigation. Thus, the CT volumes and the volumeregis were loaded separately in the 3D option; this procedure permitted their visualization at various levels and allowed cuts in the anatomy (Figure 5).

The final processing model, including the software programs used and their functions, is shown in figure 6.

DISCUSSION

The fusion of three image modalities was the purpose of this study. We sought to add fMRI information to anatomical CT and MRI data. In our survey of the literature, we found no studies that described specific methods to accomplish this goal. One of our main motivations was the possibility of joining the technological benefits of using multiple imaging techniques into a single platform with practical applications in the clinic. While performing this work and overcoming the challenges it presented, our choice of solutions was guided by practicality and the possibility of routine implementation.

Most of the time invested in this research was occupied by the search for one or more free-access software packages that met our proposed goals. We sought software that was easy to manage so that the flow of processing could be managed by many types of professionals, independent of their knowledge of information technology and programming. The researchers acquired knowledge by reading articles about image fusion and the manuals of the various software packages researched. In this way, the capacity of the software programs to provide the type of interaction needed could be realistically assessed.

The research described here and its final processing model took into account the accessibility of the platform for professionals with no specialized training in computers. However, use of the methods proposed here in clinical practice may require the presence of a professional in the bioinformatics field with programming skills to positively complement the work of medical staff and add value to the techniques they use.

The fusion software packages currently in use by the Department of Radiotherapy and the HIAE surgical center neuronavigator were evaluated for implementation in this work. Although these programs produce excellent fusions of CT and MRI images, they do not export the final recorded volume, or if they do, they do so in a format that is not compatible with widely used platforms of medical image analysis. Thus, these software programs do not provide the possibility of fusion with fMRI images.

The use of high magnetic field (1.5 T) MRI equipment has allowed the advancement of fMRI studies in recent years. Studies of the application of the results to clinical practice, however, have shown that there is a discrepancy of approximately 20% when the location of fMRI activity is compared with that of intra-operative electrocortical stimulation⁽¹¹⁾. The greater availability of equipment with 3 T fields has made possible new advances that utilize the advantages of doubling the magnetic field to improve these results. The increased signal-to-noise ratios of these high-field scanners permits the application of more accurate acquisition protocols that reduce the thickness of the slices, increase the size of the matrix and generate images with better spatial resolution. These parameters have also contributed to the decrease of magnetic susceptibility artifacts, which are very common with echo-planar acquisitions in 3 T fMRI.

Lower TE values contribute to lower acquisition times that increase temporal resolution. While maintaining the same TR, which regulates the temporal resolution) used in 1.5 T scanners, the statistical power of the sample in conjunction with other parameters is increased, thereby reducing the likelihood of false positives. The low TEs also decrease artifacts from the skull base without jeopardizing the statistical fMRI measurements from adjacent structures such as the cerebellum⁽¹²⁾. In addition to the cerebellum, areas with BOLD effects in the motor and somatosensory cortex were observed⁽¹³⁾ that promoted the accuracy of the performed movement and improved performance from the resulting motor action data.

In structural images, the 3 T field contributes to increased signal-to-noise ratios for the acquisition of brain volumes in isotropic voxels. Images constructed from 1-mm³ isotropic voxels enable three-dimensional reconstructions that allow identification of anatomical characteristics. Coverage of the entire brain at this high level of spatial resolution makes it possible to investigate anatomical variations and relationships between structures, information that is key to neurosurgeons.

The method described in this paper allows the use of images that would normally be collected in the pre-surgery routine of an individual. CT and MRI are imaging methods that are used in most large neurosurgical centers, and, if the equipment provides sufficient spatial resolution, the protocols need not be changed for the application of this technique. Once the rotations are corrected for the interpretation of the information to be synthesized at a single point, the data can be used for co-registration.

To facilitate the fusion of images using the methods described here, image transfer networks can be replaced with external drives, and workstations can be adapted. In addition to possessing better processing power and a better graphics platform, the Macintosh operating system (Apple, USA) is the only system that allows installation of the OsiriX software and the complete conversion package of the Register software. Register has proven easier to use and more intuitive than other programs. Register also provides faster responses to the registration actions and a solution to the problem of exporting the image formed after fusion in a format acceptable to other programs used in this work.

The incorporation of several software packages, even if optimized, is not optimal for routine use. We believe that the evolution of this system will involve the creation of a unique software program that has all of the necessary tools for conversion, merging and 3D processing. While opportunities to improve the process clearly exist, the results obtained here can now be used for research purposes.

The fusion model proposed in this paper follows the trend of the incorporation of diagnostic imaging in neurosurgical techniques, which enables the surgeon, through interaction with the tool, to visualize key neurosurgical features⁽¹⁴⁾ of the patient prior to surgery. The evaluation of the location of key neurosurgical points on the skull surface described by studies on cadavers⁽¹⁵⁾, their average distances in relation to the skull cap (via CT), and the adjacent sulci and gyri (via MRI), were compatible using this model⁽¹⁶⁾. Furthermore, the performance of fMRI using specific paradigms to study target brain areas adjacent to a lesion in a given individual is important because anatomical and functional characteristics have extensive inter-subject variability.

CONCLUSION

The integration of data from three imaging methods (CT, MRI, and fMRI) in a computer platform was possible based on images routinely acquired prior to neurosurgery. These data demonstrate the functional and structural aspects of the subject's brain. In the future, consideration should be given to the fusion imaging process as a virtual tool in neurosurgical planning. This process may lead to the development of appropriate software based on the needs of individual institutions.

ACKNOWLEDGEMENTS

We would like to thank Professor Dr. Guilherme Carvalhal Ribas for support and encouragement and the Instituto do Cérebro (InCe) and Instituto Israelita de Ensino e Pesquisa Albert Einstein (IIEPAE) of Hospital Israelita Albert Einstein (HIAE) for the opportunity to conduct this study. We would also like to express our appreciation to the staff of the Department of Magnetic Resonance at HIAE, the Department of Neurosciences and Behavior at the Instituto de Psicologia da Universidade de São Paulo and my advisor, Professor Dr. Edson Amaro Junior.

This work was supported in part by awards from the Research Support Program at the Hospital Israelita Albert Einstein, 2008.

Received on: Mar 19, 2012

Accepted on: May 14, 2012

Conflict of interest: none.

Study carried out at the Instituto do Cerebro, Hospital Israelita Albert Einstein – HIAE, São Paulo (SP), Brazil.

• 1. Amaro Junior E, Yamashita H. Aspectos básicos de tomografia computadorizada e ressonância magnética. Rev Bras Psiquiatr. 2001;23(Supl 1):2-3.
• 2. Westbrook C, Kaut C. Ressonância magnética prática. 2a ed. Mundim FD, tradutor. Rio de Janeiro: Guanabara Koogan; 2000.
• 3. Ogawa S, Lee TM, Kay AR, Tank DW. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc Natl Acd Sci U S A. 1990; 87(24):9868-72.
• 4. Matthews PM, Honey GD, Bullmore ET. Applications of fMRI in translational medicine and clinical practice. Nat Rev Neurosci. 2006;7(9):732-44.
• 5. Bouza AL. Desenvolvimento de uma técnica para fusão de imagens como complemento ao planejamento cirúrgico em condições estereotáxicas [dissertação]. São Paulo: Universidade Federal de São Paulo, Escola Paulista de Medicina; 1999.
• 6. Pace D, Hata N. Image guided therapy in Slicer3: planning for image guided neurosurgery [Internet]. 2008 [cited 2012 Apr 4]. Available from: http://www.slicer.org/publications/item/view/1608
• 7. Rosset A, Spadola L, Ratib O. OsiriX: an open-source software for navigating in multidimensional DICOM images. J Digit Imaging. 2004;17(3):205-16.
• 8. Talairach J, Tournoux P, Musolino A. Anatomical stereotaxic studies of the frontal-lobe in the management of the epilepsies. Epilepsia. 1988;29(2):205.
• 9. Smith SM. Fast robust automated brain extraction. Hum Brain Mapp. 2002; 17(3):143-55.
• 10. Rousset A, Spadola L, Pysher L, Ratib O. Informatics in radiology (infoRAD): navigating the fifth dimension: innovative interface for multidimensional multimodality image navigation. RadioGraphics. 2006;26(1):299-308.
• 11. Fandino J, Kollias SS, Wieser HG, Valavanis A, Yonekawa Y. Intraoperative validation of functional magnetic resonance imaging and cortical reorganization patterns in patients with brain tumors involving the primary motor cortex. J Neurosurg. 1999;91(2):238-50.
• 12. Schmitz BL, Aschoff AJ, Hoffmann MH, Grön G. Advantages and pitfalls in 3T MR brain imaging: a pictorial review. AJNR Am J Neuroradiol. 2005; 26(9):2229-37.
• 13. Kandel ER, Schwartz JH, Jessel T. Essentials of neural science and behaviour. EUA: Appleton & Lange, 1995.
• 14. Ribas GC, Yasuda A, Ribas EC, Nishikuni K, Rodrigues AJ Jr. Surgical anatomy of microneurosurgical sulcal key points. Neurosurgery. 2006;59(4 Suppl 2): ONS 177-210; discussion ONS210-1.
• 15. Ribas GC. Microanatomia cirúrgica dos pontos-chave dos sulcos e giros cerebrais [tese]. São Paulo: Universidade de São Paulo, Faculdade de Medicina. São Paulo; 2005.
• 16. Rocha, LG. Relação dos pontos-chave cirúrgicos no crânio com áreas eloquentes detectadas por ressonância magnética funcional [dissertação]. São Paulo: Universidade de São Paulo, Instituto de Psicologia; 2009.

Publication Dates

• Publication in this collection
  20 Sept 2012
• Date of issue
  June 2012

History

• Received
  19 Mar 2012
• Accepted
  14 May 2012