Multi-view autostereoscopic system for 3D visualization in anatomy

Abstract

Introduction

The use of 3D imaging in the medical field has proven to be a benefit to doctors when diagnosing patients. As for different medical applications, 3D visualization systems have advantages in terms of a better spatial understanding of anatomical structures, better performance of tasks that require high level of dexterity, increased learning performance, and improved communications with patients or between doctors.

Methods

In this technical report, we show how to employ a multi-view autostereoscopic system to provide 3D images without any special glasses or equipment, describing a new way to obtain 3D visualization using sets of 2D images instead of real volumetric data such as magnetic resonance imaging (MRI) or computed tomography (CT). We also propose an application of the images in neuroanatomy.

Results

We obtained three-dimensional images of anatomical parts for visualization without glasses with resolution of 336 × 210 pixels’.

Conclusion

The proposed method was able to generate three-dimensional high-resolution images and has great potential to be used in various areas such as anatomy and physiological studies.

Keywords
3D medical imaging; 3D visualization systems; Multi-view autostereoscopic display; Anatomy; Neuroanatomy

Introduction

There are numerous techniques to create and display 3D moving pictures. The basic condition is to display 2D offset images that are filtered separately to the left and right eye. Two strategies have been used to achieve this by having the viewer wear eyeglasses to filter the separate offset images to each eye, or having the light source split the images directionally into the viewer's eyes (no glasses required). Most of the 3D display techniques for projecting stereoscopic image pairs to the viewer are in the filtering type ( Halldorsson and EADS Deutschland GmbH, 2002 ; Urey et al., 2011 ). Both 2D offset images are then combined in the viewer’s brain to generate the perception of 3D depth.

The 2D images in front of or at the back of the Holographic screens are used to obtain images outside of the screen plane ( Lunazzi et al., 2009 ) or to construct multi-view stereoscopy ( Magalhães et al., 2013 ), but some image formation systems offer limited field of view (FOV) and low light intensity due to the required double diffraction process ( Lunazzi, 1990 ). The optical principles of multi-view autostereoscopy have been known for over 60 years ( Okoshi, 1976 ), however, only become available recently due to high resolution displays. Autostereoscopic display techniques ( Dodgson, 2005 ; Zhang et al., 2010 ) used optical components on the display, rather than worn by the user, to enable each eye to see a different image.

The use of 3D imaging in the medical field has proven to be a benefit to doctors when diagnosing patients and 3D models of the human body have assisted medical manufacturers in developing better medical devices and treatments. In the different medical domains, such as radiology, minimally invasive surgery and teaching/training, a stereoscopic display has advantages ( Van Beurden et al., 2009 ) in terms of a better spatial understanding of anatomical structures, better perception of ambiguous anatomical structures ( Zonneveld and Fukuta, 1994 ), better performance of tasks that require high level of dexterity ( Liao et al., 2010 ), increased learning performance ( Owczarczyk and Owczarczyk, 1990 ), and improved communications with patients or between doctors.

Using anatomical parts fixed in formalin is widely accepted, however, the duration of anatomy courses has been decreased in the curriculum ( Aziz et al., 2002 ; Cahill et al., 2000 ; Drake et al., 2009 ; McLachlan et al., 2004 ), which has caused the limited access to cadavers in many medical schools around the world, for ethical or bureaucratic purposes. Evidence pointed out the importance of imaging techniques in student learning ( Marks, 2000 ), especially for the development of 3D perception of anatomical structures.

One disadvantage of the 3D visualization in the multi-view autostereoscopic system is that they require volumetric acquisition data such as magnetic resonance (MR) or CT which are expensive for regular usage. In previous works, we investigated how to use multi-view autostereoscopic systems to display volumetric images of MR and CT ( Magalhães et al., 2012b ) and MR angiography ( Magalhães et al., 2012a ), in this work we describe a way to obtain 3D visualization using sets of 2D images instead of real volumetric data allowing an inexpensive wide broad of applications.

Methods

Visualizing 3D images

In this section, we describe how we can visualize 3D anatomical images using sets of 2D images instead of volumetric data ( Magalhães et al., 2012a ; 2012b ). The multi-view autostereoscopic system provides a glasses-free tool for 3D medical imaging visualization. It consists of a multi-view autostereoscopic monitor (we employ Lumina 3D screens, 22 and 46 inches) connected with a computer by a high definition multimedia interface, as shown in Figure 1 . Lumina 3D screens utilize parallax barrier technology ( Hardesty, 2011 ; Sherriff, 2001 ) with brightness of 300 cd/m², contrast ratio 1000:1, pixel pitch of 0.265 × 0.265 mm. The observed image appears in a volume outside or inside the screen.

Figure 1
Multi-view autostereoscopic system is composed of computer (PC) and multi-view autostereoscopic monitor (AM). The viewer (Ob) observes a 3D image.

We used the software Visumotion 3D in a Windows based computer (Intel Core I7, 16Gb Ram) for the visualization. The optimal viewing occurs between 2 and 5 meters from the screen for the 22” screen and 2.5 to 6 m for the 46” screen. At smaller distances discomfort may be generated, whereas the 3D aspect of the images is not well perceived at greater distances.

Obtaining images

Here, we describe how we can obtain sets of 2D images to be processed as 3D images. For 3D visualization, we need to get multiple viewpoints, e. g., snapshots of the studied object from multiple points of view, in our case we used anatomical brain parts. The pictures were made by the rotation of the object in the axis of its center. For the rotation, we employed a simple rotation table covered by a thin layer of black opaque fabric. The fabric provides good image background contrast and protects the anatomic structures. Also, a photographic table was used under the rotation table, covered by the fabric for neutral background contrast. The object was illumined with two 60 W light bulbs allowing good brightness and contrast to the 2D images. A digital high definition camera model Nikon D750 24.3 MP equipped with macro lens Nikkor 18-105 mm focal length range was used for the sequence of quickly photos. The lens was utilized to achieve good resolution with zoomed image.

We used the neuro-anatomical parts of the Microsurgery laboratory of the School of Medical Sciences of the State University of Campinas as anatomical structure. An object (cerebellum) was positioned in the center of the table and a sequence of 10 snapshots were taken in 10 seconds corresponding to a rotation of the table (and consequently the cerebellum) of 10 degrees, which means a difference of 1 degree between each viewpoint ( $\delta$ ). We employed a graphic editor (Photoshop) to finish the background treatment in each image.

Processing images

With the set of images, we generated 3 × 3 matrix images. Every matrix I _α representing one set of five different viewpoints of the rotation object ( Figure 2 ). Each element of the matrix corresponds to a different viewpoint of the scene ( $a_{\alpha }$ ).

$I_{\alpha }=\left[\begin{array}{ccc}{a}_{\alpha }& a_{\alpha +\delta }& a_{\alpha +2\delta }\\ a_{\alpha +3\delta }& a_{\alpha +4\delta }& 0\\ 0& 0& 0\end{array}\right]$ , (1)

where: $\alpha$ is an angle between $0\le \alpha \le 2\pi$ and $\delta$ is the step angle between two consecutive snapshots, we choose $\delta =\pi /180$ , which means that $I_{\alpha }=I_{\alpha +2\pi }$ and we may obtain a continuous rotation of the image with 360 photos.

Figure 2
Construction of the matrix images.

Figure 3 shows one example of these matrices. We use five images because our screen only supports five points of view for each viewer.

Figure 3
Image matrix composed of five viewpoints of a cerebellum image.

We created hypertext preprocessor (PHP) algorithm to generate the matrices from the processed photos. The software adjusts the resolution of the images in such a way that each matrix results with 1680 × 1050 pixels, which is the maximum resolution supported by VisuMotion software ( VisuMotion, 2017 ), and every $a_{\alpha }$ with 560 × 350.

The multi-view autostereoscopic screen is driven by a software ( VisuMotion, 2017 ) that interpolates columns of pixels of each image $a_{\alpha }$ and a parallax barrier spatially directs the image to the viewer ( Dodgson, 2005 ) creating three points for stereoscopic vision in which each observer can see the image in 3D with parallax.

Results

In the typical neuro-anatomical visualization with the standard computer monitors, the lack of real depth of the image could cause misunderstanding of anatomical structures. The utilization of a multi-view autostereoscopic system, which can produce real depth in visualization without the utilization of any special device such as glasses or other equipment, have advantages in terms of a better spatial understanding. Based on the 3D visualization system, we are producing a guide for 3D neuro-anatomical dissection that will be used in the neuro-anatomy courses.

The multi-autostereoscopic monitor has a resolution of 1680 × 1050 allowing 5 views of 336 × 210 pixels’, which means a horizontal resolution of 336 pixels for 3D visualization.

To show the neuro-anatomical structures, we presented two stereoscopic pairs showing the angular difference between shoots. We used a digital camera and we rotated it with the angle described in each caption.

An overview of some of the 3D images obtained with the multi-autostereoscopic monitor is in Figure 4 .

Figure 4
Some samples of 3D images of anatomical structures are visualized with the multi-view autostereoscopic monitor.

Discussion

We could not perceive distortions in 3D visualization, however when the observer is too close to the monitor (less than 1.2 m) discomfort when focusing the eyes is felt. The image can be viewed by multiple viewers with high quality and no difficulty in focusing eyes. The viewers can perceive better depth between 2 to 4 meters from the monitor.

The limitation we measure in FOV was 55 degrees from the normal angle of the monitor to each size with a total FOV angle of 110 degrees. For a 70±5 mm distance between each eye, the limit an observer can perceive depth is from 1.9±0.2 m to 5.1±0.3 m from the monitor and this perception of depth is reduced with the increase in distance from the monitor. Farther than 5.5 m the image is still visible but due to long distance, depth is not appreciated. The magnification of the visualized 3D images is controlled when generating the image matrixes ( Figure 3 ), in the images presented in Figures 4 , 5 and 6 the magnification observed in the 42 inches monitor was of 7x. This magnification is limited only for the camera lens employed.

Figure 5
A cerebellum is shown; the images appear rotated of 4 degrees from one to another.

Figure 6
The lateral ventricle of the brain is shown; the images appear rotated of 5 degrees from one to another.

When we choose 2 or 3 degrees instead of 1 degree between the viewpoints, e.g. decreasing the frame rate of the sequence of photo shoots (from 1 s^-1 to 0.3 s^-1), the image results more impressive because it appears furthest from the monitor, however the visualization is less comfortable for the viewer. When the angles between each of the stereoscopic pairs are bigger; the image appears more off the screen, but the big difference causes fatigue in viewing.

Because the resolution of 3D visualization is limited by the monitor, with an increase of resolution we could have greater 3D resolution for the images. Since the visualization software does not compress the 2D images, with the same set of images, we do not expect improvement in resolution however we observed loss of resolution in visualization with a computer with system requirements lower than recommended by ( VisuMotion, 2017 ).

This kind of 3D image will help physicians and medical students to interpret the anatomy with 3D visualization. The image gives a better spatial understanding of the anatomical structures and favors a better perception of ambiguous anatomical structures. A future study will evaluate student’s perception and learning with the 3D tools presented in this report.

In conclusion, in this paper we presented and described a method to obtain 3D visualization of anatomical images using sets of 2D images instead of real volumetric data. We show the first results of multi-view autostereoscopic system for 3D neuro-anatomical imaging and show the possibility of applying a multi-autostereoscopic system into generating 3D anatomical guides or atlas.

Acknowledgements

We thank FAPESP for financial support (grants 2013/13272-9, 2014/07501-8 and 2015/50304-1). We also thank Alline Camargo for handling the background treatment in the pictures.

• How to cite this article: Magalhães DSF, Mansoor S, Weng Y, Ghizoni E, Barbosa T, Silveira FA, Toledo RS, Li LM. Multi-view autostereoscopic system for 3D visualization in anatomy. Res Biomed Eng. 2018; 34(3):xx-xx. DOI: 10.1590/2446-4740.180066.

References

• Aziz MA, McKenzie JC, Wilson JS, Cowie RJ, Ayeni SA, Dunn BK. The human cadaver in the age of biomedical informatics. Anat Rec. 2002; 269(1):20-32. http://dx.doi.org/10.1002/ar.10046. PMid:11891622.
  » http://dx.doi.org/10.1002/ar.10046
• Cahill DR, Leonard RJ, Marks SC Jr. A comment on recent teaching of human anatomy in the United States. Surg Radiol Anat. 2000; 22(2):69-71. http://dx.doi.org/10.1007/s00276-000-0069-4. PMid:10959670.
  » http://dx.doi.org/10.1007/s00276-000-0069-4
• Dodgson NA. Autostereoscopic 3D displays. IEEE Computer Society. 2005; 38(8):31-6. http://dx.doi.org/10.1109/MC.2005.252.
  » http://dx.doi.org/10.1109/MC.2005.252
• Drake RL, McBride JM, Lachman N, Pawlina W. Medical education in the anatomical sciences: the winds of change continue to blow. Anat Sci Educ. 2009; 2(6):253-9. http://dx.doi.org/10.1002/ase.117. PMid:19890982.
  » http://dx.doi.org/10.1002/ase.117
• Halldorsson T. inventor; EADS Deutschland GmbH, assignee. System and method for stereoscoping imaging. United States patent US Patent 6412949. 2002.
• Hardesty L. Better glasses-free 3-D: a fundamentally new approach to glasses-free 3-D displays could save power, widen the viewing angle and make 3-D illusions more realistic [internet]. Massachusetts: MIT News Office; 2011. [cited 2017 Feb 20]. Available from: http://news.mit.edu/2011/glasses-free-3d-0504
  » http://news.mit.edu/2011/glasses-free-3d-0504
• Liao H, Ishihara H, Tran HH, Masamune K, Sakuma I, Dohi T. Precision-guided surgical navigation system using laser guidance and 3D autostereoscopic image overlay. Comput Med Imaging Graph. 2010; 34(1):46-54. http://dx.doi.org/10.1016/j.compmedimag.2009.07.003. PMid:19674871.
  » http://dx.doi.org/10.1016/j.compmedimag.2009.07.003
• Lunazzi JJ, Magalhães DS, Rivera NI, Serra RL. Holo-television system with a single plane. Opt Lett. 2009; 34(4):533-5. http://dx.doi.org/10.1364/OL.34.000533. PMid:19373365.
  » http://dx.doi.org/10.1364/OL.34.000533
• Lunazzi JJ. 3D photography by holography. Opt Eng. 1990; 29(1):9-14. http://dx.doi.org/10.1117/12.55558.
  » http://dx.doi.org/10.1117/12.55558
• Magalhães DSF, Li LM, Serra RL. inventors; Unicamp, assignee. Multi-stereoscopic holographic system for visualization of medical images of patient using e.g. MRI technology. Patent BR201102733. 2013.
• Magalhães DSF, Ribeiro FH, Lima FO, Serra RL, Moreno AB, Li LM. 3D brain MR angiography displayed by a multi-autostereoscopic screen. In: Proceedings of SPIE Medical Imaging; 2012; San Diego. Washington: International Society for Optical Engineering; 2012a.
• Magalhães DSF, Serra RL, Vannucci AL, Moreno AB, Li LM. Glasses-free 3D viewing systems for medical imaging. Opt Laser Technol. 2012b; 44(3):650-5. http://dx.doi.org/10.1016/j.optlastec.2011.09.015.
  » http://dx.doi.org/10.1016/j.optlastec.2011.09.015
• Marks SC Jr. The role of three-dimensional information in health care and medical education: The implications for anatomy and dissection. Clin Anat. 2000; 13(6):448-52. http://dx.doi.org/10.1002/1098-2353(2000)13:6<448::AID-CA10>3.0.CO;2-U. PMid:11111898.
  » http://dx.doi.org/10.1002/1098-2353(2000)13:6<448::AID-CA10>3.0.CO;2-U
• McLachlan JC, Bligh J, Bradley P, Searle J. Teaching anatomy without cadavers. Med Educ. 2004; 38(4):418-24. http://dx.doi.org/10.1046/j.1365-2923.2004.01795.x. PMid:15025643.
  » http://dx.doi.org/10.1046/j.1365-2923.2004.01795.x
• Okoshi T. Three-dimensional imaging techniques. New York: Academic Press; 1976.
• Owczarczyk J, Owczarczyk B. Evaluation of true 3D display systems for visualizing medical volume data. Vis Comput. 1990; 6(4):219-26. http://dx.doi.org/10.1007/BF02341046.
  » http://dx.doi.org/10.1007/BF02341046
• Sherriff L. Sharp's 3D LCD: how's that work, then? LCD sandwiches and parallax barriers [internet]. The Register; 2001. [cited 2018 Aug 23]. Available from: https://www.theregister.co.uk/2004/08/12/3d_illusion
  » https://www.theregister.co.uk/2004/08/12/3d_illusion
• Urey H, Chellappan KV, Erden E, Surman P. State of the art in stereoscopic and autostereoscopic displays. Proc IEEE. 2011; 99(4):540-55. http://dx.doi.org/10.1109/JPROC.2010.2098351.
  » http://dx.doi.org/10.1109/JPROC.2010.2098351
• Van Beurden MHPH, Van Hoey G, Hatzakis H, Ijsselsteijn WA. Stereoscopic displays in medical domains: a review of perception and performance effects. In: Proceedings of the XIV Human Vision and Electronic Imaging; 2009; San Jose, California. Washington: SPIE; 2009. http://dx.doi.org/10.1117/12.817748.
  » http://dx.doi.org/10.1117/12.817748
• VisuMotion. VisuMotion 3D Movie Center v1.6. Visualization software [internet]. 2017. [cited 2017 Mar 16]. Available from: http://www.visumotion.com
  » http://www.visumotion.com
• Zhang Y, Ji Q, Zhang W. Multi-view autostereoscopic 3D display. In: Proceedings of the Optics Photonics and Energy Engineering (OPEE) International Conference; 2010; China. USA: IEEE; 2010. p. 58-61.
• Zonneveld FW, Fukuta K. A decade of clinical three-dimensional imaging: a review part II: clinical applications. Invest Radiol. 1994; 29(5):574-89. http://dx.doi.org/10.1097/00004424-199405000-00010. PMid:8077099.
  » http://dx.doi.org/10.1097/00004424-199405000-00010

Publication Dates

• Publication in this collection
  Jul-Sep 2018
• Date of issue
  Sept 2018

History

• Received
  24 Aug 2018
• Accepted
  12 Sept 2018