Change search
ReferencesLink to record
Permanent link

Direct link
Guidance and Visualization for Brain Tumor Surgery
Linköping University, Department of Medical and Health Sciences, Division of Radiological Sciences. Linköping University, Center for Medical Image Science and Visualization (CMIV). Linköping University, Faculty of Science & Engineering.
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Image guidance and visualization play an important role in modern surgery to help surgeons perform their surgical procedures. Here, the focus is on neurosurgery applications, in particular brain tumor surgery where a craniotomy (opening of the skull) is performed to access directly the brain region to be treated. In this type of surgery, once the skull is opened the brain can change its shape, and this deformation is known as brain shift. Moreover, the boundaries of many types of tumors are difficult to identify by the naked eye from healthy tissue. The main goal of this work was to study and develop image guidance and visualization methods for tumor surgery in order to overcome the problems faced in this type of surgery.

Due to brain shift the magnetic resonance dataset acquired before the operation (preoperatively) no longer corresponds to the anatomy of the patient during the operation (intraoperatively). For this reason, in this work methods were studied and developed to compensate for this deformation. To guide the deformation methods, information of the superficial vessel centerlines of the brain was used. A method for accurate (approximately 1 mm) reconstruction of the vessel centerlines using a multiview camera system was developed. It uses geometrical constraints, relaxation labeling, thin plate spline filtering and finally mean shift to find the correct correspondences between the camera images.

A complete non-rigid deformation pipeline was initially proposed and evaluated with an animal model. From these experiments it was observed that although the traditional non-rigid registration methods (in our case coherent point drift) were able to produce satisfactory vessel correspondences between preoperative and intraoperative vessels, in some specific areas the results were suboptimal. For this reason a new method was proposed that combined the coherent point drift and thin plate spline semilandmarks. This combination resulted in an accurate (below 1 mm) non-rigid registration method, evaluated with simulated data where artificial deformations were performed.

Besides the non-rigid registration methods, a new rigid registration method to obtain the rigid transformation between the magnetic resonance dataset and the neuronavigation coordinate systems was also developed.

Once the rigid transformation and the vessel correspondences are known, the thin plate spline can be used to perform the brain shift deformation. To do so, we have used two approaches: a direct and an indirect. With the direct approach, an image is created that represents the deformed data, and with the indirect approach, a new volume is first constructed and only after that can the deformed image be created. A comparison of these two approaches, implemented for the graphics processing units, in terms of performance and image quality, was performed. The indirect method was superior in terms of performance if the sampling along the ray is high, in comparison to the voxel grid, while the direct was superior otherwise. The image quality analysis seemed to indicate that the direct method is superior.

Furthermore, visualization studies were performed to understand how different rendering methods and parameters influence the perception of the spatial position of enclosed objects (typical situation of a tumor enclosed in the brain). To test these methods a new single-monitor-mirror stereoscopic display was constructed. Using this display, stereo images simulating a tumor inside the brain were presented to the users with two rendering methods (illustrative rendering and simple alpha blending) and different levels of opacity. For the simple alpha blending method an optimal opacity level was found, while for the illustrative rendering method all the opacity levels used seemed to perform similarly.

In conclusion, this work developed and evaluated 3D reconstruction, registration (rigid and non-rigid) and deformation methods with the purpose of minimizing the brain shift problem. Stereoscopic perception of the spatial position of enclosed objects was also studied using different rendering methods and parameter values.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2016. , 70 p.
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1762
National Category
Medical Image Processing Radiology, Nuclear Medicine and Medical Imaging Surgery Computer Engineering
Identifiers
URN: urn:nbn:se:liu:diva-130791DOI: 10.3384/diss.diva-130791ISBN: 9789176857724 (Print)OAI: oai:DiVA.org:liu-130791DiVA: diva2:954971
Public defence
2016-09-30, Hugo Theorellsallen (norra entrén), Campus US, Linköping, 09:00 (English)
Opponent
Supervisors
Funder
Swedish Foundation for Strategic Research Knowledge FoundationVINNOVAVårdal Foundation, 2009/0079Swedish Childhood Cancer Foundation, MT2013-0036
Available from: 2016-08-24 Created: 2016-08-24 Last updated: 2016-09-09Bibliographically approved
List of papers
1. Superficial vessel reconstruction with a multiview camera system
Open this publication in new window or tab >>Superficial vessel reconstruction with a multiview camera system
Show others...
2016 (English)In: Journal of Medical Imaging, ISSN 2329-4302, E-ISSN 2329-4310, Vol. 3, no 1, 015001-1-015001-13 p.Article in journal (Refereed) Published
Abstract [en]

We aim at reconstructing superficial vessels of the brain. Ultimately, they will serve to guide the deformationmethods to compensate for the brain shift. A pipeline for three-dimensional (3-D) vessel reconstructionusing three mono-complementary metal-oxide semiconductor cameras has been developed. Vessel centerlinesare manually selected in the images. Using the properties of the Hessian matrix, the centerline points areassigned direction information. For correspondence matching, a combination of methods was used. The processstarts with epipolar and spatial coherence constraints (geometrical constraints), followed by relaxation labelingand an iterative filtering where the 3-D points are compared to surfaces obtained using the thin-plate spline withdecreasing relaxation parameter. Finally, the points are shifted to their local centroid position. Evaluation invirtual, phantom, and experimental images, including intraoperative data from patient experiments, showsthat, with appropriate camera positions, the error estimates (root-mean square error and mean error) are∼1 mm.

Place, publisher, year, edition, pages
SPIE - International Society for Optical Engineering, 2016
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-123661 (URN)10.1117/1.JMI.3.1.015001 (DOI)
Projects
ARIOR
Funder
Swedish Childhood Cancer Foundation, MT2013-0036
Available from: 2016-01-05 Created: 2016-01-05 Last updated: 2016-09-15Bibliographically approved
2. Non-rigid Deformation Pipeline for Compensation of Superficial Brain Shift
Open this publication in new window or tab >>Non-rigid Deformation Pipeline for Compensation of Superficial Brain Shift
2013 (English)In: Medical Image Computing and Computer-Assisted Intervention, MICCAI 2013: 16th International Conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part II, Springer Berlin/Heidelberg, 2013, 141-148 p.Conference paper (Refereed)
Abstract [en]

The correct visualization of anatomical structures is a critical component of neurosurgical navigation systems, to guide the surgeon to the areas of interest as well as to avoid brain damage. A major challenge for neuronavigation systems is the brain shift, or deformation of the exposed brain in comparison to preoperative Magnetic Resonance (MR) image sets. In this work paper, a non-rigid deformation pipeline is proposed for brain shift compensation of preoperative imaging datasets using superficial blood vessels as landmarks. The input was preoperative and intraoperative 3D image sets of superficial vessel centerlines. The intraoperative vessels (obtained using 3 Near-Infrared cameras) were registered and aligned with preoperative Magnetic Resonance Angiography vessel centerlines using manual interaction for the rigid transformation and, for the non-rigid transformation, the non-rigid point set registration method Coherent Point Drift. The rigid registration transforms the intraoperative points from the camera coordinate system to the preoperative MR coordinate system, and the non-rigid registration deals with local transformations in the MR coordinate system. Finally, the generation of a new deformed volume is achieved with the Thin-Plate Spline (TPS) method using as control points the matches in the MR coordinate system found in the previous step. The method was tested in a rabbit brain exposed via craniotomy, where deformations were produced by a balloon inserted into the brain. There was a good correlation between the real state of the brain and the deformed volume obtained using the pipeline. Maximum displacements were approximately 4.0 mm for the exposed brain alone, and 6.7 mm after balloon inflation.

Place, publisher, year, edition, pages
Springer Berlin/Heidelberg, 2013
Series
Lecture Notes in Computer Science, ISSN 0302-9743 (print), 1611-3349 (online) ; 8150
National Category
Engineering and Technology Medical and Health Sciences
Identifiers
urn:nbn:se:liu:diva-106901 (URN)10.1007/978-3-642-40763-5_18 (DOI)000342835100018 ()978-3-642-40762-8 (print) (ISBN)978-3-642-40763-5 (online) (ISBN)
Conference
16th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2013), Nagoya, Japan, September 22-26, 2013
Available from: 2014-05-23 Created: 2014-05-23 Last updated: 2016-09-15Bibliographically approved
3. Non-rigid point set registration of curves: registration of the superficial vessel centerlines of the brain
Open this publication in new window or tab >>Non-rigid point set registration of curves: registration of the superficial vessel centerlines of the brain
2016 (English)In: Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, SPIE - International Society for Optical Engineering, 2016, Vol. 9786, 8 p.978611-1-978611-8 p.Conference paper (Refereed)
Abstract [en]

In this study we present a non-rigid point set registration for 3D curves (composed by 3D set of points). Themethod was evaluated in the task of registration of 3D superficial vessels of the brain where it was used to matchvessel centerline points. It consists of a combination of the Coherent Point Drift (CPD) and the Thin-PlateSpline (TPS) semilandmarks. The CPD is used to perform the initial matching of centerline 3D points, whilethe semilandmark method iteratively relaxes/slides the points.

For the evaluation, a Magnetic Resonance Angiography (MRA) dataset was used. Deformations were appliedto the extracted vessels centerlines to simulate brain bulging and sinking, using a TPS deformation where afew control points were manipulated to obtain the desired transformation (T1). Once the correspondences areknown, the corresponding points are used to define a new TPS deformation(T2). The errors are measured in thedeformed space, by transforming the original points using T1 and T2 and measuring the distance between them.To simulate cases where the deformed vessel data is incomplete, parts of the reference vessels were cut and thendeformed. Furthermore, anisotropic normally distributed noise was added.

The results show that the error estimates (root mean square error and mean error) are below 1 mm, even inthe presence of noise and incomplete data.

Place, publisher, year, edition, pages
SPIE - International Society for Optical Engineering, 2016. 8 p.
Series
, Progress in Biomedical Optics, ISSN 1605-7422 ; 9786
Keyword
Non-rigid registration, brain shift correction, vessel registration
National Category
Medical Image Processing
Identifiers
urn:nbn:se:liu:diva-126347 (URN)10.1117/12.2208421 (DOI)000382315800036 ()978-1-5106-0021-8 (ISBN)
Conference
Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling, San Diego, California, United States, February 27, 2016
Projects
ARIOR
Funder
Swedish Childhood Cancer Foundation, MT2013-0036
Available from: 2016-03-22 Created: 2016-03-22 Last updated: 2016-09-27Bibliographically approved
4. GPU-based ray-casting of non-rigid deformations: a comparison between direct and indirect approaches
Open this publication in new window or tab >>GPU-based ray-casting of non-rigid deformations: a comparison between direct and indirect approaches
2014 (English)In: Proceedings of SIGRAD 2014, Visual Computing, June 12-13, 2014, Göteborg, Sweden / [ed] Mohammad Obaid; Daniel Sjölie; Erik Sintorn; Morten Fjeld, Linköping University Electronic Press, 2014, 67-74 p.Conference paper (Refereed)
Abstract [en]

For ray-casting of non-rigid deformations, the direct approach (as opposed to the traditional indirect approach) does not require the computation of an intermediate volume to be used for the rendering step. The aim of this study was to compare the two approaches in terms of performance (speed) and accuracy (image quality).

The direct and the indirect approach were carefully implemented to benefit of the massive GPU parallel power, using CUDA. They were then tested with Computed Tomography (CT) datasets of varying sizes and with a synthetic image, the Marschner-Lobb function.

The results show that the direct approach is dependent on the ray sampling steps, number of landmarks and image resolution. The indirect approach is mainly affected by the number of landmarks, if the volume is large enough.

These results exclude extreme cases, i.e. if the sampling steps are much smaller than the voxel size and if the image resolution is much higher than the ones used here. For a volume of size 512×512×512, using 100 landmarks and image resolution of 1280×960, the direct method performs better if the ray sampling steps are approximately above 1 voxel. Regarding accuracy, the direct method provides better results for multiple frequencies using the Marschner-Lobb function.

The conclusion is that the indirect method is superior in terms of performance, if the sampling along the rays is high, in comparison to the voxel grid, while the direct is superior otherwise. The accuracy analysis seems to point out that the direct method is superior, in particular when the implicit function is used.

Place, publisher, year, edition, pages
Linköping University Electronic Press, 2014
Series
Linköping Electronic Conference Proceedings, ISSN 1650-3686 (print), 1650-3740 (online) ; 106
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-107553 (URN)9789175192123 (ISBN)
Conference
Swedish Computer Graphics Association (SIGRAD), Göteborg, Sweden, June 12-13, 2014
Available from: 2014-06-16 Created: 2014-06-16 Last updated: 2016-09-15Bibliographically approved
5. Single-Monitor-Mirror Stereoscopic Display
Open this publication in new window or tab >>Single-Monitor-Mirror Stereoscopic Display
2013 (English)In: Journal of Graphics Tools, ISSN 2165-347X, Vol. 17, no 3, 85-97 p.Article in journal (Refereed) Published
Abstract [en]

A new single-monitor-mirror stereoscopic display is presented. The stereoscopic display system is composed of one monitor and one acrylic first-surface mirror. The mirror reflects one image for one of the eyes. The geometrical transformations to compute correctly the stereo pair are derived and presented. System considerations such as mirror placement and implications are also discussed.

In contrast to other similar solutions that use fixed configurations, we try to optimize the display area by controlling the mirror placement. Consequently, one of the images needs to be skewed. Advantages of the system include absence of ghosting and flickering.

We also developed the rendering engine for direct volume rendering (DVR) of volumetric datasets mostly for medical imaging visualization and using OpenGL for polygonal datasets and stereoscopic digital photography. The skewing process in this case is integrated into the ray-casting of DVR. Using geometrical transformations, we can compute precisely the directions of the rays, producing accurate stereo pairs. A similar operation is also performed using OpenGL.

Place, publisher, year, edition, pages
Taylor & Francis, 2013
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-120584 (URN)10.1080/2165347X.2015.1028690 (DOI)
Projects
arior
Funder
Swedish Childhood Cancer Foundation, MT2013-0036Swedish Foundation for Strategic Research VINNOVAVårdal Foundation
Available from: 2015-08-17 Created: 2015-08-17 Last updated: 2016-09-15Bibliographically approved
6. Stereoscopic static depth perception of enclosed 3D objects
Open this publication in new window or tab >>Stereoscopic static depth perception of enclosed 3D objects
2013 (English)In: SAP '13 Proceedings of the ACM Symposium on Applied Perception, New York, USA: Association for Computing Machinery (ACM), 2013, 15-22 p.Conference paper (Refereed)
Abstract [en]

Depth perception of semi-transparent virtual objects and the visu-alization of their spatial layout are crucial in many applications, in particular medical applications. Depth cues for opaque objects have been extensively studied, but this is not the case for stereo-scopic semi-transparent objects, in particular in the case when one 3D object is enclosed within a larger exterior object.

In this work we explored different stereoscopic rendering methodsto analyze their impact on depth perception accuracy of an enclosed3D object. Two experiments were performed: the first tested the hypotheses that depth perception is dependent on the color blending of objects (opacity - alpha) for each rendering method and that one of two rendering methods used is superior. The second experiment was performed to corroborate the results of the first experiment and to test an extra hypothesis: is depth perception improved if an auxiliary object that provides a relationship between the enclosed objectand the exterior is used?

The first rendering method used is simple alpha blending with Blinn-Phong shading model, where a segmented brain (exterior object) and a synthetic tumor (enclosed object) were blended. The second rendering method also uses Blinn-Phong, but the shading was modified to preserve silhouettes and to provide an illustrative rendering. Comparing both rendering methods, the brighter regionsof the first rendering method will become more transparent in the second rendering method, thus preserving silhouette areas.

The results show that depth perception accuracy of an enclosed object rendered with a stereoscopic system is dependent on opacity for some rendering methods (simple alpha blending), but this effect is less pronounced than the dependence on object position in relation to the exterior object. The illustrative rendering method is less dependent on opacity. The different rendering methods also perform slightly differently; an illustrative rendering method was superior and the use of an auxiliary object seems to facilitate depth perception.

Place, publisher, year, edition, pages
New York, USA: Association for Computing Machinery (ACM), 2013
Keyword
depth perception, stereoscopy, enclosed 3D objects
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-106896 (URN)10.1145/2492494.2492501 (DOI)978-1-4503-2262-1 (ISBN)
Conference
2013 ACM Symposium on Applied Perception, SAP 2013; Dublin, Ireland, August 22 - 23, 2013
Funder
VINNOVA
Available from: 2014-05-23 Created: 2014-05-23 Last updated: 2016-09-15Bibliographically approved

Open Access in DiVA

Guidance and Visualization for Brain Tumor Surgery(2804 kB)24 downloads
File information
File name FULLTEXT01.pdfFile size 2804 kBChecksum SHA-512
96a1f8705490558931aa105623e2bfc5388a82ff985f280832731f6016538d33b163edd4407cd360b739591eadc25ac9a8c4c43294bd26aff8412b13a0249fda
Type fulltextMimetype application/pdf
omslag(8086 kB)13 downloads
File information
File name COVER01.pdfFile size 8086 kBChecksum SHA-512
be05c2635a5f2cb2b52b9c81110cfbf2a5ccdd0ad8213721b25cd422920486e6985dee6000576af72eb48ab276def62db3f8f2c8e48580f97c86ee17b42f65a6
Type coverMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Maria Marreiros, Filipe Miguel
By organisation
Division of Radiological SciencesCenter for Medical Image Science and Visualization (CMIV)Faculty of Science & Engineering
Medical Image ProcessingRadiology, Nuclear Medicine and Medical ImagingSurgeryComputer Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 24 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 508 hits
ReferencesLink to record
Permanent link

Direct link