I. INTRODUCTION
Applied medical research is becoming rapidly developed and more dependent on imaging for diagnosis the serious and critical diseases of human body. Thus, dedicated image analysis software is needed for quantitative medical imaging [1]. Medical imaging is the technique used to create images of the human body for clinical purposes, especially for analyzing some anatomy related abnormalities.
Several data visualization and analysis environments have been developed over the past decades, which claim to meet the requirements of the users in the field of computer and computational science [2]. Which visualization and analysis environments are available for diagnosis different diseases based on different data types, such as electromagnetic signals and images;
MRI is the most common medical imaging technique used for diagnosing brain-related diseases. The major advantage of MRI is that it provides more detailed images than X-ray, ultrasound or CT [3]. MRI can also produce images in various shades of color, indicating the different conditions of the tissue under the investigation.
The most common way to diagnose the disease is to compare images of suspect organ with images of a healthy person’s organ [4]. However, sometimes, a simple two-dimensional comparison is not sufficient to recognize the disease, so clinicians need to observe the organ in a three dimensional view [5]. The primary purpose in designing this software was to 3D render, visualize and segment the brain MR images to observe and measure volume shrinkage and deformities of the hippocampus.
In this work, we develop a new 3D rendering software for MR images visualization and analysis for hippocampus segmentation [6]. The main framework is based on Microsoft .NET and it integrated with C# wrapper of VTK library (Activiz.NET) as a rendering kernel. Visualization in C# language using VTK potential can be implemented using Activiz.NET library, which is an advanced, open-source software package containing .NET wrapper for all VTK objects.
The Visualization Toolkit (VTK), a widely used library for visualization, is a primary resource for achieving rapid development of medical imaging tools in cost-effective way [7]. VTK is an open-source, freely available software system for 3D computer graphics, modeling, image processing, volume rendering, scientific visualization, and information visualization. VTK also includes ancillary support for 3D interaction widgets, two- and three-dimensional annotation, and parallel computing. At its core, VTK is implemented as a C++ toolkit, requiring users to build applications by combining various objects into an application [8]. The system also supports automated wrapping of the C++ core into Python, Java, and Tcl, so VTK applications may also be written using these interpreted programming languages.
II. PROGRAMMING WITH VTK
The VTK is an object-oriented system; the access of class and instance data members is carefully controlled in VTK. In general, all data members are either protected or private. Access to them is through Set and Get methods, with special variations for Boolean data, modal data, strings and vectors. VTK is the most notable algorithm toolkit that aims to provide an algorithm library for medical image processing and visualization in this application development process [9-12]. VTK possesses the classical object-oriented design for taking full advantage of the C++ language capabilities.
Computer languages are usually two types: compiled languages are interpreted languages. Compiled languages are usually high performing than interpreted languages, but interpreted languages provide greater flexibility to rapid application development. Our experience has shown that interpreted applications can be built significantly faster than compiled applications, mainly through the elimination of the compile/link cycle. Also, interpreted applications are often written in high level languages than compiled languages. This results in simpler to make the application with more compact code that is faster to write and debug [13]. Compiled systems, however, are absolutely necessary when developing high performance visualization application such as real time medical image processing and visualization. Compiled systems also provide low-level access computer resources.
Visualization transforms data into images that efficiently and accurately represent information about the data. Hence, visualization deals with the issues of transformation and representation. Transformation is the process of converting data from its original form into graphics primitives, and eventually into computer images.
VTK applications are largely constructed by connecting vtkAlgorithms together. Each algorithm inspects the dataset or datasets it is given and produces some derived data for the algorithms connected to it. The connected set of filters forms a data-flow network. VTK uses reference counting heavily to eliminate redundant memory consumption and timestamps in the demand-driven network to eliminate redundant computation. Algorithms are strongly type-checked to enforce compatible filter connectivity [14]. VTK has hundreds of algorithms with which to work, from the vtkAbstractMapper to the vtkXMLWriter. Some algorithms are extremely focused in what they can do, while others are completely general such as the vtkPythonAlgorithm.
VTK consists of several major subsystems. Probably the subsystem the most associated with visualization packages is the data flow/pipeline architecture. In concept, the pipeline architecture consists of three basic classes of objects:
-
vtkDataObjects- objects to represent data
-
vtkAlgorithm-objects to process transform, filter or map data objects from one form into another
-
vtkExecutive- objects to execute a pipeline which controls a connected graph of interleaved data and process objects
The algorithm objects also introduce their own special complexity. Some algorithms may take multiple inputs and/or produce multiple outputs of different types. Some can operate locally on data (e.g., compute the center of a cell) while others require global information, for example, to compute a histogram. In all cases, the algorithms treat their inputs as immutable, algorithms only read their input in order to produce their output [15-18]. This is because data may be available as input to multiple algorithms, and it is not a good idea for one algorithm to trample on the input of another.
Volume rendering is a term used to describe a process where information exists throughout a 3D space instead of simply on a 2D surface defined in 3D space. The 3D sampling points do not have color attributes themselves, including gray values; however, volume rendering calculates sampling points contributing to the screen pixels by an optical model, based on re-sampling. The algorithm can generate a 3D data field as a whole high-quality image with easy application of parallel processing [19-23].
The vtkImageData object can be used to represent one, two, and three-dimensional image data. As a subclass of vtkDataSet, vtkImageData can be represented by a vtkActor and rendered with a vtkDataSetMapper. In the 3D, data can be considered a volume. Alternatively, it can be represented by a vtkVolume and rendered with a subclass of vtkVolumeMapper. Since some subclasses of vtkVolumeMapper use geometric techniques to render the volume data, the distinction between volumes and actors mostly arises from the different terminology and parameters used in volumetric rendering as it is opposed to the underlying rendering method.
The vtkFixedPointVolumeRayCastMapper uses a ray casting technique for volume rendering. Algorithmically, it is quite similar to the vtkRayCastMapper. We chose to use ray casting due to the flexibility of this technique, which allows us to support all the features of the software ray cast mapper but with the acceleration of the GPU.
Ray casting is inherently an image-order rendering technique, with one or more rays cast through the volume per image pixel.
The image-order rendering process for the vtkVolume is initiated when the front-facing polygons of the volume’s bounding box are rendered with a custom fragment program [24]. This fragment program is used to cast a ray through the volume at each pixel, with the fragment location indicating the starting location for that ray.
III. DESIGN AND IMPLIMENTATION
What follows is an outline of some of the important details concerning design and implementation of the software.
The software was designed for the development of interactive volume rendering and segmentation application in general, having a number of design goals in mind:
-
Software should be easy to use on different computers without any pre configuration.
-
Software should be useable for use with different kind of image formats.
-
Software should be able to do arbitrary directional volume cutting with segmentation capability.
-
Logic, model and graphical user interface should be separated where possible, so that the graphical user interface and controller is exchangeable, also easy to extend current application.
Besides these general goals, focused the following major tasks need to be accomplished by the implementation:
-
Read series of DICOM files into VTK: Read series of 2D medical images using DICOM image reader
-
Volume rendering and visualization: Reconstruct the 3D image by sequentially stacked images
-
Image interpolation: Linear interpolation algorithm used to generate 3D volume image from existing 2D series of images
-
Slice manipulation: Segmentation tool needs to have different directional (orthogonal and arbitrary) sliced images from 3D volume.
-
Segmentation tool: Tools receive user interaction and call relevant functions to perform segmentation.
-
Undo and Refresh: All slice segmentation operations should be undoable step by step or all at once.
There are two distinct parts in our design. The first is the graphics model, which is abstract model for 3D graphics. The second is the visualization model, which is a data flow model of the visualization process.
The graphic model captures the important features of a 3D graphics system. Our software has eight basic graphic objects in the model.
-
Render Window- mange a window on the display device. One or more renderers draw into a single render window to generate scenes.
-
Renderer- coordinates the rendering of lights, camera, and actors.
-
Light- illuminates the actor and background in a scene.
-
Camera- defines the view position, focal point, and other camera characteristics.
-
Actor- an object drawn by a renderer in the scene. Actors are defined in terms of mapper, properties, and transform objects.
-
Property- represents the rendered attributes of an actor including object color, lighting, texture map, shading style.
-
Mapper- represents the geometric definition of an actor and maps object through a lookuptable. More than one actor may refer to the same mapper.
-
Transform-an object that consists of a 4x4 transformation matrix and method to modify matrix.
The visualization model based on the data flow paradigm adopted in many software systems. In this paradigm, modules are connected together in a pipeline. The modules perform algorithmic operations n data as it flows through the pipeline [25]. The execution of this visualization pipeline is controlled in response to demands driven or event driven. The appeal of this model is that it is flexible, and can be quickly adapted to different data types or algorithms.
Our visualization model consists of two basic types of objects: process objects and data objects (see figure 7). Process objects are the modules of visualization pipeline. Data objects, also referred to as datasets, represent and enable operations on the data flow through visualization pipeline.
A major concern when implementing visualization in data flow form is the amount of memory consumed. C# employs automatic memory management, which frees developers from manually allocating and freeing the memory occupied by objects. Automatic memory management policies are implemented by a garbage collector. The memory management life cycle of an object is as follows:
-
When the object is created, memory is allocated for it, the constructor is run, and the object is considered live.
-
If the object, or any part of it, cannot be accessed by any possible continuation of execution, other than the running of destructors, the object is considered no longer in use and it becomes eligible for destruction. The C# compiler and the garbage collector may choose to analyze code to determine which references to an object may be used in the future.
-
Once the object is eligible for destruction, at some unspecified later time the destructor for the object is run. Unless overridden by explicit calls, the destructor for the object is run once only.
-
Finally, at some time after the object becomes eligible for collection, the garbage collector frees the memory associated with that object.
The garbage collector maintains information about object usage, and uses this information to make memory management decisions, such as where in memory to locate a newly created object, when to relocate an object, and when an object is no longer in use or inaccessible.
IV. RESULTS AND EVALUATION
In this section, we demonstrate our medical images 3D rendering application software. Fig.8 shows the 3D rendering of series of DICOM images and demonstrate the 3D volume cutting function how to works.
In order to compare and evaluate, the segmente results of proposed software with existing medical imaging software. Here, we compared our software with FreeSurfer and MRICro software.
Following as the Fig.9 shows hippocampus segmentation and 3D rendering results of segmented hippocampus.
We used image datasets from six different patients of the age group (25-40 years) for comparison of segmented image data using different software. Among them, three patients are male and three patients are female. Following table shows segmented hippocampus volumetric data.
Following figures give graphical representation of segmented hippocampus volumetric measurements of MRICro, FreeSurfer and Our software.
V. CONCLUSION
Medical image analysis is more than just the algorithms. Visualization of the original image data and processed results, interaction with the data, and the data itself are also important. VTK is a powerful library of medical imaging algorithms, especially for 3D visualization. However, medical image segmentation is not well-supported by VTK. Furthermore, more complex interaction methods, including undo/redo of interactions and run-time data management, both of which are required for convenient-to-use interactive applications, are beyond the scope of VTK.
In this research, we have implemented 3D rendering software for MR images using VTK and Microsoft .NET framework. Our software provides arbitrary directional volume cutting and segmentation functions which is not provided by many existing medical imaging software. This application software is on-going project and we expect to improve the functionality, user-friendliness, flexibility and robustness though the feedback of the researches. Further work concerns add semi automatic segmentation algorithm to the application software.
