3D tissue segmentation from CT

Building 3D model of tissue based on series of computed tomography scans.

Neural networks
Computer vision
3D

2014

About project

Task

To develop software capable of building 3D-model from a series of CT-images.

Solution

  • DICOM-format read and write library was developed
  • Neural network for extracting a tissue on CT-image was made
  • 3D model of tissue was constucted from points cloud to find its volume.
  • A tool for visualization of results was developed

Task

Recognition of objects from graphic images — is a common way of using machine learning methods. Similar problems arise in many areas that use sensors to obtain data.

In particular, processing of information from medical images is an example of such area. There are a lot of applications of machining learning possibilities: segmentation of organs, selection of objects, search for areas containing tumor, building of 3D-representation on the basis of a series of images, and so on.

Pic 1. Image of liver from CT
Pic 2. CT process animation

One of the main problems is that original picture is very noisey. Indeed, to obtain CT scans X-rays are used. The stronger the X-rays are, the better quality of the output images. It is clear that it does not add health to the patients. Therefore, doctors are always seeking a new ways to reduce radiation level. And so here comes computer technology to help medicine and extract information from the original image.

Solution

We had the following objectives: from a series of CT scans we had to build a 3D model of the patient's liver and estimate its volume. The volume of the tissue is an important criterion for choosing a method of treating cancer patients: an operation or medical treatment. Modern software tools often cannot solve this task, so specialists have to select tissue manually, encircling it with the mouse on each frame, that takes a tremendous amount of time.

If you build 3D models of the body on a cloud of points and the assessment of its volume does not represent the fundamental complexity, the problem of segmentation, ie, the separation of interest to us body (in this case, the liver) from the other components of the frame is a daunting task. Indeed, in the field of contact body other tissues even visually impossible to draw a line in front of the liver.

Pic 3. Liver selection from CT-image: before and after

To solve this problem, we used a special neural network, with information on 3D environment of each pixel slice. Additionally, an important contribution to the solution is the use of texture information: i.e. pattern constituting the pattern of a particular tissue.