Regional appearance modeling for deformable model-based image segmentation

Medical imaging

Novel appearance prior for model-based image segmentation in medical imaging

Although considered as a natural process for human beings, the task of extracting objects from images through segmentation is still requiring a tremendous attention in the computer vision community. Segmentation consists in extracting regions of interest out of images. In medical imaging, the goal of segmentation is to extract anatomical structures of interest such as organs, bones and tissues. Model-based image segmentation is a very common and efficient technique used for segmentation. A mesh is first initialized into the image and then deformed using forces that attract the mesh to fit regions of interest. Those generated meshes may be used both for therapy planning and quantification for diagnosis.

In this thesis, a novel appearance prior for model-based image segmentation in medical imaging is presented. This appearance prior, denoted as Multimodal Prior Appearance Model (MPAM), is built upon a classification of intensity profiles with model order selection to automatically select the number of profile classes. Unlike classical PCA-based approaches, the clustering is considered as regional because intensity profiles are classified for each mesh and not for each vertex.

Illustration of the similar appearance of 4 livers and 2 shins cut at the knee level after classification of the intensity profiles.

In practice, the Multimodal Prior Appearance Model (MPAM) is built from a training set of meshes and images. Intensity profiles are extracted at each vertex of those meshes and classified using unsupervised clustering. The determination of the number of profile classes (i.e. appearance regions) is performed by a novel model order selection criterion. To spatially smooth the clustering of profiles, a spatial regularization is performed on the classification. Finally, the classification (i.e. appearance information) from each dataset is projected on a reference mesh.

In a second part, a boosted clustering based on spectral clustering is presented. The boosted clustering aims at optimizing the clustering of profiles associated with the MPAM for segmentation purposes. Spectral clustering consists in classifying profiles in the spectral space, after defining the similarity between data points through spectral graphs, spectral functions and affinity matrices. Boosted clustering uses a similarity measure during a local search to ensure that intensity profiles are well represented by the classification.


In this thesis, we propose a novel appearance prior for the segmentation of anatomical structures using deformable models. MPAM is first built upon an EM clustering of intensity profiles with model order selection to automatically select the number of profile classes in an attempt to study the appearance around anatomical structures. Unlike classical PCA-based Appearance Priors (PCAP), the clustering is considered as regional because each point may be associated with several profile modes and each profile mode is estimated on each subject and not across subjects. Segmentation results from comparative tests on liver profiles show that our MPAM outperforms PCAP despite the fact that less profile modes are used.

Result of the based segmentation models on a CT image of the liver.

Asclepios Research Team

François Chung is an Industrial Engineer in computer science (M.Sc.) from Institut Supérieur Industriel de Bruxelles (ISIB), Belgium. In 2005, he followed Ph.D. courses in computer vision and robotics at the University of Girona, Spain. He also worked on image registration in the Underwater Vision Laboratory of the Computer Vision and Robotics group (VICOROB). Currently, he is a Mines ParisTech Ph.D. candidate in medical imaging at the Asclepios Research Team, INRIA Sophia-Antipolis, France. His research interests include computer vision, medical imaging, classification and segmentation.

A photo of me during MRI acquisition in St Mary Hospital of London.

I am doing my PhD at the INRIA Sophia Antipolis, which is quite close to Mines ParisTech based in Sophia Antipolis, on the French Riviera, between Cannes and Monaco, and more generally between the Mediterranean Sea and the mountains. This is a very nice environment, where getting sun at the beach in summer and skiing in the mountains in the winter is possible. Most of the time, I'm working at INRIA to interact with my research team but I'm occasionally going to Mines ParisTech for courses and lectures.