User Interactive Image Segmentation By Gaussian Mixture Model

in Japanese

Introduction

In the previous page, I showed the simple image segmentation by the Gaussian Mixture Model (GMM). I made use of the implementation of the EM Algorithm that the OpenCV library provides to us. In this page, I will make the image segmentation user-interactive, i.e.,

1. A user roughly specifies some segments into which he/she expects to separate an image by mouse moving.
2. From the pixels in each specified segment, the program calculates the GMM that indicates the probability that a pixel belongs to the segment.
3. The program applies all GMMs to the entire image to separate it into user specified segments.

Formulation

Suppose that we roughly specify M regions on an image. Thus, we get M samples , where each sample has 3D vectors as (R,G,B). We can construct the GMM from as

, where represents the probability that a given pixel () belongs to the segment . We find the maximum probability among the M probabilities for the pixel, and assign it into the corresponding region.

Demo 1

Firstly we roughly tell the program two segments, the ray and the background regions.

The results are as follows:
background

ray

Demo 2

We roughly tell the program three regions, trees, cats(?), and leaves.

The results are as follows:
trees

leaves

cats

Demo 3

We roughly tell the program two regions, the bird and the background.

The results are as follows:
background

bird

Development Environment

1. Mac OS X 10.8.3
2. Processor：3.06 GHz Intel Core 2 Duo
3. Memory：4GB
4. Xcode4.6.1 with Apple LLVM 4.2（C++ Language Dialect → C++11, C++ Standard Library → libc++）
5. boost-1.51.0 built by the Apple LLVM 4.1. See here
6. opencv-2.4.3 built by the Apple LLVM 4.2. See here.

My Source Code

Here is my source code. I tested it on the above environment.

Usage

The execution file name is InteractiveImageSegmentation. Running it without any arguments, it prints the following usage statements on the standard output: The argument --nclusters indicates the number of the components that constitute the Gaussian Mixture Model. It corresponds to the variable in the formulation described on the top of the page.

The following command displays the input image. Holding down either of the 0-9 keys, you move the mouse on the image with pressing the left button on the mouse. The numbers from 0 to 9 identify the samples. For example, we can specify the background area with holding down 0 key and the bird area with holding down 1 key. The maximum number of the samples we can spedify is 10.

Pressing "e" begins the image segmentation. If we define two regions, two GMMs are calculated. On the standard output, the means are displayed. As we set --nclusters to 5, five 3D vectors for each GMM are printed.

Pressing "c" clears all samples. You can make another sampling.