in Japanese

## Introduction

In the previous page, I showed the application that is using the OpenNI and the Open Dynamics Engine libraries with the Xtion Pro Live. Recently I have introduced the SceneAnalyzer node, which is supported by the OpenNI library, to the application. In this page, I provide you with an explanation on the new application. (Here is my first introduction to the ODE.)

## Demo

I start by showing a demo movie.

## Developement Environment

1. Mac OS X 10.8.2
2. Processor：3.06 GHz Intel Core 2 Duo
3. Memory：4GB
4. Xcode4.5.1

## Primary Libraries

1. boost-1.51.0
2. pcl-1.6.0
3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
4. ode-0.12

## My Source Code

Here is my source code. I do not confirm the behavior of the application on environments other than mine.

## Compile Options

Path Purpose
/usr/include/ni/ to include the OpenNI headers
/opt/local/include/ to include the Boost and the ODE headers
/usr/local/include/pcl-1.6/ to include the PCL headers
/opt/local/include/eigen3/ to include the Eigen headers on which the PCL depends
/opt/local/include/vtk-5.10/ to include the VTK headers on which PCL dependes

Library Search Path:
Path Purpose
/opt/local/lib to link the Boost library
/usr/local/lib to link the PCL library

 -lboost_thread-mt -lboost_filesystem-mt -lboost_unit_test_framework-mt -lboost_chrono-mt -lboost_program_options-mt -lboost_system-mt /usr/lib/libOpenNI.dylib -framework OpenGL -framework GLUT -lpcl_filters -lpcl_kdtree -lpcl_segmentation -lpcl_surface -lpcl_search -lpcl_features -lpcl_common -lode -lpcl_io

## Usage

An execution file name is InteractiveObjects. Running it with an argument --help, it writes the following usage statements on the standard output. As shown above, the default window size is 320x240. I encountered the following error when running it with arguments --width 640 --height 480: The 21th-22th lines are output from the OpenNI library. I may be mistaken, but I think that the SceneAnalyzer node does not support the resolution 640x480.
Running it without any arguments (i.e. default size), it opens a viewer and begins to display frames captured by the Xtion Pro Live. Typing "m" on the keyboard under the situation where the application focuses on the viewer, a triangular mesh is constructed from a point cloud. The point cloud is calculated from the depth data in the frame at the moment when typing "m." The triangular mesh is used to create the background objects which consist of a floor, a wall, and so on. Then typing "r" brings virtual rigid bodies (balls and cylinders) into the captured scenes. The cylinders are fastened on to the floor. In the viewer, you can see that the balls collide with the cylinders, the background objects, and the other balls. Typing "o" enables an object detection, and the viewer begins to draw the collision between the detected object (human body) and the balls. To quit the application, type "q."

## Brief Overview Of Source Code

I will show you different code from the previous one.

#### Transforming Depth into Point Cloud

In the method Device::create_cloud, a depth data captured by the Xtion Pro Live is transformed into three point clouds:
1. a point cloud which is used to draw RGB images (cloud_)
2. a point cloud which is used to calculate a triangular mesh which describes the background objects (background_cloud_)
3. a point cloud which is used to calculate a triangular mesh which describes the objects detected by the SceneAnalyzer node (object_cloud_)
The 11th-13th lines and the 39th-44th lines in the code shown below are related to the SceneAnalyzer node. The point cloud object_cloud_ is captured per frame, while background_cloud_ is captured at the moment when typing "m" and it is fixed until typing "m" again.

#### Rigid Bodies

I use the ODE library to describe the rigid bodies. The ODE objects in the application are defined as follows: The prefix character "d" indicates the class provided by the ODE library. The object which moves based on the rigid body dynamics is represented by the dBody object. The object which requires the collision detection is represented by the dGeom object. In the application, a ball moves under the rigid body dynamics and collides with each other and other objects. Therefore, it is defined as the 5th-9th lines, where dSphere is the derived class from dGeom.
On the other hand, a cylinder is fastened on to the floor, so it does not move, but only collides with the balls. For that reason, as shown in the 18th line, I defined it as the object of the class dCylinder which is the derived class from dGeom.

Additional details (e.g. the transformation of a point cloud to a triangular mesh, registration of a mesh to the ODE, a collision detection, and so on) are seen in the previous page or my source code.

in English

こんな感じです。

## 開発環境

1. Mac OS X 10.8.2
2. プロセッサ：3.06 GHz Intel Core 2 Duo
3. メモリ：4GB
4. Xcode4.5.1

## 主要ライブラリ

1. boost-1.51.0
2. pcl-1.6.0
3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
4. ode-0.12

## ソース

こちらです。自分のマシーンでしか動作確認していません。

## コンパイルオプション

パス 目的
/usr/include/ni/ OpenNIヘッダー
/opt/local/include/ boost/odeヘッダー
/usr/local/include/pcl-1.6/ PCLヘッダー
/opt/local/include/eigen3/ eigenヘッダー(PCLがこれに依存している）
/opt/local/include/vtk-5.10/ vtkヘッダー(PCLがこれに依存している）

Library Search Path:
パス 目的
/opt/local/lib boostライブラリ
/usr/local/lib PCLライブラリ

 -lboost_thread-mt -lboost_filesystem-mt -lboost_unit_test_framework-mt -lboost_chrono-mt -lboost_program_options-mt -lboost_system-mt /usr/lib/libOpenNI.dylib -framework OpenGL -framework GLUT -lpcl_filters -lpcl_kdtree -lpcl_segmentation -lpcl_surface -lpcl_search -lpcl_features -lpcl_common -lode -lpcl_io

## ソースの概要

#### ポイントクラウドへの変換

Device::create_cloudで行います。以下3つのポイントクラウドを生成します。
1. カラー画像描画のためのポイントクラウド（cloud_
2. 背景の三角メッシュを算出するためのポイントクラウド（background_cloud_
3. 人物の三角メッシュを算出するためのポイントクラウド（object_cloud_
object_cloud_を生成するため、OpenNIのSceneAnalyzerを導入しました。それに関連するコードが11行目から13行目、39行目から44行目にあります。SceneAnalyzerが捉えた物体はフレーム単位で三角メッシュに変換され、ODEに登録されます。一方、背景となる三角メッシュは「m」が押された瞬間のフレームから作られ、それ以降は固定されます。

#### 剛体オブジェクト

その他、ポイントクラウドから三角メッシュへの変換、三角メッシュのODEへの登録、衝突の検出などは以前のページ、あるいはソースをご覧ください。

in Japanese

## Introduction

In the previous page, I implemented an application with the use of the OpenNI library. The application includes the simple simulation by the Newtonian dynamics. Recently, I have replaced the simulation with the Open Dynamics Engine(ODE). In this page, I give you an explanation on the new application. (Here is my first introduction to the ODE.)

## Development Environment

1. Mac OS X 10.8.2
2. Processor：3.06 GHz Intel Core 2 Duo
3. Memory：4GB
4. Xcode4.5.1

## Primary Libraries

1. boost-1.51.0
2. pcl-1.6.0
3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
4. ode-0.12

## My Source Code

Here is my source code. I do not confirm the behavior of the application on other environments.

## Compile Options

Path Purpose
/usr/include/ni/ to include the OpenNI headers
/opt/local/include/ to include the Boost and ODE headers
/usr/local/include/pcl-1.6/ to include the PCL headers
/opt/local/include/eigen3/ to include the Eigen header on which the PCL depends
/opt/local/include/vtk-5.10/ to include the VTK headers on which the PCL depends

Library Search Path:
Path Purpose
/opt/local/lib to link the Boost libraries
/usr/local/lib to link with the PCL libraries

 -lboost_thread-mt -lboost_filesystem-mt -lboost_unit_test_framework-mt -lboost_chrono-mt -lboost_program_options-mt -lboost_system-mt /usr/lib/libOpenNI.dylib -framework OpenGL -framework GLUT -lpcl_filters -lpcl_kdtree -lpcl_segmentation -lpcl_surface -lpcl_search -lpcl_features -lpcl_common -lode -lpcl_io

## Usage

An execution file name is BouncingBallWithOde. Running it with an argument --help, it writes the following usage statements on the standard output. Running it without any arguments, it opens a viewer and begins to display frames captured by the Xtion Pro Live. Typing "m" on the keyboard under the situation where the application focuses on the viewer, a triangular mesh is constructed from a point cloud. The point cloud is calculated from the depth data in the frame at the moment when typing "m". Typing "b" generates virtual balls in the captured scenes, and the interactions between the balls and the objects in the environment are drawn in real time. You can generate balls as many times as needed. Though an error occurs in the 29th line, no major disruptions are observed. To quit the application, type "q."

## Brief Overview of Source code

#### View Point of OpenGL

The view point of the OpenGL is as follow: A unit of length in the application is mm. You might need some changes to the arguments in the 6th line according to the environment captured by your depth sensor.

#### Transforming Depth into Point Cloud

A depth data is transformed into a point cloud in the method Device::create_cloud. In this application, two point clouds are generated, one (cloud_) is for displaying the color frames and another (cloud_xyz_) is for calculating a triangular mesh. RGB values are set in the 25,26th lines. The depth data is transformed into the point cloud in the 29th-30th lines. The point cloud used to calculate the triangular mesh is generated in the 33th line.

#### Drawing Color Image

The point cloud cloud_ is passed to the method Viewer::display_point_cloud as follows: The arguments buf and size indicate the pointer to the head address of cloud_ and the size of cloud_, respectively.

#### Calculation of Triangular Mesh

In the method Device::convert_to_polyton_mesh, the triangular mesh is constructed from the point cloud cloud_xyz_ by the use of the PCL library. (Here is my first introduction to the PCL library.) You might need to change parameters in the 27th-36th lines according to your environment.

#### Making Balls

A ball treated in the application is an ODE object. It consists of a dBody object and a dGeom one. The former manages the rigid body dynamics and the latter manages the collision detection. dSphere is the derived class from the base class dGeom. In the method PhysicalSimulator::create_balls(), balls are registered to the ODE framework. world_ in the 20th line and space_ in the 41th line are the dWorld and the dSpace objects, respectively. In the ODE framework, the object which moves based on the rigid body dynamics has to be registered to the dWorld object. The object which collides with other objects has to be registered to the dSpace object.

#### Registration of Triangular Mesh

The triangular mesh is registered to the ODE framework in the method PhysicalSimulator::register_triangular_mesh. triangular_mesh_ is the dGeomID object.

#### Collision Detection

The ODE library supports the method dCollide which handles the collision detection. The collision detection is carried out per frame. In the loop, the various properties are set to the collision points. The variable bounce in the 14th line corresponds to the reflection coefficient.

in English

## 動画

こんな感じ（アップロードしたら画質が悪くなった）。

## 開発環境

1. Mac OS X 10.8.2
2. プロセッサ：3.06 GHz Intel Core 2 Duo
3. メモリ：4GB
4. Xcode4.5.1

## 主要ライブラリ

1. boost-1.51.0
2. pcl-1.6.0
3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
4. ode-0.12

## ソース

こちらです。自分のマシーンでしか動作確認していません。

## コンパイルオプション

パス 目的
/usr/include/ni/ OpenNIヘッダー
/opt/local/include/ boost/odeヘッダー
/usr/local/include/pcl-1.6/ PCLヘッダー
/opt/local/include/eigen3/ eigenヘッダー(PCLがこれに依存している）
/opt/local/include/vtk-5.10/ vtkヘッダー(PCLがこれに依存している）

Library Search Path:
パス 目的
/opt/local/lib boostライブラリ
/usr/local/lib PCLライブラリ

 -lboost_thread-mt -lboost_filesystem-mt -lboost_unit_test_framework-mt -lboost_chrono-mt -lboost_program_options-mt -lboost_system-mt /usr/lib/libOpenNI.dylib -framework OpenGL -framework GLUT -lpcl_filters -lpcl_kdtree -lpcl_segmentation -lpcl_surface -lpcl_search -lpcl_features -lpcl_common -lode -lpcl_io

## ソースの概要

#### ポイントクラウドへの変換

Device::create_cloudで行います。カラー画像描画のためのポイントクラウド（cloud_）と三角メッシュ算出用のポイントクラウド（cloud_xyz_）の２種類を生成しています。 25,26行目でRGB値を設定しています。29,30行目でデプスから３次元座標への変換を行っています。33行目で三角メッシュ用のポイントクラウドを取得しています。

#### カラー画像の描画

カラー画像用のポイントクラウド（cloud_）は、OpenGLのglInterleavedArraysに以下（5行目）のように渡されます。bufcloud_の先頭ポインタ、sizecloud_のサイズです。

#### ボールの投入

ボールはODEのオブジェクトです。これは、剛体力学の計算に関与するdBodyと衝突計算に関与するdGeomから構成されます。 dSpheredGeomの子クラスです。 そして、ボールはPhysicalSimulator::create_balls()で環境内に投入されます。 20行目のworld_dWorldのオブジェクトです。41行目のspace_dSpaceのオブジェクトです。ODEでは、剛体力学の計算を必要とするオブジェクトはdWorldに、衝突計算を行うオブジェクトはdSpaceに登録しなければなりません。

## 2012年10月6日土曜日

### マジックナンバー70

3次までテイラー展開すると

ここで、 を仮定し、 で置き換えると

となります。これの実数解は0.6988854...です。おおよそ0.7です。 つまり となります。両辺に100をかければ

となります。これがマジックナンバーの正体です（たぶん）。 上の計算から、nは大きいほど、xは小さいほど精度は上がることがわかります。