2012年8月27日月曜日

Mountain LionはX11を含まない。

Point Cloud Library(PCL)のViewerを起動したら、こんなエラーが出ました。

ERROR: In /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_graphics_vtk5/vtk5/work/VTK/Rendering/vtkXOpenGLRenderWindow.cxx, line 1496 vtkXOpenGLRenderWindow (0x7ffc10445250): bad X server connection. DISPLAY=Segmentation fault: 11

bad X server connectionとあるので、Applicationディレクトリ内のX11をクリックしたら、Mountain LionはX11を含みませんという主旨のダイアログがでました。必要ならここからインストールできるよとあるので、それに従ってインストール。 直りました。

2012年8月25日土曜日

OpenNI MultiViewer

Introduction

Recently, I bought a second Xtion Pro Live. In this page I describe a multi-viewer which captures images simultaneously from two devices. The viewer derives a 3D point cloud from a depth image, and displays it with corresponding colors obtained from an RGB image using the OpenGL.

Development Environment

  1. iMac
  2. Mac OS X 10.8.1
  3. Processor:3.06GHz Intel Core 2 Duo
  4. Memory:4GB
  5. Xcode4.4.1

Primary Libraries

  1. Boost
  2. OpenNI
  3. OpenGL

Source Code

Here is my source code. I confirmed that it works well on my machine.

Compile Options

Header Search Path:
Path Purpose
/usr/include/ni/ to include the OpenNI headers
/opt/local/include/ to include the Boost headers

Library Search Path:
Path Purpose
/opt/local/lib to link with the Boost libraries

To use the OpenGL, I also set "-framework OpenGL" and "-framework GLUT." More detailed information is in my source code.

Usage

An executable file name is "OpenNiMultiViewer." Running it with an argument --help, it prints the following statements. After connecting two devices with the computer, running it without any arguments, it opens two viewers for images captured by devices. To quit the application, type an escape key under the situation where the application focuses on viewers.

Addition

  1. To change the number of devices, you have to extend my code.
  2. My application paints white color in the region where the device is unable to take the depth.
  3. The Xtion Pro Live uses infrared ray to take the depth. If there is more than one device, the number of regions where the devices are unable to take the depth increases because of interference between infrared rays.

OpenNIのMultiViewer

はじめに

Xtion Pro Liveの2台目を購入しました。2台同時に描画できるMutiViewerを実装したので紹介します。デプス画像から3次元点群を導出し、各点にRGB画像から取得した色を付けて描画します。

開発環境

  1. Mac OS X 10.8.1
  2. プロセッサ:3.06GHz Intel Core 2 Duo
  3. メモリ:4GB
  4. Xcode4.4.1

主要ライブラリ

  1. Boost
  2. OpenNI
  3. OpenGL

ソース

こちらです。自分のマシーンでしか動作確認してません。

コンパイルオプション

Header Search Path:
パス 目的
/usr/include/ni/ OpenNIヘッダー
/opt/local/include/ Boostヘッダー

Library Search Path:
パス 目的
/opt/local/lib Boostライブラリ

その他、OpenGLを使用するため、-framework OpenGLなどを設定しています。詳細はソースをご覧ください。

使い方

実行ファイ名は、OpenNiMultiViewerです。引数--helpを付けて実行すると以下を出力します。 Xtion Pro Liveを2台接続し引数なしで実行すると、窓が2つ開き映像が映し出されます。--width/--heightの値はデフォルト値でしか動作確認していません。 窓にフォーカスがある状態で、キーボードから「ecs」を打つと終了します。

補足

  1. Xtion Pro Liveの台数を可変にするには手直しが必要です。
  2. 深さを取得できない領域は白抜きになります。
  3. Xtion Pro Liveは赤外線を使用します。2台以上あるとお互いの赤外線が干渉するため、深さを取得できない領域が多くなります。

2012年8月5日日曜日

OpenNI Application "Bouncing Ball"

Introduction

I implemented an application which
  1. generates a point cloud from a depth data captured by the Xtion Pro Live,
  2. calculates a normal vector for each point in the point cloud using the Point Cloud Library(PCL),
  3. introduces virtual balls into an environment on the viewer,
  4. and simulates collisions between the balls and the objects in the environment.
The application does not retrieve normal vectors from each frame, but from a certain frame just one time. The simulation is executed in real time under those normal vectors. I used the OpenGL library to draw frames.

Development Environment

  1. Mac OS X 10.7.4
  2. Processor:3.06 GHz Intel Core 2 Duo
  3. Memory:4GB
  4. Xcode4.3.3

Primary Libraries

  1. Boost-1.50.0
  2. PCL-1.6.0
  3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
  4. Sensor-Bin-MacOSX-v5.1.2.1

Source Codes

Here is my source codes. I confirmed that the application works well on my machine.

Compile Options

Header Search Path:
Path Purpose
/usr/include/ni/ to include the OpenNI headers
/opt/local/include/ to include the Boost headers
/usr/local/include/pcl-1.6/ to include the PCL headers
/opt/local/include/eigen3/ to include the Eigen headers on which the PCL depends
/opt/local/include/vtk-5.10/ to include the VTK headers on which the PCL depends

Library Search Path:
Path Purpose
/opt/local/lib to link with the Boost libraries
/usr/local/lib to link with the PCL libraries

In addition, to use the OpenGL, I set "-framework OpenGL." More detailed information is in my source codes.

Usage

The executable file name is OpenNiViewer. Running it without any arguments, it prints the following statements. Running it with setting the path to a file "SamplesConfig.xml" to --config, it opens a viewer for a video captured by the Xtion Pro Live. The file "SamplesConfig.xml" is enclosed in my source codes. I don't verify the behavior of the application with values other than default ones for --width and --height. Typing "p" on the keyboard under the situation where the application focuses on the viewer, the derivation of normal vectors begins. After seeing "Normals has been extracted" on the standard output, typing "i" yields virtual balls in the captured scenes, and the interactions between the balls and the objects in the environment are drawn in real time. To quit the application, type the escape key.

Brief Overview of Source Codes

View Point of OpenGL

The view point of the OpenGL is as follows: A unit of length in the application is mm. You need to change the argument in the 8th line according to your environment.

Transforming Depth into Point Cloud

A depth data is transformed into a point cloud in the method OpenNiDevice::create_cloud. I generate two point clouds, one (cloud_) is for displaying the color video and another (cloud_xyz_) is for calculating normal vectors. RGB values are set in the 26th, 27th, and 28th lines. The depth data is transformed into the point cloud in the 33th line. The point cloud for calculating normal vectors is created in the 34th line.

Drawing Color Image

The point cloud used for the color video, cloud_, is passed to the OpenGL method glInterleavedArrays as

Derivation of Normal Vectors

The cloud used for deriving normal vectors (cloud_xyz_) is processed in the method OpenNiDevice::calculate_normals. The method initialize in the 6th line is as follows: The region of the local nearest search is set in the 9th line. This argument varies depending on your environment. The plane is calculated using points inside the region. The normal vector of the plane is what we want. The normal vectors is derived with each typing "p" on the keyboard. If you change position of the Xtion Pro Live, you have to type "p" again.

Configuration of Balls

The virtual balls is registered in the method OpenNiViewer::register_balls. The radius of the ball is set in the 25th line. The initial position, the initial velocity, and the acceleration are set in the 31th, 32th, and 33th lines, respectively. These values also vary depending on your environment.

Configuration of Walls

A wall is placed at the front of the viewer to avoid the balls disappearing from view.

Search of Normal Vectors

For each frame, the normal vectors located adjacent to the balls are detected by means of the kd-tree. kdtree_ in the 8th line is the object of the class pcl::KdTreeFLANN<pcl::PointXYZ>. Because the object returns nan if it fails to detect neighbors, the returned value is checked in the 12th line.

2012年8月4日土曜日

Mountain Lionへのアップデート

Mountain Lionへアップデートしたら、デプスセンサXtion Pro Liveが起動しなくなりました。起動時にこんな警告が出ます。
ネットで調べると、primesensorのドライバが動作していないようです。バイナリ版をインストールし直しました。 念のため、OpenNIもインストールし直します。 無事動くようになりました。

さて、Xcodeを4.3から4.4へアップデートしたら、macportsがこけました。 解決方法はこちらです。