"High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm," Opt. Eng., (2007)

[11] S Zhang and S-T Yau, "High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm," Opt. Eng., 46(11), 113603, 2007; doi:10.1117/1.2802546

This paper describes a high-resolution, real-time, threedimensional shape measurement system using the modified two-plusone phase-shifting algorithm. The data acquisition speed is as high as 60 frames/ s with an image resolution of 640
480 pixels per frame. Experiments demonstrated that the system was able to acquire the dynamic changing objects such as facial geometric shape changes when the subject is speaking, and the modified two-plus-one phase-shifting algorithm can further alleviate the error due to motion. Applications of this system include manufacturing, online inspection, medical imaging, compute vision, and computer graphics. 

"Phase error compensation for a 3-D shape measurement system based on the phase shifting method," Opt. Eng., (2007)

[10] S Zhang and PS Huang, "Phase error compensation for a 3-D shape measurement system based on the phase shifting method," Opt. Eng., 46(6), 063601, 2007; doi:10.1117/1.2746814

This paper describes a novel phase error compensation method for reducing the measurement error caused by nonsinusoidal waveforms in phase-shifting methods. For 3-D shape measurement systems using commercial video projectors, the nonsinusoidal waveform of the projected fringe patterns as a result of the nonlinear gamma of projectors causes significant phase measurement error and therefore shape measurement error. The proposed phase error compensation method is based on our finding that the phase error due to the nonsinusoidal waveform depends only on the nonlinearity of the projector’s gamma. Therefore, if the projector’s gamma is calibrated and the phase error due to the nonlinearity of the gamma is calculated, a lookup table that stores the phase error can be constructed for error compensation. Our experimental results demonstrate that by using the proposed method, the measurement error can be reduced by 10 times. In addition to phase error compensation, a similar method is also proposed to correct the nonsinusoidality of the fringe patterns for the purpose of generating a more accurate flat image of the object for texture mapping. While not relevant to applications in metrology, texture mapping is important for applications in computer vision and computer graphics.

"Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector," Appl. Opt., (2007)

[9] S Zhang and S-T Yau, "Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector," Appl. Opt., 46(1), 36-43, 2007; doi: 10.1364/AO.46.000036

A structured light system using a digital video projector is widely used for 3D shape measurement. However, the nonlinear
of the projector causes the projected fringe patterns to be nonsinusoidal, which results in phase error and therefore measurement error. It has been shown that, by using a small look-up table (LUT), this type of phase error can be reduced significantly for a three-step phase-shifting algorithm. We prove that this algorithm is generic for any phase-shifting algorithm. Moreover, we propose a new LUT generation method by analyzing the captured fringe image of a flat board directly. Experiments show that this error compensation algorithm can reduce the phase error to at least 13 times smaller.

"Multilevel quality-guided phase unwrapping algorithm for real-time 3-D shape reconstruction,"Appl. Opt., (2007)

[8] S Zhang, X Li and S-T Yau, "Multilevel quality-guided phase unwrapping algorithm for real-time 3-D shape reconstruction,"Appl. Opt. 46(1), 50-57, 2007 (Selected for February 5, 2007 issue of The Virtual Journal for Biomedical Optics); doi: 10.1364/AO.46.000050

A multilevel quality-guided phase unwrapping algorithm for real-time 3D shape measurement is presented. The quality map is generated from the gradient of the phase map. Multilevel thresholds are used to unwrap the phase level by level. Within the data points in each level, a fast scan-line algorithm is employed. The processing time of this algorithm is approximately 18.3 ms for an image size of 640
480 pixels in an ordinary computer. We demonstrate that this algorithm can be implemented into our real-time 3D shape measurement system for real-time 3D reconstruction. Experiments show that this algorithm improves the previous scan-line phase unwrapping algorithm significantly although it reduces its processing speed slightly

"High-resolution real-time three-dimensional shape measurement," Opt. Eng., (2006)

[7] S Zhang and PS Huang, "High-resolution real-time three-dimensional shape measurement," Opt. Eng., 45(12), 123601, 2006 ; doi: 10.1117/1.2402128 

We describe a high-resolution, real-time 3-D shape measurement system based on a digital fringe projection and phase-shifting technique. It utilizes a single-chip digital light processing projector to project computer-generated fringe patterns onto the object, and a high-speed CCD camera synchronized with the projector to acquire the fringe images at a frame rate of 120 frames/ s. A color CCD camera is also used to capture images for texture mapping. Based on a three-step phaseshifting technique, each frame of the 3-D shape is reconstructed using three consecutive fringe images. Therefore the 3-D data acquisition speed of the system is 40 frames/ s. With this system, together with the fast three-step phase-shifting algorithm and parallel processing software we developed, high-resolution, real-time 3-D shape measurement is realized at a frame rate of up to 40 frames/ s and a resolution of 532
500 points per frame

"GPU-assisted high-resolution, real-time 3-D shape measurement," Opt. Express, (2006)

[6] S Zhang, D Royer and S-T Yau, "GPU-assisted high-resolution, real-time 3-D shape measurement," Opt. Express, 14, 9120-9129, 2006 (Selected for November 13, 2006 issue of The Virtual Journal for Biomedical Optics); doi: 10.1364/OE.14.009120

This paper describes a Graphics Processing Unit (GPU)-assisted real-time three-dimensional shape measurement system. Our experiments demonstrated that the absolute coordinates calculation and rendering speed of a GPU is more than four times faster than that of a dual CPU workstation with the same graphics card. By implementing the GPU into our system, we realized simultaneous absolute coordinate acquisition, reconstruction and display at 30 frames per second with a resolution of approximately 266K points per frame. Moreover, a 2+1 phase-shifting algorithm was employed to alleviate the measurement error caused by motion. Applications of the system include medical imaging, manufacturing, entertainment, and security.

"Novel method for structured light system calibration," Opt. Eng., (2006)

[4] S Zhang and PS Huang, "Novel method for structured light system calibration," Opt. Eng., 45(8), 083601, 2006; doi:10.1117/1.2336196

System calibration, which usually involves complicated and time-consuming procedures, is crucial for any 3-D shape measurement system. In this work, a novel systematic method is proposed for accurate and quick calibration of a 3-D shape measurement system we developed based on a structured light technique. The key concept is to enable the projector to “capture” images like a camera, thus making the calibration of a projector the same as that of a camera. With this new concept, the calibration of structured light systems becomes essentially the same as the calibration of traditional stereovision systems, which is well established. The calibration method is fast, robust, and accurate. It signifi- cantly simplifies the calibration and recalibration procedures of structured light systems. This work describes the principle of the proposed method and presents some experimental results that demonstrate its performance.

"Fast three-step phase-shifting algorithm," Appl. Opt., (2006)

[3] PS Huang and S Zhang, "Fast three-step phase-shifting algorithm," Appl. Opt., 45(21), 5086-5091, 2006 (Cover Feature); doi: 10.1364/AO.45.005086

We propose a new three-step phase-shifting algorithm, which is much faster than the traditional three-step algorithm. We achieve the speed advantage by using a simple intensity ratio function to replace the arctangent function in the traditional algorithm. The phase error caused by this new algorithm is compensated for by use of a lookup table. Our experimental results show that both the new algorithm and the traditional algorithm generate similar results, but the new algorithm is 3.4 times faster. By implementing this new algorithm in a high-resolution, real-time three-dimensional shape measurement system, we were able to achieve a measurement speed of 40 frames per second at a resolution of 532
500 pixels, all with an ordinary personal computer.

"High-resolution, real-time 3-D absolute coordinate measurement based on a phase-shifting method,"Opt. Express, (2006)

[5] S Zhang and S-T Yau, "High-resolution, real-time 3-D absolute coordinate measurement based on a phase-shifting method,"Opt. Express, 14, 2644-2649, 2006 ; doi: 10.1364/OE.14.002644

We describe a high-resolution, real-time 3D absolute coordinate measurement system based on a phase-shifting method. It acquires 3D shape at 30 frames per second (fps), with 266K points per frame. A tiny marker is encoded in the projected fringe pattern, and detected by software from the texture image and the gamma map. Absolute 3D coordinates are obtained from the detected marker position and the calibrated system parameters. To demonstrate the performance of the system, we measure a hand moving over a depth distance of approximately 700 mm, and human faces with expressions. Applications of such a system include manufacturing, inspection, entertainment, security, medical imaging.

"Trapezoidal phase-shifting method for three-dimensional shape measurement," Opt. Eng., (2005)

[2] PS Huang, S Zhang, and F-P Chiang, "Trapezoidal phase-shifting method for three-dimensional shape measurement," Opt. Eng. 44(12), 123601, 2005; doi:10.1117/1.2147311

We propose a novel structured light method, namely a trapezoidal phase-shifting method, for 3-D shape measurement. This method uses three patterns coded with phase-shifted, trapezoidalshaped gray levels. The 3-D information of the object is extracted by direct calculation of an intensity ratio. Compared to traditional intensityratio-based methods, the vertical or depth resolution is six times better. Also, this new method is significantly less sensitive to the defocusing effect of the captured images, which makes large-depth 3-D shape measurement possible. If compared to sinusoidal phase-shifting methods, the resolution is similar, but the data processing speed is at least 4.5 times faster. The feasibility of this method is demonstrated in a previously developed real-time 3-D shape measurement system. The reconstructed 3-D results show similar quality to those obtained by the sinusoidal phase-shifting method. However, since the data processing speed is much faster
4.6 ms per frame, both image acquisition and 3-D reconstruction can be done in real time at a frame rate of 40 fps and a resolution of 532
500 points. This real-time capability allows us to measure dynamically changing objects, such as human faces. The potential applications of this new method include industrial inspection, reverse engineering, robotic vision, computer graphics, medical diagnosis, etc. 

"High-resolution acquisition, learning and transfer dynamic 3D facial expressions," Computer Graphics Forum, (2004)

[1] Y Wang, X Huang, C-S Lee, S Zhang, Z Li, D Samaras, D Metaxas, A Elgammal, and P Huang, "High-resolution acquisition, learning and transfer dynamic 3D facial expressions," Computer Graphics Forum, 23(3), 2004; doi: 10.1111/j.1467-8659.2004.00800.x

Synthesis and re-targeting of facial expressions is central to facial animation and often involves significant manual work in order to achieve realistic expressions, due to the difficulty of capturing high quality dynamic expression data. In this paper we address fundamental issues regarding the use of high quality dense 3-D data samples undergoing motions at video speeds, e.g. human facial expressions. In order to utilize such data for motion analysis and re-targeting, correspondences must be established between data in different frames of the same faces as well as between different faces. We present a data driven approach that consists of four parts: 1) High speed, high accuracy capture of moving faces without the use of markers, 2) Very precise tracking of facial motion using a multi-resolution deformable mesh, 3) A unified low dimensional mapping of dynamic facial motion that can separate expression style, and 4) Synthesis of novel expressions as a combination of expression styles. The accuracy and resolution of our method allows us to capture and track subtle expression details. The low dimensional representation of motion data in a unified embedding for all the subjects in the database allows for learning the most discriminating characteristics of each individual’s expressions as that person’s “expression style”. Thus new expressions can be synthesized, either as dynamic morphing between individuals, or as expression transfer from a source face to a target face, as demonstrated in a series of experiments.