"High-resolution, high-speed, three-dimensional video imaging with digital fringe projection techniques," Journal of Visualized Experiments, (2013)

L. Ekstrand*, N. Karpinsky*, Y. Wang*, and S. Zhang, "High-resolution, high-speed, three-dimensional video imaging with digital fringe projection techniques," Journal of Visualized Experiments (JoVE), (82), e50421, 2013. (Associated with Video Illustrations) (invited); doi: 10.3791/50421

Digital fringe projection (DFP) techniques provide dense 3D measurements of dynamically changing surfaces. Like the human eyes and brain, DFP uses triangulation between matching points in two views of the same scene at different angles to compute depth. However, unlike a stereo-based method, DFP uses a digital video projector to replace one of the cameras. The projector rapidly projects a known sinusoidal pattern onto the subject, and the surface of the subject distorts these patterns in the camera’s field of view. Three distorted patterns (fringe images) from the camera can be used to compute the depth using triangulation.

Unlike other 3D measurement methods, DFP techniques lead to systems that tend to be faster, lower in equipment cost, more flexible, and easier to develop. DFP systems can also achieve the same measurement resolution as the camera. For this reason, DFP and other digital structured light techniques have recently been the focus of intense research (as summarized in1-5). Taking advantage of DFP, the graphics processing unit, and optimized algorithms, we have developed a system capable of 30 Hz 3D video data acquisition, reconstruction, and display for over 300,000 measurement points per frame. Binary defocusing DFP methods can achieve even greater speeds.

Diverse applications can benefit from DFP techniques. Our collaborators have used our systems for facial function analysis9, facial animation10, cardiac mechanics studies11, and fluid surface measurements, but many other potential applications exist. This video will teach the fundamentals of DFP techniques and illustrate the design and operation of a binary defocusing DFP system.

"Optimal fringe angle selection for digital fringe projection technique," Appl. Opt., (2013)

[55] Y. Wang*, and S. Zhang, "Optimal fringe angle selection for digital fringe projection technique," Appl. Opt. 52(29),  7094-7098, 2013; doi: 10.1364/AO.52.007094

Abstract

Existing digital fringe projection (DFP) systems mainly use either horizontal or vertical fringe patterns for three-dimensional shape measurement. This paper reveals that these two fringe directions are usually not optimal where the phase change is the largest to a given depth variation. We propose a novel and efficient method to determine the optimal fringe angle by projecting a set of horizontal and vertical fringe patterns onto a step-height object and by further analyzing two resultant phase maps. Experiments demonstrate the existence of the optimal angle and the success of the proposed optimal angle determination method.

"Flexible real-time natural 2D color and 3D shape measurement," Opt. Express, 2013;

P. Ou, B. Li*, Y. Wang*, and S. Zhang, "Flexible real-time natural 2D color and 3D shape measurement," Opt. Express,21(14), 16736-16741, 2013; doi: 10.1364/OE.21.016736

The majority of existing real-time 3D shape measurement systems only generate non-nature texture (i.e., having illumination other than ambient lights) that induces shadow related issues. This paper presents a method that can simultaneously capture natural 2D color texture and 3D shape in real time. Specifically, we use an infrared fringe projection system to acquire 3D shapes, and a secondary color camera to simultaneously capture 2D color images of the object. Finally, we develop a flexible and simple calibration technique to determine the mapping between the 2D color image and the 3D geometry. Experimental results demonstrate the success of the proposed technique.  

"Three bit representation of three-dimensional range data," Appl. Opt. , (2013)

[54] N. Karpinsky*, Y. Wang*, and S. Zhang, "Three bit representation of three-dimensional range data," Appl. Opt. 52(11), 2286-2293, 2013; doi: 10.1364/AO.52.002286

Abstract

Our previous research has shown that 3D range data sizes can be substantially reduced if they are converted into regular 2D images using the Holoimage technique. Yet, this technique requires all 24 bits of a standard image to represent one 3D point, making it impossible for a regular 2D image to carry 2D texture information as well. This paper proposes an approach to represent 3D range data with 3 bits, further reducing the data size. We demonstrate that more than an 8.2∶1 compression ratio can be achieved with compression root-mean-square error of only 0.34%. Moreover, we can use another bit to represent a black-and-white 2D texture, and thus both 3D data and 2D texture images can be stored into an 8 bit grayscale image. Both simulation and experiments are presented to verify the performance of the proposed technique.

"Natural method for three-dimensional range data compression," Appl. Opt. , (2013)

[53] P. Ou and S. Zhang, "Natural method for three-dimensional range data compression," Appl. Opt. 52(9), 1857-1863, 2013;doi: 10.1364/AO.51.004058

Abstract

Prior studies on converting three-dimensional (3D) range data into regular two-dimensional (2D) color images using virtual fringe projection techniques showed great promise for 3D range data compression, yet they require resampling the raw scanned data. Due to this resampling, the natural 3D range data are altered and sampling error may be introduced. This paper presents a method that compresses the raw sampling points without modifications. Instead of directly utilizing the 3D recovered shape, this method compresses the s map, the scale factor of a perspective projection from a 3D space to a 2D space. The s map is then converted to 2D color image for further compression with existing 2D image compression techniques. By this means, 3D data obtained by 3D range scanners can be compressed into 2D images without any resampling, providing a natural and more accurate method of compressing 3D range data. Experimental results verified the success of the proposed method.

"3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique," Opt. Express , (2013)

[52] Y. Wang*, J. I. Laughner, I. R. Efimov, and S. Zhang, "3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique," Opt. Express 21(5), 5822-5832, 2013 (Cover feature)  (Selected for May 22, 2013 issue of The Virtual Journal for Biomedical Optics); doi: 10.1364/OE.21.005822

Abstract

This paper presents a two-frequency binary phase-shifting technique to measure three-dimensional (3D) absolute shape of beating rabbit hearts. Due to the low contrast of the cardiac surface, the projector and the camera must remain focused, which poses challenges for any existing binary method where the measurement accuracy is low. To conquer this challenge, this paper proposes to utilize the optimal pulse width modulation (OPWM) technique to generate high-frequency fringe patterns, and the error-diffusion dithering technique to produce low-frequency fringe patterns. Furthermore, this paper will show that fringe patterns produced with blue light provide the best quality measurements compared to fringe patterns generated with red or green light; and the minimum data acquisition speed for high quality measurements is around 800 Hz for a rabbit heart beating at 180 beats per minute.

"Phase-optimized dithering technique for high-quality 3D shape measurement," Opt. Laser Eng., (2013)

[51] J. Dai and S. Zhang, "Phase-optimized dithering technique for high-quality 3D shape measurement," Opt. Laser Eng. 51(6), 790-795, 2013; doi: 10.1016/j.optlaseng.2013.02.003

Abstract

Our recent study showed that the Bayer-dithering technique could substantially improve 3D measurement quality for the binary defocusing method. Yet, the dithering technique was developed to optimize the appearance or intensity representation, rather than the phase, of an image. This paper presents a framework to optimize the Bayer-dithering technique in phase domain by iteratively mutating the status (0 or 1) of a binary pixel. We will demonstrate that the proposed optimization technique can drastically reduce the phase error when the projector is nearly focused.

"Genetic method to optimize binary dithering technique for high-quality fringe generation," Opt. Lett. ,(2013)

[50] W. Lohry* and S. Zhang, "Genetic method to optimize binary dithering technique for high-quality fringe generation," Opt. Lett. 38(4), 540-542, 2013; doi: 10.1117/1.OE.51.11.113602

Abstract

The recently proposed dithering techniques could substantially improve measurement quality when fringes are wide, but offer limited improvement when fringes are narrow. This Letter presents a genetic algorithm to optimize the dithering technique for sinusoidal structured pattern representation. We believe both simulation and experimental results show that this proposed algorithm can substantially improve fringe quality for both narrow and wide fringe patterns. 

"3D range geometry video compression with the H.264 codec," Opt. Laser Eng. , (2013)

[49] N. Karpinsky* and S. Zhang, "3D range geometry video compression with the H.264 codec," Opt. Laser Eng. 51(5), 620-625, 2013; doi: 10.1016/j.optlaseng.2012.12.021

Abstract

Advances in three-dimensional (3D) scanning have enabled the real-time capture of high-resolution 3D videos. With these advances brings the challenge of streaming and storing 3D videos in a manner that can be quickly and effectively used. This research addresses this challenge by generalizing the Holovideo technique to video codecs that use the YUV color space such as the H.264 codec. With the H.264 codec, we have achieved a compression ratio of over 6086:1 (Holovideo to OBJ) with a reasonably high quality; utilizing an NVIDIA GeForce 9400 m GPU, we have realized 17 frames per second encoding, and 28 frames per second decoding speed, making it a viable solution for real-time 3D video compression.