"Towards superfast three-dimensional optical metrology with digital micromirror device (DMD) platforms," Opt. Eng., (2015)

[67] T. Bell* and S. Zhang, "Towards superfast three-dimensional optical metrology with digital micromirror device (DMD) platforms," Opt. Eng., 53(11), 112206, 2014; doi: 10.1117/1.OE.53.11.112206

Decade-long research efforts toward superfast three-dimensional (3-D) shape measurement leveraging the digital micromirror device (DMD) platforms are summarized. Specifically, we will present the following technologies: (1) high-resolution real-time 3-D shape measurement technology that achieves 30 Hz simultaneous 3-D shape acquisition, reconstruction, and display with more than 300,000 points per frame; (2) superfast 3-D optical metrology technology that achieves 3-D measurement at a rate of tens of kilohertz utilizing the binary defocusing method we invented; and (3) the improvement of the binary defocusing technology for superfast and high-accuracy 3-D optical metrology using the DMD platforms. Both principles and experimental results are presented.

"High-resolution, real-time three-dimensional shape measurement on graphics processing unit," Opt. Eng., (2014)

[64] N. Karpinsky*, M. Hoke*, V. Chen*, and S.  Zhang, "High-resolution, real-time three-dimensional shape measurement on graphics processing unit," Opt. Eng. 53(2), 024105, 2014;  doi: 10.1117/1.OE.53.2.024105

Abstract

A three-dimensional (3-D) shape measurement system that can simultaneously achieve 3-D shape acquisition, reconstruction, and display at 30 frames per second (fps) with 480,000 measurement points per frame is presented. The entire processing pipeline was realized on a graphics processing unit (GPU) without the need of substantial central processing unit (CPU) power, making it achievable on a portable device, namely a laptop computer. Furthermore, the system is extremely inexpensive compared with similar state-of-art systems, making it possible to be accessed by the general public. Specifically, advanced GPU techniques such as multipass rendering and offscreen rendering were used in conjunction with direct memory access to achieve the aforementioned performance. The developed system, implementation details, and experimental results to verify the performance of the proposed technique are presented.

"Digital micromirror transient response influence on superfast 3D shape measurement," Opt. Laser Eng., (2014)

[63] Y. Wang*, B.  Bhattacharya, E. H. Winer, P. Kosmicki, W. H. El-Ratal, and S. Zhang, "Digital micromirror transient response influence on superfast 3D shape measurement," Opt. Laser Eng. 58, 19-26, 2014; doi: 10.1016/j.optlaseng.2014.01.015

Abstract

Nowadays, the high speed (e.g., kilo-Hertz) refreshing rate of the digital micro-mirror device (DMD) has enabled superfast 3D shape measurement using the binary defocusing technique. This research finds that when the system reaches its extreme binary pattern refreshing rate, the transient response of the DMD induces a coupling effect (i.e., two neighboring patterns blend together) that may cause substantial measurement error. Since this transient response repeats itself, this systematic measurement error is substantially reduced to a negligible level when the timing between the projector and the camera is properly adjusted. Experimental results are presented to demonstrate the observed phenomena, and the success of utilizing the proposed method to overcome the problems associated with the transient response of the DMD.

"Objective comparison toolmarks from the cutting surfaces of slip-joint pliers," AFTE Journal, (2014)

[57] T. Grieve, L. S. Chumbley, J. Kreiser, M. Morris,  L. Ekstrand*, and S. Zhang, "Objective comparison toolmarks from the cutting surfaces of slip-joint pliers," AFTE Journal  46(2), 176-185, 2014

Abstract

Experimental results from a statistical analysis algorithm for objectively comparing toolmarks via data files obtained using optical profilometry data are described. The algorithm employed has successfully been used to compare striated marks produced by screwdrivers. In this study, quasi-striated marks produced by the cutting surfaces of slip-joint pliers were examined. Marks were made by cutting both copper and lead wire. Data files were obtained using an optical profilometer that uses focus variation to determine surface roughness. Early efforts using the comparative algorithm yielded inconclusive results when the comparison parameters used were the same as those employed successfully for screw-driver marks. Further experiments showed that the algorithm could successfully be used to separate known matches from non-matches by changing the comparison parameters. Results are presented from the analysis of the copper wires.

"High-resolution, high-speed, three-dimensional video imaging with digital fringe projection techniques," Journal of Visualized Experiments, (2013)

L. Ekstrand*, N. Karpinsky*, Y. Wang*, and S. Zhang, "High-resolution, high-speed, three-dimensional video imaging with digital fringe projection techniques," Journal of Visualized Experiments (JoVE), (82), e50421, 2013. (Associated with Video Illustrations) (invited); doi: 10.3791/50421

Digital fringe projection (DFP) techniques provide dense 3D measurements of dynamically changing surfaces. Like the human eyes and brain, DFP uses triangulation between matching points in two views of the same scene at different angles to compute depth. However, unlike a stereo-based method, DFP uses a digital video projector to replace one of the cameras. The projector rapidly projects a known sinusoidal pattern onto the subject, and the surface of the subject distorts these patterns in the camera’s field of view. Three distorted patterns (fringe images) from the camera can be used to compute the depth using triangulation.

Unlike other 3D measurement methods, DFP techniques lead to systems that tend to be faster, lower in equipment cost, more flexible, and easier to develop. DFP systems can also achieve the same measurement resolution as the camera. For this reason, DFP and other digital structured light techniques have recently been the focus of intense research (as summarized in1-5). Taking advantage of DFP, the graphics processing unit, and optimized algorithms, we have developed a system capable of 30 Hz 3D video data acquisition, reconstruction, and display for over 300,000 measurement points per frame. Binary defocusing DFP methods can achieve even greater speeds.

Diverse applications can benefit from DFP techniques. Our collaborators have used our systems for facial function analysis9, facial animation10, cardiac mechanics studies11, and fluid surface measurements, but many other potential applications exist. This video will teach the fundamentals of DFP techniques and illustrate the design and operation of a binary defocusing DFP system.

"Optimal fringe angle selection for digital fringe projection technique," Appl. Opt., (2013)

[55] Y. Wang*, and S. Zhang, "Optimal fringe angle selection for digital fringe projection technique," Appl. Opt. 52(29),  7094-7098, 2013; doi: 10.1364/AO.52.007094

Abstract

Existing digital fringe projection (DFP) systems mainly use either horizontal or vertical fringe patterns for three-dimensional shape measurement. This paper reveals that these two fringe directions are usually not optimal where the phase change is the largest to a given depth variation. We propose a novel and efficient method to determine the optimal fringe angle by projecting a set of horizontal and vertical fringe patterns onto a step-height object and by further analyzing two resultant phase maps. Experiments demonstrate the existence of the optimal angle and the success of the proposed optimal angle determination method.

"Intensity-optimized dithering technique for high-quality 3d shape measurement," Opt. Laser Eng., 2014

J. Dai, B. Li*, and S. Zhang, "Intensity-optimized dithering technique for high-quality 3d shape measurement," Opt. Laser Eng. 53, 79-85, 2014; doi: 10.1016/j.optlaseng.2013.08.015

Our previously proposed phase-based optimization method [1] has proven successful in improving the measurement quality when a dithering technique is used. This paper presents an intensity-based optimization method for 3D shape measurement with binary dithering techniques. Both simulations and experiments find that the phase-based optimization method can generate high-quality phase under a given condition, but it is sensitive to the amount of defocusing. In contrast, the proposed intensity-based optimization method can consistently generate high-quality phase with various amounts of defocusing.

"Some recent advances on superfast 3D shape measurement with digital binary defocusing techniques," Opt. Laser Eng. 2014

B. Li*, Y. Wang*, J. Dai, and W. Lohry*, and S. Zhang, "Some recent advances on superfast 3D shape measurement with digital binary defocusing techniques," Opt. Laser Eng. 54, 236-246, 2014 (invited); doi:10.1016/j.optlaseng.2013.07.010

The digital binary phase-shifting technique has been demonstrated for its merits over the conventional sinusoidal phase-shifting method in terms of measurement speed and simplicity. Yet, the measurement depth range is small when a squared binary method is used. Our recent research focuses on improving its measurement accuracy without sacrificing measurement speed, and increasing its depth range without losing measurement quality. This paper will summarize our recent work on the following three major areas: (a) realization of kHz 3D shape measurement with binary phase-shifting methods; (b) binary pattern improvement with pulse width modulation and binary dithering/halftoning techniques; and (c) applications of superfast 3D shape measurement techniques. Principle of each technique will be presented, and experimental results will be shown to verify its performance.

"High-quality fringe pattern generation using binary pattern optimization through symmetry and periodicity," Opt. Laser Eng., 2014

J. Dai, B. Li*, and S. Zhang, "High-quality fringe pattern generation using binary pattern optimization through symmetry and periodicity," Opt. Laser Eng., 52, 195-200, 2014; doi: 10.1016/j.optlaseng.2013.06.010

This paper presents a novel method to construct binary patterns for high-quality 3D shape measurement. The algorithm generates small patches using symmetry and periodicity, randomly initializes each pixels, optimizes the small patches through mutations, and finally tiles the optimized patches into full size patterns using again symmetry and periodicity. We will demonstrate that the proposed method can achieve substantial phase quality improvements over the dithering techniques for different amounts of defocusing.

"Three bit representation of three-dimensional range data," Appl. Opt. , (2013)

[54] N. Karpinsky*, Y. Wang*, and S. Zhang, "Three bit representation of three-dimensional range data," Appl. Opt. 52(11), 2286-2293, 2013; doi: 10.1364/AO.52.002286

Abstract

Our previous research has shown that 3D range data sizes can be substantially reduced if they are converted into regular 2D images using the Holoimage technique. Yet, this technique requires all 24 bits of a standard image to represent one 3D point, making it impossible for a regular 2D image to carry 2D texture information as well. This paper proposes an approach to represent 3D range data with 3 bits, further reducing the data size. We demonstrate that more than an 8.2∶1 compression ratio can be achieved with compression root-mean-square error of only 0.34%. Moreover, we can use another bit to represent a black-and-white 2D texture, and thus both 3D data and 2D texture images can be stored into an 8 bit grayscale image. Both simulation and experiments are presented to verify the performance of the proposed technique.

"Natural method for three-dimensional range data compression," Appl. Opt. , (2013)

[53] P. Ou and S. Zhang, "Natural method for three-dimensional range data compression," Appl. Opt. 52(9), 1857-1863, 2013;doi: 10.1364/AO.51.004058

Abstract

Prior studies on converting three-dimensional (3D) range data into regular two-dimensional (2D) color images using virtual fringe projection techniques showed great promise for 3D range data compression, yet they require resampling the raw scanned data. Due to this resampling, the natural 3D range data are altered and sampling error may be introduced. This paper presents a method that compresses the raw sampling points without modifications. Instead of directly utilizing the 3D recovered shape, this method compresses the s map, the scale factor of a perspective projection from a 3D space to a 2D space. The s map is then converted to 2D color image for further compression with existing 2D image compression techniques. By this means, 3D data obtained by 3D range scanners can be compressed into 2D images without any resampling, providing a natural and more accurate method of compressing 3D range data. Experimental results verified the success of the proposed method.

"3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique," Opt. Express , (2013)

[52] Y. Wang*, J. I. Laughner, I. R. Efimov, and S. Zhang, "3D absolute shape measurement of live rabbit hearts with a superfast two-frequency phase-shifting technique," Opt. Express 21(5), 5822-5832, 2013 (Cover feature)  (Selected for May 22, 2013 issue of The Virtual Journal for Biomedical Optics); doi: 10.1364/OE.21.005822

Abstract

This paper presents a two-frequency binary phase-shifting technique to measure three-dimensional (3D) absolute shape of beating rabbit hearts. Due to the low contrast of the cardiac surface, the projector and the camera must remain focused, which poses challenges for any existing binary method where the measurement accuracy is low. To conquer this challenge, this paper proposes to utilize the optimal pulse width modulation (OPWM) technique to generate high-frequency fringe patterns, and the error-diffusion dithering technique to produce low-frequency fringe patterns. Furthermore, this paper will show that fringe patterns produced with blue light provide the best quality measurements compared to fringe patterns generated with red or green light; and the minimum data acquisition speed for high quality measurements is around 800 Hz for a rabbit heart beating at 180 beats per minute.

"Phase-optimized dithering technique for high-quality 3D shape measurement," Opt. Laser Eng., (2013)

[51] J. Dai and S. Zhang, "Phase-optimized dithering technique for high-quality 3D shape measurement," Opt. Laser Eng. 51(6), 790-795, 2013; doi: 10.1016/j.optlaseng.2013.02.003

Abstract

Our recent study showed that the Bayer-dithering technique could substantially improve 3D measurement quality for the binary defocusing method. Yet, the dithering technique was developed to optimize the appearance or intensity representation, rather than the phase, of an image. This paper presents a framework to optimize the Bayer-dithering technique in phase domain by iteratively mutating the status (0 or 1) of a binary pixel. We will demonstrate that the proposed optimization technique can drastically reduce the phase error when the projector is nearly focused.

"Genetic method to optimize binary dithering technique for high-quality fringe generation," Opt. Lett. ,(2013)

[50] W. Lohry* and S. Zhang, "Genetic method to optimize binary dithering technique for high-quality fringe generation," Opt. Lett. 38(4), 540-542, 2013; doi: 10.1117/1.OE.51.11.113602

Abstract

The recently proposed dithering techniques could substantially improve measurement quality when fringes are wide, but offer limited improvement when fringes are narrow. This Letter presents a genetic algorithm to optimize the dithering technique for sinusoidal structured pattern representation. We believe both simulation and experimental results show that this proposed algorithm can substantially improve fringe quality for both narrow and wide fringe patterns. 

"3D range geometry video compression with the H.264 codec," Opt. Laser Eng. , (2013)

[49] N. Karpinsky* and S. Zhang, "3D range geometry video compression with the H.264 codec," Opt. Laser Eng. 51(5), 620-625, 2013; doi: 10.1016/j.optlaseng.2012.12.021

Abstract

Advances in three-dimensional (3D) scanning have enabled the real-time capture of high-resolution 3D videos. With these advances brings the challenge of streaming and storing 3D videos in a manner that can be quickly and effectively used. This research addresses this challenge by generalizing the Holovideo technique to video codecs that use the YUV color space such as the H.264 codec. With the H.264 codec, we have achieved a compression ratio of over 6086:1 (Holovideo to OBJ) with a reasonably high quality; utilizing an NVIDIA GeForce 9400 m GPU, we have realized 17 frames per second encoding, and 28 frames per second decoding speed, making it a viable solution for real-time 3D video compression.

"Accurate calibration for 3D shape measurement system using a binary defocusing technique," Opt. Laser Eng., (2013)

L. Merner*, Y. Wang*, and S. Zhang, "Accurate calibration for 3D shape measurement system using a binary defocusing technique," Opt. Laser Eng. 51(5), 514-519, 2013; doi: 10.1016/j.optlaseng.2012.10.015

This paper introduces a novel method to calibrate 3D shape measurement systems that use the binary defocusing technique. Specifically, this method calibrates the pixelwise z as low-order polynomial functions of absolute phase; (x, y) coordinates are calculated from camera calibration with known z value; and the camera is calibrated using the standard flat checkerboard method. Because this method does not require estimation of the projector’s parameters, it can be adopted for any phase measurement system including those employing out-of-focus projectors. Our experiment found that the root-mean squared (rms) error for the depth measurement is less than 70 mm when the measurement depth range is about 100 mm, which is at the same level of the calibration stage +-50 mm.

"Fourier transform profilometry using a binary area modulation technique," Opt. Eng., (2012)

W. Lohry* and S. Zhang, "Fourier transform profilometry using a binary area modulation technique," Opt. Eng. 51(11),113602, 2012; doi: 10.1117/1.OE.51.11.113602

A recent study found that it is very difficult to use the squared binary defocusing technique to eliminate the influence of third-order harmonics without compromising fringe quality, and thus it is challenging to utilize Fourier transform profilometry to achieve high-quality three-dimensional measurement. A novel approach is presented to effectively eliminate the third-order harmonics by modulating the squared binary structured patterns. Both simulation and experiments are presented to verify the performance of the proposed technique.

"Three-dimensional shape measurement with binary dithered patterns," Appl. Opt.,(2012)

Y. Wang* and S. Zhang, "Three-dimensional shape measurement with binary dithered patterns," Appl. Opt. 51(27), 6631-6636, 2012; doi: 10.1364/AO.51.006631

The previously proposed binary defocusing technique and its variations have proven successful for high-quality three-dimensional (3D) shape measurement when fringe stripes are relatively narrow, but they suffer if fringe stripes are wide. This paper proposes to utilize the binary dithering technique to conquer this challenge. Both simulation and experimental results show the phase error is always less than 0.6% even when the fringe stripes are wide and the projector is nearly focused.

"Three-dimensional range data compression using computer graphics rendering pipeline," Appl. Opt., (2012)

S. Zhang, "Three-dimensional range data compression using computer graphics rendering pipeline," Appl. Opt. 51(18), 4058-4064, 2012 (Cover feature); doi: 10.1364/AO.51.004058

This paper presents the idea of naturally encoding three-dimensional (3D) range data into regular two dimensional (2D) images utilizing computer graphics rendering pipeline. The computer graphics pipeline provides a means to sample 3D geometry data into regular 2D images, and also to retrieve the depth information for each sampled pixel. The depth information for each pixel is further encoded into red, green, and blue color channels of regular 2D images. The 2D images can further be compressed with existing 2D image compression techniques. By this novel means, 3D geometry data obtained by 3D range scanners can be instantaneously compressed into 2D images, providing a novel way of storing 3D range data into its 2D counterparts. We will present experimental results to verify the performance of this proposed technique.

"Composite phase-shifting algorithm for absolute phase measurement," Opt. Laser Eng., (2012)

S. Zhang, "Composite phase-shifting algorithm for absolute phase measurement," Opt. Laser Eng. 50, 1538-1541, 2012;doi: 10.1016/j.optlaseng.2012.06.005

This paper presents a method to recover absolute phase by using only four images: three phase-shifted patterns and one stair pattern. The stair pattern is designed in such a way that the stair changes are perfectly aligned with the phase jumps, and thus absolute phase can be recovered by referring to the stair pattern. Due to system noises and camera and/or projector blurring, a computational framework is also proposed. Because this technique only requires four fringe images for absolute phase recovery, it has the merit of measurement speed. And since the absolute phase is obtained, this technique is suitable for measuring step-height objects. We have developed a digital fringe projection system to verify the performance of the proposed technique.