Thermal Image formation process

Introduction:

Thermal energy is transferred through the means of conduction, convection, and radiation. All real sources reflect, transmit, and absorb radiant energy at different wavelengths, which accounts for 100% of the total radiant incident to the body.  A blackbody is essentially an opaque object in the spectrum under consideration, which means it has a high emissivity. Skin acts as a near-ideal blackbody, with an emissivity in the range of 0.91-0.98 depending on sex and measurement location [Steketee73]; ideal black bodies absorb all radiation incident on them and therefore have an emissivity ε = 1.  Similar to the rate of thermal energy emission, the blackbody radiations spectral distribution depends only on the absolute temperature of the blackbody [Infratec09].

To acquire a thermal image, an object that emits thermal IR radiation is placed in from of an IR lens as in Figure 1.  The IR lens focuses the radiation onto the focal plane array (FPA) IR sensitive detectors.  The FPA utilizes the photoelectric effect to generate electrical signal. The video processor converts the data into a 2D video frame, which is then captured via a frame-grabber in the computer.  These images are returned as gray-levels, where the intensity of the gray level is proportional to the temperature on the objects surface [Gunapala06, Rogalski06a, Rogalski06b]. The CVIP lab uses a Quantum type detector, which has higher detectivity and faster response speeds than thermal type detectors.  Quantum detectors detect electrical current when free electron-hole pairs are created resultant when the detector absorbs photons in a specified range of wavelengths; this is a direct result of the photoelectric effect.

Fig. 1:    The image formation process

Experimental Setup:

The CVIP lab setup uses a cooled (70K) Indigo Phoenix QWIP LWIR Camera System with Real Time Imaging Electronics (Product #420-0011-007, Rev. 120) and Talon Ultra 5.2 image acquisition software (Ver. 4.5.1.27). This setup also uses the Advisor Vital Signs Monitor to measure the pulse signal with an electrocardiography (EKG) device attached to the subject’s chest.  Data are saved using a Data Translation DT9816 16-bit A/D converter.

The CVIP Lab’s thermal camera is capable of detecting the minute temperature variations emanating from superficial and/or large vasculature structures.  Thermal imaging and computer vision techniques can be used to reveal the underlying vascular structures as well as pulse information for the facial regions.  For recognition, the entire vascular network is used and the bifurcation points are combined with fingerprint recognition techniques.  For work with vital signs analysis, wavelets are used to extract the heart rate from a subject.

Methods:

Facial Recognition

For identification purposes, bifurcation points–i.e. points where a ridge lines fork–on the vascular map are identified and saved into a feature-vector based on the K-Nearest Neighbor (KNN) method of comparison.  A typical face contains 50-80 legitimate thermal minutia points (TMPs).  The feature vectors (LM) contain the local orientation of the TMP, the distances to the K-nearest neighbors, angular information between the TMP and its neighbors.

 

The acquisition and vasculature extraction process can lead to the formation of spurious TMPs, which appear as a result of “bubbles” appearing in the vascular network prior to thinning as illustrated in Figure 2.  These can be removed using additional morphological operations.

Fig. 2: Three highlighted spurious TMPs showing (left) the original vascular structure prior to thinning, (middle) the results of thinning, and (right) the results after applying the dilation to the original image.

 

Issues including spurious TMPs, unaccounted global parameters (rotation and scale, primarily), camera type, and minor coding errors.  The spurious TMPs are now removed prior to encoding using the process described in the previous section.  The type of thermal camera also affects the encoding process.  At the moment, it is necessary to use the same camera model and type for the gallery and probe image captures to ensure recognition.  The use of additional cameras to populate the gallery or probe images will require unique filtering steps for those cameras to compensate.  Global parameters, such as overall rotation and scale dramatically affect recognition performance and are corrected manually until coding solutions are implemented successfully.  After implementing solutions to most of these recognition results improve as in Figure 3.

Fig. 3: Recognition performance compensating for global parameters via manual image adjustment.

Vital Signs

Vascular maps are used again for this experimental work, although the vascular lines are used to select and track a measurement site on the face to acquire the heart rate and arterial pulse waveforms.  Figure 4 illustrates the selection of one of these structures under close and far poses.

Fig. 4. Sample thermal images and overlays consisting of the measurement sites and vascular maps at (a) near and (b) distant poses.

For the experiments subjects are connected to an EKG and are recorded sitting in two poses at rest.  Thermal video and EKG sampling start simultaneously. While acquiring data, the operator denotes a rough average or range of heart rates for later reference if the results appear erroneous.  Fig. 5 illustrates the equipment setup.

Fig. 5. Experimental setup with thermal camera, EKG and data acquisition devices.

 

Despite the effects of the small movements, the vessels maintain constant relative shape and are tracked throughout the video.  The tracking information is used to create a signal to be filtered using wavelets. The underlying principle of the continuous wavelet trans-form (CWT) and multi-resolution analysis (MRA) is that wavelets, which are compactly supported in time, are used instead of sinusoids to reveal spectral components at specific instances in time.  The result from a 1D signal is a 2D matrix of wavelet coefficients, with rows corresponding to scales and columns corresponding to translations in time, where the scales vary inversely with frequency.

In contrast to the Fourier trans-form, the CWT preserves the spatial or temporal location of the frequencies present in the signal, which is ideal for analyzing time-varying pulse signals. The CWT is formulated using (2). The location of XW is given by s and τ in 2D space.  The signal can be reconstructed with the inverse continuous wavelet transform (ICWT) in (3) using the wavelet coefficients in XW, mother wavelet, and admissibility constant, cψ.

(2)

(3)

 

The magnitudes of the spectral components from the multiple measurement sites are combined using a weighted mean of the measurement sites.  Fig. 5 shows typical filtering results.

Fig. 5. Wavelet filtering process
(a) original signal; (b) wavelet transform of data; (c) filtering; (d) time-domain reconstruction; and (e) heart rate identification via FFT.

 

The thermal video results were compared to the ground truth measurement from the EKG and the accuracy was determined using the complement of the absolute normalized difference (CAND) measure in (4)

(4)

Our method works well with the small, inevitable motions that can cause noise during the signal acquisition process and this represents a step forward from the previous work in this area and demonstrates that the tracking method used in the project is useful to decrease the amount of noise in the thermal videos.  Accuracy is typically 93% accurate or better.

Fig. 6. Histogram of heart rate extraction accuracy expressed as a CAND measure

 

Results:

Experimental setup

Experimental setup with thermal camera, EKG and data acquisition devices.

Wavelet filtering process

(a) original signal; (b) wavelet transform of data; (c) filtering; (d) time-domain reconstruction; and (e) heart rate identification via FFT

The image formation process

Additional Morphological Operations

Three highlighted spurious TMPs showing (left) the original vascular structure prior to thinning, (middle) the results of thinning, and (right) the results after applying the dilation to the original image.

Research Team:

Publications:

  1. [Atamaz06]        Y. Atamaz Y. and F. Govsa. Anatomy of the superficial temporal artery and its branches: its importance for surgery. Surg. Radiol. Anat. 28(1):248–253, 2006.

  2. [Buddharaju07] P. Buddharaju, et al. Physiology-Based Face Recognition in the Thermal Infrared Spectrum. TPAMI. 29(10):613-26, 2007.

  3. [Gunapala06]     Gunapala, S. et al. (2006). Towards dualband megapixel QWIP focal plane arrays.  Infrared Phys. Techn. © 2006.  doi:10.1016/j.infrared.2006.10.005.

  4. [Infratec09]        InfraTec (2009). Basics.  InfraTec – Your specialist for Thermography and Infrared Research.  Accessed 29 June 2009.  <http://www.infratec.de/index.php?id=59&gt;.  © 2009.

  5. [Chekmenev06]  S. Chekmenev, A. Farag, and E. Essock. Multi-resolution Approach for Non-Contact Measurements of Arterial Pulse using Thermal Imaging. Proc. CVPR. 2006.

  6. [Chekmenev07] S. Chekmenev, A. Farag, and E. Essock. Thermal Imaging of the Superficial Temporal Artery: An Arterial Pulse Recovery Model. CVPR. 2007.

  7. [Cotterill09]       S. Cotterill. “The Cardiovascular System.” CancerIndex. University of Newcastle upon Tyne, 31 Aug. 2000. Web. 11 Aug. 2009.      <http://www.cancerindex.org/medterm/medtm8.htm>.

  8. [Douchet08]       A. Doucet and A. Johansen. A Tutorial on Particle Filtering and Smoothing: Fifteen years later. Tech.  Dec. 2008.

  9. [Dowdall07]       J. Dowdall, I. Pavlidis, and P. Tsiamyrtzis. “Coalition  Tracking.” CVIU 106(2007): 205-19. 2007.

  10. [Gault10]            Travis R. Gault, Nicholas Blumenthal, Aly A. Farag and Tom Starr.  “Extraction of the Superficial Facial Vasculature, Vital Signs Waveforms and Rates Using Thermal Imaging,”  Proc. IEEE OTCBVS’10.  San Francisco, CA. 13 June 2010.

  11. [Milnor93]          W. Milnor. “Cardiovascular physiology.” NY: Oxford UP, 1990.

  12. [Rogalski06a]     Rogalski, A.  (2006).  Competitive technology of third generation infrared photon detectors.  Opto-Electronics Rev. 14(4): 87-101.  2006.

  13. [Rogalski06b]     Rogalski, A. (2006).   Third generation infrared detectors.  Proc. Of the Symposium on Photonics Technologies for 7th Framework Prog.  Wroclaw. 12-14 October 2006.

  14. [Steketee73]       J. Steketee. Spectral Emissivity of Skin and Pericardium. Phys. Med. Biol., 18(10):686-94, 1973

  15. [Tipler03]           Tipler, P. and R. Llewellyn (2003).  Modern Physics, 4th ed.  New York:  W. H. Freeman and Company.  pgs 132-141.  © 2003.

  16. [Wu08]              S. Wu, W. Lin, and S. Xie. Skin Heat Transfer Model of Facial Thermograms and its Application in Face Recognition. Pattern Recognition 41(2008): 2718-729.

  17. [Yang03]            S. Yang and I. Verbauwhede.  A Secure Fingerprint Matching Technique.  WBMA.  Nov. 2003

 


News