Research Our Current Projects
NLOS
What is Non-Line-of-Sight Imaging?
Single photon cameras are fast enough to image the motion of light. This allows us to analyze light transport in way that are not possible with an ordinary camera. By measuring and inverting multibounce light transport in a scene, we can for example reconstruct images of scenes from indirect light after it has reflected off relay surfaces. In other words: Our NLOS imaging system can project a virtual camera onto any surface. Our reconstruction shows us what this virtual projected camera sees. This makes it possible to see into rooms through windows, into caves, or around obstacles on the road
Non-Line-of-Sight Imaging using Phasor Field Virtual Wave Optics
Quanta Photography
Single-Photon 3D Imaging in Strong Sunlight
Single-photon avalanche diodes (SPADs) are an emerging image sensor technology. Due to their unique ability to precisely time-tag individual photons, SPADs have great potential for high-resolution long-range LiDAR systems. However, due their peculiar image formation model, extreme ambient light incident on a SPAD LiDAR causes severe distortions (called pile-up) leading to large depth errors. Currently, these distortions are avoided by artificially lowering the incident photon flux. However, this has the undesirable side-effect of lowering the signal-to-noise ratio. We address the following basic question: What is the optimal photon flux that a SPAD-based LiDAR should be operated in? Our theoretical analysis leads to a simple, practical, closed form mathematical expression that depends on the strength of the ambient light. Surprisingly, the optimal flux level is considerably higher than what is conventionally used. We propose an adaptive approach for achieving this optimal flux by attenuating the incident light based on an estimate of ambient light strength at each scene point. Our simulations and hardware experiments show that this adaptive attenuation method is robust and holds for a wide range of illumination conditions.
Read more
Passive Inter-Photon Imaging
Digital camera pixels capture images by converting incident light energy into an analog electrical current, and then digitizing it into a binary representation. This direct measurement method, while conceptually simple, suffers from limited dynamic range: electronic noise dominates under low illumination, and the pixel’s finite full-well capacity causes saturation under bright illumination. Here we show that inter-photon timing information captured by a dead-time-limited single-photon detector can be used to estimate scene intensity over a much higher range of brightness levels. We experimentally demonstrate imaging scenes with a dynamic range of over ten million to one. The proposed techniques, aided by the emergence of single-photon sensors such as single-photon avalanche diodes (SPADs) with picosecond timing resolution, will have implications for a wide range of imaging applications: robotics, consumer photography, astronomy, microscopy and biomedical imaging.
Read more
Surgical Fluorescence Imaging
Medical vision problems provide exception challenges to our cameras and algorithms. We are developing methods that utilize single photon cameras for fluorescence imaging in fluorescence guided surgery, fluorescence lifetime imaging, and other types of radiation detection. Our imaging systems have to deal with ambient light, motion, and weak signals present in the operating room and our demonstrated methods are ideal for compensating these effects.
The application of advanced imaging techniques outside the laboratory is often complicated by background light. These techniques, such as fluorescence imaging require a dark environment, which is problematic in many industrial and medical applications. With transient lighting, it is possible to capture images on systems with light sensitive detectors and cameras while maintaining normal room illumination. This is achieved through quick switching light sources and detectors to provide time separation for multiple detectors and illumination in a common space. We have demonstrated this method on a two photon microscope and are currently developing systems for fluorescence guided surgery.
Computational Radiation CamerasDetecting X-Rays, Protons, and Neutrons can be achieved with scintillators that absorb the particle and emit fluorescence light that can be detected. We are investigating novel detector setups using advanced scintillator materials and single photon cameras for improved radiation detection for applications in nuclear nonproliferation and medical imaging.
PERISCOPE
PERIapsis Subsurface Cave OPtical Explorer
The discovery of cave skylights on the surface of the moon has stimulated a desire to explore and map the lunar subsurface. These openings provide a ready made pathway into caves that may harbor unique resources and grant protection from space weather for endemic or human life. As a result these caves could be of great value for both robotic and human exploration. However, access into and mobility around these caves is extremely challenging to perform in situ with current technology, and without reconnaissance it is difficult to identify which caves have the most scientific and resource potential. We propose a precursor to in situ exploration of lunar caves, using time of flight imaging from an orbiter to explore and map in 3D cave entrances and near-surface interiors (PERISCOPE). Our system illuminates the cave entrance with nanosecond light pulses and uses the time response of light returning from inside the cave to reconstruct images of the cave's interior. To view the project overview on the NASA website click here.
TERA
Time Encoded Remote Aperture
In conventional imaging, resolution decreases as the distance between the object and the observer increases. While this degradation seems inevitable for images, it does not happen to time encoded information. Time Encoded Remote Aperture (TERA) imaging uses the information encoded in the time dependence of light coming from a scene to reconstruct images of that scene. This time encoded aperture information can travel undistorted over large distances and does not require an imaging system with a large aperture to achieve a high resolution. TERA Imaging has potential applications in endoscopic imaging, microscopy, remote sensing, and astronomy.
Hyperspectral Confocal Microscopy
Improving the spectral resolution in confocal microscopy is important for both materials research and the study of fluorescent biological samples. When studying dynamic fluorescent samples, it is ideal to image at hight spectral resolution while retaining good viability and signal sensitivity, as well as high temporal fidelity. In particular there is a great need to image at high speeds with spectral resolution to image dynamic events such as cell division and cancer invasion and progression. We have developed a high-speed spectral confocal microscope that utilizes a low-loss Amici prism for spectral separation in conjunction with a multi-point confocal design for high speed image capture. Our system achieves 15 channels of spectral resolution at a 4 frames per second imaging rate with minimal loss of overall signal.
Femto Photography
By using an ultra-fast laser, producing 50-femtosecond long pulses, and capturing the returning light with a streak camera, videos of the interaction of light with the scene can be produced. The results of several of these experiments are shown in You Tube Videos. Links to these videos are shown below.