Researchers from Stanford University’s Computational Imaging Lab have developed a novel method of non-line-of-sight imaging. They call it keyhole imaging because the contents of a room can be captured from a single point as small as a keyhole.
Non-line-of-sight (NLOS) imaging as a concept isn’t new and has been used to image objects in a closed room from a single point source for some time now. NLOS systems were designed to see around objects by bouncing a laser off a surface, off surrounding objects, and then back to the origin point.
“Existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled,” Stanford’s researchers explain. “In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques.”
Keyhole imaging: A time-resolved detector and pulsed laser illuminate and image a point visible through a keyhole (left). As a hidden person moves, the detector captures a series of time-resolved measurements of the indirectly scattered light (center). From these measurements, we reconstruct both hidden object shape (e.g., for a hidden mannequin) and the time-resolved trajectory (right).
As InceptiveMind explains, there is one major downside to this method: the requirement for a large reflective surface inside the target area.
Stanford’s researchers tried a different approach to solve this problem. Dubbing it “keyhole imaging,” the method can capture a series of measurements along a tiny optical path in order to image a space on the other end. That path can be very small: even as tiny as a keyhole, hence the name.
Experimental setup: Stanford’s researchers’ optical system sends a laser pulse through the keyhole of a closed door. On the other side of the door, the hidden object moves along a translation stage. When third-bounce photons return, they are recorded and time-stamped by a SPAD. Top-right inset: A beam splitter (BS) is used to place the laser and SPAD in a confocal configuration.
“Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object’s shape from unknown viewpoints,” the Stanford researchers explain.
“We derive inverse methods based on expectation-maximization to recover the object’s shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.”
This new method solves the issues the researchers sought out to address but isn’t without downsides. It results in significantly worse image quality than previous NLOS methods. In the image below, the objects that are imaged are shown to be murky, blurry outlines of their original forms.
Experimental results: First row: Images of the hidden objects. Second row: Reconstructions of the hidden objects using GD when their trajectories are known. Third row: EM reconstructions of the hidden objects when their trajectories are unknown. Fourth row: EM estimates of the trajectories of the hidden objects, each of which follows a different trajectory, where the dot color indicates position over time.
Despite this, the researchers argue that it still provides enough detail to make educated guesses on the shape of the hidden object, especially when using artificial intelligence recognition systems. The team believes this technology could be used by the military, for example, to determine the risks of entering a space before breaching.
The full research documentation can be read on Computational Imaging.
Continue reading...
Non-line-of-sight (NLOS) imaging as a concept isn’t new and has been used to image objects in a closed room from a single point source for some time now. NLOS systems were designed to see around objects by bouncing a laser off a surface, off surrounding objects, and then back to the origin point.
“Existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled,” Stanford’s researchers explain. “In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques.”
Keyhole imaging: A time-resolved detector and pulsed laser illuminate and image a point visible through a keyhole (left). As a hidden person moves, the detector captures a series of time-resolved measurements of the indirectly scattered light (center). From these measurements, we reconstruct both hidden object shape (e.g., for a hidden mannequin) and the time-resolved trajectory (right).
As InceptiveMind explains, there is one major downside to this method: the requirement for a large reflective surface inside the target area.
Stanford’s researchers tried a different approach to solve this problem. Dubbing it “keyhole imaging,” the method can capture a series of measurements along a tiny optical path in order to image a space on the other end. That path can be very small: even as tiny as a keyhole, hence the name.
Experimental setup: Stanford’s researchers’ optical system sends a laser pulse through the keyhole of a closed door. On the other side of the door, the hidden object moves along a translation stage. When third-bounce photons return, they are recorded and time-stamped by a SPAD. Top-right inset: A beam splitter (BS) is used to place the laser and SPAD in a confocal configuration.
“Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object’s shape from unknown viewpoints,” the Stanford researchers explain.
“We derive inverse methods based on expectation-maximization to recover the object’s shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.”
This new method solves the issues the researchers sought out to address but isn’t without downsides. It results in significantly worse image quality than previous NLOS methods. In the image below, the objects that are imaged are shown to be murky, blurry outlines of their original forms.
Experimental results: First row: Images of the hidden objects. Second row: Reconstructions of the hidden objects using GD when their trajectories are known. Third row: EM reconstructions of the hidden objects when their trajectories are unknown. Fourth row: EM estimates of the trajectories of the hidden objects, each of which follows a different trajectory, where the dot color indicates position over time.
Despite this, the researchers argue that it still provides enough detail to make educated guesses on the shape of the hidden object, especially when using artificial intelligence recognition systems. The team believes this technology could be used by the military, for example, to determine the risks of entering a space before breaching.
The full research documentation can be read on Computational Imaging.
Continue reading...