Explore the vast range of titles available.

Digital holography at the rate of 1 000 000 frames per second (fps), which is the fastest speed in digital holography to the best of knowledge, has been achieved. High‐speed recording of digital holography needs a high‐speed camera. However, the number of available pixels is very small in general, due to the characteristics of the high‐speed camera. Then, to record the hologram with such a small number of pixels and obtain a clear phase image of the object, parallel phase‐shifting digital holography (PPSDH) was adopted. This technique was used to successfully obtain images of the phase variation of electrical discharging phenomenon at the rate of 1 000 000 fps.

The computer vision algorithms used currently in advanced driver assistance systems rely on image-based RGB cameras, leading to a critical bandwidth–latency trade-off for delivering safe driving experiences. To address this, event cameras have emerged as alternative vision sensors. Event cameras measure the changes in intensity asynchronously, offering high temporal resolution and sparsity, markedly reducing bandwidth and latency requirements 1 . Despite these advantages, event-camera-based algorithms are either highly efficient but lag behind image-based ones in terms of accuracy or sacrifice the sparsity and efficiency of events to achieve comparable results. To overcome this, here we propose a hybrid event- and frame-based object detector that preserves the advantages of each modality and thus does not suffer from this trade-off. Our method exploits the high temporal resolution and sparsity of events and the rich but low temporal resolution information in standard images to generate efficient, high-rate object detections, reducing perceptual and computational latency. We show that the use of a 20 frames per second (fps) RGB camera plus an event camera can achieve the same latency as a 5,000-fps camera with the bandwidth of a 45-fps camera without compromising accuracy. Our approach paves the way for efficient and robust perception in edge-case scenarios by uncovering the potential of event cameras 2 . Use of a 20 frames per second (fps) RGB camera plus an event camera can achieve the same latency as a 5,000-fps camera with the bandwidth of a 45-fps camera without compromising accuracy.

Terahertz (THz) radiation is poised to have an essential role in many imaging applications, from industrial inspections to medical diagnosis. However, commercialization is prevented by impractical and expensive THz instrumentation. Single-pixel cameras have emerged as alternatives to multi-pixel cameras due to reduced costs and superior durability. Here, by optimizing the modulation geometry and post-processing algorithms, we demonstrate the acquisition of a THz-video (32 × 32 pixels at 6 frames-per-second), shown in real-time, using a single-pixel fiber-coupled photoconductive THz detector. A laser diode with a digital micromirror device shining visible light onto silicon acts as the spatial THz modulator. We mathematically account for the temporal response of the system, reduce noise with a lock-in free carrier-wave modulation and realize quick, noise-robust image undersampling. Since our modifications do not impose intricate manufacturing, require long post-processing, nor sacrifice the time-resolving capabilities of THz-spectrometers, their greatest asset, this work has the potential to serve as a foundation for all future single-pixel THz imaging systems. Terahertz imaging is promising in many applications, but still relies on complex equipment. Here, the authors develop a simplified solution that enables terahertz real-time imaging using a single-pixel detector and rapid reconstruction methods.

Hyperspectral (HS) imaging provides rich spatial and spectral information and extends image inspection beyond human perception. Existing approaches, however, suffer from several drawbacks such as low sensitivity, resolution and/or frame rate, which confines HS cameras to scientific laboratories. Here we develop a video-rate HS camera capable of collecting spectral information on real-world scenes with sensitivities and spatial resolutions comparable with those of a typical RGB camera. Our camera uses compressive sensing, whereby spatial–spectral encoding is achieved with an array of 64 complementary metal–oxide–semiconductor (CMOS)-compatible Fabry–Pérot filters placed onto a monochromatic image sensor. The array affords high optical transmission while minimizing the reconstruction error in subsequent iterative image reconstruction. The experimentally measured sensitivity of 45% for visible light, the spatial resolution of 3 px for 3 dB contrast, and the frame rate of 32.3 fps at VGA resolution meet the requirements for practical use. For further acceleration, we show that AI-based image reconstruction affords operation at 34.4 fps and full high-definition resolution. By enabling practical sensitivity, resolution and frame rate together with compact size and data compression, our HS camera holds great promise for the adoption of HS technology in real-world scenarios, including consumer applications such as smartphones and drones.A hyperspectral camera based on a random array of CMOS-compatible Fabry–Pérot filters is demonstrated. The hyperspectral camera exhibits performance comparable with that of a typical RGB camera, with 45% sensitivity to visible light, a spatial resolution of 3 px for 3 dB contrast, and a frame rate of 32.3 fps at VGA resolution.

Optical imaging is commonly performed with either a camera and wide-field illumination or with a single detector and a scanning collimated beam; unfortunately, these options do not exist at all wavelengths. Single-pixel imaging offers an alternative that can be performed with a single detector and wide-field illumination, potentially enabling imaging applications in which the detection and illumination technologies are immature. However, single-pixel imaging currently suffers from low imaging rates owing to its reliance on configurable spatial light modulators, generally limited to 22 kHz rates. We develop an approach for rapid single-pixel imaging which relies on cyclic patterns coded onto a spinning mask and demonstrate it for in vivo imaging of C. elegans worms. Spatial modulation rates of up to 2.4 MHz, imaging rates of up to 72 fps, and image-reconstruction times of down to 1.5 ms are reported, enabling real-time visualization of dynamic objects. Imaging rates in single-pixel imaging has been limited by the dependence on configurable spatial light modulators. Here, the authors use cyclic Hadamard patterns coded onto a spinning mask to demonstrate dynamic imaging with rates up to 72 frames per second and real time reconstruction capabilities.

High performance X-ray detector with ultra-high spatial and temporal resolution are crucial for biomedical imaging. This study reports a dynamic direct-conversion CMOS X-ray detector assembled with screen-printed CsPbBr 3 , whose mobility-lifetime product is 5.2 × 10 −4  cm 2  V –1 and X-ray sensitivity is 1.6 × 10 4  µC Gy air –1  cm –2 . Samples larger than 5 cm × 10 cm can be rapidly imaged by scanning this detector at a speed of 300 frames per second along the vertical and horizontal directions. In comparison to traditional indirect-conversion CMOS X-ray detector, this perovskite CMOS detector offers high spatial resolution (5.0 lp mm −1 ) X-ray radiographic imaging capability at low radiation dose (260 nGy). Moreover, 3D tomographic images of a biological specimen are also successfully reconstructed. These results highlight the perovskite CMOS detector’s potential in high-resolution, large-area, low-dose dynamic biomedical X-ray and CT imaging, as well as in non-destructive X-ray testing and security scanning. Biomedical X-ray imaging requires high spatial and temporal resolution of the detectors. Liu et al. report a screen-printed perovskite direct-conversion X-ray CMOS imager with a spatial resolution of 5 lp mm −1 and a speed of 300 fps for low-dose 2D radiography and 3D computed tomography imaging.

There is much interest in employing terahertz (THz) radiation across a range of imaging applications, but so far, technologies have struggled to achieve the necessary frame rates. Here, we demonstrate a THz imaging system based upon efficient THz-to-optical conversion in atomic vapor, where full-field images can be collected at ultrahigh speeds using conventional optical camera technology. For a 0.55-THz field, we show an effective1−cm2sensor with near diffraction-limited spatial resolution and a minimum detectable power of(190±30)fWs−1/2per(40×40)μm2pixel capable of video capture at 3000 frames per second. This combination of speed and sensitivity represents a step change in the state of the art of THz imaging and will likely lead to its uptake in wider industrial settings.

Single-pixel imaging (SPI) has emerged as a powerful technique that uses coded wide-field illumination with sampling by a single-point detector. Most SPI systems are limited by the refresh rates of digital micromirror devices (DMDs) and time-consuming iterations in compressed-sensing (CS)-based reconstruction. Recent efforts in overcoming the speed limit in SPI, such as the use of fast-moving mechanical masks, suffer from low reconfigurability and/or reduced accuracy. To address these challenges, we develop SPI accelerated via swept aggregate patterns (SPI-ASAP) that combines a DMD with laser scanning hardware to achieve pattern projection rates of up to 14.1 MHz and tunable frame sizes of up to 101×103 pixels. Meanwhile, leveraging the structural properties of S-cyclic matrices, a lightweight CS reconstruction algorithm, fully compatible with parallel computing, is developed for real-time video streaming at 100 frames per second (fps). SPI-ASAP allows reconfigurable imaging in both transmission and reflection modes, dynamic imaging under strong ambient light, and offline ultrahigh-speed imaging at speeds of up to 12,000 fps. The authors present single-pixel imaging accelerated via swept aggregate patterns (SPI-ASAP), which combines a digital micromirror device with laser scanning for fast and reconfigurable pattern projection, and a lightweight reconstruction algorithm. They demonstrate real-time video streaming at 100 fps, and up to 12,000 fps offline.

Capturing fine spatial, spectral, and temporal information of the scene is highly desirable in many applications. However, recording data of such high dimensionality requires significant transmission bandwidth. Current computational imaging methods can partially address this challenge but are still limited in reducing input data throughput. In this paper, we report a video-rate hyperspectral imager based on a single-pixel photodetector which can achieve high-throughput hyperspectral video recording at a low bandwidth. We leverage the insight that 4-dimensional (4D) hyperspectral videos are considerably more compressible than 2D grayscale images. We propose a joint spatial-spectral capturing scheme encoding the scene into highly compressed measurements and obtaining temporal correlation at the same time. Furthermore, we propose a reconstruction method relying on a signal sparsity model in 4D space and a deep learning reconstruction approach greatly accelerating reconstruction. We demonstrate reconstruction of 128 × 128 hyperspectral images with 64 spectral bands at more than 4 frames per second offering a 900× data throughput compared to conventional imaging, which we believe is a first-of-its kind of a single-pixel-based hyperspectral imager. The authors showcase a video-rate hyperspectral imager based on a single-pixel photodetector that can achieve high-throughput hyperspectral video recording at a low bandwidth. Specifically, they propose a joint spatial-spectral encoding scheme which can encode the scene into highly compressed single-pixel measurements and obtain temporal correlation at the same time.

To rapidly process temporal information at a low metabolic cost, biological neurons integrate inputs as an analog sum, but communicate with spikes, binary events in time. Analog neuromorphic hardware uses the same principles to emulate spiking neural networks with exceptional energy efficiency. However, instantiating high-performing spiking networks on such hardware remains a significant challenge due to device mismatch and the lack of efficient training algorithms. Surrogate gradient learning has emerged as a promising training strategy for spiking networks, but its applicability for analog neuromorphic systems has not been demonstrated. Here, we demonstrate surrogate gradient learning on the BrainScaleS-2 analog neuromorphic system using an in-the-loop approach. We show that learning self-corrects for device mismatch, resulting in competitive spiking network performance on both vision and speech benchmarks. Our networks display sparse spiking activity with, on average, less than one spike per hidden neuron and input, perform inference at rates of up to 85,000 frames per second, and consume less than 200 mW. In summary, our work sets several benchmarks for low-energy spiking network processing on analog neuromorphic hardware and paves the way for future on-chip learning algorithms.