Ultrafast compact all-fiber imaging via deep learning

High-speed imaging through multimode optical fibers is demonstrated, leveraging the high intermodal dispersion to transform 2D spatial information into 1D temporal pulsed signal streams. Deep learning algorithms is applied to reconstruct the images of micron-scale objects at ultrahigh frame rates.

Like Comment
Read the paper

Optical fibers are vital media for long-distance information transmission. Recent years have seen increasing interests in applying deep learning algorithms to recover and detect images from the scattered lights through single- or multimode-fibers [1-3].

While ultrahigh-speed optical-fiber-based endoscopy are frequently required for in vivo and in situ applications, arming the high imaging speed with the fiber endoscopy can be vital for exploring transient biomedical and physical phenomena. However, conventional approaches entails traditional cameras that generally require a balance between the imaging speed and the frame depth due to a limited readout speed from the pixel arrays to memory. Inspired by the time-stretching scheme [4], a compact all-fiber imaging scheme is proposed and experimentally demonstrated at very high speeds. As is illustrated in Fig. 1, the high-speed imaging based on the transformation of two-dimensional spatial information into one-dimensional temporal pulsed streams by leveraging high intermodal dispersion in a multimode fiber. Neural networks are trained to reconstruct images from the temporal waveforms. 

Figure 1. Evolution of ultrashort pulses. The pulses entering the MMF will be split into lots of sub-pulses due to the large intermodal dispersion in the MMF. Each sub-pulse contains the energy of a certain mode in principle.

The operation schematics and experimental layout is shown as Fig. 2. Here, we combine the advantages of the time-stretching method [4] and fiber endoscopy to propose a one-pixel method to enable all-fiber high-speed detection of images, eliminating the requirements and costs on expensive high-speed/resolution cameras. Leveraging a single multimode fiber (MMF) as the probe, real-time image acquisition with a high frame rate of over 15 Mfps and a shutter time of 45.1 ps is experimentally demonstrated, in which 10,000 frames could be recorded in a single shot. Moreover, the maximum system frame rate may be further enhanced to 53.5 Mfps. The 2D spatial information encoded in scattered light from the target images is transformed into one-dimensional (1D) time-domain pulsed waveforms, thanks to the big intermodal dispersion effect in an MMF. An artifical neural network model is trained to reconstruct images from the temporal waveforms recorded by an ultrafast photodiode connected to the output end of the fiber. 

Figure 2.  Experimental layout. a Schematic of the experimental setup. b Flow of the reconstruction process from waveforms to images. c Schematic of the fiber-end ball, the side-pump fiber coupler, and the flow of illumination light (orange arrows) and signal light (red arrows). d Structures of the side-pump coupler and triple-cladding fiber probe, where illumination light (orange arrows) is coupled into and transmitted through the cladding of the fiber, while signal light (red arrows) is collected and transmitted through the fiber core in the opposite direction. The cross section of this coupling region is also shown. The triple-cladding fiber consists of the fiber core (red), first cladding layer (blue), and second cladding layer (orange). e Micrograph of the fiber-end ball. Scale bar: 500 μm. 

Besides, an all-fiber structure by combining a fiber-output pulse laser, a triple-cladding fiber probe, and a side-pump coupler is also applied, which may enable high levels of integration and system stability. The overall imaging performance characterizations are manifested in Fig. 3. The performance of our demonstrated proof-of-principle system may be further improved. This scheme can be further modified to detect 3D objects by combining it with the existing time-of-flight technique as instance.

Figure 3. a Imaging performance analysis. Example images of the letters and clothes and the recovered images. b Imaging of random types of objects. Several recovered images of different types.  c Ultra-high-speed imaging. Waveforms collected during the transient time when the DMD refreshes from an image 3 to an image 0. Insets: Real images of the DMD captured during the refresh process using a commercial high-speed camera. (a1–a17) show the images reconstructed from some of these waveforms.

Article Information

All-fiber high-speed image detection enabled by deep learning, Nature Communications 13, Article number: 1433 (2022).

DOI: 10.1038/s41467-022-29178-8

Link: https://www.nature.com/articles/s41467-022-29178-8


[1]  Caramazza, P., et al. Transmission of natural scene images through a multimode fibre. Nature Communications 10, 2029 (2019).
[2] Borhani, N., et al. Learning to see through multimode fibers. Optica 5, 960–966 (2018).
[3] Meng, Y., et al, Optical meta-waveguides for integrated photonics and beyond, Light: Science & Applications 10, 235 (2021).
[4] Goda, K., et al. Serial time-encoded amplified imaging for realtime observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).

Yuan Meng

PhD, Tsinghua University