Fourier Ptychography

Problem Formulation

Fourier ptychography is a computational imaging technique based on optical microscopy that consists in the synthesis of a wider numerical aperture from a set of full-field images acquired at various coherent illumination angles, resulting in increased resolution compared to a conventional microscope.

To have a high resolution picture, we need a camera with wide numerical aperture. Unfortunately the cameras with wide numerical aperture is very expensive or even imposible to make. However, image computing can help to get a high resolution image from a set of low resolution images. Consier $o(r)$ to represent a high resolution image at pixel $r$ where $r$ denotes the lateral coordinates at the image plane. Then, there are $m$ LED. Each LED generates illumination at the sample that is treated as a (spatially coherent) local plane wave with a unique spatial frequency $f_i$. The exit wave from the sample is the multiplication of the two $u(r) = o(r)\exp(jf_i\cdot r)$. Thus, the sample’s spectrum $O(f)$ is shifted to be centered around $k_m$. At the pupil plane, the field corresponding to the Fourier transform of the exit wave $O(f−f_i)$ is low-pass filtered by the pupil function $P(f)$. Therefore, the intensity at the image plane resulting from a single LED illumination (neglecting magnification and noise) is \[ \ell_i(r) = \big| \mathcal{F}_{[O(f-f_i)P(f)]}(r) \big|^2 \] where $\mathcal{F}$ represents 2D Fourier transform. Thus, the goal is to recover $o(r)$ from $\ell_m(r)$'s. We note that $\big| \mathcal{F}_{[O(f-f_i)P(f)]}(r) \big|^2$ can be represented as $\big|\langle o, a_i\rangle\big|^2$ where $a_i$ is 2D DFT matrix that is elementwise multiplied with matrix $P$ (a disk matrix where all elements are zero except a disk).

Now, consider recovering a high resolution video from a set of low resolution images from each frame. More especificly, we want to recover $o_k(r)$ from $\ell_{ik}(r)$ for $i=1,\ldots,m$ and $k=1,\ldots, q$.

S.Nayer, P.Narayanamurthy, N.Vaswani, "Phaseless PCA: Low-Rank Matrix Recovery from Column-wise Phaseless Measurements", ICML 2019.

S.Nayer, P.Narayanamurthy, N.Vaswani, "Provable low rank phase retrieval", IEEE Transactions on Information Theory 2020.

Z.Chen, G.Jagatap, S.Nayer, C.Hegde, N.Vaswani, "Low Rank Fourier Ptychography", The 43nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), April 2018.

G.Jagatap, Z.Chen, S. Nayer, C.Hegde, N.Vaswani,, "Sample efficient fourier ptychography for structured data", IEEE Transactions On Computational Imaging, 2019.

Visual comparison of super-resolved reconstructions

Real data reconstruction under the uniform random camera sub-sampling