# Answering some questions about structured illumination microscopy

James D. Manton1,* 1 MRC Laboratory of Molecular Biology, Cambridge, CB2 0QH,
UK
*<EMAIL_ADDRESS>
(16 November 2021)

###### Abstract

Structured illumination microscopy (SIM) provides images of fluorescent
objects at an enhanced resolution greater than that of conventional
epifluorescence wide-field microscopy. Initially demonstrated in 1999 to
enhance the lateral resolution two-fold, it has since been extended to enhance
axial resolution two-fold (2008), applied to live-cell imaging (2009) and
combined with myriad other techniques, including interferometric detection
(2008), confocal microscopy (2010) and light sheet illumination (2012).
Despite these impressive developments, SIM remains, perhaps, the most poorly
understood ‘super-resolution’ method. In this article, we provide answers to
the 13 questions regarding SIM proposed by Prakash et al., along with answers
to a further three questions. After providing a general overview of the
technique and its developments, we explain why SIM as normally used is still
diffraction-limited. We then highlight the necessity for a non-polynomial, and
not just non-linear, response to the illuminating light in order to make SIM a
true, diffraction-unlimited, super-resolution technique. In addition, we
present a derivation of a real-space SIM reconstruction approach that can be
used to process conventional SIM and image scanning microscopy (ISM) data and
extended to process data with quasi-arbitrary illumination patterns. Finally,
we provide a simple bibliometric analysis of SIM development over the past two
decades and provide a short outlook on potential future work.

## Introduction

Structured illumination microscopy (SIM) is a method in fluorescence
microscopy which is capable of providing resolutions better than those that
can be achieved using uniform illumination in epifluorescence wide-field
microscopy. While first demonstrated, at the turn of the millenium, to enhance
only lateral resolution [1, 2, 3, 4], subsequent developments allow for the
enhancement of axial resolution as well [5, 6]. SIM has become a popular
technique, particularly for live cell imaging [7, 8, 9], when better
resolution is required as, unlike super-resolution methods such as single
molecule localisation microscopy (SMLM) [10, 11, 12, 13, 14] and stimulated
emission depletion microscopy (STED) [15, 16, 17], it does not require high
laser powers nor special fluorophores. However, as will be discussed further
later, in its normal form it only provides a roughly two-fold enhancement of
resolution.

It should be noted that, in 1997, an alternative form of structured
illumination microscopy was presented by Neil et al. [18]. Here the goal was
not to improve resolution, but to add optical sectioning to wide-field
microscopy. This form of structured illumination has been commercially
successful, with further developments of the technology available from Aurox
(Clarity), Andor (Revolution DSD2) and Zeiss (Apotome). For the purposes of
this article, SIM will be taken to correspond only to those methods designed
to improve resolution (often referred to as SR-SIM, as opposed to the OS-SIM
of Neil et al.).

In this article, we aim to provide a general overview of SIM and answer the 13
questions recently proposed by Prakash et al. [19] (sections 1–13), along with
three more of our own invention (sections 14–16). A similar attempt at
answering the 13 questions of Prakash et al. has recently been presented by
Heintzmann [20]. For some questions, the answers given by Heintzmann are
significantly different from those presented here: where this is the case,
this will be highlighted in later sections. In Section 1, we consider the so-
called ‘diffraction limit’ and show that SIM as described here does not break
it, even though it provides resolutions beyond those achievable with
conventional wide-field imaging. Following this, in Section 2, we consider
whether an impressive demonstration of SIM combined with total internal
reflection fluorescence (TIRF) microscopy, which achieves a resolution of
$84\text{\,}\mathrm{n}\mathrm{m}$, breaks the diffraction limit. We then look
at the conditions required for SIM to break the diffraction limit in Section 3
and Section 4.

In the second part of the manuscript, we show that, despite their prevalence,
Fourier methods are not necessarily required to reconstruct images (Section 5)
and study the challenges of using SIM in thick samples (Section 6 and Section
7). We then argue that image scanning microscopy (ISM), while experimentally a
different method, is a form of SIM and show how arbitrary illumination
patterns can be used (Section 8). Following this, we study the effects of
illumination sparsity in Section 9. We note that SIM cannot be used to improve
the resolution of transmission microscopy (Section 10) and propose a careful
approach to the use of machine learning with super-resolution (Section 11).

In the last part of the manuscript, we discuss the practicalities of building
a SIM system (Section 12) and consider how to combine SIM with SMLM (Section
13). We then investigate how the axial resolution can be improved beyond a
two-fold limit without a non-linear response (Section 14), how to combine SIM
with light sheet microscopy (Section 15) and how quickly SIM images can be
acquired (Section 16). Finally, we use a simple bibliometric analysis to track
interest in SIM since its inception and conclude with an outlook on future
developments (Section 17).

##  1 What is super-resolution microscopy and should diffraction-limited
linear SIM be classed as ‘super-resolution’?

Super-resolution microscopy refers to a set of techniques that can be used to
break ‘the diffraction barrier’, a seemingly unsurmountable goal rewarded with
the 2014 Nobel Prize in Chemistry. The trick in this question is deciding
exactly what the diffraction barrier is…

There are a number of resolution metrics in use, such as the Rayleigh and
Sparrow criteria, that derive from trying to answer the question of how
closely two stars can appear while still being distinct in naked-eye
astronomical observations [21, 22]. These criteria are, however, heuristic and
have no fundamental physical basis. Ernst Abbe was the first to propose a
physically-motivated limit to the resolution of a light microscope by studying
diffraction from a grating [23]. He considered using a microscope to image a
grating using transmitted light and noted that, in order for the image to have
any periodicity, at least two of the diffracted orders from the grating must
be captured by the objective. For head-on illumination, i.e. normal to the
grating, he noted that three orders (the $-1$, 0 and $+1$ orders) were
captured, giving a minimum grating periodicity of:

$d_{\textrm{head-on}}=\frac{\lambda_{0}}{n\sin\alpha},$ (1)

where $\lambda_{0}$ is the vacuum wavelength, $n$ the refractive index of the
sample and $\alpha$ the semiaperture angle (i.e. the maximum angle from the
objective’s optical axis from which orders can be collected). For convenience,
the product $n\sin\alpha$ is known as the numerical aperture (NA).

Referring to Figure 1 on page 1a (top row), we see that these three orders are
located symmetrically about the centre of the back focal plane. By tilting the
illumination, we can shift the location of these orders laterally. Given that
we only need two orders, not three, for a grating-like structure to be seen,
we can imagine that by tilting the illumination such that the zeroth order is
right at the edge of the back focal plane, we could increase the order spacing
by a factor of two and hence half the grating period (Figure 1 on page 1a,
bottom row). This leads to the Abbe equation for oblique illumination:

$d_{\textrm{oblique}}=\frac{\lambda}{2n\sin\alpha}.$ (2)

What about fluorescence imaging, where light from the sample is emitted in all
directions? Abbe never studied the case of fluorescence imaging, but it turns
out that the right hand side of Abbe’s equation for oblique illumination is
correct for this case as well:

$d^{\textrm{lateral}}_{\textrm{fluorescence}}=\frac{\lambda}{2n\sin\alpha}.$
(3)

More specifically, this is the lateral resolution, i.e. that within the focal
plane. Perpendicular to the focal plane, the axial resolution is given by

$d^{\textrm{axial}}_{\textrm{fluorescence}}=\frac{\lambda}{n(1-\cos\alpha)}.$
(4)

Why are these equations correct? To explain this, we will follow an argument
similar to that of Gustafsson [24], which in turn builds on arguments by
McCutchen [25] and Ewald [26]. First, we must note that we will use the scalar
approximation of the electric field, although the argument can be generalised
by considering each component of the vectorial electric field separately. We
will also work in the far-field, as near-field components decay exponentially
quickly and will not be detected by a normal microscope. In addition,
fluorescence is an incoherent process, with emission from a source bearing no
phase relationship to the stimulating light or emission from another source
(we ignore the small probability of stimulated emission). As such, we can
limit our consideration to just one source, as the overall response from the
sample is the incoherent sum of the responses from all the sources present.

Consider the amplitude of the electric field, $\mathcal{E}(\vec{r})$, produced
by a source radiating monochromatically. The source produces a spherical wave
which, in the far field, can be decomposed, via a Fourier transform, into a
sum of plane waves with different directions. All these plane waves will have
the same wavelength, $\lambda$, and so all the wavevectors, $\vec{k}$, will
have the same length. Hence, as there are no preferred directions and all
wavevectors have the same length, the Fourier transform of
$\mathcal{E}(\vec{r})$, $\tilde{\mathcal{E}}(\vec{k})$ (we will use the
notation that the Fourier transform of $X(\vec{r})$ is written as
$\tilde{X}(\vec{k})$ throughout), will have the form of a spherical shell,
centered on the origin and with radius $1/\lambda$ (Figure 1 on page 1b). This
detected far field is very much not the same as the original emitted field, as
the evanescent components have been lost. The consequences of this can most
clearly be seen by noting that a true spherical wave would decay smoothly with
increasing distance from the origin, while this field has local minima and
maxima all over space.

However, our image is not given by the amplitude of the electric field, but
the intensity. This can be found by multiplying the amplitude by its complex
conjugate, giving $\mathcal{I}=\mathcal{E}\mathcal{E}^{*}$. This means that in
the Fourier domain, we have

$\tilde{\mathcal{I}}(\vec{k})=\tilde{\mathcal{E}}(\vec{k})\ast\tilde{\mathcal{E}}^{*}(-\vec{k}),$
(5)

where $\ast$ denotes convolution. Hence, $\tilde{\mathcal{I}}(\vec{k})$ is the
autocorrelation of $\tilde{\mathcal{E}}(\vec{k})$ and so will have the form of
a ball with twice the radius of the sphere of $\tilde{\mathcal{E}}(\vec{k})$,
$2/\lambda$ (Figure 1 on page 1c).

This is interesting, as it shows that there is a limit to the information
carried by the far field, irrespective of the objective lens used: only
spatial frequencies up to $k_{c}=2/\lambda$ are present. Converting back to
real space, this critical spatial frequency corresponds to a spatial scale of
$\lambda/2$. If we consider that our source may be located in a medium of
refractive index $n$, then we can convert $\lambda$ to a vacuum wavelength,
$\lambda_{0}=n\lambda$, and so obtain the far-field limit:

$d=\frac{\lambda_{0}}{2n}.$ (6)

Now, let us look at what happens if we use an objective lens to capture the
light from our source. The objective can only capture wavevectors over a
certain angular range, meaning that the Fourier domain electric field
amplitude distribution has the form of a spherical cap, rather than a full
sphere (Figure 1 on page 1d). As such, the autocorrelation of this has the
form of a toroidal solid, shown in cross-section in Figure 1 on page 1e.

We can identify that the maximum lateral spatial frequency is given by
$k_{l}=2n\sin\alpha/\lambda_{0}$, while the maximum axial spatial frequency is
given by $k_{a}=n(1-\cos\alpha)/\lambda_{0}$, where $\alpha$ is the
semiaperture angle. Converting to real space, this gives:

$d_{\textrm{lateral}}=\frac{\lambda_{0}}{2n\sin\alpha}$ (7)

and

$d_{\textrm{axial}}=\frac{\lambda_{0}}{n\left(1-\cos\alpha\right)},$ (8)

as was quoted previously.

Considering not just a single source, but a distribution of sources emitting
light given by a distribution $E(\vec{r})$, the toroidal solid formed by
autocorrelating the spherical cap defines a transfer function,
$\tilde{H}(\vec{k})$. Given this, the image spectrum, $\tilde{D}(\vec{k})$, of
the emission spectrum $\tilde{E}(\vec{k})$ is given by

$\tilde{D}(\vec{k})=\tilde{E}(\vec{k})\times\tilde{H}(\vec{k}),$ (9)

where $\times$ denotes point-wise multiplication. In real space

$D(\vec{r})=E(\vec{r})\ast H(\vec{r}),$ (10)

where $H(\vec{r})$ is known as the point spread function.

9 makes clear that the image spectrum only contains spatial frequencies for
which $\tilde{H}(\vec{k})$ is non-zero. As the form of $\tilde{H}(\vec{k})$ is
based only on the phenomenon of far-field diffraction, this defines a true
diffraction limit based on the spatial frequency bandwidth of
$\tilde{H}(\vec{k})$.

Now we are in a position to consider the mechanism of structured illumination
and see how it circumvents this limit. For simplicity, we will restrict our
consideration to one dimension and replace $\vec{r}$ with $x$.

The fundamental basis of SIM is the realisation that, in our image formation
equation, 10, we are not truly interested in the emission distribution
function, $E(x)$, but instead the distribution of sources $S(x)$. These
distributions are related by $E(x)=S(x)\times I(x)$, where $I(x)$ is the
illumination distribution. In the case of uniform illumination, $I$ merely
acts as a scalar multiplier of $S(x)$. However, for non-uniform illumination
we have an updated image formation equation:

$D(x)=E(x)\ast H(x)=\left[S(x)\times I(x)\right]\ast H(x).$ (11)

If we consider this image formation equation in the Fourier domain, then we
have

$\tilde{D}(k)=\tilde{E}(k)\times\tilde{H}(k)=\left[\tilde{S}(k)\ast\tilde{I}(k)\right]\times\tilde{H}(k).$
(12)

This suggests that, while the maximum spatial frequency that can be recorded
is limited by the support of $\tilde{H}(k)$, a careful choice of
$\tilde{I}(k)$ may let us shift the sample spectrum, $\tilde{S}(k)$, around
via the convolution such that other spatial frequencies are transmitted.

SIM, in its simplest variant, achieves this by using illumination with two
coherent beams, providing a pattern of the form

$I(x)=1+\cos(px+\phi),$ (13)

or, in the Fourier domain,

$\tilde{I}(k)=2\delta(k)+e^{i\phi}\delta(k+p)+e^{-i\phi}\delta(k-p),$ (14)

where $\delta$ denotes the Dirac delta function. Hence, for a given phase,
$\phi_{m}$, we have

$\tilde{D}_{m}(k)=\left[2\tilde{S}(k)+e^{i\phi_{m}}\tilde{S}(k+p)+e^{-i\phi_{m}}\tilde{S}(k-p)\right]\times\tilde{H}(k).$
(15)

This shows that we have the usual wide-field component,
$\tilde{S}(k)\tilde{H}(k)$, overlaid with two shifted copies of the sample
spectrum. This alone is insufficient to ‘see’ $\tilde{S}(k\pm p)$, but by
recording multiple images with different phases of the illumination pattern,
these components can be linearly separated and used to reconstruct an image
with improved resolution. As there are three components to separate, at least
three phase shifts are required.

For the purposes of this article, we shall not discuss the details of the
image reconstruction routine, nor how SIM can be used to enhance the
resolution in all directions (i.e. both laterally and axially). Instead, the
interested reader is directed to the excellent article by Gustafsson et al.,
which provides detail on both these issues [5]. Further information on the
individual steps of reconstruction is presented by Lahrberg et al. [27]
(pattern estimation), Wicker [28] (phase estimation), and Smith et al. [29]
(noise-appropriate recombination). An overview of the entire reconstruction
routine, as well as open-source software, is presented by Lal et al. [30].
ImageJ-compatible software for checking the quality of raw and reconstructed
data is provided by SIMcheck [31] and for reconstruction by fairSIM [32].

It is clear from 15 that higher resolutions are achieved with finer
illumination periods. If we illuminate the sample with the same objective lens
that we use to capture fluorescence, then the finest illumination we can
create is

$d_{\textrm{illumination}}=\frac{\lambda_{\textrm{illumination}}}{2n\sin\alpha}.$
(16)

Combining this with the emission diffraction limit, we see that the total
resolution is given by

$\displaystyle d_{\textrm{SIM}}=$
$\displaystyle\left(\frac{1}{d_{\textrm{illumination}}}+\frac{1}{d_{\textrm{emission}}}\right)^{-1}$
$\displaystyle=$
$\displaystyle\left(\frac{2n\sin\alpha}{\lambda_{\textrm{illumination}}}+\frac{2n\sin\alpha}{\lambda_{\textrm{emission}}}\right)^{-1}$
(17)

and so we have

$d_{\textrm{SIM}}=\frac{\lambda_{\textrm{emission}}}{\left(\beta+1\right)2n\sin\alpha},$
(18)

where
$\beta=\lambda_{\textrm{emission}}\big{/}\lambda_{\textrm{illumination}}$.
Hence, for the case where
$\lambda_{\textrm{emission}}=\lambda_{\textrm{illumination}}$, we see that SIM
provides double the resolution of uniform (epifluorescence) illumination.

While this is beyond the resolution of diffraction-limited imaging with
uniform illumination, the resolution is still limited, in some sense, by
diffraction. In order to not be diffraction-limited, we would need to be able
to create patterns with arbitrarily fine periods. Extensions of SIM that
provide this will be discussed in Section 3. These methods, theoretically,
create images that do not have a spatial band-limit. To distinguish SIM, and
other techniques, that provide improved resolution but are limited by
diffraction in both illumination and emission, rather than just in emission,
the term ‘extended resolution’ has been proposed [33, 34, 35, 36].

This allows us to classify methods into four distinct groups based on their
theoretical resolving power:

  1. 1.

_Standard resolution_ — as achieved by a normal wide-field epifluorescence
microscope;

  2. 2.

_Extended resolution_ — beyond standard resolution but still limited by the
combination of the illumination and emission diffraction limits, as achieved
by SIM and ISM (also referred to by some authors as _restricted super-
resolution_);

  3. 3.

_Enhanced resolution_ — beyond extended resolution, including methods with a
finite resolution limit, such as photoactivated non-linear SIM;

  4. 4.

_Super resolution_ — the subset of enhanced resolution methods not limited by
diffraction and theoretically capable of infinite resolution, discussed
further in Section 3, such as stimulated emission depletion microscopy and
saturated SIM.

Note that some authors, including Heintzmann in his answers to these questions
[20], would classify all methods capable of imaging beyond the _extended
resolution_ limit as _super-resolution_ methods, but this is not a sufficient
condition for ‘breaking’ the diffraction barrier [37]. Furthermore, it is
worth highlighting the difference between ‘imaging at the diffraction limit’,
i.e. with _standard resolution_ , and ’diffraction-limited resolution’, which
may be at any of the levels apart from diffraction-unlimited _super
resolution_.

Figure 1:  The origin of the diffraction limit. (a) Back focal plane (BFP) for
head-on and oblique transmission imaging of a grating at the Abbe limits. $\pm
1,0$ refers to the diffraction orders from the grating. (b) Wavevectors from a
single source lie on a spherical shell, with a 2D cross-section of this shell
shown. (c) The transfer function of free space is the autocorrelation of (b),
giving a ball of radius $2k$ (again, shown in cross-section). (d) Only a
subset of wavevectors are captured by the objective lens, determined by the
semiaperture angle, $\alpha$. (e) As with (c), the transfer function is given
by the autocorrelation of (d), with the lateral and axial diffraction limits
given by the magenta and green geometrical quantities calculated in (d).

##  2 Should high-NA TIRF-SIM, which can achieve lateral resolutions down to
84 nm, be considered as diffraction limited?

Total internal reflection fluorescence (TIRF) microscopy exploits the nature
of total internal reflection to produce a short-range, evanescent illuminating
field tethered to a refractive index boundary. Given two materials with
different refractive indices, $n_{1}>n_{2}$, Snell’s law states that light
incident on the boundary, from the $n_{1}$ side, at angles equal to or greater
than $\arcsin\left(n_{2}/n_{1}\right)$ cannot be transmitted and is instead
reflected. However, Maxwell’s equations demand continuity at the boundary and
so a rapidly-decaying evanescent field is created on the $n_{2}$ side.

For light incident at an angle $\alpha$, the exponential decay length of the
evanescent field is given by

$l=\frac{\lambda}{4\pi\sqrt{n_{1}^{2}\sin^{2}\alpha+n_{2}^{2}}},$ (19)

and is typically of the order of $100\text{\,}\mathrm{n}\mathrm{m}$. While
this ensures that only a thin section of the sample is illuminated, it does
not enhance axial resolution as such as the illumination cannot be scanned
over the sample. TIRF-SIM uses two such high-angle beams to (i) reduce
background fluorescence, (ii) provide finer patterns than can be created
directly in the $n_{2}$ medium [38].

When imaging a watery sample, for which $n=1.33$, the maximum possible
numerical aperture of a water-immersion objective is $n$ (i.e.
$\sin\alpha=1$), while oil immersion objectives with numerical apertures of
1.4, 1.49, 1.5 and even 1.7 are available. These objectives can form a pattern
right at the coverslip boundary at the limit of their numerical apertures and
have the near-field be patterned with the same spatial period, even though
there is no way of creating such a fine pattern in water using far-field
illumination.

Li et al. used such a 1.7-NA objective with between
30–$100\text{\,}\mathrm{W}\,\mathrm{c}\mathrm{m}^{-2}$ of
$488\text{\,}\mathrm{n}\mathrm{m}$ illumination at 1.58 NA and
~$510\text{\,}\mathrm{n}\mathrm{m}$ detection and claim a resolution of
$84\text{\,}\mathrm{n}\mathrm{m}$ [36]. Using a modified version of 1, where
we allow the numerical apertures of illumination and detection to be
different, we can back-calculate the effective detection NA for their claim to
be 1.38, consistent with the high end of the range of reported cytosolic
refractive indices [39]. The corresponding non-SIM TIRF resolution for this
case would then be $2.2\times$ worse, at $185\text{\,}\mathrm{n}\mathrm{m}$,
while the maximum possible far-field SIM resolution would be
$90\text{\,}\mathrm{n}\mathrm{m}$. Nevertheless, this
$84\text{\,}\mathrm{n}\mathrm{m}$ is still diffraction-limited, as the period
of the pattern created at the boundary is still limited by diffraction on the
$n_{1}$ side. It is interesting to note that the authors rejected the
possibility of collecting super-critical fluorescence (i.e. near-field
emission that is converted to propagating light by interacting with the
boundary) to further enhance their resolution calculation (i.e. the detection
NA is limited to the refractive index of the sample), despite other
measurements with the same objective showing that significant super-critical
light can be present [40].

It is particularly instructive to compare this answer (‘yes’) with the
companion given by Heintzmann: ”a clear ‘no’” [20]. For Heintzmann, taking 1,
all forms of SIM break the emission Abbe limit and hence break the diffraction
limit. This equation can be used to define a new limit, that of ‘restricted
super resolution’, introduced by Sheppard [41]. For TIRF-SIM, the structured
illumination pattern has a finer frequency than that permitted by the
refractive index of the sample, and so Heintzmann says that TIRF-SIM breaks
this ‘restricted super resolution’ criterion. Hence, the suggestion is that
TIRF-SIM is not diffraction-limited even under the considerations of this more
strict approach.

However, the fact remains that the fineness of the illumination pattern is
limited by diffraction, just in a medium of refractive index $n_{1}$ rather
than $n_{2}$. Even in the situation where the sample was mounted on a diamond
coverslip (ignoring the deleterious effects of the strong dispersion in such a
material) and we had an objective lens with numerical aperture matching that
of the refractive index of the material, 2.4, TIRF-SIM of the same sample as
Li et al. used could not produce a resolution better than

$d=\left(\frac{2n_{1}\sin\alpha}{\lambda_{\textrm{illumination}}}+\frac{2n_{2}\sin\alpha}{\lambda_{\textrm{emission}}}\right)^{-1}=\left(\frac{2\times
2.4}{$488\text{\,}\mathrm{n}\mathrm{m}$}+\frac{2\times
1.38}{$510\text{\,}\mathrm{n}\mathrm{m}$}\right)^{-1}=$65.6\text{\,}\mathrm{n}\mathrm{m}$.$
(20)

Hence, this author would class this as _extended resolution_ , not _super
resolution_ or even _enhanced resolution_.

##  3 Can non-linear SIM become broadly applicable and live-cell compatible?

Non-linear SIM breaks the assumption that the fluorescent response from the
sample is linearly related to the intensity of the illuminating light
($E(x)\neq S(x)\times I(x)$). The non-linearity adds extra harmonics (i.e.
spatial frequency components above that of the linear case) to the effective
illumination pattern, further extending the resolution. A patent application
(DE19908883A1) describing this idea was submitted by Heintzmann and Cremer in
March 1999, barely two months after the publication of the first SIM paper.
The simplest case to consider is a normal SIM system in which a further
sinusoidal pattern, $R(x)=1+\cos(px+\phi)$, is used to control the
fluorescence response, e.g. by photoswitching a fraction of the molecules
between dark and light states. Now, the camera data acquired have the form:

$D(x)=\left[I(x)\times R(x)\times S(x)\right]\ast H(x),$ (21)

and in the Fourier domain:

$\tilde{D}(k)=\left[\tilde{I}(k)\ast\tilde{R}(k)\ast\tilde{S}(k)\right]\times
H(x).$ (22)

Assuming that $I(x)$ and $R(x)$ have the same period, $p$, we now have a total
of five delta functions, at $k\in\left\\{-2p,-p,0,p,2p\right\\}$ (see Figure 2
on page 2b middle row). By phase-shifting both $I(x)$ and $R(x)$
simultaneously, we can unmix these five components with only five images,
producing a reconstruction with sample information at three times the
conventional limit.

Despite the further resolution increase, even this approach is diffraction-
limited, as $I(x)$ and $R(x)$ are both individually diffraction-limited.
Hence, the spatial frequencies available through this method are limited to a
certain band. In effect, all we have done is taken the normal SIM equation
(11) and replaced $I$ with $I_{\textrm{eff}}=I^{2}$. We could imagine
hypothetically extending the scheme even further, with yet another sinusoidal
pattern controlling the response of the sample to the light which controls the
fluorescence response, but this would correspond to $I_{\textrm{eff}}=I^{3}$
and so, while providing higher resolution (to four times the usual limit),
would still be diffraction-limited. Hence, while these methods would provide
_enhanced resolution_ beyond _extended resolution_ , they do not belong to the
class of diffraction-unlimited _super-resolution_ methods.

In order to achieve true super-resolution imaging with SIM, we need not just a
non-linear response, but a non-polynomial one. This is because a polynomial
response can be considered as a (scaled, repeated) convolution of the
illumination spectrum with itself and so, as we start with a finite number of
harmonics, we must end with a larger, but still finite, number. In contrast, a
non-polynomial response would provide an infinite number of harmonics and
hence cannot be said to be limited by diffraction (see Figure 2 on page 2a,b
bottom rows).

To recap, a uniform illumination pattern provides _standard resolution_ at the
diffraction limit. Patterned illumination with a linear fluorescence response
to illumination light provides _extended resolution_ up to twice the
diffraction limit (assuming excitation and emission light have the same
wavelength). Replacing the linear response with a non-linear, polynomial
response provides further _enhanced resolution_ , albeit one that is still
bandlimited. Finally, implementing a non-polynomial response allows for
diffraction-unlimited _super-resolution_ imaging.

The first non-polynomial SIM method to be proposed and demonstrated involves
saturating the fluorescence, such that a linear increase in illumination
intensity results in a sub-linear increase in fluorescence [42] (compare the
top and bottom rows of Figure 2 on page 2a). Saturation occurs when a
significant fraction of the illuminated molecules are in the excited state
when further excitation photons are incident, as there are insufficient
ground-state molecules that can be excited by the photon flux. So far, the
only demonstration of saturated SIM was provided by Gustafsson in 2005 [43],
which showed the ability to resolve a close-packed monolayer of
$50\text{\,}\mathrm{n}\mathrm{m}$-diameter fluorescent beads. Here, pulsed
illumination was used to provide an $I_{\textrm{eff}}\propto
1-\exp\left[-(I\sigma+k_{f})t_{p}\right]$, where $\sigma$ is the fluorescence
excitation cross-section of the fluorophore used, $k_{f}=1/\tau$ (where $\tau$
is the fluorescence lifetime) and $t_{p}$ is the pulse duration.

While, theoretically, an infinite number of harmonics were present, the
signal-to-noise ratio limited the number of ‘usable’ harmonics to three, i.e.
a five-fold resolution enhancement, although five harmonics were present in
calibration data. This is a good illustration of the point that, while _super-
resolution_ methods may theoretically be capable of providing superior
resolution to _enhanced-resolution_ methods, in practice the distinction is
often irrelevant.

Given that the average illumination irradiance was only
$35\text{\,}\mathrm{m}\mathrm{W}\,\mathrm{c}\mathrm{m}^{-2}$, one may wonder
why this method has never been demonstrated on biological samples. This is
because considering only the average irradiance is misleading, as the
illumination was pulsed. Averaging instead over the pulse duration gives a
peak irradiance of
$10\text{\,}\mathrm{M}\mathrm{W}\,\mathrm{c}\mathrm{m}^{-2}$, comparable to
the levels used for multiphoton microscopy. Photodamage mechanisms are known
to be non-linear with respect to light intensity, such that a doubling in
intensity produces more than double the damage. Nevertheless, it is not clear
that this irradiance is necessarily too damaging for imaging biological
samples, especially as the saturation irradiance of EGFP has been measured to
be similar, at $11\text{\,}\mathrm{k}\mathrm{W}\,\mathrm{c}\mathrm{m}^{-2}$
[44], and multiphoton microscopy has successfully been used to image many
varied biological samples. While multiphoton microscopy uses redder
wavelengths of illumination, which is suggested to contribute less to
photodamage, standard use of confocal fluorescence microscopy also often
operates at illumination irradiances close to the saturation irradiances of
similar fluorophores.

Figure 2:  Real space (a) and Fourier domain (b) comparisons of linear SIM
(top row), non-linear SIM with a quadratic non-linearity (middle row), and
non-polynomial SIM with fluorescence saturation (bottom row). The pattern
period and spatial frequency are denoted by $x_{p}$ and $k_{p}$, respectively.
Note that, in addition to containing an extra harmonic at $k=\pm 2k_{p}$, the
quadratic case has a stronger component at $k=\pm k_{p}$ than in the linear
case. Furthermore, while harmonics beyond the plot range are present in the
saturated fluorescence case, their relative strengths compared to the DC peak
at $k=0$ are so weak that the harmonic at $k=\pm k_{p}$ is already barely
visible. The saturated fluorescence case was calculated assuming a mean
excitation irradiance of 1 photon per cross-section per lifetime and
continuous wave excitation.

Another non-polynomial SIM approach, published in 2012 by Rego et al.,
involved saturated depletion of fluorescence, in which the photoswitchable
nature of the fluorescent protein Dronpa was used to confine fluorescence to
thin bands [45]. This followed a 2008 demonstration of the same mechanism by
Hirvonen et al., although here the authors were unable to successfully
reconstruct a high-resolution image [46]. The idea of saturating the depletion
of fluorescence is one example of the RESOLFT concept [37], which was
demonstrated in a parallelised, wide-field manner in 2007 [47]. In the work of
Rego et al., images with resolutions better than
$50\text{\,}\mathrm{n}\mathrm{m}$ were produced despite the required
irradiances being of the order of only
$10\text{\,}\mathrm{W}\,\mathrm{c}\mathrm{m}^{-2}$. Unfortunately, only fixed
cells were imaged as one reconstructed frame required $945\text{\,}\mathrm{s}$
of imaging time.

In order to develop a live-cell compatible method, Li et al. abandoned the use
of non-polynomial SIM and instead focussed on developing a technique that
would work without the need for saturation [36]. To this end, they used the
photoswitchable fluorescent protein Skylan-NS for patterned activation, rather
than depletion. This enabled the timelapse imaging of live cells at
resolutions better than the $84\text{\,}\mathrm{n}\mathrm{m}$ of their high-NA
linear SIM approach (see Section 2), producing a movie of 30 frames at
$65\text{\,}\mathrm{n}\mathrm{m}$ resolution. Furthermore, by increasing the
activation irradiance to $490\text{\,}\mathrm{W}\,\mathrm{c}\mathrm{m}^{-2}$,
they began to saturate the transition, creating a non-polynomial method with
an extra harmonic above the noise floor. This increased the resolution to
$45\text{\,}\mathrm{n}\mathrm{m}$, but only provided 12 frames at useful
signal-to-noise ratios.

Hence, both non-linear and non-polynomial SIM have been demonstrated to be
live-cell compatible. However, while linear SIM can image live samples for
hundreds of time points, non-linear approaches are currently limited to a few
tens of time points. In addition, while non-polynomial methods theoretically
provide unlimited resolution, signal-to-noise concerns show that they are, in
practice, less well-suited to extending resolution than carefully chosen non-
linear methods such as the patterned activiation of Li et al.. As an example
of this, compare the ‘quadratic’ and ‘saturated’ plots in Figure 2 on page 2b.
While the saturated case has more harmonics in total, the strengths of those
at $k_{p}$ and $2k_{p}$ are much stronger in the quadratic case. Given a
finite photon budget, more photons must be spent to raise these harmonics
above the spectrally-flat noise floor using saturation. This problem is
exacerbated at higher illumination powers as, while the strength of higher
harmonics increase, the strength of the first harmonic decreases (see
References).

Numerical simulations suggest that saturation using continuous wave
illumination, for which an illumination of $I(x)$ is mapped to an effective
excitation of $I/(I+1/\tau)$, produces harmonics that decay geometrically
beyond the first harmonic, with the strength of the first harmonic decaying
with increased saturation strength. For pulsed illumination, a similar pattern
of harmonics is seen, with a short pulse length mimicking the effect of a
higher intensity. For polynomial illumination response with $N$ harmonics
(i.e. $2N+1$ peaks in the spectrum) and perfect modulation contrast, the
strength of the $n$th harmonic is given by

$s=\frac{(N!)^{2}}{(N+n)!(N-n)!},$ (23)

where $s=1$ would correspond to a harmonic as strong as the DC peak ($n=0$).
These strengths can be read off from every other row in Pascal’s triangle
(dividing by the central element), with uniform illumination corresponding to
the first row.

##  4 Do you need ‘switching’ of states for non-linear super-resolution
imaging?

So far, all of our discussions of non-linear SIM and super-resolution imaging
have used continuous models. Despite this, we know that light is quantised
into photons and fluorescent samples are quantised into fluorophores. As such,
all the approaches discussed in Section 3 feature ‘switching’ of states, but
this is also true of normal fluorescence imaging. Hence, switching is not a
sufficient condition for non-linear super-resolution microscopy. While
switching of states can provide useful non-linearities, it alone is not
sufficient to encompass all possible non-linearities (e.g., Rabi oscillations
[48]), merely those that have been successfully demonstrated to work with SIM
so far. Hence, switching is theoretically not a necessary condition for non-
linear super resolution, but has been utilised in all methods implemented so
far. For more details on alternative super-resolution mechanisms, see
Heintzmann’s answer in reference [20].

##  5 Do high-quality SIM images require reconstruction in Fourier space?

In general, SIM reconstructions can be split into two separate steps: (i)
estimating the parameters of the pattern of illumination used (period,
orientation and phase) and (ii) using these parameters to separate and
recombine information contained within the raw images. In many
implementations, both of these steps have been conducted in the Fourier
domain, although there is no inherent reason why they must be.

Pattern estimation can potentially be skipped if the system is well-calibrated
and the sample does not distort the illumination pattern. If the pattern
parameters are unknown, they are typically acquired via cross-correlating
images with different phases. Assuming the pattern parameters are well-known,
either via calibration or through a pattern estimation routine that processes
the raw data, a typical reconstruction would separate the different
information components, shift them to their proper locations in Fourier space,
and recombine them to form an image.

Separating the information components is typically thought of as solving the
Fourier space matrix equation

$\begin{bmatrix}\tilde{D}_{1}(k)\\\ \tilde{D}_{2}(k)\\\ \tilde{D}_{3}(k)\\\
\end{bmatrix}=\underbrace{\begin{bmatrix}1&1&1\\\ 1&e^{2\pi i/3}&e^{-2\pi
i/3}\\\ 1&e^{4\pi i/3}&e^{-4\pi i/3}\\\
\end{bmatrix}}_{M}\begin{bmatrix}2\tilde{H}(k)\tilde{S}(k)\\\
\tilde{H}(k)\tilde{S}(k+p)\\\ \tilde{H}(k)\tilde{S}(k-p)\\\ \end{bmatrix},$
(24)

where $M$, the mixing matrix, has been calculated for the case of equal
$2\pi/3$ phase steps with no global phase offset, for simplicity. However,
this mixing matrix acts pixel-wise and so there is no reason to not consider
the real space equivalent instead:

$\begin{bmatrix}D_{1}(x)\\\ D_{2}(x)\\\ D_{3}(x)\\\
\end{bmatrix}=\underbrace{\begin{bmatrix}1&1&1\\\ 1&e^{2\pi i/3}&e^{-2\pi
i/3}\\\ 1&e^{4\pi i/3}&e^{-4\pi i/3}\\\
\end{bmatrix}}_{M}\begin{bmatrix}2H(x)\ast S(x)\\\ H(x)\ast S(x)e^{2\pi
ipx}\\\ H(x)\ast S(x)e^{-2\pi ipx}\\\ \end{bmatrix}.$ (25)

This particular mixing matrix can be seen as a discrete Fourier transform
along the phase direction, but this is not generally true and is not what is
meant by a reconstruction in Fourier space (as this operates along the spatial
directions). If we allow our image processing routine to use complex numbers
then we can separate each component by calculating the matrix inverse of $M$
and applying it to the raw data.

Shifting each component to the correct location in Fourier space can be
achieved in real space, via the Fourier shift theorem, by multiplying with a
phase ramp (see References). It should, however, be noted that in many
reconstructions a Fourier transform and inverse transform of a padded spectrum
are used to artificially reduce the pixel size before shifting, as the raw
data are not Nyquist-sampled for the higher resolution reconstruction. Staying
in real space either requires oversampled raw data or an alternative upscaling
technique.

Given shifted components, all that remains is to combine them to form a SIM
image. This can be achieved with a direct sum, but most reconstructions choose
to weight different parts of the separated components differently depending on
their exact location in Fourier space, in an attempt to counteract the effects
of the OTF.

As an alternative to this view of separating and recombining components, we
can think of the process of structured illumination imaging as akin to
amplitude modulation (AM), as used in early radios. Here, the carrier signal
is our illumination pattern, the sample structure is our message signal and
the recorded data is the product of these two, as is the AM signal. To
demodulate an AM signal and recover the message signal, a radio can use a
‘product detector’, which heterodynes (i.e. multiplies) the AM signal with a
signal from a local oscillator designed to match the carrier signal exactly.
This suggests that a similar approach might be applicable to SIM data.

Consider a two-beam SIM system where, for simplicity, we will only consider
one pattern orientation and perfect modulation contrast. In the usual way, the
data acquired, $D_{m}(x)$, where $m$ is used as a phase index, are equal to
the product of the illumination and sample, convolved with the PSF, $H(x)$:

$\displaystyle D_{m}(x)$ $\displaystyle=\left[I(x)\times S(x)\right]\ast H(x)$
(26) $\displaystyle=\left[\left(1+\cos(px+\phi_{m})\right)\times
S(x)\right]\ast H(x).$ (27)

As usual, in the Fourier domain:

$\tilde{D}_{m}(k)=\bigg{[}2\tilde{S}(k)+e^{i\phi_{m}}\tilde{S}(k+p)+e^{-i\phi_{m}}\tilde{S}(k-p)\bigg{]}\tilde{H}(k).$
(28)

Let us now consider multiplying our AM signals, $D_{m}(x)$, with a ‘local
oscillator’, i.e. another sinusoidal pattern with the same phase and period:

$M_{m}(x)=\alpha+\beta\cos(px+\phi_{m}).$ (29)

In the Fourier domain, our local oscillator takes the form:

$\tilde{M}_{m}(k)=\alpha\delta(k)+\beta e^{i\phi_{m}}\delta(k+p)+\beta
e^{-i\phi_{m}}\delta(k-p).$ (30)

Now, let us heterodyne our signals:

$P_{m}(x)=D_{m}(x)\times M_{m}(x),$ (31)

such that in the Fourier domain we have:

$\tilde{P}_{m}(k)=\tilde{D}_{m}(k)\ast\tilde{M}_{m}(k).$ (32)

Calculating this gives:

$\displaystyle\tilde{P}_{m}(k)=$ $\displaystyle
2\alpha\tilde{S}(k)\tilde{H}(k)$ $\displaystyle+\alpha
e^{i\phi_{m}}\tilde{S}(k+p)\tilde{H}(k)+\alpha
e^{-i\phi_{m}}\tilde{S}(k-p)\tilde{H}(k)$ $\displaystyle+2\beta
e^{i\phi_{m}}\tilde{S}(k+p)\tilde{H}(k+p)$ $\displaystyle+2\beta
e^{2i\phi_{m}}\tilde{S}(k+2p)\tilde{H}(k+p)+\beta\tilde{S}(k)\tilde{H}(k+p)$
$\displaystyle+2\beta e^{-i\phi_{m}}\tilde{S}(k-p)\tilde{H}(k-p)$
$\displaystyle+\beta\tilde{S}(k)\tilde{H}(k-p)+2\beta
e^{-2i\phi_{m}}\tilde{S}(k-2p)\tilde{H}(k-p).$ (33)

If we impose the conditions that

$\sum_{m}e^{i\phi_{m}}=0\quad\&\quad\sum_{m}e^{2i\phi_{m}}=0$ (34)

then most of these terms cancel when summed, giving:

$\displaystyle\sum_{m}\tilde{P}_{m}(k)=$ $\displaystyle
2m\alpha\tilde{S}(k)\tilde{H}(k)$
$\displaystyle+m\beta\tilde{S}(k)\tilde{H}(k+p)+m\beta\tilde{S}(k)\tilde{H}(k-p).$
(35)

Hence, the reconstructed image, obtained by multiplying each raw image with
our local oscillator and summing all the images, has the same form as a
traditional SIM reconstruction if we set $\alpha=\beta=1$. This form also
makes clear why a SIM acquisition with only two phases does not reconstruct
properly: in this case $\sum_{m}e^{i\phi_{m}}=0$ as desired, but
$\sum_{m}e^{i\phi_{m}}=2$ and so we are left with terms of the form
$\tilde{S}(k\pm 2p)\tilde{H}(k\pm p)$ that we do not want.

Figure 3 on page 3 shows an example of this reconstruction process for
simulated data. Here, the ground truth sample (Figure 3 on page 3a) was
engineered to be periodic, to avoid ‘cross’ artefacts in the spectra (Figure 3
on page 3d,e,f). In addition, its form makes the Moiré effect of the fringe-
like illumination particularly visible (Figure 3 on page 3h).

Considering a three-beam system, for which

$\displaystyle\tilde{D}_{m}(k)=$
$\displaystyle\bigg{[}3\tilde{S}(k)+2e^{i\phi_{m}}\tilde{S}(k+p)+2e^{-i\phi_{m}}\tilde{S}(k-p)$
$\displaystyle+e^{2i\phi_{m}}\tilde{S}(k+2p)+e^{-2i\phi_{m}}\tilde{S}(k-2p)\bigg{]}\tilde{H}(k)$
(36)

and

$\displaystyle\tilde{M}_{m}(k)=$ $\displaystyle\alpha\delta(k)+\beta
e^{i\phi_{m}}\delta(k+p)+\beta e^{-i\phi_{m}}\delta(k-p)$
$\displaystyle+\gamma e^{2i\phi_{m}}\delta(k+2p)+\gamma
e^{-2i\phi_{m}}\delta(k-2p),$ (37)

this approach provides

$\displaystyle\frac{1}{m}\sum_{m}\tilde{P}_{m}(k)=$ $\displaystyle
3\alpha\tilde{S}(k)\tilde{H}(k)$
$\displaystyle+2\beta\tilde{S}(k)\tilde{H}(k+p)+2\beta\tilde{S}(k)\tilde{H}(k-p)$
$\displaystyle+\gamma\tilde{S}(k)\tilde{H}(k+2p)+\gamma\tilde{S}(k)\tilde{H}(k-2p)$
(38)

and so we see that we should set $\alpha=\beta=\gamma=1$. Indeed, for $n$
beams, the correct $\tilde{M}_{m}(k)$ is one which is spectrally flat, i.e.
all delta functions have equal amplitude. This reconstruction approach will be
revisited in Section 8 to consider the case of non-sinusoidal illumination.
Similar real-space reconstruction approaches were also considered by Cragg &
So early-on in the development of SIM [4, 49].

Intriguingly, the idea of using a patterned illumination and a local
oscillator to enhance the resolution of an imaging system predates the idea of
structured illumination microscopy by more than 30 years. In a series of
papers, beginning in 1966, Lukosz investigated the possibility of enhancing
the resolution of an imaging system by inserting opaque masks into object and
image space [50, 51, 52]. While the details of the method are quite different
to the modern idea of structured illumination microscopy, the underlying idea
and the realisation that an increase in spatial bandwidth must be accompanied
by a decrease in temporal bandwidth were already present.

Figure 3:  An overview of a reconstruction using the real-space method
described in the main text. (a) Ground truth sample structure. (b) Wide-field
image obtained with uniform illumination. (c) Complete SIM reconstruction.
(d,e,f) Logarithmically-scaled Fourier spectra of (a,b,c). (g) Linearly
deconvolved wide-field image. (h) Single frame of SIM acquisition. (i) Frame
from (h) multiplied by local oscillator. (j) Linearly deconvolved SIM
reconstruction. (k) Logarithmically-scaled Fourier spectrum of (h). Peaks
circled in cyan correspond to the DC peaks of $\tilde{S}(k\pm p)$ in
$\tilde{D}_{m}(k)$. (l) Logarithmically-scaled Fourier spectrum of (i). Peaks
circled in cyan correspond to the undesired DC peaks of $\tilde{S}(k\pm p)$
and $\tilde{S}(k\pm 2p)$ in $\tilde{P}_{m}(k)$. Note that these peaks are not
present in (f) as they are annihilated when all phases $m$ are summed over
(the illumination peaks are deliberately chosen to be well inside the cyan
line, for clarity).

##  6 Can SIM be used for deep tissue imaging?

Using SIM for deep tissue imaging is complicated by two issues: (i) sample-
induced aberrations and (ii) insufficient stripe contrast caused by out-of-
focus and scattered light. We will restrict the discussion here to that of
sample-induced aberrations and discuss (ii) in Section 7. In larger samples,
it is common for the local refractive index to not be uniform over the entire
sample. This means that light that arrives at or departs from a given location
in the sample experiences a path-dependent phase shift with respect to light
travelling in a uniform refractive index. These unanticipated phase shifts
mean that the light is not properly focussed by the microscope system. This is
true for both illumination light and fluorescence emission.

Fortunately, it has been shown by Arigovindan et al. that aberrations
predominantly affect the fluorescence emission light, as the effect of phase
shifts on the illumination pattern can be taken into account in the
reconstruction routine [53]. However, this assumes that the variation across
the field-of-illumination is small (i.e. the ‘isoplanatic patch’ is large),
otherwise the image must be split into blocks over which the pattern
parameters are fixed. Correcting for aberrations in the fluorescence emission
can be achieved using an ‘adaptive optic’, typically a deformable mirror,
which applies a complementary phase shift to each part of the wavefront to
cancel the aberrations. Usefully, this can not only be used to compensate for
sample-induced aberrations, but any residual aberrations from the microscope
itself. However, as the same correction is applied to every point in the
field-of-view there is once again a requirement that the variation across the
field is small. This is a limitation of all wide-field adaptive optics systems
and is in constrast to point-scanning systems, for which the aberration
correction can be updated for every scan position, if necessary [54].

Such adaptive optics SIM systems have been constructed by a number of groups,
with most differences being in the way an appropriate wavefront correction is
obtained. Using a multiphoton guide star allows for direct wavefront sensing,
in which the wavefront is measured using a Shack–Hartmann sensor, but
typically requires an expensive femtosecond laser which may cause excess
sample damage [55, 56]. In contrast, sensorless schemes assume that the
wavefront can be decomposed into a sum of orthogonal modes, and simply see
which combination of modes maximise an image quality metric. Recently,
Žurauskas et al. exploited the nature of the structured illumination itself to
futher improve the performance of such a sensorless scheme [57]. These
approaches have the benefit of not requiring a guide star, but mean that a
large number of images must be acquired to obtain a good correction (at best
$2N+1$ for $N$ modes) [54].

##  7 How can the fundamental limitation of SIM, i.e. generating sufficient
stripe contrast in densely labelled and/or extended biological structures due
to out-of-focus light, be addressed?

Even for situations in which the adaptive optics schemes discussed in Section
6 are not required, obtaining high-quality SIM reconstructions can be
challenging in thicker samples due to the out-of-focus light contaminating the
image and hence reducing stripe contrast. For single-cell imaging, the out-of-
focus light produced by 2-beam or 3-beam SIM is normally insufficient to
preclude high-quality reconstructions. However, for tissue imaging, background
fluorescence from scattered light and out-of-focus regions of the sample can
quickly degrade imaging performance.

In general, there are two approaches to solve this problem: (i) avoid
producing out-of-focus light in the first place; (ii) somehow remove the out-
of-focus light before it hits the detector. In conventional fluorescence
microscopy, (i) is the approach taken by multiphoton and light sheet
microscopy, while (ii) is that taken by confocal microscopy. All three
approaches can usefully be combined with SIM.

In 2012, Andresen & Pollok et al. created a sinusoidal multiphoton SIM pattern
by scanning an array of 32 beamlets perpendicular to the beamlet line, with
phase-stepping and reconstruction in the usual manner [58]. However, as each
beamlet is incoherent with the others, the fine fringe patterns necessary for
a high resolution enhancement would have too low a modulation contrast in this
scheme, precluding a full doubling of resolution. Later, various authors
combined multiphoton excitation with some form of image scanning microscopy
(discussed further in Section 8), which uses a focussed spot for excitation
and a different reconstruction scheme [59]. This has also been combined with
adaptive optics to improve the imaging performance at depth further still
[60].

Initially, light sheet microscopy was first combined with incoherent OS-SIM to
enhance optical sectioning, rather than resolution [61]. Soon after, Planchon
et al. used an incoherent superpositions of Bessel beams to provide enhanced
resolution as well [62]. However, this still used the reconstruction approach
of OS-SIM, resulting in a spatially-varying point spread function and a
concomitant loss of ability to describe the image formation process linearly.
Gao et al. developed this further, using the traditional reconstruction
approach of SR-SIM to enhance effective resolution further still and maintain
linear image formation [63]. Here, the authors showed that this Bessel light
sheet structured illumination approach provided clearer images of live LLC-PK1
cells in anaphase than wide-field SIM, despite the superior theoretical
resolution of the latter. In the final development of Bessel light sheet SIM,
Chen, Legant & Wang et al. used a coherent superposition of Bessel beams to
create a so-called ‘lattice’ light sheet, featuring improved modulation
contrast and a higher duty-cycle [64]. However, one drawback of all these
approaches is that only one pattern orientation is used, resulting in an
anisotropic resolution enhancement. This is further discussed in Section 15.

The prototypical example of combining SIM with confocal microscopy is image
scanning microscopy, discussed in Section 8. One of the main drawbacks of this
approach is the need to acquire an entire image for each scan position of the
confocal spot, meaning that hundreds or thousands of images must be recorded
to reconstruct one enhanced resolution image. As such, multifocal structured
illumination microscopy (MSIM) was introduced, using multiple spots of
illumination in parallel [65]. Following this, in was realised independently
by multiple research groups that the reconstruction procedure was amenable to
an all-optical implementation [66, 67, 68]. A further approach, using a
32-segment confocal detector and multichannel photomultiplier tube has also
been introduced in Zeiss’ Airyscan product [69].

In all these approaches, the use of a confocal pinhole (either physical or
digital) introduces optical sectioning, making these techniques less sensitive
to out-of-focus light than traditional SIM. However, the effective resolution
and contrast in non-scattering samples is lower, due to the less effective way
in which the illumination brings higher spatial frequencies into the passband.
Compared with traditional confocals, apart from the need for smaller pixel
pitches to satisfy the Nyquist-Shannon sampling criterion, there is no
drawback to using these SIM-like approaches, with benefits not only in
resolution but in signal-to-noise ratio, due to the superconcentration of
light [70, 71].

##  8 Should image scanning microscopy be considered a form of SIM and what
forms of structured illumination could be used other than stripes?

Image scanning microscopy (ISM), first proposed by Sheppard in 1988 [72] and
first demonstrated by Müller and Enderlein in 2010 [73], is experimentally
quite different from SIM. In ISM, a tightly-focussed illumination spot is
scanned over the sample, with a camera image of the emission recorded at every
scan position. For each position in the spot image, the sample location most
likely to have produced that intensity is not, as one might assume, at the
location corresponding to the image position, but instead is located $\beta/2$
between the illumination scan position and image position, where
$\beta=\lambda_{\textrm{emission}}\big{/}\lambda_{\textrm{illumination}}$ is
the Stokes’ shift factor (i.e. for no Stokes’ shift, the most-likely location
is halfway between).

In ISM processing, every pixel in each image is relocated in this way, with
the corresponding intensity being added to an upsampled pixel grid. At first,
this seems very different from the SIM discussed so far. However, the idea
that non-uniform illumination can provide higher resolution information is
still the same, albeit with a very different form of structured illumination.

Indeed, the real-space reconstruction approach introduced in Section 5 can be
extended to process ISM data as well, albeit not in a way that uses photon
reassignment. For the purposes of visualisation and computational speed,
Figure 4 on page 4 uses multiple focussed illumination spots in parallel, as
produced by MSIM [65]. As before, each raw image (Figure 4 on page 4h) is
multiplied by a pattern that mimics the spatial period and phase of the
illumination, but is spectrally flat. These multiplied images (Figure 4 on
page 4i) are then summed to produce the final reconstruction (Figure 4 on page
4c). As shown by a comparison of the spectra in Figure 4 on page 4e and f, the
processed image contains information at higher spatial frequencies than in the
simple sum of the raw data.

Furthermore, the real-space reconstruction approach can be modified to handle
quasiarbitrary illumination patterns. As an example of this, Figure 5 on page
5 shows a workflow for using a projection of the text ‘SIM’ as an illumination
pattern. References shows the steps in the acquisition and reconstruction as
data is collected. While not necessarily spectrally optimal, a comparison of
the spectra of the raw sum and remultiplied sum, along with linearly
deconvolved images, shows that higher resolution information has successfully
been extracted. An alternative approach in which the illumination pattern is
not known, called ‘blind SIM’, has also been investigated [74, 75, 76, 77].

Figure 4:  The real-space method applied to (parallelised) ISM data (MSIM).
(a) Ground truth sample structure. (b) Wide-field image obtained with uniform
illumination. (c) Complete SIM reconstruction. (d,e,f) Logarithmically-scaled
Fourier spectra of (a,b,c). (g) Linearly deconvolved wide-field image. (h)
Single frame of acquisition. (i) Frame from (h) multiplied by local
oscillator. (j) Linearly deconvolved SIM reconstruction. (k,l)
Logarithmically-scaled Fourier spectrum of (h,i). Cyan cicles in (e,f)
correspond to the bandlimit of the wide-field data.  Figure 5:  The real-space
method applied to an ISM-like acquisition where, instead of a focussed spot, a
projection of the text ‘SIM’ is used for illumination. (a) Ground truth sample
structure. (b) Wide-field image obtained with uniform illumination. (c)
Complete SIM reconstruction. (d,e,f) Logarithmically-scaled Fourier spectra of
(a,b,c). (g) Linearly deconvolved wide-field image. (h) Single frame of
acquisition. (i) Frame from (h) multiplied by local oscillator. (j) Linearly
deconvolved SIM reconstruction. (k,l) Logarithmically-scaled Fourier spectrum
of (h,i). Cyan cicles in (e,f) correspond to the bandlimit of the wide-field
data.

##  9 How does sparse illumination compare to dense illumination in linear and
non-linear SIM?

As discussed in Section 7, SIM requires the illumination pattern to have good
contrast in order to work well. For thicker samples, where much out-of-focus
light contaminates the image of the plane of interest, sparse illumination is
helpful. However, for maximum performance at high spatial frequencies, a dense
illumination is required. In addition, a greater number of phase shifts must
be used.

This can be understood by considering real space and the Fourier domain as
reciprocal spaces. Hence, sparse illumination results in a dense Fourier
spectrum and vice versa. A dense Fourier spectrum means that more components
must be unmixed, and so a greater number of phase shifts must be used.

Furthermore, dense illumination is more readily combined with total internal
reflection fluorescence microscopy. To see this, consider an MSIM-like
illumination pattern, where the back focal plane of the objective has the
illumination shown in Figure 6 on page 6a, with circular polarisation. This
produces an illumination in the focal plane as shown in Figure 6 on page 6b.
If we wished for the illumination to be confined to the TIRF-regime, then we
must ensure that only the outer annulus of the back focal plane is
illuminated, as shown in Figure 6 on page 6c. This results in the illumination
shown in Figure 6 on page 6d. Here, the interference in the pattern causes
unwanted illumination, leading to excess photodamage and noise.

Figure 6:  Simulated back focal planes (a,c) and illumination patterns (b,d)
for MSIM (a,b) and TIRF-MSIM (c,d). A non-linear colourmap is used to enhance
the visibility of the undesired interference patterns between spots in (d).

Recently, Ingerman et al. considered a related question specifically for dense
illuminations: is it better to use two-dimensional patterns or sequentially
rotated one-dimensional patterns for linear and non-linear SIM [78]? Perhaps
surprisingly, they found that two-dimensional patterns were better for high-
resolutions in non-linear SIM while one-dimensional patterns were superior for
linear SIM, although the performance advantage of 2D patterns disappeared when
non-photoswitchable background was introduced. In addition, for the non-linear
case, they found that two-dimensional patterns require almost two-fold more
exposure to off-switching light as one-dimensional patterns, for the same
resolution enhancement. This suggests that one-dimensional methods are
superior if photodamage from off-switching light is present.

These considerations began more than a decade before appearing in print, but
publication was delayed by the deteriorating health and untimely death of Mats
Gustafsson in 2011. The version initially submitted in 2008 was revived and
modified by Ingerman, London and Heintzmann as a tribute, being finally
published in 2019.

##  10 Can SIM be used to improve the resolution of (Rayleigh scattering)
transmission microscopy?

No. As noted in Section 1, the resolution limit for oblique illumination is
already twice as good as that for head-on illumination. A structured
illumination setup can provide such oblique illumination, but there is no
further resolution to be gained. For further information, see the excellent
article by Wicker & Heintzmann [79].

##  11 Can we generate ‘true’ super-resolution images from simple instruments
enhanced with machine-learning-based algorithms?

Machine learning has proven to be a powerful tool for diverse image processing
applications, such as image segmentation, denoising, deconvolution etc. [80,
81, 82, 83]. However, care should be taken to distinguish between approaches
that try to _extract_ information from an image and those that try to _add_
information to an image. With good training, it has been possible to develop
multiple deep-learning approaches that convert an image, or set of images
(such as a SIM acquisition sequence), into a higher resolution image [84, 85,
86, 87]. While processing a set of images may fall into the ‘extract’
category, processing a single image in such a way must necessarily fall into
the ‘add’ category. As such, machine-learning will never be able to truly
replace, rather than augment, a super-resolution technique. Nevertheless, such
single-image approaches may be able to improve the effective resolution in a
manner similar to deconvolution.

##  12 Can research-grade super-resolution (SIM) microscopes be built cost-
efficiently?

All structured illumination microscopes are relatively cheap — the expense
lies in the time required to make them work well! A basic system can be
constructed using a rotation mount, piezo translation mount, diffraction
grating, polariser and a standard laser, plus some lenses and mirrors (as in
the very first SIM publications [1, 2, 5]). However, such a system will be too
slow for live-cell imaging and can severely limit the throughput of fixed-
sample imaging [8, 9]. Constructing a SIM system fast enough for live-cell
imaging requires a way of generating the sinusoidal illumination pattern
without physical rotation of a grating. In general, two methods are common:
spatial light modulators (SLMs) and interferometric systems.

Spatial light modulators can, to a first approximation, be thought of as
reprogrammable diffraction gratings and used in a similar way. However,
instead of shifting and rotating the SLM, the pattern displayed by the SLM is
instead updated to match. This arrangement maintains the common-path layout of
grating-based systems, minimising the sensitivity to air currents and
temperature fluctuations. However, fast-switching ferroelectric SLMs have a
very low diffraction efficiency (often $<5\%$) and so require powerful laser
sources to be used. Fast-switching nematic SLMs, with much higher diffraction
efficiencies, are now commercially available, albeit at high cost and not yet
capable of the frame rates of ferroelectric SLMs.

Interferometric systems normally feature some kind of Michelson/Twyman–Green
or Mach–Zehnder interferometer, where pattern orientation is changed by
changing optical paths and phase steps are accomplished by changing the length
of one arm of the interferometer [88, 89]. These approaches are much more
photon-efficient than SLM-based systems and, as the pattern period changes
with wavelength, rather than being fixed, can be used to ensure the same
effective illumination NA over all wavelengths in use. This is the approach
taken by the commercial OMX Blaze system [90]. However, such interferometers
are extremely sensitive to air currents and temperature fluctuations, as any
phase shift in one arm of the interferometer directly couples to a phase shift
in the illumination pattern. In an attempt to circumvent this issue, fibre-
based interferometeric SIM systems have recently been developed [91, 92].

Overall, SLM-based systems are by far the most common, in both research-grade
and commercial instruments. In particular, Forth Dimension Displays have
cemented themselves as the preferred supplier of ferroelectric SLMs for the
SIM community, due to their high pixel count and rapid switching speed.
Nevertheless, successful operation of such an SLM is tricky for a newcomer,
due to effects from the binary nature of the pixels, interpixel vias, duty-
cycle requirements and low photon-efficiency. As such, there has recently been
much interest in the development of systems that use a different type of SLM,
namely digital micromirror devices (DMDs) [93, 94]. Found in many projectors,
these devices rely on tilting tiny micromirrors to direct light in one
direction or another, acting as a binary amplitude display. While these
provide high photon-efficiencies, their tilted-mirror nature means that they
act akin to blazed gratings, with a strongly wavelength-dependent distribution
of power into different diffraction orders. As such, their use is currently
restricted to those labs interested in researching how best to make use of
DMDs for SIM, rather than following the tried-and-tested approach of
ferroelectric SLMs and powerful lasers.

##  13 How can information about single-molecule detection be best combined
with the knowledge of the illumination structure?

This is an area of ongoing research, prompted by the development of MINFLUX
[95]. In MINFLUX, an intensity minimum at the centre of a doughnut beam is
used to localise single fluorophores by iteratively updating the doughnut
position and minimising the photon flux. While this approach can provide
excellent localisation precisions ($<$1\text{\,}\mathrm{n}\mathrm{m}$$), it is
inherently slow as it only probes one molecule at a time. Since the initial
publication of the MINFLUX method, there have been a number of publications
which combine full-field fringe-like structured illumination with the MINFLUX
concept to provide high-precision localisations at faster rates [96, 97, 98,
99]. Most recently, Schmidt et al. have shown that these methods only use half
the localisation information encoded by the structured illumination and
demonstrate a further method that, unlike previous approaches, achieves the
Cramér–Rao lower bound [100].

##  14 Can the axial resolution of a SIM microscope be enhanced beyond a two-
fold improvement?

Enhancing the axial resolution of a SIM system requires accessing the sample
from the other side, either to introduce extra wavevectors for illumination,
or to capture extra wavevectors of fluorescence. Perhaps the most impressive
SIM microscope yet built, the I5S of Shao et al., does both [6]. By using a
second opposing high-numerical-aperture objective, a further three beams of
illumination are introduced. The same objective is also used to capture
fluorescence, with the two objectives being aligned to such a precision that
the fluorescence captured by both can be made to interfere via a beamsplitter,
despite the ~$1\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$ coherence
length of the light. This provides an almost-isotropic, better-
than-$100\text{\,}\mathrm{n}\mathrm{m}$ resolution in all three dimensions,
providing volumes of objects with no preferred viewing direction.

Despite this impressive performance, I5S is not widely used, due to the
challenging alignment and stability requirements for its operation. In
addition, the very short coherence length of fluorescence emission
(~$1\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$) means that sample-induced
phase differences restrict the technique to imaging thin single cells.
Recently, Manton et al. proposed that a lower-numerical-aperture, long-
working-distance dipping objective could be used to introduce a single
counterpropagating beam and provide much of the axial resolution enhancement
of I5S in a much simpler setup [101]. While such a system is theoretically
capable of providing 3D resolutions better than
$125\text{\,}\mathrm{n}\mathrm{m}$, the transfer function at high spatial
frequencies is lower than that of an I5S microscope without interferometric
detection. As such, we expect that six-beam structured illumination provided
by two opposing high-numerical-aperture objectives will be the future method
of choice for extended-resolution imaging when isotropic resolution is key.

##  15 Can light sheet microscopy and SIM, with multiple pattern orientations,
be properly combined?

As noted in Section 7, light sheet and SIM appear to be a perfect match: light
sheet microscopy is limited in resolution due to the relatively low numerical
aperture objectives used, but generates very little out-of-focus light — the
bane of SIM. However, traditional light sheet SIM (LSFM-SIM) suffers from the
relatively coarse illumination pattern that can be provided by the excitation
objective in LSFM systems and the fact that only one orientation of the
pattern is present. As such, while improvements in axial resolution can be
made, very little useful enhancement of lateral resolution is possible.

In an attempt to counter these issues, Manton & Rees proposed a LSFM-SIM
system featuring three mutually orthogonal objectives [102]. However, the use
of 0.8 NA objectives, necessary for reasons of steric hindrance, meant that
the maximum possible lateral resolution would still only be around
$230\text{\,}\mathrm{n}\mathrm{m}$ (i.e. approximately the theoretical
resolution of a 1.1 NA water-dipping objective as used in some other light
sheet microscope designs). While they also suggested the use of
interferometric excitation through two objectives to improve the resolution to
$120\text{\,}\mathrm{n}\mathrm{m}$, the experimental complexity of such a
system means that it is unlikely to ever be realised in practice.

In 2017, Chang et al. demonstrated a three-objective system featuring
interferometric excitation through two objectives and counterpropagating light
sheets [103]. While the use of counterpropagating sheets produces a fine
fringe period, issues with steric hindrance limited the orientations over
which patterns could be produced, precluding a reconstruction with isotropic
resolution. In addition, the large distance between arms of the interferometer
mean that the system is sensitive to temperature-induced phase shifts, as
discussed in Section 12, and sample mounting is made more difficult.

Furthermore, there is a theoretical issue with using interferometric
excitation through multiple objectives that has so far been unaddressed. In
the traditional SIM approach, we consider the illumination to be described by
a discrete sum of purely harmonic functions (i.e. functions which consist of a
single spatial frequency). As shown in Figure 7 on page 7a, this condition is
satisfied for lattice light sheet microscopy [64] as the pupil pattern
consists solely of vertical lines, which project orthographically onto the
Ewald sphere (i.e. they still look like straight lines when viewed from above,
with the separation between lines being constant — the distribution is a
discrete sum of harmonic functions along the pattern direction). However, now
consider a system in which a single vertical line in the pupil is used to
produce a light sheet with one objective, with another identical pupil and
objective opposing (see Figure 7 on page 7b). Now, when the Ewald sphere is
viewed from above, the distribution is not a discrete sum of harmonic
functions along the pattern direction. While the limitations on numerical
aperture provided by use of a single objective can be circumvented by using
C-shaped illumination profiles in ancillary objective pupils (Figure 7 on page
7c), it appears that the use of counterpropagating sheets is not allowed, at
least not by standard reconstruction approaches.

Figure 7:  Pupils and Ewald spheres for light sheet structured illumination
microscopy approaches. (a) Single pupil and illumination pattern for lattice
light sheet microscopy. As desired, the Fourier intensity distribution
consists of harmonic functions laterally. (b) Opposed pupils using
counterpropagating light sheets for SIM. Here, the components of the Fourier
intensity distribution are stretched out along the pattern direction,
precluding conventional reconstructions in which harmonic components are
assumed. (c) A potential method for producing lattice-like illumination
patterns at spatial frequencies beyond that achievable with one objective.
Here, C-shaped illumination profiles in the auxilliary objectives are used to
introduce the off-centre Ewald sphere components, ensuring a harmonic Fourier
intensity distribution.

##  16 How quickly can SIM image a sample?

Fundamentally, the speed of SIM is restricted by the requirement that enough
photons are detected for the signal-to-noise ratio to permit a high-quality
reconstruction. So far, the fastest SIM acquisitions demonstrated were
acquired at $798\text{\,}\mathrm{H}\mathrm{z}$ and
$714\text{\,}\mathrm{H}\mathrm{z}$, corresponding to image frame rates of
$88.7\text{\,}\mathrm{H}\mathrm{z}$ and $79\text{\,}\mathrm{H}\mathrm{z}$
[104, 105]. Given that even a single fluorophore can emit at rates in excess
of 100 million photons per second, it appears that current speed limits are
more practical than fundamental.

In an attempt to reduce the acquisition time for volumetric data by reducing
the number of stage movements that must be completed, SIM has been combined
with multifocal microscopy [106]. Here, images from multiple depths within the
sample are recorded at the same time. While the initial demonstration in 2017,
by Abrahamsson et al., used an aberration-free multifocus scheme, the limited
speed of the commercial SIM system used prevented fast acquisitions [107]. A
more recent demonstration, in 2020, of high-speed acquisition used an
aberration-inducing multifocus prism, limiting the depth over which images
could be acquired in parallel [108].

While these experiments demonstrate an impressive effort in pushing hardware
to its limits, other effort has been expended on improving the reconstruction
process. To enable live visualisation of a sample with extended resolution,
Markwirth et al. developed a software system that can provide GPU-accelerated
reconstructions in almost real-time [109]. Many groups, along with Zeiss in
their Elyra 7, have adopted ‘interleaved reconstruction’, in which rolling
sets of three orientations are used to produce reconstructions, rather than
only producing one image from each group of three-orientation data [110].
While this produces smoother videos, it is a form of interpolation and does
not truly increase speed [111].

A number of attempts have been made at reducing the number of raw images
required for a successful reconstruction, down from nine. Heintzmann was the
first to propose that a smaller set of images could be sufficient, identifying
that using as few as four images might be possible [112]. Lal et al. have
proposed such a reconstruction using just four images, but the illumination
patterns used do not average out to a uniform intensity [113]. This means
that, for time-lapse imaging, the illumination pattern would ‘burn’ into the
sample via spatially-dependent photobleaching. Orieux et al. have developed a
Bayesian approach to SIM reconstruction and used this to develop a method that
uses four images and has no specific constraints on the illumination [114].
Furthermore, Ströhl and Kaminski investigated the possibility of a further
reduction to just three images and developed a maximum likelihood approach to
deal with the resulting underdetermined system [115].

##  17 Outlook

At the time of writing, SIM is more than 22 years old. A PubMed analysis,
searching for publications with the phrase “structured illumination” in their
title or abstract suggests that more than 100 SIM-related articles have been
published every year since 2015 (Figure 8 on page 8a). An analysis of the
number of citations, as indexed by Google Scholar, of the top 10 most-cited
SIM papers shows that the total number of citations of this cohort now exceeds
1000 per year (Figure 8 on page 8b). Ignoring the unusual peak in 2017/18, it
appears that interest in SIM continues to increase.

Indeed, the estimation of the number of SIM-related papers is likely to be a
significant underestimate — SIM as a technique has now matured to the point
where it is no longer a ‘shiny new toy’ that must be mentioned in the title or
abstract. Nowadays, it suffices to just mention SIM in the main text alongside
well-respected techniques such as confocal microscopy.

Perhaps the most important development in the past few years is that of all-
optical versions of ISM, providing extended-resolution imaging at high frame
rates and without the need for computational reconstruction [66, 67, 68, 116].
These instruments present the user with a system that effectively works as a
familiar confocal or spinning disk microscope, albeit one with double the
resolving power. Furthermore, by avoiding computational reconstruction, these
approaches avoid the reconstruction artefacts that can often plague SIM [117].
However, in the absence of background fluorescence, at high spatial
frequencies traditional SIM still provides superior contrast and is more
compatible with TIRF imaging. As such, further investigation into which
techniques provide better performance in different samples and imaging
scenarios will be welcome, as would an all-optical SIM reconstruction
approach.

For longer-term time-lapse imaging, light sheet fluorescence microscopy (LSFM)
is now the technique of choice for minimal sample damage where fluorescence
contrast is required. While combinations of LSFM and SIM have already been
demonstrated (see Section 7 and Section 15), an instrument that achieves
maximal performance of both components has yet to be demonstrated.

Further enhancements to the capabilities of SIM for time-lapse imaging are
being provided by enhanced reconstruction routines that use various forms of
regularisation to successfully reconstruct images with lower signal levels
[118, 119]. In combination with brighter fluorophores, further development of
these techniques and other denoising procedures will help provide gentle,
extended-resolution imaging of proteins of interest at endogenous expression
levels.

As well as enhancing the resolution of images and movies, SIM has also been
demonstrated to enhance the resolution of biophysical microscopy methods.
While fluorescence lifetime imaging microscopy (FLIM) has been combined with
OS-SIM [120] and used corretively with SR-SIM [121], so far no true
combination of FLIM and SIM has been achieved. In 2019, Zhanghao et al.
reported their method of polarised structured illumination microscopy (pSIM),
providing extended-resolution imaging of dipoles in biopolymers such as
cytoskeletal networks and $\lambda$-DNA [122]. Most recently, in 2019 Colin-
York et al. have combined traction force microscopy (TFM) with 3D-SIM [123],
while in 2021 Barbieri et al. combined TFM with TIRF-SIM [124].

Altogether, it seems that there is still much room for further work in SIM,
both in terms of technique development and applications. It will be
interesting to survey the field again in 20 years’ time and see how many of
the outstanding challenges have been solved and which unanticipated
developments arose. If the past 20 years are any guide, the answers will be
surprising…

Figure 8:  A bibliometric analysis of SIM publications, from inception to the
end of 2020. (a) shows the number of PubMed-indexed SIM publications per year,
peaking at 159 in 2018. (b) shows the number of citations to the top 10 most-
cited SIM publications, with the number at the top of each bar stack
indicating the total number of citations of the cohort that year, peaking at
1187 in 2017. (c) shows the same citation information as (b), but now
unstacked. This view of the data demonstrates that the peak in 2017 is not due
to any one individual publication.

## Acknowledgements

The author is grateful to Jérôme Boulanger, Reto Fiolka, Rainer Heintzmann,
Chris Rowlands, Florian Ströhl and Andrew York for various SIM-related
discussions/arguments over the years, and thanks Jérôme Boulanger for the
ground-truth image used in Figures 3, 4 and 5. The author acknowledges support
from Fitzwilliam College, Cambridge, through a Research Fellowship.

## References

  * [1] Rainer Heintzmann and Christoph G. Cremer  “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating”  In _BiOS Europe’98_ , 1999, pp. 185–196  DOI: 10.1117/12.336833
  * [2] Mats G.. Gustafsson  “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy”  In _Journal of Microscopy_ 198.2, 2000, pp. 82–87  DOI: 10.1046/j.1365-2818.2000.00710.x
  * [3] Jan T. Frohn, Helmut F. Knapp and Andreas Stemmer  “True optical resolution beyond the Rayleigh limit achieved by standing wave illumination”  In _Proceedings of the National Academy of Sciences_ 97.13, 2000, pp. 7232–7236  DOI: 10.1073/pnas.130181797
  * [4] George E. Cragg and Peter T.. So  “Lateral resolution enhancement with standing evanescent waves” Publisher: Optical Society of America  In _Optics Letters_ 25.1, 2000, pp. 46–48  DOI: 10.1364/OL.25.000046
  * [5] Mats G.. Gustafsson et al.  “Three-Dimensional Resolution Doubling in Wide-Field Fluorescence Microscopy by Structured Illumination”  In _Biophysical Journal_ 94.12, 2008, pp. 4957–4970  DOI: 10.1529/biophysj.107.120345
  * [6] Lin Shao et al.  “I5S: Wide-Field Light Microscopy with 100-nm-Scale Resolution in Three Dimensions”  In _Biophysical Journal_ 94.12, 2008, pp. 4971–4983  DOI: 10.1529/biophysj.107.120352
  * [7] Liisa M. Hirvonen, Kai Wicker, Ondrej Mandula and Rainer Heintzmann  “Structured illumination microscopy of a living cell”  In _European Biophysics Journal_ 38.6, 2009, pp. 807–812  DOI: 10.1007/s00249-009-0501-6
  * [8] Peter Kner et al.  “Super-resolution video microscopy of live cells by structured illumination”  In _Nature Methods_ 6.5, 2009, pp. 339–342  DOI: 10.1038/nmeth.1324
  * [9] Lin Shao, Peter Kner, E. Rego and Mats G.. Gustafsson  “Super-resolution 3D microscopy of live whole cells using structured illumination”  In _Nature Methods_ 8.12, 2011, pp. 1044–1046  DOI: 10.1038/nmeth.1734
  * [10] E. Betzig et al.  “Breaking the Diffraction Barrier: Optical Microscopy on a Nanometric Scale”  In _Science_ 251.5000, 1991, pp. 1468–1470  DOI: 10.1126/science.251.5000.1468
  * [11] Eric Betzig et al.  “Imaging Intracellular Fluorescent Proteins at Nanometer Resolution”  In _Science_ 313.5793, 2006, pp. 1642–1645  DOI: 10.1126/science.1127344
  * [12] Samuel T. Hess, Thanu P.. Girirajan and Michael D. Mason  “Ultra-High Resolution Imaging by Fluorescence Photoactivation Localization Microscopy”  In _Biophysical Journal_ 91.11, 2006, pp. 4258–4272  DOI: 10.1529/biophysj.106.091116
  * [13] Michael J. Rust, Mark Bates and Xiaowei Zhuang  “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)”  In _Nature Methods_ 3.10, 2006, pp. 793–796  DOI: 10.1038/nmeth929
  * [14] Alexey Sharonov and Robin M. Hochstrasser  “Wide-field subdiffraction imaging by accumulated binding of diffusing probes”  In _Proceedings of the National Academy of Sciences_ 103.50, 2006, pp. 18911–18916  DOI: 10.1073/pnas.0609643104
  * [15] Stefan W. Hell and Jan Wichmann  “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy”  In _Optics Letters_ 19.11, 1994, pp. 780  DOI: 10.1364/OL.19.000780
  * [16] Thomas A. Klar and Stefan W. Hell  “Subdiffraction resolution in far-field fluorescence microscopy”  In _Optics Letters_ 24.14, 1999, pp. 954  DOI: 10.1364/OL.24.000954
  * [17] Thomas A. Klar et al.  “Fluorescence microscopy with diffraction resolution barrier broken by stimulated emission”  In _Proceedings of the National Academy of Sciences_ 97.15, 2000, pp. 8206–8210  DOI: 10.1073/pnas.97.15.8206
  * [18] Mark Neil, R. Juškaitis and T. Wilson  “Method of obtaining optical sectioning by using structured light in a conventional microscope”  In _Optics Letters_ 22.24, 1997, pp. 1905–1907  DOI: 10.1364/OL.22.001905
  * [19] Kirti Prakash et al.  “Super-resolution structured illumination microscopy: past, present and future”  In _arXiv:2102.13649 [physics, q-bio]_ , 2021  arXiv: http://arxiv.org/abs/2102.13649
  * [20] Rainer Heintzmann  “Answers to fundamental questions in superresolution microscopy” Publisher: Royal Society  In _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 379.2199, 2021, pp. 20210105  DOI: 10.1098/rsta.2021.0105
  * [21] Lord Rayleigh  “Investigations in optics, with special reference to the spectroscope”  In _The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science_ 8.49, 1879, pp. 261–274  DOI: 10.1080/14786447908639684
  * [22] C.. Sparrow  “On Spectroscopic Resolving Power”  In _The Astrophysical Journal_ 44, 1916, pp. 76  DOI: 10.1086/142271
  * [23] Ernst Abbe  “Beiträge zur Theorie des Mikroskops und der mikroskopischen Wahrnehmung”  In _Archiv für Mikroskopische Anatomie_ 9.1, 1873, pp. 413–468
  * [24] Mats G.. Gustafsson, David A. Agard and John W. Sedat  “Sevenfold improvement of axial resolution in 3D wide-field microscopy using two objective lenses”  In _SPIE Proceedings_ 2412, 1995, pp. 147–156  DOI: 10.1117/12.205334
  * [25] C.. McCutchen  “Generalized Aperture and the Three-Dimensional Diffraction Image” Publisher: Optical Society of America  In _JOSA_ 54.2, 1964, pp. 240–244  DOI: 10.1364/JOSA.54.000240
  * [26] P.P. Ewald  “Zur Theorie der Interferenzen der Röntgentstrahlen in Kristallen”  In _Physikalische Zeitschrift_ 11, 1913, pp. 465–472
  * [27] Marcel Lahrberg, Mandeep Singh, Kedar Khare and Balpreet Singh Ahluwalia  “Accurate estimation of the illumination pattern’s orientation and wavelength in sinusoidal structured illumination microscopy” Publisher: Optical Society of America  In _Applied Optics_ 57.5, 2018, pp. 1019–1025  DOI: 10.1364/AO.57.001019
  * [28] Kai Wicker  “Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space”  In _Optics Express_ 21.21, 2013, pp. 24692–24701  DOI: 10.1364/OE.21.024692
  * [29] Carlas S. Smith et al.  “Structured illumination microscopy with noise-controlled image reconstructions” Publisher: Cold Spring Harbor Laboratory Section: New Results  In _bioRxiv_ , 2021, pp. 2021.03.11.434940  DOI: 10.1101/2021.03.11.434940
  * [30] A. Lal, C. Shan and P. Xi  “Structured illumination microscopy image reconstruction algorithm”  In _IEEE Journal of Selected Topics in Quantum Electronics_ PP.99, 2016, pp. 1–1  DOI: 10.1109/JSTQE.2016.2521542
  * [31] Graeme Ball et al.  “SIMcheck: a Toolbox for Successful Super-resolution Structured Illumination Microscopy” Number: 1 Publisher: Nature Publishing Group  In _Scientific Reports_ 5.1, 2015, pp. 15915  DOI: 10.1038/srep15915
  * [32] Marcel Müller et al.  “Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ”  In _Nature Communications_ 7, 2016, pp. ncomms10980  DOI: 10.1038/ncomms10980
  * [33] Mats GL Gustafsson  “Extended resolution fluorescence microscopy”  In _Current Opinion in Structural Biology_ 9.5, 1999, pp. 627–628  DOI: 10.1016/S0959-440X(99)00016-0
  * [34] Euiheon Chung, Daekeun Kim and Peter T. So  “Extended resolution wide-field optical imaging: objective-launched standing-wave total internal reflection fluorescence microscopy” Publisher: Optical Society of America  In _Optics Letters_ 31.7, 2006, pp. 945–947  DOI: 10.1364/OL.31.000945
  * [35] Andreas Stemmer, Markus Beck and Reto Fiolka  “Widefield fluorescence microscopy with extended resolution”  In _Histochemistry and Cell Biology_ 130.5, 2008, pp. 807  DOI: 10.1007/s00418-008-0506-8
  * [36] Dong Li et al.  “Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics”  In _Science_ 349.6251, 2015, pp. aab3500  DOI: 10.1126/science.aab3500
  * [37] Stefan W. Hell  “Toward fluorescence nanoscopy” Bandiera_abtest: a Cg_type: Nature Research Journals Number: 11 Primary_atype: Reviews Publisher: Nature Publishing Group  In _Nature Biotechnology_ 21.11, 2003, pp. 1347–1355  DOI: 10.1038/nbt895
  * [38] Reto Fiolka, Markus Beck and Andreas Stemmer  “Structured illumination in total internal reflection fluorescence microscopy using a spatial light modulator” Publisher: Optical Society of America  In _Optics Letters_ 33.14, 2008, pp. 1629–1631  DOI: 10.1364/OL.33.001629
  * [39] P.. Liu et al.  “Cell refractive index for cell biology and disease diagnosis: past, present and future” Publisher: The Royal Society of Chemistry  In _Lab on a Chip_ 16.4, 2016, pp. 634–644  DOI: 10.1039/C5LC01445J
  * [40] Anindita Dasgupta et al.  “Direct supercritical angle localization microscopy for nanometer 3D superresolution” Number: 1 Publisher: Nature Publishing Group  In _Nature Communications_ 12.1, 2021, pp. 1180  DOI: 10.1038/s41467-021-21333-x
  * [41] Colin J.. Sheppard  “Fundamentals of superresolution”  In _Micron_ 38.2, Special issue on Super-resolution and other Novel Microscopies, 2007, pp. 165–169  DOI: 10.1016/j.micron.2006.07.012
  * [42] Rainer Heintzmann, Thomas M. Jovin and Christoph Cremer  “Saturated patterned excitation microscopy—a concept for optical resolution improvement”  In _JOSA A_ 19.8, 2002, pp. 1599–1609  URL: https://www.osapublishing.org/abstract.cfm?uri=josaa-19-8-1599
  * [43] Mats G.. Gustafsson  “Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution”  In _Proceedings of the National Academy of Sciences of the United States of America_ 102.37, 2005, pp. 13081–13086  DOI: 10.1073/pnas.0406877102
  * [44] Ulrich Kubitscheck, Oliver Kückmann, Thorsten Kues and Reiner Peters  “Imaging and Tracking of Single GFP Molecules in Solution”  In _Biophysical Journal_ 78.4, 2000, pp. 2170–2179  DOI: 10.1016/S0006-3495(00)76764-6
  * [45] E.. Rego et al.  “Nonlinear structured-illumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution”  In _Proceedings of the National Academy of Sciences_ 109.3, 2012, pp. E135–E143  DOI: 10.1073/pnas.1107547108
  * [46] Liisa Hirvonen, Ondrej Mandula, Kai Wicker and Rainer Heintzmann  “Structured illumination microscopy using photoswitchable fluorescent proteins”  In _Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XV_ 6861 SPIE, 2008, pp. 133–140  DOI: 10.1117/12.763021
  * [47] Miriam A. Schwentker et al.  “Wide-field subdiffraction RESOLFT microscopy using fluorescent protein photoswitching” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/jemt.20443  In _Microscopy Research and Technique_ 70.3, 2007, pp. 269–280  DOI: 10.1002/jemt.20443
  * [48] Rainer Heintzmann and Mats G.. Gustafsson  “Subdiffraction resolution in continuous samples”  In _Nature Photonics_ 3.7, 2009, pp. 362–364  DOI: 10.1038/nphoton.2009.102
  * [49] Peter T.. So, Hyuk-Sang Kwon and Chen Y. Dong  “Resolution enhancement in standing-wave total internal reflection microscopy: a point-spread-function engineering approach” Publisher: Optical Society of America  In _JOSA A_ 18.11, 2001, pp. 2833–2845  DOI: 10.1364/JOSAA.18.002833
  * [50] W. Lukosz  “Optical Systems with Resolving Powers Exceeding the Classical Limit*” Publisher: Optical Society of America  In _JOSA_ 56.11, 1966, pp. 1463–1471  DOI: 10.1364/JOSA.56.001463
  * [51] W. Lukosz  “Optical Systems with Resolving Powers Exceeding the Classical Limit. II” Publisher: Optical Society of America  In _JOSA_ 57.7, 1967, pp. 932–941  DOI: 10.1364/JOSA.57.000932
  * [52] A. Bachl and W. Lukosz  “Experiments on Superresolution Imaging of a Reduced Object Field*” Publisher: Optical Society of America  In _JOSA_ 57.2, 1967, pp. 163–169  DOI: 10.1364/JOSA.57.000163
  * [53] Muthuvel Arigovindan, John W. Sedat and David A. Agard  “Effect of depth dependent spherical aberrations in 3D structured illumination microscopy” Publisher: Optical Society of America  In _Optics Express_ 20.6, 2012, pp. 6527–6541  DOI: 10.1364/OE.20.006527
  * [54] Martin J. Booth  “Adaptive optical microscopy: the ongoing quest for a perfect image” Number: 4 Publisher: Nature Publishing Group  In _Light: Science & Applications_ 3.4, 2014, pp. e165–e165  DOI: 10.1038/lsa.2014.46
  * [55] Raphaël Turcotte et al.  “Dynamic super-resolution structured illumination imaging in the living brain” Publisher: National Academy of Sciences Section: Biological Sciences  In _Proceedings of the National Academy of Sciences_ 116.19, 2019, pp. 9586–9591  DOI: 10.1073/pnas.1819965116
  * [56] Ruizhe Lin and Peter A. Kner  “Structured illumination microscopy with direct wavefront sensing”  In _Adaptive Optics and Wavefront Control for Biological Systems VII_ 11652 SPIE, 2021, pp. 10–13  DOI: 10.1117/12.2582806
  * [57] Mantas Žurauskas et al.  “IsoSense: frequency enhanced sensorless adaptive optics through structured illumination” Publisher: Optical Society of America  In _Optica_ 6.3, 2019, pp. 370–379  DOI: 10.1364/OPTICA.6.000370
  * [58] Volker Andresen et al.  “High-Resolution Intravital Microscopy” Publisher: Public Library of Science  In _PLOS ONE_ 7.12, 2012, pp. e50915  DOI: 10.1371/journal.pone.0050915
  * [59] Peter W. Winter et al.  “Two-photon instant structured illumination microscopy improves the depth penetration of super-resolution imaging in thick scattering samples” Publisher: Optical Society of America  In _Optica_ 1.3, 2014, pp. 181–191  DOI: 10.1364/OPTICA.1.000181
  * [60] Wei Zheng et al.  “Adaptive optics improves multiphoton super-resolution imaging” Number: 9 Publisher: Nature Publishing Group  In _Nature Methods_ 14.9, 2017, pp. 869–872  DOI: 10.1038/nmeth.4337
  * [61] Philipp J. Keller et al.  “Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy”  In _Nature Methods_ 7.8, 2010, pp. 637–642  DOI: 10.1038/nmeth.1476
  * [62] Thomas A. Planchon et al.  “Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination”  In _Nature Methods_ 8.5, 2011, pp. 417–423  DOI: 10.1038/nmeth.1586
  * [63] Liang Gao et al.  “Noninvasive Imaging beyond the Diffraction Limit of 3D Dynamics in Thickly Fluorescent Specimens”  In _Cell_ 151.6, 2012, pp. 1370–1385  DOI: 10.1016/j.cell.2012.10.008
  * [64] Bi-Chang Chen et al.  “Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution”  In _Science_ 346.6208, 2014, pp. 1257998  DOI: 10.1126/science.1257998
  * [65] Andrew G. York et al.  “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy”  In _Nature Methods_ 9.7, 2012, pp. 749–754  DOI: 10.1038/nmeth.2025
  * [66] Giulia M.R. De Luca et al.  “Re-scan confocal microscopy: scanning twice for better resolution”  In _Biomedical Optics Express_ 4.11, 2013, pp. 2644  DOI: 10.1364/BOE.4.002644
  * [67] Stephan Roth, Colin JR Sheppard, Kai Wicker and Rainer Heintzmann  “Optical photon reassignment microscopy (OPRA)”  In _Optical Nanoscopy_ 2.1, 2013, pp. 5  DOI: 10.1186/2192-2853-2-5
  * [68] Andrew G. York et al.  “Instant super-resolution imaging in live cells and embryos via analog image processing”  In _Nature Methods_ 10.11, 2013, pp. 1122–1126  DOI: 10.1038/nmeth.2687
  * [69] Joseph Huff  “The Airyscan detector from ZEISS: confocal imaging with improved signal-to-noise ratio and super-resolution” Number: 12 Publisher: Nature Publishing Group  In _Nature Methods_ 12.12, 2015, pp. i–ii  DOI: 10.1038/nmeth.f.388
  * [70] Colin J.. Sheppard et al.  “Interpretation of the optical transfer function: Significance for image scanning microscopy”  In _Optics Express_ 24.24, 2016, pp. 27280–27287  DOI: 10.1364/OE.24.027280
  * [71] Stephan Roth, Colin J.. Sheppard and Rainer Heintzmann  “Superconcentration of light: circumventing the classical limit to achievable irradiance”  In _Optics Letters_ 41.9, 2016, pp. 2109  DOI: 10.1364/OL.41.002109
  * [72] C… Sheppard  “Super-resolution in Confocal Imaging”  In _Optik_ 80, 1988, pp. 83–84
  * [73] Claus B. Müller and Jörg Enderlein  “Image Scanning Microscopy”  In _Physical Review Letters_ 104.19, 2010, pp. 198101  DOI: 10.1103/PhysRevLett.104.198101
  * [74] E. Mudry et al.  “Structured illumination microscopy using unknown speckle patterns”  In _Nature Photonics_ 6.5, 2012, pp. 312–315  DOI: 10.1038/nphoton.2012.83
  * [75] R. Ayuk et al.  “Structured illumination fluorescence microscopy with distorted excitations using a filtered blind-SIM algorithm” Publisher: Optical Society of America  In _Optics Letters_ 38.22, 2013, pp. 4723–4726  DOI: 10.1364/OL.38.004723
  * [76] Aurélie Jost et al.  “Optical Sectioning and High Resolution in Single-Slice Structured Illumination Microscopy by Thick Slice Blind-SIM Reconstruction” Publisher: Public Library of Science  In _PLOS ONE_ 10.7, 2015, pp. e0132174  DOI: 10.1371/journal.pone.0132174
  * [77] Li-Hao Yeh, Lei Tian and Laura Waller  “Structured illumination microscopy with unknown patterns and a statistical prior” Publisher: Optical Society of America  In _Biomedical Optics Express_ 8.2, 2017, pp. 695–711  DOI: 10.1364/BOE.8.000695
  * [78] E.. Ingerman, R.. London, R. Heintzmann and M… Gustafsson  “Signal, noise and resolution in linear and nonlinear structured-illumination microscopy” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/jmi.12753  In _Journal of Microscopy_ 273.1, 2019, pp. 3–25  DOI: https://doi.org/10.1111/jmi.12753
  * [79] Kai Wicker and Rainer Heintzmann  “Resolving a misconception about structured illumination”  In _Nature Photonics_ 8.5, 2014, pp. 342–344  DOI: 10.1038/nphoton.2014.88
  * [80] Ignacio Arganda-Carreras et al.  “Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification”  In _Bioinformatics_ 33.15, 2017, pp. 2424–2426  DOI: 10.1093/bioinformatics/btx180
  * [81] Uwe Schmidt, Martin Weigert, Coleman Broaddus and Gene Myers  “Cell Detection with Star-convex Polygons”  In _arXiv_ 11071, 2018, pp. 265–273  DOI: 10.1007/978-3-030-00934-2˙30
  * [82] J. Boulanger et al.  “Patch-Based Nonlocal Functional for Denoising Fluorescence Microscopy Image Sequences” Conference Name: IEEE Transactions on Medical Imaging  In _IEEE Transactions on Medical Imaging_ 29.2, 2010, pp. 442–454  DOI: 10.1109/TMI.2009.2033991
  * [83] Min Guo et al.  “Rapid image deconvolution and multiview fusion for optical microscopy” Number: 11 Publisher: Nature Publishing Group  In _Nature Biotechnology_ 38.11, 2020, pp. 1337–1346  DOI: 10.1038/s41587-020-0560-x
  * [84] Chang Ling et al.  “Fast structured illumination microscopy via deep learning” Publisher: Optical Society of America  In _Photonics Research_ 8.8, 2020, pp. 1350–1359  DOI: 10.1364/PRJ.396122
  * [85] Luhong Jin et al.  “Deep learning enables structured illumination microscopy with low light levels and enhanced speed” Number: 1 Publisher: Nature Publishing Group  In _Nature Communications_ 11.1, 2020, pp. 1934  DOI: 10.1038/s41467-020-15784-x
  * [86] Chang Qiao et al.  “Evaluation and development of deep neural networks for image super-resolution in optical microscopy” Number: 2 Publisher: Nature Publishing Group  In _Nature Methods_ 18.2, 2021, pp. 194–202  DOI: 10.1038/s41592-020-01048-5
  * [87] Zafran Hussain Shah et al.  “Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images” Publisher: Optical Society of America  In _Photonics Research_ , 2021  DOI: 10.1364/PRJ.416437
  * [88] Gerrit Best et al.  “Structured illumination microscopy of autofluorescent aggregations in human tissue”  In _Micron_ 42.4, Special issue on Super and High resolution imaging, 2011, pp. 330–335  DOI: 10.1016/j.micron.2010.06.016
  * [89] Wenjie Liu et al.  “Three-dimensional super-resolution imaging of live whole cells using galvanometer-based structured illumination microscopy” Publisher: Optical Society of America  In _Optics Express_ 27.5, 2019, pp. 7237–7248  DOI: 10.1364/OE.27.007237
  * [90] William M. Dougherty and Steven Charles Quarre  “Method and system for fast three-dimensional structured-illumination-microscopy imaging”, 2013  URL: https://patents.google.com/patent/US8570650B2/en?q=METHOD+AND+SYSTEM+FOR+FAST+THREE-DIMENSIONAL+STRUCTURED-ILLUMINATION-MICROSCOPY+IMAGING&oq=METHOD+AND+SYSTEM+FOR+FAST+THREE-DIMENSIONAL+STRUCTURED-ILLUMINATION-MICROSCOPY+IMAGING
  * [91] Taylor A. Hinsdale, Sjoerd Stallinga and Bernd Rieger  “High-speed multicolor structured illumination microscopy using a hexagonal single mode fiber array” Publisher: Optical Society of America  In _Biomedical Optics Express_ 12.2, 2021, pp. 1181–1194  DOI: 10.1364/BOE.416546
  * [92] Jakub Pospíšil et al.  “Highly compact and cost-effective 2-beam super-resolution structured illumination microscope based on all-fiber optic components” Publisher: Optical Society of America  In _Optics Express_ 29.8, 2021, pp. 11833–11844  DOI: 10.1364/OE.420592
  * [93] Alice Sandmeyer et al.  “DMD-based super-resolution structured illumination microscopy visualizes live cell dynamics at high speed and low cost” Publisher: Cold Spring Harbor Laboratory Section: New Results  In _bioRxiv_ , 2019, pp. 797670  DOI: 10.1101/797670
  * [94] Peter T. Brown, Rory Kruithoff, Gregory J. Seedorf and Douglas P. Shepherd  “Multicolor structured illumination microscopy and quantitative control of polychromatic coherent light with a digital micromirror device” Publisher: Cold Spring Harbor Laboratory Section: New Results  In _bioRxiv_ , 2020, pp. 2020.07.27.223941  DOI: 10.1101/2020.07.27.223941
  * [95] Francisco Balzarotti et al.  “Nanometer resolution imaging and tracking of fluorescent molecules with minimal photon fluxes” Publisher: American Association for the Advancement of Science Section: Research Article  In _Science_ 355.6325, 2017, pp. 606–612  DOI: 10.1126/science.aak9913
  * [96] Loïc Reymond et al.  “SIMPLE: Structured illumination based point localization estimator with enhanced precision” Publisher: Optical Society of America  In _Optics Express_ 27.17, 2019, pp. 24578–24590  DOI: 10.1364/OE.27.024578
  * [97] Lusheng Gu et al.  “Molecular resolution imaging by repetitive optical selective exposure” Number: 11 Publisher: Nature Publishing Group  In _Nature Methods_ 16.11, 2019, pp. 1114–1118  DOI: 10.1038/s41592-019-0544-2
  * [98] Jelmer Cnossen et al.  “Localization microscopy at doubled precision with patterned illumination” Number: 1 Publisher: Nature Publishing Group  In _Nature Methods_ 17.1, 2020, pp. 59–63  DOI: 10.1038/s41592-019-0657-7
  * [99] Pierre Jouchet et al.  “Nanometric axial localization of single fluorescent molecules with modulated excitation” Number: 4 Publisher: Nature Publishing Group  In _Nature Photonics_ 15.4, 2021, pp. 297–304  DOI: 10.1038/s41566-020-00749-9
  * [100] Martin Schmidt et al.  “Camera-based localization microscopy optimized with calibrated structured illumination” Number: 1 Publisher: Nature Publishing Group  In _Communications Physics_ 4.1, 2021, pp. 1–9  DOI: 10.1038/s42005-021-00546-y
  * [101] James D. Manton et al.  “Concepts for structured illumination microscopy with extended axial resolution through mirrored illumination” Publisher: Optical Society of America  In _Biomedical Optics Express_ 11.4, 2020, pp. 2098–2108  DOI: 10.1364/BOE.382398
  * [102] James D. Manton and Eric J. Rees  “triSPIM: light sheet microscopy with isotropic super-resolution”  In _Optics Letters_ 41.18, 2016, pp. 4170  DOI: 10.1364/OL.41.004170
  * [103] Bo-Jui Chang, Victor Didier Perez Meza and Ernst H.. Stelzer  “csiLSFM combines light-sheet fluorescence microscopy and coherent structured illumination for a lateral resolution below 100 nm”  In _Proceedings of the National Academy of Sciences_ 114.19, 2017, pp. 4869–4874  DOI: 10.1073/pnas.1609278114
  * [104] Yuting Guo et al.  “Visualizing Intracellular Organelle and Cytoskeletal Interactions at Nanoscale Resolution on Millisecond Timescales”  In _Cell_ 175.5, 2018, pp. 1430–1442.e17  DOI: 10.1016/j.cell.2018.09.057
  * [105] Liyan Song et al.  “Fast structured illumination microscopy using rolling shutter cameras”  In _Measurement Science and Technology_ 27.5, 2016, pp. 055401  DOI: 10.1088/0957-0233/27/5/055401
  * [106] P. Prabhat, S. Ram, E.. Ward and R.. Ober  “Simultaneous imaging of different focal planes in fluorescence microscopy for the study of cellular dynamics in three dimensions” Conference Name: IEEE Transactions on NanoBioscience  In _IEEE Transactions on NanoBioscience_ 3.4, 2004, pp. 237–242  DOI: 10.1109/TNB.2004.837899
  * [107] Sara Abrahamsson et al.  “Multifocus structured illumination microscopy for fast volumetric super-resolution imaging”  In _Biomedical Optics Express_ 8.9, 2017, pp. 4135–4140  DOI: 10.1364/BOE.8.004135
  * [108] Adrien Descloux et al.  “High-speed multiplane structured illumination microscopy of living cells using an image-splitting prism” Publisher: De Gruyter Section: Nanophotonics  In _Nanophotonics_ 9.1, 2020, pp. 143–148  DOI: 10.1515/nanoph-2019-0346
  * [109] Andreas Markwirth et al.  “Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction” Number: 1 Publisher: Nature Publishing Group  In _Nature Communications_ 10.1, 2019, pp. 4315  DOI: 10.1038/s41467-019-12165-x
  * [110] Ying Ma et al.  “Structured illumination microscopy with interleaved reconstruction (SIMILR)” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/jbio.201700090  In _Journal of Biophotonics_ 11.2, 2018, pp. e201700090  DOI: https://doi.org/10.1002/jbio.201700090
  * [111] Abderrahim Boualam and Christopher J. Rowlands  “Method for assessing the spatiotemporal resolution of structured illumination microscopy (SIM)” Publisher: Optical Society of America  In _Biomedical Optics Express_ 12.2, 2021, pp. 790–801  DOI: 10.1364/BOE.403592
  * [112] Rainer Heintzmann  “Saturated patterned excitation microscopy with two-dimensional excitation patterns”  In _Micron_ 34.6, Super-Resolution, 2003, pp. 283–291  DOI: 10.1016/S0968-4328(03)00053-2
  * [113] A. Lal et al.  “A Frequency Domain SIM Reconstruction Algorithm Using Reduced Number of Images” Conference Name: IEEE Transactions on Image Processing  In _IEEE Transactions on Image Processing_ 27.9, 2018, pp. 4555–4570  DOI: 10.1109/TIP.2018.2842149
  * [114] F. Orieux et al.  “Bayesian Estimation for Optimized Structured Illumination Microscopy” Conference Name: IEEE Transactions on Image Processing  In _IEEE Transactions on Image Processing_ 21.2, 2012, pp. 601–614  DOI: 10.1109/TIP.2011.2162741
  * [115] Florian Ströhl and Clemens F. Kaminski  “Speed limits of structured illumination microscopy” tex.date-modified= 2017-08-09 13:45:05 +0000  In _Optics Letters_ 42.13, 2017, pp. 2511–2514  DOI: 10.1364/OL.42.002511
  * [116] Takuya Azuma and Takayuki Kei  “Super-resolution spinning-disk confocal microscopy using optical photon reassignment” Publisher: Optical Society of America  In _Optics Express_ 23.11, 2015, pp. 15003–15011  DOI: 10.1364/OE.23.015003
  * [117] Justin Demmerle et al.  “Strategic and practical guidelines for successful structured illumination microscopy” Number: 5 Publisher: Nature Publishing Group  In _Nature Protocols_ 12.5, 2017, pp. 988–1010  DOI: 10.1038/nprot.2017.019
  * [118] Jérôme Boulanger et al.  “Nonsmooth convex optimization for structured illumination microscopy image reconstruction” Publisher: IOP Publishing  In _Inverse Problems_ 34.9, 2018, pp. 095004  DOI: 10.1088/1361-6420/aaccca
  * [119] Xiaoshuai Huang et al.  “Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy” Number: 5 Publisher: Nature Publishing Group  In _Nature Biotechnology_ 36.5, 2018, pp. 451–459  DOI: 10.1038/nbt.4115
  * [120] Taylor Hinsdale et al.  “Optically sectioned wide-field fluorescence lifetime imaging microscopy enabled by structured illumination” Publisher: Optical Society of America  In _Biomedical Optics Express_ 8.3, 2017, pp. 1455–1465  DOI: 10.1364/BOE.8.001455
  * [121] Frederik Görlitz et al.  “Mapping Molecular Function to Biological Nanostructure: Combining Structured Illumination Microscopy with Fluorescence Lifetime Imaging (SIM + FLIM)” Number: 3 Publisher: Multidisciplinary Digital Publishing Institute  In _Photonics_ 4.3, 2017, pp. 40  DOI: 10.3390/photonics4030040
  * [122] Karl Zhanghao et al.  “Super-resolution imaging of fluorescent dipoles via polarized structured illumination microscopy” Number: 1 Publisher: Nature Publishing Group  In _Nature Communications_ 10.1, 2019, pp. 4694  DOI: 10.1038/s41467-019-12681-w
  * [123] Huw Colin-York et al.  “Spatiotemporally Super-Resolved Volumetric Traction Force Microscopy” Publisher: American Chemical Society  In _Nano Letters_ 19.7, 2019, pp. 4427–4434  DOI: 10.1021/acs.nanolett.9b01196
  * [124] Liliana Barbieri et al.  “Two-dimensional TIRF-SIM–traction force microscopy (2D TIRF-SIM-TFM)” Number: 1 Publisher: Nature Publishing Group  In _Nature Communications_ 12.1, 2021, pp. 2169  DOI: 10.1038/s41467-021-22377-9

The effect of increased illumination power on harmonic strength in saturated
structured illumination microscopy.

Shifting high-resolution information back to correct locations in Fourier
space.

Demonstration of homodyne reconstruction method as applied to scanning ‘SIM’
as an excitation pattern.