Specification

The invention relates to an imaging system for microscopic far field superresolution imaging, as well as to a method for superresolution.

Superresolution microscopy is known in the art. Superresolution microscopy refers to a method of microscopy that allows for generating an image of a sample comprising luminescent emitters that has a spatial resolution higher than the optical resolution of the optical system it has been acquired with. Particularly, the term superresolution microscopy refers to far field superresolution microscopy in distinction to near-field microscopy.

The key for achieving superresolution is to encode information on the emitter distribution in the sample by means of a switchable optical property that the emitters exhibit. Typically, the encoding is facilitated by an emitter property that allows the emitters to switch between a bright (on) and a dark (off) state. Some methods allow a rather uncontrolled, i.e. stochastic switching between these two states, while some rely on a controlled a localized switching.

Particularly in case of the methods that allow for a stochastic switching of the emitters, it is necessary to have some degree of control over the switching behavior.

For example, in methods that rely on imaging a series of images, while in each image only a very limited number of emitters are providing an optical signal to the detector, such that on average less than one emitter per optical volume is in its bright state. Thus, the on- and off- states have to be controlled in terms of their duration and switching rate.

The optical volume is defined as the volume that corresponds to the optical resolution of the imaging system. Often, this volume is a diffraction-limited volume. In some cases, however, diffraction-limited imaging is not achieved.

From the series of images each emitter in each image of the series that provided a signal will be represented by a spatially isolated point spread function (PSF). In the series of images, the position of emitters of the subset of emitters that is in its on state is determined by determining a center of the PSF in terms of its highest intensity. The center can be determined with higher accuracy than the width of the PSF. As the emitters stochastically switch between on and off states in each image a different subset of emitters will give rise to an isolated PSF in the image.

The coordinates of all successful localizations of PSFs might then be combined to arrive at the high-resolution image, which resolution is particularly limited by the number of photons and which resolution is higher than the optical resolution of the imaging system.

A major drawback of this so-called single-molecule localization microscopy (SMLM) approach, is that acquisition times are long, in order to arrive at a complete or high enough sampling of the emitter localizations. Usually, around 10.000 frames have to be acquired in order to reconstruct such a high-resolution image.

Moreover, the control of the switching rate and duration of the emitters is achieved by either selecting very specific fluorophores or by adding chemical compounds to the sample that are configured to adjust the switching behavior.

Despite progress in the field of SMLM and other superresolution methods, higher resolution in space therefore inevitably comes with increased acquisition times, as acquisition is currently governed by the desired temporal or spatial resolution rather than the information content of the signal. Long image acquisition times in turn translate directly to photo-damage done to any biological sample.

An object of the present invention is to provide a method and an imaging system that does not, or only minimally, compromises time for space, that has a low propensity for photo-bleaching, and that can scale between the nanometer and millimeter range covering tens of thousands of cells.

The object is achieved by the system having the features of claim 1.

Advantageous embodiments are described in the subclaims.

According to a first aspect of the invention, an imaging system for microscopic far-field superresolution imaging is disclosed having an optical resolution, wherein the imaging system comprises at least the following components: - An excitation light source configured to provide excitation light, particularly coherent excitation light within a predefined excitation wavelength range,

- A first objective arranged and configured to illuminate a sample comprising emitters that are excitable with the excitation light and that emit emission light upon excitation with excitation light, wherein the sample is arranged or may be arranged in a sample space of the system,

- A sample carrier arranged and configured to support the sample in the sample space, wherein the sample carrier comprises a first transparent window for supporting the sample, wherein the sample carrier is arranged such that excitation light from the first objective first propagates through the first transparent window of the sample carrier before reaching the sample space,

- A far-field detection optics configured and arranged such in the system that emission light is collected from the sample space and propagated towards an image space along a detection path,

Particularly a filter element arranged in the detection path and configured to filter the collected emission light from the far-field detection optics,

- A detector arranged in the image space and configured to record the emission light, particularly the filtered emission light from the sample,

- A colloidal suspension, particularly a transparent colloidal suspension, comprising particles as the colloid, particularly wherein the particles are dielectric particles, wherein the particles exhibit light scattering, absorbing, and plasmonic properties in the excitation wavelength range, wherein the suspension is arranged in the sample space or between the first window of the sample carrier and the first objective, such that excitation, particularly an excitation strength or an excitation cross-section, of the emitters with excitation light varies randomly both spatially and temporally at the positions of the emitters, due to the light scattering, absorbing and/or plasmonic properties of the particles undergoing Brownian motion in the suspension, such that light emitted from the emitters also varies randomly both spatially and temporally due to the random excitation.

The imaging system allows for producing and controlling temporally and spatially variant emitter signals, without the need of addition of chemical compounds to the sample in order to control a fluctuating emission behaviour of the emitters. According to the invention, the fluctuations (magnitude and frequency) in the emission light are for example adjustable by means of a concentration adjustment of the particles in the suspension. Moreover, the system offers a large degree of freedom in terms of where the suspension of particles can be placed with respect to the sample.

It is noted that the sample may not to be understood to form a mandatory part of the system, but is solely used in the context of the current description and claims to illustrate the arrangement of components or the architecture of the system and/or the working principle of the method.

The temporally and spatially variant excitation and emission is particularly not induced by emitter quenching processes.

The particles are particularly nanoparticles.

In one embodiment the particles are colloidal particles.

According to another embodiment of the invention, the particles comprise or are silver (Ag) particles, particularly colloidal silver particles.

The emitters might comprise fluorophores, such as organic fluorophores or genetically encoded fluorophores.

The emitters are particularly luminescent emitters that are excitable by the excitation light and emit on a different wavelength than the excitation light, such that the emission light can be filtered by a wavelength dependent filter.

According to another embodiment of the invention, the detector is configured to sample the detected light at a timescale that is in the range or faster than the diffusion time of the particles required to diffuse through a volume of the size of the optical resolution of the system, such that subsequent recorded signals by the detector comprise an information at least on the temporal and/or the spatial correlation of the random excitation caused by the Brownian motion of the particles.

For example, in case the detector is an array detector with light sensitive pixels forming the array of the array detector, the array detector is configured to acquire images such that variations, i.e. fluctuations of the emitted light in the pixels of the detector are correlated in space and time, as the frame rate may be adjusted such that the diffusion rate of the particles through a volume corresponding to a pixel in the sample space is smaller than the frame rate. This way the correlation in the random emission signal may be recorded allowing to analyse the recorded images with a compressed sensing method as will be detailed later.

According to another embodiment of the invention, a concentration of the emitters is chosen such that an emitter density conforms to a sparsity criterion for compressed sensing methods.

For biological samples this sparsity criterion is usually met without further adjustment.

In one embodiment the imaging system comprises a microscope that is configured to illuminate the sample by means of a wide-field illumination with the excitation light. The term wide-field illumination thereby comprises illumination of the sample by conventional wide-field illumination, small angle illumination, and/or total internal reflection (TIR) illumination. The system is further configured to image the sample in the sample space in a parallel fashion on the array detector, that is sensitive enough to detect photons from single emitters. Such array detectors are for example CMOS cameras, or electron multiplying CCD cameras (EMCCD cameras).

The imaging process involves the signal from the emitters to be optically convolved with the PSF of the system that eventually defines the optical resolution of the system.

The excitation light source might be a coherent light source, such as a laser or lasingbased light source.

In contrast to dynamic speckle patterns created for example by a rotating diffuser, the suspension of dielectric particles with plasmonic properties introduces stronger fluctuations in the excitation light than other particles lacking the plasmonic property without compromising overall intensities. Particularly, in combination with the Brownian motion of the particles in the suspension, this results in spatially and temporally correlated fluctuations of the excitation light at length scales shorter than the optical resolution, leading to a generation of information on the position of the individual emitters that is better than the optical resolution, particularly wherein said information and thus the emitter positions may be extracted from the varying emission light by means of a compressed sensing method.

The system particularly comprises a beamsplitter that is arranged such in the light path of the system that excitation light can be filtered from the emitted light. In addition, the system can further comprise an emission filter that blocks any excitation light to a higher degree than the beamsplitter. The excitation light and the emission light of the emitters at least partially comprise a different wavelength.

The array detector might comprise an optical assembly, e.g. an imaging lens, that allows for magnified imaging the sample on the array detector.

The system might comprise further optical elements configured to shape, filter or alter the wave front of the excitation and/or emission light.

The system is particularly arranged and adapted to illuminate the sample simultaneously and not by means of a point- or line-scanning method.

Alternatively, or additionally, the system is arranged and adapted for confocal excitation and/or confocal detection, such as a spinning disk wide field detection microscope or a confocal point detection microscope, particularly wherein the detector is a point-detector.

According to another embodiment of the invention, the microscope is a confocal laser scanning microscope.

The sample carrier may comprise a sample holder that may have a sample chamber for immersing the sample with the suspension of particles.

Particularly, the first transparent window is a coverslide or a cover slip having a thickness of only around hundred to two hundred microns.

The detection optics might be comprised in the first objective, such that illumination and collection of light is achieved by the same objective. Alternatively, the detection optics it might be provided by means of a second objective that is arranged and configured to collect the emitted light at a different angle than the first objective. Such imaging system therefore may include a light sheet imaging modality.

Using a second objective for detection particularly allows that the emitted light from the emitters does not propagate through the suspension on its way to the detector.

It is noted that excitation and detection of light is facilitated particularly by means of conventional and known microscope geometries and architectures.

In one embodiment of the invention, the suspension of particles comprises a transparent liquid surrounding the particles

According to another embodiment of the invention, the suspension is transparent at least within a wavelength range between 300 and 800 nm. The term “transparent” particularly comprises the notion of a hued, colored or tinted suspension, that is the suspension exhibits wavelength-dependent absorption properties.

According to another embodiment of the invention, the dielectric particles have a size such that they exhibit plasmonic resonance with the excitation light. For example, particles of 50 nm in diameter will show strong interactions with excitation light at a wavelength of 488 nm.

According to another embodiment of the invention, the particles are not bound or are connected to the emitters or a vicinity of the emitters.

According to another embodiment of the invention, the particles are inert particles in terms of a chemical reaction taking place with the emitters.

According to another embodiment of the invention, the optical resolution, particularly the optical resolution of the acquired images of the imaging system is diffraction limited.

According to another embodiment of the invention, the Brownian motion of the particles causes spatial correlations, particularly on a length scale smaller than the diffraction limit and/or the optical resolution of the system, in the excitation of the emitters, allowing a varying excitation on length scales that are shorter than the optical resolution of the imaging system such that there is stronger incoherence in the excitation of the emitters.

According to another embodiment of the invention, particularly in case the suspension is arranged between the first window of the sample carrier and the first objective, the interaction of the excitation light and the particles cause the generation of a spatio- temporally varying speckle illumination of the sample space, particularly wherein a speckle size of the speckles is diffraction limited.

According to this embodiment the system allows for generation of image sequences that in which the speckles spatio-temporally vary in the images of the sequence, such that a compressed sensing method may be reconstruct a super-resolved image despite the speckle size being in the diffraction range and thus not in the sub-diffraction range.

This stands in stark contrast to superresolution imaging methods that rely on the assumption that the origin of the fluctuations in the recorded signal have a subdiffraction origin. In case the suspension is arranged between the first window of the sample carrier and the first objective, the fluctuations are diffraction limited in size and therefore may not satisfy the assumption of other superresolution methods that the fluctuations have their origin on a sub-diffraction scale (i.e. independent switching of the emitters). Particularly, the fluctuations induced by the suspension arranged between the first window of the sample carrier and the first objective will not cause the emitters to emit a fluorescence signal completely independent, e.g. completely uncorrelated of a neighbouring emitter, as the fluctuations are controlled by the speckle size that is diffraction limited.

According to another embodiment of the invention, a concentration of the particles in the suspension is adjusted such that in a volume corresponding to the optical resolution of the imaging system in the sample space not more than 10 particles, particularly not more than 5 particles, more particularly not more than 1 particle are present at the same time on average.

This embodiment allows for the system to acquire images of the sample that exhibit intensity fluctuations that are neither averaged out during the acquisition of a single image (or supress the signal of the emitters completely) due to a too high concentration of the particles, nor do the signals only fluctuate seldomly in a series of images which would prolong acquisition times.

It is noted that the concentration of particles is particularly adjusted such that on average more than one, particularly more than 5 emitters in a volume corresponding to the optical resolution, e.g. a diffraction limited volume, are excitable by the excitation light or the plasmonic particles.

This embodiment stands in contrast to single-molecule localization methods that rely on a lower average number of emitters signalling, i.e. being in an on-state, in a diffraction limited volume.

According to another embodiment of the invention, the array detector has an adjustable frame rate with a corresponding reciprocal frame time, at which the array detector may record the images from the sample, wherein a viscosity of the suspension is adjusted such that a diffusion time of the particles in the suspension through a volume corresponding to the optical resolution of the imaging system in the sample space is in the same order of magnitude or longer than the frame time, particularly wherein the diffusion time is not shorter than a tenth of the frame time, particularly not shorter than a fifth of the frame time. The term “volume corresponding to the optical resolution of the imaging system” has to be understood, as a volume that is defined by the optical resolution of the imaging system. The optical resolution might differ along the three dimensions of space in the sample, and even might be locally varying due to varying sample properties or limitations inherent to optical components of the imaging system.

In the cases in which the optical resolution corresponds to the optical diffraction limit, said volume corresponds to the diffraction limited volume.

This embodiment allows a well-adjusted camera frame rate with regard to the temporal fluctuations in the sample. In case the frame rate of the camera is too slow, the fluctuating signals are averaged within a single frame, i.e. image, and thus the information content about the positions of the emitters is reduced, leading to longer acquisition times. On the other hand, if the camera frame rate is too fast with regard to the fluctuation time scale and thus the diffusion rate, the intensity of the signal of the emission light is reduced in each image, which causes a higher amount of noise being present in each image, which too, reduces the information content with regard to the emitters positions.

According to another embodiment of the invention, the particles are smaller than 130 nm in diameter, particularly smaller than 100 nm, more particularly smaller than 75 nm, even more particularly larger than 40 nm or 30 nm in diameter.

Particularly, the particles have an average size in the range of 30 nm to 100 nm.

According to another embodiment of the invention, the particles have an average diameter between 50 nm to 75 nm.

The term “average diameter” particularly refers to the average of diameters of a plurality of particles.

The term “diameter” particularly refers to an average hydrodynamic diameter in the suspension or to an average diameter of the particle. It is noted that the particles do not have to be round, in order to appropriately define the diameter. For example, the diameter of the particle is the diameter that would correspond to a sphere having the same volume as the particle. If the particles are smaller than 20 nm their optical properties regarding absorption, scattering or plasmonic excitation might become too low in order to induce the fluctuation in the emitter emissions.

According to another embodiment of the invention, the particles comprise a dielectric compound or wherein the particles comprise or consist of a metal compound or a metal alloy, such silver and/or gold, particularly wherein the particles are silver particles or wherein the particles are gold particles.

Particularly, silver is known to exhibit plasmonic excitation properties in the visible light range, particularly such that excitation is altered depending on the distance of the particle to the emitter.

Gold particles in turn might be suitable for two-photon excitation and/or for imaging in the red to infrared spectral region.

According to another embodiment of the invention, the suspension has a viscosity that is at least 20 times higher than the viscosity of water under the same conditions, e.g. at room temperature and normal atmospheric pressure, and/or wherein the suspension has a refractive index, wherein said refractive index is within a range of ± 20% of the refractive index of the window, particularly within a range of ± 20% of the refractive index of glass, particularly within 1.2 and 1.8.

By adjusting the viscosity, the diffusion rate of the particles can be adjusted such that frame rate of the array detector and diffusion rate of the particles through a volume corresponding to the optical resolution allow a best possible resolution of the fluctuations in the recorded signals.

According to this embodiment, the refractive index of the suspension matches the refractive index of the window, such that optical aberrations causing a deterioration of the optical resolution of the system caused by a refractive index mismatch between the window and the suspension are kept low.

According to another embodiment of the invention, the first objective is an immersion objective, having an immersion medium arranged between the objective and the sample carrier, wherein the immersion medium consists of the suspension with the dielectric particles.

This embodiment allows for imaging a sample at high-resolution, without the need to apply the suspension to the sample.

According to another aspect of the invention, a liquid immersion medium for oil or water immersion objectives for optical microscopes is disclosed, wherein the immersion medium is, or comprises the suspension with the particles, particularly wherein said suspension has the refractive index of water or the refractive index of immersion oil, particularly the refractive index of glass, more particularly a refractive index in the range of 1.50 and 1.53, particularly at room temperature.

The concentration, composition and optical properties of the particles are disclosed for the embodiment relating to the system and are applicable to the immersion medium as well. Particularly, the particles exhibit light scattering, absorbing, and/or plasmonic properties upon exposure to excitation light, such that spatio-temporal fluctuations in an excitation light passing through the liquid immersion medium may be generated.

According to another embodiment of the invention, the sample carrier comprises a second transparent window arranged opposite the first transparent window, wherein the second window faces, i.e. is closer to the first objective and the first window faces, i.e. is closer to the sample space, wherein the suspension is arranged in a space between the first and the second window.

According to this embodiment, the sample carrier allows for superresolution imaging of the sample, without having the suspension in direct contact to the sample or the first objective.

Moreover, particularly when the first objective is an oil immersion objective, used with an immersion oil, no additional refractive index mismatches are induced by the addition of the second window. In case the refractive index of the suspension is adjusted as well to the refractive index of the immersion oil, no additional aberrations are induced at all, by the sample carrier. According to this embodiment, the first and the second window are essentially stacked on top of each other along the optical axis, while preserving a gap that comprises the suspension.

Particularly, the gap has an equal height throughout the first and the second window, i.e. the windows extend parallel to each other.

According to another embodiment of the invention, the first and the second transparent window are comprised by a container, wherein a container top and bottom side comprise the first and the second window respectively, wherein the container comprises a container wall at its lateral portions that seals the suspension in the space between the first and the second window.

According to another embodiment the container is comprised in the sample carrier.

According to another embodiment of the invention the gap is in the range of one to 200 microns, particularly wherein a thickness of the first and the second window are within 50 to 200 microns.

According to a separate and independent aspect of the invention, a container for superresolution imaging is disclosed, wherein the container comprises a top and bottom side which comprise the first and the second window respectively, wherein the container comprises a container wall at its lateral portions that seals the suspension of particles in the space between the first and the second window.

Particularly, the second window faces, i.e. is closer to, the first objective and the first window faces, i.e. is closer to, the sample space.

According to another embodiment of the separate aspect of the invention, the space has a height in the range of one to 200 microns, particularly wherein a thickness of the first and the second transparent window are within 50 to 200 microns.

The height of the container particularly extends along the optical axis, when the container is arranged in the imaging system. The container comprises a second transparent window arranged opposite the first window, wherein the second window faces, i.e. is closer to the first objective and the first window faces, i.e. is closer to the sample space, wherein the suspension is arranged in the space between the first and the second window.

The properties of the suspension, such as viscosity and medium, and the particles, such as material, optical properties and sizes, can be adjusted or chosen as described throughout to specification even if related to other embodiments and aspects of the invention.

According to another embodiment of the invention, the liquid of the suspension comprises or consists of 97% 2,2’-thiodiethanol (TDE).

According to another embodiment of the invention, the container is comprised by a or the sample carrier.

According to another embodiment of the invention, the container is comprised by a microscope objective.

According to another embodiment of the invention, the detection optics comprise a second objective configured and arranged to collect the emitted light from the sample in the sample space and to propagate the emitted light from the sample to via the detection path and the filter element to the array detector.

This embodiment allows for example for light sheet imaging, particularly in a 90° configuration.

Particularly, the excitation path and the emission path of the imaging system are completely separate light paths.

According to another embodiment of the invention, the system comprises a computer connected to the detector, wherein the computer is configured to receive data from the detector, wherein the data comprise information about a series of recorded images of the detector, wherein in at least some pixels of the images a fluctuating signal, particularly a fluctuating sum signal, is comprised that originates from fluctuating emissions and/or fluctuating excitations of a plurality of emitters that are imaged to the at least some pixels of the images, particularly wherein said fluctuating emissions and/or said fluctuating excitations are caused by the random moving particles in the suspension, wherein the computer is configured to process the data by means of a compressed sensing method that is configured to output a high-resolution image of the sample that has a spatial resolution better than the optical resolution of the system.

This embodiment allows for generating a superresolution image by means of a compressed sensing method, with the system according to the invention. The compressed sensing method is configured to evaluate the recorded fluctuations of the series of images, and to determine the high-resolution image.

Particularly, the series of images comprises not more than 1.000 frames, particularly not more than 100 frames, more particularly not more than 50 frames, wherein from that series the determined high-resolution image is generated to completion, meaning that the high-resolution image is not generated only partially. The determined high- resolution image for example comprises information on emitter positions not only of a fraction of the emitters but for all emitters that contributed to the evaluated signal sufficiently and/or the Nyquist criterion for sampling a structure of the emitters in the sample is met.

According to another embodiment of the invention, the imaging system comprises a motion-inducing device that is configured to prevent sedimentation of the particles in the suspension, particularly wherein said motion-inducing device comprises a shaker, an ultrasonic sound source and/or an electric and/or a magnetic field generating device, wherein said electric and/or magnetic field is configured to induce motion of the particles.

Particularly, the motion-inducing device is connected to or arranged at the container or at the sample carrier.

According to a second aspect of the invention, a method, particularly a computer- implemented method, a computer program comprising computer program code or a computer program product, for superresolution imaging with a microscope imaging system, particularly with an imaging system according the first aspect of the invention, is disclosed, said method, computer program or computer program product being configured to image a sample or to acquire data of the imaging of the sample having emitters that are excitable by excitation light and that are located at the sample, particularly fixedly attached to the sample, wherein the method comprises the steps of:

- Arranging the sample in a sample space of the imaging system, Particularly acquiring a point spread function of the imaging system, particularly a locally varying point spread function of the system from the sample or a different sample,

Exposing the sample to excitation light,

- Arranging a colloidal suspension, particularly a transparent colloidal suspension, comprising particles as the colloid in the sample space or between the first window of the sample carrier and the first objective, wherein the particles exhibit light scattering, absorbing, and/or plasmonic properties in the excitation wavelength range, such that excitation, particularly an excitation strength or an excitation cross-section, of the emitters with excitation light varies randomly both spatially and temporally at the positions of the emitters, due to the light scattering, absorbing and/or the plasmonic properties of the particles undergoing Brownian motion in the suspension, such that light emitted from the emitters also varies randomly both spatially and temporally due to the random excitation.

- Acquiring a series of t images yr of the sample, particularly wherein t is an index for the images of the series, having the optical resolution of the imaging system by repeatedly imaging the emitted light from the sample with a far-field detection optics and an array detector of the imaging system, particularly filtering or pre-processing the images,

- Applying a compressed sensing method executed on a computer to the series of images for determining a series of t non-negative high-resolution images Xf, particularly wherein the determined series of high-resolution images has the same number of images as the acquired series of images, wherein the compressed sensing method minimizes a L1-norm of the series of high-resolution images under a constraint that an Euclidian norm || ... || ₂ of deviations between the series of the determined high-resolution images Xf and the acquired series of images yr is lower than a threshold value s, and wherein the deviations are Poisson-distributed or Gaussian-distributed, wherein the series of high-resolution images Xf have a spatial resolution that is better than the optical resolution of the imaging system, wherein the images of the acquired series exhibit spatio-temporal intensity fluctuations induced by the moving particles in the suspension, particularly wherein the intensity fluctuations originate from light recorded from subsets of emitters that are excited at random at each image of the series of images and particularly wherein at least some emitters are positioned at a distance to each other, in a volume corresponding to the optical resolution of the imaging system, wherein the distance of at least some emitters is shorter than the optical resolution of the imaging system.

The computer program may be stored as a compute product on a non-volatile storage medium, that when executed by a computer executes the method according to the invention, particularly the step comprising the compressed sensing method. Particularly, after executing or during execution of the computer program at least one of the series of high-resolution images is displayed or an average of the high-resolution images.

Particularly, the particles are dielectric particles.

The fluctuations that are induced in the series of images by the randomly moving particles in the suspension are expected to be Gaussian or Poisson distributed with respect to their temporal behaviour around an average or mean fluctuation time.

For this reason, other methods that evaluate fluctuations of signals in images that are based on an independent component analysis (ICA), would not be able to increase the spatial resolution of the images, as such ICA requires the recorded signals to be nonGaussian distributed.

The term fluctuations in the images, particularly relates to temporal changes in intensity in the pixels of the image over time. As the pixel’s intensity changes in the plurality of pixels, the fluctuations are spatio-temporal fluctuations.

It is noted that while Poisson, i.e. photon noise also induces fluctuation in the series of images such fluctuations cannot be used to increase the spatial resolution. Therefore, it is important that the fluctuations are induced by the particles. The term “compressed sensing method” refers to a computer-implemented method that is configured to reconstruct a high-resolution image from the recorded images, wherein the recorded images, particularly the sample structure and the emitter density in the sample, suffice the sparsity requirement necessary for successfully compressed sensing.

For executing the compressed sensing method, a computer with a processor and a storage device for storing the series of images is provided. The computer comprises a processor on which the compressed sensing method may be executed. The images might be transmitted from the array detector to the computer in form of digital data.

It is noted that features, definitions, effects as well as advantages that are disclosed in the embodiments relating to the imaging system, are applicable in the same manner to the method and vice versa, even though they may not be explicitly recited for each aspect of the invention.

According to another embodiment of the second aspect, the particles of the suspension comprise particles made of a dielectric compound or wherein the particles comprise or consist of a metal compound or a metal alloy, such as silver and/or gold, particularly wherein the particles are silver or gold particles.

Particularly silver is known to exhibit plasmonic excitation properties in the visible light range, such that excitation is altered depending on the distance of the particle to the emitter.

Gold particles in turn might be suitable for two-photon excitation and/or for imaging in the red to infrared spectral region.

According to another embodiment of the invention, an average low-resolution image y _m is determined from an average or a sum of the images yr of the acquired series, and wherein the compressed sensing method determines an average high-resolution image x _m corresponding to an average or a sum of the series of high-resolution images Xf.

The average low-resolution image may be determined from a temporal mean value for each pixel of the images of the acquired series. Similarly, the average high-resolution image may be determined from temporal mean value for each pixel of the images of the series of high-resolution images.

Particularly, the average high-resolution image is well-suited to relay the generated information of the increased spatial resolution of the sample to a user, for example by displaying the average high-resolution image, particularly together with the low- resolution image.

According to another embodiment of the invention, each image of the acquired series has a width of / pixels and a height of j pixels, particularly wherein the pixels are arranged in pixel rows and pixel columns of the image, wherein each pixel has a pixel value and comprises information about an area in the sample from which light is imaged onto the pixel, wherein a sampling factor s is selected that is larger than 1 , and is for example an integer number, wherein each image of the series of the determined t high-resolution images Xf, has /*s pixels along a width of the high-resolution images and j*s pixels along a height of the high-resolution images.

In case s is not an integer value, appropriate interpolating methods can be chosen. By increasing the pixel number of the high-resolution images, the sampling of the spatial structures becomes higher and therefore the high-resolution images can be displayed at higher accuracy, i.e. the higher spatial resolution is met by a higher sampling of the high-resolution images, such that finer structures can become visible in the images.

According to another embodiment of the invention, the high-resolution images and particularly the average high-resolution image are determined by the compressed sensing method by means of a convex optimization method, wherein the deviations between the series of the determined high-resolution images Xf and the acquired series of images yr are determined by applying the Euclidian norm || ... || ₂ to Ax - y, wherein A is a matrix, that might comprise information on the point spread function of the imaging system, wherein y corresponds to a vector comprising the pixel values of the images yr of the acquired series of images and particularly the average low-resolution image y _m , wherein x corresponds to a vector comprising the pixel values of the high- resolution images x _f and particularly the average high-resolution image x _m , wherein the convex optimization method determines the matrix A and x such that ||A% - y|| ₂ < £ and the L1-norm on x is minimized, wherein s is a positive threshold value, particularly wherein s is determined from a noise comprised in the images, such that s indicates a limit to the convex optimization method.

Ways to determine the matrix A is known for compressed sensing methods in the art.

Particularly, the matrix A (also referred to as sensing matrix in the context of the current specification) comprises and reflects a localized spatio-temporal randomness arising from the Brownian motion of particles in the suspension and their effect on light over the varying speckle pattern illuminations that cause the fluctuations in the signal. It is noted that the sparsity prior for the compressed sensing method enhances an achievable spatial/optical resolution beyond the two-fold enhancement of structured illumination microscopy (SIM), and the number of measurements, i.e. the number of recorded low resolution images, required to recover the unknown x is drastically reduced due to the efficient randomness of the sensing matrix A. By explicitly enhancing the incoherence of the sensing matrix A, the compressive sensing method satisfies the randomness requirement. This allows to overcome the well-known tradeoff between achievable resolution and measurement time.

According to another embodiment of the invention the threshold value s is indicative of a noise characteristic of the recorded images, wherein the noise characteristic is determined from noise of the array detector and a Poisson noise of the detected light on the array detector.

The determination reflects a physical meaningful limit for resolution increase, such that an over-restoration of the high-resolution images is prevented.

According to another embodiment of the invention, the average high-resolution is displayed on a display.

As elaborated above, this allows for a user to comprehend the achieved increased resolution.

According to another embodiment of the invention, the point-spread function of the imaging system is acquired in a separate measurement.

For the determination of the Matrix A it might be advantageous to provide the PSF to the method, such that an accurate restoration of the high-resolution images is achieved.

The more accurate the PSF is determined, the more accurate the restoration can be. For this purpose, the PSF might be determined with a very high signal-to-noise ratio from a different measurement with a sample dedicated to the PSF measurement. Such a sample might comprise separately arranged bright emitters, such as fluorescent beads comprising a plurality of single fluorophores.

According to another embodiment of the invention, the PSF is determined as a function of location of a field of view of the system, particularly as a function of location in the images, such that a locally varying PSF can be used for determining the high-resolution images. This in turn leads to even better restoration results.

According to another aspect of the invention, the problem is solved by a computer program, comprising computer program code that when executed executes the compressed sensing method according to the invention.

Examples

Particularly, exemplary embodiments are described below in conjunction with the Figures. The Figures are appended to the claims and are accompanied by text explaining individual features of the shown embodiments and aspects of the present invention. Each individual feature shown in the Figures and/or mentioned in said text of the Figures may be incorporated (also in an isolated fashion) into a claim relating to the device according to the present invention.

The invention provides superresolution imaging by exploiting the mathematical theory of compressed sensing (CS) that builds on the fact that most images can be described by a number of parameters much lower than the total number of pixels. CS leverages on this low mathematical complexity (sparsity) in order to recover images from far fewer measurements than normally required.

The method according to the invention allows for computationally generating superresolved images from a sequence of diffraction-limited images. In these images each pixel of the image measures the compressed form of a spatially random subset of sub- diffraction-modulated true signal.

From repeated samplings of the sample, the signal measured by each low-resolution pixel can be spatially decompressed into multiple pixels, effectively creating a new image with a higher sampling rate and an increased spatial resolution. The term low- resolution (LR) pixel refers to the pixels of the images acquired by the array detector, i.e. the images that are provided to the method in order to increase the resolution. The term low-resolution image is also referred to as the acquired image in the context of the current specification, i.e. the LR frame or image is an image acquired with the imaging system and has a spatial resolution that corresponds to the optical resolution of the imaging system.

The relationship between the low-resolution images and the high-resolution or superresolution images can be modelled by a system of linear equations, y = Ax.+noise, (1) where y is the vector form of the series of low-resolution images, A is the sensing matrix describing the linear mapping from the source to the observation, containing information about the illumination modulation that may comprise the spatio-temporal varying speckle patterns generated by the suspension and the PSF, x is the vector from of the high-resolution images that are to be determined, and noise represents the measurement noise. Spatially resolving (i.e. decompressing or reconstructing) x from the 20 low-resolution images is the inverse problem, for which L1-norm regularization is used to solve for x with the assumption that it is sparse (a small fraction of non-zero pixels). Since it is not known a priori where intensity was modulated, this represents a less conventional method of compressed sensing [1], but based on theoretical analysis, it can be shown that x can be resolved by putting a set of constraints on the sensing matrix and x, while minimizing the real non-negative function, minimize ||x|| _t subject to x > 0 and || x - y|| ₂ < E (2) where x > 0 is a non-negativity constraint on x, i.e. the pixels of the high-resolution image must not be negative and s is the maximal error constraint, which states that the predicted high-resolution images should closely match the measured low- resolution images and that deviations must follow a Poisson model. It is noted that in the context of the current specification the terms high-resolution and super-resolution (SR) image are used interchangeably. Further, each SR image in x comprises s x s more pixels than each low-resolution image y. Thus, a virtual grid of s * s pixels on each pixel of each low-resolution (LR) image is overlaid or generated. The pixels of the virtual grid of s x s pixels are referred to as superresolution (SR) pixels. It is noted that in the context of the current specification s might be referred to as sampling factor or scale factor interchangeably.

Each super resolution pixel models the contribution of the luminescence, particularly the fluorescence coming from the emitters correspondingly located in the sample space within that pixel. Therefore, calculating the correct intensities of every pixel in all SR-images represents a higher resolution reconstruction (Fig. 9). The method according to the invention computes the SR pixel intensities for all frames at once, by solving a comparably large convex optimization problem, for which a modified version of the solver SCS [10] driven by a MATLAB interface code YALMIP [11] can be used.

Therefore, the minimization problem is converted to a convex program with a linear objective function, and the solution may be found using an established first-order solver [2] (Fig. 1).

The following sections formulate the convex optimization problem: the involved variables, objective function and constraints.

In a first step an acquired sequence of low-resolution (LR) images is provided and encoded into a vector as follows: Denote the LR frames by y (i, j, t), where i, j are pixel coordinates and t=1, ...,T is the frame number. Additionally, set y (i, j, 0) = y (i, j) the average of y (i, j, t) over t. Compared to an individual pixel y (i, J, f), y has reduced noise due to the averaging: the corresponding noise variance fory (/, j) is a ( , J, t) ² = a(/, 7) ² /T. The entire vector y = y (/, j, f) has dimension N = w*h*(T+1), where w and h are the width and height of any LR frame.

In a second step a solution /W-component vector x encoding the reconstructed SR image is prepared; x is the concatenation of three smaller vectors x _M , x _F , x _B that are defined as follows:

■ x _M = x _M (/, j) are the w*s*/?*s unknowns representing the SR solution image.

■ x _F = x _F (/, j, t) are w*s*h*s*T unknowns capturing the effect of particle (particularly silverj-induced intensity modulation: they represent the difference between a reconstructed pixel in frame t and its corresponding time average. These are auxiliary variables that are used in formulating the convex optimization problem but do not otherwise contribute to the final SR output.

■ x _B = (x _B (1), x _B (2), x _B (3), x _B (4)) is an optional 4-vector encoding non-specific background illumination: four illumination sources are distributed (evenly spaced) across the border of the reconstruction region. For every measured pixel the predicted background contribution is a distance-weighted average of the four intensities x _B (1), x _B (2), x _B (3), x _B (4). In a third step, the convex optimization problem is formulated as follows. Note that, for easier writing of formulas, the notation x (i, j, 0) = x _M (i, j) and x (i, j, t) = x _F (i, j, t) is used.

The optimization problem is to minimize the objective function, subject to the following constraints:

Maximal error constraint (Constraint 1) states that the predicted SR image x has to closely match the measured LR image y and that differences between their intensities are Gaussian distributed (see section “Noise model”). Therefore, the maximal error constraint may be formulated as: || Ax - y|| ₂ < £, where:

■ x and y are previously defined.

■ s is the reduced chi-square target defined in the section "Noise model" and corresponds to a noise characteristic of the array detector.

■ A = P*Q*R is a matrix (N * M) formed by the matrix product of:

• R; which is an M x M matrix used to vary the weight of particle-mimicking variables x _F in the problem by randomly multiplying each variable with either -1 (means the pixel emits less photons than its value in x _M ) or +1 (means the pixel emits more photons than its value in x _M ). The net effect is that the particle-mimicking variables can have a negative contribution due to this randomization of signs. On the x _M and x _B variables R acts as the identity matrix.

• Q; which is an M x M matrix associated with the convolution of each pixel with the high-resolution, estimated PSF and addition of the linearly interpolated background; i.e., if u = Qx and i = ip(i, j) is the PSF then u(i, j, t) = f(i,J)+ {h,kj ip(i-h, j-k)*( x _M (i, j) + x _F (i, j,t)) where i (i,j) is zero outside the support of the PSF, and f expresses the contribution of the background xB. Q acts as the null matrix on the subspace spanned by x _B that is, background illumination is not considered when evaluating how close the SR solution is to the LR image. The solver SCS may be modified to handle convolution with the PSF using FFT (Fast Fourier Transform) instead of matrix-vector products.

• P; is an N x M matrix that sums the intensities of the s x s SR pixels and maps the SR pixels to the corresponding LR pixel, thereby downscaling the predicted images. A Non-neqative solution (Constraint 2) requires that all variables x, being measures of the intensity of light, be non-negative.

Upper bound values for Aq-mimickinq variables (Constraint 3) states that x _F (/, j, t) < 0.7 * x _M (/, j). This stems from the fact that, empirically, the extent of intensity modulation observed was ~ 30 % relative to mean.

Lower bound values for Aq-mimickinq variables (Constraint 4): by the same reasoning as for Constraint 3, a constraint may be imposed such that x _F (/, j, f) > 0.05 * x _M (/, /)■

Density constraint (Constraint 5) stems from the motivation to impose a structure on the solution. The idea is that there should be no pixel pairs (H, j1) and (i2, j2) which are resolved in the solution if their distance is less than the theoretically achievable resolution supported by the method according to the invention. This constraint places a condition on the solution x (i, j, t), requiring that, for all t, the functions Lt (i, j):= x (i, j, t) satisfy a kind of “concavity” condition for close enough points: if P1= (H, j1) and P2= (i2, j2) are at Euclidean distance less than s, then it is required that Lt (P3) > (1/k)*( Lt (P1) + Lt (P2)) for P3 = midpoint of the line segment joining P1 and P2, and for some fixed integer constant k (when k= 2 this is the usual concavity constraint).

In a first example demonstrating the feasibility of the method according to the invention, a synthetic template reference library (pixel size 18 nm) composed of 11 pairs of input signals, i.e. emitters, is provided, wherein the pairs of emitters are separated by distances ranging from 36 to 18 nm (Fig. 1 B). 12 runs are simulated, wherein each run comprises a series of 100 diffraction-limited images, i.e. low-resolution images at pixel size 72 nm (the typical pixel size in an sCMOS camera at 90x magnification) using either an intensity modulation scheme, or no modulation as control. Camera noise and Poisson photon noise are added to the sequence of images.

When modulation is applied to the emitters, the intensities of the emitters in the template are varied randomly by ± 30% relative to a mean intensity of the emitters.

Reconstruction using pixel size 18 nm (i.e. setting the sampling factor s = 4) resolves separations down to 36 nm, which is the smallest theoretically separable distance in the library, but only when intensity modulation was applied (Fig. 2, left graph).

Precision dropped with decreasing separation distances and increased with the number of imaged frames, reaching a plateau at 20 frames (precision score for 36 nm: 70 ± 11% (mean ± s.e.m.)). This outperforms reconstruction in the absence of modulation (Fig. 2 right graph, no separation at distances < 72 nm) and standard deconvolution algorithms (Huygens) by a factor of ~4, and standard wide-field (Low res) by a factor of ~8 (Fig. 3).

In a second experiment, the method according to the invention is executed on computationally simulated microtubules (Fig. 4). After generation of 20 low-resolution images at pixel size 72 nm with intensity modulation, said images are reconstructed into super-resolved (SR) images using grid pixel sizes of 36 or 24 nm. Line profiles through selected filaments showed that the method according to the invention resolves closely aligned microtubules, and that filaments that cannot be well separated with pixel size 36 nm can be resolved better with pixel size 24 nm. Additionally, Pearson correlation analysis shows that reconstructions errors in dense regions decrease as the sampling frequency increases, i.e. smaller pixel size/ higher scale factor for the SR images. Thus, by increasing the sampling factor s used during decompression, the spatial resolution in the SR image can be increased.

Thus, the in-silico findings show that weak spatio-temporal random modulation of signal (dynamic modulation) is sufficient to achieve a compressive device for superresolution imaging without a priori knowledge of the position of those modulated emitters. It is noted that in the context of the specification, the term “modulation” and “variation” in the context of signal fluctuations are used synonymously.

For measurements on non-virtual samples the corresponding imaging system comprises far-field optics microscope, such as a TIRF or Widefield fluorescence microscope with an array detector, for example in form of a light sensitive camera that is configured to detect fluctuating light from single emitters. In correspondence to the mathematical ground work, it can be noted that light passing through a medium containing scattering particles corresponds to applying a random Gaussian matrix on light [3], To achieve this effect, silver (Ag) nanoparticles of diameter 55-75 nm suspended in a suspension of 97% 2,2’-thiodiethanol (97% TDE) are used. Ag particles 502 of such dimensions exhibits strong, wavelength-dependent scattering and absorption of light that may be exploited by placing said randomly moving particles in the light path close the sample (Fig. 5). When light hits an Ag particle 502, the light intensity reaching the emitters 506 at the sample is lowered due to the particle 502 and the corresponding fluorescence emission of these emitters is lowered too, while neighboring emitters that remain exposed to the full amount of light will not be affected. This fits the CS requirement for spatial modulation of signal (incoherence). The second CS requirement, randomness in signal sampling, is achieved by the high viscosity of 97% TDE, which slows down Ag particle motion, ensuring that movement by Brownian motion primarily occurs between consecutive frames. The concentration of the Ag particles is adjusted such that enough random illumination events are obtained for standard fluorophores without strongly affecting the overall signal-to-noise ratio of the signal. Acquiring a sequence of LR images with this setup subjects the signal that each pixel samples to a unique spatiotemporal light modulation (dynamic modulation) scheme.

In a third experiment DNA origami STED nanorulers are used with a mark-to-mark distance of 90 nm [4, 5], Using a reconstruction pixel size of 27 nm, the average distances were properly reconstructed from a sequence of 30 frames, taken at an interval of 100 ms (Fig. 6). Next, we stained microtubules in COS-7 cells using indirect immunofluorescence (Fig. 7) and acquired 50 frames in the aforementioned imaging setup.

Replaying these LR-images as a movie shows small shadings of the fixed microtubule sample that stochastically move through the field of view, while consecutive frames taken in the absence of Ag particles do not show such fluctuations (not shown). This confirms that the fluctuations are caused by Ag particles that absorb and scatter the incoming light as they undergo Brownian motion.

Using as few as 20 frames of the acquired LR images allow reconstruction the low- resolution information into a superresolution image, choosing a sampling factor of 2, i.e. a pixel size of 36 nm. Line profiles through 25 microtubule filaments show an average width of ~80 nm when using Ag, compared to -143 nm when not using Ag, demonstrating that the method enables superresolution microscopy even with a small sampling factor of 2 and without the need for a prohibitively large amount of Ag particles (Fig. 7). Since Ag particles of diameter 55-75 nm have the potential to modulate light of multiple wavelengths, the method may be applied to multi-color experiment using emitters that emit in different wavelength ranges.

In another experiment, COS-7 cells are transfected with an outer mitochondrial membrane marker mEGFP-TOM20, and mitochondrial dynamics are imaged at high speed and at superresolution for extended periods of time (Fig.8). For this purpose, 1,000 consecutive images were taken at a frame rate of 25 frames/s, and two modes were used for super-resolved image reconstruction. In the first mode, 20 consecutive frames are used for reconstruction into one super-resolved average image of 36 nm pixel size, resulting in a time interval of 800 ms in the final time-lapse sequence. In the second mode, super-resolved images are generated using a sliding window of 10 frames every 5 frames, resulting in a time interval of 200 ms in the final time-lapse sequence. The high spatial resolution of the method according to the invention ensures detection of densely packed mitochondria. Additionally, it is possible to capture mitochondrial dynamics at very high temporal resolution across a large field of view (Fig. 8), providing high spatial detail of fast-moving mitochondria-derived vesicles. Even though some photobleaching might be observed over the 100 frames, the reconstruction was robust to changes in overall intensities, indicating a proper modeling of the measurement noise in each pixel as the sum of Poisson photon count noise and camera readout noise. Such processes cannot be captured with a 3- fold lower sampling frequency due to motion blur (Fig. 4C). Thus, in combination with live cell imaging, the method according to the invention, also enables nanoscale 2D spatial imaging at sub second temporal resolution with minimal photobleaching.

Silver particles used for the experiments are from econix silver nanospheres (Nanocomposix) and have a diameter of 50-70 nm and a concentration ~ 5.2 x10 ¹² particles/mL.

Estimation of camera characteristics

Camera characterization may be done following the approach of Huang et al. [7] and Diekmann et al. [8], For dark pixel offset, and read noise measurements, it is ensured that the camera chip is in darkness during the acquisition of 2000 frames. The baseline (offset) for each pixel is determined by the mean value per pixel and the read noise by the standard deviation over all frames. Single pixel gain is estimated as follows: (a) measuring a sequence of 2000 frames for a series of exposure times by directly shining light on the objective (starting with 1 ms up to 20 ms exposure, (b) for each exposure time, calculating the variance and the mean, (c) calculating the camera gain (e-/grey level) from the variance and the mean (both per pixel) by regression using the following equation: x — offset Gain = - variance — a where x is the mean of the sequence of frames taken at the first step, variance contains the full noise (camera read noise + Poisson shot noise), and a is the variance of the camera read noise, offset relates to the baseline of each pixel as elaborated above. The measured signal is transformed to photoelectrons by subtracting the measured offset value and dividing by the measured camera gain.

Noise model

As previously described [6, 7], it is assumed that for every camera pixel, the noise is caused by Poisson noise and camera read noise. Other noise sources may be ignored since they typically have a small magnitude. Poisson distributions are similar to Gaussian ones if the A parameter of the Poisson distribution is large enough, and since measured signals have a comparably high intensity, the Poisson may be approximated with a Gaussian distribution. Risch et al. show how compressed sensing problems can be solved with multivariate Gaussian errors [9], The full error distribution is therefore the sum of the Gaussian approximating the Poisson distribution and the camera readout noise (since both noise sources are Gaussian and Gaussians can be summed by summing their means and variances).

Widefield Microscopy

Widefield images may be recorded using a VisiTIRF Cell Explorer, a custom modified Nikon (Eclipse Ti-E) inverted microscope with a motorized linear TIRF condenser by Visitron systems. The main objective used in the experiments is a 60XS Oil (CFI Plan Fluor) with iris, NA 1.25 and working distance 0.22 mm. Otherwise, an 100X Oil NA 1.49 (CPI Plan Apo TIRF) objective may be applicable as well. However, due to its short working distance, the suspension may be added on top of the objective directly. To reach 90X magnification with the 60X objective, an intermediate magnification of 1.5X inside the body of the microscope was used. The microscope is equipped with a Perfect Focus System (NIR laser based PFS) and a Hamamatsu Orca Flash4.0 V3 camera with a pixel size of 6.5 pm. Since this is a sCMOS camera with pixel-based statistics, all pixel readouts may be characterized before analysis.

Sample preparation for compressive imaging

Double sticky tape was placed on ultra-thin glass slide (thickness: 100 pm) of the same dimensions like the Nunc, Lab-Tek 8-well chambered coverglass system with borosilicate glass bottom 1.0 (#155411, ThermoFisher Scientific). 7 pL of silver nanoparticle suspension (4% silver stock v/v/ in 97% TDE (diluted in ddH2O water)) were added in every well-spot. Before preparing the silver suspension with TDE, silver stock was vigorously vortexed for 60 sec and then mixed thoroughly by pipetting in 97% TDE. Nunc, Lab-Tek 8-well chambered coverglass system with borosilicate glass bottom 1.0 carrying the stained sample of interest was finally fixed on top of the adhesive such that the silver layer will be between the sample and the objective. The total height of the setup was therefore -220 pm. Fixed cells were mounted in PBS and live cells in phenol-red free imaging media. In addition to the viscosity required for random motion of silver particles, 97% TDE has the added advantage of matching the refractive index of oil and glass, which preserves the signal intensity of structures imaged through the Ag-containing liquid layer.

Experiments on fixed cells

All imaging relating to the invention was done on the VisiTIRF Widefield microscope (refer to Widefield Microscopy). Illumination of the sample was performed using laser stimulation in epi mode and straightness of the light coming out of the objective was ensured by checking the position of the laser beam on the ceiling. To ensure a high- quality, symmetric PSF, it is ensured that the stage is flat with a maximum deviation of ± 2 pm between each edge before performing the experiments. Acquisition is done in a continuous mode with a total number of frames set to 20-100/channel, if the channel would be used for reconstruction. Exposure time of the camera may be set to 50-100 ms.

Estimation of the point spread function (PSF)

One parameter in the compressed sensing reconstruction is the PSF, which describes the image formation process of a small object over pixels. Generally, the microscope PSF is measured and estimated for each channel. For measurement, diluted Tetraspeck beads suspension was added for 1 hour. Afterwards, this suspension was removed and only beads that stick to the surface of the slide remain. From these beads the PSF can be estimated with high accuracy.

Figure legends

In the following additional short descriptions are given on the content of the Figures.

Super Resolution in silico with structurally perturbed compressed sensing. Fig. 1 : Relationship between the low-resolution compressed image (y) (and the superresolution decompressed image (x) .On the left column the schematic image is shown depicting a low-resolution image on the top and a high-resolution decompressed image x where two emitters (white pixels) as ground truth can be seen. The decompressed image comprises s ² -times more pixels. Imaging (compression) the two emitters results in the low-resolution image, wherein applying the compressed sensing method (decompression) recovers the two emitters.

The upper panel describes in more detail the image formation process, where a sensing matrix describes the linear mapping from x to y .For 20 simulated frames the following is done: (i) signal in x 100 is randomly modulated 101 with a modulation matrix (Z\) that for example mimics a varying excitation profile. The result 102 is convolved with the PSF 103, each s x s pixels are summed up 105 into a new compressed pixel (s: the scale factor). This process is repeated 106 such that a series of low-resolution images is generated 107. Lower panel: L1-norm - regularized sparse recovery of x using minimize ||x|| _x subject to x > 0 and || Ax - y|| ₂ < E) 108, where (E) designates the mismatch between measured and predicted intensities and m < n. The optimization function is solved using a first order splitting conic solver. For this purpose, the series of low-resolution images 107 is subjected to the compressed sensing method yielding the emitter distribution 109, 100

Fig. 2: Quantification of the effect of increasing number of frames on reconstruction precision (shown as a grey scale LUT, the lower the grey value the better the separation precision). The absolute difference to the template distance is used to calculate the precision score (%). Mean precision score for the 12 runs in the presence (left graph), or absence (right graph) of random intensity modulation. Black entries in the matrix indicates that no separation took place in the 12 runs.

Fig. 3: Comparison of precision scores (%) with 20 frames, in the presence of intensity modulation 300, in the absence of modulation 301 , with standard deconvolution using Huygens software 302 or simple intensity thresholding 303.

Fig. 4: Reconstruction of synthetic microtubules 403, 407 with pixel size 36 nm (upper panel) or 24 nm (lower panel. Ground truth images of synthetic microtubules 400, 404. Pixel intensities were randomly modulated and the frames were convolved with a PSF 401 , 405 of the respective pixel size to produce 20 low-resolution images 402, 406 with a larger pixel size (72 nm). Right: reconstructed images Scale bars 75 nm. Intensity units are arbitrary digital units (DN). The method according to the invention, also referred to as Brownian Excitation Absorption Microscopy (BEAM) uses compressive imaging.

Fig. 5: Imaging system for compressive imaging. Parallel light 500 coming out of the objective 501 passes through a viscous layer of 97% TDE 503 containing Ag particles 502 of diameter 50-75 nm. The Ag particles 502 undergo Brownian motion in the viscous suspension 503 giving rise to the signal fluctuations in the detected signal. The dark grey shaded areas indicate a strong excitation by the excitation light 500, wherein the brighter areas indicate a lower excitation level, due to the AG particles 502.

Fig. 6: Gattaquant STED-90 nm nanoruler reconstruction. STED-90 nm Oregon Green slides were assembled over a layer of Ag suspension and 20 consecutive frames were acquired and used for reconstruction with pixel size 27 nm. Graph shows quantification of distances from 132 reconstructed rulers (Mean ± S.D.).

Fig. 7: Reconstruction of real microtubules. COS-7 cells stained for p-tubulin are mounted on a layer of Ag particles containing suspension and 20 consecutive frames were taken and used for reconstruction with pixel size 36 nm.

Fig. 8: Schematic representation of the two imaging modalities used to capture mitochondrial dynamic behavior. COS-7 cells transiently expressing EGFP-TOM20 were continuously imaged at a frame rate of 25 fps for 1000 images and a total time of 40 s. For the decompression step, either the whole sequence may be chopped into blocks of 20 frames and each 20 frames are used to reconstruct one super-resolved image or 10 frames with a moving window of 5 frames maybe used. The images represent a movie reconstruction corresponding to 0,8 s time intervals between each image.

Fig. 9: Scheme of dividing original image pixels into s-times oversampled grids with smaller pixel sizes (digital grid).

Fig. 10: Schematically depicts a system 1 according to the invention. The system 1 comprises a light source 2, for example a laser, that emits laser light that is guided toward a first objective 3, via a wavelength dependent beam-splitter 14. The light propagates through the first objective 3, which in the present case is an oil immersion objective. Between the objective 3 and the sample 4 comprising a plurality of excitable emitter (not shown), the sample carrier 5 is arranged. In this example, the sample carrier 5 is formed by a container comprising a first window 6, a second window 12 that are arranged opposite from each other and extend parallel to each other. The container walls are formed by a wall element 15 that laterally encloses a container volume. In the container volume the suspension 10 is comprised in which the particles 11 randomly diffuse. The suspension may have the same refractive index as the first and the second window, wherein the windows may be made of glass.

The excitation light passes through the suspension, which causes a varying excitation of the emitters at the sample 4. The emitters comprised in the sample emit light that is collected by the first objective 3. The detected light passes through the beam splitter 14, and is focussed with an imaging lens 7 onto the detector 9 after begin filtered with a filter 8 for residual light that is not within a selected detection wavelength.

The detector outputs a signal that comprises information about the emission signal of the emitters. Said signal may be used for reconstructing a series of low-resolution images of the emitters in the sample 4.

The computer may be configured to reconstruct a series of high-resolution images from said low-resolution images.

Further, it is noted that the sample carrier may be used independently of the other components of the system and the specification of the system.

The suspension 10 comprised in the container volume may have a selected viscosity and particles of a selected size distribution.

In Fig. 11 a similar embodiment of Fig. 10 is shown. In contrast to Fig, 10, the sample carrier 5 essentially consist only of the first window 6. The suspension comprising the particles 10 is provided to the system 1 in form of an immersion medium directly between the objective and the first window.

In Fig. 12 A, left panel, a plot is shown that specifies that number of frames (x-axis) that are required in order to increase the probability of success (gray shading) in resolving an underlying structure having a predefined number of non-zero pixels (y- axis). The density is a measure of how many pixels are non-zero relative to the total number of pixels. The underlying structures for density of 30 (upper row of right panel) and 4 emitters (lower row of right panel) are shown in the plots in the right panel. Here, the underlying structure of emitters is shown as the ground truth (“GT”), and the resulting super-resolved images after processing 4 frames (“SR 4 frames”) and 20 frames (“SR 20 frames”). It can be seen that apart form the intensity of the emitters after 4 frames the SR 4 frames images for density 4 the underlying structure is resolved completely, wherein for density 30, the underlying structure is not completely resolved after 4 frames. After 20 frames, however, both structures are completely resolved, i.e. each emitter can be seen as a single separate spot.

In Fig. 12 B a similar plot of a simulated emitter distribution is shown, wherein a line density of lines that are arranged in a relative spacing of the Full-Width at Half Maximum (FWHM) of the Point Spread Function is shown. Depending on the number of lines and the relative spacing in relation to the FWHM of the PSF, the method according to the invention is able to resolve the underlying emitter distribution.

In the left column of Fig. 12 B, the line density is FWHM/1.5 meaning the lines are arranged below 1.5 times below diffraction limit. The method resolves the two-line structure (upper row) as well as the four-line structure (lower row). For an increased line density of three lines in the PSF (FWHM / 3) the method fails to resolve the four- line structure (lower row) but is capable to resolve the two-line structure (upper row). If the density is increased even further (FWHM /4), the method does not resolve the two- lines nor the four-line structure.

References

[1] Z. Yang, C. Zhang, L. Xie, Robustly Stable Signal Recovery in Compressed Sensing With Structured Matrix Perturbation. IEEE Trans. Signal Process. 60, 4658-4671 (2012).

[2] B. O’Donoghue, E. Chu, N. Parikh, S. Boyd, Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding. J. Optim. Theory Appl. (2016), doi: 10.1007/s10957-016-0892-3.

[3] A. Liutkus, D. Martina, S. Popoff, G. Chardon, O. Katz, G. Lerosey, S. Gigan, L. Daudet, I. Carron, Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium. Sci. Rep. 4, 5552 (2014).

[4] J. J. Schmied, M. Raab, C. Forthmann, E. Pibiri, B. Wunsch, T. Dammeyer, P. Tinnefeld, DNA origami-based standards for quantitative fluorescence microscopy. Nat. Protoc. 9, 1367-1391 (2014).

[5] J. J. Schmied, A. Gietl, P. Holzmeister, C. Forthmann, C. Steinhauer, T. Dammeyer, P. Tinnefeld, Fluorescence and super-resolution standards based on DNA origami. Nat. Methods. 9, 1133-1134 (2012). [6] S. Culley, D. Albrecht, C. Jacobs, P. M. Pereira, C. Leterrier, J. Mercer, R. Henriques, Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nat. Methods (2018), doi:10.1038/nmeth.4605.

[7] F. Huang, T. M. P. Hartwich, F. E. Rivera-Molina, Y. Lin, W. C. Duim, J. J. Long, P. D. Uchil, J. R. Myers, M. A. Baird, W. Mothes, M. W. Davidson, D. Toomre, J. Bewersdorf, Video-rate nanoscopy using sCMOS camera-specific singlemolecule localization algorithms. Nat. Methods. 10, (2013).

[8] R. Diekmann, K. Till, M. Muller, M. Simonis, M. Schuttpelz, T. Huser, Characterization of an industry-grade CMOS camera well suited for single molecule localization microscopy - high performance super-resolution at low cost. Sci. Rep. 7,14425 (2017).

[9] I. Rish, G. Grabarnik, in 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton) (IEEE, 2009);

[10] B. O’Donoghue, E. Chu, N. Parikh, S. Boyd, Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding. J. Optim. Theory Appt. (2016).

[11] J. Lof berg, in Proceedings of the IEEE International Symposium on Computer- Aided Control System Design (2004).

*****