Field of the invention

The invention relates to structured illumination microscopy, and, in particular, though not exclusively, to methods and systems for structured illumination scanning microscopy and a computer program product for executing such methods .

Background of the invention

Super resolution microscopy methods are known for resolving spatial resolutions beyond the diffraction limit. An example of such a super-resolution method is the well-known structured illumination microscopy (SIM) as e.g. described in the article by Gustafsson et al, Journal of Microscopy, Vol. 198, Pt 2, May 2000, pp. 82-87. Herein, a sample is

illuminated with a series of light patterns (also referred to as structured light) comprising bright maxima (associated with high light intensities) and dark minima (associated with low light intensities), which cause normally inaccessible high- resolution information to be encoded into observed images.

Recorded images are subsequently processed to extract the high-resolution information and construct a super-resolution image, i.e. an image having a resolution surpassing the diffraction limit.

The goal of SIM is to obtain a super-resolution image of an unknown sample structure - or more precisely - of an unknown spatial distribution of fluorescent dyes. Structured illumination light, typically generated using a diffraction grating, is incident on the spatial distribution of

fluorescent dyes and causes the fluorescent dyes to emit emission light. The intensity of emission light originating from a point on the sample is proportional to the product of dye concentration at this point and the effective light intensity of the structured light at this point. The resulting emission light is thus a product of two patterns, namely the structured light and the light of an (unknown) distribution of fluorescent molecules. As a result, the emission light will contain observable Moire effects from which normally

inaccessible high-resolution information can be derived as explained in Gustafsson et al .

The light pattern typically includes a two or three dimensional arrangement of parallel lines in one or more planes perpendicular to the optical axes. This light pattern is used to expose a sample several times, wherein after each exposure, the light pattern is shifted and/or rotated. Recoded images of the emission light of each exposure are used to reconstruct a high resolution image using a known mathematical image reconstruction algorithm. An example of such method is described by Lai et. al , in document arXiv : 1602.06904vl

(retrievable from website https://arxiv.org) . For a successful reconstruction of a high-resolution image, it is important that the parameters of the structured light pattern on the sample can be determined as accurate as possible. Deviations from the exact position will cause artefacts in the

reconstructed image.

For example a 2D arrangement of parallel lines may be represented by a sinusoid having an amplitude, a modulation depth, an initial phase, a direction and a spatial frequency, wherein the amplitude relates to the light intensity of the bright maxima and dark minima of a light pattern and the modulation depth relates to the difference in light

intensities. The spatial frequency defines the number of bright maxima per distance perpendicular to the line- direction. Furthermore, the initial phase defines the position of a light patterns on a sample relative to each other. For reconstructing a high-resolution image it is important to know the parameters of the structured light pattern as accurate as possible .

For structured light generated on the basis of a diffraction pattern it is difficult to control the parameters that define the pattern on the sample. Therefore, the

parameters such as the phase information and the modulation depth, is typically retrieved a posteriori from the recorded images using image analysis algorithms. These algorithms can only retrieve these parameters if the recorded images comprise a clear enough "footprint" of the used structured light pattern. This means that the recorded images must comprise discernible bright maxima and dark minima corresponding with the maxima and minima of the used structured light pattern otherwise the algorithms cannot retrieve the positions of the bright maxima on the sample.

Emission light passes through at least one NA- limiting optical element (e.g. an objective lens) before being incident on an image sensor. However, it is known that such NA-limiting element acts as a spatial low pass filter in the sense that it attenuates the amplitudes of high spatial frequency components. Hence, if the spatial frequency of an applied structured light pattern is too high, i.e. if the structured light pattern is too fine, the amplitude associated with this frequency may be attenuated significantly by the lens system. As a result, the bright maxima and dark minima corresponding to the structured light pattern may only be weakly present in a recorded image. The footprint of the structured light pattern may then not be discernible anymore by the image processing algorithms, which may impede

determination of the parameters such as the phase information and the modulation depth. The lens system thus imposes an upper limit to the spatial frequency that can be used for structured light patterns. This is disadvantageous, since the spatial frequency of the applied structured light patterns are preferably as high as possible, because finer structured light patterns yield higher resolutions.

US 2013/0314717 Al discloses a SIM system that includes a modulator to modulate the light such that the sample is exposed to structured light. In this scheme, a modulated focal spot of light is scanned over the sample.

During scanning, the modulation is controlled such that the sample is exposed to a 2D pattern of parallel lines wherein the lines make an angle with the scanning direction. In the proposed system the scanning movement of the focal spot over the sample must be accurately timed with respect the modulation in order to generate an accurate 2D pattern of parallel lines of a high spatial frequency. Further, also in this document the parameters of the light pattern are derived from the captured images.

Hence, there is a need in the art for improved structured light microscopy systems. In particular, there is a need in the art for systems and methods for SIM which allow the use of high frequency light patterns.

Summary of the invention

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely

hardware embodiment, an entirely software embodiment

(including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit, " "module" or "system." Functions described in this disclosure may be implemented as an algorithm executed by a microprocessor of a computer. Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium (s) having computer readable program code embodied, e.g., stored, thereon.

Any combination of one or more computer readable medium (s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , an optical fiber, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro ^¬ magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber, cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java(TM), Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming

languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) .

Aspects of the present invention are described below with reference to flowchart illustrations and/or block

diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be

implemented by computer program instructions. These computer program instructions may be provided to a processor, in particular a microprocessor or central processing unit (CPU) , of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer, other programmable data processing apparatus, or other devices create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks .

These computer program instructions may also be stored in a computer readable medium that can direct a

computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function (s) . It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block

diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It is an object of the present disclosure to provide methods and systems for structured illumination microscopy using a scanning microscope. In particular, it is an object of the present disclosure to provide methods and systems for scanning structured illumination microscopy (SSIM) that enable the use of periodic light patterns having spatial frequencies that are equal or higher than the spatial frequencies and that eliminate the use of posteriori image analysis for determining the light pattern parameters which are used by the

reconstruction algorithm.

In an aspect, the invention relates to a method of forming a high-resolution image of a sample using a scanning microscope controlled by a processor. The method may include: the processor receiving or generating control information for controlling a scanning microscope, the control information defining a plurality of different periodic patterns on the basis of one or more pattern parameters; the processor using the control information to control the scanning microscope to expose the sample to multiple illumination patterns, each exposure to an illumination pattern causing one or more optical excitations in the sample, the light originating from said optical excitations forming an emission light signal; the processor controlling an imaging system to capture multiple images, each image being associated with an emission light signal of one of the multiple exposures, and, the processor using an image reconstruction algorithm for forming a high- resolution image on the basis of at least one of the one or more pattern parameters and the captured images. In an

embodiment, the one or more pattern parameters may include at least one of: a spatial frequency, a periodicity direction, an initial phase, an illumination intensity.

Hence, the method according to the invention uses control information defining a plurality of different periodic patterns on the basis of one or more pattern parameters which is used by a processor, e.g. a computer, of the scanning microscope to expose a sample to multiple different period illumination patterns, i.e. a light pattern that provides a certain light distribution, e.g. a 2D light distribution or a 3D light distribution, within the sample.

As the scanning microscope can accurately control the position of a focused illumination light spot on the sample, a periodic 2D or 3D illumination pattern can be accurately written into the sample using the control information. Here, the focused illumination light spot may comprise a light intensity profile, for example a Gaussian intensity profile, so that areas of the pattern may be understood to have been illuminated with a light intensity exceeding a predetermined threshold light intensity. In the context of this application, the intensity of light may relate to the physical quantity irradiance.

The parameters defining the periodic patterns - which are determined before exposing the sample - may be used in the reconstruction algorithm. The invention thus eliminates or at least substantially reduces the need for a posteriori image analysis in order to determine illumination light pattern parameters that are used by the reconstruction algorithm. This way, light patterns may be used that have a higher spatial frequency than the light patterns that are used in SIM systems known in the prior art.

In an embodiment, controlling the scanning microscope may include controlling a scanning mirror of the scanning microscope to move at least one focused illumination light spot through the sample positioned a sample holder. In an embodiment, the position of the sample and the light spot may be determined on the basis of a reference position associated with the sample or the sample holder. Hence, a reference position associated with the sample (e.g. on the sample) or the sample holder may be used by the scanning microscope in order to determine the position of the focused illumination light spot in the sample. This way, the positions and the orientation of the different illumination patterns that are written into the sample are accurately known and directly correspond to the pattern parameters that are used by the processor of the scanning microscope to write the illumination patterns into the sample. These pattern parameters are used by reconstruction algorithm in order to determine a high- resolution image. This obviates the need to retrieve

information of the illumination pattern a posteriori from the recorded images using image analysis techniques.

In an embodiment, exposing the sample to multiple illumination patterns may include moving the position of a focused illumination light spot of the scanning microscope in accordance with each of said multiple periodic patterns.

In an embodiment, a reference position associated with the sample may be used to define a coordinate system, which may be used by the processor to move the illumination light spot to predetermined positions relative to the

reference position. The relative position of the different illumination patterns with respect to the reference position are determined on the basis of a calibration of an optical system, wherein the calibration links settings of the optical system, such as respective positions of said scanning mirror, to positions of the illumination light spot on the sample.

Hence, a periodic pattern associated with certain pattern parameters may be defined as a set of coordinates on the sample, which the processor may translate into a set of mirror positions for moving the focused illumination light spot in accordance with the pattern.

In one embodiment, the method comprises the processor obtaining calibration data. The calibration data associate a first state of the optical system to a region of an image of a sample, e.g. a reference sample, captured by the imaging system. Herein, the region represents a part of the sample that is exposed to illumination when the optical system is in said first state. The region may comprise or consist of one or more image pixels. The reference sample may be the same sample of which a high-resolution image is formed in accordance with methods described herein.

It should be appreciated that states of the optical system may relate to respective positions and/or orientations of one or more movable elements of the optical system.

Examples of a movable element of an optical system are a scanning mirror, a rescanning mirror, and a sample holder, such as a sample stage. These moveable elements may be movable with respect to a surface supporting the microscope and their movement may be controllable.

Preferably, the region is discernable in the captured image. In one embodiment, obtaining the calibration data comprises identifying said region in the captured image.

Identifying the region may comprise identifying aberrant image pixel values, for example relatively bright image pixel values in the image, the aberrant image pixel values constituting said image region. In one embodiment, identifying said region in the captured image may comprise executing a phase retrieval algorithm known in the art.

It should be appreciated that the scanning microscope may be a scanning microscope in the sense that it may be configured to perform sample scanning. In such embodiment, the sample may be positioned in a sample holder that can be controlled to move with respect to a focused illumination light spot, wherein the focused illumination light spot is fixed relative to a surface supporting the scanning

microscope. In such embodiment, emission light may be collected by an imaging system such as an array detector or camera, and an image of the sample may be constructed based on the emission light that is captured while scanning the sample point by point by moving the sample with respect to the fixed illumination light spot. In such embodiment, exposing a sample to an illumination pattern thus comprises repeatedly

controlling the sample holder to move with respect to the fixed focused illumination light spot. Further, in such embodiment, a state of the optical system described above may relate to a position and/or orientation of the sample holder relative to the focused illumination light spot.

In one embodiment, the processor uses the image reconstruction algorithm for forming the high-resolution image of the sample on the basis of the calibration data. The calibration data may namely be valid for any image captured by the optical system irrespective of what kind of sample is imaged. To exemplify, the calibration data may associate the exact center of a calibration image with a first state of the system. Then, if the optical system captures a particular image of an actual sample and in this process adopts the first state, then, in accordance with the calibration data, the first state of the optical system is associated with the exact center of the particular image, which means that the exact center of the particular image represents a part of the actual sample, which part was exposed to illumination when the optical system was in the first state. Thus, the calibration data may be understood to define a reference position

associated with a sample based on which the exact position of an illumination pattern on the sample, and thus the initial phase of an illumination pattern on the sample, can be

determined without having to perform phase retrieval

algorithms. This enables to apply illumination patterns that are so fine that phase retrieval algorithms cannot, at least to a lesser extent, retrieve the initial phase from an image.

In one embodiment, the processor obtaining calibration data comprises the processor controlling the optical system to adopt a first state herewith controlling the scanning microscope to expose a first part of the reference sample to a focused illumination light spot. This embodiment further comprises the processor controlling the imaging system to capture a calibration image of the reference sample, the calibration image comprising a first region representing said exposed first part of the reference sample. This embodiment further comprises the processer storing the first state of the optical system in association with the first region of the calibration image.

In one embodiment, the calibration data associate a plurality of states of the optical system, e.g. a plurality of positions and/or orientations of a scanning mirror of the optical system, to respective regions of one or more images of the reference sample captured by the imaging system, wherein each particular region of said respective regions represents a part of the reference sample that is exposed to illumination when the optical system is in the state associated with the particular region.

In one embodiment, obtaining the calibration data comprises the processor controlling the optical system to adopt a second state, e.g. to control the at least one

scanning mirror to be in a second position and/or adopt a second orientation, herewith controlling the scanning

microscope to expose a second part of the reference sample to a focused illumination light spot. This embodiment comprises the processor controlling the imaging system to capture one or more calibration images, at least one calibration image of the one or more calibration images comprising the first region representing said exposed first part of the reference sample and at least one calibration image of the one or more

calibration images comprising a second region representing the second exposed part of the reference sample. This embodiment also comprises the processer storing the second state of the optical system in association with the second region. The optical system may be controlled to adopt the first and second state in the sense that the optical system is controlled to move the focused illumination light spot relative to the sample to expose the sample to a calibration illumination pattern. In one embodiment, a single calibration image comprises both the first and the second region described above .

In one embodiment, controlling the optical system to adopt the first state, and optionally to adopt the second state, comprises controlling an orientation and/or position of at least one movable element of the optical system, such as a scanning mirror and/or rescanning mirror and/or sample holder of the optical system. A movable element, such as a mirror or sample holder, in such an optical system may have a position and/or orientation that depends on a voltage that is applied to it, in particular that is applied to a control mechanism of the movable element, which control mechanism may comprise one or more actuators and/or electro motors. Thus, controlling an orientation and/or position of a movable element may comprise controlling a voltage that is applied to the movable element, in particular to a control mechanism of the movable element.

In an embodiment, the illumination light spot may comprise a predetermined light intensity pattern comprising at least two spatially arranged light intensity maxima. In an embodiment, the spatially arranged light intensity maxima may define a spatial frequency of the illumination pattern. In an embodiment, the at least two light intensity maxima may be arranged next to each other in a plane perpendicular to the optical axis of the microscope. In another embodiment, the at least two light intensity maxima may be arranged in an axial direction parallel to optical axis of the microscope.

In an embodiment, the predetermined light intensity pattern may comprise at least a 2D arrangement of two or more light intensity maxima; and/or, wherein the predetermined light intensity pattern comprises a 3D arrangement of multiple light intensity maxima.

In an embodiment, the illumination light spot may be a shaped illumination light spot, the shaped light spot comprising at least a first width and at least a second width small than the first width. In an embodiment, during writing, the second width may be arranged perpendicular to the

direction in which the focused illumination light spot is moving . In an embodiment, capturing multiple images may include controlling a rescanning mirror of the scanning microscope to reflect the emission light signal towards an imaging system and to move a focused emission light signal in accordance with the illumination pattern over the imaging plane of the imaging system. The optical response may comprise emission light that is emitted by fluorophores in the sample. The illumination light spot may excite the fluorophores and, as a result, the fluorophores may emit the emission light. The optical response may comprise a collimated emission light beam that is formed using an optical system, such as a lens system.

In an embodiment, controlling the scanning and the rescanning mirror may include rotating the scanning mirror and the rescanning mirror back and forth over a predetermined first angular amplitude and a second angular amplitude

respectively. In an embodiment, the ratio between the first and second angular amplitude may be unequal to one. In an embodiment, the second angular amplitude may be selected two times the first angular amplitude.

In an embodiment, forming a high-resolution image further includes: determining a Fourier transform of the different multiple illumination patterns on the basis of the one or more pattern parameters .

In an embodiment, the scanning microscope may comprise at least a pinhole for filtering out-of-focus light out of the illumination light signal. In an embodiment, the scanning microscope may comprise one or more optical elements configured to project an illumination light spot in a plane of interest of the sample onto a confocal conjugate plane of the plane of interest. The use of a pinhole may significantly increase the signal-to-noise ratio of the recorded image data. This is because the sample is exposed to a illumination pattern by scanning an illumination light spot over the sample. This way, a scanning structure illumination microscope may be realized that has confocal characteristics.

In an embodiment, the control information may be adapted to control one or more actuators for controlling a scanning mirror and/or rescanning mirror in order to move a focused illumination light spot in a lateral direction in the sample; and/or, to control one or more optical elements, preferably one or more lenses, in order to move the focused illumination light spot in an axial direction in the sample.

In an aspect, the invention may relate to a scanning microscopy system for forming a high-resolution image of a sample comprising: a computer adapted to control the scanning microscope, the computer comprising a computer readable storage medium having at least part of a program embodied therewith; and, a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the

computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising:

receiving or generating control information for controlling a scanning microscope, the control information defining a plurality of different periodic patterns on the basis of one or more pattern parameters, preferably the one or more pattern parameters, including at least one of: a spatial frequency, a periodicity direction, an initial phase; using the control information to control the scanning microscope to expose the sample to multiple illumination patterns, each exposure to an illumination pattern causing one or more optical excitations in the sample, the light originating from said optical

excitations forming an emission light signal; controlling an imaging system to capture multiple images, each image being associated with an emission light signal of one of the

multiple exposures, and, using an image reconstruction

algorithm for forming a high-resolution image on the basis of at least one of the one or more pattern parameters and the captured images.

In an embodiment, the executable operations may further comprise: controlling a rescanning mirror of the scanning microscope to reflect the emission light signal towards an imaging system and to move a focused emission light signal in accordance with the illumination pattern over the imaging plane of the imaging system; preferably controlling the scanning and the rescanning mirror includes rotating the scanning mirror and the rescanning mirror back and forth over a predetermined first angular amplitude and a second angular amplitude respectively; more preferably the ratio between the first and second angular amplitude unequal to one; even more preferably, the second angular amplitude being selected two times the first angular amplitude.

In a further aspect, the invention may relate to a control module for controlling a scanning microscopy system comprising: a computer adapted to control the scanning

microscope, the computer comprising a computer readable storage medium having at least part of a program embodied therewith; and, a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the

computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising:

receiving or generating control information for controlling a scanning microscope, the control information defining a plurality of different periodic patterns on the basis of one or more pattern parameters, preferably the one or more pattern parameters, including at least one of: a spatial frequency, a periodicity direction, an initial phase; using the control information to control the scanning microscope to expose the sample to multiple illumination patterns, each exposure to an illumination pattern causing one or more optical excitations in the sample, the light originating from said optical

excitations forming an emission light signal; controlling an imaging system to capture multiple images, each image being associated with an emission light signal of one of the

multiple exposures, and, using an image reconstruction

algorithm for forming a high-resolution image on the basis of at least one of the one or more pattern parameters and the captured images.

The invention may also relate to a computer program product comprising software code portions configured for, when run in the memory of a computer, executing the method steps according to any of process steps described above.

The invention will be further illustrated with reference to the attached drawings, which schematically will show embodiments according to the invention. It will be understood that the invention is not in any way restricted to these specific embodiments.

Brief description of the drawings

Fig. 1 depicts a scanning microscopy system according to an embodiment of the invention.

Fig. 2A-2C depict a process of illuminating a sample with an illumination pattern using a scanning microscopy system according to an embodiment of the invention.

Fig. 3 depicts illumination patterns for use with a scanning microscopy system according to an embodiment of the invention .

Fig. 4 depicts the reconstruction of an image using a structured illumination microscopy technique.

Fig. 5A-5C depicts various illumination light spots for use in a scanning microscopy system according an

embodiment of the invention.

Fig. 6 depicts a scanning microscopy system according to another embodiment of the invention.

Fig. 7A depicts a calibration of the scanning

microscopy system, in particular of the rescanning mirror with respect to an imaging system.

Fig. 7B illustrates the principle that the initial phase of a structured illumination light pattern can be determined, even if the spatial frequency of the pattern is so high that it does not have a clear footprint in a captured image . Detailed description

Fig. 1 depicts a scanning microscopy system according to an embodiment of the invention. The system may comprise a light source 100 for generating illumination light that is directed onto a sample 112 located on a sample holder (not shown) . Illumination light 122 may be directed onto a sample using a first optical system 130, which may include one or more refractive and/or reflective optical elements. For example, in an embodiment, the first optical system may include one or more collimating and/or focusing lenses

102,110, mirrors 104, a set of orthogonal scanning mirrors, schematically depicted by mirror 108 and/or dichroic mirrors 106.

In particular, a lens 102 may be used in order to form a collimated beam of illumination light 122 which may be reflected via a set of mirrors 104,106,108 towards a lens 110 in order to form a focused illumination light spot 126 on the sample. The focused illumination light spot may excite local optical excitations, which generate emission light 124. The emission light may pass through lens 110 and may be directed via a second optical system 132 comprising one or more

refractive and/or reflective optical elements to an imaging system 118.

The second optical system may include a lens 110 for forming a collimated beam of the emission light that is reflected by a first scanning mirror 108. The beam of emission light may pass one or more optical elements, e.g. dichroic mirror 106, and reflected by a further second scanning mirror 114, which directs the emission light beams to a focussing lens 116. The second scanning mirror may also be referred to as a rescanning mirror. In an embodiment, the first and/or second scanning mirror may be implemented by a set of

orthogonal scanning mirrors.

The focussing lens may focus the emission light beam into focussed emission light spot 128 onto the imaging plane of imaging system 118. Here, the imaging system may include one or more image sensors, e.g. one or more CMOS image sensors or one or more CCDs. Dichroic mirror 106 may be configured such that it functions as a reflector for illumination light 122 and such that it is transparent for emission light 124 (which typically is of a longer wavelength than illumination light 122) .

The first scanning mirror 108 which reflects the illumination light towards the sample and the emission light towards the further the second scanning mirror 114 may be configured as a rotatable mirror which may be controlled by a computer system 120. The computer system may control the actuators of the scanning mirror so that the light spot can be moved in a 2D plane of the sample. Further, the computer system may control the optics of the system in in order to set a focussing height within the sample. By controlling the focus height different 2D planes in the sample at different focus heights can be illuminated with light patterns. This way, the focused illumination light spot 126 can be moved through the sample 112 in accordance with a predetermined 2D or 3D

pattern .

For example, when writing a 2D periodic line pattern in the sample, the scanning mirror 108 may be configured to rotate back and forth over a first angular amplitude Al . This way, the mirror can move ("scan") the focused illumination light spot through sample causing continuous local optical illumination in the scanning direction of the light spot.

Controlling the mirror in three dimensions and controlling the focus of the light spot thus allows writing a 2D or 3D light structure in the sample.

The computer may comprise one or more processor connected to a memory comprising computer code that, when executed by the one or more processor, may transform a

predetermined pattern into control information for the optics (the mirrors and the lenses) so that light pattern can be written in the sample.

In an embodiment, the pattern may be defined by a mathematical function including a number of parameters (e.g. frequency, initial phase, amplitude, modulation depth, etc.) which can be easily transformed into control information. By executing the computer code, the control information may control the actuators and optical elements of the system in order to write the pattern into the sample. In an embodiment, the optical system comprises a feedback system for controlling a position and/or orientation of a mirror, such as the scanning mirror and rescanning mirror. The feedback system comprises a device for measuring the position and/or orientation of the mirror and outputs a signal indicative of the measured position and/or orientation towards the computer system. The computer system may be configured to, based on the received signal, adjust control signals that are used to control the mirror. In an example, the mirror is to be controlled such that it makes an angle of 45 degrees to a reference plane. If the device for measuring the orientation then outputs a signal that indicates that the mirror makes an angle of 44 degrees with respect to the reference plane, the computer system may adjust its control signals, e.g. increase or decrease an applied voltage, in order to bring the mirror in the desired position. Of course, the device for measuring the orientation and/or position of the mirror repeatedly, preferably continuously, measures the orientation and/or position and repeatedly, preferably continuously, outputs the signal towards the computer system in order to allow accurate control of the orientation and/or position of the mirror.

In an embodiment, the one or more processors connected to the memory comprising the computer code may be integrated in a structured light microscopy (SIM) module 132 that is configured to transform a predetermined 2D or 3D pattern in to control information associated with a desired illumination pattern that is used in the SIM process.

Illumination patterns can be designed in advance using e.g. a software application that may include a drawing program for designing a desired light pattern, wherein the defined pattern represents an exact copy of the illumination light pattern that is written into the sample.

During scanning, emission light originating from the moving illumination light spot is formed into a collimated beam 124 of emission light. The collimated beam may be reflected via the scanning mirror 108 towards the second scanning mirror 114, a rotatably mounted scanning mirror, which is controlled to reflect the light beam towards the focusing lens 116 in order form a moving emission light spot 128 onto the imaging plane of the imaging system 118.

The second scanning mirror 114 may be configured to rotate back and forth over a second angular amplitude A2 in order to move (scan) the focused emission light spot over the imaging plane of the image sensor, while the focussed

illumination light spot is moved (scanned) over the sample by the scanning mirror. The actuators and/or electro-motors of the scanning and rescanning mirror may be controlled by a computer system 120 so that the mirrors can be moved

synchronously. Here, the frequency of the back and forth rotation of the scanning mirror is identical to the frequency of the back and forth rotation of the rescanning mirror.

In an embodiment, computer system may be configured as a central computer for centrally controlling the actuators and the optics. In another embodiment, the computer system may be a distributed system wherein different processor may control different parts of the system. For example, in an embodiment, each scanning mirror may be controlled by a separate processor in order to provide fast low delay control of the scanning mirrors.

Thus, when the scanning mirror is controlled by the computer system to scan an area of the sample by the moving illumination light spot, the rescanning mirror is controlled by the computer system to scan associated areas of pixels of the imaging plane of the image sensor so that the pixels are exposed by moving emission lights spots. When a sample is optically excited by the illumination beam that moves in accordance with a illumination pattern over the sample, emission light originating from the excited areas is imaged onto the imagining plane.

The structured light microscopy (SIM) module 132 may control the scanning and rescanning mirror in order to expose the sample multiple times, each time using a different

illumination pattern, typically a periodic illumination pattern, so that during an exposure some parts of the samples are not exposed to the illumination light and other parts are. Examples of illumination patterns are described hereunder in more detail with reference to Fig. 2 and Fig. 3.

For each of these multiple exposures, the spatial position of said illumination pattern is spatially shifted and/or rotated with respect to a reference position associated with the sample. The exposure of the sample includes

controlling a scanning mirror of a first optical system to move (scan) at least one focused illumination light signal through the sample in accordance with the illumination

pattern, wherein the focused illumination light signal causes one or more optical excitations in the sample, the light originating from said optical excitations forming an emission light signal.

During the exposure of the sample, multiple images are captured, wherein each image is associated with one of the multiple exposures. Here, during the capturing of an image the rescanning mirror is controlled to reflect the emission light signal towards the imaging system and to move (scan) a focused emission light signal in accordance with the illumination pattern over the imaging plane of the imaging system.

Thereafter, the SIM module may construct a high resolution image on the basis of the multiple captured images.

In an embodiment, the second angular amplitude A2 of the rescanning mirror 114 may be larger than the first angular amplitude Al of the scanning mirror 108 so that high- resolution images of the scanned sample area can be obtained. For example, in an embodiment, the second angular amplitude may be twice the first angular amplitude, so that the

reconstruction of the image can be improved when compared to the case when the second angular amplitude is chosen to be equal to the first angular amplitude. The ratio between the second and first angular amplitude may be in the range 1-5, particularly 2-5, more particularly 2-4, even more

particularly 2-3. The light source 100 may comprise one or more light sources, e.g. one or more lasers, and one or more light filters. In an embodiment, the light source may be connected to the computer system. The one or more light sources and filters may be used to control the wavelengths or bands of wavelengths the illumination light comprises. In an embodiment, the light source may be configured to generate white light. In another embodiment, the light source may be configured to generate light consists of two or more

predetermined wavelengths or bands of wavelength. For example, in an embodiment, the light source may generate light of a first wavelength selected from the blue band of the visible spectrum and light of a second wavelength selected from the yellow band of the visible spectrum.

The optical elements of optical system 132 may be configured to control different wavelengths. For example, mirror 104 may reflect the illumination light onto dichroic mirror 106 that may be configured to reflect light of a first group of wavelengths, e.g. blue light and yellow light.

Dichroic mirror 106 may be further configured to pass light of a second group of wavelengths, e.g. red and green light. This way, the illumination light may be reflected by the dichroic mirror onto the scanning mirror.

In an embodiment, the illumination light may comprise light of a first wavelength, e.g. yellow light, which may cause a first type of optical excitations in the sample, the first type of excitations generating first emission light.

Further, the illumination light may comprise light of a second wavelength, e.g. blue light, which may cause a second type of optical excitations in the sample, the second type of

excitations generating second emission light. Thus the above- mentioned first group of wavelengths, which may be reflected by dichroic mirror 106, may comprise said first wavelength and said second wavelength.

In an embodiment, sample 122 may be a material, e.g. a biological material, that is imaged using a reflective microscopy technique. In an embodiment, sample 112 may be an optically active sample. For example, in an embodiment, the sample may be a material, e.g. a biological material,

comprising one or more types of fluorescent/ luminescent materials, such as fluorophores . In another embodiments, the optically active sample may be non-luminescent optical active material. For example, the sample may be a material that can be imaged on the basis of second harmonics generation (SHG) microscopy or third harmonics generation (THG) microscopy. In yet a further embodiment, the sample may be a material that can be imaged using a Rahman microscopy technique, such as Coherent Anti-Stokes Raman Scattering (CARS) microscopy.

For example, the sample comprise at least a first type of fluorophores , e.g. red fluorophores , and a second type of fluorophores, e.g. green fluorophores . The light of the first wavelength of the illumination light may cause the first fluorophores in the sample to emit the first emission light and the light of the second wavelength may cause the second fluorophores in the sample to emit second emission light.

Hence, the emission light may comprise light that is emitted as a result of photoluminescence, preferably

fluorescence. In an embodiment, a blue light component of the illumination light may cause green fluorophores at the focused illumination light spot to emit green light. Similarly, yellow light component of the illumination light may cause red fluorophores at the focused illumination light spot to emit red light. In this example, the emission light thus comprises green light and red light.

Hence, from the above it follows that different photo-luminescent sites which are sensitive to light of different wavelengths and which - in response - emit light of different wavelengths, may be used. This way, the emission light originating from the sample may include optical

information associated with different excitation sites of the sample. In an embodiment, filters such as color filters (not shown) may be positioned in front of the imaging system. A color filter may be configured as a bandpass filter that is configured to pass light in a predetermined band of

wavelengths .

The scanning mirror both directs the illumination light signal from the light source 100 towards the sample 112 as well as the emission light signal from the sample 112

towards the rescanning mirror 114. This way, the movement of the emission light signal originating from the sample is neutralized by the movement of the scanning mirror. Due to the movement of the scanning mirror, the path of emission light from scanning mirror to rescanning mirror is static. In other words, the scanning mirror 108 "descans" the emission light 124 and reflects the emission light as a static emission light beam towards the rescanning mirror. Further, the path of the emission light between rescanning mirror and light detector may be dynamic during scanning due to the movement of the rescanning mirror.

Fig. 2A-2C depict a process of illuminating a sample with an illumination pattern using a scanning microscopy system according to an embodiment of the invention. In

particular, Fig. 2A depicts a top view of a sample 206 wherein a position in the sample may be defined on the basis of a coordinate system, e.g. a 3D Cartesian coordinate system x,y,z. The origin of the coordinate system in the sample may be calibrated with respect to a reference position 202 i so that after calibration the coordinate system may be used to

accurately position the light spot in the sample, which may have lateral dimensions in the x, y plane and an axial

dimension in the z-direction. The reference position may be a position on the sample or associated with the sample, e.g. a position on the sample (holder), e.g. an (optical) marker that is used by the system to position and align the sample within respect to the optical system, in particular the optical axis of the optical system, of the scanning microscope.

Fig. 2A further depicts part of an illumination pattern 204 i , in this example a periodic arrangement of parallel lines, which is formed by writing the illumination pattern into the sample. To this end, the computer system, may generate or receive control information for controlling the scanning microscope, wherein the control information defines a plurality of different periodic patterns, e.g. a plurality of different stripe patterns, each being defined using one or more pattern parameters, e.g. the number of stripes, the length of the stripes, the spatial frequency, the direction of the periodic tripe pattern (periodicity direction) , an initial phase of the pattern, an illumination light intensity, etc. The computer system may use the control information to control the scanning mirror for writing the multiple illumination patterns in the sample. Each exposure to an illumination pattern causes one or more optical excitations in the sample and an emission light signal, formed on the basis of the light originating from said optical excitations, may be captured by the imaging system.

When looking at a cross-section 200 of the sample along a direction perpendicular to the scanning direction (in this case parallel to the x-direction) , a periodic intensity function identifying high and low illuminated areas in the x- direction of the sample can be identified, wherein the spatial period of the function is identified by a parameter d.

Due to the fact that a focused illumination spot 208i is shaped according to a certain point spread function (PSF) , the intensity pattern in a cross section 200 of the sample in the direction of the periodicity may be described as a sine ^¬ like function rather than a sguare wave function with sharp edges. Hence, the periodic intensity function of the first exposure in Fig. 2A may be expressed as a sinusoid function 201i and an initial phase cpo :

(2πχ

/(r) = / ₀ ^sin hr

wherein I ₀ is the averaged intensity, A is measure of the modulation depth, 1/d the spatial frequency and cpo an (initial) phase of the intensity function. The initial phase cpo is directly related to the position of the illumination structure in the sample as is illustrated in Fig. 2A.

The point spread function may cause a certain

blurring so that when the spatial period is decreased, the modulation depth of the periodic intensity function will become smaller. The decrease of the modulation depth with increasing spatial frequency is a general problem with

conventional SIM that typically uses a grating in order to expose a sample with a periodic illumination structure. Due to the decrease modulation depth, it is no longer possible to determine pattern parameters such as the initial phase and the frequency on the basis of the captured images.

This problem is solved by the methods and systems described in this application. The scanning microscope

according to the invention may accurately control the position of a focused illumination light spot in the sample, a periodic 2D or 3D illumination pattern can be accurately written into the sample using the control information.

Pattern parameters, including the initial phase and the spatial frequency, defining the different periodic

patterns - which are determined before exposing the sample - may be used in the reconstruction algorithm. The invention thus eliminates or at least substantially reduces the need for a posteriori image analysis in order to determine illumination light pattern parameters that are used by the reconstruction algorithm. This way, light patterns may be used that have a higher spatial frequency than the light patterns that are used in SIM systems known in the prior art.

Once a first illumination pattern is written into the sample and a corresponding image of the illuminated sample is captured by the imaging system and stored in a memory storage, the computer system may repeat the process of exposure and image capturing several times wherein each time, the

illumination pattern is shifted and/or rotated with respect to the reference position. As shown in Fig. 2B and 2C, shifted versions of the periodic stripe structure may be written into the sample. The shifted illumination patterns of Fig. 2B and 2C may be expressed by similar sine functions 201.2,3 having initial phases (pi and (p2 respectively. Hence, the spatial shift may be expressed in terms of a spatial phase shift of the periodic intensity function in the x-direction.

Fig. 3 depicts different illumination patterns for use in an image reconstruction algorithm. As shown in Fig. 3, first the sample may be exposed to a first set of light patterns 306i-3, e.g. a set of shifted periodic tripe patterns wherein the stripes may have a predetermined orientation in the 2D plane, e.g. aligned to the y-axis as e.g. described with reference to Fig. 2A-2C. Thereafter, the computer system may repeat the exposure of the sample using a second set of light patterns 304i-3 wherein the second set is a rotated version (in this example 45 degrees) of the first set.

Similar, the process may be repeated using a third set 306i-3 which is a rotated version of the first or second set) . Each pattern in the sets of patterns is defined by pattern

parameters such as an initial phase which may be directly used in the reconstruction algorithm.

Although the examples in this disclosure are described with reference to a periodic stripe pattern, many other illumination patterns may be used. In an embodiment, the illumination pattern may have a periodic 2D pattern that has a periodicity in one direction (e.g. such as the pattern

depicted in Fig. 2A) . In another embodiment, the illumination pattern may be a periodic 2D pattern that has a periodicity in two directions, e.g. an array of blocks having a periodic in the x and y direction. In yet a further embodiment, the illumination pattern may be a 3D periodic pattern. The sample may be exposed at different focus depths in the (axial) z- direction. Hence, in that case, the illuminated areas are not only separated in the x,y plane shown, but also separated in the axial direction.

Fig. 4 depicts a process of reconstructing an image on the basis of images obtained by structured illumination microscopy according to an embodiment of the invention. In an embodiment, the reconstruction process may be executed by computer system of the scanning microscopy system as for example descripted with reference to Fig. 1.

As described above, the illumination pattern may be formed by a periodic illumination pattern which may be

represented by a sinusoid, which in Fourier space may be represented by three delta functions 402 as shown in Fig.

The Fourier transform may be expressed as follows:

( ^k+ i)- ^e *"{ ^k -i) including a first delta function at frequency k=-l/d, a second delta function at frequency k=0 with an intensity Io and a third delta function at frequency k=+l/d.

The optical response of the sample to the structured illumination is a product of the applied light intensity pattern, I (r) , and the fluorophore distribution, S (r) . Hence, the optical response in reciprocal space is a convolution of the Fourier transform of the illumination pattern 402 and the Fourier transform of the (unknown) fluorophore structure S(k) 404 , which represents an image that needs to be reconstructed. The optical response 406 can be represented by:

OR(k) = /(fe) ® 5(/c) = y 5(fc) -S [k

The microscope, which may comprise one or more lenses, that may be used to capture image data of the optical response may be characterized by an Optical Transfer Function OTF(k) 408 . The OTF shows that the microscope system

effectively acts as a low pass filter, attenuating high frequencies. The dashed vertical lines indicate a cut-off frequency. Frequencies higher than this cut-off frequency will not pass through the example microscope system.

The Fourier transform of the captured images D(k) 410 is a product (in reciprocal space) of the optical response OR(k) 406 and the OTF(k) 408 . This relation may be expressed by the following formula:

D(k) = OR(k) * OTF(k) = y S(k)-S[k OTF(k) wherein the system does not capture image data associated with frequencies higher than the cut-off frequency of the

microscope system.

The Fourier transform of the illumination pattern is known. Further, the frequency 1/d and initial phase cpo are pattern parameters which are determined prior to the exposure and can be used directly in the computation. This way, it is mathematically possible to compute the unknown Fourier transform of the fluorophore structure S(k) for a range of frequencies including frequencies

exceeding the cut-off frequencies of the OTF of the microscope system. This process may be executed on the basis of a known image reconstruction algorithm 412. The computed Fourier transform 414 of the fluorophore structure comprises

frequencies that are higher than the cut-off frequency of the OTF of the microscope. This computed Fourier transform may be used to reconstruct a high resolution image.

In the above described example, the reconstructed image comprises enhanced resolution in only one direction, namely the direction in which the periodicity exists. In order to obtain enhance resolution in other directions as well, the above described method has to be repeated using illumination patterns having different periodicity directions, i.e.

patterns that are periodic in different directions (as e.g. shown in Fig. 3) .

Hence, in the image reconstruction process described with reference to Fig. 4, pattern parameters such as the initial phase and the spatial frequencies defining the

patterns that are used for exposing the sample are used in the reconstruction process. As the scanning microscope is capable of accurately controlling the position of the illumination light spot in the sample, a high resolution image can be reconstructed .

Fig. 5A-5C depicts various illumination light spots for use in a scanning microscopy system according an

embodiment of the invention. In particular, FIG. 5A-5C

illustrates differently shaped illumination light spots, i.e. illumination light spots with different light intensity profiles, that can be used to illuminate areas on the sample, i.e. write the periodic illumination pattern onto or into the sample. In this figure, the darker areas are associated with high light intensities and the lighter areas with low light intensities. The illumination light spots move in the

direction of the arrows, i.e. in the writing direction. The dashed lines indicate one or more areas of the plurality of areas, i.e. the illuminated areas on the sample, after the illumination light spot has moved over the sample in the direction as specified by the arrow.

FIG. 5A shows an illumination light spot that has a simple Gaussian shaped light intensity profile. FIG. 5B shows an illumination light spot that comprises a predetermined light intensity pattern. In this embodiment, the illumination source of the scanning microscopy system and the optics associated with the illumination source may be adapted to generate an illumination light spot 5042 comprising at least two intensity maxima wherein d defines the distance between the maxima. When using such illumination light spot, at least two areas on the sample may be illuminated simultaneously thus proving an efficient way of writing multiple parts of the periodic illumination structure simultaneously.

Alternatively and/or additionally, in an embodiment, the illumination light spot may be shaped, e.g. oval instead of circular so that the width of the spot in the scanning direction is larger than the width of the spot in the

direction perpendicular to the scanning direction. Minimizing the width of the illumination spot perpendicular to the scanning direction (writing direction) allows writing of periodic illumination structures that have higher spatial frequency which is beneficial for achieving higher

resolutions.

In case the illumination light spot is shaped into a predetermined shape, e.g. an oval shape as shown in Fig. 5B, the orientation of the shaped light spot has to be changed when changing the scanning direction so that the width of the spot perpendicular to the scanning direction is always the smallest width. This way, a fine resolution of the

illumination pattern is ensured in every scanning direction as shown in Fig. 5B.

Fig. 5C depicts a further embodiment wherein the illumination light spot comprises a plurality of maxima so that a plurality of lines are generated when moving the illumination light sport in predetermined directions. For example, the maxima may be arranged in a square configuration, so that when moving the light sport in a first direction a plurality of lines are simulations written onto the sample and when moving the light sport in a second direction a plurality of lines are simultaneously written onto the sample.

In the example of Fig. 5C, the plurality of maxima may be arranged in a configuration having three axis of rotation symmetry, i.e. the x-axis, y-axis and an 45 degree axis. When the scanning direction of the illumination light spot is parallel to one of the directions of axis of symmetry, the same striped pattern can be generated.

The shape of the illumination light spot and the number of maxima in the illumination light spot may be

controlled using an element that can shape the phase-profile and/or intensity profile of the wave front of the excitation laser beam, such as a deformable mirror of a spatial light modulator. Such optical element is controlled by a computer with some control software.

Fig. 6 depicts a scanning microscopy system according to another embodiment of the invention. In particular, Fig. 6 depicts a scanning microscopy system similar to the system described with reference to Fig. 1, comprising a light source, a sample 612, an imaging system 618 and optical elements including a mirror 604, dichroic mirror 606, scanning mirror 608, focusing lens 610, rescanning mirror 614 and focusing lens 616.

In contrast with the system of Fig. 1, the system in Fig. 6 comprises an optical arrangement 621 for enabling confocal measurements wherein the optical arrangement may comprise a pinhole 623. The scanning microscopy may be

configured to focus illumination light spot 613 in a plane of interest P. Additionally, the system may be configured to arranged to focus emission light from the plane of interest P onto a plane 623 P' . For example, the system may comprise one or more optical elements, e.g. lenses 620 and 622, arranged to focus emission light from the plane of interest P onto plane P' . Plane P' thus may be a confocal conjugate plane of plane of interest P. In a further embodiment, the scanning microscopy system may comprise a point spread function (PSF) module 603 that is configured to modify the illumination light spot by modifying the shape of the illumination light spot and/or introducing a predetermined arrangement of light intensity maxima in the illumination light spot as e.g. described with reference to Fig. 5A-5C.

Fig. 7A illustrates an exemplary calibration of the optical system. The calibration comprises exposing a reference sample 754a to a reference light pattern 704a and capturing an image 758a, for example using methods described above.

The dashed outline 750 indicates a first state of the optical system. When the optical system is in the first state, the rescanning mirror 714 is oriented at a first angle with respect to a reference plane, e.g. the imaging plane of the imaging system, and the scanning mirror 708 is oriented at a second angle with respect to the reference plane. When the optical system is in the second state 752, at least one of the scanning mirror and rescanning mirror is oriented differently with respect to the orientations in the first state. In this example, the rescanning mirror 714 is oriented at a third angle with respect to the reference plane that is different from the first angle and the scanning mirror 708 is positioned at a fourth angle with respect to the reference plane that is different from the second angle.

When the optical system is in the first state 750, a focused illumination light spot is incident on the sample 754a at position 761. When the optical system in in the second state 752, the focused illumination light spot is incident on the sample at position 762.

The reference sample 754a in this example has a substantially homogeneous fluorescent layer, preferably of thickness less than 200 nm, more preferably less than 100 nm, which means that a focused illumination light spot positioned anywhere on the reference sample will cause optical

excitations which will cause light emission from the reference sample at the position of the focused illumination light spot. In a rescanning microscopy system, a focused emission light spot may be formed on the basis of this light emission, which light spot may be scanned over an imaging plane 756 as

described above.

Reference numeral 756 indicates an imaging plane of an imaging system, for example of a CCD camera. In this example, the imaging plane 756 comprises a plurality of pixels two of which are indicated by 775a and 775b respectively.

During the scanning of the focused illumination light spot, a focused emission light spot is scanned over the imaging plane 756 in accordance with the pattern indicated by 760a. When the illumination light spot is at position 761 on the sample 754a, the emission light spot on the imaging plane 756 is at

position 763. Furthermore, when the illumination light spot is at position 762 on the sample 754a, the emission light spot on the imaging plane 756 is at position 764. Thus, the focused emission light spot is incident on positions 762 and 764 on the imaging plane 756 when the optical system is in the first state 750 respectively second state 752.

Image 758a is the image captured by the imaging system during the calibration procedure as a result of the exposure of reference sample 754a to the reference

illumination light pattern 704a. It should be understood that the pixels 755 of the imaging plane 756 may be one-to-one associated with image pixels of image 758a. Each image pixel may thus indicate how many photons were received by its associated pixel 755 of the imaging plane 756 during a certain time period. The three vertical paths 770 indicate bright regions in the image 758a as a result of exposing the

reference sample 756a to the reference illumination light pattern 704a and scanning the emission light spot over the imaging plane 756 in accordance with pattern 760a.

The bright regions 770 comprise a region 766 that represents illuminated part 761 of the sample 754a and

comprises a region 768 that represents illuminated part 762 of the sample 754a.

The calibration method may then comprise storing calibration information by storing the first state of the optical system, for example the respective angles that the scanning and/or rescanning mirrors make with respect to a reference plane, in association with region 766 of image 758a and/or in association with position 763 on the imaging plane 756 and/or in association with one or more pixels at position 763 of the imaging plane. Similarly, the calibration method may comprise storing calibration information by storing the second state of the optical system, for example the respective angles that the scanning and/or rescanning mirror make with a reference plane, in association with region 768 of image 758a and/or in association with position 764 on the imaging plane 756 and/or in association with one or more pixels at position 764 of the imaging plane.

Storing a state may be performed by storing control information for bringing the optical system in that state. In an example, storing a state may comprise storing a voltage that is to be applied to actuators of a scanning and/or rescanning mirror for causing the scanning and/or rescanning mirror to orient such that the optical system adopts the state .

The emission light pattern 770 is discernible in image 758a. Based on image 758a alone, the exact regions in the image can be determined that represent parts of the sample that have been exposed to the illumination pattern 704a. Phase retrieval algorithms known in the art may be executed for finding the exact position of the pattern 770 in the image 758a.

A state of the optical system may thus, in one embodiment, be defined by the respective orientations of the scanning and rescanning mirror. Preferably, when obtaining the calibration data, in particular when controlling the optical system to adopt the first state, the first state is not arbitrarily chosen, but preferably is predetermined based on pre-calibrations of the optical system.

To illustrate, the scanning mirror is preferably oriented such that the reference illumination pattern 704a, for which the optical system may have to adopt the first and second state, falls properly onto the reference sample 754a. To this end, the scanning mirror may have to be pre-calibrated with respect to a sample holder according to methods known in the art. In one example, pre-calibrating the scanning mirror with respect to the sample holder may comprise imaging a test sample positioned in the sample holder, wherein the test sample comprises a known fluorescent structure, such as fluorescent lines that are spaced 1 micrometer apart over a length of 1 mm.

Another pre-calibration may relate to calibrating the scanning mirror and rescanning mirror with respect to each other, which may be performed as follows. First, a sample comprising a number of fluorescent beads is positioned in the microscope, e.g. in or on a sample holder. Then, while the excitation laser of the microscope is shut off and any pinhole is removed, an external light source illuminates the entire sample and thus excites all beads. The fluorescent emission light of the beads travels from scanning mirror to rescanning mirror and is incident on the imaging system and an image of the beads appears on a display connected to the imaging system. The scanning and/or rescanning mirror may be moved, e.g. rotated, such that the emission light is properly

incident on the imaging system such that the image on the display properly shows the beads. Then, the scanning mirror is moved, e.g. periodically rotated over a first amplitude, and an attempt is made to move the rescanning mirror as well in such manner that the light bundle incident on the imaging system does not move. The beads should then appear steady on the display. The movement of the rescanning mirror then compensates for the movement of the scanning mirror. As such, each possible orientation of the scanning mirror may be associated one to one with an orientation of the rescanning mirror .

In one example, a first angular amplitude of rotation of the scanning mirror is compensated by a second angular amplitude of rotation of the rescanning mirror. The first and second angular amplitude need not be the same depending on the optics between the scanning and rescanning mirror. After the first and second amplitude have been found, the second

amplitude may be doubled when using the microscope to capture an image of a sample, because this may enable to enhance the resolution of the captured image as described in Biomed Opt. Express; 2013 Nov 1; 4(11) : 2644-2656.

FIG. 7B illustrates capturing one of the multiple images during an embodiment of the method for forming a high resolution image of a sample 754b. The sample 754b comprises coarse structures 772, such as structure 772a and 772b.

Furthermore, the sample 754b comprises fine structures 774, such as 774a and 774b.

The embodiment comprises a processor controlling a scanning microscope to expose the sample 754b to illumination pattern 704b. The illumination pattern 704b in this example is finer than the reference illumination pattern 704a as shown.

Note that exposing the sample to illumination pattern 704b comprises moving a focused illumination light spot over the sample 754b in accordance with the illumination pattern 704b. While exposing the sample to the illumination pattern 704b, the optical system may be understood to be controlled to adopt the first state 750 and optionally the second state 752, in this example identical to the first and second state mentioned with reference to Fig. 7A, so that the focused illumination light spot is incident on the sample 754b at position 775. The focused emission light spot may be

understood to scan over the imaging plane 756 in accordance with pattern 760b.

The image 758b of the sample 754b shows the coarse structures 772 in the sample 754b. Image 758b however does not show the fine structures 774 of the sample, because these fine structures are associated with high frequencies that are attenuated too much by the optical transfer function of the optical system as schematically illustrated in Fig. 4,

reference numeral 408. In image 758b, the applied illumination pattern, or formulated differently, the regions 784

representing parts of the sample 754b that have been exposed to illumination, are indicated for reference and are not actually present in the image 758b and cannot be distinguished in the image 758b as will be explained below. In this example, the illumination pattern 704b is so fine, that the image 758b does not have a clear enough

footprint of the pattern. The intensity minima and maxima of the illumination pattern 704b cannot be distinguished in the image 758b, which for example prevents a phase retrieval algorithm known in the art to determine the exact regions in the image that represent parts of a sample exposed to

illumination, and thus to retrieve the initial phase of the illumination pattern on the sample 754b. In conventional systems using for example a grating to expose a sample to an illumination pattern, such a fine illumination pattern 704b cannot be used, because there is no way to retrieve the initial phase of the illumination pattern 704b in the sense that the exact regions in the image 758b representing parts of the sample that are exposed to illumination light cannot be determined based on the image 758b.

However, because during exposing the sample 754b the optical system has adopted the first state 750, and because the processor has stored the region 766 in association with the first state 750, the processor can determine that region 766 indicated in image 758b represents a part of the sample 754b that was exposed to illumination. Thus, the exact

position, and thus initial phase, of the illumination pattern 704b on sample 754b can be determined. Similarly, region 768 in image 758b may likewise be determined to represent a part of the sample that is exposed to illumination light based on the second state 752 being associated with region 768 and the optical system having adopted the second state while exposing the sample 754b to illumination pattern 704b.

In an example, the method comprises determining a region 780 in an image as representing a part of the sample exposed to illumination on the basis of the calibration information. The region 780 may be associated with a further state of the optical system. It is thus not required that the optical system adopts an identical state during exposing the sample to illumination pattern 704b and during the calibration procedure described with reference to FIG. 7A. Based on the calibration data further links can be made between states of the optical system and respective regions in captured images.

It should be appreciated that the method further comprises capturing further images associated with respective further illumination patterns to which the sample 754b is exposed which allows to form a high-resolution image on the basis of the obtained images using an image reconstruction algorithm. Input to the image reconstruction algorithm is for example that region 766 in image 758b represents part of the sample that was exposed to illumination, which allows

determining of the initial phase. It will be understood that a reconstructed image may indeed show the fine structures 774.

Further, in case the sample 754b comprises sufficient coarse structures 772, sample 754b may be used as reference samp1e 754a.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a, " "an, " and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and

described in order to best explain the principles of the invention and the practical application, and to enable other of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.