RELATED APPLICATION'S)

This application claims the benefit of United States Provisional Patent Application Serial No. 61/983,926, filed on April 24, 2014, the entire contents of which are incorporated herein by reference.

BACKGROUND

Until about a decade ago, resolution in far- field light microscopy was thought to be limited to 200-250 nanometers in the focal plane, concealing details of sub-cellular structures and constraining its biological applications. Breaking this diffraction barrier by the seminal concept of stimulated emission depletion ("STED") microscopy has made it possible to image biological systems at the nanoscale with light. STED microscopy and other members of reversible saturable optical fluorescence transitions ("RESOLFT") family achieve a resolution greater than 10- fold beyond the diffraction barrier by engineering the microscope's point-spread function ("PSF") through optically saturable transitions of the (fluorescent) probe molecules.

However, slow progress in 3D super-resolution imaging has limited the application of previously available techniques to two-dimensional ("2D") imaging. The best 3D resolution until recently had been 100 nanometers axially at conventional lateral resolution. 4Pi microscopy achieved this through combination of two objective lenses of high numerical aperture, in an interferometric system. 4Pi microscopy was only recently shown to be suitable for biological imaging. Only lately the first 3D STED microscopy images have been published exceeding this resolution moderately with 139 nanometer lateral and 170 nanometer axial resolutions. While this represents a 10-fold smaller resolvable volume than provided by conventional microscopy, it is still at least 10-fold larger than a large number of sub-cellular components, for example synaptic vesicles.

Current understanding of fundamental biological processes on the nanoscale (e.g., neural network formation, chromatin organization) is limited because these processes cannot be visualized at the necessary sub-millisecond time resolution. Current biological research at the sub-cellular level is constrained by the limits of spatial and temporal resolution in fluorescence microscopy. Fluorescence microscopy is a special form of light microscopy. Fluorescence microscopy uses fluorescence to highlight structures in fixed and living biological specimens instead of using absorption, phase or interference effects. The fluorescence is delivered either by inorganic dyes, proteins, synthetic beads or by auto fluorescent structures within a sample. The most prominent difference in fluorescence microscopy is that it employs incident light instead of transmitted light. Therefore, the beam path for fluorescence microscopy substantially differs from that of transmitted light techniques. Also, fluorescence microscopy is limited in resolution by the wavelength of light, referred to as the "diffraction limit", which restricts lateral resolution and axial resolution at typical excitation and emission wavelengths when a sample emits fluorescence that is detected by the microscope.

The diameter of most organelles is below the diffraction limit of light, limiting spatial resolution and concealing sub-structure. Although recent developments have improved spatial resolution and even overcome the traditional diffraction barriers, comparable improvements in temporal resolution are still needed.

SUMMARY

Super-resolution is a technique that enhances the resolution of an imaging system. Super-resolution microscopy is a form of light microscopy and allows the capture of images with a higher resolution than the diffraction limit. As such, there is a need in the art for super-resolution using a multifocal structured illumination microscopy system that produces a multi-focal excitation pattern of the sample for each high-resolution image. In light of the problems and deficiencies noted above, the present invention provides microscopy systems and methods for super-resolution microscopy using multifocal structured illumination.

In accordance with one embodiment, a method is provided for super-resolution microscopy using multifocal structured illumination. A multi-foci excitation light may be formed from a light beam. The multi-foci excitation light may be scanned toward a sample using a side of a scanning mirror. The sample may be illuminated with the multi-foci excitation light to produce a multi-foci emission light. The multi-foci emission light may be directed onto the side of the scanning mirror. The multi-foci emission light may be steered through an emission microlens array under multi- focal structured illumination conditions to form a modified multi-focal emission pattern. The modified multi-focal emission pattern may be collected with a detector. A super-resolution image may be constructed from the modified multi- focal emission pattern.

In accordance with one embodiment, a microscopy system is provided for super- resolution microscopy using multifocal structured illumination. The system may include a light source that transmits a light beam. An excitation microlens array may be oriented along an excitation path that receives the light beam therethrough to form a multi-foci excitation light. A single-sided scanning mirror can direct the multi- foci excitation light onto a sample such that the sample produces a multi-foci emission light. The single-sided scanning mirror may rescan fluorescence emissions of the multi-foci emission light using the single-sided scanning mirror. An emissions microlens array may be oriented along an emission path subsequent to the single-sided scanning mirror and adapted to form a modified multi- focal emission pattern. A detector may be oriented along the emission path subsequent to the emissions microlens array and adapted to collect the modified multi- focal emission pattern.

There has thus been outlined, rather broadly, the more important features of the invention so that the detailed description thereof that follows may be better understood, and so that the present contribution to the art may be better appreciated. Other features of the present invention will become clearer from the following detailed description of the invention, taken with the accompanying drawings and claims, or may be learned by the practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a multifocal structured illumination microscopy system using a scanning mirror to achieve super-resolution in accordance with one embodiment.

FIG. 2 is a homogenous square excitation profile of a light beam in accordance with one embodiment.

FIG. 3 is a composite assembly including an emission microlens array and pinhole array in accordance with one embodiment. FIG. 4 is a pinhole array having various pinhole sizes and dimensions in accordance with one embodiment.

FIG. 5 is a multifocal structured illumination microscopy system using two scanning mirrors to achieve super-resolution in accordance with one embodiment.

FIG. 6 is an additional multifocal structured illumination microscopy system using two scanning mirrors and two cameras to achieve super-resolution in accordance with one embodiment.

FIG. 7 is an additional multifocal structured illumination microscopy system having microlens arrays moving in unison to achieve super-resolution in accordance with another embodiment.

FIG. 8 is an additional multifocal structured illumination microscopy system using a glass block which enables two microlens arrays to move in unison to achieve super- resolution in accordance with one embodiment.

FIG. 9 illustrates a method for using a scanning mirror to achieve super-resolution in a multifocal structured illumination microscopy system.

These drawings are provided to illustrate various aspects of the invention and are not intended to be limiting of the scope in terms of dimensions, materials, configurations, arrangements or proportions unless otherwise limited by the claims. DETAILED DESCRIPTION

While these exemplary embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, it should be understood that other embodiments may be realized and that various changes to the invention may be made without departing from the spirit and scope of the present invention. Thus, the following more detailed description of the embodiments of the present invention is not intended to limit the scope of the invention, as claimed, but is presented for purposes of illustration only and not limitation to describe the features and characteristics of the present invention, to set forth the best mode of operation of the invention, and to sufficiently enable one skilled in the art to practice the invention. Accordingly, the scope of the present invention is to be defined solely by the appended claims.

Definitions In describing and claiming the present invention, the following terminology will be used.

The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a lens" includes reference to one or more of such elements and reference to "exposing" refers to one or more such steps.

As used herein with respect to an identified property or circumstance, "substantially" refers to a degree of deviation that is sufficiently small so as to not measurably detract from the identified property or circumstance. The exact degree of deviation allowable may in some cases depend on the specific context.

As used herein, "adjacent" refers to the proximity of two structures or elements.

Particularly, elements that are identified as being "adjacent" may be either abutting or connected. Such elements may also be near or close to each other without necessarily contacting each other. The exact degree of proximity may in some cases depend on the specific context.

As used herein, a plurality of items, structural elements, compositional elements, and/or materials may be presented in a common list for convenience. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, no individual member of such list should be construed as a de facto equivalent of any other member of the same list solely based on their presentation in a common group without indications to the contrary.

As used herein, the term "at least one of is intended to be synonymous with "one or more of." For example, "at least one of A, B and C" explicitly includes only A, only B, only C, and combinations of each.

Concentrations, amounts, and other numerical data may be presented herein in a range format. It is to be understood that such range format is used merely for convenience and brevity and should be interpreted flexibly to include not only the numerical values explicitly recited as the limits of the range, but also to include all the individual numerical values or sub-ranges encompassed within that range as if each numerical value and subrange is explicitly recited. For example, a numerical range of about 1 to about 4.5 should be interpreted to include not only the explicitly recited limits of 1 to about 4.5, but also to include individual numerals such as 2, 3, 4, and sub-ranges such as 1 to 3, 2 to 4, etc. The same principle applies to ranges reciting only one numerical value, such as "less than about 4.5," which should be interpreted to include all of the above -recited values and ranges. Further, such an interpretation should apply regardless of the breadth of the range or the characteristic being described.

Any steps recited in any method or process claims may be executed in any order and are not limited to the order presented in the claims. Means-plus-function or step-plus- function limitations will only be employed where for a specific claim limitation all of the following conditions are present in that limitation: a) "means for" or "step for" is expressly recited; and b) a corresponding function is expressly recited. The structure, material or acts that support the means-plus function are expressly recited in the description herein. Accordingly, the scope of the invention should be determined solely by the appended claims and their legal equivalents, rather than by the descriptions and examples given herein.

Embodiments of the Invention

The disclosure provides for super-resolution microscopy using multifocal structured illumination. A multi-foci excitation light may be formed from a light beam. The multi- foci excitation light may be directed and scanned toward a sample using a side of a scanning mirror. The sample may be illuminated with the multi-foci excitation light to produce a multi-foci emission light. The multi-foci emission light may be directed onto the side of the scanning mirror. The multi-foci emission light may be steered through an emission microlens array under multi-focal structured illumination conditions to form a modified multi-focal emission pattern. In some cases, the multi-foci emission light may be steered through an emission microlens array which is placed one focal length before the focus would have been formed by the scan lens thus producing emission foci with up to twice the sharpness to form a modified multi-focal emission pattern. The modified multifocal emission pattern is collected with a detector. A super-resolution image may be constructed from the modified multi-focal emission pattern.

In accordance with an embodiment shown in FIG. 1, a multifocal structured illumination microscopy system 100 is provided for super-resolution. The multifocal structured illumination microscopy system 100 may include a light source 102, beam expanders 104, 106, mirrors 108, 110, 112, 114, and 144, a square aperture 116, a tube lens 118, an excitation microlens array 120, an emissions microlens array 122, scan lenses 124, 126, 128, and 130, an object 132 (e.g., a sample), a focal point 134 with objective lens 132, a scanning mirror 136, such as a galvanometer, a dichroic mirror 142, and an image capturing device 138, such as a camera.

In one embodiment, the beam expanders 104, 106 may be positioned subsequent to the light source 102 along an excitation path of the light beam. Mirrors 108, 110, also disposed along the excitation path, may be placed between the beam expanders 104, 106 and the excitation microlens array 120. A first scan lens 124 may be oriented along the excitation path and positioned between the excitation microlens array 120 and the dichroic mirror 142. The microlens array can take the form of an actual microlens array or devices that can produce a similar output such as a digital light processing (DLP) chip. The excitation light can be directed from the scanning mirror 136 towards a sample and associated objective lens 132. In one embodiment, the objective lens 132 may be positioned a distance from the scanning mirror 136 allowing for the light beam to travel along the excitation path toward a sample at the focal point 134 to cause stimulation and emission from the sample. The emission light beam returns through the objective toward the scanning mirror 136 along an emissions path. An optional second scan 130 or a similar set of lenses, the square aperture 116, and the tube lens 118 may be positioned between the object 132 and the scanning mirror 136. The scan lenses can be oriented in order to assure that beams hitting the scanning mirror have identical scanning parameters while being scanned. An optional mirror 137 can be used to direct excitation light toward the sample. It should be noted that scanning, de-scanning and/or re-scanning may be achieved using the single scanning mirror 136, such as a galvanometer, a two dimensional (2D) scanning piezo mirror, a micro -electromechanical systems (MEMS) mirror, or other suitable rapidly positionable mirror.

In one aspect, an emissions path of the light beam may include an emissions path loop that includes a third scan lens 126, mirror 112, emissions microlens array 122, pinhole array 140, mirror 114, and a fourth scan lens 128, arranged in such order. This configuration of the emissions microlens array can be desirable because by placing the emissions microlens one focal length before the focus that would have been formed by the second scan lens emission foci will be up to twice as sharp. Typically, the emissions microlens can be oriented to maintain multi- focal structured illumination conditions. These conditions are known in the art and are more fully extensively detailed in various articles such as those referenced herein. However, as a general guideline, the emissions microlens can be placed within about 30%, and often within about 5% of one focal length of the focal point formed by the second relay/scan lens, although resolution improvements may be reduced slightly. The scanning mirror 136 may a single sided scanning mirror which can be angularly rotated for scanning the light beam. Mirror 144 may be oriented a distance from the scanning mirror 136 and positioned between the scanning mirror 136 and the capturing device 138. An optional focusing lens 146 can be used to focus emission light onto a corresponding detector region of the capturing device 138. The capturing device 138 may be configured to detect the probe molecule luminescence. The capturing device 138 may optionally be cooled with liquid cooling. The capturing device 138 may include an image construction module (e.g., a data computing module), such as a field-programmable gate array (FPGA), and can be built integrally with the capturing device 138, the multifocal structured illumination microscopy system 100, and/or can be provided separately. The capturing device 138 can take captured images from different focal planes or object planes and combine them to produce two or three dimensional image output. It should be noted that the image construction module, such the FPGA, may be provided and adapted to construct a super-resolution image from the modified multi-focal emission pattern. An image construction module as used herein can be circuitry or a processor and software. Use and reconstruction of images using multi- focal structured illumination microscopy (MSIM) is known and can be accomplished using techniques described in York et al, "Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy;' Nat. Methods 9(7), 749-754 (2012), 10.1038/nmeth.2025; York et al, "Instant super-resolution imaging in live cells and embryos via analog image processing;' Nat. Methods 10(11), 1122-6 (2013), 10.1038/nmeth.2687; Sheppard, C.J.R., "Super- resolution in Confocal Imaging " Optik 80, no. 2, 53-54 (1998); Roth et al, "Optical Photon Reassignment Microscopy arXiv: 1306.6230 [physics. optics] (2013); De Luca et al, "Re-scan confocal microscopy: scanning twice for better resolution " Biomed Opt Express 4(11), 2644-2656 (2013); Schulz et al, "Resolution doubling in fluorescence microscopy with confocal spinning-disk image scanning microscopy " Proc Nat Acad Sci vol. 110, no. 52, 21000-21005 (2013), 10.1073/pnas. l315858110, and Shroff et al, International Patent Application Publication No. WO 2013/126762, which are each incorporated in their entireties as if fully reproduce herein. Acordingly, this invention is an improvement to such recent developments in multi-focal structured illumination microscopy (i.e. MSIM or analog-SIM). Thus, this invention can be implemented as described herein with reference to the underlying physics and design conditions which background is known in the field.

In one embodiment, the light source 102 may be one of various wavelengths, numbers of light sources, and types of light sources, but is typically a coherent light. Although a specific light source, such as a laser may be mentioned herein, other types of light sources can also be used to provide the functions of activation and readout as described herein, provided the light source is compatible with micro lenses. For example, a 405 nm laser or other lasers may be used to activate a subset of probe molecules. A selected range of intensities may be used to convert only a sparse subset of molecules at a time, e.g. to activate at least one molecule with at least one activation photon. Although powers can vary, a power ranging from about 0.01 μW to 5.0mW may be suitable in some cases. The power used can depend on the object 132, such as a particular probe molecule and sample characteristics, and specific optical elements used throughout the system. The light source 102 may be used to expose photoconvertible fluorescent probes in a natural state prior to conversion, excite the converted fluorescent probe, subsequently providing for emission of light by the capturing device 138, such as a CCD camera.

Although other probe molecules may be suitable, the probe molecules used herein can generally be fluorophores. The fluorophores may be imaged either sequentially or simultaneously. The multifocal structured illumination microscopy system 100 can include a fluorophore localization module (not shown) configured to localize each fluorophore in three dimensions. The sample can include cells having photo activatable or photoswitchable fluorescent molecules (PAFMs) residing in a biological membrane, including photoactivatable or photoswitchable fluorescent proteins or photoactivatable or photoswitchable fluorescent lipids or lipids with photoactivatable or photoswitchable fluorescent molecules attached by a chemical bond. In one aspect, beam expanders 104, 106 may be oriented along the excitation path and may be used to expand the light beam to yield a desired beam size and/or dimension. Mirrors 108 and 100 may be oriented subsequent to the beam expanders 104, 106 for directing the light beam from the light source 102 towards an excitation microlens array 120. In one embodiment, the excitation microlens array 120 may be oriented along the excitation path between the focus lens 110 and the first scan lens 124. The scan lenses focus and collimate the beam so that it has the identical attributes when it reflects off the scanning galvos as well as in assuring the beam has proper attributes at the tube lens and microlenses. The light beam may pass through the excitation microlens array 120, which creates and/or forms the multi-foci excitation light. The light beam (e.g., the multi-foci excitation light) may then be recollimated by the first scan lens 124.

The dichroic mirror 142, separated a distance from the first scan lens 124 and the scanning mirror 136, may be positioned and/or angled for deflecting the multi-foci excitation light to the scanning mirror 136, such as a single sided galvo scanning mirror, onto the second scan lens 130. The multi-foci excitation light may be scanned by one side of the scanning mirror 136 and imaged onto a sample having fluorescent probes via the second scan lens 130 and the tube lens 118 and the objective 132 (e.g.,. the multi-foci excitation light may be scanned by rotating the scanning mirror 136. The fluorescence emission from the sample at the focal point 134 can be passed back through objective lens 132 and the multi-foci emission light passes though the tube lens 118 and the scan lens 130. That is, illuminating the sample produces a multi-foci emission light from the multi- foci excitation light.

It should be noted that in the event of using a Gaussian excitation profile and/or a 1- dimensional (ID) scanner, a portion of the sample that is not imaged by the multi-foci excitation light may be exposed to light. This often results in photo bleaching and reduces the brightness of the probes. As such, optional square aperture 116 may be placed at the image plane 150 between the second scan lens 130 and the tube lens 118. The square aperture 116 may be adjusted in such way that only the parts of the object 132 in the field of view (FOV) are exposed to the multi-foci excitation light. Hence, the square aperture 116 may be placed in the image plain 150 in order to prevent photo bleaching in regions of the object 132 that are not imaged. The square aperture can be accomplished using off the shelf adjustable apertures or custom with designs.

As mentioned, illuminating the sample with the multi-foci excitation light produces a multi-foci emission light. As such, the multi-foci emission light travels along an emissions path from the object 132 back to the scanning mirror 136. In one embodiment, the multi-foci emission light may be rescanned by the scanning mirror 136 and directed back towards the third scan lens 126. In other words, the multi-foci emission light may be scanned by the scanning mirror 136 and follow along an emissions path loop. That is, one side of the scanning mirror 136, which may be rapidly rotating, may steer the multi-foci emission light to the third scan mirror 126, and passes therethrough, which is then reflected by mirror 112. The multi-foci emission light proceeds to and passes through the emissions micro lens array 122, within the emissions path loop, to form a modified multi- focal emission pattern. The multi-foci emission light (e.g., the modified multi-focal emission pattern) may pass through the pinhole array 140 and reflect off mirror 114 towards a fourth scan lens 128 ending back at the scanning mirror 136 to complete the emissions path loop. The pinhole array 140 may be placed in the imaging plane in order to regain sectioning ability. The Pinholes are used to physically reject out-of focus light, thus providing higher contrast images at each focal depth imaged.

In some cases, the pinhole array 140 and the emissions microlens array 122 can be oriented along the emissions path and secured on a common rotating axis. Although, the pinhole array and emissions microlens array can be spaced apart; in one aspect, the pinhole array 140 may be coupled and combined with the emissions microlens array 122 into a composite assembly in which these two elements are contacting one another. By coupling the pinhole array 140 to the emissions microlens array 122, the combined pinhole array 140 and emissions microlens array 122 overcome the challenges of the current state of the art where a microlens array and pinhole array are on different mounts and need to be aligned in respect to each other. Furthermore, the different mounts can drift over time which requires readjustment which is time consuming and also difficult to automate. As such, the pinhole array 140 may be combined and coupled to the emissions microlens array 122.

Super-resolution may be achieved by placing the emissions microlens array 122 before the image plane 155 formed between the third scan lens 126 and the fourth scan lens 128. If placed one focal length before the focus would have been formed, the emissions micro lens array 122 can thus produce emission foci with up to twice the sharpness (the exact factor depends on the exact position of the microlens array but it can range from 0.75 to 1.3 times the focal length, and in some cases from 0.9 to 1.1 times). The capturing device 138, such as a camera, will be collecting the light for one full scan. The resulting image can have improved resolution by as much as V2. Also, the image captured by the capturing device 138 may have the potential to improve the resolution by up to a factor of 2.

As such, the multifocal structured illumination microscopy system 100 as described herein improves efficiency and spacing over the current state of the art with a more compact design. For example, the emission throughput plays a significant role in a variety of applications, such as in biology applications, since a majority of objects (e.g. microscopic sample) are not bright and therefore the signal to noise ratio on the camera is extremely low.

Moreover, the multifocal structured illumination microscopy system 100 may be used as an add-on to conventional microscopes by plugging the multifocal structured illumination microscopy system 100 into a side port (which already exists in most microscopes).

In one embodiment, the super-resolution of the multifocal structured illumination microscopy system 100 may also be combined with two-photon microscopy in order to yield super-resolution two-photon images. The advantage of two-photons emission is the capability to focus the beam the beam to a defined depth allowing for the removal of the pinhole array, while still providing an improvement in resolution by a factor of two. Also, the super-resolution of the multifocal structured illumination microscopy system 100 may also be combined with a single molecule localization microscopy. Typically, in a single molecule localization the sample is imaged with wide field microscopy and the molecules are localized in the post analysis. As such, the wide field imaging may be replaced by the super resolution technique described with respect to the multifocal structured illumination microscopy system 100. As such, the improved resolution of the images can result in better localization accuracy. Alternatively the system can be constructed to bypass the microlenses and provide standard widefield imaging. Standard widefield imaging can also be used for superresolution.

Turning now to FIG. 2, a homogenous square excitation profile 200 of a light beam is depicted in accordance with one embodiment. It should be noted that a multi-foci excitation light may have a Gaussian profile, meaning the intensity in the imaged sample varies from one location to another. To circumvent the Gaussian profile problem, a beam shaping device (e.g., a Powell lens and aperture, a modified fiber or square fiber and the like) may be used to shape the multi-foci excitation light to have a homogenous rectangular excitation profile 200. The homogenous rectangular excitation profile 200 may have a square-shape field of view (FOV) and rectangle-shape (e.g., the rectangular shape of the multi-foci excitation light) that is twice as long as the square FOV (e.g., 2 times the FOV) in the scan direction.

Thus, the homogenous rectangular excitation profile 200 allows an entire sample to be excited by the multi-foci excitation light with the same intensity. Furthermore, since the field of view (FOV) is square, by exciting the square region of the FOV of the multi-foci excitation light, the power of the light source (e.g., light source 102 of FIG 1) is used more efficiently. The beam shaping device can be placed in various points of the illumination path. As an example in Figure 1 the beam shaping device 107 could be placed after the beam expander (the combination of 104 and 108) and before the microlens 120.

For example, the homogenous rectangular excitation profile 200 of the multi-foci excitation light may be used to excite the sample (e.g., object 132 in FIG. 1) in the case of using a ID scanner. If a 2D scanner is used, a homogenous square profile may be used. Hence, the multi-foci excitation light illuminating an object (e.g., the sample) may be homogenous and rectangular using a beam shaping device.

As previously mentioned in FIG. 1, the pinhole array 140 may be combined and coupled to the emissions microlens array 122. Coupling the pinhole array 140 with the emissions microlens array 122 reduces the need to have two independent and adjustable mounts and prevents drifting. As such, FIG. 3 illustrates an emission microlens array- pinhole array 300 in accordance with one embodiment. In one aspect, the emission microlens array 322 and pinhole array 340 of FIG.1, may be combined into one unit by first aligning the two arrays in respect to each other and then gluing the emission microlens array 322 together with the pinhole array 340. In an alternative approach, as depicted in FIG. 3, an emission microlens array 322 may be combined with the pinhole array 340 by putting a pinhole mask 340 on the back of the emission microlens array 322. In this case, the thickness of the emission microlens array 322 can to be designed in such way that an image plane may be precisely formed at an edge of the flat back edge of the microlens array. As such, the pinhole array mask 340 may be fabricated on the back of the emission microlens array 340, as illustrated in FIG. 3. In one optional aspect, the microlens array and pinhole array can be fabricated from a common wafer.

Turning now to FIG. 4, a pinhole array 400 having various pinhole sizes and dimensions is depicted in accordance with one embodiment. FIG.4 illustrates both an excitation and emission microlens array, such as the excitation microlens array 120 and the emissions microlens array 122 of FIG. 1. In FIG. 4, pinhole array 400 that is comparable to the pinhole array 140 in FIG. 1, with separate regions 402, 404 of various lens sizes and pitch. In one aspect, region 402 may include smaller lenses with smaller pitch sizes resulting in faster imaging but having a higher cross talk between different foci. In one aspect, region 404 may include larger lenses with larger pitch sizes and will reduce the cross talk. As example the mirco lens array could be a 1" square including lenses with a lmm focal length, and a pitch of 220 microns. The focal length of the mircolenses could range from 0.5 to 6mm and a pitch from 150-400. The larger lenses with larger pitch sizes are important for a thicker sample but the larger lenses also result in slower imaging times. Additionally, a pinhole size in a pinhole array, such as pinhole array 400, may be different for different regions of the pinhole array. Thus, the smaller pinholes in region 402 increase sectioning ability but may reduce throughput, whereas the larger pinholes in region 404 decrease sectioning but result in an increase in throughput.

As such, in one aspect, a pinhole array, such as pinhole array 400, may include regions with different pinhole size. The desired pinhole region of the pinhole array 400, such as region 402, 404, may be inserted accurately into an optical path via piezo actuation (e.g., using a piezo actuator). The excitation microlens array, such as the excitation microlens array 120 of FIG.1, and the pinhole array, such as pinhole array 140 of FIG. 1, may have regions with different lens (and pitch) size. Also, the excitation microlens array 120 of FIG.1, and the pinhole array, such as pinhole array 140 of FIG. 1, may be arranged in one of a variety of patterns, such as in a hexagonal pattern. By having a microlens array and a pinhole arrays with different options, a user may decide what tradeoff works best for a specific sample to be imaged.

Referring to FIG. 5, a multifocal structured illumination microscopy system 500 is shown which is similar in many regards to the multifocal structured illumination microscopy system 100 of FIG. 1. Here, the multifocal structured illumination microscopy system 500 may be arranged in an alternative configuration incorporating two scanning mirrors for super-resolution.

In one embodiment, the excitation microlens array 520 may be in the excitation path and the emissions microlens array 522 and pinhole array 540 oriented in the emission path. In one aspect, similar to FIG. 1, the first scan lens 524 may be oriented along the excitation path and positioned between the emissions microlens array 522 and the dichroic mirror 542. The objective lens 532 is positioned a defined distance from the scanning mirror 536 allowing for the light beam to travel along the excitation path and returning along an emissions path. The second scan lens 530, the square aperture 516, and optional tube lens 518 may be positioned between the objective 532 and the scanning mirror 536. The square aperture 516 may be positioned at or near image plane 550. An optional mirror 537 can be oriented to reflect excitation light toward the objective lens 532 and sample. The sample located at the focal point 534 may be illuminated with the multi-foci excitation light at focal point 534 to produce a multi-foci emission light which then traverses back along the emission path which coincides with the excitation path up to the scanning mirror 536.

However, in contrast to FIG. 1, there is no emissions path loop, but rather, a second scanning mirror 575 may be positioned a distance along the modification emission path, which portion of the emission path may be substantially linear, from the scanning mirror 536 to second scanning mirror 575. The third scan lens 526, the emissions microlens array 522, the pinhole array 540, and the fourth scan lens 528 may arranged and oriented in the linear emission path between the scanning mirror 536 and the second scanning mirror 575. The third scan lens 526 may be positioned closest to the scanning mirror 536 while the fourth scan lens 528 may be positioned closest to the second scanning mirror 575. The emissions microlens array 522 and the pinhole array 548 may be oriented along the emissions path between the third scan lens 526 and the fourth scan lens 528.

In one aspect, similar to FIG. 1, the multi-foci emission light travels along an emissions path from the sample at focal point 534 back to the scanning mirror 536. However, the multi-foci emission light may be rescanned by the scanning mirror 536 and directed back towards the third scan lens 526 and passes therethrough, which then passes through the emissions microlens array 522 and the pinhole array 540, which forms a modified multi-focal emission pattern. Super-resolution may be achieved by placing the emissions microlens array 522 one focal length before the focus that would have been formed by the third scan before the image plane 555 formed between the third scan lens 526 and the fourth scan lens 528. The modified multi- focal emission pattern may then pass through the fourth scan lens 528 and scanned by one side of the second scanning mirror 575 towards the capturing device 538, which collects the modified multi-focal emission pattern.

It should be noted that the scanning mirror 536 and the second scanning mirror 575 may be synchronized and scanned as equal amplitudes. As such, the multifocal structured illumination microscopy system 500, with only one single focus of the object 532, achieves super-resolution by scanning and descanning using only one scanner mirror, such as using scanner mirror 536, and the rescanning of the light beam using a second scanning mirror, such as the second scanning mirror 575.

In this way, the multifocal structured illumination microscopy system 500 eliminates and reduces the number of additional optical components from the emission path, such as the focus lenses 112 and 114 of FIG. 1, further improving throughput of the multi-foci emission light. An additional advantage of the multifocal structured illumination microscopy system 500 is being less prone to drift given the reduction and decrease in the number of optical elements in the emission path, which will make the system more stable over time. Also, by using the multifocal structured illumination microscopy system 500 and using two scanner mirrors, such as two 2D Galvo scanner mirrors, the light beam only needs to cover the field of view (instead of twice the field of view), therefore, utilizing more power of the light (e.g., laser power) to actually image the sample. That is, the multifocal structured illumination microscopy system 500 enhances energy efficiency by using the light beam (e.g., laser power) more efficiently.

Referring to FIG. 6, a multifocal structured illumination microscopy system 600 is shown which is similar in many regards to the multifocal structured illumination microscopy system 500 of FIG. 5 and the multifocal structured illumination microscopy system 100 of FIG. 1. Here, the multifocal structured illumination microscopy system 600 may be arranged in an alternative configuration incorporating two galvanometers for super- resolution but also includes an additional focus lens 658 and second camera 660 for super- resolution in accordance with one embodiment.

In one embodiment, the excitation micro lens array 620 may be in the excitation path (receiving light from a light source - not shown) and the emissions micro lens array 622 and pinhole array 640 oriented in the emission path. In one aspect, similar to FIG. 1, the first scan lens 624 may be oriented along the excitation path and positioned between the emissions microlens array 620 and the dichroic mirror 642. The objective lens 632 is positioned a defined distance from the scanning mirror 636 allowing for the light beam to travel along the excitation path and returning along an emissions path. The second scan lens 630, the square aperture 616, and the tube lens 618 may be positioned between the objective lens 632 and the scanning mirror 636. The square aperture 616 may be positioned at or near image plane 650. An optional mirror 637 can be oriented to direct light toward the sample. The sample may be illuminated at the focal point 634 of the objective 632 with the multi-foci excitation light to produce a multi-foci emission light.

However, in contrast to FIG. 1, there is no emissions path loop, but rather, a second scanning mirror 675 may be positioned a distance, which may be substantially linear, from the scanning mirror 636 to the scanning mirror 675. The third scan lens 626, the emissions microlens array 622, the pinhole array 640, and the fourth scan lens 628 may arranged and oriented in the emission path between the scanning mirror 636 and the second scanning mirror 675. The third scan lens 626 may be positioned closest to the scanning mirror 636 while the fourth scan lens 628 may be positioned closest to the second scanning mirror 675. The emissions microlens array 622 and the pinhole array 648 may be oriented along the emissions path between the third scan lens 626 and the fourth scan lens 628. In one aspect, similar to FIG. 1, the multi-foci emission light travels along an emissions path from the sample through objective 632 back to the scanning mirror 636. However, the multi-foci emission light may be rescanned by the scanning mirror 636 and directed back towards the third scan lens 626 and passes therethrough, which then passes through the emissions microlens array 622 and the pinhole array 640, which forms a modified multi-focal emission pattern. Super-resolution may be achieved by placing the emissions microlens array 622 one focal length before the focus that would have been formed by the third scan before the image plane 655 formed between the third scan lens 626 and the fourth scan lens 628. The modified multi- focal emission pattern may then pass through the fourth scan lens 628 and scanned by one side of the second scanning mirror 675 towards the capturing device 638, which collects the modified multi-focal emission pattern.

It should be noted that the scanning mirror 636 and the second scanning mirror 675 may be synchronized and scanned as equal amplitudes. As such, the multifocal structured illumination microscopy system 600, with only one single focus of the object 632, achieves super-resolution by scanning and descanning using only one scanner mirror, such as using scanner mirror 636, and the rescanning of the light beam using a second scanning mirror, such as the second scanning mirror 675.

As more clearly depicted in FIG. 6, the multifocal structured illumination microscopy system 600 may also be combined with multi-color excitation and detection or with polarization dependent imaging. For example, multi-color imaging may be achieved by splitting a detection path (e.g., the path starting from the second scanning mirror 675 and proceeding to the two capturing devices 638, 675) into two cameras, such as capturing device 638 and 660 using a second dichroic mirror 610 in the emission path. In this case, each capturing device 638 and 660 may detect a different color. The same principle can be applied to different polarization by simply switching the second dichroic mirror 610 with a polarization beam splitter. In an alternative embodiment, multi-color imaging may be realized by splitting the multi-foci emission light onto two separate the regions on a capturing device, such as camera 638, where each region records a different color and/or polarization. The multifocal structured illumination microscopy system 600 imaging technique provides a multi-color or polarization imaging by inserting a second dichroic mirror 610 (and/or a polarization beam splitter) in the emission path subsequent to the second scanning mirror 675 and focusing the new path on the additional capturing device 660. The multifocal structured illumination microscopy system 600 provides instantaneous multicolor imaging with super-resolution enabling, multiple probes to be examined and/or studied at the same time. Alternatively the beams could be layout so that two or more colors could be imaged on to the different sections of the same camera (not shown). For example, the multifocal structured illumination microscopy system 600 provides biologists with the ability to examine live or dynamic samples with instantaneous imaging in two or more colors.

Referring to FIG. 7, an additional multifocal structured illumination microscopy system 700 having micro lens arrays moving in unison for super-resolution is shown which is similar in many regards to the multifocal structured illumination microscopy system 100 of FIG. 1, the multifocal structured illumination microscopy system 500 of FIG. 5., and/or the multifocal structured illumination microscopy system 600 of FIG. 6.

In FIG. 7, the multifocal structured illumination microscopy system 700 may include a light source 702, mirrors 708, 710, 712, 714, a tube lens 718, an excitation microlens array 720, an emissions microlens array 722, scan lenses 724, 726, 728, and 730, an objective 732 (e.g., a sample), a scanning mirror 736, such as a galvanometer, a dichroic mirror 742, and a capturing device 738, such as a camera.

In one embodiment, the excitation microlens array 720 may be positioned subsequent to the light source 102 along an excitation path of the light beam. Scan lens 724 and mirrors 708, 710, also disposed along the excitation path, may be placed between the excitation microlens array 720 and the dichroic mirror 742. Mirrors 710 and 712 may be oriented along the excitation path and be positioned between the dichroic mirror 742 and the emissions microlens array 722. The excitation microlens array 720 and the emissions microlens array 722 can be mechanically coupled to allow contemporaneous movement. The microlens arrays can be one-dimensional (i.e. line arrays) or two-dimensional arrays. This would enable the user to easily make matching changes for both the excitation or and emissions microlens array. The system could also incorporate non-microlens arrays such as a ID rectangle or larger ID pinhole to provide other imaging modalities to match the objective and sample characteristics.

The emissions microlens array 722 and the pinhole array 740 may be arranged and oriented in the emission path between the mirrors 712, 714. The scan lens 728 steers the light beam towards the scanner 750, which scans the light beam towards the scanning mirror 736.

In one embodiment, an objective 732 may be positioned a distance from the scanning mirror 736 allowing for the light beam to travel along the excitation path and return to the scanning mirror 736 along an emissions path. A second scan lens 730 and a tube lens 718 may be positioned between the objective 732 and the mirror 775. It should be noted that scanning, de-scanning and/or re-scanning may be achieved using the single scanning mirror 736, such as a galvanometer, a two sided galvanometer scanning mirror, a two dimensional (2D) scanning piezo mirror, a micro-electromechanical systems (MEMS) mirror, or the like. As an example of an alternative configuration, if the system is used with ID microlens arrays, the scanners 750 and 775 enable the system to scan in the additional dimension of a ID scanner used in position 736.

A capturing device 738 may be configured to detect the probe molecule luminescence. The capturing device 738 may optionally be cooled with liquid cooling. An image construction module as used herein can be circuitry or a processor and software. The capturing device 738 may include an image construction module (e.g., a data computing module), such as a field-programmable gate array (FPGA), and can be built integrally with the capturing device 738, the multifocal structured illumination microscopy system 700, and/or separately. The capturing device 738 can take captured images from different focal planes or object planes and combine them to produce a three dimensional image output. It should be noted that the image construction module, such the FPGA, may be provided and adapted to construct a super-resolution image from the modified multifocal emission pattern.

More specifically, in this arrangement, the multifocal structured illumination microscopy system 700 may be arranged in the alternative configuration incorporating a galvanometer 736 positioned between two additional scanners 750, 775 for super- resolution. Also, the excitation microlens array 720 and the emissions microlens array 722 may be positioned and located for varying and/or moving the excitation microlens array 720 and the emissions microlens array 722 in unison. In one aspect, the excitation microlens array 720 and the emissions microlens array 722 may be combined as one unit for varying and/or moving the excitation microlens array 720 and the emissions microlens array 722 in unison. Alternatively, a length of the excitation path and/or emissions path may adjusted or altered for varying and/or moving the excitation microlens array 720 and the emissions microlens array 722 in unison.

As depicted, the excitation microlens array 720 is oriented in the excitation path and the emissions microlens array 722 may be oriented in the emissions path, and may be closely positioned and our combined as a single unit, for being moved or varied in unison.

Referring to FIG. 8, an additional multifocal structured illumination microscopy system 800 using a glass block for enabling two microlens arrays to move in unison for super-resolution is shown, which is similar in many regards to the multifocal structured illumination microscopy system 700 of FIG. 7.

In one embodiment, the multifocal structured illumination microscopy system 800 may include a light source 802, mirrors 808, 814, a tube lens 818, an excitation microlens array 820, an emissions microlens array 822, scan lenses 824, 826, 828, and 830, an object 832 (e.g., a sample), a focal point 834 of the object 832, a scanning mirror 836, such as a galvanometer, a dichroic mirror 842, and a capturing device 838, such as a camera.

In one embodiment, the excitation microlens array 820 may be positioned subsequent to the light source 102 along an excitation path of the light beam. Scan lens 824 and mirrors 808, also disposed along the excitation path, may be placed between the excitation microlens array 820 and the dichroic mirror 842. The excitation microlens array 820 and the emissions microlens array 822 can be are mechanically coupled to allow contemporaneous movement. The microlens arrays can be one-dimensional (i.e. line arrays) or two-dimensional arrays. This would enable the user to easily make matching changes for both the excitation or and emissions microlens array. The system could also incorporate non-micro lens arrays such as a ID rectangle or larger ID pinhole to provide other imaging modalities to match the objective and sample characteristics.

The emissions microlens array 822 and the pinhole array 840 may be arranged and oriented in the emission path between the mirrors 814. The scan lens 828 steers the light beam towards the scanner 850, which scans the light beam towards the scanning mirror 836.

In one embodiment, an objective lens 832 may be positioned a distance from the scanning mirror 836 allowing for the light beam to travel along the excitation path and return to the scanning mirror 836 along an emissions path. A second scan lens 830 and a tube lens 818 may be positioned between the objective 832 and the scanning mirror 836. It should be noted that scanning, de-scanning and/or re-scanning may be achieved using the single scanning mirror 836, such as a galvanometer, a two sided galvanometer scanning mirror, a two dimensional (2D) scanning piezo mirror, a micro electromechanical systems (MEMS) mirror, or the like. As an example of an alternative configuration, if the system is used with ID microlens arrays, the scanners 850 and 875 enable the system to scan in an additional dimension of a ID scanner used in position 836.

A capturing device 838 may be configured to detect the probe molecule luminescence. The capturing device 838 may optionally be cooled with liquid cooling. An image construction module as used herein can be circuitry or a processor and software. The capturing device 838 may include an image construction module (e.g., a data computing module), such as a field-programmable gate array (FPGA), and can be built integrally with the capturing device 838, the multifocal structured illumination microscopy system 800, and/or separately. The capturing device 838 can take captured images from different focal planes or object planes and combine them to produce a three dimensional image output. It should be noted that the image construction module, such the FPGA, may be provided and adapted to construct a super-resolution image from the modified multifocal emission pattern. The step of constructing the super-resolution image may further include a software processing step such as deconvolution to increase the resolution improvement to two (or double). This processing step could be done with a processing unit or a FPGA. Such processing can be performed in close to real time.

However, in contrast to FIG. 7, a glass block 802 may be orient along the emissions path and positioned between the dichroic mirror 842 and the emissions microlens array 822. The glass block 802 enables the multifocal structured illumination microscopy system 800 to move the excitation microlens array 820 and the emissions microlens array 822 in unison for super-resolution. In one aspect, the excitation and emissions micro lenses 820, 822 and the pinholes in the pinhole arrays 840 may be bypassed to provide a user normal imaging of the object 832.

Turning now to FIG. 9, an embodiment of a method for using a single-sided scanning mirror for super-resolution in a multifocal structured illumination microscopy system is illustrated. The method 900 may be representative of some or all of the operations executed by one or more embodiments described herein. In the illustrated embodiment shown in FIG. 9, the method 900 may start at block 902. The method 900 may form a multi-foci excitation light from an light beam at block 904. Moving to block 906, the method 900 may include scanning the multi-foci excitation light toward a sample using a side of a scanning mirror. At block 908, the sample may be illuminated with the multi-foci excitation light to produce a multi-foci emission light. Next, at block 910, the method 900 may include directing the multi-foci emission light onto the side of the scanning mirror and steering the multi-foci emission light through an emission microlens array to form a modified multi- focal emission pattern. At block 912, the method 900 may include collecting the modified multi- focal emission pattern with a detector. From block 912, the method 900 may move to block 914 and may include construction a super- resolution image from the modified multi-focal emission pattern. The method 900 may end at block 918.

As described herein, a method is provided for super-resolution microscopy using multifocal structured illumination. A multi-foci excitation light may be formed from a light beam. The multi-foci excitation light may be scanned toward a sample using a side of a scanning mirror. The sample may be illuminated with the multi-foci excitation light to produce a multi-foci emission light. The multi-foci emission light may be directed onto the side of the scanning mirror. Thus, only a single-sided scanning mirror is needed. The multi- foci emission light may be steered through an emission microlens array to form a modified multi-focal emission pattern. The modified multi-focal emission pattern may be collected with a detector. A super-resolution image may be constructed from the modified multifocal emission pattern.

In one aspect, the method may form the multi-foci excitation light from the light beam by passing the light beam through an excitation microlens array. In one aspect, the method may include moving an excitation microlens array in unison with the emission microlens array.

In one aspect, the method may include imaging the modified multi-focal emission pattern onto different sectors of the detector.

In one aspect, the method may include deflecting the multi-foci excitation light toward the scanning mirror using a dichroic mirror.

In one aspect, the method may include shaping the multi-foci excitation light to be homogenous and rectangular.

In one aspect, the method may include passing the multi-focal excitation light through a square aperture in an image plane prior to the collecting.

In one aspect, the method may include aligning a pinhole array with the emission microlens array, wherein the pinhole array is coupled to the emission microlens array, the pinhole array and the emission microlens array each having one of a plurality of a sizes and pitches.

In one aspect, the scanning mirror may be at least one of a 2D galvanometer scanning mirror, a micro -electromechanical systems (MEMS) mirror, and/or a 2D scanning piezo mirror.

A microscopy system may also be provided for super-resolution microscopy using multifocal structured illumination. The system may include a light source that transmits a light beam. An excitation microlens array may be oriented along an excitation path that receives the light beam therethrough to form a multi-foci excitation light. A single-sided scanning mirror can direct the multi-foci excitation light onto a sample such that the sample produces a multi-foci emission light. The single-sided scanning mirror may rescan fluorescence emissions of the multi-foci emission light using the single-sided scanning mirror. An emissions microlens array may be oriented along an emission path subsequent to the single-sided scanning mirror and adapted to form a modified multi-focal emission pattern. A detector may be oriented along the emission path subsequent to the emissions microlens array and adapted to collect the modified multi- focal emission pattern.

In one aspect, the multifocal structured illumination system may include a glass block oriented along the excitation path and/or the emissions path for moving the excitation mircolens array in unison with the emission microlens array. In one aspect, the multifocal structured illumination system may include a dichroic mirror oriented along the excitation path that deflects the multi-foci excitation light.

In one aspect, the multifocal structured illumination system may include a scan lens and a tube lens, oriented along the excitation path subsequent to the single-sided scanning mirror that receives and passes the multi-foci excitation light therethrough.

In one aspect, the multifocal structured illumination system may include a pinhole array and a scan lens oriented along the emission path subsequent to the scanning mirror, the pinhole array coupled to and aligned with the emissions micro lens array.

In one aspect, the multifocal structured illumination system may include a beam- shaping device oriented along the excitation path that shapes the multi-foci excitation light to be homogenous and rectangular.

In one aspect, the single-sided scanning mirror can be a galvanometer scanning mirror, a micro-electromechanical systems (MEMS) mirror, and/or a 2D scanning piezo mirror.

In one aspect, the multifocal structured illumination system may include a second single-sided scanning mirror oriented along the excitation path and subsequent to the single-sided scanning mirror that rescans the fluorescence emissions. The detector (e.g., the capturing device 138 of FIG. 1 or the capturing device 175 of FIG. 6) may collect the fluorescence emissions of the multi-foci emission light. The detector e.g., the capturing device 138 of FIG. 1 or the capturing device 175 of FIG. 6) may also be configured to provide multi-color excitation and detection and/or polarization of the super-resolution image.

Some of the functional units described in this specification have been labeled as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.

Modules may also be implemented in software for execution by various types of processors. An identified module of executable code may, for instance, comprise one or more blocks of computer instructions, which may be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which comprise the module and achieve the stated purpose for the module when joined logically together.

Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices. The modules may be passive or active, including agents operable to perform desired functions.

The technology described here can also be stored on a computer readable storage medium that includes volatile and non- volatile, removable and non-removable media implemented with any technology for the storage of information such as computer readable instructions, data structures, program modules, or other data. Computer readable storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other computer storage medium which can be used to store the desired information and described technology.

The foregoing detailed description describes the invention with reference to specific exemplary embodiments. However, it will be appreciated that various modifications and changes may be made without departing from the scope of the present invention as set forth in the appended claims. The detailed description and accompanying drawings are to be regarded as merely illustrative, rather than as restrictive, and all such modifications or changes, if any, are intended to fall within the scope of the present invention as described and set forth herein.