Reflection of the Laser

Inventor: Edward CALVIN

BACKGROUND OF THE INVENTION FIELD OF THE INVENTION

At least one embodiment of the invention relates generally to automated analysis of particles in liquid samples using a laser-scanning confocal microscope. More specifically, embodiments described herein enable a novel method and apparatus for focusing a laser scanning microscope using the reflection of the laser beam off of the vessel containing the liquid sample.

DESCRIPTION OF THE RELATED ART

Measurement of microscopic particles is an integral part of many applications in clinical and life sciences testing as well as research applications. A laser-scanning cytometer that images such particles in a liquid sample and measures the size, shape, light absorption, and fluorescence emission of the particles is one means of making such measurements.

One possible embodiment of a laser-scanning cytometer (LSC) comprises an optically clear vessel containing the sample, an objective lens that focuses excitation light from a laser onto the sample and collects light emitted by the sample, a mirror or set of mirrors used to scan the laser across the sample in a two-dimensional pattern, and at least one detector used to measure the amount of light emitted by the sample at each point in the scan.

Applications frequently require the analysis of many samples at a time. Automating the imaging of the samples so that a large number of samples can be analyzed without requiring the constant attention of an operator is highly desirable. In order to facilitate this automation, the LSC must include a means of focusing each sample. The optical resolution of some LSC systems may be on the order of one micron, and obtaining acceptable focus may require positioning the objective lens with respect to the plane containing the particles in the sample (the sample plane) within a few microns of the optimal focus distance from the sample plane. BRIEF SUMMARY

Described here is a novel invention that accomplishes automated autofocusing of the objective lens in the LSC in a fast, simple, robust manner.

Embodiments of the invention are utilized in concert with a sample vessel that contains a liquid sample. The sample may contain microscopic particles such as cells or man-made particles such as plastic microspheres. The sample vessel has an optically clear lower surface through which particles in the sample may be observed, measured or imaged. The particles in the sample lie on the lower surface of the sample vessel (the sample plane). It will be appreciated that the bottom surface has a lower surface proximate to an objective lens and an upper surface upon which the microscopic particles are positioned.

An objective lens focuses light from a laser onto a point in the sample plane, i.e., the surface layer of the sample vessel upon which the microscopic particles are positioned. The objective lens collects light emanating from the particles in the sample in response to illumination by the laser. Other embodiments of this invention include a light source that is not a laser but which is focused to a small spot by the objective lens (such as a highly collimated beam of light from a light-emitting diode).

A scan mirror positioned in the path of the laser beam between the laser and the objective lens causes the point at which the laser beam intersects the sample plane to traverse across the sample plane.

An actuator raises or lowers the objective lens to focus the lens on the sample plane. One device that could be used as an actuator is a stepper motor with a captive lead screw configured to raise or lower the objective lens as the stepper motor rotates.

Embodiments of the invention utilize an algorithm implementing the method of autofocusing described herein. The steps in the algorithm include the following:

Position the objective lens at a position known to be below the point where the objective lens is focused on the lower surface of the sample vessel, accounting for uncertainty in the estimate of this position.

Turn on the laser and begin to oscillate the scan mirror in one axis. The scan mirror should oscillate at a speed that is fast enough that the point illuminated by the laser in the focal plane of the objective lens moves by a distance that is greater than the apparent size of the reflection of the laser beam, and ideally several diameters of the reflection of the laser beam.

Begin capturing images using the CMOS camera of the reflection of the laser beam off of the surface of the sample vessel. The reflection will initially be a large spot of relatively low intensity because the objective lens is not focused on any surface. Begin raising the height of the objective lens with respect to the sample plane while recording images using the CMOS camera at height increments. The height increments could be limited to be no larger than the required focusing accuracy of the LSC system, which will vary according to the type of sample being analyzed and the type of analysis being performed. As the focal plane of the objective lens approaches the lower surface of the sample vessel, the reflection will increase in brightness and the size of the illuminated spot will shrink to a minimum. The reflection will then diminish in brightness and become less sharp until the focal plane of the objective lens approaches the upper surface of the sample vessel, at which point the reflection will again become brighter and the size of the illuminated spot will shrink to a minimum. As the objective lens continues to move upwards, the illuminated spot will again become less bright and less sharp as the focal point of the objective lens moves past the upper surface of the sample vessel and into the liquid of the sample.

Stop moving the objective lens at a location where the focal point is known to be above the sample plane, accounting for uncertainty in the estimate of that position.

For the images collected at each height of the objective lens, calculate a focus score by applying a mathematical function to the pixel values in each image that increases in value as the image of the reflection of the laser beam becomes more intense (higher pixel values) and smaller (occupies a smaller number of pixels).

Identify the lower surface of the sample vessel. The lower surface is identified by finding the first maximum of the focus score encountered as the objective lens is raised from the start position.

Identify the upper surface of the sample vessel by finding the maximum focus score after the objective lens is raised above the lower surface. Smoothing or filtering the focus score data may be necessary to avoid incorrectly identifying a small local variation in the focus score as a maximum focus score.

Identify the sample plane by adding to the height of the upper surface of the sample vessel a distance that is approximately one half the thickness of the particles in the sample, so that the focal plane of the objective lens transects the midpoint of particles lying on the surface of the sample vessel.

Move the objective lens to the location determined in the previous step. In one embodiment, the operator may turn off the laser and stop the scan mirror.

The algorithm is useful where the position of the lower and upper surfaces of the sample vessel are not known a priori with a high degree of accuracy. Because there is no surface below the lower surface of the sample vessel off which the laser can reflect, the method will find this surface very reliably by starting the objective lens at a point sufficiently low that the focal point of the objective lens is known to be below the sample vessel.

After the process described in the previous steps above, an alternative algorithm that only searches for the upper surface of the sample vessel may be utilized for other samples, which minimizes the time require to focus on multiple samples. The upper surface of the sample vessel is the surface upon which the particles may be positioned, assuming the particles are denser than the sample liquid and have settled under the force of gravity. This algorithm is faster than the algorithm described in the previous steps above because it searches over a smaller range of objective lens positions. Because the interface between the liquid of the sample and the air above the sample could be confused with one of the other surfaces, this method requires an initial estimate of the height of the sample vessel such that the focal point of the objective lens is known to start above the upper surface of the sample vessel but within the liquid of the sample. The initial estimate of the height of the sample vessel is known a priori by the operator and supplied to the algorithm as an input.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates a LSC system including a laser-scanning confocal analysis system, an objective lens, an actuator which adjusts the position of the objective lens, a vessel containing the sample, and a computer used to control the LSC system and to record data from the LSC system.

Figure 2 illustrates one possible embodiment of the LSC system including a CMOS camera used for auto-focusing.

Figure 3 illustrates a second possible embodiment of the LSC system including a CMOS camera used for auto-focusing.

Figure 4a shows images of the reflection of the laser off of the sample vessel surface at several focus positions of the objective lens.

Figure 4b shows the focus score as a function of distance of the objective lens from the sample plane.

Figure 5 shows one possible algorithm for auto-focusing the LSC using this apparatus and method.

Figure 6 shows an abbreviated version of the algorithm illustrated in Figure 5 where the approximate height of the sample vessel is known with greater accuracy.

Figure 7a shows the initial position of the objective lens and focal plane at the beginning of the algorithm illustrated in Figure 5. Figure 7b shows the final position of the objective lens and focal plane at the end of the algorithm illustrated in Figure 5.

Figure 7c shows the initial position of the objective lens and focal plane at the beginning of the algorithm illustrated in Figure 6.

Figure 7d shows the final position of the objective lens and focal plane at the end and end of the algorithm illustrated in Figure 6.

DETAIL DESCRIPTION OF THE INVENTION

Figure 1 illustrates the laser-scanning confocal system described in this invention. A sample vessel (103) contains a liquid sample. The sample may contain microscopic particles such as cells or man-made particles such as plastic microspheres. The sample vessel has an optically clear lower surface through which particles in the sample may be observed, measured or imaged. The particles in the sample lie on the lower surface of the sample vessel (the sample plane). It will be appreciated that the bottom surface has a lower surface proximate to an objective lens and an upper surface upon which the microscopic particles are positioned.

In the examples described in this disclosure, the objective lens is positioned vertically below the vessel containing the sample plane, and the sample plane is oriented horizontally. Moving the objective lens higher relative to the sample plane is equivalent to positioning the objective lens closer to the sample plane. Other embodiments might employ different orientations of the sample plane and objective lens. It will be appreciated that the invention could be equally applied to other orientations. For simplicity, this disclosure uses the convention that the sample plane is always positioned above the objective lens.

An objective lens (102) focuses light from a laser onto a point in the sample plane, i.e., the surface layer of the sample vessel upon which the microscopic particles are positioned. The objective lens collects light emanating from the particles in the sample in response to illumination by the laser. Other embodiments of this invention include a light source that is not a laser but which is focused to a small spot by the objective lens (such as a highly collimated beam of light from a light-emitting diode).

An actuator (104) raises or lowers the objective lens to focus the lens on the sample plane. One device that could be used as an actuator is a stepper motor with a captive lead screw configured to raise or lower the objective lens as the stepper motor rotates.

The system shown in Figure 1 may include a means of automatically loading a sample for analysis. Loading the samples is outside the scope of this invention and therefore not illustrated. Example means of positioning samples for analysis can be found in prior art. The laser- scanning cytometer (LSC) system is controlled by a computer (101) with software implementing the method described in this invention disclosure.

Figure 2 illustrates one possible embodiment of the LSC system.

A laser (203) is used to illuminate particles in the sample, e.g., positioned on the sample plane. The laser is combined into the optical path using a dichroic beam splitter (202) which reflects light at the wavelength of the laser and transmits light at other wavelengths.

An X-Y Scan Mirror (204) that is controlled by the computer aims the laser at different points within the sample plane. The X-Y Scan Mirror (204) could comprise a pair of galvanometer-positioned mirrors oriented to rotate in directions orthogonal to one another or a single mirror mounted on orthogonal gimbals able to rotate around two axes independently. In another embodiment, a rotating mirror with polygonal faces could be used to aim the laser at different points within the sample plane. The objective lens (102) focuses light from the laser to a point lying in the sample plane. By raster-scanning the laser across the entire field of view of the objective lens, the system is able to image particles in the sample plane.

The objective lens collects light emanating from particles in the sample and focuses it onto an image plane. The image plane of the objective lens is coincident with the front surface of the confocal pinhole aperture (212) and aligned so that the image of the point illuminated by the laser is coincident with the pinhole aperture. One or more detectors (one is shown here; 210) measure the fluorescence light from the particle at each point measured in the sample plane, thereby forming an image of the sample. It will be appreciated that the objective lens, scan lens (205), and focusing lens (201) work in combination to create an image of the sample plane on the front surface of the pinhole aperture. Suitable choices of these lenses are readily apparent to those familiar with design of optical systems such as microscopes.

A beamsplitter (213) reflects a small amount of light collected by the objective lens towards the CMOS camera (200). In this embodiment the beamsplitter reflects approximately 10% of the light from the optical path towards the CMOS camera and transmits approximately 90% of the light to the fluorescence detector (210).

A focusing lens (201) is used to create an image of the sample plane on the detector surface inside the CMOS camera. The CMOS camera creates digital images and communicates these to the computer for analysis.

In the embodiment of Figure 2, the position of the reflection of the laser as observed through the CMOS camera appears constant in space because as the scan mirror moves to aim the laser towards different points in the sample, the area projected onto the CMOS camera detector surface moves in tandem with the laser beam. Thus, while the laser scans across the sample plane, the reflection of the laser as imaged by the CMOS camera appears to be a point that is static in space. Although the dichroic beamsplitter (202) reflects a majority of the laser light, between 1% and 10% of the laser light will be transmitted through the dichroic and subsequently directed towards the CMOS camera by the beamsplitter (213). This quantity of light is sufficient for the CMOS camera to measure.

Figure 3 illustrates another embodiment of the LSC system that implements this invention. In Figure 3, the position of the beamsplitter and CMOS camera have been moved. In Figure 3, the CMOS camera images a field of view of the objective lens that remains constant while the laser scans the sample. While the laser scans the sample, the reflection of the laser as imaged by the CMOS camera as positioned in Figure 3 will also change.

Either of the embodiments shown in Figure 2 and Figure 3 would be acceptable implementations of this invention.

Figure 4a shows the image of the reflection of the laser off of the upper surface of the sample vessel (the surface upon which the microscopic particles are positioned) as the position of the objective lens in Figure 2 is changed with respect to the position of the sample vessel (which remains fixed in space). As the objective lens nears the position with the best focus, the image of the laser beam narrows to a point. Because the objective lens focuses the laser beam while also focusing light emanating from points in the focal plane, the position where the laser beam is focused to the smallest point coincides with the position where the objective lens is focused on the plane of a surface of the sample vessel that reflects light (which could be the upper surface or the lower surface of the sample vessel).

A focus score may be calculated for the images obtained from the CMOS camera at each changed position of the objective lens. One possible method of calculating a focus score that exhibits a maximum value when the objective lens focal plane coincides with a surface of the sample vessels is specified here:

1. For each pixel in the image, determine if the pixel intensity exceeds a threshold. The threshold could be set at 10% of the maximum possible intensity measured by the CMOS camera.

2. If the pixel exceeds the threshold: a. add the pixel intensity value to a quantity called the Aggregate Intensity; and b. increment the count of pixels that exceed the threshold.

3. After processing each pixel in the manner described in steps 1 and 2, divide the Aggregate Intensity by the count of pixels in the image that exceed the threshold.

This quotient is the focus score for the image. Figure 4b shows the focus score as determined by this method obtained as the objective lens is positioned at different heights with respect to the sample plane. The focus score reaches a maximum value when the objective lens is focused on the lower surface of the sample vessel. The interface between the sample vessel (which is glass or clear plastic) and air is highly reflective compared to the interface between the sample vessel and the liquid in the sample (which is primarily water). It will be appreciated that this phenomenon is caused by the larger difference in refractive indices of air and glass (or plastic) compared to the smaller difference in refractive indices between glass (or plastic) and water for light in the visible spectrum.

For reference, typical refractive indices of the materials relevant to this invention are listed in Table 1. The exact values of these indices are not important to the understanding of this invention.

Table 1 Refractive indices of select optical materials at 600nm

Figure 5 illustrates an algorithm implementing the method of autofocusing described in this invention disclosure. The steps in the algorithm include the following:

5a: Position the objective lens at a position known to be below the point where the objective lens is focused on the lower surface of the sample vessel, accounting for uncertainty in the estimate of this position. This position is specified in figure 7a.

5b: Turn on the laser and begin to oscillate the scan mirror in one axis. The scan mirror should oscillate at a speed that is fast enough that the point illuminated by the laser in the focal plane of the objective lens moves by a distance that is greater than the apparent size of the reflection of the laser beam, and ideally several diameters of the reflection of the laser beam. It will be appreciated that the exposure time used for the CMOS camera and the amplitude of the angle traversed by the scan mirror will factor into the oscillation speed necessary to meet these criteria.

5c: Begin capturing images using the CMOS camera of the reflection of the laser beam off of the surface of the sample vessel. The reflection will initially be a large spot of relatively low intensity because the objective lens is not focused on any surface.

5d: Begin raising the height of the objective lens with respect to the sample plane while recording images using the CMOS camera at height increments. The height increments could be limited to be no larger than the required focusing accuracy of the LSC system, which will vary according to the type of sample being analyzed and the type of analysis being performed. As the focal plane of the objective lens approaches the lower surface of the sample vessel, the reflection will increase in brightness and the size of the illuminated spot will shrink to a minimum. The reflection will then diminish in brightness and become less sharp until the focal plane of the objective lens approaches the upper surface of the sample vessel, at which point the reflection will again become brighter and the size of the illuminated spot will shrink to a minimum. As the objective lens continues to move upwards, the illuminated spot will again become less bright and less sharp as the focal point of the objective lens moves past the upper surface of the sample vessel and into the liquid of the sample.

5e: Stop moving the objective lens at a location where the focal point is known to be above the sample plane (shown in figure 7a), accounting for uncertainty in the estimate of that position.

5f: For the images collected at each height of the objective lens, calculate a focus score as described above.

5g: Identify the lower surface of the sample vessel. The lower surface is identified by finding the first maximum of the focus score encountered as the objective lens is raised from the start position of 5c. It will be appreciated that the focus scores of the images may be smoothed using a moving average or other method of filtering so that small variations in the focus score are not counted as maxima.

5h. Identify the upper surface of the sample vessel by finding the maximum focus score after the objective lens is raised above the lower surface. As in 5g, smoothing or filtering the focus score data may be necessary to avoid incorrectly identifying a small local variation in the focus score as a maximum focus score.

5i: Identify the sample plane by adding to the height of the upper surface of the sample vessel a distance that is approximately one half the thickness of the particles in the sample, so that the focal plane of the objective lens transects the midpoint of particles lying on the surface of the sample vessel. For small particle sizes this step may be omitted and the sample plane may be defined to be the same as the upper surface of the sample vessel. The approximate thickness of the particles is known a priori by the user.

5j : Move the objective lens to the location determined in step 5i. In one embodiment, the operator may turn off the laser and stop the scan mirror.

The algorithm illustrated in Figure 5 is useful where the position of the lower and upper surfaces of the sample vessel are not known a priori with a high degree of accuracy. Because there is no surface below the lower surface of the sample vessel off which the laser can reflect, the method will find this surface very reliably by starting the objective lens at a point sufficiently low that the focal point of the objective lens is known to be below the sample vessel.

Figure 6 illustrates an alternative algorithm embodying the method of this invention that only searches for the upper surface of the sample vessel. The upper surface of the sample vessel is the surface upon which the particles may be positioned, assuming the particles are denser than the sample liquid and have settled under the force of gravity. This algorithm is faster than the algorithm illustrated in Figure 5 because it searches over a smaller range of objective lens positions. Because the interface between the liquid of the sample and the air above the sample could be confused with one of the other surfaces, this method requires an initial estimate of the height of the sample vessel such that the focal point of the objective lens is known to start above the upper surface of the sample vessel but within the liquid of the sample. The initial estimate of the height of the sample vessel is known a priori by the operator and supplied to the algorithm as an input.

In one or more embodiments, after the coarse focusing steps of Figure 5 are performed, subsequent samples may be focused faster utilizing the steps of Figure 6, for example by performing for each additional sample: positioning the objective lens above the sample plane; turning on the light source and beginning to oscillate the scan mirror in said one axis; capturing the images using the CMOS camera of the reflection of the light source off of surfaces of the sample vessel; moving the objective lens closer to the sample plane while recording the images using the CMOS camera at the height increments; stopping the movement of the objective lens at the location where the focal plane is known to be below the sample plane but above the lower surface of the sample vessel; calculating the focus score using the images collected at each height of the objective lens; identifying the upper surface of the sample vessel by finding the first maximum of the focus score encountered as the objective lens is moved from the start position; identifying the sample plane location by adding to the height of the upper surface of the sample vessel the distance that is approximately one half the thickness of the particles in the sample; moving the objective lens to the sample plane location.

Figure 7a shows the initial of the objective lens and focal plane of the objective lens with respect to the sample vessel during the algorithm specified in Figure 5. Figure 7b shows the final position of the objective lens and focal plane. At the start of the procedure the focal plane is located in the air beneath the sample vessel. At the end of the procedure the focal plane is located in the liquid of the sample located above the upper surface of the sample vessel.

Figure 7c shows the initial position of the objective lens and focal plane of the objective lens with respect to the sample vessel during the algorithm specified in Figure 6. Figure 7d shows the final position of the objective lens and focal plane. At the beginning of the procedure the focal plane is located in the liquid of the sample located above the upper surface of the sample vessel. At the end of the procedure the focal plane is located within the material (either glass or clear plastic) proximate to the lower surface of the sample vessel. The lower surface of the sample vessel could be as thin as approximately 100 microns, in the case of microtiter plates with thin-film plastic bottom surfaces, or as thick as more than a millimeter in the case of glass microscope slides or microtiter plates with injection-molded plastic bottom surfaces.

It will be appreciated that the sample vessel could take multiple forms. The sample vessel shown in the figures represents a single well of a multi-well microtiter plate. Alternately, the sample vessel could be a microscope slide with the sample positioned on top of the slide (either with or without a coverslip). Other configurations are also possible.

It will be appreciated that moving the sample vessel relative to the objective lens would be substantially equivalent to moving the objective lens relative to the sample with regard to this invention, and that moving either the sample vessel or the objective lens are effective embodiments of this invention.

BENEFITS OF THE INVENTION

This method and apparatus offer multiple benefits, listed below.

1. Because the point where the laser reflects off of surfaces is constantly moving (scanning) in a direction orthogonal to the direction of motion of the objective lens during the autofocusing process, inaccuracy caused by imperfections in the surfaces of the sample vessel are averaged out.

2. Autofocusing methods using a static laser reflection would result in inaccurate performance if a particle in the sample coincided with the reflection of the laser beam, since the particle creates additional unexpected surfaces. This invention does not suffer from interference of particles coinciding with the laser beam because the laser beam scans a large path relative to the size of the particles in the sample during the focusing process and averages out the effect of one or more particles lying in the path of the laser beam.

3. The effects of diffraction and interference of the laser beam, which is coherent, off of surfaces is minimized because the diffraction will change as the laser scans across the surfaces of the sample vessel and will average out over the exposure time of each image. Diffraction and interference patterns from a static laser beam would create artifacts that confuse autofocusing algorithms.

4. The method requires the addition of a monochrome digital camera, a focusing lens and a beamsplitter, none of which is expensive.

5. The method requires very little laser power. The illumination from the laser can be kept at a level that will not photobleach the fluorescence of sensitive samples.

6. The accuracy of this method derives from the rate at which the laser beam converges to a minimum. This rate of convergence can be adjusted by changing the diameter of the laser beam to make the focusing more or less sensitive to position of the objective lens. It will be readily understood that large diameter beams focused by the objective lens will converge to a smaller minimum (known as the waist of the beam) than smaller diameter beams.

7. The method can be made arbitrarily fast by choosing a camera with a sufficiently high frame rate and a sufficiently fast actuator; and 8. The method only requires liquid to be present in the sample vessel but does not require particles to be present in the sample. Many focusing methods, such as methods based on image contrast, require a minimum amount of objects in the focal plane of the objective lens. These methods fail if either there are too few objects to focus on, or if there are objects above or below the focal plane that come into focus at different objective lens heights.