BACKGROUND

[0001] Fluid droplets are utilized in a variety of applications such as printing, additive manufacturing, environmental testing and biomedical diagnostics. For example, such fluid droplets may comprise an ink, a binder or other similar materials with respect to printing and additive manufacturing. With respect to environmental testing and biomedical diagnostics, such fluid droplets may comprise a reactant, a stain or an analyte. In many applications, the provision of the fluid droplet is automated through the use of a fluid ejector.

BRIEF DESCRIPTION OF THE DRAWINGS

[0002] FIG. 1 is a block diagram illustrating portions of an example computer-readable medium containing example imaging and fluid ejection control instructions.

[0003] FIG. 2 is a block diagram schematically illustrating portions of an example imaging and fluid ejection control system.

[0004] FIG. 3 is a flow diagram of an example imaging and fluid ejection control method.

[0005] FIGS. 4A, 4B and 4C are side views schematically illustrating portions of an example imaging and fluid ejection control system during the deposition of different droplets onto a deposition site.

[0006] FIG. 5A is a sectional view schematically illustrating portions of an example imaging and fluid ejection control system. [0007] FIG. 5B is a bottom view of the example system of FIG. 5A taken along line 5B-5B.

[0008] FIGS. 6A, 6B and 6C are side views schematically illustrating portions of an example imaging and fluid ejection control system during the deposition of different droplets onto a deposition site.

[0009] Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements. The figures are not necessarily to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or implementations consistent with the description; however, the description is not limited to the examples and/or implementations provided in the drawings.

DETAILED DESCRIPTION OF EXAMPLES

[00010] Disclosed are example imaging and fluid ejection control instructions, methods and systems that alter non-location parameters of droplets being ejected onto a deposition site in response to changing conditions at the deposition site. The instructions methods and systems carry out imaging of the deposition site and ejection of a fluid droplet both while the deposition site is supported at a single imaging location. As a result, the non location parameters of the droplets may be determined and adjusted based upon captured images of the deposition site with a previously deposited first droplet without repositioning or realignment of the deposition site with respect to an imager or with respect to a different fluid ejector that ejects a second droplet.

[00011] In some implementations, the first droplet is ejected while the deposition site is at the imaging location and the deposition site with the first droplet is also imaged while the deposition site is at the imaging location. The captured image of the deposition site with the first droplet is used to determine a non-location parameter for a second droplet to be subsequently deposited onto the deposition site. Thereafter, the deposition site may be relocated for receiving a second droplet having the determined non-location parameter.

[00012] In some implementations, the first droplet is ejected onto the deposition site and the deposition site is relocated to the imaging location. While at the imaging location, the deposition site is imaged. The captured image of the deposition site with the first droplet is used to determine a non location parameter for a second droplet to be separately deposited onto the deposition site. Thereafter, while the deposition site remains at the imaging location, the second droplet is deposited onto the deposition site.

[00013] In some implementations, the first droplet is ejected while the deposition site is at the imaging location and the deposition site with the first droplet is also imaged while the deposition site is at the imaging location. The captured image of the deposition site with the first droplet is used to determine a non-location parameter for a second droplet to be subsequently deposited onto the deposition site. Thereafter, the second droplet is ejected onto the deposition site while the deposition site remains at the same imaging location. [00014] In the various disclose examples, a fluid ejector in an imager may be concurrently aimed at a deposition site such that imaging of the deposition site with a deposited droplet may occur without movement of the deposition site. For purposes of this disclosure, the concurrent “aiming” of a fluid ejector and imager towards a deposition site means that an individual nozzle opening of a fluid ejector extends generally opposite to the deposition site such that a droplet ejected by the fluid ejector will travel in a direction generally perpendicular to the target so as to land on the deposition site and that the field-of-view of the imager concurrently encompasses and is focused upon the deposition site without movement of the target, the fluid ejector and/or the imager relative to one another. In some implementations, the field- of-view of the imager encompasses a less than total portion of the target. In an example implementation, the field-of-view [please insert a metric or quantitative value indicating the extent to which the imager is focused on the relatively small area of the deposition site (the objective to distinguish over a camera that simply captures the entire media or target]).

[00015] For purposes of this disclosure, a “non-location parameter” for a droplet refers to a characteristic of the droplet other than the current physical location or trajectory of the droplet or the targeted location or targeted trajectory of the droplet. In other words, a non-location parameter is unrelated to the accuracy of the location at which an ejected droplet finally lands. Examples of “non-location parameters” include, but are not limited to, the volume of liquid forming the droplet, the temperature of the liquid of the droplet, the chemical composition of the liquid of the droplet.

[00016] The time at which the droplet is ejected may, in some circumstances, constitute a “non-location parameter” where the time parameter is unrelated to the targeted location or trajectory of the droplet. For example, time may constitute a “non-location parameter” where the time parameter relates to a frequency at which multiple droplets are being ejected or the time between the consecutive ejection of fluid droplets. Time may constitute a “non-location parameter” where the time parameter is not dependent upon the rate of movement of the deposition site, but is dependent upon the current state of the deposition site such as the state of an ongoing process at the deposition site. Examples of an ongoing process that may be taking place at the deposition site include chemical reactions, the absorption of materials, the expulsion or emission of materials, biological activities and material phase changes such as freezing, melting, sublimation and evaporation.

[00017] In some implementations, the captured images may be utilized to further adjust the application of a stimulus or multiple stimuli to the deposition site. Examples of stimuli include, not limited to, electromagnetic radiation such as light, visible and invisible, heat, gas, a velocity of the applied gas, electrical charge, vibrational movement and combinations thereof. In some implementations, a first stimulus or group of stimuli may be applied to the deposition site prior to the imaging of the deposition site and a second stimulus or group of stimuli, different than the first stimulus or stimuli, may be applied to the deposition site based upon the imaging of the deposition site.

In some implementations, the first stimulus or first stimuli and the second stimulus or second stimuli may both be applied to the deposition site while the deposition site is at the imaging location.

[00018] Disclosed is an example non-transitory computer-readable medium containing imaging and fluid ejection control instructions to direct operation of a processor. The instructions may comprise first droplet ejection instructions to cause ejection of a first droplet towards a deposition site, image acquisition instructions to acquire signals representing an image of the deposition site supported at an imaging location following receipt of the first droplet, parameter determining instructions to cause determining of a non location parameter for a second droplet based upon the image and second droplet ejection instructions to cause ejection of the second droplet with the non-location parameter towards the deposition site, wherein the ejection of one of the first droplet and the second droplet towards the deposition site occurs while the deposition site is supported at the imaging location.

[00019] Disclosed is an example imaging and fluid ejection control method that may include ejecting a first droplet onto a deposition site, imaging the deposition site following receipt of the first droplet and while the deposition site is supported at an imaging location, determining a non-location parameter for a second droplet based upon the imaging of the deposition site following receipt of the first droplet and ejecting the second droplet having the non location parameter onto the deposition site, wherein the ejecting of one of the first droplet and second droplet is to occur while the deposition site is supported at the imaging location.

[00020] Disclosed is an example imaging and fluid ejection control system. The system may comprise a fluid ejector to eject a first droplet onto a deposition site, an imager to capture an image of the deposition site following receipt of the droplet and while the deposition site is supported at an imaging location, a processor and a non-transitory computer readable medium containing instructions. The instructions may direct the processor to determine a non-location parameter for a second droplet based upon the image and output control signals to cause ejection of the second droplet having the non-location parameter towards the deposition site. The ejection of one of the first droplet and the second droplet towards the deposition site is to occur while the deposition site is supported at the imaging location.

[00021] FIG. 1 is a block diagram schematically illustrating portions of an example non-transitory computer-readable medium 32. Medium 32 contains instructions for directing a processor to carry out various actions. Medium 32 comprises first droplet ejection instructions 34, image acquisition instructions 36, parameter determining instructions 38 and second droplet ejection instructions 40.

[00022] First droplet ejection instructions 34 comprise instructions that direct the processor to output control signals that cause ejection of a first droplet towards a deposition site, a location on a target. Such ejection may be carried out using a fluid actuator that displaces fluid within a fluid chamber through an ejection orifice or nozzle opening. In one implementation, the ejection of the droplet is using a thermal resistive fluid ejector. In other implementations, the ejection of the droplet may be by other fluid injectors. The deposition site may be a landing spot on a substrate or slide, a landing spot in a well of a well plate, a landing spot on a living organism or other locations.

[00023] Image acquisition instructions comprise instructions that direct the processor to acquire signals representing an image of the deposition site while the deposition site is supported at an imaging location and following receipt of the first droplet. The image of the deposition site, represented by the signals, may be captured by an imager having a field-of-view that encompasses the deposition site at the imaging location. In some implementations, the imaging location is the same location at which the deposition site was located when the deposition site received the first droplet. In other implementations, the imaging location may be a different location of the deposition site, following movement of the deposition site from the location at which the deposition size was located when the deposition site received the first droplet. For example, the different location may be a location at which the deposition site is to receive a second droplet of fluid.

[00024] Parameter determining instructions comprise instructions that direct the processor to determine a non-location parameter for a second droplet, to be ejected onto the deposition site, based upon the image acquired pursuant to the image acquisition instructions.

[00025] Second droplet ejection instructions comprise instructions that cause the processor droplet control signals that cause ejection of the second droplet with the non-location parameter towards the deposition site. The non location parameter of the second droplet may be different as compared to the same non-location parameter of the first droplet, based upon the image. For example, the first droplet may have a first volume of the second droplet has a second different volume. The first droplet may have a first temperature while the second droplet has a second different temperature. The first droplet may have a first chemical composition or makeup while the second droplet has a second different chemical composition or makeup. The first droplet fluid may have been ejected as part of a series of ejections having a first frequency of the second droplet may be ejected as part of a series of ejections having a second different frequency.

[00026] In some implementations, the ejection of the second droplet is carried out using the same fluid ejector that ejected the first droplet. The characteristics of the fluid being ejected by the same fluid ejector may be altered prior to ejection of the second droplet. In some implementations, the fluid being ejected by the same fluid ejector may be heated or cooled. In some implementations, the same fluid ejector may be operated in a different fashion, supplied was a different electrical pulse, such that the second droplet has a greater volume. In some implementations, a valve mechanism may be actuated to supply a different fluid having a different chemical composition to the same fluid ejector prior to ejection of the second droplet.

[00027] In some implementations, the ejection of the second droplet is carrying out using a different fluid ejector as compared to the fluid ejector that ejected the first droplet. Following deposition of the first fluid droplet, a carriage, table or other support may be moved by an actuator to relocate the deposition site to a different location for receiving the second fluid droplet. As indicated above, the ejection of the first droplet or the ejection of the second droplet towards a deposition site occurs while the deposition site is supported at the imaging location. The imaging of the deposition site occurs either following ejection of the first droplet without movement of the deposition site or immediately prior to ejection a second droplet without movement of the deposition site. As a result, the selection or modification of the non-location parameter of the second droplet may be more timely determined such that the selection or modification of the non-location parameter is more in tune with the current conditions of the deposition site following ejection of the first droplet or immediately prior to ejection of the second droplet.

[00028] FIG. 2 is a diagram schematically illustrating portions of an example imaging and fluid ejection control system 120. System 120 comprises fluid ejector 124, imager 126, processor 128 and a non-transitory computer-readable medium 32 (described above) containing instructions for processor 128. Fluid ejector 124 comprises a device to selectively eject fluid droplets towards and onto a deposition site 140. In an example implementation fluid ejector 124 is electrically powered and controlled through the transmission of electrical signals. In an example implementation, fluid ejector 124 comprises a fluid ejection chamber that is supplied with fluid from a fluid reservoir, the fluid to be ejected by a fluid actuator that is selectively actuated to displace fluid within the chamber through an ejection orifice or nozzle opening.

[00029] In an example implementation, the fluid actuator may comprise a thermal resistor which, upon receiving electrical current, heats to a temperature above the nucleation temperature of the fluid so as to vaporize a portion of the adjacent fluid to create a bubble which displaces the fluid through the associated orifice. In other implementations, the fluid actuator may comprise other forms of fluid actuators. In other implementations, the individual fluid actuators may be in the form of a piezo-membrane based actuator, an electrostatic membrane actuator, mechanical/impact driven membrane actuator, a magneto-strictive drive actuator, an electrochemical actuator, and external laser actuators (that form a bubble through boiling with a laser beam), other such microdevices, or any combination thereof.

[00030] Imager 126 comprises a device that images the deposition site 140 by capturing an image or images of the deposition site 140, before deposition of a droplet by fluid ejector 124, during deposition of the droplet by fluid ejector 124 and/or following deposition of the droplet by fluid ejector 124. In an example implementation, imager 28 may comprise a lens which focuses light or the image of the deposition site onto an imaging array. In an implementation, the lens may comprise a flat lens. Particular examples of the lens include Fresnel lenses, zone plate lenses and meta-lenses. The lens may include an amplitude mask for computational imaging. The imaging array may comprise a complementary metal-oxide-semiconductor (CMOS), a charge coupled device (CCD) sensor array or other types of imaging devices or arrays.

[00031] Processor 128 carries out instructions read from or provided by computer readable medium (CRM) 32. As described above, the various instructions on medium 32 direct processor 128 to output control signals causing first and second droplets 142-1 and 142-2, respectively, to be ejected onto deposition site 140. The instructions on medium 32 further direct imager 126 to capture an image of the deposition site 140. The instructions on medium 32 direct processor 128 to analyze the captured image on a pixel by pixel basis or in other fashions with optical recognition so as to identify a current state of the deposition site 140. This step may include a breadth of image processing methods and their combinations (e.g. color histogram, intensity in region of interest, segmentation, background, subtraction, template alignment, pattern recognition via machine learning and deep learning, and the like). For example, the instructions on medium 32 may direct the processor 128 to identify the current state of an ongoing process that may have started or that may be taking place at the deposition site following deposition of the first droplet 142-1 , such as chemical reactions, the absorption of materials, the expulsion or emission of materials, biological activities and material phase changes such as freezing, melting, sublimation and evaporation. Based upon such analysis, the instructions on medium 32 further direct processor 128 to determine a non-location parameter for a second subsequent droplet 142-2 to be ejected onto the deposition site 140.

[00032] As further described above, in some implementations, the imaging location for deposition site 140 is the same location at which the deposition site 140 was located when the deposition site received the first droplet 142-1. In other implementations, the imaging location may be a different location of the deposition site, following movement of the deposition site from the location at which the deposition site was located when the deposition site received the first droplet. For example, the different location may be a location at which the deposition site is to receive the second droplet 142-2. As also described above, in some implementations, the second droplet 142-2 is ejected by a different fluid ejector 124’. In other implementations, the second droplet 142-2 is ejected by the same fluid ejector 124 that ejected the first droplet 142-1. [00033] FIG. 3 is a flow diagram of an example imaging and fluid ejection control method 200. Method 200 facilitates more timely selection or modification of a non-location parameter of a second droplet to be ejected such that the non-location parameter is more in tune with the current conditions of the deposition site following ejection of a first droplet or immediately prior to ejection of the second droplet. Although method 200 as described in the context of being carried out by system 120, it should be appreciated method 200 may likewise be carried out with any of the systems described below or with other similar systems.

[00034] As indicated by block 204, fluid ejector 124 ejects a first droplet 142-1 onto a deposition site 140. As indicated by block 208, following receipt of the first droplet and while the deposition site is supported at an imaging location, deposition site 140 is imaged. Imager 126 captures images of the deposition site for analysis.

[00035] As indicated by block 212, instructions 32 direct a processor, such as processor 128, to determine a non-location parameter for a second droplet 142-2, to be ejected onto deposition site 140, based upon the imaging of the deposition site 140 following receipt of the first droplet. As indicated by block 216, a second droplet 142-2 having the non-location parameter is ejected onto the deposition site 140. Ejection and deposition of the first droplet 142-1 and/or the second droplet 142-2 occurs while the deposition site 140 is supported at the imaging location, the location at which deposition site 140 was imaged by imager 126. The ejection of the second droplet 142-2 may be by the same fluid ejector 124 that ejected the first fluid droplet 142-1 or may be ejected by a different fluid ejector 124’.

[00036] FIGS. 4A, 4B and 4C are side view schematically illustrating portions of an example imaging and fluid ejection control system 320 during the ejection of two droplets. System 320 comprises fluid ejectors 324-1 , 324- 2, 324-3 (collectively referred to as fluid ejectors 324), fluid supplies 325-1 ,

325-2, 325-3 (collectively referred to as fluid supplies 325), imagers 326-1,

326-2, 326-3 (collectively referred to as imagers 326), stimulus applicators 327-1, 327-2, 327-3 (collectively referred to as stimulus applicators 327), deposition site 140, actuator 344 and controller 350.

[00037] Fluid ejectors 324 each similar to fluid ejectors 124 described above except that fluid ejectors 324 have architectures to eject volumes V1 , V2, and V3, respectively. In some implementations, volumes V1 , V2 and V3 are the same. In other implementations, some of volumes V1 , V2 and V3 are different.

[00038] Fluid supplies 325-1 , 325-2 and 325-3 supply fluid to fluid ejectors 324-1 , 324-2 and 324-3, respectively. As shown by broken lines, the fluid supplied by the different fluid supplies 325 may have characteristics or non-location parameters. As schematically indicated, fluid supply 325-1 supplies a fluid having a temperature T1 and a chemical composition C1.

Fluid supply 325-2 supplies the second fluid having a temperature T2 and a chemical composition C2. Fluid supply 325-3 supplies the second fluid having a temperature T3 and a chemical composition C3.

[00039] In some implementations, temperature T3 is different than temperature T2. In some implementations, compositions C3 is different than composition C2. In some implementations, both temperature T3 and composition C3 are different than temperature T2 and composition C2, respectively. In some implementations, both of temperatures T3 and T2 are different than temperature T1. In some implementations, both of compositions C3 and C2 are different than composition C1. In some implementations, temperature T2 and composition C2 may be the same as temperature T1 and composition C1 while one or both of temperature T3 and composition C3 are different than temperature T1 and composition C1 , respectively.

[00040] Imagers 326 are each similar to imager 126 described above. Imager 326-1 is to capture an image of the deposition site 140 while deposition site 140 is at the imaging location opposite to fluid ejector 324-1 (shown in FIG. 4A). Imagers 326 and their respective associated ejectors 324 are concurrently aimed at the same deposition spot or site 140. Imager 326-1 facilitates the capturing of images of deposition site 140 before, during and/or after a first fluid droplet 142-1 has been deposited upon deposition site 140 and without any movement of deposition site 140 between the deposition of droplet 142-1 in the imaging of deposition site 140 with the deposited droplet 142-1. Imager 326-2 is to capture an image of the deposition site 140 while deposition site 140 is at the imaging location opposite to fluid ejector 324-2. Imager 326-1 facilitates the capturing of images of deposition site 140 before, during and/or after a first fluid droplet 142-2 has been deposited upon deposition site 140 and without any movement of deposition site 140 between the deposition of droplet 142-2 and the imaging of deposition site 140 with the deposited droplet 142-2. Imager 326-3 is to capture an image of the deposition site 140 while deposition site 140 is at the imaging location opposite to fluid ejector 324-3. Imager 326-1 facilitates the capturing of images of deposition site 140 before, during and/or after a first fluid droplet 142-3 has been deposited upon deposition site 140 and without any movement of deposition site 140 between the deposition of droplet 142-4 in the imaging of deposition site 140 with the deposited droplet 142-3.

[00041] Stimulus applicators 327-1 , 327-2, 327-3 comprise devices that are to apply stimulus to deposition site 140 when deposition site 140 is at the first imaging location 345-1 shown in FIG. 4A, when deposition site 140 is at the second imaging location 345-2 shown in FIG. 4B and when deposition site 140 is at the third imaging location 345-3 shown in FIG. 4, respectively. Examples of stimuli include, but are not limited to, electromagnetic radiation such as light, visible and invisible, heat, gas, a velocity of the applied gas, electrical charge, vibrational movement and combinations thereof. In some implementations, a first stimulus or group of stimuli may be applied to the deposition site 140 prior to the imaging of the deposition site and a second stimulus or group of stimuli, different than the first stimulus or stimuli, may be applied to the deposition site 140 based upon the imaging of the deposition site. In some implementations, the first stimulus or first stimuli and the second stimulus or second stimuli may both be applied to the deposition site while the deposition site is at a single imaging location.

[00042] As shown by broken lines, in some implementations, fluid ejectors 324 and imagers 326 may be provided as part of a single package or unit, such as a single die 329. In some implementations, fluid sources 325, in the form of fluid reservoirs, may also be provided as part of the single package or die 329. In some implementations, when provided, stimulus applicators 327 may also be provided as part of the single package or die 329. In such an implementation, the fluid ejectors 325 and imagers 326 may be partially encapsulated by a packaging material such as a moldable compound that is been solidified or hardened about the individual fluid ejection dies and/or imagers 326. In other implementations, each fluid ejector 324 and is associated imager 326 may be formed as individual packages or dies. For example, fluid ejector 324-1 and imager 326-1 may be formed as a first die, fluid ejector 324-2 and imager 326-2 may be formed of a second different die and fluid ejector 324-3 and imager 326-3 may be formed as a third die. In yet other implementations, each of the fluid ejectors and imagers may be separate individual units or packages.

[00043] Actuator 344 comprises a powered device that is to move deposition site 140 between the first imaging station or position 345-1 , the second imaging station or position 345-2 and the third imaging station or position 345-3. In one implementation, actuator 344 may comprise an electric motor, and electric solenoid, hydraulic or pneumatic cylinder-piston assembly or the like operably coupled to deposition site 140 by a transmission. For example, in one implementation, deposition site 140 and actuator 344 may be part of an x-y table that is controlled movable between a multitude of different positions relative to different fluid ejectors such as fluid ejectors 324.

[00044] As shown by broken lines, in some implementations, actuator 344 may be operably coupled to fluid ejectors 324 and imagers 326 so as to move fluid ejectors 324 and imagers 326 relative to deposition site 140, wherein deposition site 140 remains stationary during such movement. In some implementations, actuator 344 may concurrently move both deposition site 140 and fluid ejectors 324 relative to one another. In yet other implementations, actuator 344 may be omitted such as where deposition site 140 is manually located or repositioned or where deposition site 140 is part of a live biological creature, where imagers 326 sense the positioning and movement of the creature to determine when to eject fluid droplets so as to land droplets 142 onto a specific targeted location at the deposition site 140. [00045] Controller 350 controls the relative movement of deposition site 140 relative to fluid ejectors 324 and imagers 326. Controller 350 further controls capturing of images by imagers 326 and the ejection of fluid by fluid ejectors 324. Controller 350 comprises a processor 128 and computer- readable medium 32 (described above). Medium 32 contains instructions 34, 36, 38 and 40 which direct processor 128 to carry out various actions.

[00046] In the example illustrated, as shown by FIG. 4A, controller 350 outputs control signals causing actuator 344 to locate deposition site 140 at the first imaging location 345-1 , wherein deposition site 140 and fluid ejector 324-1 are generally opposite to one another and wherein deposition site 140 is within the field of view of imager 326-1. Controller 350 outputs control signals causing fluid ejector 324-1 to eject a first droplet 142-1 onto deposition site 140. Droplet 142-1 comprises a liquid having a first non-location parameter such as the first volume V1 , the first temperature T1 and the first chemical composition C1. Following deposition of droplet 142-1 , controller 350 outputs control signals causing imager 326-1 to capture and provide an image or images of deposition site 140.

[00047] Following acquisition of the imager images captured by imager 326-1 , controller 350 analyzes the image or such images on a pixel by pixel basis or in other fashions with optical recognition so as to identify a current state of the deposition site 140. This step may include a breadth of image processing methods and their combinations (e.g. color histogram, intensity in region of interest, segmentation, background, subtraction, template alignment, pattern recognition via machine learning and deep learning, and the like). For example, the instructions on medium 32 may direct the processor 128 to identify the current state of an ongoing process that may have started or that may be taking place at the deposition site following deposition of the first droplet 142-1, such as chemical reactions, the absorption of materials, the expulsion or emission of materials, the staining of particular cells or other objects to distinguish such objects or indicate their presence, biological activities and material phase changes such as freezing, melting, sublimation and evaporation. Based upon such analysis, the instructions on medium 32 further direct processor 128 to determine a non-location parameter for a second subsequent droplet to be ejected onto the deposition site 140.

[00048] Based upon the determined non-location parameter for the second subsequent droplet to be ejected onto the deposition site 140, controller 350 outputs control signals causing actuator 344 to locate deposition site 140 with the deposited droplet 142-1 for receiving the second subsequent droplet having the determined non-location parameter. In the example illustrated, actuator 344 moves deposition site 140. As described above, in other implementations, actuator 344 may instead move die 329 relative to deposition site 140.

[00049] FIGS. 4B and 4C illustrate two alternative scenarios. FIG. 4B illustrates a scenario in which the determined non-location parameter for the second droplet is offered by fluid ejector 324-2. As a result, controller 350 outputs control signals causing actuator 344 to locate deposition site 140 at the second imaging location 345-2, wherein deposition site 140 and fluid ejector 324-2 are generally opposite to one another and wherein deposition site 140 is within the field of view of imager 326-2. Controller 350 outputs control signals causing fluid 324-2 to eject a second droplet 142-2 onto deposition site 140, onto the region upon which droplet 142-1 was previously deposited. Droplet 142-2 comprises a liquid having a second non-location parameter such as the second volume V2, the second temperature T2 and the second chemical composition C2. In an example implementation, one or more of the volume V2, the temperature T2 or the composition C2 may be different than the respective volume V1 , temperature T2 or composition C2, respectively, of the first droplet 142-1.

[00050] While at imaging location 345-2 and following deposition of droplet 142-2, controller 350 may output control signals causing imager 326-2 to capture images of deposition site 140. Following acquisition of the imager images captured by imager 326-2, controller 350 analyzes the image or such images on a pixel by pixel basis or in other fashions with optical recognition so as to identify a current state of the deposition site 140. This step may include a breadth of image processing methods and their combinations (e.g. color histogram, intensity in region of interest, segmentation, background, subtraction, template alignment, pattern recognition via machine learning and deep learning, and the like). For example, the instructions on medium 32 may direct the processor 128 to identify the current state of an ongoing process that may have started or that may be taking place at the deposition site following deposition of the second droplet 142-2, such as chemical reactions, the absorption of materials, the expulsion or emission of materials, biological activities and material phase changes such as freezing, melting, sublimation and evaporation. Based upon such analysis, the instructions on medium 32 further direct processor 128 to determine a second non-location parameter for a third subsequent droplet to be ejected onto the deposition site 140.

[00051] Based upon the determined non-location parameter for the third subsequent droplet to be ejected onto the deposition site 140, controller 350 outputs control signals causing actuator 344 to locate deposition site 140 with the deposited droplets 142-1 and 142-2 for receiving the third subsequent droplet from yet a third fluid ejector offering a droplet having the second non location parameter. In the example illustrated, actuator 344 moves deposition site 140. As described above, in other implementations, actuator 344 may instead move die 329 relative to deposition site 140. In some scenarios, the third fluid ejector may be fluid ejector 324-3, wherein deposition site 140 is subsequently moved to imaging location 345-3. In other scenarios, the third fluid ejector may be an additional fluid ejector (not shown) that may or may not be supported as part of the single die 329.

[00052] In some implementations, imager 326-2 may be used to capture additional images of deposition site 140 with the previously deposited droplet 142-1 immediately prior to the possible deposition of droplet 142-2 at imaging location 345-2. In such implementations, controller 350 may analyze the additional images to confirm that the current state of deposition site 140 has not sufficiently changed, since the time at which deposition site 140 was at imaging location 345-1, to warrant a change in the previously determined non location parameter of the second droplet to be ejected onto the deposition site. In response changes satisfying a predetermined threshold, controller 350 may identify a new second non-location parameter, different than the non-location parameter. Based upon the analysis of the additional images, controller 350 may once again locate deposition site 140 for receiving a droplet from a second different fluid ejector that offers the second non-location parameter. This process may be repeated until the state of the deposition site 140 immediately prior to the deposition of the second fluid droplet is appropriate, as determined by controller 350 based upon the captured images, for receiving the second fluid droplet without further repositioning of deposition site 140 and/or die 329.

[00053] FIG. 4C illustrates a scenario in which the determined non location parameter for the second droplet is offered by fluid ejector 324-3. As a result, controller 350 outputs control signals causing actuator 344 to locate deposition site 140 at the third imaging location 345-3, wherein deposition site 140 and fluid ejector 324-3 are generally opposite to one another and wherein deposition site 140 is within the field of view of imager 326-3. Controller 350 outputs control signals causing fluid 324-3 to eject a second droplet 142-3 onto deposition site 140, onto the region upon which droplet 142-1 was previously deposited. Droplet 142-3 comprises a liquid having a second non location parameter such as the second volume V3, the second temperature T3 and the second chemical composition C3. In an example implementation, the volume V3, the temperature T3 and/or the composition C3 may be different than the respective volume V1 , temperature T2 and/or composition C2, respectively, of the first droplet 142-1.

[00054] As described above with respect to FIG. 4B, while at imaging location 345-3 and following deposition of droplet 142-3, controller 350 may output control signals causing imager 326-3 to capture images of deposition site 140. Following acquisition of the image or images captured by imager 326-3, controller 350 analyzes the image or such images on a pixel by pixel basis or in other fashions with optical recognition so as to identify a current state of the deposition site 140. This step may include a breadth of image processing methods and their combinations (e.g. color histogram, intensity in region of interest, segmentation, background, subtraction, template alignment, pattern recognition via machine learning and deep learning, and the like). For example, the instructions on medium 32 may direct the processor 128 to identify the current state of an ongoing process that may have started or that may be taking place at the deposition site following deposition of the second droplet 142-3, such as chemical reactions, the absorption of materials, the expulsion or emission of materials, biological activities and material phase changes such as freezing, melting, sublimation and evaporation. Based upon such analysis, the instructions on medium 32 further direct processor 128 to determine a second non-location parameter for a third subsequent droplet to be ejected onto the deposition site 140.

[00055] Based upon the determined non-location parameter for the third subsequent droplet to be ejected onto the deposition site 140, controller 350 outputs control signals causing actuator 344 to locate deposition site 140 with the deposited droplets 142-1 and 142-3 for receiving the third subsequent droplet from yet a third fluid ejector offering a droplet having the second non- location parameter. In the example illustrated, actuator 344 moves deposition site 140. As described above, in other implementations, actuator 344 may instead move die 329 relative to deposition site 140. In some scenarios, the third fluid ejector may be fluid ejector 324-2, wherein deposition site 140 is subsequently moved to imaging location 345-2. In other scenarios, the third fluid ejector may be an additional fluid ejector (not shown) that may or may not be supported as part of the single die 329.

[00056] As shown by FIGS. 4A-4C, system 320 facilitates an automated sequential multi-droplet processing of a deposition site. After each droplet or group of droplets has been ejected onto deposition site, the resulting reaction or change may be captured and analyzed to determine the characteristics or non-location parameters of the next droplet or group of droplets that should be ejected onto the same deposition site. This process may be repeated multiple times in sequence. Because the reaction or state of the deposition site is captured by an imager following deposition of a droplet and without movement of the deposition site, the imaging of the resulting reaction or change in state is closer in time to the deposition of the prior droplet, resulting in faster and real-time feedback control.

[00057] In some implementations, the imaging of the deposition site may be triggered by controller 350 following deposition of the droplet 142-1 and following movement of the deposition site 140 (or movement of die 329). For example, following deposition of droplet 142-1, controller 350 may output control signals causing actuator 344 to locate deposition site 140 at imaging location 345-2. At such time, and prior to the deposition of droplet 142-2, controller 350 may direct imager 326-2 to capture an image of deposition site 140. As result, the imaging of deposition site 140 occurs at a time that may have allowed additional reaction time or time for other changes since the original deposition of droplet 142-1. In addition, imaging of deposition site 140 may occur closer in time to the possible deposition of the second droplet. [00058] Following acquisition of the images captured by imager 326-2, controller 350 may analyze the image or such images on a pixel by pixel basis or in other fashions with optical recognition so as to identify a current state of the deposition site 140. This step may include a breadth of image processing methods and their combinations (e.g. color histogram, intensity in region of interest, segmentation, background, subtraction, template alignment, pattern recognition via machine learning and deep learning, and the like). For example, the instructions on medium 32 may direct the processor 128 to identify the current state of an ongoing process that may have started or that may be taking place at the deposition site following deposition of the first droplet 142-1, such as chemical reactions, the absorption of materials, the expulsion or emission of materials, biological activities and material phase changes such as freezing, melting, sublimation and evaporation. Based upon such analysis, the instructions on medium 32 further direct processor 128 to determine a non-location parameter for a second subsequent droplet to be ejected onto the deposition site 140.

[00059] In response to fluid ejector 324-2 offering a droplet having the determined non-location parameter, controller 350 may output control signals causing fluid ejector 324-2 to eject and deposit a droplet or groups of droplets having the second non-location parameter onto the deposition site 140. In response to fluid ejector 324-2 failing to offer a droplet or group of droplets having the second non-location parameter, controller 350 may consult a database to identify a fluid ejector that does offer the non-location parameter. Controller 350 may output control signals causing actuator 344 to move deposition site 140 to a location for receiving a droplet from the identified fluid ejector that does offer the non-location parameter. As described above, in some implementations, actuator 344 may instead move die 329 to locate the identified fluid ejector that offers droplet or droplets with the non-location parameter, opposite deposition site 140. [00060] In such circumstances where fluid ejector 324-2 may not offer a droplet having the non-location parameter, such that one or both of deposition site 140 and die 329 are moved to locate deposition site 140 to receive a droplet having the non-location parameter from a first different fluid ejector (different than fluid ejector 342-2), controller 350 may output control signals causing the deposition site 140 to be once again imaged at the new location prior to ejection by the different fluid ejector. The newly captured images may be analyzed by controller 352 confirm that the state of the deposition site 140 has not sufficiently changed to warrant a change in the non-location parameter of the second droplet to be ejected onto the deposition site. In response to identifying a second non-location parameter, different than the non-location parameter, based upon the analysis of the newly captured images, controller 350 may once again relocate the position site 140 for receiving a droplet from a second different fluid ejector that offers the second non-location parameter. This process may be repeated until the state of the deposition site 140 immediately prior to the deposition of the second fluid droplet is appropriate, as determined by controller 350 based upon the captured images, for receiving the second fluid droplet without further repositioning of deposition site 140 and/or die 329.

[00061] FIGS. 5A and 5B schematically illustrate portions of an example fluid ejection and imaging system 420, an example implementation of system 320. System 420 comprises circuitry platform 450, fluid actuators 456-1 , 456- 2, 456-3 (collectively referred chest fluid actuators 456), imaging elements 463-1, 463-2, transparent substrate 464, lens 466-1, 466-2, target illuminators 468-1 , 468-2, packaging 470, target 472, actuator 474 and controller 350. In the example illustrated, portions of circuitry platform 450 and portions of transparent substrate 464 along with fluid actuators 456 form fluid ejectors 324-1 , 324-2 and 324-3. Portions of circuitry platform 450 and portions of transparent substrate 464 further form imagers 326-1 , 326-2 and 326-3. [00062] Circuitry platform 450 circuitry platform 1050 of system 1020 supports imaging elements 463-1 , 463-2, arranged as arrays, on opposite sides of fluid actuators 456. Circuitry platform 450 includes electrically conductive wires or traces for transmitting signals between controller 350 (described above) and imaging elements 463. Circuitry platform 450 further comprises transistors and other electronic componentry for powering and actuating elements 463. Circuitry platform 450 may be in the form of a thin film, a circuit board or a single electronic die.

[00063] Circuitry platform 450 further supports fluid actuators 456. Platform 450 incorporates electrically conductive traces, transistors and other electronic components for powering and controlling fluid actuators 456. The electrically conductive traces may connect electronic componentry, such as transistors, of circuitry platform 450 to controller 350. Each of fluid actuators 456 extends adjacent to or proximate an ejection chamber 480 which is fluidly connected to the respective one of fluid supplies 325. Each of ejection chambers 480 is associated with a fluid ejection orifice or nozzle opening 482 through which fluid droplets are ejected. Each of fluid actuators 456 with its associated ejection chamber 480 and orifice opening 42 forms one of fluid ejectors 324.

[00064] Imaging elements 463 comprise individual elements that receive light focused by lenses 466 through substrate 464 and outputs electrical signals to controller 350, representing captured images, based upon the received light. Imaging elements 463 may comprise a complementary metal- oxide-semiconductor (CMOS), a charge coupled device (CCD) sensor array or other types of imaging elements.

[00065] Transparent substrate 464 comprises a layer or multiple layers sandwiched between lenses 466 and imaging elements 463. Transparent substrate 464 spaces lenses 466 from imaging elements 463 to enhance focusing of the light from target 472 onto imaging on 463. In one implementation, transparent substrate 464 has a thickness of 20 microns or more. In some implementations, transparent substrate 464 has a thickness of no greater than 2 mm. For optical performance, transparent substrate 464 may have a thickness of 100-500 microns. In an example implementation, transparent substrate 464 may be formed from a transparent material such as SU8 ( a Bisphenol A Novolac epoxy that is dissolved in an organic solvent (gamma-butyrolactone GBL or cyclopentanone, depending on the formulation) and up to 10 wt% of mixed Triarylsulfonium/hexafluoroantimonate salt as the photoacid generator) , quartz or other optical glass material (i.e. Borosilicate BK7, Crown K5 and the like). In other implementations, transparent substrate 464 may be formed from other transparent materials or may have other thicknesses. In some implementations, transparent substrate 464 may be omitted.

[00066] Lenses 466 focus the light from target 472 through transparent substrate 464 and onto imaging elements 463. In an implementation, the lens lenses 466 may comprise flat lenses. In an example implementation, lenses 466 each comprise a flat lens having a thickness of 1 pm or less, facilitating a short working distance of less than 2 mm without difficult alignment given its flat form. Particular examples of the lenses 466 include Fresnel lenses, zone plate lenses and meta-lenses. The lens may include an amplitude mask for computational imaging.

[00067] In an example implementation, lenses 466-1 , 466-2 are each focused on the same deposition site to provide different perspectives of the deposition site, facilitating the construction of stereoscopic or three- dimensional images of the deposition site. In another example implementation, lenses 466 are focused on different portions of target 472, providing a wider field of view and, in some implementations, facilitating imaging of multiple wells of a well plate. [00068] Target illuminators 468 comprise electronic components that illuminates portions of target 472 with light that may be reflected from the deposition site and that may be received by lenses 466. In an example implementation, each target illuminator 468 may comprise a light emitting diode. In an example implementation, each target illuminator 468 may comprise a laser diode for monochromatic imaging to reduce the effect of chromatic aberrations off-axis of the optical system. In other implementations, each target illuminator 468 may comprise other light-emitting devices. The two target illuminators 468-1 , 468-2 provide illumination of the target 472 for each of the two different pairs of lenses 466 and imaging elements 463 that extends on opposite sides of each fluid ejector 324 in which form an imager 326 for the particular fluid ejector 324. Although the sectional view illustrates imaging elements 463 and lenses 466 as extending on opposite sides of orifice 480, it should be appreciated that in some implementations, imaging elements 463 and lenses 466 may be in the form of (a) a single imaging array and a single continuous lens or (B) multiple imaging arrays and/or multiple lenses that collectively surround or encircle ejection orifice 480, providing a larger field of view or providing additional perspectives for the construction of a stereoscopic or 3D image of a deposition site.

[00069] In the examples illustrated, both a circuitry platform and a transparent substrate are shared by both the imagers 326 and the fluid ejectors 324. In other implementations, the imagers 326 and the fluid ejectors 324 may share the circuitry platform, wherein each of imagers 326 has a dedicated transparent substrate 464 while the fluid ejector has a dedicated chamber layer. In other implementations, each of the imagers 326 and the fluid ejector 324 may have distinct dedicated circuitry platforms, wherein the transparent substrate 464, used by the imagers 326 also forms the fluid ejection chambers 480.

[00070] Target 472 is in the form of a well plate comprising multiple individual wells 490-1 , 490-2, 490-3, 490-4 and so on (collectively referred to as wells 490. Each of wells 490 comprises a volume to receive a solution or material as well as to receive droplets 142 ejected through orifices 482. Each of wells 490 may include registration markings 492 (schematically shown) rather than a transparent finishing. Such registration markings 492 may facilitate identification of individual wells by the imagers 326 of system 420.

In some implementations, the registration markings 1082 may comprise well- outlines or fiducial marks (crosses, posts and the like) imprinted, embossed, laser engraved or scribed into the wells 1082. Each of wells 490 may additionally or alternatively include landing pads 494 (schematically shown) for registration with respect to ejection orifices 480.

[00071] In an example implementation, each of wells 490 comprises a micro-reaction micro well having a cross-sectional area on a scale of less than one mm ² . Because ejection orifices 480 and imagers 326 are aimed or focused on the same location or spot, providing built-in alignment of ejection offices for 80 with the concurrently imaged deposition site (the interior of a well), the individual wells 490 may be precisely located for both imaging and the reception of a fluid droplet or multiple droplets. As a result, the wells 490 may have smaller cross-sections and the array may have a greater density of wells. Real-time monitoring of the placement of droplets or real-time monitoring of the positioning of wells 490 is facilitated to facilitate faster sample processing and analysis.

[00072] FIG. 5B is a bottom view of a portion of system 420 taken along line 5B-5B of FIG. 5A. FIG. 5B illustrates one example of how the fluid ejectors 324 and imagers 326 of system 420 may be arranged or laid out on a single integrated packaging, such as a single integrated die. In the example illustrated, the fluid ejectors 324are arranged in rows or columns along packaging 470. In the example illustrated, each of fluid ejectors 324 has its own opposite dedicated pair of lenses 466. In the example illustrated, imaging elements 463 are formed as a single continuous band or strip of elements extending along the row or column of fluid ejectors 324. Distinct portions of the continuous band or strip of elements 463 may be associated with distinct fluid ejectors 324. In the example illustrated, target illuminators 468 are also provided as a single continuous row or column of light emitters, such as light emitting diodes. In other implementations, each of fluid ejectors 324 may have an associated pair of imaging array elements 463 and/or target illuminators 468.

[00073] In the example shown in FIGS. 4A-4C, each of fluid ejectors 324 are supplied with different liquids having different non-location parameters (temperature composition) from their associated fluid supplies 325 or offer droplets of different volumes. In the examples, controller 350 causes actuator 344 to locate deposition site 140 relative to the different fluid ejectors 324 based upon the determined non-location parameter of the next droplet to be ejected onto the deposition site. FIGS. 6A, 6B and 6C schematically illustrate portions of an example imaging and fluid ejection control system 620 in which a single fluid ejector may offer droplets or groups of droplets having different selected non-location parameters. Operation of the single fluid ejector is adjusted based upon captured images indicating the current state of the deposition site so as to provide the determined non-location parameter. System 620 comprises fluid ejector 624, fluid supplies 625-1 , 625-2 (shown in FIG. 6C) (collectively referred to as supplies 625), valve 626, heater 629, stimulus applicator 631 and controller 650.

[00074] Fluid ejector 624 is similar to fluid ejector 324 except that fluid ejector 624 is actuated between different states in which fluid ejector 624 ejects differently sized droplets having different volumes V1 and V2. For example, in implementations where fluid ejector 624 comprises a thermal resistive fluid actuator that nucleated fluid within a chamber to form a bubble that displaces fluid within the chamber through nozzle opening, fluid ejector 624 may be supplied with different mounting political current to create different amount of heat and creatively sized bubbles to eject additionally sized droplets. In implementations where fluid ejector 624 comprises a piezoresistive fluid actuator that moves a membrane to displace fluid within a chamber through a nozzle opening or orifice, different amount of electrical power may be supplied to the actuator to differently displace the membrane and eject differently sized droplets. Thus, fluid actuator 624 may supply droplets having two different non-location parameters, different volumes.

[00075] Fluid supplies 625-1 , 625-2 are similar to fluid supplies 325-1 and 325-2, described above except that such supplies 625 are each selectively connected to fluid ejector 624 by valve 626. In implementations where fluid supplies 625 contain fluids of different chemical compositions, valve 626 may be selectively actuated to cause fluid ejector 624 to eject different droplets having different non-location parameters in the form of different chemical compositions.

[00076] Heater 629 comprise a device to adjust the temperature of the liquid being supplied by fluid supplies 625. In one implementation, heater 629 comprises a thermal resistive heater to heat fluid as it passes from fluid supplies 625 to fluid ejector 624. Heater 629 facilitates the ejection of different droplets having a different non-location parameter in the form of different temperatures through a single fluid ejector 624.

[00077] Controller 650 is similar to controller 350 described above except that controller 650 outputs control signals adjusting the state of fluid ejector 624, the state of valve 626 and/or the state of heater 629 to vary and control a non-location parameter or multiple non-location parameters of the droplet or droplets being ejected by fluid ejector 624. FIGS. 6A-6C illustrate how deposition site with a previously deposited droplet may be imaged and analysis of the captured image or images may be used to control or adjust characteristics of a second subsequent droplet deposited on the deposition site.

[00078] As shown by FIG. 6A, controller 650 has outputted control signals causing fluid ejector 624 to eject a first droplet 142-1. Controller 650 output control signal such that the droplet 142-1 has a first volume V1 , a temperature T 1 and a chemical composition C1. As further shown by FIG.

6A, controller 650 outputs control signals causing imager 326 to capture an image of deposition site 140 and the deposited droplet 142-1 (which may be in any of a variety of state states).

[00079] Following acquisition of the imager images captured by imager 326, controller 650 analyzes the image or such images on a pixel by pixel basis or in other fashions with optical recognition so as to identify a current state of the deposition site 140. Such analysis may include a breadth of image processing methods and their combinations (e.g. color histogram, intensity in region of interest, segmentation, background, subtraction, template alignment, pattern recognition via machine learning and deep learning, and the like). For example, the instructions on medium 32 may direct the processor 128 to identify the current state of an ongoing process that may have started or that may be taking place at the deposition site following deposition of the first droplet 142-1, such as chemical reactions, the absorption of materials, the expulsion or emission of materials, biological activities and material phase changes such as freezing, melting, sublimation and evaporation. Based upon such analysis, the instructions on medium 32 further direct processor 128 to determine a non-location parameter for a second subsequent droplet to be ejected onto the deposition site 140.

[00080] FIGS. 6B and 6C illustrate two alternative scenarios that may take place depending upon the non-location parameter for the second droplet of fluid to be deposited on deposition site 140, as determined by controller 650. FIG. 6B illustrates controller 650 outputs control signals causing adjusting the heating provided by heater 629 such that ejector 624 ejects a second droplet 142-2 having a temperature T2. In the example illustrated, valve mechanism 626 was not changed such that the droplet 142-2 has the same composition C1 as droplet 142-1. The droplet 124-2 may have the same volume V1 or a different volume V2 as compared to the volume V1 of droplet 142-1. [00081] FIG. 6C illustrates a scenario where a different non-location parameter was determined by controller 650 based upon the images of deposition site 140 captured by imager 326 following deposition of droplet 142-1. In FIG. 6C, controller 650 is illustrated as outputting control signals causing fluid ejector 624 to eject a second droplet 124-3 having a non-location parameter or multiple non-location parameters that are different as compared to droplet 124-2. In the example illustrated, controller 650 is illustrated as having output control signals resulting in valve 626 disconnecting fluid supply 625-1 from fluid ejector 624 and connecting fluid supply 625-2 to fluid ejector 624. As result, droplet 142-3 has a chemical composition C2 that it different in the chemical composition of droplet 142-2 (C20). As further shown by FIG. 6C, the droplet 142-3 may have either temperature T1 orT2 (depending upon the state of heater 629 as controlled by controller 650) and either a first volume V1 or a second volume V2, depending upon the operation of fluid ejector 624 by controller 650.

[00082] In some implementations, system 620 may comprise a multitude of imaging and ejection subsystems with each subsystem comprising a fluid ejector 624, multiple available fluid supplies 625 with valve 626, a heater 629 and in imager 326. In such an implementation, controller 650 may additionally output control signals causing actuator 344 to move deposition site 140 and/or the imaging and ejection subsystems relative to one another to locate deposition site 140 opposite to a selected one of the subsystems for receiving a droplet or a group of droplet having a non-location parameter as determined by controller 650 based upon the acquired images of the deposition site 140 following deposition of an earlier droplet. In such an implementation, system 620 may operate in a fashion similar to the operation system 320 described above with respect to 4A-4C except that each subsystem itself may offer a variety of different non-location parameters of the droplets being ejected at the particular subsystem. [00083] Although the present disclosure has been described with reference to example implementations, workers skilled in the art will recognize that changes may be made in form and detail without departing from disclosure. For example, although different example implementations may have been described as including features providing various benefits, it is contemplated that the described features may be interchanged with one another or alternatively be combined with one another in the described example implementations or in other alternative implementations. Because the technology of the present disclosure is relatively complex, not all changes in the technology are foreseeable. The present disclosure described with reference to the example implementations and set forth in the following claims is manifestly intended to be as broad as possible. For example, unless specifically otherwise noted, the claims reciting a single particular element also encompass a plurality of such particular elements. The terms “first”, “second”, “third” and so on in the claims merely distinguish different elements and, unless otherwise stated, are not to be specifically associated with a particular order or particular numbering of elements in the disclosure.