ANTIMICROBIC SUSCEPTIBILITY TESTING USING RECURRENT NEURAL NETWORKS

RELATED APPLICATION

[0001] This application is related to, and claims the benefit of, provisional patent application 63/115,768 titled “Antimicrobic Susceptibility Testing Using Recurrent Neural Networks,” filed in the United States Patent Office on November 19, 2020. That application is hereby incorporated by reference in its entirety.

BACKGROUND

[0002] Various types of tests related to patient diagnosis and therapy can be performed by analysis of the patient’s microorganisms, or “microbes.” Microbes are microscopic living organisms such as bacteria, fungi, or viruses, which may be single-celled or multicellular. Biological samples containing the patient's microorganisms may be taken from a patient's infections, bodily fluids or abscesses and may be placed in test panels or arrays, combined with various reagents, incubated, and analyzed to aid in treatment of the patient. Automated biochemical analyzers have been developed to meet the needs of health care facilities and other institutions to facilitate analysis of patient samples and to improve the accuracy and reliability of assay results when compared to analysis using manual operations and aid in determining effectiveness of various antimicrobials. An antimicrobial is an agent that kills microorganisms or inhibits their growth, such as antibiotics which are used against bacteria and antifungals which are used against fungi. However, with ever changing bacterial genera and newly discovered antimicrobials, the demand for biochemical testing has increased in both complexity and volume.

[0003] An important family of automated microbiological analyzers function as a diagnostic tool for determining both the identity of an infecting microorganism and of an antimicrobic effective in controlling growth of the microorganism. Automated microbiological analyzers function as a diagnostic tool for determining both the identity of an infecting microorganism and of an antimicrobic effective in controlling growth of the microorganism. In performing the diagnostic tests, identification and in vitro antimicrobic susceptibility patterns of microorganisms isolated from biological samples are ascertained. Conventional versions of such analyzers may place a small sample to be tested into a plurality of small sample test wells in panels or arrays that contain different enzyme substrates or antimicrobics in serial dilutions. Identification (ID) testing of microorganisms, and antimicrobic susceptibility testing (AST) for determining Minimum Inhibitory Concentrations (MIC) of an antimicrobic effective against the microorganism may utilize color changes, fluorescence changes, the degree of cloudiness (turbidity) in the sample test wells created in the arrays, or other information derived from the testing. Both AST and ID measurements and subsequent analysis may be performed by computer controlled microbiological analyzers to provide advantages in reproducibility, reduction in processing time, avoidance of transcription errors and standardization for all tests run in the laboratory.

[0004] In ID testing of a microorganism, a standardized dilution of the patient's microorganism sample, known as an inoculum, is first prepared in order to provide a bacterial or cellular suspension having a predetermined known concentration. This inoculum is placed in a plurality of test wells that may contain or thereafter be supplied with predetermined test media. Depending on the species of microorganism present, this media will facilitate changes in color, turbidity, fluorescence, or other characteristics after incubation. These changes are used to identify the microorganism in ID testing.

[0005] In AST testing, a plurality of test wells are filled with inoculum and increasing concentrations of a number of different antimicrobial agents, for example antibiotics. The different antimicrobial agents may be diluted in a growth medium or liquid medium to concentrations that include those of clinical interest. After incubation, the turbidity will be increased or unchanged in test wells where growth has not been inhibited by the antimicrobics in those test wells. The MIC of each antimicrobial agent is measured by lack of growth with respect to each concentration of antimicrobial agent. It follows that the lowest concentration of antimicrobial agent displaying a lack of growth is the MIC.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] While the specification concludes with claims which particularly point out and distinctly claim the invention, it is believed the present invention will be better understood from the following description of certain examples taken in conjunction with the accompanying drawings, in which like reference numerals identify the same elements and in which:

[0007] FIG. 1A depicts a portion of a diagrammatic view of an exemplary biological testing system;

[0008] FIG. IB depicts another portion of the diagrammatic view of the biological testing system of FIG. 1A;

[0009] FIG. 2 depicts a perspective view of an exemplary incubator system and an exemplary optics system of the biological testing system of FIG. IB;

[00010] FIG. 3 depicts a perspective view of the optics system of FIG. 2;

[00011] FIG. 4 depicts another perspective view of the optics system of FIG. 2 showing an

XY stage of the optics system;

[00012] FIG. 5 depicts a diagrammatic view of portions of the optics system of FIG. 2;

[00013] FIG. 6 depicts a diagrammatic view of an exemplary computer system;

[00014] FIG. 7 depicts a chart of theoretical growth curves for a microbe;

[00015] FIG. 8 depicts a schematic view of an exemplary growth test well for use in the biological testing system of FIGS. 1A and IB;

[00016] FIG. 9 depicts an exemplary image analysis cycle for use in the biological testing system of FIGS. 1A and IB;

[00017] FIG. 10 depicts an exemplary raw image captured by the optics system of FIG. 2;

[00018] FIG. 11 depicts an exemplary enhanced image derived from the raw image of FIG.

10;

[00019] FIG. 12 depicts an exemplary gradient image derived from the raw image of FIG. 10;

[00020] FIG. 13 depicts an exemplary image enhancement flowchart;

[00021] FIG. 14 depicts an exemplary dynamic image enhancement flowchart; [00022] FIG. 15 depicts an exemplary segmented image derived from the raw image of FIG. 10;

[00023] FIG. 16 depicts an exemplary image segmentation flowchart;

[00024] FIG. 17 depicts an exemplary optimized antimicrobic susceptibility testing method of the present invention;

[00025] FIG. 18 depicts an exemplary architecture for a machine learning model that some embodiments may use to apply recurrent neural networks to the task of MIC identification;

[00026] FIG. 19 depicts an individual GRU that could be used to recognize a particular growth pattern;

[00027] FIG. 20 depicts a flowchart showing processing a GRU could perform; and

[00028] FIG. 21 depicts an architecture that could be used to train a model to generate microorganism counts.

[00029] The drawings are not intended to be limiting in any way, and it is contemplated that various embodiments of the invention may be carried out in a variety of other ways, including those not necessarily depicted in the drawings. The accompanying drawings incorporated in and forming a part of the specification illustrate several aspects of the present invention, and together with the description serve to explain the principles of the invention; it being understood, however, that this invention is not limited to the precise arrangements shown.

SUMMARY

[00030] Various aspects and embodiments of the invention are set out in the claims appended hereto.

[00031] In a first aspect of the invention, there is provided a method comprising: (a) creating a plurality of test mixtures in a plurality of test wells, wherein: (i) each test mixture from the plurality of test mixtures is inoculated using a biological sample; (ii) each test mixture from the plurality of test mixtures comprises an antimicrobial solution comprising an antimicrobial agent; (iii) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that test mixture differs from the antimicrobial solution in each other test mixture from the plurality of test mixtures; and (iv) the same biological sample is used to inoculate each test mixture from the plurality of test mixtures; (b) incubating each of the test mixtures; (c) for each test mixture from the plurality of test mixtures, at a plurality of imaging times, wherein each of the imaging times takes place after incubation has begun, capturing an image of that test mixture; (d) obtaining a plurality of growth predictions by performing steps comprising, for each test mixture from the plurality of test mixtures, providing a data sequence to a machine learning model, wherein: (i) the data sequence comprises a plurality of input items; (ii) each input item from the plurality of input items corresponds to an imaging time from the plurality of imaging times; and (iii) the machine learning model is adapted to recognize, and to make growth predictions based on, temporal sequences; and (e) generating a minimum inhibitory concentration, MIC, determination for the biological sample based on the plurality of growth predictions.

[00032] In another aspect of the invention, there is provided a biological testing system comprising a processor configured with a set of computer instructions operable, when executed, to cause the system to perform a method comprising: (a) creating a plurality of test mixtures in a plurality of test wells, wherein: (i) each test mixture from the plurality of test mixtures is inoculated using a biological sample; (ii) each test mixture from the plurality of test mixtures comprises an antimicrobial solution comprising an antimicrobial agent; (iii) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that test mixture differs from the antimicrobial solution in each other test mixture from the plurality of test mixtures; and (iv) the same biological sample is used to inoculate each test mixture from the plurality of test mixtures; (b) incubating each of the test mixtures; (c) for each test mixture from the plurality of test mixtures, at a plurality of imaging times, wherein each of the imaging times takes place after incubation has begun, capturing an image of that test mixture; (d) obtaining a plurality of growth predictions by performing steps comprising, for each test mixture from the plurality of test mixtures, providing a data sequence to a machine learning model, wherein: (i) the data sequence comprises a plurality of input items; (ii) each input item from the plurality of input items corresponds to an imaging time from the plurality of imaging times; and (iii) the machine learning model is adapted to recognize, and to make growth predictions based on, temporal sequences; and (e) generating a minimum inhibitory concentration, MIC, determination for the biological sample based on the plurality of growth predictions.

[00033] In another aspect of the invention, there is provided a computer program product comprising a non-transitory computer readable medium having stored thereon a set of computer instructions operable, when executed, to cause a biological testing system to perform a method comprising: (a) creating a plurality of test mixtures in a plurality of test wells, wherein: (i) each test mixture from the plurality of test mixtures is inoculated using a biological sample; (ii) each test mixture from the plurality of test mixtures comprises an antimicrobial solution comprising an antimicrobial agent; (iii) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that test mixture differs from the antimicrobial solution in each other test mixture from the plurality of test mixtures; and (iv) the same biological sample is used to inoculate each test mixture from the plurality of test mixtures; (b) incubating each of the test mixtures; (c) for each test mixture from the plurality of test mixtures, at a plurality of imaging times, wherein each of the imaging times takes place after incubation has begun, capturing an image of that test mixture; (d) obtaining a plurality of growth predictions by performing steps comprising, for each test mixture from the plurality of test mixtures, providing a data sequence to a machine learning model, wherein: (i) the data sequence comprises a plurality of input items; (ii) each input item from the plurality of input items corresponds to an imaging time from the plurality of imaging times; and (iii) the machine learning model is adapted to recognize, and to make growth predictions based on, temporal sequences; and (e) generating a minimum inhibitory concentration, MIC, determination for the biological sample based on the plurality of growth predictions.

DETAILED DESCRIPTION

[00034] The following description of certain examples of the invention should not be used to limit the scope of the present invention. Other examples, features, aspects, embodiments, and advantages of the invention will become apparent to those skilled in the art from the following description, which is by way of illustration, one of the best modes contemplated for carrying out the invention. As will be realized, the invention is capable of other different and obvious aspects, all without departing from the invention. Accordingly, the drawings and descriptions should be regarded as illustrative in nature and not restrictive.

[00035] It will be appreciated that any one or more of the teachings, expressions, versions, examples, etc. described herein may be combined with any one or more of the other teachings, expressions, versions, examples, etc. that are described herein. The following- described teachings, expressions, versions, examples, etc. should therefore not be viewed in isolation relative to each other. Various suitable ways in which the teachings herein may be combined will be readily apparent to those of ordinary skill in the art in view of the teachings herein. Such modifications and variations are intended to be included within the scope of the claims.

[00036] I. Biological Testing System Hardware

[00037] FIGS. 1A and IB depict a diagrammatical example of various hardware components available in a biological testing system 1. Biological testing system 1 facilitates an optimized antimicrobial susceptibility testing (AST) method 101 (FIG. 17). Biological testing system 1 broadly includes a consumable preparation system 3, an inoculating system 5, an incubator system 7, and an optics system 9. The various systems within biological testing system 1 coordinate with each other and work automatically once loaded with adequate material by a user.

[00038] To operate biological testing system 1, the user first acquires an appropriate microbe sample. As shown in FIG. 1 A, a microbe sample may be obtained from an agar plate 11 or, under certain circumstances, from a blood sample. Next, the user prepares an inoculum suspension by transferring the microbes into a tube containing a suitable liquid medium or broth. One such tube is shown in FIG. 1A as an inoculum 13. In some versions of biological testing system 1, the liquid medium or broth may be an approximately 0.5 mM phosphate buffered solution with small amounts of sodium and potassium chloride to aid in maintaining the viability of the microbes introduced into solution without adversely interfering with the MIC determination or other associated testing. The phosphate buffered solution may be used in both ID testing and AST testing to minimize the need for different inoculum across the two systems and leverage the efficiencies in having a single broth. Each inoculum 13 is placed into an inoculum rack 15 and the entire inoculum rack 15 is placed into inoculating system 5. Once in inoculating system 5, the inoculum in each inoculum 13 is adjusted if necessary to a standard turbidity value of 0.5 McFarland to create an inoculum 17. In some versions of biological testing system 1, a 1 microliter plastic loop or swab may be provided to the user to easily pick colonies from the agar plate and to minimize the amount of adjustment needed to bring the inoculum to the desired turbidity value. Once adjusted to the desired turbidity value, the inoculum is finalized. The finalized inoculum will be referred to hereinafter as inoculum 17, as depicted in FIG. IB. Inoculum

17 may be further diluted into a 1 :250 dilution and converted into an inoculum 18. As will be discussed in more detail below, the inoculum contained in each inoculum 17 is applied to an identification (ID) array holder 21, while the inoculum contained in each inoculum

18 is applied to an AST array holder 23. Both ID array holder 21 and AST array holder 23 are assembled by consumable preparation system 3 and provided to inoculating system 5 for use with inoculum 17 and inoculum 18.

[00039] Consumable preparation system 3 is loaded with magazines of test arrays 19, which may contain various antimicrobials or other agents required by biological testing system 1 disposed in a series of test wells 20. For example, test array 19 may comprise an antimicrobic dilution array or an identification array. Consumable preparation system 3 may also be loaded with bulk diluents (not shown) and/or various other elements for preparing and finalizing ID array holder 21 and AST array holder 23 and the inoculate therein. Primarily, consumable preparation system 3 operates to retrieve test arrays 19 as required and combine each retrieved test array 19 into either ID array holder 21 or AST array holder 23. Test arrays 19 may be selected and assembled by a robotic gripper (not shown) or other mechanical features as dictated by the prescribed testing. For example, a physician may order biological testing using the antibiotic amoxicillin. Test arrays 19 relating to amoxicillin testing are therefore retrieved and assembled into the appropriate ID array holder 21 and AST array holder 23. All or some portions of test array 19 may be formed of a styrene material to aid in reducing fluorescent crosstalk, fallout, and/or bubbles when digitally examining each test well 20. [00040] Once inoculum 17, inoculum 18, ID array holder 21, and AST array holder 23 are assembled, inoculating system 5 dispenses the generally undiluted inoculum from inoculum 17 into test wells 20 of ID array holder 21 and the diluted inoculum from inoculum 18 into test wells 20 of AST array holder 23. The time between applying inoculum 17 to ID array holder 21 or inoculum 18 to AST array holder 23 and the start of logarithmic growth of the microbes disposed therein is known as “lag time.” Lag time may be decreased by using enhanced broth such as a broth with yeast extract, vitamins, and/or minerals. Lag time may also be decreased by increasing the inoculum. In some versions of biological testing system 1, the amount of inoculum may be doubled to decrease the lag time by approximately 30 minutes without affecting the accuracy of the MIC determination. The dispensing may be accomplished via an elevator assembly 26 having an XY robot or XYZ robot (not shown) with a gripper (not shown) and pipettor (not shown), along with various circuitry, channels, and tubing as necessary. The XYZ robot is tasked with retrieving inoculum from inoculum racks 15 and dispensing the inoculum into test wells 20 of ID array holder 21 and AST array holder 23. Once ID array holder 21 and AST array holder 23 are sufficiently loaded with inoculum, each ID array holder 21 and AST array holder 23 are moved into incubator system 7 by way of an elevator assembly

26.

[00041] The XY robot or XYZ robot may be, or include, a Cartesian coordinate robot or gantry robot which comprises three principal axes (x-axis, y-axis and z-axis) of control which are linear. That is, the robot generally moves in a straight line rather than rotates along axes which are perpendicular (at right angles to) each other. A Cartesian coordinate robot or gantry robot may comprise three sliding joints that correspond to movement up- down (the z-axis), in-out (the y-axis), and back-forth (the z-axis), with respect to a plane or object. Alternatively, the elevator assembly may comprise any of an Articulate robot, a Cylindrical coordinate robot, a Spherical coordinate robot, a SCARA (Selective Compliance Assembly Robot Arm) robot, and/or serial manipulators.

[00042] As shown in FIG. 2, incubator system 7 includes slots 27 for holding a large number of ID array holders 21 and AST array holders 23. Each array holder is placed into a corresponding slot 27 by an XYZ robot 29 using a gripper 31. XYZ robot 29 operates to move in any portion of the XYZ plane and position gripper 31 proximate the desired ID array holder 21 or AST array holder 23. While in incubator system 7, each array holder incubates in specific desired environmental conditions. For example, incubator system 7 may be set to incubate array holders at thirty-five degrees Celsius. At certain time intervals during the incubation, XYZ robot 29 retrieves a particular ID array holder 21 or AST array holder 23 and move the selected array holder into the optics system 9.

[00043] As shown in FIG. 2-4, optics system 9 includes features that are configured to observe, monitor, review, and/or capture images for each test well 20 of an ID array holder 21 or AST array holder 23. Specifically, each ID array holder 21 is monitored by an ID fluorimeter 33, and each AST array holder 23 is monitored by an AST camera 35. To accomplish the monitoring, XYZ robot 29 retrieves the particular array holder with gripper 31 and places the selected array holder onto an XY-stage 37. The XY-stage 37 moves in the XY plane to position the array holder under the associated monitoring element, namely, the ID array holders 21 are disposed under the ID fluorimeter 33 and the AST array holders 23 are disposed under the AST camera 35 for monitoring and observation in optics system 9. XY-stage 37 includes finely tuned motor control to allow each test well 20 of the associated array holder to be positioned accurately within the observation frame of either ID fluorimeter 33 or AST camera 35.

[00044] FIG. 5 illustrates an exemplary architecture for an AST optics portion 39 of optics system 9. AST optics portion 39 includes an illumination source 41, an objective lens 43, a tube lens 45, and a fold mirror 47. Illumination source 41 may comprise a condenser LED system for providing monochromatic illumination of each test well 20 of AST array holder 23. Objective lens 43 may comprise a Nikon 20x 0.45NA ELWD objective lens, an Olympus 10X objective lens or any other suitable kind of lens. Objective lens 43 may comprise a 20x objective lens with each pixel covering about 0.33 microns. A 20x objective lens provides both the ability to detect a reasonable number of cells at the beginning of cell growth (around 100-200 cells) and the ability to detect cell morphology. Objective lens 43 may comprise a lOx objective lens and/or a 5MP camera for a larger dynamic range and/or a bigger sample of each test well 20) while maintaining enough resolution to count the microbes therein. The magnification of Objective lens 43 may be in the range of 5x to 50x. The selection of the magnification of the Objective lens 43 may depend on various factors, such as the size of an image sensor or the camera being used, the size of the microbes being photographed, the required pixel size or resolution of the captured image, and the like. In some exemplary embodiments of optics system 9, only one picture or image per test well 20 per pass is acquired by objective lens 43. Objective lens 43 may focus slightly off the bottom of test well 20 to eliminate background noise from the bottom of test well 20. In some versions of optics system 9, objective lens 43 is configured to focus approximately 5-10 microns from the bottom of test well 20. In some versions of optics system 9, objective lens 43 is configured to focus 8 microns from the bottom of test well 20. Objective lens 43 may also include a Z-stage 44 for allowing objective lens 43 to move in the Z-axis, relative to XY-stage 37. Thus, between XY-stage 37 moving AST array holder 23 in the XY plane and Z-stage 44 moving objective lens 43 in the Z-axis, each test well 20 of array holder 23 may be moved in any three-dimensional space to precisely align test wells 20 with the frame of AST camera 35. Tube lens 45 may be embodied in an achromatic tube lens. AST camera 35 may comprise a Sony IMX253 camera, various types of 5 megapixel cameras provided by other manufacturers such as Canon, Thorlabs, or Sentech, or any other suitable kind of camera. In some versions of optics system 9, XY-stage 37 and Z-stage 44 are replaced with an XYZ-stage to provide all three axes of three-dimensional movement of test wells 20.

[00045] Referring now to FIG. 6, the various components of biological testing system 1 may incorporate one or more computing devices or systems, such as exemplary computer system 49. For example, any one of consumable preparation system 3, inoculating system 5, incubator system 7, and/or optics system 9 may incorporate one or more computing systems such as exemplary computer system 49. Alternatively, each of these subsystems of biological testing system 1 may function via commands from one overall computing system such as exemplary computer system 49.

[00046] Computer system 49 may include a processor 51, a memory 53, a mass storage memory device 55, an input/output (VO) interface 57, and a Human Machine Interface (HMI) 59. Computer system 49 may also be operatively coupled to one or more external resources 61 via a network 63 or I/O interface 57. External resources may include, but are not limited to, servers, databases, mass storage devices, peripheral devices, cloud-based network services, or any other suitable computer resource that may used by computer system 49.

[00047] Processor 51 may include one or more devices selected from microprocessors, micro-controllers, digital signal processors, microcomputers, central processing units, field programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other devices that manipulate signals (analog or digital) based on operational instructions that are stored in memory 53. Memory 53 may include a single memory device or a plurality of memory devices including, but not limited, to read-only memory (ROM), random access memory (RAM), volatile memory, nonvolatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache memory, or any other device capable of storing information. Mass storage memory device 55 may include data storage devices such as a hard drive, optical drive, tape drive, non-volatile solid state device, or any other device capable of storing information.

[00048] Processor 51 may operate under the control of an operating system 65 that resides in memory 53. Operating system 65 may manage computer resources so that computer program code embodied as one or more computer software applications, such as an application 67 residing in memory 53, may have instructions executed by the processor 51. In an alternative embodiment, processor 51 may execute application 67 directly, in which case the operating system 65 may be omitted. One or more data structures 69 may also reside in memory 53, and may be used by processor 51, operating system 65, or application 67 to store or manipulate data.

[00049] The I/O interface 57 may provide a machine interface that operatively couples processor 51 to other devices and systems, such as network 63 or external resource 61. Application 67 may thereby work cooperatively with network 63 or external resource 61 by communicating via I/O interface 57 to provide the various features, functions, applications, processes, or modules comprising embodiments of the invention. Application 67 may also have program code that is executed by one or more external resources 61, or otherwise rely on functions or signals provided by other system or network components external to computer system 49. Indeed, given the nearly endless hardware and software configurations possible, persons having ordinary skill in the art will understand that different versions of the invention may include applications that are located externally to computer system 49, distributed among multiple computers or other external resources 61, or provided by computing resources (hardware and software) that are provided as a service over network 63, such as a cloud computing service.

[00050] HMI 59 may be operatively coupled to processor 51 of computer system 49 in a known manner to allow a user to interact directly with the computer system 49. HMI 59 may include video or alphanumeric displays, a touch screen, a speaker, and any other suitable audio and visual indicators capable of providing data to the user. HMI 59 may also include input devices and controls such as an alphanumeric keyboard, a pointing device, keypads, pushbuttons, control knobs, microphones, etc., capable of accepting commands or input from the user and transmitting the entered input to the processor 51.

[00051] A database 71 may reside on mass storage memory device 55, and may be used to collect and organize data used by the various systems and modules described herein. Database 71 may include data and supporting data structures that store and organize the data. In particular, database 71 may be arranged with any database organization or structure including, but not limited to, a relational database, a hierarchical database, a network database, or combinations thereof. A database management system in the form of a computer software application executing as instructions on processor 51 may be used to access the information or data stored in records of the database 71 in response to a query, where a query may be dynamically determined and executed by operating system 65, other applications 67, or one or more modules.

[00052] II. Optimized AST System and Method

[00053] In some versions, system 1 as discussed above may be used to facilitate some or all of the features provided in optimized AST method 101 such as shown in figure 17.

[00054] In some conventional processes, MIC is determined through manual visual inspection of test wells after waiting a period to allow the microbes to grow. However, as shown in FIG. 7, traditional methods of determining MIC are limited by what a human can visually perceive relative to the growth of the microbes within the test wells. Further, the presence of antimicrobic dilution concentrations that are below the MIC concentration may slow the growth rate, and take even longer to perceive. As shown in FIG. 7, the first six to seven doublings of the microbial sample cannot be observed visually by the human eye. However, the information provided through these initial doublings is often indicative of the MIC. As shown in FIGS. 7-20, optimized AST method 101 utilizes digital microscopy to monitor the count of microbes in one more test wells from the point of inoculation and thus allows for more rapid and accurate detection of the MIC.

[00055] FIG. 8 provides an illustration of how data that can be used in an optimized AST method 101 can be captured from a test well. Test well 20 is embodied by a “384” style test well, having a clear viewing bottom. The volume of inoculum in test well 20 may be set to 20 microliters to reduce the amount of materials such as bulk diluents required by AST method 101 and/or minimize light artifacts and provide sampling of a consistent number of microbes from the 1 :250 dilution of the 0.5 MacFarland inoculum. Decreasing the volume of inoculum in test well 20 generally increases the light artifacts. In some instances, capturing a single vertical plane 103 at 20x objective whereby AST camera 35 is set at 0.33 microns/pixel may be sufficient for sampling the inoculum and ensuring that each individual microbe is recognizable. However, these parameters are configurable and may change as desired by the user or the underlying needs of the system. Capturing three focal sites spaced about 5 microns apart within single vertical plane 103 may also provide a sufficient number of microbes in the sample to count for use in optimized AST method 101. These three focal sites are labeled site 105, site 107, and site 109 in FIG. 8. In an exemplary version of optimized AST method 101, each focal site is approximately 700 x 700 microns within single vertical plane 103, rather than capturing three sites in three different vertical planes. AST camera 35, optics system 9, and computer 49 are configured to capture an image of each focal site in consecutive time periods, manipulate each of these images, and thereafter use the data derived from these manipulated images to make a MIC determination.

[00056] Image analysis cycle 102 is generally depicted in FIG. 9 and comprises an image capture step 111, an image enhancement step 113, an image segmentation step 115, and an object counting step 117. Optimized AST method 101 may include performing image analysis cycle 102 repetitively until a MIC is determined. Image analysis cycle 102 utilizes one or more instances of computer 49 and the various elements thereof to perform image capture step 111, image enhancement step 113, image segmentation step 115, and data extraction step 117, as well as any sub steps provided therein.

[00057] Image analysis cycle 102 begins with image capture step 111 captures a raw image 119 (FIG. 10) of the inoculum via AST camera 35 of optics system 9 and stores raw image 119 in memory 53. Raw image 119 may be considered to be an image which has not had any static or dynamic enhancement, filters, noise reduction or any other image correction. While raw image 119 is depicted as a single image, raw image 119 may be a composite of several separate images taken in vertical plane 103, for example, a composite of site 105, site 107, and site 109. Raw image 119 may also be a composite of several images taken in different planes within the inoculum sample or may be a single image. As illustrated in FIG. 10, raw image 119 may include deficiencies such as uneven illumination. Uneven illumination may be a result of the meniscus created by the inoculum in combination with the walls of the associated test well 20, which may affect the path of illumination source 41. Uneven background intensity may also be caused by environmental issues such as plastic deformation or from a variety of other sources. After raw image 119 is captured in image capture step 111, image capture step 111 moves to image enhancement step 113.

[00058] As shown in FIGS. 9, and 11-14, image enhancement step 113 processes raw image 119 to create an enhanced image 121 (FIG. 11), whereby enhanced image 121 is more suitable for the process of counting and recognizing microbes such as bacteria. During image enhancement step 113, one or more attributes of raw image 119 are modified. These attributes may include basic gray level transformations, noise filtering, and median filtering. For example, to resolve the problem of non-uniform illumination, a median filter may be applied to raw image 119 to arrive at a gradient image 123 (FIG. 12). This may be accomplished by selecting a pixel radius sufficient to provide a resulting image containing only the gradient of the background illumination of raw image 119. Gradient image 123 is then subtracted from raw image 119 to correct for the uneven illumination and generate enhanced image 121 free of uneven illumination. [00059] Image enhancement step 113 may apply image enhancements either statically, dynamically, or both. For example, the pixel radius for the median filter may be a statically set constant value or may be adaptively derived dynamically from characteristics of each raw image 119 captured through optics system 9. As shown in FIG. 13, some versions of image enhancement step 113 may include a step 125, whereby a determination is made as to whether the working image should undergo enhancement. If step 125 determines the working image should be enhanced, step 125 proceeds to a step 127. In step 127, a determination is made as to whether a static enhancement should be applied. If step 127 determines a static enhancement should be applied, step 127 proceeds to a step 129. If step 127 determines a static enhancement should not be applied, step 127 proceeds to a step 131. In step 127, a static enhancement is applied to the working image and step 127 proceeds back to step 125. In step 131, a dynamic enhancement is applied to the working image and step 131 proceeds back to step 125. If step 125 determines the working image should not be further enhanced, step 125 proceeds to end.

[00060] FIG. 14 illustrates an example of a method of dynamic image enhancement 133. Method of dynamic image enhancement 133 is directed to dynamically determining the appropriate pixel radius for use in a median filter enhancement. Method of dynamic image enhancement 133 begins with a step 135, whereby the size of the microbes depicted in raw image 119 is determined. The size of the microbes in raw image 119 may change depending on various circumstances and parameters associated with the inoculum and the overall optics system 9. Thus, while the literal size of the particular microbe being tested is generally constant in nature, the relative size of the microbes depicted in raw image 119 is dynamic and variable because of differences in parameters such as the lens objectification. Once step 135 determines the size of the microbes in raw image 119, step 135 proceeds to a step 137. In step 137, the pixel radius is derived from the determined size of the microbes. As a general example, if step 135 determines the maximum length of any given microbe in raw image 119 is five pixels, the pixel radius may be determined to be greater than five so that the filtered image only represents the gradient contained in the background. After step 135 derives the pixel radius based off the dynamically determined size of the microbes, step 135 proceeds to step 139. In step 139, gradient image 123 is generated based on processing raw image 119 with the derived pixel radius. After gradient image 123 is generated, step 139 moves to a step 141. In step 141, gradient image 123 is subtracted from raw image 119 to generate enhanced image 121. Thereafter, method of dynamic image enhancement 133 proceeds to end.

[00061] As shown in FIGS. 9, 15, and 16, image segmentation step 115 is used to partition the image into distinct regions containing pixels representing either the microbes as the foreground or the background. Image segmentation step 115 converts enhanced image 121 into a segmented image 143, as shown in FIG. 15. Image segmentation step 115 produces a binary image from enhanced image 121, where every pixel is equal to a value of either 0 or 1, where 0 refers to the background and 1 refers to a portion of a particular microbe.

[00062] Image artifacts such as noise may be removed by applying a noise reduction filter prior to applying the segmentation algorithm. Segmentation can be obtained using static threshold value or using an adaptive image thresholding method such as the Otsu cluster based thresholding algorithm. In this algorithm, the gray-level samples are clustered in two parts as background and foreground (object), or alternatively are modeled as a mixture of two Gaussians. The threshold value for the particular image thresholding algorithm used may be determined dynamically, depending on the overall image provided to image segmentation step 115 and the relative grayscale levels of the image. For example, inoculating system 5 or another element of system 1 may be configured to apply nigrosin to each test well 20 to enhance the image, as nigrosin does not attach to certain microbes such as bacteria. This may alter the relative greyscale levels in raw image 119 and require a different threshold value for the segmentation algorithm, as compared to a raw image 119 without nigrosin. In some versions of optimized AST method 101, threshold values may be determined dynamically by searching for edges within several areas of the image. These edges are the transition point between the background and a microbe. Thus, the threshold value can then be calculated as the average greyscale value for pixels on each side of the located edge.

[00063] As shown in FIG. 16, some versions of image segmentation step 115 may begin with a step 145. In step 145, a determination is made regarding whether to apply a noise reduction filter to enhanced image 121. If step 145 determines a noise reduction filter should be applied, step 145 proceeds to a step 147 where the noise reduction filter is applied. Step 147 thereafter proceeds to a step 149. If step 145 determines that a noise reduction filter should not be applied, step 145 proceeds directly to step 149. In step 149, a decision is made regarding whether to dynamically determine a threshold value. If step 149 decides a threshold value should be dynamically determined, step 149 proceeds to a step 151 where the threshold value is determined. Step 149 thereafter proceeds to a step 153. If step 149 decides not to dynamically determine a threshold value, a static predetermined threshold value is used and step 149 proceeds directly to step 153. In step 153, enhanced image 121 is segmented using the selected threshold value and step 153 and thereafter image segmentation step 115 proceeds to end.

[00064] Once image segmentation step 115 generates segmented image 143, image segmentation step 115 proceeds to data extraction step 117. In data extraction step 117, the background and foreground pixels are considered to derive information, such as the number of microbes in the sample. In some versions of data extraction step 117, the actual microbe count is compared with an average microbe count to determine if an error occurred within the image capture process. The comparison may incorporate a standard deviation with the average microbe count to generalize the microbe comparison.

[00065] Data extraction step 117 may be configured to derive information regarding the number of microbes in the image. In some embodiments of data extraction step 117, the number of foreground pixels in segmented image 143 may be counted in accordance with a predefined width and/or length to determine the number of microbes in the imaged portion of the inoculum. The counting algorithm may be divided into two separate algorithms, one for counting rod shaped microbes and one for counting spherical shaped microbes as the profile of the underlying microbes provides a corresponding different foreground pixel shape in segmented image 143. For example, the counting algorithm may be configured to consider a square of 2 x 2 pixels a microbe for counting purposes for spherical shaped microbes, or may consider a rectangle of 1 x 4 pixels a microbe for counting purposes for rod shaped microbes. Further, the counting algorithm may be configured to process both algorithms in order to capture the different three-dimensional orientations of rod shaped microbes. For example, if an elongated rod is positioned endwise towards AST camera 35, it will have a much different profile when viewed in two dimensions through AST camera 35. Therefore, both of the counting algorithms may be used during the counting phase of image analysis cycle 102. Alternatively, the counting algorithm may be configured to consider and count any foreground pixels surrounded by background pixels as a microbe.

[00066] Once data extraction step 117 derives the desired information from segmented image 143, image analysis cycle 102 terminates. Optimized AST method 101 iteratively performs image analysis cycle 102 at set time intervals to determine how the microbes in each test well 20 are changing and reacting to the particular antimicrobic dilution pairing. Further, optimized AST method 101 iteratively performs image analysis cycle 102 on each test well 20 associated with the microbes to determine how the microbes are reacting to each concentration of the antimicrobic dilution. For example, presume the microbes being tested are E. coli bacteria and three test wells 20 are being tested, with each test well 20 having a 20-microliter solution therein. The first test well 20 may contain an antimicrobic dilution of 1 microgram per milliliter (pg/ml), the second test well 20 may contain an antimicrobic dilution of 2 pg/ml, and the third test well 20 may contain an antimicrobic dilution of 4 pg/ml. Optimized AST method 101 performs image analysis cycle 102 on each of the three test wells at each set time interval to determine (a) how each antimicrobic dilution is affecting the microbes; and (b) how each antimicrobic dilution is performing relative to the other antimicrobic dilutions. If the data indicates the 1 pg/ml antimicrobic dilution is as effective as the 2 and 4 pg/ml antimicrobic dilutions at neutralizing the microbes, the 1 pg/ml antimicrobic dilution is the MIC.

[00067] An exemplary version of optimized AST method 101 is illustrated in FIG. 17 and begins with a step 155. In step 155, the system waits for a set time period threshold to allow the microbes within the selected test well 20 enough time to provide new information regarding the growth rate or reaction to the particular antimicrobic dilution. Optimized AST method 101 may preferably be configured to utilize a time period of 30 minutes for this threshold, though different thresholds may be used in different embodiments and/or with different microorganisms. For example, some bacteria or yeast or other microbes may react very quickly and provide information relevant to making an MIC determination within an hour. In this scenario, optimized AST method 101 may be configured to perform image analysis cycle 102 every five minutes to capture data regarding the rapidly changing environment within test wells 20. Other microbes may react relatively slowly to antimicrobic dilutions, and therefore a time period threshold of one hour may be more appropriate. Once step 155 waits the specified time period threshold, step 155 moves to a step 157.

[00068] In step 157, one iteration of image analysis cycle 102 is performed on a particular microbe with a selected test well 20. As discussed above, an iteration of image analysis cycle 102 derives data regarding the growth rate of the microbes within the selected test well 20. After an iteration of image analysis cycle 102 is performed, step 157 moves to a step 159. In step 159, the data collected in step 157 is stored and/or updated in memory, which may be in the form of a database, a flat file, or any other similar memory or storage device. In some embodiments of optimized AST method 101, step 159 stores the data collected in step 157 in database 71 (FIG. 6). Once step 159 stores/updates the collected data, step 159 moves to a step 161.

[00069] In step 161, optimized AST method 101 determines whether enough data has been collected to determine a MIC. This could be done, for example, by determining whether the data collected regarding the test well matched the data used to train a machine learning model used in determining a MIC and/or whether the machine learning model was able to make a determination with sufficient confidence to be usable. If more data is needed to accurately determine a MIC, step 161 returns to step 155 and waits to perform another image analysis cycle 102 to collect more data at a future time interval. If step 161 determines a sufficient amount of data has been collected, step 161 proceeds to a step 163.

[00070] The determination of whether the machine learning model is able to make a determination of an MIC with sufficient confidence to be usable can be based on a plurality of different criteria. A sufficient confidence value/level may be based on a threshold value, such that a useful determination may be considered to be when one or more of the determined MIC have a confidence value above the threshold value. The threshold value and the confidence value may both be expressed as a percentage value, e.g. 80%, 90%, 95%, etc., as a decimal representation, e.g. within a range between 0 and 1, or as any other suitable metric. The method may also determine that more data is needed to accurately determine an MIC if a plurality of confidence levels of a plurality of MIC determinations are within a specified confidence range of one another. In the example in which the confidence value is expressed as a percentage, the specified confidence range may be ±0.01% to ±5%. For example, if there are a plurality of MIC determinations having confidence values of 80%, 81% and 79%, which equates to a confidence range of ±2%, the method may determine that more data is needed to accurately determine the MIC for the biological sample.

[00071] In step 163, the MIC is determined. The determination is based on the data collected during each iteration of image analysis cycle 102 for all of the antimicrobic dilutions for a microbial sample as well as for a control well where no antimicrobial agent was present. As described in the following section, this determination may involve processing the collected data with a machine learning model that had previously been trained to make determinations of whether an antimicrobial dilution in a test well would inhibit growth and then determining the MIC based on the test well with the lowest concentration where the machine learning model made a determination that growth would be inhibited.

[00072] III. Recurrent Neural Networks for Determining Minimum Inhibitory Concentration (MIC)

[00073] FIG. 18 depicts an exemplary architecture for a machine learning model that some embodiments may use to apply recurrent neural networks to the task of MIC identification. In an embodiment following FIG. 18, a growth sequence vector 1801 could be generated by periodically (e.g., every 30 minutes) capturing data from test wells and organizing it into a vector data structure. This growth sequence vector 1801 could then be provided to a preprocessing module 1802 to put the data from the growth sequence vector 1801 in a form more suitable for subsequent processing by other modules as shown in FIG. 18. For example, in some embodiments where subsequent processing is performed using linear equations, a preprocessing module could take raw count data and apply a log base 2 transformation to transform the raw data into a linear scale representing organism doublings. Other transformations may also be applied, either in addition to or as alternatives to the logarithmic transformation described above. For example, in some embodiments, differences between time step values could be calculated (e.g., to obtain figures for changes in doubling rather than simple organism doubling values).

[00074] The machine learning model may, therefore, receive, be provided with, or otherwise obtain a plurality of images, or image items, for each test mixture from a plurality of test mixtures, using the image capture techniques and methods described above, for example as shown in Figure 8. The machine learning model may also receive, be provided with, or otherwise obtain a plurality of imaging times at which each of the plurality of images was captured. The plurality of imaging times may be used in conjunction with the plurality of images to recognize the growth patterns for microbes in the biological sample, and based on the training data provided to the machine learning model or otherwise, make growth rate predictions based on temporal sequences. The temporal sequences may be determined based on the time evolution, or rate of change of the growth rate, of the microbes as determined from the plurality of images and the plurality of imaging times for each test mixture of the plurality of test mixtures. The determination of a temporal sequence and the time evolution, or rate of change of the growth rate, of microbes within each test mixture of the plurality of test mixtures may be performed by any appropriate mathematical method or algorithm.

[00075] To illustrate, tables 1-4, below, provide examples of raw organism count data such as could be included in a growth sequence vector 1801, and how that count data could be transformed into doubling change values using a preprocessing module 1802 as described above.

Table 1 : Exemplary organism count values for 0-180 minutes. Note that each row represents a separate dilution, and therefore a separate test well. In some embodiments, each row would be represented as a separate growth sequence vector.

Table 2: Exemplary organism count values for 180-360 minutes. Note that each row represents a separate dilution, and therefore a separate test well. In some embodiments, each row would be represented as a separate growth sequence vector.

Table 3: Exemplary organism doubling change values that could be provided by a preprocessing module based on the count values of table 1.

Table 4: Exemplary organism doubling change values that could be provided by a preprocessing module based on the count values of table 2.

[00076] In an embodiment following the architecture of FIG. 18, after preprocessing was complete, the output of the preprocessing module 1802 would be provided to a network cluster 1803, e.g., as a series of 1x1 matrices representing the values obtained at individual time steps from the preprocessing calculations described above. This network cluster 1803 would preferably comprise a plurality of recurrent neural networks, such as long short-term memory (LSTM) or gated recurrent unit (GRU) networks. In an embodiment following the architecture of FIG. 18, such recurrent neural networks can be trained to recognize patterns which are relevant to specific organisms based on a dense layer 1807 identifying certain recurrent networks as better or worse for specific organisms. This would result in the dense layer 1807 up or down-weighting those recurrent neural networks accordingly, which would, in turn, cause them to become more effective at identifying patterns associated with their particular organisms through the training process. An illustration of a unit which could be trained in this manner and which could be included in a network cluster 1803, in this case a GRU, is provided in FIG. 19, and the calculations it would perform are discussed in the context of FIG. 20 and equations 1-4.

[00077] Turning now to FIGS. 19 and 20, FIG. 19 illustrates an individual GRU that could be used to recognize a particular growth pattern in the network cluster 1803 of an embodiment following the architecture of FIG. 18, while FIG. 20 illustrates in flowchart form the processing that such a GRU would perform. As shown in FIG. 20, at each time step (e.g., for every 30 minute data point) a reset matrix (r[t]) would be generated 2001 to specify information from previous time steps to discard or downweight. This can be done using Equation 1, which represents mathematically the substeps 2002 2013 2014 2005 shown in flow chart form as making up the reset matrix generation 2001 of FIG. 20.

Equation 1 : r[t] = o(Wr * x[t] + Ur * h[t- 1 ] + br)

In those substeps, the input for a particular time step (x[t], where t is the time step in question) is multiplied 2002 by a reset kernel matrix (Wr) which weights the input to determine how much of the information carried over from the previous step (h[t-l ]) should be forgotten. A similar multiplication 2003 is performed using a hidden state (h[t-l]) carried over from the previous time step (which, on the first time step, could be simply set to zeros), multiplying that hidden state by a reset recurrent matrix (Ur) which weights each of the items in the hidden state to determine the information from the previous step that should be forgotten. The results of these multiplications would then be added 2004 to a reset bias matrix (br) to obtain a raw reset matrix. This raw reset matrix would then be provided as input to a sigmoid activation function 2005 which would normalize its values by mapping them onto the range [0, 1], thereby providing the reset matrix (r[t]) which would be used to decide what (and how much) information from previous steps to discard.

[00078] Once it had been generated, the reset matrix r[t] could be combined 2006 with the hidden state from the previous time step (h[t- 1 ]), such as by taking the Hadamard product (represented in FIG. 19 and Equations 2 and 4 by •) of the reset matrix and the hidden state, this could then be used, along with the input for the then current time step (x[t]), to generate 2007 a candidate matrix (h[t]) which could be used to update the state value of the GRU. In some embodiments, this type of generation 2007 could comprise application of Equation 2.

Equation 2: h[t] = (|)(Wh*x[t] + Uh(r[t]»h[t- 1 ]) + bh)

In that equation, (|) is a hyperbolic tangent function which maps the output onto [-1, 1], and Wh, Uh and bh are candidate kernel, recurrent and bias matrices which would be applied in a manner similar to that described for the reset kernel, recurrent and bias matrices used to generate 2001 the reset matrix (r[t]).

[00079] Once the candidate matrix (h[t]) had been generated 2007, an embodiment following FIG. 20 would also generate 2008 an update matrix (z[t]) using Equation 3 and update kernel, recurrent and bias matrices (respectively, Wz, Uz, and bz) in a manner similar to that described above for the generation 2001 of the reset matrix using Equation 1.

Equation 3: z[t] = o(Wz*x[t] + Uz*h[t-1] + bz)

This update matrix (z[t]) could then be combined 2009 with the candidate matrix (h[t]) in a manner similar to that used to combine the reset matrix and the hidden state from the previous step (e.g., taking the Hadamard product). An additional intermediate combined matrix could then be generated 2010 by subtracting the update matrix (z[t]) from 1, and combining it with the hidden state matrix from the previous step (h[t-l]). These two combined matrices could then be added 2011 together to provide a final matrix (h[t]), which could be expressed mathematically as the result of Equation 4.

Equation 4: h[t] = (1 -z[t]) • h[t- 1 ] + z[t] • h[t]

This final matrix (h[t]) could serve as both the output of the GRU at time step t, as well as the hidden state that could propagate forward to the next time step. In this way, embodiments utilizing a GRU such as shown in FIG. 19 could take advantage of the ability of recurrent neural networks to identify patterns over time to make more accurate determinations of whether the particular concentration and antimicrobial in a particular test well would inhibit microbial growth.

[00080] After the GRUs (or other types of networks) from the network cluster 1803 had processed the output of the preprocessing module 1802, the result would preferably be a set of 1x1 matrices, with one matrix being provided by each unit in the network cluster 1803 (e.g., if the network cluster 1803 consisted of 24 GRUs, then the output of the network cluster 1803 would be 24 1x1 matrices). This set of matrices could then be combined with a vector representing the organism for which the MIC determination was being made. As shown in FIG. 18, information on an organism could follow a different path than that described for the growth sequence vector. For example, in some embodiments, a machine learning model following the architecture of FIG. 18 could be adapted to make MIC determinations for a plurality or organisms, and each of those organisms could be given a particular identification using one hot coding. For instance, in a model adapted to make MIC predictions for four organisms, the first could be given the ID [1 0 0 0], the second could be given the ID [0 1 0 0], the third could be given the ID [0 0 1 0], and the fourth could be given the ID [0 0 0 1], When using a model following the architecture of FIG. 18, an organism vector 1804 with information identifying the organism for which MIC is being identified could be provided as an input to an expansion module 1805, which could transform the organism vector 1804 into a form compatible with the output of the network cluster 1803. For example, in an embodiment where the output of the network cluster 1803 consisted of a set of 24 matrices, the expansion module could duplicate the organism vector 24 times to create a data structure which included one vector for each of the output matrices.

[00081] After the organism vector 1804 had been put into a form compatible with the output of the network cluster 1803, the outputs of the expansion module 1805 and the network cluster 1803 could be provided to a combination function 1806. This combination function 1806 could then join them, such as by multiplication or concatenation. The combined output could then be provided to a dense layer 1807, which, in some embodiments, could be implemented as a linear equation that applies weights to the output of the combination function. In embodiments using this type of approach, the portions of the combined output that are populated by the output of the network cluster 1803 could be seen as providing measures of growth, while the portions of the combined output that are populated by the output of the expansion module could be seen as acting as a mechanism for applying a different bias based on the organism in question.

[00082] Once it had been calculated, the output of the dense layer 1807 would be provided to an activation function 1808 for normalization (e.g., a sigmoid activation function), and the output of the activation function 1808 could be seen as a determination, based on the information gathered as of the time step that the growth sequence vector 1801 was provided as input, of whether the antimicrobial concentration of test well from which the growth information was gathered would successfully inhibit growth of the microorganism. For example, in the case of a sigmoid activation function, a value of 1 could be interpreted as a determination that growth would not be inhibited, a value of 0 could be interpreted as a determination that growth would be inhibited, and values between 0 and 1 could be treated as determinations of growth or no growth at confidence levels based on how close the value was to 0 or 1. This could then be compared with determinations of growth or no growth for the other test wells, and lowest concentration of antimicrobial in a test well with a determination of no growth could be treated as the MIC. Additionally, in some embodiments, an additional bias value (referred to as a dense bias) may be added to the output of the dense layer 1807 to cause the activation function 1808 to be triggered by different types of outputs, thereby allowing for further tuning.

[00083] Of course, it should be understood that, while some embodiments may use an architecture as shown in FIG. 19 for determining MIC, variations are also possible and may be implemented by those of ordinary skill in the art. For example, while some embodiments may include a network cluster 1803 which outputs a 1x1 matrix or single scalar value for each unit making up the cluster, it is also possible that a network cluster 1803 may provide output in the form of a set of scalar values/lxl matrices for each unit, where the set of scalar values/lxl matrices includes one value for each time step value from the growth sequence vector 1801. In such an embodiment, the dense layer 1807 may include a weight for each of the scalar values/lxl matrices for each of the units from the network cluster 1803, thereby allowing for more information to be considered in making the determination of whether the concentration of antimicrobial in a particular test well would or would not inhibit microorganism growth. Other variations on the dense layer 1807 are also possible, and may be made independent of the outputs of the network cluster 1803. For example, while the discussion of FIG. 18 referred to embodiments where the dense layer 1807 is implemented as a linear equation that applies weights to the output of the combination function, in other embodiments, a dense layer 1807 could be implemented in other manners, such as a multi-layer neural network with input nodes for each value provided by the combination function and one or more hidden layers through which those input notes would be connected to an output activation function. Accordingly, the above description of variations should be understood as being illustrative only, and should not be treated as limiting.

[00084] As another example of a type of variation that could be implemented by those of ordinary skill in the art, in some embodiments multiple instances of models using an architecture as shown in FIG. 18 could be used in determining MIC. For example, in some cases there could be multiple instances where each instance had different trainable layers (e.g., network clusters made up of LSTMs rather than GRUs, network clusters and/or dense layers trained with a bias toward predicting growth or no growth, network clusters with different numbers of units such as 16 or 32 rather than 24, etc.). In such embodiments, the multiple instances could be applied in parallel at each time step, and the determination of growth or no growth could be made using a voting protocol which would allow low confidence determinations from individual instances to be combined to make an earlier determination than would be possible using any of the individual instances on its own. As another example of how multiple instances of models using an architecture such as shown in FIG. 18 could be used, some embodiments may include multiple instances where each instance was trained to make a determination at a particular time step. For example, there could be one instance trained to make a determination four hours after inoculation, one instance trained to make a determination five hours after inoculation, and one instance trained to make a determination six hours after inoculation. In such an embodiment, if an instance was not able to make a determination with sufficient confidence, then the data collection process could continue until enough data had been gathered for a determination by the next instance, and this could then be repeated until ultimately a sufficiently confident determination could be made.

[00085] Variations are also possible in how a model following the architecture of FIG. 18 could be trained. For example, while it is possible that the trainable portions of a model following the architecture of FIG. 18 (set off using grey fill for ease of reference) could be trained using standard feedback and reinforcement techniques, it is also possible that some modifications could be made to facilitate the training. For instance, in some embodiments, an architecture such as shown in FIG. 18 could be modified for training purposes by inserting a filtering layer between the network cluster 1803 and the combination function 1806. Such a filtering layer could reduce the amount of data passed on from the network cluster 1803 (e.g., if the network cluster made a MIC determination for time steps separate from each other by 30 minutes, a filtering layer could pass on only the determinations made every hour to the combination function 1806). As another example, while some embodiments may perform training only on final outputs of a model (e.g., if the model was being trained to make a MIC determination after 6 hours, then the training would be on whether the determination made at 6 hours was correct), other embodiments may compare outputs at each time step with target outputs provided by the training data, so that a loss function and appropriate updating could be performed for all time steps where training data was available. Other variations are also possible and will be immediately apparent to those of ordinary skill in the art in light of this disclosure. Accordingly, the above description of training variations should be understood as being illustrative only, and should not be treated as limiting.

[00086] As an illustration of how these types of training approaches could be applied in practice, consider tables 5-12, below. In those tables, tables 5-9 provides GRU kernel weights, GRU recurrent weights, GRU bias weights, dense kernel weights, and a dense bias weight for a model implemented using the Keras API (an open source project governed by the Keras Special Interest Group and available at https://keras.io) and Tensorflow backend (an open source machine learning platform developed by Google, Inc. and available at https://www.tensorflow.org/) comprising a network cluster with 24 GRUs trained to make MIC determinations at 4, 5 and 6 hours based on training data for 56 microorganism strains when combined with various antimicrobials and the various combinations were labeled with growth or no growth information determined after 16 hours. In particular, table 5 provides a 1x72 array, in which the first 24 values would be mapped to Wz, the second 24 values would be mapped to Wr and the third 24 values would be mapped to Wh in an embodiment following FIG. 19. Table 6 provides a 72x24 array, in which the first 24x24 array (i.e., the first 24 bracketed sets of values) would be mapped to Uz, the next 24x24 array would be mapped to Ur and the third 24x24 array would be mapped to Uh in an embodiment following FIG. 19. Table 7 provides a 1x72 array, in which the first 24 values would be mapped to bz, the second 24 values would be mapped to br and the third 24 values would be mapped to bh in an embodiment following FIG. 19. Table 8 provides a 1x1344 array of values which would be multiplied by the output of the combination function 1806 in the dense layer 1807. Table 9 provides a bias value which would be added to the output of the dense layer 1807 before it was provided to the activation function 1808. Table 10 provides growth/no growth determinations made at 4, 5 and 6 hours by a system trained as described above for Pseudomonas aeruginosa and the drug Ceftazidime. Tables 11-12 provide microorganisms and antimicrobials included in the training data described above and for which the model was confirmed to be effective.

Table 5: Exemplary GRU Kernel Weights

Table 6: Exemplary GRU Recurrent Weights

Table 7: Exemplary GRU Bias Weights

Table 8: Exemplary Dense Kernel Weights

Table 9: Exemplary Dense Bias Weight

Table 10: Exemplary 4, 5 and 6 Hour MIC Determinations

Table 12: Exemplary Microorganisms and Microorganism Classes

[00087] It should be understood that, while the above description has focused on use of machine learning models to make MIC determinations, some embodiments may use such models for other purposes as well. For example, in some embodiments, a model using the architecture of FIG. 18 and trained to make MIC determinations as described above could be used to generate new models for deriving microbial counts from images, or even to bootstrap a complete MIC determination system. To consider how this may take place, consider the architecture of FIG. 21, which shows how an architecture as illustrated in FIG. 18 could be modified to train a model to provide counts that could subsequently be used in MIC determination.

[00088] In the architecture of FIG. 21, rather than taking a growth sequence vector 1801 as input, input regarding a test well would be provided in the form of an image sequence vector 2101 - e.g., a series of images of a test well such as could be provided by periodically imaging the test well as described herein. This image vector could then be processed by a machine learning model (e.g., a Keras API 2D convolution layer 2102) to extract salient features (e.g., signal strengths for how strongly particular pixel windows, such as 3x3 pixel windows in a NxN pixel image, match a particular pattern) from the images. This feature information could then be provided to a 2D transformation module 2103 which could put it in a form which could be combined with organism information in a manner similar to what was described previously in the context of FIG. 18 (e.g., by taking a global average of strength values from a Keras API 2D convolution layer 2102 to obtain a density value for an image, then duplicating that image to create a vector that could be combined with an organism vector as described previously). Meanwhile, the organism vector 1804, would be transformed with a 2D Convolution expansion module 2105, that could, in a manner similar to the expansion module 1805 of FIG. 18, duplicate the organism vector so that it is compatible for combination with the information from the image sequence vector 2101. The outputs of the convolution expansion module 2105 and the 2D transformation module 2013 could then be combined using an image input combination function 2014 (e.g., multiplication, concatenation), and the combined information could be fed to a dense layer 2106 adapted to provide a sequence of values that would take the place of the growth sequence vector 1801 in a model following the architecture of FIG. 18.

[00089] Using an architecture such as shown in FIG. 21, a model could be trained to generate microorganism counts by connecting the outputs to a model which had already been trained to make MIC determinations based on organism count information. The trainable portions of the MIC determination model could then be locked, and the count generation model could then be trained based on whether its output allowed the MIC determination model to make correct growth/no-growth determinations. Once the count generation model had been trained sufficiently to allow the MIC determination model to give reasonable answers, the trainable potions of the MIC determination model could be unlocked, and the combined model (i.e., count generation + MIC determination) model could be given end to end training as a further fine tuning measure.

[00090] IV. Exemplary Combinations [00091] It will be appreciated that embodiments of the invention may be implemented using a variety of different information processing systems. In particular, although the figures and the discussion thereof provide an exemplary computing system and methods, these are presented merely to provide a useful reference in discussing various aspects of the invention. Embodiments of the invention may be carried out on any suitable data processing device, such as a personal computer, laptop, personal digital assistant, mobile telephone, server computer, etc. Of course, the description of the systems and methods has been simplified for purposes of discussion, and they are just one of many different types of system and method that may be used for embodiments of the invention. It will be appreciated that the boundaries between logic blocks are merely illustrative and that alternative embodiments may merge logic blocks or elements, or may impose an alternate decomposition of functionality upon various logic blocks or elements.

[00092] It will be appreciated that the above-mentioned functionality may be implemented as one or more corresponding modules as hardware and/or software. For example, the above- mentioned functionality may be implemented as one or more software components for execution by a processor of the system. Alternatively, the above-mentioned functionality may be implemented as hardware, such as on one or more field-programmable-gate-arrays (FPGAs), and/or one or more application-specific-integrated-circuits (ASICs), and/or one or more digital-signal-processors (DSPs), and/or one or more graphical processing units (GPUs), and/or other hardware arrangements. Method steps implemented in flowcharts contained herein, or as described above, may each be implemented by corresponding respective modules; multiple method steps implemented in flowcharts contained herein, or as described above, may be implemented together by a single module.

[00093] It will be appreciated that, insofar as embodiments of the invention are implemented by a computer program, then one or more storage media and/or one or more transmission media storing or carrying the computer program form aspects of the invention. The computer program may have one or more program instructions, or program code, which, when executed by one or more processors (or one or more computers), carries out an embodiment of the invention. The term “program” as used herein, may be a sequence of instructions designed for execution on a computer system, and may include a subroutine, a function, a procedure, a module, an object method, an object implementation, an executable application, an applet, a servlet, source code, object code, byte code, a shared library, a dynamic linked library, and/or other sequences of instructions designed for execution on a computer system. The storage medium may be a magnetic disc (such as a hard drive or a floppy disc), an optical disc (such as a CD-ROM, a DVD-ROM or a BluRay disc), or a memory (such as a ROM, a RAM, EEPROM, EPROM, Flash memory or a portable/removable memory device), etc. The transmission medium may be a communications signal, a data broadcast, a communications link between two or more computers, etc.

[00094] The following examples relate to various non-exhaustive ways in which the teachings herein may be combined or applied. It should be understood that the following examples are not intended to restrict the coverage of any claims that may be presented at any time in this application or in subsequent filings of this application. No disclaimer is intended. The following examples are being provided for nothing more than merely illustrative purposes. It is contemplated that the various teachings herein may be arranged and applied in numerous other ways. It is also contemplated that some variations may omit certain features referred to in the below examples. Therefore, none of the aspects or features referred to below should be deemed critical unless otherwise explicitly indicated as such at a later date by the inventors or by a successor in interest to the inventors. If any claims are presented in this application or in subsequent filings related to this application that include additional features beyond those referred to below, those additional features shall not be presumed to have been added for any reason relating to patentability.

[00095] Example 1

[00096] A method comprising: (a) creating a plurality of test mixtures in a plurality of test wells, wherein: (i) each test mixture from the plurality of test mixtures is inoculated using a biological sample; (ii) each test mixture from the plurality of test mixtures comprises an antimicrobial solution comprising an antimicrobial agent; (iii) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that test mixture differs from the antimicrobial solution in each other test mixture from the plurality of test mixtures; and (iv) the same biological sample is used to inoculate each test mixture from the plurality of test mixtures; (b) incubating each of the test mixtures; (c) for each test mixture from the plurality of test mixtures, at a plurality of imaging times, wherein each of the imaging times takes place after incubation has begun, capturing an image of that test mixture; (d) obtaining a plurality of growth predictions by performing steps comprising, for each test mixture from the plurality of test mixtures, providing a data sequence to a machine learning model, wherein: (i) the data sequence comprises a plurality of input items; (ii) each input item from the plurality of input items corresponds to an imaging time from the plurality of imaging times; and (iii) the machine learning model is adapted to recognize, and to make growth predictions based on, temporal sequences; and (e) generating a MIC determination for the biological sample based on the plurality of growth predictions.

[00097] Example 2

[00098] The method of example 1, wherein: (a) the machine learning model comprises: (i) a network cluster comprising a plurality of recurrent neural networks; and (ii) a dense layer comprising a feed forward neural network; (b) obtaining the plurality of growth predictions comprises, for each test mixture from the plurality of test mixtures: (i) providing a temporal sequence based on the data sequence for that test mixture to each recurrent neural network from the plurality of recurrent neural networks; (ii) obtaining, from each recurrent neural network from the plurality of recurrent neural networks, an intermediate growth prediction for that test mixture; and (iii) obtaining, from the dense layer, a growth prediction for that test mixture based on the intermediate growth predictions for that test mixture obtained from the recurrent neural networks.

[00099] Example 3

[000100] The method of example 2, wherein: (a) the machine learning model is adapted to receive an identification of a microorganism corresponding to the biological sample; and (b) the dense layer is adapted to, for each test mixture, weight the intermediate growth predictions for that test mixture from the plurality of recurrent neural networks based on the identification of the microorganism.

[000101] Example 4

[000102] The method of example 2, wherein the machine learning model is adapted to, for each test mixture from the plurality of test mixtures, generate the temporal sequence based on the data sequence for that test mixture by performing steps comprising: (a) for each imaging time from the plurality of imaging times, obtaining a doubling value by applying a log base 2 transformation to the input item corresponding to that imaging time comprised by the data sequence for that test mixture; and (b) for each doubling value except the doubling value corresponding to a first imaging time, obtaining a doubling value change by subtracting a doubling value corresponding to a preceding imaging time.

[000103] Example 5

[000104] The method of example 2, wherein the plurality of recurrent neural network comprises 16 recurrent neural networks.

[000105] Example 6

[000106] The method of example 5, wherein the plurality of recurrent neural networks comprises 24 gated recurrent units.

[000107] Example 7

[000108] The method of example 1, wherein: (a) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that text mixture has a corresponding antimicrobial concentration which is different from the antimicrobial concentrations which correspond to the other test mixtures from the plurality of test mixtures; (b) the plurality of growth predictions comprises, for each test mixture from the plurality of test mixtures, a growth prediction corresponding to that test mixture; and (c) generating the MIC determination for the biological sample based on the plurality of growth predictions comprises determining that a lowest concentration corresponding to a test mixture with a corresponding growth prediction of no growth as the MIC determination.

[000109] Example 8

[000110] The method of example 7, wherein the plurality of test mixtures comprises a growth mixture, in which the corresponding antimicrobial concentration is no antimicrobial.

[000111] Example 9

[000112] A biological testing system comprising a processor configured with a set of computer instructions operable, when executed, to cause the system to perform a method comprising: (a) creating a plurality of test mixtures in a plurality of test wells, wherein: (i) each test mixture from the plurality of test mixtures is inoculated using a biological sample; (ii) each test mixture from the plurality of test mixtures comprises an antimicrobial solution comprising an antimicrobial agent; (iii) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that test mixture differs from the antimicrobial solution in each other test mixture from the plurality of test mixtures; and (iv) the same biological sample is used to inoculate each test mixture from the plurality of test mixtures; (b) incubating each of the test mixtures; (c) for each test mixture from the plurality of test mixtures, at a plurality of imaging times, wherein each of the imaging times takes place after incubation has begun, capturing an image of that test mixture; (d) obtaining a plurality of growth predictions by performing steps comprising, for each test mixture from the plurality of test mixtures, providing a data sequence to a machine learning model, wherein: (i) the data sequence comprises a plurality of input items; (ii) each input item from the plurality of input items corresponds to an imaging time from the plurality of imaging times; and (iii) the machine learning model is adapted to recognize, and to make growth predictions based on, temporal sequences; and (e) generating a MIC determination for the biological sample based on the plurality of growth predictions.

[000113] Example 10

[000114] The system of example 9, wherein: (a) the machine learning model comprises: (i) a network cluster comprising a plurality of recurrent neural networks; and (ii) a dense layer comprising a feed forward neural network; (b) obtaining the plurality of growth predictions comprises, for each test mixture from the plurality of test mixtures: (i) providing a temporal sequence based on the data sequence for that test mixture to each recurrent neural network from the plurality of recurrent neural networks; (ii) obtaining, from each recurrent neural network from the plurality of recurrent neural networks, an intermediate growth prediction for that test mixture; and (iii) obtaining, from the dense layer, a growth prediction for that test mixture based on the intermediate growth predictions for that test mixture obtained from the recurrent neural networks.

[000115] Example 11 [000116] The system of example 10, wherein: (a) the machine learning model is adapted to receive an identification of a microorganism corresponding to the biological sample; and (b) the dense layer is adapted to, for each test mixture, weight the intermediate growth predictions for that test mixture from the plurality of recurrent neural networks based on the identification of the microorganism.

[000117] Example 12

[000118] The system of example 10, wherein the machine learning model is adapted to, for each test mixture from the plurality of test mixtures, generate the temporal sequence based on the data sequence for that test mixture by performing steps comprising: (a) for each imaging time from the plurality of imaging times, obtaining a doubling value by applying a log base 2 transformation to the input item corresponding to that imaging time comprised by the data sequence for that test mixture; and (b) for each doubling value except the doubling value corresponding to a first imaging time, obtaining a doubling value change by subtracting a doubling value corresponding to a preceding imaging time.

[000119] Example 13

[000120] The system of example 10, wherein the plurality of recurrent neural network comprises 16 recurrent neural networks.

[000121] Example 14

[000122] The system of example 13, wherein the plurality of recurrent neural networks comprises 24 gated recurrent units.

[000123] Example 15

[000124] A computer program product comprising a non-transitory computer readable medium having stored thereon a set of computer instructions operable, when executed, to cause a biological testing system to perform a method comprising: (a) creating a plurality of test mixtures in a plurality of test wells, wherein: (i) each test mixture from the plurality of test mixtures is inoculated using a biological sample; (ii) each test mixture from the plurality of test mixtures comprises an antimicrobial solution comprising an antimicrobial agent; (iii) in each test mixture from the plurality of test mixtures, the antimicrobial solution in that test mixture differs from the antimicrobial solution in each other test mixture from the plurality of test mixtures; and (iv) the same biological sample is used to inoculate each test mixture from the plurality of test mixtures; (b) incubating each of the test mixtures; (c) for each test mixture from the plurality of test mixtures, at a plurality of imaging times, wherein each of the imaging times takes place after incubation has begun, capturing an image of that test mixture; (d) obtaining a plurality of growth predictions by performing steps comprising, for each test mixture from the plurality of test mixtures, providing a data sequence to a machine learning model, wherein: (i) the data sequence comprises a plurality of input items; (ii) each input item from the plurality of input items corresponds to an imaging time from the plurality of imaging times; and (iii) the machine learning model is adapted to recognize, and to make growth predictions based on, temporal sequences; and (e) generating a MIC determination for the biological sample based on the plurality of growth predictions.

[000125] Example 16

[000126] The computer program product of example 15, wherein: (a) the machine learning model comprises: (i) a network cluster comprising a plurality of recurrent neural networks; and (ii) a dense layer comprising a feed forward neural network; (b) obtaining the plurality of growth predictions comprises, for each test mixture from the plurality of test mixtures: (i) providing a temporal sequence based on the data sequence for that test mixture to each recurrent neural network from the plurality of recurrent neural networks; (ii) obtaining, from each recurrent neural network from the plurality of recurrent neural networks, an intermediate growth prediction for that test mixture; and (iii) obtaining, from the dense layer, a growth prediction for that test mixture based on the intermediate growth predictions for that test mixture obtained from the recurrent neural networks.

[000127] Example 17

[000128] The computer program product of example 16, wherein: (a) the machine learning model is adapted to receive an identification of a microorganism corresponding to the biological sample; and (b) the dense layer is adapted to, for each test mixture, weight the intermediate growth predictions for that test mixture from the plurality of recurrent neural networks based on the identification of the microorganism.

[000129] Example 18

[000130] The computer program product of example 16, wherein the machine learning model is adapted to, for each test mixture from the plurality of test mixtures, generate the temporal sequence based on the data sequence for that test mixture by performing steps comprising: (a) for each imaging time from the plurality of imaging times, obtaining a doubling value by applying a log base 2 transformation to the input item corresponding to that imaging time comprised by the data sequence for that test mixture; and (b) for each doubling value except the doubling value corresponding to a first imaging time, obtaining a doubling value change by subtracting a doubling value corresponding to a preceding imaging time.

[000131] Example 19

[000132] The computer program product of example 16, wherein the plurality of recurrent neural network comprises 16 recurrent neural networks.

[000133] Example 20

[000134] The computer program product of example 19, wherein the plurality of recurrent neural networks comprises 24 gated recurrent units.

[000135] V. Miscellaneous

[000136] It should be understood that, in the above examples and the claims, a statement that something is “based on” something else should be understood to mean that it is determined at least in part by the thing that it is indicated as being based on. To indicate that something must be completely determined based on something else, it is described as being “based EXCLUSIVELY on” whatever it must be completely determined by.

[000137] It should be understood that any of the examples described herein may include various other features in addition to or in lieu of those described above. By way of example only, any of the examples described herein may also include one or more of the various features disclosed in any of the various references that are incorporated by reference herein. [000138] It should be understood that any one or more of the teachings, expressions, embodiments, examples, etc. described herein may be combined with any one or more of the other teachings, expressions, embodiments, examples, etc. that are described herein. The above-described teachings, expressions, embodiments, examples, etc. should therefore not be viewed in isolation relative to each other. Various suitable ways in which the teachings herein may be combined will be readily apparent to those of ordinary skill in the art in view of the teachings herein. Such modifications and variations are intended to be included within the scope of the claims.

[000139] It should be appreciated that any patent, publication, or other disclosure material, in whole or in part, that is said to be incorporated by reference herein is incorporated herein only to the extent that the incorporated material does not conflict with existing definitions, statements, or other disclosure material set forth in this disclosure. As such, and to the extent necessary, the disclosure as explicitly set forth herein supersedes any conflicting material incorporated herein by reference. Any material, or portion thereof, that is said to be incorporated by reference herein, but which conflicts with existing definitions, statements, or other disclosure material set forth herein will only be incorporated to the extent that no conflict arises between that incorporated material and the existing disclosure material.

[000140] Having shown and described various versions of the present invention, further adaptations of the methods and systems described herein may be accomplished by appropriate modifications by one of ordinary skill in the art without departing from the scope of the present invention. Several of such potential modifications have been mentioned, and others will be apparent to those skilled in the art. For instance, the examples, versions, geometries, materials, dimensions, ratios, steps, and the like discussed above are illustrative and are not required. Accordingly, the scope of the present invention should be considered in terms of the following claims and is understood not to be limited to the details of structure and operation shown and described in the specification and drawings.