EVENT CORRELATION USING DATA HAVING

DIFFERENT TIME RESOLUTIONS

Cross Reference to Related Application

[0001]This application claims priority to U.S. Provisional Application Serial No. 61/769,590, filed February 26, 2013, the entirety of which is incorporated by reference.

Statement Regarding Federally Sponsored Research

[0002]This disclosure was made with government support under NIH grant ROl GM085808 awarded by the National Institutes of Health. Microfabrications were performed at the Cornell NanoScale Facility supported by NSF Grant ECS-0335765 awarded by the National Science Foundation. The government has certain rights in this disclosure.

Field of the Disclosure

[0003]This disclosure relates to a system, a device, a method and a program for obtaining high time resolution data from low time resolution data. This disclosure also relates to a system, a device, a method and a program for correlating, in time, events recorded with data having a different time resolution.

Background

[0004] Different events can be detected using devices having a different time resolution. For example, events can be detected using an imaging device. However, imaging devices typically have low time resolution frame rates when compared with events detected with high time resolution. A frame rate is limited by an exposure time and a readout time.

[0005] Other detection devices may not have these limitations. For example, changes in electric potential or other electrochemical events can be detected by devices with high time resolution. When the timing of the detected events is compared, the precision is low.

Summary

[0006] A method comprises storing a timing of a reference event. The reference event occurs during a frame time of an image frame from a plurality of imaging events. The frame time is an exposure time and a readout time for an imaging source. Each imaging event has at least one set of image frames. Each set of image frames is recorded simultaneously with the timing of the reference event and each set of image frames has a plurality of image frames. The method further comprises receiving at least one channel of image data for each imaging event from the imaging source. One channel of image data corresponds to one set of image frames, respectively. The method further comprises calculating, for each imaging event, image values for a representative frame set. The representative frame set is determined based on the at least one set of image frames. One image value is calculated for each image frame of the representative frame set. Each image value is based on a characteristic of a plurality of pixels within a respective image frame of the representative frame set. The representative frame set has the same number of image frames as the at least one set of image frames. The

characteristic is determined from the at least one channel of image data. The method further comprises assigning, for each imaging event, the calculated image value of a respective image frame as a trace value for each trace point during the exposure time of the respective image frame and a portion of the readout time before the exposure time of the respective image frame and a portion of the readout time following the exposure time of the respective image frame to generate an intensity trace for a respective imaging event. The assigning is performed for each image frame in the representative frame set for the respective imaging event. The method further comprises shifting in time, the intensity trace for each respective imaging event to align the timing of the reference event for each imaging event of the plurality of imaging events. The method further comprises averaging, trace values of trace points aligned in time from each shifted intensity trace to generate an average continuous intensity trace over time. The method further comprises determining, a timing of an event based on the average continuous intensity trace.

[0007]Also disclosed is a device comprising a storage device configured to store at least one channel of image data for a plurality of imaging events received from an imaging source and a timing of a reference event. Each imaging event has at least one set of image frames. Each set of image frames is recorded simultaneously with the timing of the reference event. Each set of image frames has plurality of image frames. The reference event occurs during a frame time of an image frame from the plurality of imaging events. The frame time is an exposure time and a readout time for the imaging source. The device further comprises a processor comprising hardware configured to: calculate, for each imaging event, image values for a representative frame set. The representative frame set is determined based on the at least one set of image frames. One image value is calculated for each image frame of the representative frame set. Each image value is based on a characteristic of a plurality of pixels within a respective image frame of the representative frame set. The representative frame set has the same number of image frames as the at least one set of image frames. The characteristic is determined from the at least one channel of image data. The processor is further configured to assign, for each imaging event, the calculated image value of a respective image frame as a trace value for each trace point during the exposure time of the respective image frame and a portion of the readout time before exposure time of the respective image frame and a portion of the readout time following the exposure time of the respective image frame to generate an intensity trace for a respective imaging event. The assigning is performed for each image frame in the representative frame set for the respective imaging event. The processor is further configured to shift, in time, the intensity trace for each respective imaging event to align the timing of the reference event for each imaging event of the plurality of imaging events. The processor is further configured to average, trace values of trace points aligned in time from each shifted intensity trace to generate an average continuous intensity trace over time. The processor is further configured to determine, a timing of an event based on the average continuous intensity trace. The device further comprises a display configured to display the average continuous intensity trace.

[0008]Also disclosed is a computer readable storage device having a program, which when executed by a computer, causes the computer to execute a method of: calculating, for each imaging event of a plurality of imaging events, where each imaging event has at least one set of image frames, each set of image frames being recorded simultaneously with the timing of the reference event and each set of image frames having plurality of image frames, image values for a representative frame set. The representative frame set is determined based on the at least one set of image frames. One image value is calculated for each image frame of the representative frame set. Each image value being based on a characteristic of a plurality of pixels within a respective image frame of the representative frame set. The representative frame set has the same number of frames as the at least one set of image frames. The characteristic is determined from the at least one channel of image data. The computer readable storage device having a program, which when executed by a computer, causes the computer to execute a method further comprising assigning, for each imaging event of the plurality of imaging events, the calculated image value of a respective image frame as a trace value for each trace point during the exposure time of the respective image frame and a portion of the readout time before exposure time of the respective image frame and a portion of the readout time following the exposure time of the respective image frame to generate an intensity trace for a respective imaging event. The assigning being performed for each image frame in the representative frame set for the respective imaging event. The computer readable storage device having a program, which when executed by a computer, causes the computer to execute a method further comprising shifting in time, the intensity trace for each respective imaging event to align the timing of a reference event for each imaging event of the plurality of imaging events. The reference event occurs during a frame time of an image frame in each of the plurality of imaging events. The frame time is an exposure time and a readout time for an imaging source. The computer readable storage device having a program, which when executed by a computer, causes the computer to execute a method further comprising averaging, trace values of trace points aligned in time from each shifted intensity trace to generate an average continuous intensity trace over time. The computer readable storage device having a program, which when executed by a computer, causes the computer to execute a method further comprising determining, a timing of an event based on the average continuous intensity trace.

Brief Description of the Figures

[0009]Aspects of the disclosure will now be described by way of non-limiting examples with reference to the following figures:

[0010]Fig. 1 illustrates an example of a system in accordance with aspects of the disclosure;

[0011]Fig. 2 illustrates a flow chart for a method of obtaining high time resolution data from low time resolution data in accordance with aspects of the disclosure;

[0012]Fig. 3 illustrates a flow chart for a method of comparing the timing of events in accordance with aspects of the disclosure;

[0013]Fig. 4 illustrates a block diagram of an example of a total internal reflection (TIR) fluorescence system; [0014]Fig. 5 illustrates a Brightfield image showing an electrode array;

[0015]Fig. 6 illustrates an example of a fluorescence footprint of a cell expressing SCORE which is placed on the electrode array;

[0016]Fig. 7 illustrates a cross sectional view of the electrode array along the dotted line of Fig. 6;

[0017]Fig. 8 illustrates a block diagram of an example of a signal conditioning system in accordance with aspects of the disclosure;

[0018]Fig. 9 illustrates a flow diagram for a method of obtaining high time resolution data from low time resolution image frames in accordance with aspects of the disclosure;

[0019]Fig. 10 illustrates a flow chart for a method of comparing timing of a single vesicle release event and a Fluorescence Resonance Energy Transfer in accordance with aspects of the disclosure;

[0020]Fig. 11 illustrates a flow chart for a method of determining the location of a single vesicle release event in accordance with aspects of the disclosure.

[0021]Figs 12 and 13 illustrate currents recorded by electrodes of an electrode array depicted in Fig. 5;

[0022]Fig. 14 depicts average continuous intensity traces determined in accordance with aspects of the disclosure;

[0023]Fig. 15 depicts average continuous intensity trace for Fluorescence Resonance Energy Transfer fitted with two step functions and average continuous intensity trace for

amperometrically detected spikes determined in accordance with aspects of the disclosure; [0024]Fig. 16 depicts squared deviations of the two fitted step response functions depicted in Fig. 15 from the average continuous intensity trace for Fluorescence Resonance Energy Transfer also depicted in Fig. 15;

[0025]Fig. 17 illustrates an average continuous intensity trace determined in accordance with aspects of the disclosure from image data simultaneously recorded with 903 amperometric spikes with no detectable foot signal and an average continuous intensity trace determined in accordance with aspects of the disclosure from measured current data of the same 903 amperometric spikes;

[0026]Fig. 18 illustrates an average continuous intensity trace determined in accordance with aspects of the disclosure from image data simultaneously recorded with 581 amperometric spikes that showed a detectable foot signal and an average continuous intensity trace determined in accordance with aspects of the disclosure from measured current data of the same 581 amperometric spikes;

[0027]Fig. 19A illustrate detected current values (solid lines) at each of the electrodes within an electrode array and a simulated current value (dashed lines) at each of the electrodes within the electrode array from a potential release point on a grid;

[0028]Fig. 19B illustrates an integrated current for the detected currents depicted in Fig. 19A and for the simulated currents depicted in Fig. 19A;

[0029] Fig. 19C is a contour graph showing a determined location of a single vesicle release event in accordance with aspects of the disclosure based on simulated values and measured values, where the values depicted are a root of a sum of squared deviations between measured and simulated charge ratios at different electrodes as a function of position;

[0030]Figs. 20A-20D illustrate results of a first shutter opening experiment in accordance with aspects of the disclosure;

[0031]Figs. 21A-21D illustrate results of a second shutter opening experiment in accordance with aspects of the disclosure; and [0032]Fig. 22 depicts average continuous intensity traces determined in accordance with aspects of the disclosure from a whole cell patch clamp experiment where a voltage pulse stimulated calcium influx into a cell which was recorded simultaneously with total internal reflection (TIR) fluorescence imaging.

DETAILED DESCRIPTION

[0033]Fig. 1 illustrates an example of a system 1 in accordance with aspects of the disclosure. The system 1 includes a Data Processing Device 100, a High Time Resolution Data Source 150 and a Low Time Resolution Data Source 160. The use of the terms "high" and "Low" herein means that one data source has a time resolution higher than the time resolution of the other data source and does not reflect a classification based on a threshold time resolution value.

[0034]The Data Processing Device 100 includes a Processor 105, Data Storage Devices 116, such as RAM 110, ROM 112 and Persistent Storage 114, Data Input Ports (collectively 120), a Display 125, and an Input Device 130. The Data Processing Device 100 also includes a bus (Data Bus 140), which provides communication between the Processor 105, Data Storage Device(s) 116 (including RAM 110, ROM 112 and Persistent Storage 114), the Data Input Ports 120, Display 125 and Input Device 130.

[0035] Although, not shown, the Data Processing Device 100 can also include a

communication unit such as a network interface card.

[0036]In an aspect of the disclosure, the Processor 105 is configured to execute one or more programs stored in a computer readable storage device. The computer readable storage device can be RAM 110, persistent storage 114 or removable storage. For example, the Processor 105 can execute instructions in a program that may be loaded into RAM 110.

[0037]Computer readable storage device may be in a tangible or hardware form, such as, an optical or magnetic disc that is inserted or placed into a drive or other portion or device for transfer onto an internal storage device, such as a hard drive. Additionally, the computer readable storage device also may take the form of a hard drive, a thumb drive, or a flash memory that is connected to Data Processing Device 100. [0038]The Processor 105 may include one or more processing units.

[0039]RAM 110, ROM 112 and Persistent Storage 114 are only examples of Data Storage Devices 116. A storage device is any piece of hardware that is capable of storing information, such as, for example without limitation, data, programs, instructions, program code, and/or other suitable information either on a temporary basis and/or a permanent basis.

[0040]Fig. 1 depicts RAM 110, which hereinafter is used interchangeably with "Working Storage Device"; however, any other suitable volatile memory can be used.

[0041]Persistent Storage 114 can take various forms. For example, Persistent Storage 114 can contain one or more components or devices. For example, Persistent Storage 114 may be a hard drive (e.g., HDD), a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The medium or media used by Persistent Storage 114 also may be removable. For example, a removable hard drive may be used.

[0042]The Input Device 130 can provide a connection for user input such as, but not limited to, through a keyboard, a mouse, a touch panel, a touch screen or pad, and/or some other suitable input device.

[0043]The Display 125 provides a way to display information to a user. The Display 125 can be, but is not limited to, LCD or plasma.

[0044]Data received from the High Time Resolution Data Source 150 and the Low Time Resolution Data Source 160 can be initially stored in Persistent Storage 114. The data can be directly received or directly received from the High Time Resolution Data Source 150 and the Low Time Resolution Data Source 160. Although, not shown in Fig. 1, the data can be pre-processed by a signal conditioning system prior to being input into the Data Processing Device 100.

[0045]The data can be copied or moved to the Working Data Storage. Additionally, when the Processor 105 processes the data, the processed data can be stored in the Working Data Storage. The processed data can also be copied or moved to Persistent Storage 114. [0046]The data is received from the High and Low Time Resolution Sources (150 and 160) via Data Input Ports 120i and 120 ₂ , respectively. While Fig. 1 shows two data input ports, any number of data input ports can be included in the Data Processing Device 100 and two are only shown as representative ports.

[0047]The Data Input Ports 120 can be any interface between the Data Processing Device 100 and an external device, such as, but not limited to, serial and parallel connections, Digital Video Interface (DVI), High -Defin tion Multimedia Interface (HDMI), Small Computer System Interface (SCSI), Peripheral Component Interconnect (PCI) or USB.

[0048] Additionally, Fig. 1 shows one High Time Resolution Data Source 150 and one Low Time Resolution Data Source, however, any number of High and Low Time Resolution Data Sources can be used. For example, a High Time Resolution Data Source 150 can be correlated with two different events detected by two different Low Time Resolution Data Sources.

[0049] A High Time Resolution Data Source 150 can be, but is not limited to, a rapid signal recorded with an array of electrodes, a shutter opening from an illumination source, application of a field stimulation pulse from a current or voltage source, application of a voltage or current stimulus, measurement of a stimulated or spontaneous rapid current or voltage signal such as an action potential with a suitable amplifier, an optical pulse source such as a laser or a flash lamp,, a photodiode or photomultiplier signal, or any device or means for generating or probing a rapid perturbation or a device or means that can produce or detect a rapid phenomena.

[0050] A High Time Resolution Data Source 150 can be used to detect events such as, but not limited to, single vesicle release events, flash illuminations, calcium influx, electrical stimulation of a cell, and other electrophysiological stimulus, electrophysiologically recorded events, such as action potential, optically produced events, such as, uncaging of an active compound such as calcium, protons, neurotransmitters, optical activation of a genetically encoded light sensitive protein such as channelrhodopsin or light sensitive groups chemically linked to a molecule of interest. [0051] A Low Time Resolution Data Source 160 can be, but is not limited to, an imaging source such as a CCD, an EMCCD, a photomultiplier or photodiode signal, an audio source, a temperature or chemical sensor, or a sensor detecting any physical quantity of interest, and a device having a data or signal integrated over a low time resolution period. Low Time Resolution Data Sources 160 can be used to detect events such as, but is not limited to, Fluorescence changes with of specific proteins such as, GFP, CFP and Venus, and others, Fluorescence Resonance Energy Transfer (FRET), Fluorescence changes or fluorescence ratio changes of probes sensing ion concentration, pH, membrane potentials, or other quantities, any light intensity change including fluorescence and absorption, any other physical quantities acquired by data or signal integration. A gate pulse for the low resolution signal can be used to synchronize the high time resolution data with the low time resolution data. Additionally or alternatively, an integration start pulse or an integration end pulse can be used.

[0052]In accordance with aspects of the disclosure, any high time resolution event can be correlated in time with a low time resolution event which is simultaneously recorded. For example, any of the high time resolution events described herein and can be correlated with any low time resolution event described herein, which is simultaneously recorded.

[0053] Additionally, the Data Processing Device 100 can include programming environments, tools or packages which can be used to generate graphs, charts, and for data analysis or image processing. For example, an environment, tool or package can be, but is not limited to, Igor Pro (from Wavemetrics), Lab VIEW (National Instruments Corporation) or Matlab® (The MathWorks, Inc). These environments, tools and/or packages can be configured or customized by a script or user code, program code and/or user defined modules to execute functionality described herein. The script, user code, program code or user defined modules are examples of the programs, instructions, and/or program code disclosed herein.

[0054]These environments, tools or packages such as, Lab VIEW, Igor Pro and/or Matlab® are well known in the art and will not be described herein. While certain environments, tools, and packages have been described herein, any environment, tool or package can be used. [0055]Fig. 2 illustrates a flow chart for a method of obtaining high time resolution data from low time resolution data in accordance with aspects of the disclosure.

[0056]In an aspect of disclosure, the data from both the Low Time Resolution Data Source 160 and the data from the High time Resolution Data Source 150 is recorded simultaneously. Multiple Sets of Low/High time resolution data are recorded. A Set of Low/High time resolution data refers to one iteration of the data collection. The number of sets can vary and can be application specific.

[0057]A synchronization signal ("synch signal") can be used to synchronize start times for Set of Low/High time resolution data. A synch signal can be received from at least one of the High Time Resolution Data Source 150 and the Low Time Resolution Data Source 160 indicating a start of the recording of the data.

[0058]At Step 200, the Data Processing Device 100 receives one set of high time resolution data from the High Time Resolution Data Source 150 via one of the Data Input Ports (e.g., 120i _). The high time resolution data is stored in Persistent Storage 114 at step 205. The high time resolution data will subsequently be copied or moved to Working Data Storage 110 for processing.

[0059]At Step 210, the Data Processing Device 100 receives one set of low time resolution data from the Low Time Resolution Data Source 160 via another of the Data Input Ports (e.g., 120 _2). The low time resolution data is stored in Persistent Storage 114, step 215. The low time resolution data will subsequently be copied or moved to Working Data Storage 110 for processing.

[0060]Fig. 2 shows steps 200 and 205 before steps 210 and 215; however, this is for descriptive purposes only. The data from the data sources, High Time Resolution Data Source 150 and Low Time Resolution Data Source 160 can be received in any order or concurrently.

[0061]A set of low time resolution data is copied or moved to Working Data Storage 110. At Step 220, the Processor 105 calculates a representative value for each data point within the set of low time resolution data. The set of low time resolution data may include one or more data points in a time sequence. The representative value can be based on an intensity of each data point, e.g., magnitude. In an aspect of the disclosure, each data point may include a plurality of values. The representative value can be calculated by a mean of the plurality of values, the highest value of the plurality of values, and the mode of the plurality of values. For example, if the data point is an image frame, the data point includes a plurality of values indicating intensity of pixels within an image frame. In this case, the representative value can be an average intensity of the pixel values from more than one pixel in a region of interest ("ROI").

[0062]The Processor 105 stores the representative value for each data point in one set of low time resolution data in Working Data Storage 110.

[0063]At Step 225, the Processor 105 creates an intensity trace for one set of the low time resolution data. For each data point, in one set of the low time resolution data, the Processor 105 assigns the representative value as the trace value for a preset time within an intensity trace. The preset time equals a time resolution of the low time resolution data. For example, if each data point is generated every 1 second, the time resolution is 1 second and the representative value is assigned as the trace value for each trace point for 1 second.

[0064]For an imaging source, the time resolution includes the exposure time and a readout time. For other sources, the time resolution may also include a gap or dead time. The trace values for the intensity trace for the set of low time resolution data is stored in Working Data Storage 110.

[0065] Additionally, the trace values can be stored in Persistent Storag

[0066] A set of high time resolution data is copied or moved to Working Data Storage 110. The set of high time resolution data also may have one or more data points in time sequence. At Step 230, the Processor 105 examines the data points to determine a time of the reference event. The time of the reference event may be determined based on a rate of change of the data points in the time sequence. In another aspect of the disclosure, the time of the reference event may be determined based on a magnitude of each data point in the time sequence. In another aspect of the disclosure, the time of the reference event may be determined based on a comparison of the magnitude of each data point in the time sequence with a magnitude threshold. In another aspect of the disclosure, the timing of the reference event may be determining using a linear or nonlinear fit between two or more points with respect to a baseline value.

[0067]At Step 235, the Processor 105 stores the determined time of the reference event, e.g., timing, in Working Data Storage 110. Additionally, the timing can be stored in Persistent Storage 114.

[0068] Steps 200-235 are repeated for each set of Low/High time resolution data received from the Low Time Resolution Data Source 160 and High Time Resolution Data Source 150.

[0069] At step 240, a determination is made if there are additional sets of Low/High time resolution data. If there are additional data sets ("Y" at step 240), steps 200-235 are repeated for the additional data sets. If all of the data sets are processed ("N" at step 240), the process proceeds to step 245.

[0070]The Processor 105 retrieves the intensity trace for each set of the low time resolution data and the corresponding reference times from the Working Data Storage 110.

[0071]At Step 245, the Processor 105 shifts the intensity trace for each set of the low time resolution data to align the timing of the reference event for each intensity trace. A trace may not need to be shifted if the reference event is already aligned with the reference event in other intensity traces. Each individual intensity trace will be desynchronized with respect to each other, but aligned with respect to the timing of the reference event.

[0072]At Step 250, the Processor 105 averages the trace values for the aligned trace points from each of the shifted intensity trace. For example, if there are 100 data sets (100 intensity traces), the processor 105 averages the trace values for the 100 trace points which are aligned in time. The average trace values are used to create an average continuous intensity trace. The average continuous intensity trace has the same resolution as the high time resolution data. [0073]The average continuous intensity trace is stored in the Working Data Storage 110. Additionally, the average continuous intensity trace can be displayed on a Display 125. The average continuous intensity trace can also be stored in the Persistent Storage 114.

[0074]Fig. 3 illustrates a flow chart for a method of comparing the timing of events in accordance with aspects of the disclosure.

[0075]At Step 300, the Processor 105 creates an intensity trace of one set of high time resolution data. Trace values can be determined based on a magnitude or average of the magnitudes of detected signal(s). For example, the trace values can be the magnitude of detected current or voltage value over a period of interest, e.g., the recording time.

[0076]While Fig. 3 describes certain steps separately, the functionality described therein can be performed concurrently with the steps described in Figure 2. For example, when the Processor 105 determines the timing of the reference event, the Processor 105 can create the intensity trace for the set of high time resolution data. The intensity trace for each set of high time resolution data is also stored in Working Data Storage 110. Additionally, the intensity trace for each set of high time resolution data can be also stored in Persistent Storage 114.

[0077]Step 300 is repeated for each set of high time resolution data. At step 305, a determination is made if there are additional sets of high time resolution data. If there are additional sets of high time resolution data ("Y" at step 305), step 300 is repeated for the additional set. If all of the sets of high time resolution data sets are processed, e.g. "N" at step 305), the process proceeds to step 310.

[0078]The Processor 105 retrieves each intensity trace from the Working Data Storage 110. At step 310, the Processor 105 generates an average image intensity trace for the high time resolution data. While step 310 is described separately, when the Processor 105 shifts the intensity traces generated from the sets of low time resolution data to align the timing of the reference event (at step 245), the Processor 105 can also generate the average continuous intensity trace for the high time resolution data. The intensity traces for the high time resolution is similarly shifted to align the traces based on the timing of the reference event. In an aspect of the disclosure, the timing of the reference event centered on the average continuous intensity trace (t=0) for the high time resolution data.

[0079]The Processor 105 averages the trace values for the aligned trace points from each of the individual intensity traces for the respective sets of high time resolution data. For example, if there are 100 data sets (100 intensity traces), the Processor 105 averages the trace values for the 100 trace points which are aligned in time. The average trace values are used to create the average continuous intensity trace. The average continuous intensity trace is stored in the Working Data Storage 110. Additionally, the average continuous intensity trace can be also stored in Persistent Storage 114.

[0080] At Step 315, the Processor 105 superimposes the average continuous intensity trace created from the sets of low time resolution data with the average continuous intensity trace created from the sets of high time resolution data. The superimposed traces can be displayed on the display 125.

[0081]At Step 320, the Processor 105 determines a timing of an event that occurred during the recording period using the average continuous intensity trace created from the sets of low time resolution data.

[0082] The event may be indicated by a change of intensity of traces values over time, a rate of change of the trace values in the time sequence. In another aspect of the disclosure, the event may be indicated by a magnitude of each trace value in the time sequence. In another aspect of the disclosure, the event may be indicated based on a comparison of the magnitude of each data point in the time sequence with a magnitude threshold.

[0083]For example, in an aspect of the disclosure, the timing of the event is a time when a trace value has a value 50% of the maximum intensity for the average continuous intensity trace.

[0084] Further, the average continuous intensity trace generated from the sets of low time resolution data can be compared and fit to a theoretical continuous intensity trace, e.g., a step response function. In another aspect of the disclosure, the timing can be the time were a trace value has a value 50% of the maximum intensity for the step function fit to the average continuous intensity trace.

[0085]At Step 325, the Processor 105 compares the timing of the event and the timing of the reference event.

[0086]In another aspect of the disclosure, the timing of the reference event, the timing of event and other data can be also stored in the Persistent Storage 114 or other storage device for long term storage.

[0087]The above described method, system, device and program for obtaining high time resolution data from low time resolution data can be applied to any high time resolution data source and any low time resolution data source where the data is simultaneously recorded with multiple iterative recordings.

[0088] For example, the method, system, device and program can be applied to imaging sequences from an imaging source used to detect a fluorescence resonance energy transfer simultaneously recorded with a high time resolution data. The high time resolution data can be obtained, for example, using an electrode array. Other high time resolution sources can be a photodetector.

[0089] Additionally, the above-described method of comparing data from two data sources can be applied to determining a relative timing difference between the FRET event and a single vesicle release event.

[0090]Fig. 4 illustrates a block diagram of an example of a total internal reflection (TIR) fluorescence system for detecting an FRET. The TIR fluorescence system includes an Illumination Section 470, a Microscope 423, a Beam Splitting Section 480 and an Imaging Section 490.

[0091]The Illumination Section 470 includes a Light Source 405, a Filter(s) 410, and a Dichroic 415. The Illumination Section 470 can also include light Guide System/Mirrors 420. The Light Source 405 can be a lamp. For example, a mercury arc lamp (made by X-cite®, 120PC) can be used as the Light Source. The light is channeled through an Excitation Filter 410. The Excitation Filter 410 can be configured to let a certain wavelength band through. For example, a 436/10 nm excitation filter can be used. The filtered light travels through a dichroic 415, such as a 455 dichroic. The Illumination Section 470 can include one or more light guides or mirrors 420 to channel or focus the light path, as needed.

[0092]Fig. 4 illustrates the Illumination Section 470 separate from the Microscope 423, however, portions of the Illumination Section 470 can be included within a housing of the Microscope 423.

[0093]Fig. 4 depicts one example of an Illumination Section 470, however, other illumination sections including ones that have different light source(s) can be used such as the illumination sections and light sources described in Total Internal Reflection Fluorescence Microscopy in Cell Biology, D. Axelrod, Traffic 2001; 2: 764-774, the entirety of which is incorporated by reference.

[0094]The light generated by the Illumination Section 470 is channeled to a Microscope 423. For example, the Microscope 423 can be a Zeiss Axiovert 135 TV microscope. Other microscopes can be used as well such as the microscopes described in Total Internal

Reflection Fluorescence Microscopy in Cell Biology, D. Axelrod, Traffic 2001; 2: 764-774, the entirety of which is incorporated by reference.

[0095]The internal structure of a Microscope 423 such as the Zeiss Axiovert 135 TV microscope is well known in the art and therefore will not be described in detail herein. Fig. 4 illustrates certain components of the Microscope 423 for illustrates purposes only.

[0096]The Microscope 423 can be fitted with an opaque disk positioned at an equivalent upbeam of the back focal plane of the Objective 430 (aperture plane), to produce a sharp- edge shadow image such that only supercritical light passes through the objective 430.

Certain microscopes mark the aperture plane. If not marked, the aperture plane can be created using two lenses. [0097] Other configuration can be used such as described in Total Internal Reflection Fluorescence Microscopy in Cell Biology, D. Axelrod, Traffic 2001; 2: 764-774, the entirety of which is incorporated by reference.

[0098]The Microscope 423 also includes an Objective 430 and a Connector 435. The Objective 430 can be, but is not limited to, a Zeiss Plan-Fluar 1.45 NA 100X oil objective. The objective 430 can be below the Sample Holder 425 (inverted) or above the Sample Holder 425 (upright).

[0099]The Connector 435 is configured to be coupled to a Beam Splitting Section 480. For example, a DC2™ by Photometries® can be used as a Beam Splitting Section 480, e.g., beam splitter. Other optical beam splitters can be used, such as one of the OptoSplit devices by Cairn Research, Ltd. or Dual View™, by Optical Insights, LLC. Elements 440-450 would be included in the beam splitter. The Connector 435 would typically be located at the top of the housing of the Microscope 423.

[0100] A dichroic 440 can be used to divide the received light into two channels. For example, a 505 nm dichroic can be used. Two filters (445 and 450), e.g., emission filters, can be configured to allow wavelengths in a predefined band. One filter can be configured to allow, e.g., 465-495 nm band, for a CFP channel. Another filter can be configured to allow, e.g., 520-550 nm, for a Venus channel. The Beam Splitting Section 480 can also include one or more Guide System/Mirrors 455 to channel light to an Imaging Source. An Imaging Source be can an EMCCD camera 460, such as (iXon series by Andor Technology pic), with the associated imaging program. The EMCCD 460 outputs the image data to the Data Processing Device 100. The image data includes two simultaneously recorded channels of data, e.g., 465-495 nm and 520-550 nm.

[0101] The associated imaging program is stored in the Persistent Storage 114 and can be copied or moved to Working Data Storage 110 during data processing.

[0102] The EMCCD camera 460 has a frame rate equal to the sum of the exposure time and readout time. For example, the exposure time can be approximately 200 ms and the readout time, e.g., readout interval, can be approximately 19 ms, where a frame can be generated every 219ms. The EMCCD camera 460 has a pixel resolution of M by N pixels per frame. M and N are integers and can be equal.

[0103] The EMCCD camera 460 also transmits a TTL signal to the Data Processing Device 100. The TTL signal indicates exposure times for the frames in order to synchronize a timing of the image frames and the data from the high time resolution source.

[0104] Cell sample(s), under testing can be prepared in accordance with the method described in T.D. Parsons et al., Docked Granules, the Exocytic Burst, and the Need for ATP Hydrolysis in Endocrine Cells, Neuron 15, 1085-1096, November 1995,the entirety of the description of which is incorporated herein by reference.

[0105] A single vesicle release event ("release event") can be amperometrically detected by an electrode array 500. The release event described herein includes, but is not limited to, exocytotic catecholamine release.

[0106] Fig. 5 illustrates an electrode array (collectively 500) which includes four electrodes 500A D as seen from an example of a Brightfield image. A Brightfield image is an image generated using white light. The Brightfield image can be generated prior to total internal reflection (TIR) fluorescence imaging to determine a location of the electrode array.

Alternatively, the Brightfield image can be generated afterward.

[0107] Generating a Brightfield image is well known and will not be described herein in detail. In an aspect of this disclosure, the same microscope used in the total internal reflection (TIR) fluorescence system can be used for the Brightfield imaging. The light source is a white light source. The Illumination Section 470 (external) and the Beam Splitting Section 480 is not used. The same image detector, e.g., EMCCD camera can also be used.

[0108] The electrode array 500 is depicted as black images within the 12 μιη v. 12 μιη image (-6 μιη to 6 μιη) in both the vertical and horizontal direction. The 4 Pt conductors, e.g., electrodes 500A D, which are depicted in black, are covered by a translucent (and invisible) insulating layer that was removed in the region outlined by the dashed circle 505. Within the dashed circle 505, the tips of 4 electrodes are exposed for the detection of single vesicle release events, e.g., fusion events. Fig. 5 also depicts a grid 510 within the dashed circle 505, e.g., 500 nm grid, as illustrated by the dots. The grid 510 is used as potential release locations for release simulation, which will be described later, for providing expected fractions of molecules detected by the 4 individual electrodes for these locations.

[0109] The electrode array 500 can be microfabricated by contact photolithography, patterning 4 Pt electrodes on a glass coverslip (shown in Fig. 7 as 700) with a space of ~5μιη between them. A 5 μιη legend is depicted in Fig. 7. However, the spacing can vary depending on a size of the sample. The electrode array 500 is located between the glass coverslip 700 and the cell membrane.

[0110] Fig. 6 illustrates an example of a fluorescence footprint of a cell expressing SNARE Complex Reporter (SCORE) which is placed on the electrode array 500. The fluorescence footprint was generated using a light source of approximately 436 nm. Fig. 6 represents an average of 1000 exposures without subtracting for a background. For descriptive purposes the electrodes have been labeled A-D in Figs 5 and 6. The bottom of Fig. 6 shows the intensity scale for the pixels. The intensity of each pixel can be digitized giving a value between a minimum and a maximum value, e.g., gray scale level. The level depends on the number of bits in the A/D conversion. Fig. 6 depicts a 14 bit A/D conversion, e.g., values between 0 and 16383. However, different A/D conversion bits can be used, e.g. 0 to 255.

[0111] Like Fig. 5, the electrode array 500 is depicted in black. The average fluorescence in a circular area between the electrodes (dashed circle 610) with -2.5 μιη radius can serve as control for measurements at release sites.

[0112] Fig. 7 illustrates a cross sectional view of the electrode array 500 along the dotted line of Fig. 6. As depicted in Fig. 7, the heights of the electrode (depicted as back and labeled) 705 can be varied as well. For example, the height of the electrodes can be 150 nm. Above the electrodes, an insulation layer(s) can be formed 710. For example, the insulation layers can be 300 nm thick. The cells and electrode array 500 are placed in a solution (not shown). For example, the solution can contain (in mM) 140 NaCl, 5 KC1, 5 CaCl ₂ , 1 MgCl ₂ , 10 Hepes/NaOH, and 20 glucose. [0113] Fig. 8 illustrates a block diagram of an example of a signal conditioning system used for initially processing signals output from the electrode array 500.

[0114] Signals from each electrode (e.g., 500A D ) can be amplified for further processing by an N-Channel Amplifier 800. The electrodes 500 are electrically coupled to the N-Channel Amplifier using photoresist-insulated wires coupled to a contact pad.

[0115] The number of channels for the amplifier is the same as the number of electrodes. For example, a four-channel amplifier model number VA-10M4, manufactured by NPI Electronic GmbH, Tamm, Germany can be used. Each amplified signal can be filtered by a Filter 805. For example, a 500Hz filter can be used. The Electrode Array 500 outputs an analog signal. The analog signal is converted into a digital signal for further processing by the Data Processing Device 100 by an A/D Converter 810. A 16-bit A/D converter can be used. The sample rate can vary. For example, a sample rate of 1kHz made be used. The signals received from the Signal Conditioning System can be analyzed by the Data Processing Device 100 using any of the above described configured environments, tools and packages.

[0116] Although, the Signal Conditioning System is described separately from the Data Processing Device 100, the some components can be included in the Data Processing Device 100. Additionally, the signals from the electrode array 500 can be digital filtered.

[0117] Fig. 9 illustrates a flow chart for a method of obtaining high time resolution data from low time resolution image frames from an imaging source in accordance aspects of the disclosure.

[0118] The data from the EMCCD 460 and the data from the electrode array 500/ signal conditioning system are simultaneously recorded. The light from the Light Source 405 is turned on, the light traverses the Illumination Section 470, a Microscope 423 (and sample with cell), a Beam Splitting Section 480 and arrives at the EMCCD 460 in two channels. Simultaneously, a photocurrent is detected by the electrode array 500 as the light is emitted through the sample. [0119] Data from the EMCCD 460 and the electrode array 500/ signal conditioning system is recorded for a period of interest. The period of interest includes a time before the Light Source 405 is turned on and a time after. The EMCCD 460 generates image data for image frames. The frame rate for the image frame is a sum of the exposure time and the readout time. For example, if the exposure time is 200 ms and the readout time is 19 ms, one image frame represents 219 ms of time and set of new frames is generated every 219 ms. Since there are two channels of images, two frames are simultaneously recorded, one for each channel and the EMCCD 460 generates image data for each channel. The number of frames generated by the imaging source 460 depends on the period of interest. For each channel, image data for a plurality of frames is generated. A plurality of frames in the same channel is referred to as a set of image frames. Since there are two channels, there are two sets of image frames.

[0120] The electrode array 500/Signal Conditioning System generates data continuously over the period of interest.

[0121] A plurality of data sets is iteratively generated, repeatedly turning on and off the Light Source 405. The plurality of data sets generates a plurality of imaging events.

[0122] At step 900, the Data Processing Device 100 receives the electrochemical signals, e.g., current values, detected by each electrode 500A-D in the electrode array 500 via the Signal Conditioning System. For example, the Data Processing Device 100 receives four different continuously detected current values for the period of interest, one current value detected by an electrode, respectively. At step 905, the values are stored in the Persistent Storage 114, step 205. The values will be subsequently moved or copied to Working Data Storage 110 for processing.

[0123] At Step 910, the Data Processing Device 100 receives the two channels of image data from the EMCCD 460, e.g. two sets of image frames. As noted above, one channel represents emission in the CFP channel and the other represents emissions in the Venus channel. Each set of image frames contains a plurality of image frames over a period of interest. For example, the period of time can be 10 second before and after turning the light on from the Light Source 405. [0124] At Step 915, the two channels of image data are stored in Persistent Storage 114. The image data will be subsequently moves or copied to Working Data Storage 110 for processing.

[0125] At Step 920, the Processor 105 generates a representative set of image frames from the two channels of image data. A FRET is determined from a ratio of the Venus

channel/CFP channel. The plurality of image frames from the Venus channel and the plurality of image frames from the CFP channel are aligned in time. Pixel values for frames in the representative set of image frames are determined based on the relative intensities of corresponding pixels from corresponding frames in time in the Venus channel and CFP channels. For example, the image data is defined by gray levels. In an aspect of the disclosure, a background is subtracted from the gray levels of the image data from both channels. A ratio is subsequently calculated. In other aspects of the disclosure, the image data can be defined in other color spaces, such as RGB.

[0126] The pixel data for the representative set of image frames is stored in Working Data Storage 110. The representative set of image frames is dependent on an event of interest. For example, in a case where the comparison is between the CFP channel and the

amperometrically detected data, such as release event, the representative set of image frames could be the plurality of image frames from the CFP channel. Other events of interest may use a sum of the image data from both channels.

[0127] At Step 925, the Processor 105 calculates a representative value for a frame of the representative set of image frames. During the first pass through Step 925, the Processor 105 calculates the representative value for the first frame. Successive times performing Step 925, the Processor 105 calculates the representative value for the next successive frame in the representative set of image frames. The representative value can be calculated as the mean of intensity value of a plurality of pixels proximate to a ROI, the highest intensity value of the plurality of pixels proximate to the ROI , and the mode of the intensity value of the plurality of pixels proximate to the ROI. [0128] The number of pixels used and the location of the pixel can be determined based on a determined location of the single vesicle release event (or any other simultaneously recorded event).

[0129] At step 930, the Processor 105 assigns the calculated representative value to all trace points within the exposure time of the image frame and a portion of a readout time before exposure time of the image frame and a portion of the readout time after the exposure time. The exposure time is determined from TTL signals received concurrently with the two sets of image frames from the imaging source. The TTL signal is indicative of the exposure time.

[0130] In an aspect of the disclosure, the portion can be 50% of both the readout time before and after the exposure time. In another aspect of the disclosure, the portion of the readout time before and the portion of the readout time after do not have to be equal. However, the sum of the two portions, before and after, should equal the total readout time between successive exposures. Therefore, the representative value is assigned to trace points having time duration of the frame rate.

[0131] At step 935, representative values and the trace values assigned for the image frame are store in the Working Data Storage 110. While, Fig. 9 illustrates that the data being stored in step 935 for description purposes, the data can be stored in the Working Data Storage 110 after the respective steps.

[0132] At step 940, the Processor 105 determines if there are unprocessed frames in the Working Data Storage 110. If the Processor 105 determines that there are unprocessed frames (Y at step 945), the Processor 105 returns to step 925 and repeats Steps 925-940. Steps 925- 940 are repeated for each frame in the representative set of image frames.

[0133] If the processor 105 determines that there are no unprocessed frames remaining (N at step 940), the processor 105 generates an intensity trace for the representative set of image frames at Step 945.

[0134] At step 950, the Processor 105 retrieves the trace values for each image frame of the representative set of image frames, from the Working Data Storage 110. The Processor 105 aggregates the trace values in time beginning with the first image frame and continuing with each successive frame to generate the intensity trace. The intensity trace is stored in the Working Data Storage 110. The intensity trace can also be displayed on a display 125.

[0135] At step 950, the Processor 105 determines the timing of the release event, e.g., reference event, from the current values, detected by each electrode 500A-D and which were received and stored.

[0136] The amplitudes of the spike depend on a diffusion distance between the release site and the respective electrode. The largest and earliest signal comes from the electrode closest to the release site. It reports the time when the vesicle's cargo is released (the "spike starting time"), e.g., when the fusion pore dilates explosively, as well as the type of release event. An amperometric foot signal (AFS) spike is indicative of slow catecholamine release through a narrow fusion pore. The foot duration reports the delay between opening of a narrow fusion pore and its subsequent dilation. In a non-AFS spike, the spike rises abruptly to a peak and declines, as a fusion pore opens, immediately dilates, and allows the rapid escape of catecholamine.

[0137] In an aspect of the disclosure, the timing of the release event is based on the change of the current intensity over time, e.g., a time derivative. The first and largest spike is examined to determine the time of the release event. Initially, a baseline current is determined and a time of the maximum spike (T _max ). Then, the amplitude of the maximum current (I _max ) is determined along with a half width (t^) and a charge Q. A risetime and slope thereof are derived from a linear fit of the current between two points (upper and lower point) on the rising portion of the spike. The upper point is less than 90% I _max to prevent the inclusion of the upper portion of the spike, which may have been flattened or adjusted by filtering. The lower point can be between 20% -50% of the I _max . This avoids including the foot. For example, the spike starting time can determined by extrapolating a linear fit of a 50%-90% rising phase back to the baseline. The linear fit lines are indicated by a dashed line along the first detected spike, in time, in Figs. 12 and 13. Figs. 12 and 13 will be described later when discussing the results of an experiment applying aspects of the disclosure. Other methods of determining the spike starting time can be used such as methods described in Analysis of Exocytotic Events Recorded by Amperometry, E. Mosharov et al., Nature Methods, Vol. 2 No. 9, September 2005, pp. 651-658, the entirety of which is incorporated by reference.

[0138] At step 955, a determination is made if there are additional unprocessed data sets. In an embodiment, 1000 data sets are generated by the EMCCD 460 and electrode array 500/ signal conditioning system, respectively.

[0139] If there are additional data sets ("Y" at step 955), steps 900-950 are repeated for each additional sets of data. If all of the data sets are processed, (e.g., "N" at step 955), the process proceeds to step 960.

[0140] The Processor 105 retrieves the intensity trace for each set of the representative image frames and the corresponding timing of the release event (e.g., reference event) for the data set from the Working Data Storage 110.

[0141] At Step 960, the Processor 105 shifts, in time, the intensity trace for each set of the representative image frames to align the timing of the release event for each intensity trace. An intensity trace may not need to be shifted in time if the release event already is aligned with the release event in other intensity traces. Each individual intensity trace will be desynchronized with respect to each other, but aligned with respect to the timing of the release event.

[0142] At Step 965, the Processor 105 averages the trace values for each of the aligned trace points from each of the shifted intensity trace. For example, if there are 1000 data sets (1000 intensity traces), the Processor 105 averages the trace values for the 1000 trace points which are aligned in time. The average trace values are used to create an average continuous intensity trace for the time period of interest. The average continuous intensity trace has the same resolution and the high time resolution data.

[0143] This average continuous intensity trace is stored in the Working Data Storage 110. Additionally, the average continuous intensity trace can be displayed on a display 125. The average continuous intensity trace can also be stored in Persistent Storage 114. [0144] Fig. 10 illustrates a flow chart for a method of comparing the timing of the release event with the timing of a FRET.

[0145] At Step 1000, the Processor 105 creates a continuous intensity trace for sets of data from the electrode array/signal conditioning system.

[0146] The current data measured by the closest electrode in the electrode array to the release event over the time period of interest is used as the trace values, e.g., the electrode that was used to determine the timing of the release event.

[0147] While Fig. 10 describes certain steps separately, the functionality described therein can be performed concurrently with the steps described in Figure 9. For example, when the Processor 105 is determining the timing of the release event, the processor 105 can create the continuous intensity trace (Step 950). The continuous intensity trace for each set of data from the electrode array is also stored in Working Data Storage 110.

[0148] Step 1000 is repeated for each data set. At step 1005, a determination is made if there are additional unprocessed data sets. If there are additional unprocessed data sets ("Y" at step 1005), step 1000 is repeated for each unprocessed data set. If all of the sets of data from the electrode array are processed, (e.g. "N" at step 1005), the process proceeds to step 1010.

[0149] The Processor 105 retrieves each continuous intensity trace from the Working Data Storage 110. At step 1010, the Processor 105 generates an average image intensity trace for the all of sets of data from the electrode array (each iterative recording). Each continuous intensity trace is shifted to align the timing of the release event in each continuous intensity trace.

[0150] While step 310 is described separately, when the Processor 105 shifts the traces in step 960 to align the timing of the release event (the data for which simultaneously recorded), the Processor 105 can also generate the average continuous intensity trace for the sets of data from the electrode array. [0151] At step 1010, the Processor 105 averages the trace values for the aligned trace points from each of the individual continuous intensity traces. For example, if there are 1000 data sets (1000 continuous intensity traces), the Processor 105 averages the trace values for the 1000 trace points which are aligned in time. The average trace values are used to create an average continuous intensity trace.

[0152] The average continuous intensity trace is stored in the Working Data Storage 110. Additionally, the average continuous intensity trace can be stored in the Persistent Storage 114.

[0153] At Step 1015, the Processor 105 superimposes the average continuous intensity trace created for the imaging events (e.g., FRET) with the average continuous intensity trace created for the sets of data from the electrode array. The superimposed traces can be displayed on the display 125.

[0154] At Step 1020, the Processor 105 determines a timing of an event of interest, e.g., FRET. The average continuous intensity trace (of the imaging data) is used to determine timing of a FRET. The average continuous intensity trace for the imaging events, e.g., FRET time course, is characterized by a gradual increase of the trace value over the time span of two frame rates. In an aspect of the disclosure, the timing of an event, e.g., a FRET, is determined based on a time of the half maximum trace value (e.g., the time where the trace exhibits an amplitude of 50% of the maximum intensity amplitude) for the average continuous intensity trace. The average continuous intensity trace can be fitted with a step response function. In another aspect of the disclosure, the timing of the event, e.g., FRET, is determined based on the trace values of the step function fit to the average continuous intensity trace.

[0155] At Step 1025, the Processor 105 calculates the difference between the timing of the event and the timing of the release event.

[0156] The average continuous intensity trace for the FRET can be compared with a theoretically continuous intensity trace, e.g., step function. For a sequence frames with exposure times t _e and a readout time t _gap , a step response function /of the averaged traces obtained from steps with fixed amplitude A at time t ₀ can be calculated theoretically and can be expressed as follows: i t < t ₀ - t _t - -&- / = 0

if t > t ₀ + t _e +- f = A

[0157] Fig. 11 illustrates a flow chart for a method of determining a location of the release site. The location of the release site can define an ROI for the calculating the representative value of an image frame including determining the location and number of pixels used for the calculation. Precise determination of the site is important to correlate the FRET signals with a single exocytic event, e.g., release event.

[0158] Molecules released from a chromaffin granule diffuse isotropically in the space between cell membrane and coverslip as indicated by fluorescence imaging.

[0159] When a cell is placed in the space between surface patterned electrodes (see Fig. 6) a certain fraction of the catecholamine molecules released in a single exocytic event will be detected by the different electrodes, e.g., 500A-D, depending on the distance of the particular electrode from the release site with the largest fraction being oxidized at the electrode closest to the release site.

[0160] The location is determined based on a series of comparisons between stimulated integrated current values over time, from different potential release locations (of each electrodes in the electrode array) and measured integrated current values over time (from the same electrodes in the electrode array).

[0161] At Step 1100, the measured, e.g., detected current values at each electrode 500A D is integrated over time by the Processor 105. Fig. 5 depicts four electrodes in the electrode array, and therefore, four different integrated current values over time are generated, one for each electrode. By integrating the currents, the charges corresponding to a number of molecules detected by each electrode are obtained.

[0162] Multiple release points on the grid (depicted in Fig. 5) are stimulated by the Processor 105. The simulation may or may not assume that the release from individual sites is instantaneous. Diffusion is modeled using Monte Carlo simulation. A predetermined number of molecules per vesicle is used for the simulation. For example, 100,000 molecules per vesicle can be used. Other simulation parameters include a diffusion coefficient and distance between cell membrane and electrode array, size of the cell and topology including the surface bottom. For a given distance r, between a simulated release event and an electrode, a 2-dimensional diffusion time is r /4D, where D is the diffusion coefficient. The distance r and the geometry of the electrode are determined by imaging.

[0163] The diffusion time reflects an average time of arrival of a molecule at the distance r. Spike starting time reflects an arrival of a first molecule.

[0164] At step 1110, the Processor 105 determines a simulated current for each electrode, e.g., electrodes 500A D- The simulated current is integrated over the same time as above by the Processor 105 at step 1115.

[0165] For example, four different integrated current values over time are generated, one for each electrode. For each electrode, the integrated currents (both simulated and measured) are compared to calculate a Deviation at step 1120. In the case of four electrodes, four different comparisons are made and four Deviations are determined by the Processor 105.

[0166] At step 1125, the Processor 105 squares each of the Deviations. For example, each of the four calculated Deviations is squared. At step 1130, the Processor 105 adds each of the squared Deviations to generate a sum of the squared Deviations (S). For example, the four squared deviations are added together.

[0167] This process is repeated for each grid point used to simulate a release, e.g. steps 1105- 1135. For example, a 20 x 20 grid points can be used. The spacing can be 500 nm. However, a larger or smaller number of grid points can be used with smaller or larger spacing. Steps 1105-1135 are repeated for each of the 20 x 20 grid points. At step 1135, the Processor 105 determines if there are any other grid points for simulating a release site. If there are other grid points for simulating a release site ("Y" at step 1135), the Processor 105 returns to step 1105; otherwise, the process moves to step 1140.

[0168] In reality, the release location does not usually fall exactly on a grid point as shown in Fig. 5, but somewhere in between four grid points, e.g., four lattice points. At step 1140, the Processor 105 estimates the simulated fractional charges for arbitrary release locations by linear interpolation of the simulated fractional changes using the four lattice points surrounding the arbitrary release location. Steps 1105-1130 are performed for the arbitrary release locations to estimate the sum of squared Deviations (S) for each arbitrary release location.

[0169] S is used to determine the location of the actual release site. At step 1145, the position of the recorded exocytic event, e.g., release site is determining by fitting the release location to give a best fit minimizing S.

Experimental Results:

FRET and Single Release Event Experiment

[0170] An experiment was conducted to determine the temporal relation between a FRET increase and a release event.

[0171] The experiment used aspects of the disclosure to obtain high time resolution data from low time resolution image frames and subsequently compare the obtained data with simultaneously recorded data. The image data was obtaining using a total internal reflection (TIR) fluorescence system as described in Fig 4 and an electrode array as illustrated in Figs. 5 and a signal conditioning systemas depicted in Fig. 8. The signal conditioning system included an A/D converter board which, was connected to the Data Processing Device 100.

[0172] The sample used in the experiment included the DNA of SCORE which was subcloned into a viral vector pSFVl (Semliki Forest, Invitrogen). Bovine chromaffin cells were prepared in accordance with the method described in T.D. Parsons et al., Docked Granules, the Exocytic Burst, and the Need for ATP Hydrolysis in Endocrine Cells, Neuron 15, 1085-1096November 1995, the entirety of the description of which is incorporated herein by reference.

[0173] Illumination was provided by a mercury arc lamp (X-cite®, 120PC) through a 436/10nm excitation filter and a 455 dichroic. The fluorescence image was separated into CFP and Venus components with an image splitter (Dual View™, by Optical Insights, LLC) fitted with a 505nm dichroic, and emission filters at 465-495 nm for the CFP channel and 520-550 nm for the Venus channel. For each recording, a 1000 frame image sequence was collected with 200ms exposure time and 19 ms readout interval using an EMCCD camera (iXon series by Andor Technology pic) and its accompanying software such that one frame was acquired every 219 ms simultaneously for both channels. The image data was input to the Data Processing Device 100 via a PCI controller card. The Data Processing Device contained a program for performing the functionalities described herein.

[0174] A Zeiss Axiovert 135 TV microscope was equipped with a Zeiss Plan-Fluar 1.45 NA 100X oil objective and was modified for objective-type evanescent- field illumination. The Objective was below the sample, e.g., inverted.

[0175] A bovine chromaffin cell expressing SCORE was picked up with a micropipette and gently pressed onto the space between electrodes as depicted in Fig. 7. Experiments were performed in a solution containing (in mM) 140 NaCl, 5 KC1, 5 CaCl ₂ , 1 MgCl ₂ , 10

Hepes/NaOH, and 20 glucose. A TLL signal from the EMCCD was used to determine the exposure time for each frame. The TTL signal was used to determine the start and end time of the exposure portion of each frame to assign the representative value to trace points during the exposure time and a portion of the readout time before and after the exposure time. The TTL signal can also be used to identify the frame in which the spike occurred. [0176] Figs 12 and 13 illustrate amperometric currents recorded by the 4 electrodes of an electrode array 500. Each figure depicts 4 intensity traces for a portion of the time period of interest. The dashed curve shows a portion of intensity trace 1200D on an expanded vertical scale. In Fig. 12, the intensity traces are labels 1200A D to correspond with the electrode labels. The legend to the right of the intensity traces illustrates the scale in time and amplitude. The spike starting time is labeled and was determined using the linear best fit (identified by dashed line 1205) as described above. The foot duration is labels and illustrated by a double arrow. The foot starting time was taken as the first point that exceeds the baseline current noise by >2 standard deviations and the foot duration as the time interval from foot starting time to spike starting time. Fig. 12 illustrates an AFS-spike.

[0177] Fig. 13 illustrates a non-AFS spike. Native' foot signals with duration shorter than 33% of the 50%-90% rise time were categorized as non-AFS events. In Fig. 13 the intensity traces are labels 1300A D to also correspond with the electrode labels. The legend to the right of the intensity traces illustrates the scale in time and amplitude. The spike starting time is labeled and was determined using the linear best fit (identified by dashed line 1305) as described above.

[0178] Fig. 14 depicts average continuous intensity traces for both the imaging event (e.g., FRET) and average amperometric spike (release event) as a result of the experiment. The average continuous intensity traces were for 903 aligned spikes and imaging events. In the experiment, the spike can be considered the reference event or the high time resolution data and the image data can be considered the low time resolution data.

[0179] Each individual intensity trace for the imaging event was aligned with respect to the timing of the spike.

[0180] The dark wavy line 1400 depicts an average continuous intensity trace for an AFS and the dotted wavy line 1405 depicts the average continuous intensity trace for a non-AFS, both determined in accordance with aspects of the disclosure. The solid smooth line 1410 depicts a continuous intensity trace for a theoretical continuous intensity trace, e.g., a step response function. The step response function in Fig. 14 is with a fixed delay. [0181] The vertical dash line indicates the spike starting time (t=0) (identified as 1415). The solid lighter colored line 1420 depicts the average amperometric spike of the AFS and the dotted lighter colored line 1425 depicts the average amperometric spike of the non-AFS.

[0182] The horizontal dashed line indicates the 50% FRET change levels, e.g., 50% of maximum intensity (identified as 1430).

[0183] Fig. 14 reveals a clear delay between the FRET change and the spike. A delay of 91+1 ms between FRET change for an AFS event and the onset of the amperometric spike (t=0) indicating the time of fusion is shown. A delay of 120+1 ms between FRET change for an AFS event and the onset of the amperometric spike (t=0) indicating the time of fusion is shown. The delay is 30 ms longer for the AFS events than the non-AFS events.

[0184] Evidently, the FRET change is temporally correlated with the opening of the fusion pore and not its dilation. It thus indicates a conformational change in the SNARE complex that leads to subsequent fusion of the vesicle.

[0185] Fig. 15 depicts average continuous intensity traces for both the imaging events (e.g., FRET) and average amperometric spike (release event) as a result of the experiment (similar to Fig. 14). The average continuous intensity traces were for 903 aligned spikes and imaging events. The average continuous intensity trace for the imaging events 1500 was fitted to two step response functions 1505 and 1510 in accordance with the disclosure. The step response function 1505 is with a fixed delay (similar to Fig. 14). The step response function 1510 is with a convolution of step response with exponential distribution of 128 ms time constant. The vertical dash line indicates the spike starting time (t=0) (identified as 1520). The solid lighter colored line 1515 depicts the average amperometric spike.

[0186] Fig. 16 depicts squared deviations of the two fitted step response functions to the average continuous intensity trace. Deviation 1600 corresponds to step response function 1505 and Deviation 1605 corresponds to step response function 1510.

[0187] The FRET change in Fig. 15 is abrupt. If fusion followed a step in FRET with first- order kinetics, then the delays between FRET change and spike starting time would be exponentially distributed, and the step response function must be convolved with that exponential distribution (step response function 1510). A delay of 90 ms (the time of 50% maximum intensity) was obtained with an exponential time constant τ=128 ms using the step response function 1510. However, the rising phase of the step response function 1510 is shallower and does not fit the average continuous intensity trace 1500 as well as the step response with fixed delay (step response function 1505) with a 2.5-fold increase of χ . The data is better explained if FRET changes in a step followed by fusion with a fixed delay of approximately 90 ms.

[0188] Figs. 17-18 depict continuous intensity traces for both the imaging events (e.g., FRET) and average amperometric spike (release event) as a result of the experiment. Fig. 17 illustrates a average continuous intensity trace 1700 determined in accordance with the disclosure from image data simultaneously recorded 903 amperometric spikes with no detectable foot signal. Spikes 1705 were aligned at time of onset (t=0). Fig. 17 illustrates an extended view of the corresponding intensity traces from Fig. 14. In Fig. 17, the period of time is 10 seconds before and after the spikes.

[0189] Fig. 18 illustrates an average continuous intensity trace 1800 determined in accordance with the disclosure from image data simultaneously recorded with 581 amperometric spikes that showed a detectable foot signal. Spikes (green) 1805 were aligned at the transition point from foot signal to spike (t=0). The legend in both Figs. 17- 18 depicts the amplitude scale. The period of time for the observation was 10 seconds before and after the spikes.

[0190] The location of the release site was determined by fitting the release location to give a best fit minimizing a sum of squared deviations S between simulated and measured fractional charges for the 4 electrodes. The simulated values were determined by Random Walk Simulation (RWS). The coordinates of the electrodes were determined from the geometrical outline of the electrodes was obtained from imaging. The distance between the cell membrane and electrode array surface was set to 0.1 μιη and the diffusion

The simulation zone was within the 500 nm grid as depicted in Fig. 5. Random Walk Simulations (RWS) were performed to determine the fractional charges measured by the individual electrodes for release sites within the simulation zone. The simulation produces a unique current for each electrode. These currents were integrated as described above.

[0191] Figs. 19A illustrate the detected current values at the electrodes 1900A-D (solid) and the simulated current value at the electrodes 1905 A D (dotted) for one potential release site. Fig 19B illustrates the integrated current 1920 A D for the detect currents and for the simulated currents 1925 A D from Fig. 19B. Fig. 19C is a contour graph.. The contour graph showing the root of the sum of squared deviations between measured and simulated charge ratios as a function of X and Y positions. The deviation at the determined releasing site (dark mark) of the release event is 0.003.

[0192] The traces representing imaging events in Figs. 14-18 were determined using pixel coinciding with the determined release site, e.g., 2 x 2 pixels from each frame. The representative value for each frame was generated using the pixel intensities for these pixels. The representative intensity value for a frame was assigned to all the trace values for trace points for the exposure time for the frame and half the readout time before and after the frame. This assignment was repeated for each frame to generate each individual intensity trace.

Shutter Opening Experiments

[0193] Other experiments were conducted in accordance with aspects of the disclosure. A coverslip was placed on a sample holder of an inverted microscope. The coverslip had an electrode array on its surface. The electrode array was connected to a signal conditioning system. The signal conditioning system was connected to a Data Processing Device, e.g., 100. Current was detected by the electrode array, e.g., 500. Illumination was provided by a mercury arc lamp (X-cite®, 120PC) through a 436/10nm excitation filter and a 455 dichroic. An EMCCD camera (iXon series by Andor Technology pic) and its accompanying software were used to capture images such that one frame was acquired every 219 ms. The EMCCD camera was coupled to the microscope via the connector. The whole camera image was analyzed for intensity to determine the representative value for a frame. [0194] The image data was input to the Data Processing Device 100 via a PCI controller card. The Data Processing Device contained a program for performing the functionalities described herein.

[0195] A shutter opening produced a stepwise increase in light intensity. The increase light intensity initiated a photocurrent on the electrode array (e.g., 500) that marked the time of shutter opening. In the experiment, either the shutter opening or the detection of the photocurrent can be considered the reference event or the high time resolution event. The image data from the EMCCD camera can be considered the low time resolution event.

[0196] Figs. 20A-20D illustrate the results of the shutter opening experiment. The imaging frame during which the shutter opens reports a brightness that depends on the precise timing of shutter opening during the frame. When the shutter opens shortly after the exposure begins, the brightness in this frame is high (Fig. 20A). When the shutter opens in the middle of the frame the brightness is 50% (Fig. 20B), and when the shutter opens near the end of the exposure, the intensity remains rather low (Fig. 20C).

[0197] The image intensity (calculated value) was assigned to each trace point within the exposure time and half of the readout time before and after the exposure time for the frame. In the shutter experiment, each trace point represents 1 ms of time. The traces for each imaging event 2000A-2000 _c (collectively 2000) were aligned with respect to the shutter opening, e.g., reference event. Figs. 20A-20C illustrate the shutter opening with an arrow. Figs 20A-20C depict the intensity traces for the imaging events shifted in time to align the traces with respect to the shutter opening. Figs. 20A-20C illustrate examples of individual intensity trace which have been shifted. The intensity trace for the detected current by the electrode array 2005A-C is also depicted in Figs. 20A-20C. When the frames for each imaging event are aligned with respect to the time of shutter opening (reference event) with sub-frame time resolution, as indicated by the onset of the photocurrents, the imaging frames from the different imaging events are desynchronized. The image intensity reflected in the intensity trace for the frames in Figs 20A-20C are given in arbitrary units. The frame intensity in Fig. 20D is normalized. [0198] Figs. 20A-20C illustrate three imaging events. The shutter experiment included 260 imaging events. Fig. 20D illustrates the average continuous intensity trace 2010 for the 260 imaging events. Fig. 20D also includes an average continuous intensity trace 2020 for the current detected by the electrode array, e.g., 500, simultaneously with each of the 260 imaging events. Fig. 20A illustrates a legend showing the scale for the magnitude of current. Image frame times are indicated by vertical lines in Figs. 20A-20C and include 200 ms exposure time (black horizontal bars) and 19ms inter frame interval.

[0199] Fig. 20D also depicts a step response function 2030 fitted to the averaged continuous intensity trace 2020 for the 260 imaging events. Fig. 20D shows a 50% rise time at t=0+2 ms. The averaged continuous intensity trace is characterized by a gradual increase of the traces over the time span of two frames.

[0200] A second shutter experiment was conducted using a photodiode instead of the electrode array. Figs. 21A-21D illustrate the results of the shutter opening experiment with the photodiodes.

[0201] . A coverslip was placed on a sample holder of an inverted microscope. A photodiode was mounted on top of the coverslip to detect a light step change. The photodiode was connected to an A/D converter (e.g., a signal conditioning system without an amplifier). The A/D converter was connected to a Data Processing Device, e.g., 100. Illumination was provided by a mercury arc lamp (X-cite®, 120PC) through a 436/lOnm excitation filter and a 455 dichroic. An EMCCD camera (iXon series by Andor Technology pic) and its

accompanying software were used to capture images such that one frame was acquired every 219 ms. The EMCCD camera was coupled to the microscope via the connector. The whole camera image was analyzed for intensity to determine the representative value for a frame.

[0202] The image data was input to the Data Processing Device 100 via a PCI controller card. The Data Processing Device contained a program for performing the functionalities described herein.

[0203] Intensity traces for the intensity traces for the imaging events are labeled 2100A C _. The intensity traces of the measured voltage from the photodiode is labeled 2105A C The intensity traces for the measured voltage shows noise. The experiment did not attempt to filter the noise. However, the noise can be filtered out.

[0204] The average continuous intensity trace generated from 262 imaging events is labeled 2110. The average continuous intensity trace generated from 262 voltage measurements is labeled 2120. Fig. 21D also depicts a step response function 2130 fitted to the averaged continuous intensity trace 2120 for the 262 imaging events. Fig. 21 A illustrates a legend showing the scale for the magnitude of voltage. Fig. 21D show a 50% rise time at t=-0.05+ .08 ms.

Whole Cell Patch Clamp Experiment

[0205] Aspects of the disclosure were applied to determine a time relationship between a FRET change and calcium influx into a cell in response to a voltage pulse. Imaging of the cell was performed using the system described above in the experiment section. A TTL signal from the EMCCD was used to determine the exposure time of each frame. The whole cell footprint was analyzed for intensity to determine fluorescence, e.g., a representative value for an image frame. For a given cell, between 1000-1300 pixels were used to generate the representative intensity value for each frame. The representative intensity value for a frame was assigned to all the trace values for trace points for the exposure time for the frame and half the readout time before and after the frame. This assignment was repeated for each frame to generate the individual intensity trace.

[0206] The cell was prepared as described above in the experiment section. Calcium influx into the cell was detected using a patch clamp amplifier (model EPC-P by HEKA Instruments Inc.) The patch clamp amplifier was connected to the Data Processing Device 100. A patch electrode was pressed onto a cell surface. A glass pipette was used for suctioning to rupture a portion of the cell membrane. The glass pipette had an electrolyte solution which was in communication with the intracellular fluid when the membrane was ruptured. A voltage pulse was applied to the electrode controlling a voltage of the cell membrane and the current detected by the electrode was recorded simultaneously with the imaging of the cell. The simultaneous recording was repeated over 2400 times. The voltage pulse had a duration of 25 ms. The voltage pulse stimulated a calcium influx into the cell. Before and after the voltage pulse, a constant sine wave voltage was applied to the cell. The current was measured in pA

[0207] Fig. 22 depicts the results of the experiment showing average traces for the current and FRET determined in accordance with aspects of the disclosure. Over 2500 individual intensity traces for the detected current and FRET were shifted and averaged to generate the average continuous intensity traces for the current and the FRET, respectively. The individual intensity traces for the FRET were shifted to align the timing of the start of the pulse, e.g., reference event for each trace. In accordance with aspects of the disclosure, the reference event could also have been the detected current.

[0208] The average continuous intensity trace for the detected current (calcium influx) is depicted as trace 2200. A negative spike occurs almost immediately after the pulse onset. The average continuous intensity trace is centered at T=0, the time the voltage pulse commenced. The average continuous intensity trace for the FRET is depicted as trace 2205. Trace 2205 was fitted to a step response function 2210 (with no delay). Based on the half maximum intensity time, the FRET change occurred 25-30 ms after the onset of the voltage pulse. The two vertical dashed lines 2215 represents the duration of the voltage pulse.

[0209] The horizontal axis is a time after the onset of the voltage pulse in seconds. The vertical axis includes both the current (CA current) in pA and the Venus/CFP ratio, e.g., FRET. The change in the FRET ratio was approximately 0.2%.

[0210] The word "comprising", "comprise", "comprises", "including", "include" or

"includes" as used herein should not be viewed as excluding additional elements. The singular article "a" or "an" as used herein should not be viewed as excluding a plurality of elements. Unless the word "or" is expressly limited to mean only a single item exclusive from other items in reference to a list of at least two items, then the use of "or" in such a list is to be interpreted as including (1) any single item in the list, (2) all of the items in the list, or (3) any combination of the items in the list. The use of numbers, such as "first" or "second," is for distinguishing otherwise identical terminology, and is not intended to imply that operations or steps must occur in any particular order, unless otherwise indicated. [0211] Various aspects of the present disclosure may be embodied as a program, software, or computer instructions embodied or stored in a computer or machine usable or readable medium, or a group of media which causes the computer or machine to perform the steps of the method when executed on the computer, processor, and/or machine. A program storage device readable by a machine, e.g., a computer readable medium, tangibly embodying a program of instructions executable by the machine to perform various functionalities and methods described in the present disclosure is also provided, e.g., a computer program product.

[0212] The computer readable medium could be a computer readable storage device or a computer readable signal medium. A computer readable storage device, may be, for example, a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing; however, the computer readable storage device is not limited to these examples except a computer readable storage device excludes computer readable signal medium. Additional examples of the computer readable storage device can include: a portable computer diskette, a hard disk, a magnetic storage device, a portable compact disc read-only memory (CD-ROM), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical storage device, or any appropriate combination of the foregoing; however, the computer readable storage device is also not limited to these examples. Any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device could be a computer readable storage device.

[0213] A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, such as, but not limited to, in baseband or as part of a carrier wave. A propagated signal may take any of a plurality of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium (exclusive of computer readable storage device) that can communicate, propagate, or transport a program for use by or in connection with a system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0214] The terms "Data Processing Device" and "System" as may be used in the present disclosure may include a variety of combinations of fixed and/or portable computer hardware, software, peripherals, and storage devices. The Data Processing Device may include a plurality of individual components that are networked or otherwise linked to perform collaboratively, or may include one or more stand-alone components. The hardware and software components of the Data Processing Device of the present disclosure may include and may be included within fixed and portable devices such as desktop, laptop, and/or server, and network of servers (cloud).

[0215] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed concurrently, substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

[0216] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting the scope of the disclosure and is not intended to be exhaustive. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. Specific examples and embodiments describe herein were chosen and described in order to best explain the principles of the disclosure and the practical application thereof, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.