CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims all benefit including priority to United States Provisional Patent Application 62/610,545, filed December 27, 2017, and entitled:“Method, System and Computer Program Product for Tracking Defects”, which is hereby incorporated by reference in its entirety.

FIELD

[0002] The present disclosure generally relates to the field of machine learning, and in particular to a system and method for tracking incidents.

INTRODUCTION

[0003] Defect tracking tools are used by individuals and companies to monitor the functionality of their products, such as equipment or software, used by their customers. More specifically, defect tracking tools are used to keep track of equipment failure or software failure. Tracked records are used by manufacturers for statistical purposes as well as research and development of future products.

[0004] Although defect tracking tools are useful for manufacturers, these tools are not used to provide users troubleshooting or solutions to overcome the equipment failures or software failures encountered. Further, users are unaware of whether the issues are common, whether other users are experiencing the same or similar issues, or whether solutions are readily available. The only remedy available to users experiencing functionality issues is contacting the manufacturer, which is inconvenient and time-consuming, as well as subject to policies that effectively assign low priority to reporting such issues.

SUMMARY

[0005] In one embodiment, there is provided an incident report tracking system. The report tracking system comprises at least one processor and at least one memory comprising a sequence of instructions which when executed by the at least one processor cause the at least one processor to obtain an incident report, obtain incident anchor data from the incident report, obtain enrichment data associated with the incident report, normalize the anchor data and the enrichment data into data capsules, analyze the data capsules to infer one or more insights, and report the one or more insight.

[0006] In another embodiment, there is provided a method of tracking incident reports.

The method comprises obtaining an incident report, obtaining incident anchor data from the incident report, obtaining enrichment data associated with the incident report, normalizing the anchor data and the enrichment data into data capsules, analyzing the data capsules to infer one or more insights, and reporting the one or more insight.

[0007] In another embodiment, there is provided a computer-readable medium. The computer-readable medium comprises a memory that stores a sequence of instructions which when executed by the at least one processor cause the at least one processor to obtain an incident report, obtain incident anchor data from the incident report, obtain enrichment data associated with the incident report, normalize the anchor data and the enrichment data into data capsules, analyze the data capsules to infer one or more insights, and report the one or more insight.

[0008] In various further aspects, the disclosure provides corresponding systems and devices, and logic structures such as machine-executable coded instruction sets for implementing such systems, devices, and methods.

[0009] In this respect, before explaining at least one embodiment in detail, it is to be understood that the embodiments are not limited in application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.

[0010] Many further features and combinations thereof concerning embodiments described herein will appear to those skilled in the art following a reading of the instant disclosure.

DESCRIPTION OF THE FIGURES

[001 1] Embodiments will be described, by way of example only, with reference to the attached figures, wherein in the figures:

[0012] FIG. 1 illustrates, in a block schematic diagram, an example of a defect tracking interface platform and interface application environment, in accordance with some embodiments;

[0013] FIG. 2 illustrates, in a schematic diagram, an example of an incident tracking and analysis platform, in accordance with some embodiments;

[0014] FIG. 3 illustrates, in a flow diagram, an example of a method of tracking an incident, in accordance with some embodiments;

[0015] FIG. 4 illustrates, in a block schematic, a flow of generating data capsules, in accordance with some embodiments; [0016] FIG. 5 illustrates, in a block diagram, an example of a process of tracking incidents, in accordance with some embodiments;

[0017] FIG. 6 illustrates, in a flowchart, another example of a method of tracking incidents, in accordance with some embodiments;

[0018] FIG. 7 illustrates, in a block schematic, an example of a process to capture incident report data from a user computing device, in accordance with some embodiments;

[0019] FIG. 8 illustrates, in a block schematic, an example of a processes for capturing incident data from structured and unstructured external data sources, in accordance with some embodiments;

[0020] FIG. 9 illustrates, in a flowchart, another example of a method of tracking incidents, in accordance with some embodiments;

[0021 ] FIG. 10 illustrates, in a flowchart, another example of a method of tracking incidents, in accordance with some embodiments;

[0022] FIG. 11 illustrates, in a tree summation diagram, an example of classification, in accordance with some embodiments;

[0023] FIG. 12 illustrates, in a flowchart, an example of a method of user interaction with the platform, in accordance with some embodiments;

[0024] FIG. 13 depicts an example of a data structure for incident records, in accordance with some embodiments;

[0025] FIG. 14 illustrates a sample of an incident analysis report, in accordance with some embodiments;

[0026] FIG. 15 illustrates examples of incident reports, in accordance with some embodiments;

[0027] FIG. 16 illustrates, in a flow diagram, an example of a method for opening user accounts in an enterprise, environment, in accordance with some embodiments; and

[0028] FIG. 17 illustrates, in a flowchart, an example of an incident verification process, in accordance with some embodiments.

[0029] It is understood that throughout the description and figures, like features are identified by like reference numerals.

DETAILED DESCRIPTION

[0030] Embodiments of methods, systems, and apparatus are described through reference to the drawings. [0031] Described below is a method, system and computer program product for capturing, storing, consolidating and reporting information concerning incident reports. Particulars of incidents of glitches in consumer products, motor vehicles, software or virtually any product may be captured, stored, consolidated and reported. Consumers may directly report such incidents.

[0032] Incidents of glitches, defects, or other problems or issues pertaining to products, services or items may sometimes occur without a cause being determined. In some instances, not determining the cause may result in others experiencing the same problem. For problems resulting in serious injury or death, it is desirable for such issues to be detected as soon as possible. Moreover, a manufacture or service provider would benefit from knowing the problem as soon as possible to minimize the size or even need of potential recalls.

Terminology:

[0033] The following terms are used herein:

[0034] anchor data: one data item related to an incident, that may be the product involved or the occurrence time, date, location, or in some embodiments, other data items.

[0035] customer: the recipient of insights created by the methods described herein.

[0036] data capsule: an incident record that has been processed by a normalization method.

[0037] derived data: data derived from existing raw data using a mathematical, logical, or other type of transformation, such as arithmetic formula, composition, aggregation and stored locally for later use in analysis; for example deriving age from date of birth.

[0038] enrichment data: supplementary data from any source that is in some way related to an incident.

[0039] incident: an event in which a human is involved as a participant or observer. This may include a recall.

[0040] incident record: a collection of a unique identifier and anchor data, and may include enrichment data for a specific incident.

[0041] key word: a word or set of words that are characteristics of an incident and are used for discovery of incident records and enrichment data.

[0042] local enrichment data: data related to the incident that may be collected from devices involved in the incident or may be descriptive of an incident discovered in a non-user data source. [0043] parsing a pre-processing step in which data from external sources not organized by incidents is reorganized into incident records containing anchor data and local enrichment data if available from the same source.

[0044] remote enrichment data: data that may be collected from sources other than local enrichment data sources, using anchor data and, in some embodiments, key words or local enrichment data as lookup parameters.

[0045] user: a human involved in or observing an incident.

[0046] FIG. 1 illustrates, in a block schematic diagram, an example of a defect tracking interface platform and interface application environment 100, in accordance with some embodiments. FIG. 1 provides an overview of a high level architecture. Data about an incident may be captured from a computing device 102, run through an interface application 104, and sent through a network 108. Data about an incident may also come from an external incident data source 110, sent through the network 108. An incident may then be stored in an incident tracking and analysis platform 200. Supplementary data may be acquired from supplementary data sources 112 and internal data sources within 250 to enrich the incident record, and stored in the incident tracking and analysis platform 200 for further processing and reporting.

[0047] FIG. 2 illustrates, in a schematic diagram, an example of an incident tracking and analysis platform 200, in accordance with some embodiments. The platform 200 may be an electronic device connected to an interface application 104, supplementary data sources 112 and external incident sources 110 via a network 108 (or multiple networks). The platform 200 can implement aspects of the processes described herein for tracking and analysis of incidents.

[0048] The platform 200 may include at least one processor 202 on at least one server and at least one memory 204 on at least one server storing machine executable instructions to configure the at least one processor 202 (herein processor 202) to receive data (from e.g., data sources 104, 110, 112). The platform 200 may be implemented on an electronic device and can include an input/output (I/O) unit 206, a communication interface 208, and data storage device(s) 210. The processor 202 can execute instructions in memory 204 to implement aspects of processes described herein. The platform 200 may receive and transmit data from one or more of these via the I/O unit 206. When the data is received, the I/O unit 206 transmits the data to the processor 202. [0049] The I/O unit 206 can enable the platform 200 to interconnect with one or more input devices, such as a keyboard, mouse, camera, touch screen and a microphone, and/or with one or more output devices such as a display screen and a speaker.

[0050] The processor 202 can be, for example, any type of general-purpose

microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, or any combination thereof.

[0051] The data storage device(s) 210 can include memory 204, database(s) 250 and persistent storage. Memory 204 may include a suitable combination of any type of computer memory that is located either internally or externally such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically-erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like.

[0052] The communication interface 208 can enable the platform 200 to communicate with other components, to exchange data with other components, to access and connect to network resources, to serve applications, and perform other computing applications by connecting to a network (or multiple networks) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switch telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WiMAX, LTE), SS7 signalling network, fixed line, local area network, wide area network, and others, including any combination of these.

[0053] The platform 200 can be operable to register and authenticate users (using a login, unique identifier, and password for example) prior to providing access to applications, a local network, network resources, other networks and network security devices. The platform 200 can connect to different machines or entities. The platform 200 may serve one user or multiple users.

[0054] The data storage device(s) 210 may be configured to store information associated with, obtained by or created by the platform 200. Storage device(s) 210 and/or persistent storage may be provided using various types of storage technologies, such as solid state drives, hard disk drives, flash memory, and may be stored in various formats, such as relational databases, non-relational databases, flat files, spreadsheets, extended markup files, etc. [0055] The memory 204 may include a data collection module 212 for acquiring anchor data and other incident data from data sources 104, 110, 250, an enrichment data module 214 for acquiring enrichment data from supplementary data sources 112 based on the anchor data, a parsing module 216 for parsing acquired data, a normalization module 218 for normalizing acquired data, an analysis module 220 for analyzing the normalized data, and a reporting module 222 for reporting results from the analysis. The modules 212, 214, 216, 218, 220, 222 will be described in more detail below.

[0056] The data collection module 212 may acquire data through the network 108 from an interface application 104 of a computing device, and/or from unstructured and structured external data sources 110. Anchor data of an incident may be obtained and an incident record may be created with the anchor data. The data collection module 212 may comprise receiving functionality and aggregation functionality to receive and aggregate data from different sources. The interface application 104 and external data sources 110 may also include modules that collect data and send the data to the data collection module 212.

[0057] The enrich data module 214 may be used to obtain/request enrichment data from supplementary data sources 112 and data from internal data repositories 250 based on received anchor data. The parsing module 216 may parse received data to enable the normalization module 218 to standardize the data and populate an incident database with data capsules that include an incident number, the anchor data, enrichment data, and any additional details provided. The incident database may be included in the data repositories 250. It is understood that the data repositories 250 may also include one or more databases having information such as product records, recall records, user records, communication notifications, etc.

[0058] The analysis module 220 may analyze the data in the standardized incident database (e.g., data capsules) to determine one or more insights from report incident patterns. The report module 222 may report the one or more insights. In some embodiments, the report module 222 may generate a report, an electronic mail (email) or text message, or initiate a recall procedure.

[0059] FIG. 3 illustrates, in a flow diagram, an example of a method 300 of tracking an incident, in accordance with some embodiments. The method 300 may be performed by the platform 200 and comprises obtaining an incident report 310 where an incident report is initiated. Incident anchor data may be obtained 320 from the incident report. An example of an anchor data may be a data item that is related to an incident, and that may be a product involved, the incident occurrence time, date, or location, or other data items pertaining to the incident. Next, enrichment data associated with the incident report may be obtained 330 based on the anchor data. For example, local enrichment may be collected from device(s) involved in the incident, or may be descriptive of an incident discovered in a non-user data source and an incident record is created. In some embodiments, remote enrichment data, such as traffic conditions, weather conditions, sun spot activity, and other data, may be automatically obtained (based on the time, date and/or location anchor data) from internal 518 and supplementary data sources 112. This information enriches the original incident record.

[0060] At times, there may be additional anchor data within internal or external data sources 110 that is not captured as part of the original incident report. This additional anchor data may be collected and, optionally, a subsequent enrichment data search may be conducted based upon the newly acquired additional anchor data to further enrich the incident record. This loop can repeat until all current enrichment data based on all anchor data is acquired.

[0061] The incident record may then be normalized 340 using taxonomies to form an incident data capsule. These data capsules are analyzed and given a confidence score based on the number of enriched data points we were able to capture for that capsule as well as the original source and quality of the data. These incident data capsules can then be analyzed 350 to determine one or more insights which can be reported 360. Other steps may be added to the method 300.

[0062] FIG. 4 illustrates, in a block schematic, a flow 400 of generating data capsules, in accordance with some embodiments. An incident report may come from a source 410. One source may provide structured data 412 from a user device application 104 such as voice, text entry, application prompts, email, text messaging, etc. via a computing device such as a laptop, tablet, smartphone, etc., and may include one or more media and/or other types files pertaining to the incident. In some embodiments, the device may trigger the reporting of an incident (even without human intervention). Another source may provide other structured data 414 from methods that search external sources 110. For example, application programming interfaces (APIs) and other techniques may capture structured data such as datasets from regulatory bodies, incident data records from customers, and other structured data that may be available from partners and other sources. Another source may provide unstructured data 416 from methods that search external sources 110. For example, web scraping and other techniques may capture unstructured data from sources such as social media, online forums, or other public sources of information that provide unstructured data. The incident record may then be stored in one or more incident database(s) (raw data) 440 in the data repositories 250, normalized 340 and formatted based on one or more taxonomies (e.g., standardize the incident records) into data capsules, and stored in an incident database (data capsules) 445 which may be in a different or the same data repository 250, partitioned, or in one or more different databases.

[0063] FIG. 5 illustrates, in a block diagram, an example of a process 500 of tracking an incident, in accordance with some embodiments. An incident is reported 502 which includes anchor data that may be at least one of date, time, location, product, or other type of data directly associated with the incident. If the incident is reported from a computing device such as a smartphone, tablet, laptop, or other device, the information may include text or voice or image as well as local enrichment data 504. Local enrichment data may be data related to the incident that may be collected from devices involved in the incident. Additionally or alternatively, local enrichment data may be descriptive of an incident discovered in a nonuser data source such as an accelerometer, a barometer, and other sensors that may capture information. I.e., enrichment data may be pulled from an internal data source 518 and added to incident record. Enrichment data may also be pulled from supplementary data sources 112 and added to incident record.

[0064] In some embodiments, an enhanced method of local data enrichment involves capturing a time sequence of data related to the incident rather than static data at the moment the incident is reported. For example, an application in a user device or a data capture device may be configured to record all available sensor data continuously, deleting older data based on predetermined criteria. In some embodiments, such sensor data could be deleted every second for all data more than fifteen seconds old, so that the most recent fifteen seconds of data are always available for data capture following a report of an incident. It is understood that another amount of time other than fifteen seconds may be used.

[0065] In the event of an incident record being initiated by a user via an interface application 104 or a system process on an internal data source 518 or an external data source 110, data deletion may be discontinued, and data continues to be recorded until different predetermined criteria are met (e.g., typically closure of the incident report). At this point, any data not already attached to the incident record may be added, and the incident, including anchor data and other local enrichment data, along with the aggregation of time series data from prior to the incident through to closure of the incident, are communicated to the data capture method.

[0066] This method ensures that relevant data prior to the incident is captured, so that later analysis can examine conditions prior to the incident for determination of possible causes or contributing factors. Such details are lost if data capture starts only at the time of the incident report. The result is the conditions that the user or device may have reacted to are now available that were previously unknown and potentially unknowable.

[0067] The incident report 502 and local enrichment data 504 may be sent through the network 108 to an application server 508. The application server 508 may comprise elements of the platform 200. If data is received in a format that is compatible with the data analysis module 220, then parsing is not required 510 to create a single incident record. An incident number may be created and the data will be stored on the incident database 440. If parsing is required 510 to create a single incident record, the data will be stored on a data lake 512, parsed 514, and an incident number may be created and stored on the incident database 440. If the incident was reported by voice, that voice record may be translated to text and parsed, and both the voice record and text may be retained on the incident database 440 and is further processed.

[0068] Remote enrichment data to be associated with the reported incident may be gathered through the network 108 from other internal sources 518 or from supplementary data sources 112. This information may include traffic conditions, weather conditions, sun spot activities, and other information. Anchor 502, local enrichment 504, and remote enrichment data may then be stored in an incident database (raw data) 440 for further processing via the normalization process 340. This normalization process 340 structures the unstructured data according to a pre-determined taxonomy, creating a data capsule (e.g., standardizes or normalizes incident record data into data capsules) which is then stored in an incident database (data capsules) 445. These databases 440, 445 may be in the same partitioned database 250, or in one or more different databases. From the normalized incident data, analysis methods 350 may be applied, such as feature extraction, selection, classification and other processes to identify patterns and determine insights, which can be reported 360.

[0069] FIG. 6 illustrates, in a flowchart, another example of a method 600 of tracking an incident, in accordance with some embodiments. The method 600 may be performed by the application server and comprises parsing and storing 610 incident data, creating 620 an incident identifier index, seeking 630 further enrichment data from internal and external, normalizing 640 the data, analyzing 650 the data to find one or more insights, and reporting 660 the one or more insights. The parsed and stored 610 data may include anchor data and local enrichment data, and may be received via voice or text. The further enrichment data sought from internal and external sources may be based on the anchor data. Other steps may be added to the method 600. [0070] FIG. 7 illustrates, in a block schematic, an example of a process 700 for capturing incident report data from a user computing device, in accordance with some embodiments. An incident may be triggered by a user device or a device involved in the incident. The interface application 104 of the computing device captures 502 anchor data and in some embodiments may capture 504 local enrichment data. A unique incident record ID 702 and an incident record may be created 704 which are stored on an incident database 440.

Further incident details 706, may be reported 708 and stored on the incident database 440.

If no further incident details are reported 706, the incident may be flagged 710 as such and this is stored on the 440 incident database.

[0071] According to a further embodiment of the invention, a system for providing incident reports may include a computing device comprising a processor, a display and a memory. The computing device may be configured to receive a product identifier and information about an incident with the product associated with the product identifier via voice entry or touch entry. The computer device may be further configured to consolidate the information about the incident with information from a database and generate summary reports.

[0072] According to a further embodiment of the invention as shown in FIG. 8, a method 800 for capturing an incident may include a user using a mobile device 102 to report an incident, starting an incident reporting process by recording particulars of the incident in step 802, accessing an application server 508 through a network 108 to create an incident number, capture data sent from the mobile device such as date, time, location and IP address, and capture a record of the incident, storing the record of the incident including any voice, image, or text record in a data repository 250, and adding weather and traffic conditions to the record of the incident through the use of data, time, and location in step 806. If the incident report contains a voice record, the voice record is translated to text and also stored with the incident. The report may also indicate whether the user is the owner of the product, as users may report incidents involving products belonging to other parties.

[0073] FIG. 9 illustrates, in a block schematic, an example of processes 900 for capturing incident data from structured and unstructured external data sources, in accordance with some embodiments. Determining 902 which data sources are to be accessed can be performed in multiple ways. In one way, input may be received from an administrator of the system. In another way, input may be received by machine analysis. Both ways provide direction to which the method then uses 904 a key word or set of words to the methods sent via the network 108 to the unstructured data 416 and/or other structured data 414 sources selected. The methods 904 may acquire incident data including the anchor data and may attempt to acquire any local enrichment data that may be available. This information is then sent back through the network 108, filtered 908 based on key words and stored in the data lake 512 from which it is parsed 514 to create individual incident records that are stored on an incident database 440. Processing of this data continues as in FIG. 5. It should be understood that the above explains one example of a parsing flow. Other methods of parsing may be employed at this or different stages of the method for tracking, analyzing and reporting incidents.

[0074] FIG. 10 illustrates, in a flowchart, another example of a method 1000 of tracking incidents, in accordance with some embodiments. Following pre-processing of incident data as described above in FIG. 5, the method 1000 normalizes 340 and standardizes the incident data. As noted above, the following sets of data may be collected: Anchor Data, Local Enrichment Data, Remote Enrichment Data and/or Incident Data, including a unique record identifier. All text data may be standardized to facilitate analysis. This may be accomplished by mapping data to an already existing taxonomy. Taxonomies exist for incidents, and in some embodiments, for products and recalls.

[0075] Similarly, the numerical data collected also may be standardized. In some embodiments, a standard mathematical concept called Z-Score Normalization (also known as Standard Scaling) may be used. This process may be defined as the measure of how many standard deviations a raw value is below or above the population mean. Note that Standard Scaling does not make the distribution normal and, in some cases, there may be additional transformations to make these data sets normal. This can be done using Box-Cox power transformations. A Box-Cox power transformation is a way to transform non-normal dependent variables into a normal shape.

[0076] The normalization service creates an updated incident record called a data capsule that is stored and forms the basis for analysis.

[0077] Feature extraction and engineering 1002 aim to create new features from collected data. Two types of data are processed when handling data capsules: textual data from its descriptions, and numerical and/or categorical data via enrichment data sources

(temperature, weather, location, etc.). Before feature engineering begins, the textual data may be pre-processed to remove issues that can confuse the model. These include:

Stripping all HTML tags

Removing accented characters

Expanding contractions

Lowercasing all text Removing extra new lines

Removing special characters

Stemming or Lemmatizing text

Removing extra white space

Removing stopwords

Correcting spelling

Correcting grammatical

Removing repeated characters

Tokenization

At the end of this pre-processing stage, an array of clean tokenized textual data for each data capsule is available, alongside its associated numerical and categorical data from enrichment data sources.

[0078] The next step involves vectorization using term frequency-inverse document frequency (TF-IDF). For some embodiments TF-IDF is used as it is a well-known method to evaluate how important a word is in a document. Essentially, this method converts textual data to a numerical representation, specifically, a Vector Space Model (VSM). The VSM may comprise an array of numbers based on word frequency and importance that the model can use when trying to evaluate which keywords and/or phrases (n-grams) trigger a product recall.

[0079] The potential features that may be extracted from the data in data capsules may comprise one or more of the following examples:

Year of Submission: Value of year when the incident record was submitted;

Month of Submission: Month number (e.g., values between 1 to 12);

Week Day of Submission: Week Day value (e.g., values between 1 to 7);

Hour of Submission: Value of time hour (e.g., values between 0 to 23);

Year Day of Submission: Year Day (e.g., values between 1 to 365);

Month Day of Submission: Month Day (e.g., values between 1 to 31);

Country of Submission: Associated country;

State of Submission: Associated state/province;

City of Submission: Associated city; Code of Submission: Associated zip or postal code;

Character Body Length: Total number of characters in the body text;

Character Title Length: Total number of characters in the title text;

Word Body Length: Total number of words in the body text;

Word Title Length: Total number of words in the title text;

Noun Count: Total number of nouns in incident record;

Verb Count: Total number of verbs;

Adjective Count: Total number of adjectives;

Adverb Count: Total number of adverbs;

Pronoun Count: Total number of pronouns;

Punctuation Count: Total number of punctuation marks;

Stopword Count: Total number of stopwords;

Uppercase Count: Total number of uppercase characters;

Word N-Gram TF-IDF Body Text: Word importance in body text;

Word N-Gram TF-IDF Title Text: Word importance in title text;

Temperature: Temperature at location;

Humidity: Moisture content at location; and/or

Weather: Weather at location.

[0080] Feature selection 1004 may be used to identify the features or variables that affect a trend, insight or recall by a certain threshold. Not all features extracted need be used, as the goal is to reduce model complexity but increase model accuracy. In some embodiments, a combination of features may be selected based on how they rank with the following feature selection methods.

[0081] Feature selection 1004 here may involve using the Filter Method. Such algorithms may be used as they are a group of simple, standard statistical algorithms that are based on correlations with the outcome variable. The following set of statistical algorithms allows for a reduction in the number of features in training the model:

Pearson’s Correlation: Used as a measure for quantifying linear dependence between two continuous variables X and Y. Linear discriminant analysis (LDA): Finds a linear combination of features that characterizes or separates two or more classes of a categorical variable.

Analysis of Variance (ANOVA): Similar to LDA but operates using one or more categorical independent features and one continuous dependent feature. It provides a statistical test of whether the means of several groups are equal or not.

Chi-Square: Applied to the groups of categorical features to evaluate the likelihood of correlation or association between them using their frequency distribution.

Features may be selected on the basis of their scores in these aforementioned statistical tests.

[0082] Machine-learning methods can be used to create 1006 several models that may allow us to examine trends, insights and predictions. In some embodiments, a supervised machine-learning method, such as the Random Forest classifier, may be used to predict a product recall. The Random Forest classifier is sufficiently flexible, and based on an ensemble or collection of decision trees that is able to achieve better predictive performance by using an average of all the outputs.

[0083] FIG. 11 illustrates, in a tree summation diagram, an example of classification 1100, in accordance with some embodiments. As illustrated in FIG. 11 , each decision tree X or Y may be modeled in such a way that each node (A...G) represents a feature during classification. In other words, each decision tree captures a mode of asking questions until it produces an output indicating a valid correlation. The model may then find the average answer by taking into account all the decision trees used.

[0084] A supervised machine-learning method learns to map input data to an output in what is called the training phase of model development. During training, the model is provided with historical input data relevant to the problem domain. I.e., a combination of features (keywords, environment), and labels (the true target value that the model needs to learn in order to predict recalls). The model then learns the relationships between the features and labels. The decision tree forms the structure shown above, formulating the most appropriate questions to ask in order to make the most accurate estimates possible.

[0085] In other embodiments, unsupervised machine-learning methods may be used where there is an absence of training data. Instead, this model aims to uncover underlying structures and distributions in the data including clusters where there are inherent groupings in that data, or associations where there are correlations between data variables. For example, the k-means clustering method may be used to segment incidents into different classes of severity. Conversely, the a priori association method may be used to illustrate patterns such that whenever an incident occurs in certain locations, other incidents of a different nature also occur there.

[0086] Once models have been tuned via its hyper-parameters and cross-validated, they are deployed to a production environment to operate on real data.

[0087] Upon deployment of several models, an aggregation and analysis 1008 of the results of each model may be performed. Certain models may predict outcomes based on classification and regression. Other models will support the ability to identify trends and insights. In either case, there may be a manual analysis and evaluation of the results before issuing a report.

[0088] Finally, these results may be combined to create a trends, insights and predictions (TIP) report 1010 that explains the data, methodology used and any visualizations that may be helpful in understanding the results.

[0089] A system could be implemented such that it has a user database that stores all of the people who use the system. This would allow for the issuance of recall notifications (beneficial to broadcast) to the relevant people to notify of an issue.

[0090] FIG. 12 illustrates, in a flowchart, an example of a method of user interaction with the platform 200, in accordance with some embodiments. A user may register 1202 with the system and may provide one or more details such as email, name, address, telephone number and other information as part of the registration process. The user record may be added 1204 to the database and, if available, products and devices associated with the computing device that the user used to register may be downloaded 1206 to the user record, and presented 1208 to the user for confirmation. If the user confirms the product 1210, it is added 1212 to the user’s product record. If the user does not confirm the product 1210, it is not added to the user’s product record and the user may take further action 1214 such as search 1216 the database for incident records associated with a specific product that may or may not be registered on the user account or may end the interaction. If the user searched 1216 the database, the system displays 1218 the associated incidents. The user may take further action 1220 such as reporting 700 an incident on a product registered in the user account, or may end the interaction.

Airbag Incidents

[0091] The following use case refers to the methods described above and outlines an example of an application pertaining to a detection of a defect in airbags that causes unexpected airbag deployments, the detection resulting in the avoidance of multiple deaths and injuries. [0092] An incident 502 commences with the unexpected deployment of a vehicle airbag. The user, in this case of the vehicle and the system, reports the facts of the incident and provides details as prompted by a user device involved in the incident. In some

embodiments, the user device can detect that there has been an airbag deployment and reports the incident. The anchor data collected may be one or both of a timestamp and the product involved, in this case a vehicle; in some embodiments, the facts and details of this vehicle may have been pre-registered via a 1210 registration process. In some

embodiments, 504 local enrichment data may be packaged up and include the GPS location, as well as accelerometer, compass and gyroscope readings. Once this data has been transmitted through the network 108 to the enrichment service server 508, 330 remote enrichment data may be retrieved from supplementary data sources 112 via network 108 and directly from databases 518 to reverse lookup the actual location, speed of local traffic, direction, orientation of travel and other existing data points. For example, if the incident occurred in Miami, Florida travelling north at a speed of 40km/h. Additionally, at this location, it is discovered that the weather at that time was hot and sunny with a high moisture content given its proximity to the coast. An incident record including these data points is recorded in the incident database 440 and then processed through the normalization process 340, creating a data capsule that is stored in the incident database 445.

[0093] The analysis process 350 seeks the existence of multiple data capsules filed concerning unanticipated airbag deployments. FIG. 13 depicts an example of a data structure 1300 for incident records, in accordance with some embodiments. Following feature extraction 1002, feature selection 1004 would have identified that there is a positive correlation between these incidents and the location, temperature and humidity. The model 1006 is then able to use these highly-correlated features during training so that when deployed, accurate predictions can be made in the reporting process 360 specifically taking this selected subset of features into account to create a report 1010, a sample of which can be seen in FIG. 14.

[0094] Proceeding this way would have uncovered that incidents reported in hot and humid conditions contributed materially to the unexpected and injurious deployment of airbags in significantly shorter time and with higher certainty of contributing factors compared to former and current methods that resulted in avoidable deaths and injuries.

[0095] After completing the analysis and producing a report, a pattern can be seen whereby the majority of unexpected airbag deployments occurred in parts of the United States that are warm and humid. By capturing and retaining specific information in the data capsules stored in the database 445, it can be seen in the report map 1400 of airbag incidents 1410 (in FIG. 14) that incidents A and B, occurring in Oregon and Michigan, respectively, had weather patterns that were also warm and humid when the airbag incidents occurred.

Unexpected Braking Incidents

[0096] The following use case refers to the methods described above and outlines an example of an application pertaining to the detection of a faulty braking system causing unexpected near collisions, the detection of which results in the prevention of potential deaths and injuries.

[0097] An incident 502 commences with the unexpected deployment of a vehicle’s autonomous or semi-autonomous brakes. The user, in this case of the vehicle and the system, reports the fact of the incident and provides details as prompted by a user device involved in the incident. In some embodiments, the user device can detect that there has been brake deployment and reports the incident.

[0098] With near simultaneity, the driver of a following vehicle is forced to brake or braking may occur autonomously. In the event that this driver is also a user of the system, an incident may be reported and anchor data collected. In some embodiments, the fact and details of these vehicles may have been pre-registered. In some embodiments and for both users, local enrichment data 504 may be packaged up and include the GPS location, as well as accelerometer, compass and gyroscope readings. Once this data has been transmitted through the network 108 to the server 508 so that remote enrichment data 330 may be retrieved from supplementary data sources 112 via network 108, and directly from internal sources 518 to reverse lookup the actual location, speed of local traffic, direction, orientation of travel, vehicle behaviour data as recorded by either or both vehicles and other existing data points. For example, the incident occurred on a country road with no traffic and under good driving conditions, allowing both drivers to see and subsequently record that there was no obstacle for which braking was required. One or two incident records including all these data points are recorded in the incident database 440. After being processed through the normalization process 340, in this example one or two data capsules are then stored in the incident database 445.

[0099] The analysis process 350 seeks the existence of multiple data capsules filed concerning unexpected braking. An example of a data structure 1300 including normalized incident records can be found in FIG. 13. Following feature extraction 1002, feature selection 1004 would have identified that there is a positive correlation between this pair of incidents, the local driving conditions and vehicle behaviour data that would report that both vehicles operated as designed. The model 1006 is then able to use these highly-correlated features during training so that when deployed, accurate predictions can be made specifically taking this selected subset of features into account using the reporting process 360 to create a report 1010.

[0100] Proceeding this way can uncover that flaws exist in the lead vehicle’s braking algorithm or sensors that the vehicle manufacturers would interpret as normal and required behaviour by the autonomous controls in the lead vehicle. A failure to recognize such flaws by a reliance on sensor data only can result in collisions and avoidable deaths and injuries.

Ignition Switch Incidents

[0101] FIG. 15 illustrates examples of incident reports, in accordance with some embodiments. FIG. 15 shows how the embodiment addresses a real-life situation where an ignition switch incident occurred in thousands of cars for over 10 years. Hundreds of people were seriously injured and at least 125 people died. FIG. 15 shows actual incidents that a young woman reported experiencing faulty ignition switch issues.

[0102] This young woman reported her car’s faulty ignition switch more than once to her dealership but they turned her away, saying there was no problem. Although she did keep track of this in a simple notebook, limited anchor data and no enrichment data were captured, and she was unable to do a quick and accurate search to easily understand if other people were experiencing the same problems as she had experienced.

[0103] Despite the fact that this young woman had done her best to have the dealership address the ignition switch incidents she had been experiencing, while driving to her parents’ house on her 29 ^th birthday, her faulty ignition switch turned the engine off, she lost control of the vehicle, her car crashed and she died.

[0104] It is not uncommon for a user to experience multiple incidents associated with a product. When an incident occurs 1502, in this case the vehicle turning itself off while it is being driven, the user reports the incident 1504 and details about that incident are saved and displayed 1506 for the product. It should be noted that the user does not need to know why it happened, just that it happened.

[0105] Over time, there may other incidents 1508, 1510 and, again they are reported 1504, saved and the newer incident details are displayed 1508, 1510 with the first incident

1506.

[0106] At any point in time, the user may search the incident database 1515 to see if other people are experiencing the same issues with that product. The user would enter search criteria on the global glitch database screen 1518 and any incidents that meet the criteria entered are displayed 1520. Product Recall Notification

[0107] The benefit of this use case is that a user only has to register their products once, in one place, and one of the embodiments will ensure that they are notified of any product recalls in a timely and efficient manner.

[0108] The following use case refers to the methods described above and outlines an example of an application for product recalls, resulting in the prevention of potential deaths and injuries. A new user registers 1202 with the system containing these methods and in part of the application registers 1206 products owned or used by the user. Asynchronously, a process uses the information from a product just registered as a search parameter to query any or all of external sources 110, supplementary data sources 112, and internal databases 518 to discover the existence of a recall notice for the registered product. In the event of a match to a notice, a notification is created and displayed 1218 to the user in a search window. In some embodiments, if the user is not active on the system, the notification is routed to the user via contact information recorded 1204 in the registration process, using the reporting process 360 to format and distribute the report 1010 content.

[0109] After registration is complete and all products recorded, a process adds this list of products to a database monitored by a continuously running process. This process seeks new recall notices from external sources 110 and supplementary data sources 112 and matches them to products registered by users. In some embodiments, methods may determine that similar products may be subject to the recall and make a high confidence match. In the event of any match, a notification is created and routed to the user via contact information recorded in the registration process 1204, using the reporting process 360 to format and distribute the report content 1010.

[01 10] Proceeding this way, users can be quickly notified of recalls that are specific to the list of products they have registered and have interest in. This may lead to avoidance of deaths and injuries from defective products.

Food Recall Alert

[01 1 1 ] Food recalls are generally triggered for a specific food or group of foods in a specific region. In one embodiment, a public service can be executed by identifying all of the users in the region to which the food recall applies and then create a notification and route the notification to the user via contact information recorded in the registration process 1204, using the reporting process 360 to format and distribute the report content 1010. Health Incident

[01 12] The following use case refers to the methods described above and outlines an example of an application pertaining to the processing and reporting of an incident involving a user’s heath, the prompt and accurate outcome potentially resulting in the avoidance of multiple deaths and injuries.

[01 13] An incident 502 commences with a user experiencing a symptom representing a variance from their normal state of health. The user, in this case a consumer of some mix of pharmaceutical and naturaceutical medication taken consistently over time or on an as needed basis, reports the facts of the incident and provides details as prompted by a user device involved in the incident. In some embodiments, a user device can detect that there has been a state of health variance and reports the incident. The anchor data collected may be one or both of a timestamp and a recent medication consumed; in some embodiments, the facts and details of this and other consumed medication may have been pre-registered via a registration process 1210. In some embodiments, local enrichment data 504 may be packaged up and include the GPS location, as well as measurements of health indicators or changes therein, such as temperature, blood sugar and heart rate. Once this data has been transmitted through the network 108 to the enrichment service server 508, remote enrichment data 330 may be retrieved from external sources 110 via network 108 and directly from databases 518 to reverse lookup details of registered medications, their known reactions and related symptoms, and other existing data points, such as weather conditions at the location. For example, the incident may have occurred in Miami, Florida outside the user’s registered address immediately after reported consumption of a specific medication. Additionally, at this location, it is discovered that the weather at that time was hot, humid and sunny. An incident record including these data points is recorded in the incident database 440 and then processed through the normalization process 340, creating a data capsule that is stored in the incident database 445.

[01 14] The analysis process 350 seeks the existence of multiple data capsules filed concerning health variances following consumption of the specific medication reported during the data capture process 412. Following feature extraction 1002, feature selection 1004 would have identified that there is a positive correlation between these incidents and the temperature, humidity and specific medication, in combination with a subset of other medications registered by the user and that user’s registered personal characteristics, such as sex. The model 1006 is then able to use these highly-correlated features during training so that when deployed, accurate predictions can be made in the reporting 360 process specifically taking this selected subset of features into account to create a report 1010. [01 15] Proceeding this way would have uncovered that incidents reported in hot and humid conditions contributed materially to the unexpected and injurious health variances due to consumption of specific medication combinations for specific subsets of users, in significantly shorter time and with higher certainty of contributing factors compared to former and current methods that could have resulted in avoidable deaths and injuries. It would also have noted any patterns of adverse health effect for a specific combination of

pharmaceuticals and naturaceuticals, or when a person has added a new ingestible health product to the mix of pills already being taken.

Multiple User Types

[01 16] A user may be an individual or a group of individuals such as a family, a caregiver on behalf of one or more people to whom they are attending, or an enterprise that wishes to keep track of incidents associated with the products they use. The basic functions of reporting and tracking incidents remain the same; the difference is in the sign on, authentication, and account closing processes which are fairly standard across most applications.

Class Action Participants

[01 17] Because the database(s) will contain information about people who have experienced incidents with the products they use, an embodiment includes the ability to identify and communicate with individuals or enterprises that may wish to participate in a class action law suit. The benefit of this service includes the ability to keep the participants reasonably informed as to the status of any class action law suit.

[01 18] The benefit of this use case is that a user may easily be contacted and kept informed about the possibility of participating as in a legal matter relating to a product issue as well as being informed of any other information relating to a class action or personal injury lawsuit(s).

[01 19] The methods of the embodiment include the ability to identify a user based on any incidents reported with a specific product, similar to the Product Recall Notification process, as well as the ability to issue a notification to a wide group of people based on any number of variables including the region in which they live, similar to the Food Recall Alert process.

Enterprise Users

[0120] FIG. 16 illustrates, in a flow diagram, an example of a method 1600 for opening user accounts in an enterprise environment, in accordance with some embodiments. In this example, the user begins 1601 by selecting“Enterprise” as the User Type. The user enters 1602 company or organization details such as name of business/organization, type of business/organization, address, etc., followed by 1603 the business/organization’s email domain name. The user then selects 1604 number of users that may enter incidents on behalf of business/organization. According to the number of users selected, a monthly cost is displayed 1605. If the user does not accept the monthly cost 1607, the user may enter a different number of users and a revised monthly cost is displayed until the user accepts the monthly cost or abandons the sign up process. Once the user accepts the monthly cost, then they may enter 1608 a payment method.

[0121] The user then enters 1609 the names and email IDs of Administrator and Alternate Administrator. Note that all user email IDs includes the same email domain name as the business / organization as entered at 1603. The user may also enter 1610 the names and email IDs of each authorized user. Again, all user email IDs include the same email domain name as the business / organization as entered at 1603.

[0122] If the user has a product file to upload 1611 , then upload 1612 the product file. If the user does not have a product file and wishes to enter product information 1619, then allow user to enter product information 1620. If the user does not wish to enter product information, then send confirmation email 1617 to users and end the process 1621.

[0123] Once the product file is uploaded 1611 , and/or the product information is entered 1620, then the product information is displayed 1613 for acceptance 1614 by the user. If the products are accepted 1615, then the products are registered to the account. If not 1618, then the user may accept or reject each product individually. The user may be asked 1616 if there are additional products to be entered and may enter the information 1620. Once complete (no more products to enter), then send confirmation emails 1617 to each user and end process 1621.

[0124] FIG. 17 illustrates, in a flowchart, an example of an incident verification process, in accordance with some embodiments. FIG. 17 illustrates the incident verification process that may be used. Having a user verify an incident may help in the building of a confidence score. In some embodiments, an incident verification process is initiated on a user. Once an incident has been created using an application.

[0125] The discussion provides example embodiments of the inventive subject matter. Although each embodiment represents a single combination of inventive elements, the inventive subject matter is considered to include all possible combinations of the disclosed elements. Thus, if one embodiment comprises elements A, B, and C, and a second embodiment comprises elements B and D, then the inventive subject matter is also considered to include other remaining combinations of A, B, C, or D, even if not explicitly disclosed. [0126] The embodiments of the devices, systems and methods described herein may be implemented in a combination of both hardware and software. These embodiments may be implemented on programmable computers, each computer including at least one processor, a data storage system (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication interface.

[0127] Program code is applied to input data to perform the functions described herein and to generate output information. The output information is applied to one or more output devices. In some embodiments, the communication interface may be a network

communication interface. In embodiments in which elements may be combined, the communication interface may be a software communication interface, such as those for inter-process communication. In still other embodiments, there may be a combination of communication interfaces implemented as hardware, software, and combination thereof.

[0128] Throughout the foregoing discussion, numerous references will be made regarding servers, services, interfaces, portals, platforms, or other systems formed from computing devices. It should be appreciated that the use of such terms is deemed to represent one or more computing devices having at least one processor configured to execute software instructions stored on a computer readable tangible, non-transitory medium. For example, a server can include one or more computers operating as a web server, database server, or other type of computer server in a manner to fulfill described roles, responsibilities, or functions.

[0129] The technical solution of embodiments may be in the form of a software product. The software product may be stored in a non-volatile or non-transitory storage medium, which can be a compact disk read-only memory (CD-ROM), a USB flash disk, or a removable hard disk. The software product includes a number of instructions that enable a computer device (personal computer, server, or network device) to execute the methods provided by the embodiments.

[0130] The embodiments described herein are implemented by physical computer hardware, including computing devices, servers, receivers, transmitters, processors, memory, displays, and networks. The embodiments described herein provide useful physical machines and particularly configured computer hardware arrangements.

[0131] Although the embodiments have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein. [0132] Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification.

[0133] As can be understood, the examples described above and illustrated are intended to be exemplary only.