COMPUTER-IMPLEMENTED METHODS. COMPUTER-READABLE MEDIA. AND SYSTEMS FOR IDENTIFYING CAUSES OF LOSS

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 62/631,579, filed February 16, 2018. The entire content of this application is hereby incorporated by reference herein.

FIELD OF THE INVENTION

The present invention relates generally to computer-implemented methods, computer- readable media, and systems for identifying causes of loss from unstructured or semi-structured insurance claims data.

BACKGROUND OF THE INVENTION

Insurers and re-insurers seek a better understanding of accidents in order to lower the incidence of accidents. At least in the auto insurance field, most claims include a text description generated by a human ( e.g ., a claims representative, a policyholder, a police officer, and the like). These descriptions pose an obstacle to understanding the underlying causes of accidents across the large (e.g., thousands of claims per year) data sets generated by insurance companies.

SUMMARY OF THE INVENTION

One aspect of the invention provides a computer-implemented method of identifying causes of loss from insurance claims data including a plurality of unstructured or semi-structured insurance claims. The computer-implemented method includes: loading insurance claims data including a plurality of unstructured or semi -structured insurance claims into memory on a computer; for each of at least a subset of the insurance claims within the insurance claims data, creating a corresponding pre-processed claim record by: tokenizing the insurance claims loaded into memory to separate words in the insurance claims loaded into memory from punctuation, lemmatizing the words in the insurance claims loaded into memory to map morphological variations onto a common base word, removing stop words from the insurance claims loaded into memory, removing punctuation and numbers from the insurance claims loaded into memory, and replacing abbreviations and common typographical errors with associated words previously- defined in a data dictionary stored in memory; creating a Term Frequency - Inverse Document Frequency (TF-IDF) matrix in memory detailing relative frequency of a plurality of «-word terms within at least a subject of the pre-processed claim record, wherein n is a positive integer; selecting a plurality of features from the TF-IDF matrix; and creating a binary classifier for each of a plurality of causes of loss.

This aspect of the invention can include a variety of embodiments. For example, n can be an integer between 1 and 6. The selecting step can include applying a chi-squared test for each of the plurality of features within the TF-IDF matrix.

The computer-implemented method can further include applying the binary classifiers against a plurality of the pre-processed claim records not previously used in creating the binary classifiers to identify one or more causes of loss. The computer-implemented method can further include, if one or more of the insurance claims was not classified by any of the binary classifiers, designating the insurance claim as unclassifiable. The computer-implemented method can further include selecting a single cause of loss from the one or more causes of loss identified by the binary classifiers based upon a pre-defmed hierarchy. The binary classifiers can be applied in a previously specified priority order. The binary classifiers can be applied to identify a single cause of loss.

The computer-implemented method can further include identifying whether a plurality of the insurance claims was preventable, non-preventable, or partially preventable based on previously stored associations between causes of loss and prevention techniques. The computer- implemented method can further include identifying one or more prevention techniques for a plurality of the insurance claims identified as preventable or partially preventable based on previously stored associations between causes of loss and prevention techniques. The computer- implemented method can further include identifying which of the one or more prevention techniques are associated with a highest aggregate or average loss over the plurality of preventable or partially preventable insurance claims.

The computer-implemented method can further include discarding terms having a total frequency over the plurality of insurance claims of less than or equal to 2 before either creating the TF-IDF matrix or selecting a plurality of features from the TF-IDF matrix.

Another aspect of the invention provides a computer-implemented method of identifying causes of loss from insurance claims data comprising a plurality of unstructured or semi- structured insurance claims. The computer-implemented method includes: loading insurance claims data comprising a plurality of unstructured or semi-structured insurance claims into memory on a computer; for each of at least a subset of the insurance claims within the insurance claims data, creating a corresponding pre-processed claim record by: tokenizing the insurance claims loaded into memory to separate words in the insurance claims loaded into memory from punctuation, lemmatizing the words in the insurance claims loaded into memory to map morphological variations onto a common base word, removing stop words from the insurance claims loaded into memory, removing punctuation and numbers from the insurance claims loaded into memory, and replacing abbreviations and common typographical errors with associated words previously-defined in a data dictionary stored in memory; applying the binary classifiers created using the methods described herein against a plurality of the pre-processed claim records to identify one or more causes of loss for each of the insurance claims within the insurance claims data.

This aspect of the invention can have a variety of embodiments. The computer- implemented method can further include, if one or more of the insurance claims was not classified by any of the binary classifiers, designating the insurance claim as unclassifiable. The computer-implemented method can further include selecting a single cause of loss from the one or more causes of loss identified by the binary classifiers based upon a pre-defmed hierarchy.

Another aspect of the invention provides a system for identifying causes of loss from insurance claims data comprising a plurality of unstructured or semi -structured insurance claims data. The system includes: a processor; and computer-readable memory containing instructions to: implement an interface programmed to receive insurance claims data comprising one or more insurance claims; store the insurance claims data in the computer-readable memory; and invoke execution of the method as described herein on the processor. BRIEF DESCRIPTION OF THE DRAWINGS

For a fuller understanding of the nature and desired objects of the present invention, reference is made to the following detailed description taken in conjunction with the

accompanying drawing figures wherein like reference characters denote corresponding parts throughout the several views. FIG. 1 A depicts a method for building a plurality of binary classifiers to identify causes of loss for textual data according to an embodiment of the invention.

FIG. 1B depicts a method for identifying causes of loss for textual data according to an embodiment of the invention.

FIG. 2 depicts a computer system through which provisioned textual data can be interfaced, stored, and processed to identify causes of loss and/or mitigation techniques according to an embodiment of the invention.

FIG. 3 depicts a system for identifying causes of loss and/or accompanying mitigation techniques for textual data according to an embodiment of the invention.

FIG. 4 depicts a method of generating a Term Frequency - Inverse Document Frequency (TF-IDF) matrix according to an embodiment of the invention.

DEFINITIONS

The instant invention is most clearly understood with reference to the following definitions.

As used herein, the singular form“a,”“an,” and“the” include plural references unless the context clearly dictates otherwise.

As used in the specification and claims, the terms“comprises,”“comprising,”

“containing,”“having,” and the like can have the meaning ascribed to them in U.S. patent law and can mean“includes,”“including,” and the like.

Unless specifically stated or obvious from context, the term“or,” as used herein, is understood to be inclusive.

As used herein,“insurance claims data” represents the data for which a loss analysis is requested regardless of whether or not it was received from insurance companies, their system processors, their agents, insurance clients and/or their brokers, individual fleets or loss mitigation product vendors.

Ranges provided herein are understood to be shorthand for all of the values within the range. For example, a range of 1 to 50 is understood to include any number, combination of numbers, or sub-range from the group consisting 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,

16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, or 50 (as well as fractions thereof unless the context clearly dictates otherwise).

As used herein, the term“unstructured data” includes information that either does not have a pre-defmed data model or whose textual components are not organized in a pre-defmed manner. As used herein, the term“semi-structured data” includes unstructured data within a data model or other pre-defmed manner. For example, insurance claims data may be in a data model in which certain fields ( e.g ., date of accident, driver, automobile make and/or model, rental status) are well-defined, but other data (e.g., accident description) accept free-form text.

DETAILED DESCRIPTION

Embodiments of the invention provide computer-implemented methods, computer- readable media, and systems for identifying causes of loss from textual data. Although described in the context of automotive insurance claims, it could be applied to other insurance lines, e.g, casualty insurance, health insurance, disability insurance, life insurance, property insurance, aviation insurance, boiler insurance, builder’s risk insurance, crop insurance, home insurance, landlord insurance, marine insurance, liability insurance, and the like.

Although textual data poses processing challenges, it provides a valuable narrative to human readers that process an insurance claim and captures information that could not be efficiently captured in structured fields that must be programmed in advance and become tedious if too numerous.

Further complicating claims classification across various entities is the nonstandard codification process and data capture methodologies employed by each collecting company. The ability to bypass these differences by analyzing the original sources of the data provides a unique opportunity to gain insight from hundreds of thousands to millions of analyzed records, rather than the just the thousands produced by one company.

However, if one could discover valuable insight from this textual information, a company could have a better understanding of why the accidents occurred, whether they were preventable, and how. The invention described enables this deeper and automated insight.

Embodiments of the invention are particularly useful for re-insurers that may seek to better understand their underwriting risks and associated loss mitigation potential and/or share this data with their insurance company clients. In particular, even if each individual insurance company client used internally consistent styles for describing accidents— a highly unlikely condition— it is also highly unlikely that such styles would be identical across insurance companies or the agencies they contract with to process their losses. For example,

inconsistencies can arise from at least three sources. First, different people will often use different language for a similar story. Such variations can exist on an individual, generational, and/or regional level. Additionally, the report authors may have access to differing amounts of information and may perceive different information to be relevant or irrelevant. Moreover, some companies have their unique loss codes or cause-of-loss buckets.

Product manufacturers or individual fleets ( e.g ., rental car fleets, government vehicles, and the like) can process their loss descriptions outside of the insurance context for similar analysis to determine what type of loss mitigation method might be the most economically feasible for their unique loss situation. While the accident/incident description provided may not be in the context of insurance, the invention’s analysis process would be applicable.

The incident/accident data can be unstructured or semi-structured. For example, the insurance claims data can include human-generated descriptions of claims, examples of which are provided in Table 1 below. Although embodiments of the invention are particularly useful in processing unstructured data, structured data associated with an insurance claim can also be utilized. For example, structured data fields detailing the time of accident, age of driver, vehicle type, vehicle usage, and the like can also be utilized to identify features for use in building binary classifiers.

Referring now to FIGS. 1 A and 1B, one embodiment of the invention provides a computer-implemented method for identifying causes of loss from insurance data. Although the method described in FIGS. 1 A and 1B can be practiced in a single instance, it need not and, in many instances, will not be practiced in a single instance. Instead, the method of FIG. 1 A will often be performed once using a training data set in which a cause of loss was previously specified ( e.g ., by humans) to build classifiers before the method of FIG. 1B is applied to client- provisioned data. The training method of FIG. 1 A can be repeated for each data set (e.g., each time an insurance client provides data for analysis), periodically (e.g, every week, month, year, and the like), upon introduction of new causes of loss (e.g, for accidents deemed unclassifiable), and the like.

In step S102, insurance claims data is loaded into memory. The insurance data can be transferred from various media including storage disks using various technologies. For example, the insurance claims data can be submitted over the internet in a variety of file types.

In step S104, the insurance claims can be preprocessed before it is mapped onto a meaningful computation representation, such as a vectorial representation, used in later steps of the methods. Preprocessing maps claims onto a list of tokens that have linguistic meaning, e.g., «-word terms, that can then be classified in further steps. Preprocessing can include one or more of the following steps: tokenization (Sl04a), lemmatization (Sl04b), stop word removal (Sl04c), punctuation and number removal (Sl04d), and replacement of abbreviations and common typographical errors (colloquially,“typos”) (Sl04e).

Tokenization (Sl04a) separates punctuation from words.

Referring to step Sl04b, lemmatization maps each morphological variation of a word to its base form. For example, the words,“go”,“going”,“went”, and“gone” are lemmatized to their root or base form“go”. Lemmatizers include the WORDNET® system, available from Princeton University of Princeton, New Jersey. Other lemmatizers can be used, including the MORPHA™ software described in G. Minnen et ak, Applied morphological processing of English, 7(3) Natural Language Engineering 207-23 (2001) and available at

http://www.informatics.susx.ac.uk/ research/groups/nlp/carroll/morph.html.

Referring to step Sl04c,“stop words” (also called“stopwords” or“noise words”) are common words that appear in too many documents and, therefore, do not have discriminative power. That is, stop words cannot be used to capture the essence of a document such that one can differentiate one document from another. Standard lists of stop words are provided in software programs such as the SMART Information Retrieval System, available from Cornell University of Ithaca, N.Y. The SMART stop word list is available at

ftp://ftp.cs.comell.edu/pub/smart/english.stop. A collection of stop words for a particular set of documents can be manually selected using domain knowledge or created by selecting the words that occur in more than 80% of the documents. See R. Baeza-Yates & B. Ribeiro-Neto, Modern Information Retrieval §7.2 (1999). For example,“PR” (an abbreviation for“police report”) may be a stop word in an automotive accident data set.

Removal of numbers in step Sl04d can reduce overfitting in later steps caused by inherent ambiguity in numbers. (For example, the number“75” could represent a vehicle speed, a compass bearing, a street address number, a route number, a vehicle make year, and the like.)

Step Sl04e can use a custom dictionary reflecting domain knowledge (e.g, of the automotive insurance industry, of a particular insurance client, and the like). For example, in“id strikk cv”,“id” should be“IV” (“insured vehicle”) and“strikk” should be“struck”, indicating that an insured vehicle struck claimant vehicle. Additionally or alternatively, the model can adjust for regional phraseology such as determining that“driver ran the light” has the same meaning the same as“driver did not stop for light”.

As a result of step S104, a variety of free-form text descriptions may converge to a common format. For example, a description of“IV Rearend-ed CV” (along with variations thereof) can be cleaned to“Insur vehicl rear end claimant vehicl” (or other descriptions based on the custom dictionary).

In step S106, a Term Frequency - Inverse Document Frequency (TF-IDF) matrix is created in memory detailing relative frequency of a plurality of «-word terms within each insurance claim, wherein « is a positive integer. The consideration of «-word terms captures the potential significance of phrases such as the 2-word“stop light”, which may connote a more specific meaning than the l-word term“stop”.

Insurance claims can be mapped onto a vectorial representation where each report is a \ V\ dimensional vector, where Vis the vocabulary of a large collection of defect reports and \ V\ is the size of the vocabulary. The vocabulary can be obtained by extracting unique terms after preprocessing the collection. Each dimension in this space of | V\ dimensions corresponds to a unique term in the collection and the value along the dimension is a weight that indicates the importance of the term/dimension for a particular report d _j. Below is an exemplary representation for insurance claim d ₇ :

where w _t is the weight of term i in document d _j. Usually, if a term is not present in a document d _j , the corresponding value in the vector for the term is zero. Although the TF-IDF approach is described herein, other weighting schemes can be used.

TF-IDF is a composed measure that is the product of the frequency of a term in a document (TF) and its inversed document frequency (IDF). TF-IDF provides a metric measuring the importance of term. A term will receive a high value if it frequently appears in the document (TF) and does not occur often in other documents (IDF).

The TF (“term frequency”) value for term /, within document d _j can be expressed mathematically as TFi = n _Lj or can be normalized using the equation T F _L J

wherein is the number of occurrences of the considered term in document d _j and the denominator is the frequency of the most frequent term in document d _j. The denominator is used to normalize the TF values. Other normalization factors can be used such as the most-frequent term in any document in the collection or the number of occurrences of all terms in document d _j.

The IDF of a term is the percentage of distinct documents the term appears in amongst a collection of documents. The IDF measures the specificity of a term. The fewer documents a term appears in, the more specific the term is. The IDF can be expressed mathematically as:

IDF = , log

wherein | D | is the total number of documents in the corpus and | {d _{j \} t _t e d _j ] | is the number of documents where the term /, appears (that is, h, _I ¹0). In another embodiment, IDF(t ) = 1 + log (—7-:), wherein the logarithm function provides non-linear scaling, N represents the total number of reports in the collection, and d/(t) represents the number of reports containing the term t.

An exemplary implementation of a TF-IDF matrix is depicted in FIG. 4.

In step S108, a plurality of features are selected from the TF-IDF matrix. For example, a chi-squared test can be used to identify a difference in expected frequency of a cause of loss in the training set relative to the observed frequency of the cause of loss when a feature is present. The m n- grams having the largest chi-squared value can be selected for further use.

In step Sl 10, binary classifiers are created for each of a plurality of causes of loss. A binary classifier divides each of a set of insurance claims into two groups: YES or NO for each potential cause of loss.

Multiple classifier algorithms could be used instead of binary classifier algorithms. However, binary classifier algorithms are preferred because accident data tends to be unbalanced across cause of loss. For example, 40% of all accidents may be rear-end collisions and 30% may involve a parked car, while other causes of loss may occur in less than 10% of insurance claims. Multiple classifiers would prefer these larger classification“buckets” and tend to classify an insurance claim as the most prevalent cause of loss, even though other rarer causes of loss may also be present. Binary classification identifies all causes of loss and allows for optional later culling to one or more causes of loss in step S120.

Various binary classifier algorithms can be used including the following listed in Table 3 using a 70%-30% training-testing data split.

Referring again to FIG. 1B, the binary classifiers built in the method of FIG. 1 A can now be used to classify unseen insurance claims data. The initial loading and pre-processing steps Sl 12 and Sl 14 (including steps Sl l4a-Sl l4e) can be similar or identical to steps S102 and S104. In some embodiments, loading and pre-processing can be performed once. For example, insurance claims data can be received from an insurance client, loaded, pre-processed, and divided into training and real-world data sets in any order.

In step Sl 16, each binary classifier built in step Sl 10 can be applied against each insurance claim. This may produce a plurality of positive answers for a single insurance claim as seen in Table 2.

In step Sl 18, a priority or hierarchy of causes of loss can be applied to identify and associate a single cause of loss with each insurance claim. For example, an IF -ELSE loop can be applied over each insurance claim in the order of priority to check if the claim was classified as cause of loss 1, 2, When the highest priority cause of loss is identified, that cause of loss can be associated with the insurance claim and the loop can terminate. In other embodiments, searches can be performed iteratively for each cause of loss in order of priority.

In some embodiments, the priority order is set based on the number of classifications made in step Sl 18. For example, the priority order can be in order from most-to-least or least-to- most inclusive classifiers.

In still other embodiments, the binary classifiers can be applied in step Sl 16 in order of priority ( e.g ., the highest priority classifier first), with further binary classifiers only applied to the sub-set of insurance claims not yet classified by a classifier for a higher priority cause of loss.

Some insurance claims may not yield a positive answer for any cause of loss and can be designated as“unclassifiable” in step S120. String matching and/or human review can be performed to identify one or more causes of loss and such data can be used for further training of the binary classifiers in Sl 10.

Exemplary Deliverables

The classifications produced by the invention are valuable in and of themselves. The classifications can also be augmented to provide actionable advice to insurance and re-insurance companies. For example, the distribution of causes of loss produced by the methods described herein can be married with cost data from the insurance claims (e.g., by division) to identify which causes of loss are associated with the highest average cost per claim. Likewise, costs associated with the insurance claims or industry averages can be associated with the classifications ( e.g ., by multiplication) to identify which causes of loss result in the largest aggregate costs over a portfolio. Furthermore, the cause of loss distribution can be married with data regarding prevention techniques (e.g., forward collision warning device, driver training, rear backup camera, lane departure warning device, blind spot monitoring device, motorcycle warning device) and associated costs and risk reduction to identify the largest opportunities for cost savings. Additionally or alternatively, costs or risk exposure can be presented graphically (e.g, in bar or pie charts) or on numerical scale (e.g, exposure due to each cause of loss on a 0-10 scale).

Exemplary Implementation

Referring now to FIG. 2, another embodiment of the invention provides a system 200 for implementing the methods described herein.

An interface 204 can support communication with a data source 202, which can be an internal or an external client. For example, interface 204 can be a Web page or other internet technology that allow customers to upload insurance claims data. The interface 204 can verify compliance with various input requirements. The interface 204 can also relay the information to storage/memory 206 and/or processor 208. For example, the interface 204 can invoke one or more processes for execution by the processor 208.

Processor 208 can load and execute one or more functions, methods, modules, objects or other computing structures to implement the methods described herein. For example, data pre- processing, TF-IDF generation, feature selection, binary classifier generation, binary classifier application, and single cause of loss identification can be each implemented by a separate function/method/module 2l0a, 210b, 2l0c, 2l0d, 2l0e, and 21 Of, respectively. Such separate functions/methods/modules 2l0a, 210b, 2l0c, 2l0d, 2l0e, 21 Of can be invoked by an

overarching function/method/module that can pass the result of a previous

function/method/module to the next function/method/module, e.g, by reference to the resulting data.

Exemplary Technical Contributions

Prior to the development of the invention described herein, classification of insurance claims data was a tedious task requiring human labor. Applicant’s invention not only minimizes or eliminates the need for human labor and improves accuracy, but also provides several orders of magnitude of speed improvements over human classification. For example, Applicant can classify thousands of insurance claims in seconds.

Additionally, embodiments of the invention utilizing a plurality of binary classifiers are more accurate than models utilizing a single multiple classifier as discussed in greater detail herein.

EQUIVALENTS

Although preferred embodiments of the invention have been described using specific terms, such description is for illustrative purposes only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the following claims.

INCORPORATION BY REFERENCE

The entire contents of all patents, published patent applications, and other references cited herein are hereby expressly incorporated herein in their entireties by reference.

APPENDIX

Exemplary PYTHON® source code for an implementation of an embodiment of the methods described herein is provided below. lossdetect Model run

import timeit

# we keep record of the time we started the program to measure how long it takes to finish it.

start = timeit . default_timer ( )

# import necesary packages

import numpy as np

import pandas as pd

import os

import sys

import re

import random

import math

import pickle

from s klearn . externals import joblib

import csv

import string

from string import digits import nltk

from nltk import word_tokenize

from nltk . tokenize import TweetTokenizer

twitter_tokenizer = TweetTokenizer ( )

from nltk . stem. snowball import EnglishStemmer

stemmer = EnglishStemmer ( )

from s klearn . feature_extraction . text import TfidfTransformer

from sklearn . naive_bayes import MultinomialNB

from s klearn . linear_model import SGDClassifier

from s klearn . ensemble import RandomForestClassifier

from sklearn .model_selection import cross_val_score

from sklearn . neural_network import MLPClassifier

# take the filename (testing dataset) and the column name (which column represents the text description column) from parameters .

filename = sys.argv[l]

colname = sys.argv[2]

# we set the directory of each folder

folder_dir = "/data/lossmodelanalysis/"

project_dir = folder_dir + "project/"

function_dir = folder_dir + "src/"

data_dir = folder_dir + "data/"

library_dir = folder_dir + "library/"

model_dir = folder_dir + "binary/"

# load our defined functions

sys .path . append (function_dir)

from data_process_functions import *

from model_initiali zation_functions import *

from model_functions import *

# read in the test data set and the text columns

testing_data = pd . read_csv (data_dir + sys.argv[l])

col_id = ord (colname . upper () ) - ord('A')

print (len (testing_data) )

# create a stop word set

stop_word_list = set ()

stop_word_list . add ( "the " )

stop_word_list . add ( "an" )

stop_word_list . add ( "a" )

# load the before_token_map mapping table before tokenization

before_token_file = library_dir + "before_token_map . csv"

before_token_map = load_before_token_map (before_token_file) # load and import the data dictionary and save them into dictionary structure .

abbr = library_dir + "abbr.csv"

data_dictionary = library_dir + "data_dictionary . csv"

raw_term_origin , stemmed_term_origin = intialize_dictionaries (abbr, data_dictionary)

# extract the text column

test_text = testing_data . iloc [ : , col_id] . copy()

# apply the description_standardizer function to clean the text for i in range (0, len (test_text) ) :

temp_text = test_text[i]

cleaned_text = description_standardizer (str (temp_text) ,

before_token_map , raw_term_origin, stemmed_term_origin, stop_word_list) test_text[i] = cleaned_text

# now we load the pre-selected features and its weight (IDF)

index_file = pd . read_csv ( library_dir + "feature_selected . csv" ) control_words = index_file [ ' control_words ' ]

weights = index_file [' weight ' ]

# now we create the Term-Frequency matrix

word_posi = {}

for i in range (0, len (control_words ) ) :

word_posi [control_words [i] ] = i

test_data = np . zeros ( (len (test_text) , len (control_words ) ) )

for i in range (0, len (test_text) ) :

temp_words = test_text[i] .split (" ")

for j in range (0, len (temp_words ) ) :

uni = temp_words [ j ]

if uni in word_posi:

test_data[i] [word_posi [uni] ] += 1

for j in range (0, len (temp_words ) -1 ) :

bi = temp_words [ j ] + + temp_words [ j +1 ]

if bi in word_posi:

test_data[i] [word_posi [bi] ] += 1

for j in range (0, len (temp_words ) -2 ) :

thr = temp_words [ j ] + + temp_words [ j +1 ] + +

temp_words [ j +2 ]

if thr in word_posi:

test_data[i] [word_posi [thr] ] += 1 for j in range (0, len (temp_words ) -3 ) :

four = temp_words [ j ] + + temp_words [ j +1 ] + +

temp_words [ j +2 ] + + temp_words [ j +3 ]

if four in word_posi:

test_data[i] [word_posi [four] ] += 1

for j in range (0, len (temp_words ) -4 ) :

five = temp_words [ j ] + + temp_words [ j +1 ] + +

temp_words [ j +2 ] + + temp_words [ j +3 ] + + temp_words [ j +4 ]

if five in word_posi:

test_data[i] [word_posi [ five ] ] += 1

for j in range (0, len (temp_words ) -5 ) :

six = temp_words [ j ] + + temp_words [ j +1 ] + +

temp_words [ j +2 ] + + temp_words [ j +3 ] + + temp_words [ j +4 ] + + temp_words [ j+4]

if six in word_posi:

test_data[i] [word_posi [ six] ] += 1

# convert the TF matrix into TF-IDF matrix

temp_data = np . asarray ( test_data) *np . asarray (weights )

test_data = temp_data

# now we start loading all binary classifers

print ("start loading models")

# import all models

backing_rf = j oblib . load (model_dir + 'backing .pkl ' )

drivererror_rf = j oblib . load (model_dir + ' drivererror .pkl ' )

intersection_rf = j oblib . load (model_dir + ' intersection . pkl ' )

lanechange_rf = j oblib . load (model_dir + ' lanechange . pkl ' )

lanedeparture_rf = j oblib . load (model_dir + ' lanedeparture .pkl ' ) leftturn_rf = j oblib . load (model_dir + ' lettturn . pkl ' )

merging_rf = j oblib . load (model_dir + 'merging .pkl ' )

notatfault_rf = j oblib . load (model_dir + ' notatfault . pkl ' )

notauto_rf = j oblib . load (model_dir + ' notauto .pkl ' )

parked_vehicle_rf = j oblib . load (model_dir + ' parkedvehicle . pkl ' ) passing_rf = j oblib . load (model_dir + 'passing .pkl ' )

pedestrian_rf = j oblib . load (model_dir + ' pedestrian . pkl ' )

rearend_rf = j oblib . load (model_dir + ' rearend.pkl ' )

rightturn_rf = j oblib . load (model_dir + ' rightturn .pkl ' )

sideswipe_rf = j oblib . load (model_dir + ' sideswipe .pkl ' )

turningother_rf = j oblib . load (model_dir + ' turningother . pkl ' )

bicyclist_rf = j oblib . load (model_dir + ' bicyclist . pkl ' )

glassbreakage_rf = j oblib . load (model_dir + ' glassbreakage .pkl ' )

# once finish loading all models

print ( "finish loading models")

output_frame = [""] * len (test_data) print (test_data . shape)

print (len (output_frame) )

# now we should decompose the test_data into pieces .

# Each piece should have less or equal to 10k records

# the reason is that I don't want to throw all data into the model at once

# which may crash the model

number_pieces = int (test_data . shape [0] /10000) + 1

print (number_pieces )

rearend_predict = []

backing_predict = []

drivererror_predict = []

intersection_predict = []

lanechange_predict = []

lanedeparture_predict = []

lettturn_predict = []

merging_predict = []

notatfault_predict = []

notauto_predict = []

parked_vehicle_predict = []

passing_predict = []

pedestrian_predict = []

rightturn_predict = []

sideswipe_predict = []

turningother_predict = []

bicyclist_predict = []

glassbreakage_predict = []

# we process all pieces one by one

for curr in range (0, int ( number_pieces ) ) :

curr_test_data = test_data [curr*10000 : ( curr+1 ) * 10000 , :]

rearend_predict_temp = rearend_rf .predict (curr_test_data)

rearend_predict = np . concatenate ( [rearend_predict,

rearend_predict_temp] )

backing_predict_temp = backing_rf .predict (curr_test_data)

backing_predict = np . concatenate ( [backing_predict,

backing_predict_temp] )

drivererror_predict_temp = drivererror_rf .predict (curr_test_data) drivererror_predict = np . concatenate ( [drivererror_predict,

drivererror_predict_temp] )

intersection_predict_temp = intersection_rf .predict (curr_test_data) intersection_predict = np . concatenate ([ intersection_predict ,

intersection_predict_temp] )

lanechange_predict_temp = lanechange_rf .predict (curr_test_data) lanechange_predict = np . concatenate ([ lanechange_predict ,

lanechange_predict_temp ] ) lanedeparture_predict_temp = lanedeparture_rf . predict (curr_test_data) lanedeparture_predict = np . concatenate ( [lanedeparture_predict, lanedeparture_predict_temp] )

lettturn_predict_temp = leftturn_rf .predict (curr_test_data)

lettturn_predict = np . concatenate ([ lettturn_predict ,

lettturn_predict_temp] )

merging_predict_temp = merging_rf .predict (curr_test_data)

merging_predict = np . concatenate ( [merging_predict,

merging_predict_temp] )

notatfault_predict_temp = notatfault_rf .predict (curr_test_data) notatfault_predict = np . concatenate ( [notatfault_predict,

notatfault_predict_temp ] )

notauto_predict_temp = notauto_rf .predict (curr_test_data)

notauto_predict = np . concatenate ( [notauto_predict,

notauto_predict_temp] )

parked_vehicle_predict_temp =

parked_vehicle_rf .predict (curr_test_data)

parked_vehicle_predict = np . concatenate ( [parked_vehicle_predict, parked_vehicle_predict_temp] )

passing_predict_temp = passing_rf .predict (curr_test_data)

passing_predict = np . concatenate ( [passing_predict,

passing_predict_temp] )

pedestrian_predict_temp = pedestrian_rf .predict (curr_test_data) pedestrian_predict = np . concatenate ( [pedestrian_predict ,

pedestrian_predict_temp ] )

rightturn_predict_temp = rightturn_rf .predict (curr_test_data) rightturn_predict = np . concatenate ( [rightturn_predict,

rightturn_predict_temp] )

sideswipe_predict_temp = sideswipe_rf .predict (curr_test_data) sideswipe_predict = np . concatenate ( [sideswipe_predict,

sideswipe_predict_temp] )

turningother_predict_temp = turningother_rf .predict (curr_test_data) turningother_predict = np . concatenate ([ turningother_predict , turningother_predict_temp] )

bicyclist_predict_temp = bicyclist_rf .predict (curr_test_data) bicyclist_predict = np . concatenate ( [bicyclist_predict,

bicyclist_predict_temp] )

glassbreakage_predict_temp = glassbreakage_rf .predict (curr_test_data) glassbreakage_predict = np . concatenate ( [glassbreakage_predict, glassbreakage_predict_temp] ) # Now we determine what the output should be based on the result of each binary classifier

# and the priority order of each type of Cause of Loss

for i in range (0, len (output_frame) ) :

if notatfault_predict [ i ] == 1:

output_frame [i] = "NOT AT FAULT"

elif rearend_predict [ i ] == 1:

output_frame [ i ] = "REAR-END"

elif backing_predict [ i ] == 1:

output_frame [ i ] = "BACKING"

elif drivererror_predict [ i ] == 1:

output_frame [i] = "DRIVER ERROR"

elif intersection_predict [i] == 1:

output_frame [i] = "INTERSECTION"

elif lanechange_predict [ i ] == 1:

output_frame [i] = "LANE CHANGE"

elif lanedeparture_predict [ i ] == 1:

output_frame [i] = "LANE DEPARTURE"

elif leftturn_predict [i] == 1:

output_frame [i] = "LEFT TURN"

elif merging_predict [ i ] == 1:

output_frame [ i ] = "MERGING"

elif notauto_predict [ i ] == 1:

output_frame [i] = "NOT AUTO"

elif parked_vehicle_predict [ i ] == 1:

output_frame [i] = "COLLIDED WITH PARKED VEHICLE"

elif passing_predict [i] == 1:

output_frame [ i ] = "PASSING"

elif pedestrian_predict [ i ] == 1:

output_frame [i] = "PEDESTRIAN"

elif rightturn_predict [ i ] == 1:

output_frame [i] = "RIGHT TURN"

elif sideswipe_predict [i] == 1:

output_frame [ i ] = "SIDESWIPE"

elif turningother_predict [ i ] == 1:

output_frame [i] = "TURNING OTHER"

elif bicyclist_predict [i] == 1:

output_frame [ i ] = "BICYCLIST"

elif glassbreakage_predict [i] == 1:

output_frame [i] = "GLASS BREAKAGE"

else :

pass

print ("main parts done")

# modify the result based on my understanding of the model bias for i in range (0, len (output_frame) ) :

if output_frame [i] ==

if not_at_fault_checker (test_text [i] ) == 1:

output_frame [i] = "NOT AT FAULT"

else :

output_frame [ i ] = cause_standardizer (test_text [i] ) for i in range (0, len (output_frame) ) :

if output_frame [i] == "NOT AUTO" and "head on" in test_text[i] :

output_frame [ i ] = "UNKNOWN"

# import the dictionary to map each Cause of Loss to its mitigation techniques

mitigation_file = pd . read_csv ( library_dir + "mitigation_map . csv" ) score = 0

known = 0

preventable = { }

technique = { }

for i in range (0, len (mitigation_file) ) :

preventable [mitigation_file [' CAUSE OF LOSS'] [i] ] =

mitigation_file [ ' PREVENTABLE ' ] [i]

technique [mitigation_file [' CAUSE OF LOSS'] [i]] = mitigation_file [' LOSS MITIGATION TECHNIQUE'] [i]

prev = [ " " ] *len (output_frame )

tech = [ " " ] *len (output_frame )

for i in range (0, len (output_frame) ) :

prev[i] = preventable [output_frame [i] ]

tech[i] = technique [output_frame [i] ]

if prev [ i ] == ' Y ' :

score += 1

if output_frame [i] != "UNKNOWN":

known += 1

# and it will generate a score

score = round ( float ( score ) /float ( known) * 10 , 1)

# re-read the file, write the model result and output it

temp = pd . read_csv (data_dir + sys.argv[l])

temp [ ' model_result ' ] = output_frame

temp ['prev'] = prev

temp ['tech'] = tech

temp . to_csv (data_dir + "testresult . csv" )

with open (data_dir + "score.txt", 'w') as temp_file :

temp_file .write (str (score) )

stop = timeit . default_timer ( )

print (stop - start) lossdetect data process functions

# import necessary packages

import numpy as np

import pandas as pd

import os

import sys

import re

import math

import random

import pickle

import csv

import string

import nltk

from string import digits

from nltk import word_tokenize

from nltk . tokenize import TweetTokenizer

twitter_tokenizer = TweetTokenizer ( )

from nltk . stem. snowball import EnglishStemmer

stemmer = EnglishStemmer ( )

from s klearn . feature_extraction . text import CountVectorizer

from s klearn . feature_extraction . text import TfidfTransformer

from sklearn . naive_bayes import MultinomialNB

from s klearn . linear_model import SGDClassifier

from s klearn . ensemble import RandomForestClassifier

from sklearn .model_selection import cross_val_score

from sklearn . neural_network import MLPClassifier

# It will make sure that all money representation will be converted into float format, (remove space, dollar sign, comma and hyphen)

def clean_money (some_str) :

# eg :

# input = "$35.12"

# output = clean_money ( some_str )

# output = 35.12

# it makes a frequency-count of all items in a list and returns a list def reason_frequency (a_list) :

# it is the function designing for removing non-word items from a string, for text cleaning.

def modify_string (some_str) :

# it is designed to test whether an object is none value,

def isNaN(item) :

# sometimes we may have to clean the data even before the tokenization def load_before_token_map (before_token_mapping_file ) : # ititalize the dictionaries. The input are the abbr and the data dictionary,

# and the output are one mapping for raw words and one mapping for stemmed words

# it returns two dictionaries for the description_standarizer functions def intiali ze_dictionaries ( abbr , data_dictionary) :

# this is the function to clean the document, based on three dictionaries and the stop word list

def description_standardizer (descri , before_token_dictionary,

raw_dictionary, stemmed_dictionary, stop_set) :

# step 1: remove initial space and space at the ending, and then tokenization

# step 2: convert abbreviation to origin by raw_dictionary

# step 3: stem the words

# step 4: remove punctuations and numericals and stop words

# step 5: update with stemmed_dictionary

# eg :

# input = "IV STRUCK & KILLED COW ? IV DAMAGED

# output = description_standardizer (input, before_token_dictionary, raw_dictionary, stemmed_dictionary, stop_set)

# output = "insur vehicl hit kill anim insur vehicl damag"

# eg :

# input = "" insd car rear-end ov"

# output = description_standardizer (input, before_token_dictionary, raw_dictionary, stemmed_dictionary, stop_set)

# output = "insur vehicl rear end other vehicl" lossdetect model initialization functions

# import all packages:

import numpy as np

import pandas as pd

import os

import sys

import re

import math

import random

import pickle

import CSV

import string

import nltk

from string import digits

from nltk import word_tokenize

from nltk . tokenize import TweetTokenizer

twitter_tokenizer = TweetTokenizer ( )

from nltk . stem. snowball import EnglishStemmer

stemmer = EnglishStemmer ( ) from s klearn . feature_extraction . text import CountVectorizer

from s klearn . feature_extraction . text import TfidfTransformer

from sklearn . naive_bayes import MultinomialNB

from s klearn . linear_model import SGDClassifier

from s klearn . ensemble import RandomForestClassifier

from sklearn .model_selection import cross_val_score

from sklearn . neural_network import MLPClassifier

from s klearn . externals import joblib

# create word features from documents

# Each word feature is a gram. We extract work features from unigram to six gram.

# And for the ngrams with a total frequency less or equal than 2, we will not keep them because they are rare words .

def gram_creator (texts ) :

# In this function, we calculate the IDF based on the feature words extracted .

# the data is the TF matrix created. Each column is a word feature and each row is the vector representation of a document

# For each word feature, it calcuates its document frequency (how many documents have this ngram) and then calcualte the IDF (inverse document frequency) of each ngram.

def IDF_calculator (data) :

# Now based on the IDF we created on the TF matrix, we combine them together to arrive at the TF-IDF matrix

# We can think of it in this way. IDF is a word feature's weight based on how many documents contain it, or we can say it is the coefficient

# related to each word feature.

# Combine TF matrix and IDF together into a TF-IDF matrix, we are actually adjusting the weights of each feature.

def tfidf creater (data , idf) : lossdetect model functions

# import all necessary packages

import numpy as np

import pandas as pd

import os

import sys

import re

import math

import random

import pickle

import csv

import string

import nltk

from string import digits

from nltk import word_tokenize

from nltk . tokenize import TweetTokenizer twitter_tokenizer = TweetTokenizer ()

from nltk . stem. snowball import EnglishStemmer

stemmer = EnglishStemmer ( )

import sklearn

from sklearn. feature_extraction . text import CountVectorizer

from sklearn. feature_extraction . text import TfidfTransformer

from sklearn . naive_bayes import MultinomialNB

from s klearn . linear_model import SGDClassifier

from s klearn . ensemble import RandomForestClassifier

from sklearn .model_selection import cross_val_score

from sklearn . neural_network import MLPClassifier

# The function to calculate FI statistics based on two list. Each list contains only 1 or 0.

# https : //en .wikipedia . org/wiki/Fl_score

def Fl(list_l, list_2) :

# In this section, we hardcoded lots of commonly-used string patterns to determine the at fault party.

# it returns 1 (is not at fault) or 0 (is at fault)

def not_at_fault_checker (text) :

# eg :

# input = "rear end by another vehicle"

# output = not_at_fault_checker (input)

# output = 1

# eg :

# input = "rear end another vehicle"

# output = not_at_fault_checker (input)

# output = 0

# If all binary classifies can't tell the really Cause of Loss, we will look for typical string patterns to determine

# the Cause of Loss. String patterns can be a word or phrase,

def cause_standardizer (text) :

# it is the string matching part

# eg :

# input = "rear end another vehicle"

# output = cause_standardizer (input)

# output = "REAR-END"

# eg :

# input = "HIT A PEDESTRIAN IN THE TURNING"

# output = cause_standardizer (input)

# output = "PEDESTRIAN"