Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHODS AND DEVICES FOR DETECTING IMPROPER CLINICAL PROGRAMMING OF IMPLANTABLE MEDICAL DEVICES
Document Type and Number:
WIPO Patent Application WO/2020/225819
Kind Code:
A1
Abstract:
Provided are devices and methods for detection of malicious programming between a programmer device and an implantable medical device. The methods and devices disclosed act as a firewall to identify malicious commands issued by the programmer and prevent them from reaching the implantable medical device.

Inventors:
NISSIM NIR (IL)
KATZ AMOS (IL)
Application Number:
PCT/IL2020/050502
Publication Date:
November 12, 2020
Filing Date:
May 07, 2020
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
B G NEGEV TECHNOLOGIES AND APPLICATIONS LTD AT BEN GURION UNIV (IL)
FUND FOR MEDICAL RES DEVELOPMENT OF INFRASTRUCTURES & HEALTH SERVICES BARZILAI MEDICAL CENTER R E (IL)
ASSUTA MEDICAL CENTERS LTD (IL)
International Classes:
G06F21/54; A61N1/08; G06F21/52
Domestic Patent References:
WO2018086544A12018-05-17
Foreign References:
CN1649311A2005-08-03
CN106777024A2017-05-31
US20160110528A12016-04-21
CN201894778U2011-07-13
US20180322263A12018-11-08
Attorney, Agent or Firm:
SILVERMAN, Eran et al. (IL)
Download PDF:
Claims:
CLAIMS

What is claimed is:

1. A device for detection of malicious programming between a programmer and implantable medical device (IMD), the device comprising: a processing unit configured to: detect a programming sent from the programmer device to the IMD; and analyze the programming by utilizing one or more detection layers, to determine if one or more parameter attributes of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious.

2. The device according to claim 1, wherein the one or more detection layers comprises a first detection layer comprising determining the attribute of the parameters value, based on deterministic rules.

3. The device according to claim 2, wherein the deterministic rules are at least partially based on experts' knowledge.

4. The device according to any one of claims 1-3, wherein the one or more detection layers comprises a detection layer for determining the attribute of the parameters value distribution of the programming, for each corresponding parameter value in the training dataset.

5. The device according to any one of claims 1-4, wherein the one or more detection layers comprises a detection layer for determining the attribute of the probability of a change of a parameter's value and/or a detection layer for determining the attribute of the probability of the change of a parameter’s value by a delta amount.

6. The device according to any one of claims 1-5, wherein the one or more detection layers comprises a detection layer for determining the probability of each combination of values of every pair of two parameters in the training data set.

7. The device according to any one of claims 1-6, wherein the one or more detection layers comprises a detection layer comprising a machine learning algorithm to determine a deviation in the one or more parameter attributes.

8. The device according to any one of claims 1-7, wherein the detection layers comprises at least a first layer comprising deterministic rules and a second layer comprising a detection layer for determining the attribute of the parameters value distribution of the programming.

9. The device according to any one of claims 1-8, wherein the one or more detection layers comprises six detection layers.

10. The device according to any one of claims 1-9, wherein the training data set comprises parameters of benign programmings.

11. The device according to any one of claims 1-10, wherein the data set further comprises parameters of malicious programming.

12. The device according to any one of claims 1-11, wherein the analysis comprises determining a threshold value for a parameter in one or more of the detection layers.

13. The device according to claim 12, wherein the threshold value is selected from a baseline value or an actual threshold value, determined based on a constant threshold or a data-driven threshold.

14. The device according to any one of claims 1-13, wherein determination of the programing being malicious comprises a true positive rate (TPR) of over about 90% and/or a false positive rate (FPR) of less than about 10%.

15. The device according to any one of claims 1-14, wherein the implantable medical device is selected from: a pacemaker, an implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor (ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, and phrenic nerve stimulator.

16. The device according to any one of claims 1-15, wherein the IMD is a cardiac implantable electronic device (CIED).

17. The device according to any one of claims 1-16, wherein the device is further configured to issue an alert if a malicious programming has been identified.

18. The device according to any one of claims 1-17, wherein the device is further configured to prevent or block a malicious programming from reaching the IMD.

19. The device according to any one of claims 1-18, further comprising one or more of: a communication unit, a power source, a display, a user interface, an alert unit.

20. A method for detecting a malicious programming between a programmer and implantable medical device (IMD), the method comprising: detecting a programming sent from the programmer device to the IMD; and analyzing the programming by utilizing one or more detection layers, to determine if one or more parameter attributes of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious.

21. The method according to claim 20, wherein the one or more detection layers comprises a first detection layer which comprises determining the attribute of the parameters value, based on deterministic rules, based on expert's knowledge.

22. The method according to claim 21, wherein the deterministic rules are at least partially based on expert's knowledge.

23. The method according to any one of claims 20-22, wherein the one or more detection layers comprises a detection layer for determining the attribute of the parameters value distribution of the programming, for each corresponding parameter value in the training dataset.

24. The method according to any one of claims 20-23, wherein the one or more detection layers comprises a detection layer for determining the attribute of the probability of a change of a parameter's value and/or a detection layer for determining the attribute of the probability of the change of a parameter’s value by a delta amount.

25. The method according to any one of claims 20-24, wherein the one or more detection layers comprises a detection layer for determining the probability of each combination of values of every pair of two parameters in the training data set.

26. The method according to any one of claims 20-25, wherein the one or more detection layers comprises a detection layer comprising a machine learning algorithm to determine a deviation in the one or more parameter attributes.

27. The method according to any one of claims 20-26, wherein the detection layers comprises at least a first layer comprising deterministic rules and a second layer comprising a detection layer for determining the attribute of the parameters value distribution of the programming.

28. The method according to any one of claims 20-27, wherein the one or more detection layers comprises six detection layers.

29. The method according to any one of claims 20-28, wherein the training data set comprises parameters of benign programmings.

30. The method according to any one of claims 20-29, wherein the data set further comprises parameters of malicious programmings.

31. The method according to any one of claims 20-30, further comprising determining a threshold value for a parameter in one or more of the detection layers.

32. The method according to claim 31, wherein the threshold value is selected from a baseline value or an actual threshold value, determined based on a constant threshold or a data-driven threshold.

33. The method according to any one of claims 20-32, having a true positive rate (TPR) of over about 90% and/or a false positive rate (FPR) of less than about 10%.

34. The method according to any one of claims 20-33, wherein the implantable medical device is selected from: a pacemaker, an implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor (ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, and phrenic nerve stimulator.

35. The method according to any one of claims 20-34, wherein the implantable medical device is a cardiac implantable electronic device (CIED).

36. The method according to any one of claims 20-35, further comprising issuing an alert if a malicious programming is identified.

37. The method according to any one of claims 20-36, further comprising preventing or blocking a malicious programming from reaching the IMD.

38. A non-transitory computer-readable medium having stored thereon a computer program for execution by a processor configured to execute a method of detecting a malicious programming between a programmer and implantable medical device (IMD), the method comprising: detecting a programming sent from the programmer device to the IMD; and analyzing the programming by utilizing one or more detection layers, to determine if one or more parameter attributes of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious.

39. The non-transitory computer-readable medium according to claim 38, wherein the one or more detection layers comprises a first detection layer comprises determining the attribute of the parameters value, based on deterministic rules, based on expert's knowledge.

40. The non-transitory computer-readable medium according to claim 39, wherein the deterministic rules are at least partially based on expert's knowledge.

41. The non-transitory computer-readable medium according to any one of claims 38- 40, wherein the one or more detection layers comprises a detection layer for determining the attribute of the parameters value distribution of the programming, for each corresponding parameter value in the training dataset.

42. The non- transitory computer-readable medium according to any one of claims 38-41, wherein the one or more detection layers comprises a detection layer for determining the attribute of the probability of a change of a parameter's value and/or a detection layer for determining the attribute of the probability of the change of a parameter’s value by a delta amount.

43. The non-transitory computer-readable medium according to any one of claims 38-

42, wherein the one or more detection layers comprises a detection layer for determining the probability of each combination of values of every pair of two parameters in the training data set.

44. The non-transitory computer-readable medium according to any one of claims 38-

43, wherein the one or more detection layers comprises a detection layer comprising a machine learning algorithm to determine a deviation in the one or more parameter attributes.

45. The non-transitory computer-readable medium according to any one of claims 38-

44, wherein the detection layers comprises at least a first layer comprising deterministic rules and a second layer comprising a detection layer for determining the attribute of the parameters value distribution of the programming.

46. The non-transitory computer-readable medium according to any one of claims 38-

45, wherein the one or more detection layers comprises six detection layers.

47. The non-transitory computer-readable medium according to any one of claims 20-

46, wherein the training data set comprises parameters of benign programmings and/or malicious programmings.

48. The non-transitory computer-readable medium according to any one of claims 38-

47, wherein the method further comprises determining a threshold value for a parameter in one or more of the detection layers.

49. The non-transitory computer-readable medium according to claim 48, wherein the threshold value is selected from a baseline value or an actual threshold value, determined based on a constant threshold or a data-driven threshold.

50. The non-transitory computer-readable medium according to any one of claims 38- 49, the method having a true positive rate (TPR) of over about 90% and/or a false positive rate (FPR) of less than about 10%.

51. The non-transitory computer-readable medium according to any one of claims 38- 50, wherein the implantable medical device is selected from: a pacemaker, an implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor (ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, and phrenic nerve stimulator.

52. The non-transitory computer-readable medium according to any one of claims 38- 51, wherein the method further comprises issuing an alert if a malicious programming is identified and/or preventing or blocking a malicious programming from reaching the IMD.

Description:
METHODS AND DEVICES FOR DETECTING IMPROPER CLINICAL PROGRAMMING OF IMPLANTABLE MEDICAL DEVICES

TECHNICAL FIELD

The present disclosure relates generally to methods and devices for detecting and prevent improper (malicious) clinical programming between implantable medical devices (IMD) and programmers thereof.

BACKGROUND

An implanted medical device (IMD) is a medical device that is implanted within the patient’s body. The number users with implanted medical devices, such as cardiac implantable electronic devices (CIEDs), is increasing each year. According to the European Heart Rhythm Association, in Western Europe in each of the years between 2009 and 2013, the number of pacemaker implantations per million inhabitants was about 1,100 (0.11%).

Currently, implanted medical devices, such as, CIEDs, usually contain an embedded operating system (for example, Windows or Linux) and advanced electronic components, all of which are aimed at improving a patient’s physical functioning. These advanced electronic components could suffer from the same security breaches and vulnerabilities that exist in the operating systems they are based on, or alternatively, they can suffer from new vulnerabilities associated with their additional components. In both cases, attackers aim at exploiting such breaches and vulnerabilities in order to manipulate the normal behavior of the medical device and launch their attack.

In recent years, there has been an increase in the discovery of vulnerabilities and risks associated with implanted medical devices, in particular, pacemakers and implantable cardioverter defibrillator (ICDs). Although security issues associated with medical devices were first presented over a decade ago, many CIEDs may still be vulnerable to a variety of attacks. This is due to the fact that vendors invest most of their efforts in developing new medical devices with additional treatment functionalities, rather than solving security issues. During a follow-up visit of a patient with an implanted medical device, such as, for example, an ICD device, the physician assesses the patient's medical condition using a dedicated computer system referred to herein a programmer, which extracts information from the patient’ s ICD. According to this evaluation, the physician may decide to program the ICD in order to change the parameters of pacing therapy, detection and therapy parameters of the ICD. The connection between the

The programmer device allows connections from other devices and peripherals, such as a keyboard, mouse, or flash drive, via a USB socket, Bluetooth, and Ethernet, in order to allow doctors or technicians to more comfortably use the programmer device and to give them the ability to extract data from it. These connectivity capabilities open the door to cyber-attacks, enabling attackers to penetrate the programmer devices and inject malware into them. For instance, a doctor could attach an infected USB flash drive to the programmer device that can install a malware or cause a firmware update to the programmer device using malware resident on the connected USB device. In each case, the original functionality of the programmer device can be altered according to the attacker’s aims, a scenario which can endanger patients.

In recent years, several studies have explored the issue of cyber-attacks against personal medical devices (PMDs), such as CIEDs and insulin pumps, and proposed security mechanisms to protect these devices. Several solutions, such as disclosed in [ 1]— [6] , have been proposed, in which an external device aimed at determining whether the information passing from an external programmer device to an implantable device, is coming from a legitimate source (e.g., a legitimate programmer device), to prevent, for example, resource depletion attack (i.e., an attack that overconsumes the resources of the CIED, such as the battery, memory, etc.). Such solutions based their decision on information from the environment (i.e., the meta-data), such as the physical characteristics of the channel, the location of the patient, the time of the follow-up, and so on. Downsides of the abovementioned security mechanisms include, inter alia: 1) they do not protect the implanted device, such as, ICD device against attacks initiated from legitimate programmer devices, and/or 2) they require a change to the firmware of the existing medical devices, in order to share a secret key between them, and/or they require the use of a smartphone of the patient as a communication mediator between the IMD and the programmer device. Another solution is disclosed in US Patent No. US 10, 135,849, which is directed to securing medical devices through wireless monitoring and anomaly detection.

The use of machine learning (ML) methods on the clinical data itself in order to protect PMDs from cyber-attacks has been limited. One study [8] proposed an idea for using ML methods to protect CIEDs. To the best of the inventor's knowledge, no art has used and applied machine learning methods on the clinical data itself to protect CIEDs from cyber-attacks.

In addition to the various data mining and machine learning based detection mechanisms, there are also some security mechanisms for CIEDs that are not based on analyzing data, but rather rely on concepts from cryptography, biometrics, and more. For example, One-Time Pad Encryption System is disclosed [9], which uses a classic encryption technique in order to protect CIEDs from malicious access. Other studies [ 10]— [ 16] present non-data-based mechanisms that leverage the use of biometric identifiers from the patient’s body in order to authenticate access to the medical device. The interpulse interval (IPI) of the patient’s heartbeats is a well-known biometric identifier.

Thus, various attempts have been made to proposed protection mechanisms for IMDs against cyber-attacks. However, all of the proposed solutions are aimed against attacks coming from illegitimate sources, such as an improvised antenna or a stolen programmer device.

Thus, there is need in the art for systems, devices and methods for protection of IMDs, from a compromised legitimate source, such as a programmer device, whereby the systems, devices and methods allow the analysis of parameters that are being sent from the programmer device to the IMD and further determine whether they are benign or dangerous/malicious programming, based on parameters and values identified in the programming, even if the data comes from a source considered legitimate.

SUMMARY

Aspects of the disclosure, according to some embodiments thereof, relate to systems, devices and methods designed to protect implanted medical devices (IMDs) from cyber-attacks aimed at the programmer device. More specifically, but not exclusively, aspects of the disclosure, according to some embodiments thereof, relate to a novel and advantageous detection and prevention system, device and method, aimed at the detection of malicious clinical commands (also referred to herein as "programming") sent by a compromised legitimate programmer device to an implanted medical device, such as, CIEDs.

According to some embodiments, the disclosed protecting system can be deployed as an external and trusted device and/or executed method which acts as an active intermediary (i.e., man-in-the -middle or a "firewall") between the programmer device and the IMD, thus allowing the use of existing medical devices without the need to change their hardware or software. According to some embodiments, the device and methods disclosed herein are configured to analyze the clinical commands (programming) sent from the programmer, intercept identified malicious clinical commands, and prevent them from being transmitted to the IMD, before they can cause harm to the patient.

According to some embodiments, the disclosed devices and methods are highly advantageous, as they are able to look at the parameters that are being sent from the programmer device to the IMD and determine whether it is benign or dangerous/malicious programming, using the parameters and values found in the programming, even if the data comes from a source considered legitimate. Advantageously, the inspection/analysis of the clinical data is much more comprehensive than merely analyzing the meta-data, and results in a more reliable and accurate detection and prevention mechanism that better identifies malicious/dangerous clinical commands and produces fewer false alarms. Further, the devices and methods are advantageous as do not require any hardware modification of the IMD.

Thus, according to some embodiments, the disclosed advantageous devices, methods and systems are able to: provide a defense mechanism against attacks that are initiated from compromised programmer device aimed at IMDs; inspect and analyze the content of the programmings sent from the programmer device to the ICD, rather than the meta-data (e.g., information about the connection used, etc.); use a real dataset, originating from real patients’ treatments at different hospitals and clinics, of programmings sent to patients’ IMD devices; and/or can be used for detecting design and implementation bugs in the programmer’s software, preventing human errors (e.g., technicians, doctors, etc.), and may also be used by interns for educational purposes.

According to some embodiments, the detection and prevention systems, devices and methods disclosed herein utilize a plurality of layers of protection, including, for example, medical experts’ knowledge, statistical methods, and machine learning algorithms, in order to analyze, detect and prevent malicious programming. According to some embodiments, as further exemplified hereinbelow, the layers of protection utilized provided a high detection capability associated with high rates of true positive, and low rates of false positive.

According to some embodiments, there is provided a detection and prevention system designed to protect IMDs (such as, ICDs) from cyber-attacks aimed at the programmer device. In some embodiments, the disclosed system has six different layers of protection, leveraging medical experts’ knowledge, statistical methods, and machine learning algorithms. According to some embodiments, as exemplified below, the detection system was evaluated extensively in two comprehensive experiments, with respect of ICD. For the evaluation, data was gathered to use 775 benign clinical commands that are related to hundreds of different patients and 28 malicious clinical commands (created by cardiology experts). The results show that the detection system provided a high detection capability associated with high rates of true positive, and low rates of false positive.

According to some embodiments, there is provided a device for detection of malicious programming between a programmer and implantable medical device (IMD), the device includes a processing unit configured to:

detect a programming sent from the programmer device to the IMD; and analyze the programming by utilizing one or more detection layers, to determine if one or more parameter attributes of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious. According to some embodiments, the one or more detection layers may include a first detection layer which includes determining the attribute of the parameters value, based on deterministic rules.

According to some embodiments, the deterministic rules may be at least partially based on experts' knowledge.

According to some embodiments, the one or more detection layers may include a detection layer for determining the attribute of the parameters value distribution of the programming, for each corresponding parameter value in the training dataset.

According to some embodiments, the one or more detection layers may include a detection layer for determining the attribute of the probability of a change of a parameter's value and/or a detection layer for determining the attribute of the probability of the change of a parameter’s value by a delta amount.

According to some embodiments, the one or more detection layers may include a detection layer for determining the probability of each combination of values of every pair of two parameters in the training data set.

According to some embodiments, the one or more detection layers may include a detection layer which includes or utilize a machine learning algorithm to determine a deviation in one or more parameter attributes.

According to some embodiments, the detection layers may include at least a first layer comprising deterministic rules and a second layer comprising a detection layer for determining the attribute of the parameters value distribution of the programming.

According to some embodiments, the one or more detection layers may include six detection layers.

According to some embodiments, the training data set may include parameters of benign programmings. According to some embodiments, the training data set may further include parameters of malicious programming.

According to some embodiments, the analysis may include determining a threshold value for a parameter in one or more of the detection layers. In some embodiments, the threshold value may be selected from a baseline value or an actual threshold value, which may be determined based on a constant threshold or a data-driven threshold.

According to some embodiments, the determination of the programing being malicious may include a true positive rate (TPR) of over about 90% and/or a false positive rate (FPR) of less than about 10%.

According to some embodiments, the implantable medical device may be selected from: a pacemaker, an implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor (ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, and phrenic nerve stimulator. According to some embodiments, the IMD may be a cardiac implantable electronic device (CIED).

According to some embodiments, the device may be further configured to issue an alert if a malicious programming has been identified.

According to some embodiments, the device may be further configured to prevent or block a malicious programming from reaching the IMD.

According to some embodiments, the device may further include one or more of: a communication unit, a power source, a display, a user interface, an alert unit.

According to some embodiments, there is provided a method for detecting a malicious programming between a programmer and implantable medical device (IMD), the method includes one or more of the steps of:

- detecting a programming sent from the programmer device to the IMD; and

- analyzing the programming by utilizing one or more detection layers, to determine if one or more parameter attributes of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious.

According to some embodiments, the one or more detection layers of the method may include a first detection layer which includes determining the attribute of the parameters value, based on deterministic rules, based on expert's knowledge. According to some embodiments, the deterministic rules may at least partially be based on expert's knowledge. According to some embodiments, the one or more detection layers of the method may include a detection layer for determining the attribute of the parameters value distribution of the programming, for each corresponding parameter value in the training dataset.

According to some embodiments, the one or more detection layers of the method may include a detection layer for determining the attribute of the probability of a change of a parameter's value and/or a detection layer for determining the attribute of the probability of the change of a parameter’s value by a delta amount.

According to some embodiments, the one or more detection layers of the method may include a detection layer for determining the probability of each combination of values of every pair of two parameters in the training data set.

According to some embodiments, the one or more detection layers of the method may include a detection layer comprising a machine learning algorithm to determine a deviation in the one or more parameter attributes.

According to some embodiments, the one or more detection layers of the method may include a first layer comprising deterministic rules and a second layer comprising a detection layer for determining the attribute of the parameters value distribution of the programming.

According to some embodiments, the one or more detection layers of the method may include six detection layers.

According to some embodiments, the training data set of the method may include parameters of benign programmings. In some embodiments, the training data set may further parameters of malicious programmings.

According to some embodiments, the method may further include determining a threshold value for a parameter in one or more of the detection layers. According to some embodiments, the threshold value may be selected from a baseline value or an actual threshold value, determined based on a constant threshold or a data-driven threshold.

According to some embodiments, the method may have a true positive rate (TPR) of over about 90% and/or a false positive rate (FPR) of less than about 10%. According to some embodiments, the implantable medical device may be selected from: a pacemaker, an implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor (ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, and phrenic nerve stimulator. In some embodiments, the implantable medical device may be a cardiac implantable electronic device (CIED).

According to some embodiments, the method may further include issuing an alert if a malicious programming is identified.

According to some embodiments, the method may further include preventing or blocking a malicious programming from reaching the IMD.

According to some embodiments, there is provided a non-transitory computer- readable medium having stored thereon a computer program for execution by a processor configured to execute or perform a method of detecting a malicious programming between a programmer and implantable medical device (IMD), the method includes: detecting a programming sent from the programmer device to the IMD; and analyzing the programming by utilizing one or more detection layers, to determine if one or more parameter attributes of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious.

Certain embodiments of the present disclosure may include some, all, or none of the above advantages. One or more other technical advantages may be readily apparent to those skilled in the art from the figures, descriptions, and claims included herein. Moreover, while specific advantages have been enumerated above, various embodiments may include all, some, or none of the enumerated advantages.

BRIEF DESCRIPTION OF THE FIGURES

Some embodiments of the disclosure are described herein with reference to the accompanying figures. The description, together with the figures, makes apparent to a person having ordinary skill in the art how some embodiments may be practiced. The figures are for the purpose of illustrative description and no attempt is made to show structural details of an embodiment in more detail than is necessary for a fundamental understanding of the disclosure. For the sake of clarity, some objects depicted in the figures are not drawn to scale. Moreover, two different objects in the same figure may be drawn to different scales. In particular, the scale of some objects may be greatly exaggerated as compared to other objects in the same figure.

In block diagrams and flowcharts, optional elements/components and optional stages may be included within dashed boxes.

In the figures:

Figure 1 - a schematic diagram of programming routing between a programmer device and an IMD, in the absence and presence of the detection and prevention firewall, according to some embodiment;

Figures 2A-2E - Bar graphs exemplifying the threshold selection process, according to some embodiments. Fig. 2A- Example of strictly increasing probability, where the values’ probabilities are different from one another - in this case, there is no need for grouping, because there are not two or more values with the same probability- the solid line indicates the baseline threshold, and the dotted line indicates the data-driven threshold selected; Figs. 2B-C - Examples of monotonically increasing probability- selected threshold is greater than zero, where more than one value with the same probability is found, in this case, the values are grouped together, assigning them the probability that equals the sum of their probabilities. Fig. 2B presents this case before the grouping is done, and Fig. 2C presents this case after the grouping and threshold selection have taken place. The solid line indicates the baseline threshold, and the dotted line indicates the data-driven threshold selected. Figs. 2D-E - Example of monotonically increasing probability- selected threshold equals zero, where there is no such threshold value with the sum of the probabilities until and including that value which is less than or equal to the baseline threshold value. Fig. 2D presents this case before the grouping is done, and Fig. 2E presents this case after the grouping and threshold selection have taken place. The solid line indicates the baseline threshold, and the dotted line marks the dynamic threshold selected; Figure 3 is a block diagram of a device for detection of malicious programming, according to some embodiments;

Figure 4 is a flow chart of steps in a method for detection of malicious programming, according to some embodiments;

Figure 5 - shows an example of a probability histogram of the basic heart rate parameter (bpm), according to the benign programmings of an ICD collected and analyzed; and

Figure 6 - shows histogram of the probability of changes that were made for each of the parameters in each of the benign programming that were collected in the experiments described herein. The x-axis represents the enumerate identification of each parameter (260 parameters in total). The y-axis represents the probability of a change to each parameter, based on the benign data collected.

DETAILED DESCRIPTION

The principles, uses, and implementations of the teachings herein may be better understood with reference to the accompanying description and figures. Upon perusal of the description and figures present herein, one skilled in the art will be able to implement the teachings herein without undue effort or experimentation. In the figures, same reference numerals refer to same parts throughout.

According to some embodiments, there are provided herein systems (devices and methods) for detection, identification and/or prevention of a malicious programming between a legitimate programmer device and an implanted medical device. In some embodiments, the devices and methods address a unique need to protect the environment between a trusted (or seemingly trusted) programmer and an implanted device associated therewith and to alert a user regarding malicious commands arriving from the trusted and legitimate programmer, which may have been compromised (intentionally or mistakenly). The device and methods act as a firewall between the programmer device and the IMD, which is implanted and active within a subject. The devices and methods disclosed herein make use of an advantageous detection model, based at least in part on artificial intelligent and machine learning tools, which combines one or more detection layers, to results in a detection which is accurate, highly specific and having low false positive results.

In some embodiments, malicious or dangerous programmings, can occur outside the context of cyber-attacks, for example, in cases where the doctor or technician mistakenly set some parameters with dangerous values while programming the IMD, such as, CIED. Thus, the relevance of the detection system (device and methods) disclosed extends beyond the field of cyber-attacks and can determine cases of human mistake, bugs and miscellaneous anomalies. In some instances, for example, when used with CIED, it can also help cardiology specialists who want to learn how to use the programmer device and program ICD devices without inadvertently harming patients; the specialists could use the system while programming an ICD device intended for training purposes and receive alerts if they program the ICD with a dangerous programming. In some embodiments, during the training, the b parameter of the TF measure can be changed in accordance with the training purposes.

According to some embodiments, given the possible cyber-attacks on IMDs, such as ICDs, the detection system disclosed herein, is the first to: 1) provide a defense mechanism against attacks that are initiated from the legitimate programmer device, 2) observe the content of the programmings sent from the programmer device to the ICD, rather than the meta-data, and 3) uses a real dataset of programmings sent to patients’ IMD devices.

According to some embodiments, the detection system can efficiently detect malicious programming. Thus, if attacks on IMD devices are adopted by countries, organizations, or terrorists, the detection system may be able to protect the lives of millions of people.

In accordance with other embodiments, the detection system can advantageously work independently, without the need to modify the hardware or software of the IMD or programmer device.

According to some embodiments, the detection system may issue alerts about a malicious programing, while accompanying the alert with an explanation, indicating what, specifically, caused the system to make the decision, and presenting the problematic parameters and/or the value for each parameter, in order to give the health care provider, the ability to understand any situation and respond appropriately.

In some embodiments, in medical emergency scenarios, such as the urgent evacuation of a patient to the hospital, the detection system may still function as usual and allow the full functionality of the IMD and its communication to the programmer, since the detection system does not rely on the patient’s medical condition, but rather on data that is based on previous clinical programmings. In addition, the medical doctor is the one who makes the final decision, as the detection system serves as a decision-making support system that raises alerts for the doctor’s consideration. In cases in which the doctor chooses to program the IMD in a very unusual way (i.e., anomalous) as a means of treating a patient in an emergency situation, depending on the configuration thereof, the detection system may not prevent the doctor from doing so.

Thus, according to some embodiments there are provided systems, devices and methods for protection of IMDs from a compromised legitimate source, such as a programmer device, whereby the systems, devices and methods allow the analysis of parameters that are being sent from the programmer device to the IMD and further determine whether they are benign or dangerous/malicious programming, based on parameters and values identified in the programming, even if the data comes from a source considered legitimate.

Definitions

As used herein, the term "implantable medical device" and "IMD" may interchangeably be used. The terms are directed to a medical device which is configured to be implanted in the body of a subject and may perform life -sustaining functions and/or monitoring, such as, for example, cardiac pacing, cardiac defibrillation, nerve and/or neurostimulation, monitoring, and the like. IMDs may also allow monitoring, recording and/or storing patient information. The IMD may be programmed/controlled (for example by wireless routes) by a dedicated, external programmer system (such as a computer system), as further detailed herein.

According to some embodiments, an IMD may be selected from, but not limited to: pacemaker, implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor

(ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, phrenic nerve stimulator, and the like, or any combination thereof. Each possibility is a separate embodiment.

According to some embodiments, the IMD may be related to treatment, prevention, amelioration and/or monitoring of various conditions. In some embodiments, the conditions are related to heart diseases. In some embodiments, the conditions are related to chronic pain. In some embodiments, the conditions are related to neurological conditions, such as Parkinson's disease and epilepsy. In some embodiments, the conditions are related to Sleep Apnea.

According to some embodiments, IMDs related to heart disease may be selected from: pacemaker, implanted cardioverter defibrillator (ICD) and/or insertable Cardiac Monitor (ICM). Each possibility is a separate embodiment. According to some embodiments, IMDs related to heart disease are referred to as cardiac implantable electronic devices (CIEDs).

According to some embodiments, IMDs related to chronic pain may be selected from: Chronic Pain neurostimulator and/or Cluster headache neurostimulator.

According to some embodiments, IMDs related to Sleep Apnea may be selected from: upper airway stimulator and/or phrenic nerve stimulator.

According to some exemplary embodiments, the IMD is a pacemaker. A pacemaker is a compact digital medical device implanted in the body of a patient for the purpose of controlling brady arrhythmias, i.e., abnormally slow heart rhythms. The role of a pacemaker is to maintain the heart rate above a programmable predetermined rate ("basic rate") using low energy electrical pulses. The pacemaker is implanted under the patient’s chest, near the heart, and affects the heart rate by connecting directly with electrical leads. The electrical leads target the heart right atrium, right ventricle, or both. The pacemaker’s "pacing" activity is achieved by a current pulse which is sent from the pacemaker battery to each of the leads, enabling "electrical capture" (i.e., efficient pacing of the cardiac chamber paced by each lead). The current pulse amplitude and width are programmable with the aim of keeping them above a "pacing threshold" which is the minimum pulse amplitude and width needed to achieve capture. These values should be close to the threshold in order to minimize energy expenditure and maintain maximal pacemaker battery duration (see below). According to some exemplary embodiments, the IMD is an Implantable Cardioverter Defibrillator (ICDs). Implantable cardioverter defibrillators can also treat bradyarrhythmias using low energy electric pulses (like a pacemaker), but can also treat tachyarrhythmias, i.e., abnormally fast heart rhythms (which can be life-threatening), such as ventricular tachycardia (VT) or ventricular fibrillation (VF), using high energy shocks to terminate these arrhythmias. ICD is particularly effective for patients with a history of sudden cardiac arrest (SCA) resulting from previous tachyarrhythmias ("secondary prevention" of SCA) or patients with decreased left ventricular function who are at an increased risk of new tachyarrhythmias ("primary prevention" of SCA). Like pacemakers, ICDs are connected directly to the heart using 1-3 electrical leads.

As used herein, the term "Programmer", "Programmer device", "Programming device" and "programmer computer" are directed to an external device capable of communicating with the IMD to provide (transmit) operating instructions thereto and/or to receive information (data) from the IMD. The programmer device is part of the ecosystem that wraps the CIEDs and provides them with their functionality (e.g., configurations and modifications, event history log transferring, etc.). The programmer device (usually located in the clinic or hospital) allows health care providers (such as doctors and technicians) to comfortably and easily interact with the IMD (such as, CIED), remotely, for example, using wireless technology (including, wireless routes, such as, for example, WIFI- RF-ID, NFC, Bluetooth, and the like), and configure it as needed. According to some embodiments, the configurations, referred to herein as "programmings", are set according to the medical condition of the patient. In some embodiments, the programming may include one or more sets of commands/data transferred from the programmer device to the IMD relating to the operation of the IMD. In some embodiments, the programming may be "benign" or "legitimate" (i.e., authentic and correct commands), or may be "harmful"/"malicious"/"dangerous" commands. As used herein, the term "malicious programming" in this context is defined as any programming that is not aligned with the treatment that the health care provider intended to provide with the patient, or alternatively unwanted treatment that can cause the patients any inconvenience or endanger their health.

As used herein, the term "training data set" relates to a collection of programmings that may include benign programming and/or malicious programming, which serve as a basis/data set for comparison between attributes/features (values) of parameters sent in programming and their corresponding values in the data set. In some embodiments, such training data set can be collected from programmers that are being used in clinics (such as, for example, heart clinics), hospitals, and/or in some instances may be crafted in a research lab by experts.

In some embodiments, the terms "user", “doctor”, “physician”, “clinician”, “technician”,“medical personnel”, "health care provider" and“medical staff’, are used interchangeably throughout this disclosure and may refer to any person taking part in the medical procedure/programming.

As used herein, the terms“subject” and“patient” may be used interchangeably, and they may refer either to a human subject or to an animal subject.

As used herein, the term "detection system" relates to a device and/or method for the detection of malicious programming conveyed/transferred/communicated from a programmer device and an implantable device (IMD). In some exemplary embodiments, the detection system may be termed CardiWall, for example, when the IMD is a CIED, such as, a CID or a pacemaker. In some exemplary embodiments, the detection system may be termed "ImplantiWall".

According to some embodiments, there are provided systems, devices and methods for the detection and prevention of attacks on IMD, such as, ICDs, that are initiated from a compromised programmer device or by an unintentional programming mistake. In some embodiments, the systems, devices and methods are implemented in the form of a firewall utilizing a detection model for the detection and prevention of attacks on IMD from a legitimate programmer.

According to some embodiments, there is provided a designated firewall which may be implemented as a device or an independently executed software (on a suitable processor) for the detection and prevention of attacks on IMDs, such as, ICDs, that are initiated from a compromised programmer device or by an unintentional programming mistake. The detection and prevention firewall utilizes/based on a suitable anomaly identification/detection model, as detailed herein. According to some embodiments, the detection and prevention firewall may be embodied as an external and trusted device, which can act as an active intermediary between the programmer device and the IMD; thus, it can allow the use of existing medical devices without the need to change their hardware or software. In some embodiments, as detailed herein, the external device may include one or more processors configured to execute a method for detection or prevention of such attacks (based on a suitable anomaly model). According to some embodiments, the detection and prevention firewall may be implemented as a software, configured to be executed by the IMD, for example by a processor of the IMD.

According to some embodiments, the programmer device may be compromised by, for example, getting a malicious firmware-update or any other malicious intervention, from one of its external interfaces/connections, such as, for example, a flash drive, a USB connection/interface, printer, mouse, keyboard, cloud, internet, internal network, and the like. Such malicious intervention may eventually make it send malicious commands to the IMD, with which it interacts. To this aim, the disclosed detection and prevention firewall is intended to protect the IMD against such scenarios, by being a man-in-the- middle between the programmer device and the IMD.

According to some embodiments, the firewall disclosed herein is a trusted solution for the several reasons: 1) it does not have any physical access points that can be utilized as an initial penetration vector for launching an attack (such as USB port or Bluetooth which can be used for his purpose); in contrast, the programmer does have vulnerable physical access points. 2) the firewall is least as secure as the IMD itself, since it uses the same encrypted and designated wireless protocols as the IMD; therefore, in cases in which such a protocol is broken, the attacker can attack the IMD itself, rather than the protecting firewall.

In some embodiments, the disclosed firewall can act as a mediator between the programmer device and the patient's IMD and can further provide a warning/alarm to the health care provider regarding malicious and dangerous commands (even by accidental programming mistake) that are being sent from the programmer device to the patient’s ICD.

According to some embodiments, as further exemplified herein below, the firewall is based on/utilizes an anomaly detection model. In some embodiments, the construction of the anomaly detection model (i.e., the model configured to identify malicious programming) is based on collection of benign programming data only. In some embodiments, the construction of the anomaly detection model (i.e., the model configured to identify malicious programming) is based on collection of benign programming and/or malicious commands. In some embodiments, the anomaly detection model may be used to detect malicious and abnormal programmings which might be initiated via a malware resident on the programmer.

According to some embodiments, the anomaly detection model is a machine learning model, which learns only from the benign programmings and is aimed at inferring whether a new programming belongs to the class of the benign programmings or malicious programmings. In some embodiments, advantageously, the disclosed model can learn from only one type of programming class (instead of two classes: benign and malicious) for several reasons: 1. Most of the malicious programmings represent a different type of maliciousness. Therefore, learning from one malicious programming would not necessarily help the model understand that another malicious program is also malicious; 2. All malicious programmings generally contain very few parameters that make them malicious (as exemplified herein below (see Table 1 listing exemplary malicious commands with respect of ICD). In other words, most of the values of the parameters in these programmings are benign. If the model were to consider these programmings as malicious, it might learn that certain values are correlated with maliciousness, although they are completely benign; a model that learns only from benign programmings is more suited to the current reality, since until now, a relatively low number of real malicious programmings have been identified.

In some embodiments, for the construction/training of the detection model, data collection may be performed. The data collection may include benign data collection and/or malicious data collection that may be used as a training data set for the model.

In some embodiments, the data collection includes benign data collection. To this aim, data from configuration files which represent a patient’s clinic visit and contains a list of the parameters of the patient’s IMD, both when the patient arrived at the clinic and at the end of the visit (i.e., the previous programmings and the new programming). This means that each configuration file contains the changes to the parameters of the patient’s IMD made during the visit. In this domain, each configuration is referred to as a programming. In some embodiments, the configuration files may be anonymous and do not contain any personal information about the patient, that might be used or lead to her identity, thereby preserving the patients’ privacy. Each programming may include any number of parameters (for example, 150-500 parameters, 200-300 parameters, about 260 parameters, etc.) that are responsible for the treatment modality of the IMD. Note that all of the programmings thus collected from the programmer devices are considered legitimate or benign; as it is assumed that the programmer devices from which the data was collected had not been the target of an attack and thus are free from malicious or dangerous programmings.

In some embodiments, the data collection includes malicious data collection. After collecting a sufficient number of benign programmings, a malicious programmings based on the knowledge of medical experts may be created. The malicious programmings may be created by the experts, for example, by altering one or more parameters in a benign programming in such a way as to make the programming harmful to the patient. At least part of the malicious programmings thus created may be such which can endanger the life of a patient if sent to the patient’s IMD.

According to some embodiments, the detection model of the firewall may include one or more detection layers and methods, whereby each of the layers is aimed at the detection of a malicious programmings in a different manner. According to some embodiments, the detection model of the firewall may include a plurality of detection layers and methods. For example, the detection model may include at least two layers of detection. For example, the detection model may include 2-6 layers of detection. In some embodiments, one or more or all of the layers may be associated with analyzing the configured parameters and/or values associated therewith, that are included in the programmings.

In some embodiments, when the firewall receives a new programming, which is sent from the programmer device to the IMD, the programming passes through one or more or all of the detection layers. If one of the layers considers the command to be malicious, an alert may be issued, notifying the health care provider about the command and requesting the user to indicate whether or not to send the command on to the IMD. If the health care provider approves the command, it will be delivered to the IMD; if not, the firewall may block the command, and the IMD will not receive it, i.e., the command will be prevented from the IMD.

According to some embodiments, the detection layers of the detection model may include one or more of the following layers:

Layer I: Deterministic Rules - This layer contains a set of rules that are defined by medical experts (for example, cardiologists, neurologists, and the like) who know the parameters that are configured in each programming and are aware of the dangers that can result from certain values for certain parameters and/or data collected from "real- world". The scope of this layer may be limited or exhaustive and may increase in size, as the model is used and updated. In some embodiments, this layer is more comprehensive as it is being used. In some embodiments, every malicious programming in the dataset can be translated to at least one deterministic rule in real-life systems. In some embodiments, even if a human expert does not provide a deterministic rule, other layers, which are based on the data of many programmings, may be able to detect malicious programmings. In some embodiments, the data set may be continuously updated, as new and validated cases of malicious programming are encountered. In some embodiments, an expert may confirm if a suggested programing is malicious. An exemplary list of parameters, related to ICD is provided in Table 4 below.

Layer II: Parameters’ Values Outliers - For each parameter that exists in the programmings in the benign data collection (training set), the model (system) learns the distribution of each value. We define val (prog, param) as the value of the parameter param in the programming prog, and programmings as the set of all of the programmings in the dataset. The probability of obtaining the value val for each parameter param in the training set is defined in Equation 1 :

When the system tests a new programming, it will go through all of the programming’s parameters and check whether it contains a parameter par am with a value val, for which the probability of appearance P value is lower than a threshold value. The threshold value is set based on a predefined baseline threshold value (as detailed below). In cases in which there is such a parameter value, the system may issue an alert flagging this programming as malicious or anomalous. The system may also flag the parameter and its value, so the health care provider can see why the system has issued an alert.

Layer III: Parameters Change Outlier - In this layer, the model learns the number of times each specific parameter’ s value was changed during a visit for each parameter in the programmings in the benign data collected, over all the patients and visits; based on that, the system calculates the probability of a change of each of the parameters. We define val before (prog, param) and val after (prog, param) as the value of the parameter param in the programming prog before and after the visit, respectively. The probability of a parameter param to be changed is defined in Equation 2.

The system may issue an alert about a malicious programming if there are one or more values that have changed, and the P change of one of these parameters is less than a specific threshold. The system may also flag the parameter and its value, so the health care provider can see why the system has issued an alert. The threshold value is set based on a predefined baseline threshold value (as detailed below). In some embodiments, some parameters are frequently changed, and other parameters values are rarely changed.

Laver IV : Amount of Change - Similarly to layer III, this layer also learns from the changes that have been made to the values of the parameters in the programmings from the benign data collection. However, in this layer the model looks at the probability of the change of a parameter’s value by a specific delta amount. For example, the model may check the probability of changing the basic rate parameter by any possible delta value. We define the probability of having the change of delta del for parameter param in Equation 3. When the model tests a new programming which changes a parameter param with delta del, and value P delta (param, del) is less than a threshold value, the model may issue an alert flagging this programming as malicious or anomalous. The model may also flag the parameter and its value, so the health care provider can see why the system has issued an alert. The threshold value is set based on a predefined baseline threshold value (detailed below).

Layer V: Combinations of Parameters - In this layer, the model learns the probability of each combination of values of every pair of two parameters in the programmings from the benign data collected. We define the probability of obtaining the combination of the value val 1 in parameter par par 1 and the value val 2 val 2 in parameter par 2 as P comb ination ( par 1 , val 1 par 2 , val 2 ) in Equation 4. When the model tests a new programming, it may check all of the combinations of every pair of parameters’ values in order to find a combination with the probability P combination that is less than a threshold value. The threshold value is set based on a predefined baseline threshold value (as detailed below). If the model finds such a combination, it may issue an alert flagging this programming as malicious. The system may also flag the parameter and its value, so the health care provider can see why the system has issued an alert. For n parameters and m values for each parameter

combinations may be obtained. In some embodiments, in this layer, a combinations of pairs, rather than triplets or fourths may be used, since the more parameters included in the combinations, the lower the probability of getting such a combination; this is our aim, since such combinations may result in a lot of false alarms.

Layer VI: Machine Learning Algorithm for Anomaly Detection - In this layer a machine learning algorithm, such as, OneClass SVM machine learning algorithm which suits the anomaly detection problem, may be used in order to detect anomalies in new programmings. The machine learning approach is used to improve the tasks’ process. Machine learning methods have demonstrated the ability to find relations between the various features that represent data (i.e., the configuration parameters of the IMDs), and this ability, which cannot be applied by simple statistics or a human expert, might improve the accuracy of the model results. To this aim, the informative features from the data is extracted and leveraged using appropriate machine learning algorithm(s). In some exemplary embodiments, an SVM algorithm using the parameters’ values as the features is used. For numeric parameters, the original value may be used. For non-numeric parameters, the values may be normalized and used as numeric values. A support vector machine (SVM) is an example of a classifier that can be employed in the devices and methods disclosed herein. Other directed and undirected model classification approaches may include, for example, naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, k-nearest neighbors, random forest, probabilistic classification models, statistical regression that is utilized to develop models of priority, and the like, or any other suitable anomaly detection based model.

Reference is now made to Fig. 1, which illustrates programming routing between a programmer device and an IMD, in the absence and presence of the detection and prevention firewall, according to some embodiments. As shown in Fig. 1, a programmer 6 is configured to directly send information/data/programming (represented by arrows 4) to an IMD 2 (exemplary illustrated in Fig. 1 as an ICD). However, when the disclosed firewall 8 (which may be implemented as an external device or as software configured to be executed on a processor of the IMD and/or the programmer, as detailed herein) is present, the communication route between the IMD 2 and the Programmer 6 is indirect, i.e., it passes via the firewall 8. As shown in Fig. 1, the communication between the IMD and the Programmer is facilitated via firewall 8, which acts as a "man in the middle", whereby programming/instructions/data 10A from the programmer device 6 are first processed by firewall 8, (i.e., by an anomaly detection model) before being conveyed (if determined to be appropriate), to the IMD 2, via route 10B As further shown in Fig. 1, the anomaly detection model 12 may include one or more detection layers. In the example shown in Fig. 1, the model includes 6 layers, 14A-F, corresponding to layers I- VI, detailed above. Namely, Layer 14A corresponds to the deterministic rules layer (Layer I); Layer 14B corresponds to the parameter's values layer (Layer II); Layer 14C corresponds to the very change layer (Layer III); Layer 14D corresponds to the Delta of change layer (Layer IV); Layer 14E corresponds to the combination of parameters layer (Layer V); and Layer 14F corresponds to the machine learning algorithm layer (Layer VI). In some embodiments, once an anomaly is detected (i.e. a malicious programming is identified) at one or more layers, an indication to a health care provider may be provided. The indication may be in the form of an alert/alarm and may be visual, audible, tactile, or the like. In some embodiments, the alert may be presented to the health care provider (for example, on a monitor). In some embodiments, the alert may include an indication as to the suspected malicious programming identified. In some embodiments, the health care provider may confirm or ignore the alert. In some embodiments, in response to the alert the model and/or the health care provider may decide if to prevent or allow the identified programming from reaching the IMD.

According to some embodiments, various threshold calculation may be performed in order to determine various threshold utilized by the detection model, at one or more of the detection layers. According to some embodiments, as detailed above, one or more of layers II-V may use threshold values that are based on a predefined baseline value, ("baseline threshold"). Each threshold of each layer may be defined by a different baseline threshold. The health care provider (Such as physician or technician) may define the value of the baseline threshold during the calibration of the firewall. The baseline threshold’s values may be between zero and one (or any other desired scale), depending on the desired sensitivity level. After defining the value of the baseline threshold, the system may choose its actual threshold value, using, for example, one of the following methods: constant threshold and data-driven threshold, as detailed below. In some embodiments, the method used for threshold determination may be determined during the calibration. In some embodiments, when a threshold value of x is defined, the anomalous values are below x; this means that all of the values y such that y £ x are anomalies, and the values above x are legitimate.

According to some embodiments, a constant threshold may be utilized. With this method, the system may choose its actual threshold value using a constant constraint- driven threshold. The actual threshold value that is selected is equal to the baseline threshold defined in the calibration stage. For example, if a baseline threshold with the value of 0.1 is used in layer II, the actual threshold value of each of the parameters will also be 0.1. This method allows the physician or technician define the minimal acceptable probability of a value/delta/combination’s appearance. For instance, in the previous example, one may choose to accept values for which the probability of appearance is at least 0.1.

According to some embodiments, a Data-Driven Threshold may be used. With this method, the system may choose its actual threshold value using a constant constraint- driven threshold which may be modified by the data distribution. The actual threshold’s value may be chosen dynamically based on the benign data collected and the baseline threshold defined in the calibration stage. With this method, the actual threshold may be chosen in such a way that the baseline threshold represents the probability mass of the presence of a parameter or a pair of parameters. For example, if the baseline threshold is defined as 0.1, then approximately 10% of the programmings will be marked as anomalies. The lower the baseline threshold is, the lower the detection rate (sensitivity).

According to some embodiments, in the model learning process, each of the layers II-V can automatically choose the best threshold value for each parameter (or pair of parameters) based on the baseline threshold value predefined for each layer. In order to do that, a probability histogram for each parameter (or pair of parameters, according to the layer), may be created and sorted in ascending order. The threshold may be set to the maximum value, such that the sum of the probabilities until and including that value is less than or equal to the baseline threshold value. At this stage, various different cases may be considered. In the first and easiest case, the values’ probabilities are different from one another. The second case is when there is more than one value with the same probability; in this case, the method may group the values together, assigning them the probability that equals the sum of their probabilities. The third case is when there is no such threshold value with the sum of the probabilities until and including that value which is less than or equal to the baseline threshold value; in this case, the threshold value may be set at zero, because a threshold that will satisfy the probability mass of the baseline threshold cannot be chosen. The following describe, explain, and demonstrate the three cases of the data-driven threshold selection process. In each of the cases below the values (with vali ) and their probabilities (with P(val i )) are defined according to the layer the system is dealing with. For example, if the system is dealing with layer II, the values will be the parameter’s values, and the probability will be the probability of appearance of those values ( P value )· If the system is dealing with layer III, the values will be the parameters, and the probabilities will be the probability of change of those parameters

(P change )

Case I: Strictly Increasing Probability - Given that P(val 1 ) = 0.02, P(val 2 ) = 0.07 , P(val 3 ) = 0.3, P(val 4 ) = 0.6, let baseline threshold = 0.1. In this case, there is no need for grouping, because there are not two or more values with the same probability. The data-driven threshold selected will be 0.07, since the sum of all of the probabilities up to this threshold (0.02+0.07) is less than the baseline threshold (0.1). This case is presented in Fig. 2A, whereby the solid line indicates the baseline threshold, and the dotted line indicates the data-driven threshold selected.

Case II: Monotonically increasing probability- selected threshold is greater than zero. Given that P(val 1 ) = 0.05 , P(val 2 ) = 0.05, P(val 3 ) = 0.9, let Baseline threshold = 0.1. In this case, a grouping of val ± and val 2 is needed, because they have the same probability. The selected threshold value will be 0.05, because P(valf) + P(val 2 ) + P(val 3 ) = 0.15 > Baseline threshold = 0.1 and P(val 1 ) = P(val 2 ). Error! Reference source not found.2B presents this case before the grouping is done, and Fig. 2C presents this case after the grouping and threshold selection have taken place. The solid line indicates the baseline threshold, and the dotted line indicates the data- driven threshold selected.

Case III: Monotonically increasing probability- selected threshold equals zero - Given that P(val 1 ) = 0.05, P(val 2 ) = 0.05, P(val 3 ) = 0.05, P(val 4 ) = 0.85, let Baseline threshold=0.1. In this case, a grouping of val x , val 2 , and val 3 is needed. After the grouping has been made, no threshold value for which the sum of the probabilities up to it is less than or equal to baseline threshold (0.1) is found. In this case the lower probability is 0.05, and P(val 1 ) + P(val 2 ) + P(val 3 ) = 0.15 > Baseline threshold = 0.1. Thus, in this case the threshold will be set to zero. Fig. 2D presents this case before the grouping is done, and Fig. 2E represents this case after the grouping and threshold selection have taken place. The solid line indicates the baseline threshold, and the dotted line marks the dynamic threshold selected.

According to some embodiments, in order to evaluate the effectiveness of the detection model, the true positive rate (TPR), false positive rate (FPR), and area under the Receiver Operating Characteristic (ROC) curve, or the AUC may be calculated. In some embodiments, the accuracy measure is not used, because it is biased when the dataset is imbalanced. In some embodiments, a new measure may be defined, referred to as the True-False measure (TF or TF b), which is the weighted mean of the TPR and the FPR. An F-measure (presented in Equation 5) is used, as well as the IDR measure presented by [17] in order to develop the TF measure presented in Equation 6. A high value of the b parameter means more significance for low FPR, and a low value of the b parameter means more significance for high TPR. The TF measure with b= 1 is the harmonic mean of the sensitivity and specificity (or the HMSS). In some embodiments, the TF measure with b= 1 is used in order to determine the threshold (a value between zero and one) that optimizes the model detection results. The b parameter can be changed easily in a case in which the physicians are more interested in a low FPR or a high TPR.

According to some embodiments, the detection devices and methods disclosed herein can provide a high true positive rate (TPR), a low false positive rate (FPR) and a high Area under the curve (AUC). For example, the TPR may be over about 0.8, over about 0.9, over about 0.91, over about 0.92, over about 0.95, or over about 0.97. For example, the FPR may be lower than about 0.1, lower than about 0.07, lower than about 0.05, lower than about 0.04, lower than about 0.02, lower than about 0.015, lower than about 0.012, lower than about 0.01, or lower than about 0.005. For example, the AUC may be over about 0.8, over about 0.9, over about 0.91 , over about 0.92, over about 0.95, or over about 0.97.

According to some embodiments, the detection devices and methods disclosed herein can provide a high true positive rate (TPR), a low false positive rate (FPR) and a high Area under the curve (AUC). For example, the TPR may be over about 80%, over about 85%, over about 90%, over about 91%, over about 92%, over about 95%, or over about 97%. For example, the FPR may be lower than about 10%, lower than about 5%, lower than about 2%, lower than about 1.2%, lower than about 1.15%, lower than about 1.1%, lower than about 1%, lower than about 0.9% or lower than about 0.5%. For example, the AUC may be over about 80%, over about 90%, over about 91%, over about 92%, over about 95%, or over about 97%.

In some exemplary embodiments, as exemplified below, the use of the detection system with ICD, with the best configuration (according to the TF measure) that provided the best harmonic mean of sensitivity and specificity (HMSS) and the data-driven threshold method provided the following results: TPR = 0.914, FPR =0.011, and AUC = 0.947.

In some embodiments, with the configuration that provided the best harmonic mean of sensitivity and specificity (HMSS), the detection system achieved a high true positive rate (TPR) of about 91.4% and a very low false positive rate (FPR) of about 1%, with an AUC of about 94.7%.

According to some embodiments, the detection device and methods disclosed herein can analyze each new programming quickly and detection may take only a few milliseconds. This short timeframe makes such detection system suitable in emergency scenarios when the response time is critical.

According to some embodiments, there is provided a detection and prevention system aimed at the detection of malicious clinical commands (programmings) sent from a programmer device to implantable medical devices, such as, cardioverter defibrillators. The detection system includes six different layers. Layers I-IV are aimed at the detection of several kinds of anomalies, i.e., anomaly values of parameters, anomaly combinations of values of parameters and anomalies in the change of the values of parameters. Layer VI uses classic machine learning algorithm, such as, OneClass SVM, for anomaly detection, in order to improve the performance of the system.

The embodiments described above may be implemented by a variety of different hardware and/or software configurations. Reference is now made to Fig. 3, which is a schematic illustration of an exemplary firewall device, according to some embodiments. As shown in Fig. 3, firewall device 50, include a processing unit 52 including one or more processors, configured to execute the model for detection of malicious programming between a programmer device and an implanted medical device. In some embodiments, the instructions may be stored in volatile or non -volatile memory within the processing unit. The processing unit may include any type of one or more processors, whereby the processors may be or be part of a control circuitry (such as, a microcontroller, a microprocessors, FPGA, programmable logic device, digital signal processors (DSP), and the like, or combinations thereof), that may allow control over the operations (including communications with the programmer and/or the IMD) of the device. Device 50 may further include a communication unit 54, configured to allow communication with the IMD and/or with the programmer. The communication unit may include wired or wireless communication routes. For example, device 50 may connected by wireless means to the IMD. For example, device 50 may be connected to the programmer device via a communication cable or via wireless routes. The wireless communication routes may include any type of wireless communication, including, for example, radio frequency (RF), near field communication (NFC), Wi-Fi, Bluetooth, and the like. The communication routes may be secured/encrypted. In some embodiments, the communication unit may include receiver and/or transceiver. In some embodiments, the communication between the programmer device and the IMD is indirect and may be mediated by communication unit 54 of device 50. The device may further optionally include a display or GUI unit 56, to allow a user to interact with the device. The device may further include an alarm unit 58, which may be part of the display/GUI unit, or an independent alarm unit. In addition, the device may further optionally include a power source (such as a battery or main supply unit). Each of the indicated units may operate in conjunction with each other, under the control of the processing unit.

In some embodiments, suitable processors that may be utilized include, for example, a general-purpose processor, a special purpose processor, a digital signal processor (DSP), a plurality of microprocessors, programmable logic device, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an integrated circuit, Field-Programmable Gate Arrays (FPGA), Application-Specific Integrated Circuits (ASIC), and the like, or any combination thereof. Each possibility is a separate embodiment.

Reference is now made to Fig. 4, which shows a flow chart of a general method for detection of malicious programming, according to some embodiments. As detailed herein, the detection method implements a detection model configured to detect/identify malicious programming between a programmer device and an implanted medical device (IMD), wherein method includes the use of one or several detection layers, each aimed at detecting/identifying/learning various parameters, to eventually lead to a reliable output as to whether a programming (command) issued by a trusted/legitimated programming device is benign or malicious. As further detailed herein, the method may be implemented in any suitable hardware/software form. As shown in Fig. 4, the method 100 includes at step 102, the receipt/detection of new programming (i.e., a programming sent from the programmer device to an IMD). Next, at step 104, the received programming is analyzed (for example, by a processor), utilizing one or more of the detection layers (such as detection layers I- VI, detailed above), to determine if a command is malicious. At this stage, if the command is found to be benign - the command may reach the IMD (step 108). Optionally, an indication as to the command (properties, why it was determined to be benign, etc.) may be issued to a user (step 110). Alternatively, if the command is found/suspected to be malicious in step 104, as determined by one or more of the detection layers, an alert may be issued to a user at step 112. Optionally, at step 114, the malicious command may be prevented from reaching the IMD, if instructed by a user. Alternatively, the malicious command may be prevented from reaching the IMD automatically, depending on the type and nature of the command and/or anyma other operating settings.

According to some embodiments, when malicious programmings are identified, the firewall device and method disclosed herein can respond passively or actively, manually or automatically, depending, inter alia, on the configuration for the type of malicious programming and consequences related thereto. For example, if a malicious programming is identified which poses an immediate life threatening condition, the firewall may automatically hold/prevent the programming from reaching the IMD. The automatic prevention may be alerted/conveyed to the user. For example, if a malicious programming is identified which does not necessarily poses an immediate life threatening condition, the firewall may alert the user regarding the identified malicious programming and await the user to indicated whether to proceed or hold conveying the detected programming to the IMD.

In some embodiments, the processor executing the method of detection may keep a record of the historical data and commands. Such information may be stored in a memory. When new programming arrives, the firewall may compare them to the historical/training record/data to decide whether the new programing is malicious and optionally the degree of damage that may be afflicted by the programming.

In some embodiments, as disclosed herein, the detection system learns from benign programmings. Thus, a one -class learning is performed, which is also referred to as anomaly detection. In some embodiments, supervised learning, which is based on both benign and malicious samples, may be used, if malicious programing is included in the learning data set.

In some embodiments, the system may use a global model of an entire population of patients. A collection with additional benign programmings may allow to create a separate model for each patient; each model should be more accurate in detecting anomalies for the specific patient, and thus improve the detection results of the system for all of the patients.

In some embodiments, the system’s training is configured to be enriched and updated with additional benign programmings from many different patients and actual malicious programmings from the real world.

In some embodiments, actual malicious programming data that is being collected, can be used to enlarge the set of deterministic rules used in layer I, in addition to or instead or the malicious programmings that was created, as described herein.

According to some embodiments, there is provided a non-transitory computer readable medium comprising computer executable instructions that, in response to execution, cause at least one processor to perform operations, which may include: detecting a programming sent from a programmer device to an IMD; and analyzing the programming by utilizing one or more detection layers, to determine if one or more parameter attributes (features) of the programming deviate from values of corresponding parameters in a training data set, wherein a deviation between one or more attributes of the parameters of the programming and values of parameters in the training data set is indicative of the programming being malicious.

It should be understood that the firewall and detection model, devices and methods disclosed herein may be used with a wide variety of implanted medical devices, including, for example, pacemaker, implanted cardioverter defibrillator (ICD), insertable Cardiac Monitor (ICM), neurostimulators, Chronic Pain neurostimulator, Cluster headache neurostimulator, upper airway stimulator, phrenic nerve stimulator, and the like.

The following is a description of an exemplary implementation of the detection model for an implantable cardioverter defibrillator (ICD), which may be used with suitable modifications/adjustments, if needed, with other implantable devices, including other CIEDs, such as a pacemaker, or other implantable medical devices.

I. Methods

A. Data Collection

1) Benign Data Collection

A retrospective study using benign ICD configuration files collected from CIED programmers of Barzilai University Medical center was performed. This study was approved by the Institutional Review Board of Barzilai University Medical Center. Each configuration file represents a patient’ s clinic visit and contains a list of the parameters of the patient’s ICD, both when the patient arrived at the clinic and at the end of the visit (i.e., the previous programmings and the new programming). This means that each configuration file contains the changes to the parameters of the patient’s ICD made during the visit. In this domain, each configuration is referred to as a programming. The configuration files are completely anonymous and do not contain any personal information about the patient, that might be used or lead to her identity, thus the study was conducted while preserving the patients’ privacy.

Overall, the data collection contains 803 programmings from hundreds of different patients, gathered over a period of four years. Each programming contains 260 parameters that are responsible for the treatment modality of the ICD device.

Note that all of the programmings collected from the programmer devices are considered legitimate or benign; It is assumed that the programmer devices from which the data was collected had not been the target of an attack and thus are free from malicious or dangerous programmings. This assumption relies on the fact that none of the patients or their physicians reported any malfunctioning or anomalous behavior of the ICD device when the patient used the device.

2) Malicious Data Collection After collecting a sufficient number of benign programmings, two independent cardiology experts were consulted, in order to create malicious programmings based on their knowledge. The cardiology experts created malicious programmings by altering one or more parameters in a benign programming in such a way as to make the programming harmful to the patient. The experts created 28 different malicious programmings, each of which can endanger the life of a patient if sent to the patient’s ICD. The programmings were applied on an ICD that was used exclusively for this research and was not implanted in the body of any patient; thus, there was no actual harm caused by the malicious programmings applied on the ICD.

Table 1 presents the malicious programmings, including their parameters and their values. The table also contains the possible outcomes to the patient or the ICD which may result from the malicious programming and mentions when such a possible outcome will occur: at rest (R), during activity (A), or at both times (B). The last column provides a description of the potential result of each malicious programming. An explanation of each of the parameters used in the malicious programmings is presented in Table 2. The material presented in both tables is based on the experts’ clinical knowledge and experience with ICDs. Note that, in Table 1 there are eight malicious programmings that can lead to patient’s death (denoted with (*) under their ID), while the others are still considered malicious programmings, as they might cause a the patient a damage in the long term or might lead to frequent replacement of the device (associated with frequent implantations to the patient) etc.

Table 1 - Malicious programmings

Table 2 -Parameters' descriptions

* All ICDs have a few tachycardia detection zones, which are primarily used for life-threatening ventricular arrhythmias called ventricular tachycardia (VT) and ventricular fibrillation (VF); the distinction between the two is usually based on their sensed ventricular rates. Usually, ICDs are programmed to detect VT or VF upon ventricular rates which are well above the normal/physiologic fast sinus rate during exercise or excitement (called "sinus tachycardia"), to prevent inappropriate ICD intervention at physiologic heart rates. Usually, there are few sensed rate zones for VT (aiming to distinguish between "regular" VT and fast VT), and in each zone there is a predetermined minimal number of beats (called "detection counter ") required for arrhythmia detection. For example, if the VT1 zone sensing rate is 150 and the detection counter is set at 10 beats, it means that any ventricular arrhythmia faster than 150 bpm and lasting more than 10 beats will be detected and defined as VT1. Once the arrhythmia gets faster and goes above the VT1 zone rate, it will "replace" its name with VT2. VF is usually arbitrarily detected at a ventricular sensed rate above 200 or 220 bpm.

Table 1 presents a summary of all the data that has been collected. It is noted that the dataset is extremely imbalanced: 96.5% of the programmings are benign, and only 3.5% are malicious; this may make the research more challenging and more difficult but more representative of reality (most of the programmings are benign, and only a few, if at all, are malicious).

Table 1: Programming data collected for a period of four years

Next, an anomaly detection model is induced, based on the collection of benign programming data only, in order to detect malicious and abnormal programmings which might be initiated via a malware resident on the programmer. In the example provided herein, the detection model includes six detection layers and methods; each layer is aimed at the detection of malicious programmings in a different way. All of the layers are associated with analyzing the configured parameters that are included in the programmings. When the model receives a new programming, which is sent from the programmer device to the CIED, the programming passes through these six layers. If one of the layers considers the command malicious, an alert is issued, notifying the doctor about the command and requesting that the doctor indicate whether or not to send the command on to the ICD. If the doctor approves the command, it will be delivered to the ICD; if not, it will be blocked, and the ICD will not receive it. The following layers are used:

Layer I: Deterministic Rules - This layer contains a set of rules that were written and defined by cardiology experts who know the parameters that are configured in each programming and are aware of the dangers that can result from certain values for certain parameters. Table 4 presents the deterministic rules that were provided by the experts and their explanations. As seen in the table, a few deterministic rules are found in the system. This was done in order to demonstrate that when a human expert does not provide a deterministic rule, the other layers in our system, which are based on the data of many programmings, may be able to detect malicious programmings. This layer is more comprehensive in real-life systems, and in fact, every malicious programming in the dataset can be translated to at least one deterministic rule in a real-life system.

Table 4- deterministic rules

Layer II: Parameters’ Values Outliers - For each parameter that exists in the programmings in the benign data collection (training set), the system learns the distribution of each value. The probability of obtaining the value "val" for each parameter "param" in the training set may be as defined in Equation 1 above. Error! Reference source not found presents an example of the probability histogram of the basic rate parameter according to the benign programmings collected. As can be seen, the most common basic rate is 60 bpm (for over 40% of the programmings), and the least common basic rate is 85 bpm (for around 1% of the programmings). Layer III: Parameters Change Outlier - In this layer, the system learns the number of times each specific parameter’ s value was changed during a visit for each parameter in the programmings in the benign data collected, over all the patients and visits; based on that, the system calculates the probability of a change of each of the parameters. Equation 2 (above) may be used to calculate the probability of a parameter to be changed. Fig. 6 presents the histogram of the probability of the changes that were made for each of the parameters in each of the benign programmings collected. The x-axis represents the enumerate identification of each parameter (260 parameters in total). The y-axis represents the probability of a change to each parameter based on the benign data collected. As shown, there are some parameters that are frequently changed and other parameters whose values are rarely changed.

Layer IV: Amount of Change - Like layer III, this layer also learns from the changes that have been made to the values of the parameters in the programmings from the benign data collection. However, in this layer the system looks at the probability of the change of a parameter’s value by a specific delta amount. For example, the system checks the probability of changing the basic rate parameter by any possible delta value. The probability of having the change of delta del for parameter param may be calculated based on Equation 3 above. An example of the delta probabilities of the basic rate parameter is presented in Table 2. It can be seen, for example, that P delta (basic rate, 20) = 0.022.

Table 2: Delta probability for basic rate parameter

Layer V: Combinations of Parameters - In this layer, the system learns the probability of each combination of values of every pair of two parameters in the programmings from the benign data collected. Equation 4 may be used for the calculations. An example of a combination probability matrix of the two parameters: basic rate and VT1 rate is presented in Table 3. It can be seen, for instance, that Pcombination (basic rate, 60, VT1 rate, 150) = 0.129 It can be seen that with those two parameters, most of the value combinations are rarely used, if at all. Thus, these rarely used combinations can be treated as anomalies. In this layer, the combinations of pairs were used, rather than triplets or fourths, due to the fact that the more parameters included in the combinations, the lower the probability of getting such a combination; this is the aim, since such combinations will result in a lot of false alarms.

Table 3: Example of combination probability matrix for two parameters

Layer VI: Machine Learning Algorithm for Anomaly Detection - In the final layer a OneClass S VM machine learning algorithm which suits the anomaly detection problem, is used in order to detect anomalies in new programmings. In the study, the SVM algorithm was trained using the parameters’ values as the features. For numeric parameters, the original value was used. For non-numeric parameters, the values were normalized and used as numeric values. Evaluation of the detection model in detecting anomalies (malicious programming): Eperiment 1 - Identifying the Best Threshold Configuration

In this experiment, it was sought to identify the best threshold configuration for each of the layers II-V, in order to obtain the best total results for the detection model (also referred to herein as ImplantiWall or detection firewall). Here, a configuration refers to the value of the baseline threshold for each layer, which will result in the highest value of the TF measure. The baseline threshold values that are assessed are all of the values between zero and one, with an increment of 0.01 (for a total of 100 baseline threshold values). This experiment is performed on layers II-V, because they are the only layers that require a threshold value. Although layer I does not require a threshold value, the results for layer I are included in this experiment in order to present the total detection results of the system. Layer VI (ML algorithm) is the focus of the second experiment. In this experiment, four different train-test mixture are used. This is done in order to examine the differences in the accuracy of the detection results obtained by hospitals that have a lot of data to learn from and hospitals that have less data to learn from. For each such mixture, a portion of the benign programmings are randomly selected for the training set, and the remaining benign programmings are part of the test set which also includes the 28 malicious programmings collected. Each train-test mixture experiment is repeated five times and the results are averaged, in order to overcome randomness and obtain the most precise true positive and false positive rates. The following train-test mixtures are used: 1) 60%-40%, 2) 70%-30%, 3) 80%-20%, and 4) 90%-10%.

The constant threshold is investigated in Experiment 1.1, and the data-driven threshold is investigated in Experiment 1.2. In each case, the configurations that provide the best TPR (TPR=1) and FPR (FPR=0) are also presented so that the results at the extremes can be seen.

Experiment 1.1 - Using Constant Threshold

In this sub-experiment, the constant threshold is used; the threshold value selected for each layer is defined as the baseline threshold. The system (model) is run with all of the possible combinations of threshold values for each of the layers II-V, in order to identify the threshold configuration that achieves the best total results.

Experiment 1.2 - Using Data-Driven Threshold

In this sub-experiment, the previous sub-experiment is performed with the data- driven threshold. This means that the actual threshold value for each feature (attribute) in each layer will be chosen automatically in the training stage based on selected rules.

Experiment 2 - Investigating the use of a Machine Learning (ML) Algorithm

In this experiment, the influence of layer VI, which is based on classic machine learning algorithm for anomaly detection is assessed. The aim was to determine whether this layer improves the results of the first five layers of the proposed system, which do not rely on ML algorithm. The Support Vector Machine ( SVM ) [ 18 ] classifier is used with only one class of learning (because an anomaly detection problem is being dealt); more specifically, the OneClass SVM classifier [19] is used with two different kernel functions: RBF and Polynomial. The machine learning algorithm is applied using the SciKit-Leam Python package for machine learning. 100 different thresholds were used, between zero and one, with increments of 0.01. The benign dataset is divided into training and testing sets with the following train- test mixture: 1) 60%-40%, 2) 70%-30%, 3) 80%-20%, and 4) 90%-10%. The 260 parameters of the programmings will be used as features for the machine learning algorithm. For each kernel function and mixture, the thresholds that optimize the results are identified based on the TF measure mentioned above.

Results:

1) Experiment 1

In each of the sub-experiments, each of the layers II- V were tested with 100 thresholds. For each threshold configuration, all of the metrics discussed above were calculated. In each sub-experiment, five CSV result files were created for each train-test mixture (with a total of 40 files). Each file contains 100,000,000 rows representing the total results (TPR, FPR, and TF) of the detection model for each threshold configuration of the first five layers. The size of each CSV file was around 30GB (with a total size of 1.2TB). For each sub-experiment and mixture, all five CSV files were combined for the four thresholds in order to calculate the average TPR, FPR, and TF for each threshold configuration; the results are presented below. During the evaluation process it was discovered that the use of two out of six layers provided enhanced results, meaning that four out of the six proposed layers do not necessarily contribute additional detection capabilities and thus are not used with the best configuration, using the tested collected dataset.

a) Experiment 1.1

T able 7 below presents a summary of the results of Experiment 1.1. The first three columns contain the mixture of the training and test sets and the percentage of malicious programmings in the test set. The three rows of the description column for each mixture contain the threshold of the best results for layers II-V based on the highest TF value (with b =1), the highest TPR, and the lowest FPR. The four baseline threshold columns contain the baseline threshold for layers II-V. The last four columns present the TPR, FPR, TF, and AUC. The AUC was calculated using the TPR and FPR points provided by plotting the ROC curve of layer II against 100 thresholds.

Table 7: Results of the constant threshold experiment

As can be seen from the results, the threshold configuration that provides the best result for the TF for each mixture is a combination that does not necessarily use layers III-V at all (threshold = 0), but uses layers I-II with a baseline threshold of 0.01 in layer II. This means that when using the constant threshold method, layers III-V are not necessarily of use and do not necessarily provide additional benefit to our system.

It can be seen that the TPR and AUC are almost the same for each of the mixture used, but the TF increases as the size of the training set grows. The FPR, on the other hand, decreases as the size of the training set grows. This indicates that the model becomes more accurate with the increasing number of samples included in the training set, which enable the system to extract more accurate knowledge from the data. b) Experiment 1.2

Experiment 1.1 was repeated, but this time the data-driven threshold method was used. Table 8 is organized similarly to Table 7 and presents a summary of the results from this experiment. The actual data-driven threshold is not presented in this table, because it is not part of the configuration of the system (instead it is learned during the training stage). Table 8 - Results of the data-driven threshold experiment

As can be seen from the results in T able 8, the combination of baseline thresholds that provides the best result for the TF in each mixture is the same as the results of the constant threshold experiment: only uses layers I-II with a baseline threshold of 0.01 in layer II and zero for layers III-V. That means that for the data-driven threshold methods, layers III-V do not necessarily contribute to the system’s performance. The TPR obtained in the current experiment is a bit worse than the TPR obtained in the constant threshold experiment, but the FPR, TF, and AUC are much better. Table 9 provides a summary and comparison of the two experiments above. In each row, the TPR, FPR, TF, and AUC of the threshold configuration that provided the best TF (with b =1) are presented. In both sub-experiments, the best TF was achieved by using the same threshold configuration (0.01 for layer II and 0.0 for the rest of the layers). It can be seen that the data-driven threshold method provided better results in terms of the AUC and TF. Table 9 - Constant vs. data-driven threshold

Error! Reference source not found, presents the results for each layer for Experiment 1.2, using the 70%-30% mixture. The first column indicates the layer (I-V). The second column presents the threshold of the best configuration; it can be seen that there is no threshold listed for layer I, because this layer only applies deterministic rules which are not associated with any threshold. The next four columns present the TP, TPR, FP, and FPR for each layer. The subsequent columns present the malicious programmings (referred to by their ID number from Error! Reference source not found.); bright background indicates that the current layer detected the programming as malicious (Hits = True Positives), and dark background indicates undetected malicious programming (Misses = True Negatives).

Table 10 - Results for the best configurations

As can be seen, using the 70%-30% mixture, layers I and II detected 17% and 85% of the malicious programmings respectively (on average for all of the mixtures, Layer II detected 86% of the malicious programmings). The firewall detection model did not detect #7, #11, and #23 malicious programmings: Programming #7 is considered malicious because of the change in the value of the ventricular pacing parameter from BiV to RV. Layer 1 did not detect this programming, because there is no deterministic rule existing for this parameter. Layer II did not detect this programming because the value RV alone is not dangerous or malicious. The layer that should have detected this programming is layer IV (delta of change). However, layer IV did not detect the malicious programming because of the threshold used for this layer, which was zero. The same explanation applies to programming #11. Programming #23 is considered malicious due to its low value for the RV amplitude parameter. In this case, the value of this parameter alone was not considered dangerous or malicious, so layers I and II did not detect it. The layer that should have detected this programming is layer V (combinations). However, layer V did not detect this malicious programming due to the threshold used for this layer, which was zero.

21 Experiment 2

Table 4 presents a summary of the results of Experiment 2 that focused on leveraging the programmings features (attributes) using ML algorithms. The first column contains the kernel function used (RBF or Polynomial). The next three columns contain the mixture of the training and test sets and the percentage of malicious programmings in the test set. The subsequent columns present the threshold for the model and the results of the experiment (the TPR, FPR, TF, and AUC).

The results for the best TPR are not presented because the results were TPR=1 and FPR= 1 for that case, which is not an interesting result. For the same reason, the best FPR is not presented in the table, because the results for that case were FPR=0 and TPR=0.

After running the OneClass SVM classifier on the test set, a level of certainty for each programming is obtained, whether it is benign or not. Therefore, in this experiment the threshold represents the minimal level of certainty required to consider a programming as benign. Thus, if the threshold is zero, none of the programmings will be considered benign, and TPR = FPR = 1 is obtained. In the opposite case, in which the minimum certainty level is one, all of the programmings will be considered benign, and TPR = FPR = 0 is obtained. Table 4: Summary of the results of Experiment 2

As can be seen, layer VI does not necessarily contribute to the efficient detection of malicious programmings. In some instances, it may raise more false positives and fewer true positives than the first five layers combined. The AUC is worse than the one obtained by using only layers I-V. With the Polynomial kernel function, the ratio of the TPR and the FPR is almost 1. With the RBF kernel function, the results are better but are still not as strong as the results obtained in Experiment 1.

In order to show that in the exemplified use case it is best to use a one -class classifier, an experiment was performed with 125 benign programmings and eight malicious programmings in the training set, and 30 benign programmings and 20 malicious programmings in the test set. Several classifiers, including SVM, k-nearest neighbors, decision tree, random forest, were used. All of them provided an AUC of less than 0.7.

Thus, as detailed above, the detection system was tested extensively in two comprehensive experiments. For the evaluation, a collection of 803 programming samples were used. Each programming contained 260 parameters that configured the patient’s ICD. The benign programmings, 775 in total, of hundreds of different patients were obtained from different programmer devices. The malicious programmings, 28 in total, were created by two cardiology experts, and were intended to harm the patients in different ways, in case they will be delivered to their ICDs. The dataset is extremely imbalanced (only 3.5% of the programmings are malicious), which makes it more challenging to achieve a high TPR and low FPR, as there are less malicious instances to learn from. The purpose of the first experiment was to find the best threshold for each of the layers II-V, in order to achieve the best total value of the TF for the system, with the best TPR and FPR possible. The experiment was divided into two sub-experiments, one for each method of threshold selection: constant and data-driven.

In the first sub-experiment, which uses the constant threshold method, TPR = 0.928 was achieved for each of the train-test mixtures used. However, the best FPR achieved was between 0.162 and 0.201 for the 90%-10% and 60%-40% mixtures respectively. All of the AUC values obtained in this sub-experiment were around 0.9, which represents a fine result for a simple constant threshold method.

In the second sub-experiment, which uses the data-driven threshold method, better results were obtained. The FPR decreased from 0.201 to 0.084 (58% improvement) for the 60%-40% mixture and from 0.162 to 0.101 (37% improvement) for the 90%-10% mixture. The AUC increased from around 0.9 to around 0.95 (5.5% improved) for all of the mixtures.

In both sub-experiments, it was found that the best threshold configuration sets the threshold to zero for layers III-V. The layer configuration that results with the best TF is the one with the thresholds 0.1, 0, 0, and 0 for layers II, III, IV, and V, respectively.

The second experiment was designed to assess the influence of a classic machine learning algorithm for anomaly detection on our novel system. A classifier, the One Class SVM classifier, was used with two different kernel functions: RBF and Polynomial. The results of this experiment show that the RBF kernel performs better than the Polynomial, however the OneClass SVM classifier preformed mediocrity with both kernels. With the RBF kernel function, a total TPR between 0.64 and 0.79 was obtained for the 60%-40% and 90%-10% mixtures, respectively, an FPR of around 0.25, and an AUC of 0.7, which is worse than the AUC obtained in the previous experiments with the other non -ML based layers. Thus, in some instances, VI layer which is based on OneClass SVM does not necessarily contribute to the detection capabilities.

To summarize the results presented herein with respect of ICD, the use of the detection model system with the best configuration (according to the TF measure) and the data-driven threshold method provided the following results: TPR = 0.914, FPR = 0.011, and AUC = 0.947. In the description and claims of the application, the words“include” and“have”, and forms thereof, are not limited to members in a list with which the words may be associated.

As used herein, the term“about” may be used to specify a value of a quantity or parameter (e.g. the length of an element) to within a continuous range of values in the neighborhood of (and including) a given (stated) value. According to some embodiments, “about” may specify the value of a parameter to be between 80 % and 120 % of the given value. For example, the statement“the length of the element is equal to about 1 m” is equivalent to the statement“the length of the element is between 0.8 m and 1.2 m”. According to some embodiments,“about” may specify the value of a parameter to be between 90 % and 110 % of the given value. According to some embodiments,“about” may specify the value of a parameter to be between 95 % and 105 % of the given value.

As used herein, according to some embodiments, the terms“substantially” and “about” may be interchangeable.

Unless specifically stated otherwise, as apparent from the disclosure, it is appreciated that, according to some embodiments, terms such as “processing”, “computing”,“calculating”,“determining”,“estim ating”,“assessing”,“gauging” or the like, may refer to the action and/or processes of a computer or computing system, or similar electronic computing device, that manipulate and/or transform data, represented as physical (e.g. electronic) quantities within the computing system’s registers and/or memories, into other data similarly represented as physical quantities within the computing system’s memories, registers or other such information storage, transmission or display devices.

Embodiments of the present disclosure may include apparatuses for performing the operations herein. The apparatuses may be specially constructed for the desired purposes or may include a general-purpose computer(s) selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), electrically programmable read-only memories (EPROMs), electrically erasable and programmable read only memories (EEPROMs), magnetic or optical cards, or any other type of media suitable for storing electronic instructions, and capable of being coupled to a computer system bus.

The processes and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct a more specialized apparatus to perform the desired method(s). The desired structure(s) for a variety of these systems appear from the description below. In addition, embodiments of the present disclosure are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present disclosure as described herein.

Aspects of the disclosure may be described in the general context of computer- executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and so forth, which perform particular tasks or implement particular abstract data types. Disclosed embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In the description and claims of the application, the words“include” and“have”, and forms thereof, are not limited to members in a list with which the words may be associated.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. In case of conflict, the patent specification, including definitions, governs. As used herein, the indefinite articles“a” and“an” mean“at least one” or“one or more” unless the context clearly dictates otherwise.

It is appreciated that certain features of the disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the disclosure, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub -combination or as suitable in any other described embodiment of the disclosure. No feature described in the context of an embodiment is to be considered an essential feature of that embodiment, unless explicitly specified as such.

Although steps of methods according to some embodiments may be described in a specific sequence, methods of the disclosure may include some or all of the described steps carried out in a different order. The methods of the disclosure may include a few of the steps described or all of the steps described. No particular step in a disclosed method is to be considered an essential step of that method, unless explicitly specified as such.

The phraseology and terminology employed herein are for descriptive purpose and should not be regarded as limiting. Citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the disclosure. Section headings are used herein to ease understanding of the specification and should not be construed as necessarily limiting.

It is appreciated that certain features of the disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the disclosure, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub -combination or as suitable in any other described embodiment of the disclosure. No feature described in the context of an embodiment is to be considered an essential feature of that embodiment, unless explicitly specified as such.

Although stages of methods according to some embodiments may be described in a specific sequence, methods of the disclosure may include some or all of the described stages carried out in a different order. A method of the disclosure may include a few of the stages described or all of the stages described. No particular stage in a disclosed method is to be considered an essential stage of that method, unless explicitly specified as such.

Although the disclosure is described in conjunction with specific embodiments thereof, it is evident that numerous alternatives, modifications and variations that are apparent to those skilled in the art may exist. Accordingly, the disclosure embraces all such alternatives, modifications and variations that fall within the scope of the appended claims. It is to be understood that the disclosure is not necessarily limited in its application to the details of construction and the arrangement of the components and/or methods set forth herein. Other embodiments may be practiced, and an embodiment may be carried out in various ways.

The phraseology and terminology employed herein are for descriptive purpose and should not be regarded as limiting. Citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the disclosure. Section headings are used herein to ease understanding of the specification and should not be construed as necessarily limiting.

REFERENCES

[1] D. Halperin et al ,“Pacemakers and implantable cardiac defibrillators: Software radio attacks and zero-power defenses,” in Proceedings - IEEE Symposium on Security and Privacy, 2008, pp. 129-142.

[2] D. Halperin, T. Kohno, T. S. Heydt-Benjamin, K. Fu, and W. H. Maisel,

“Security and privacy for implantable medical devices,” Pervasive Comput.

IEEE, vol. 7, no. 1, pp. 30-39, 2008.

[3] S. Gollakota, H. Hassanieh, B. Ransford, D. Katabi, and K. Fu,“They can hear your heartbeats: non-invasive security for implantable medical devices,” Proc. ACM SIGCOMM 2011 Conf. SIGCOMM, vol. 41 , no. 4, pp. 2-13, 2011.

[4] F. Xu, Z. Qin, C. C. Tan, B. Wang, and Q. Li,“IMDGuard: Securing implantable medical devices with the external wearable guardian,” in Proceedings - IEEE INFOCOM, 2011, pp. 1862-1870.

[5] T. Denning, K. Fu, and T. Kohno,“Absence Makes the Heart Grow Fonder: New Directions for Implantable Medical Device Security,” Proc. 3rd Conf. Hot Top. Secur. , vol. 11, no. 1, pp. 1-14, 2008.

[6] X. Hei, X. Du, J. Wu, and F. Hu,“Defending resource depletion attacks on

implantable medical devices,” in GLOBECOM - IEEE Global

Telecommunications Conference, 2010.

[7] M. Kintzlinger and N. Nissim,“Keep an eye on your personal belongings! The security of personal medical devices and their ecosystems,” Journal of

Biomedical Informatics, vol. 95. Academic Press Inc., 01-Jul-2019.

[8] M. Zhang, A. Raghunathan, and N. K. Jha,“MedMon: Securing medical devices through wireless monitoring and anomaly detection,” IEEE Trans. Biomed. Circuits Syst. , vol. 7, no. 6, 2013.

[9] A. Kaadan and H. H. Refai,“Securing wireless medical devices,” in

GLOBECOM - IEEE Global Telecommunications Conference, 2012, pp. 942- 948.

[10] R. M. Seepers, C. Strydis, I. Sourdis, and C. I. De Zeeuw,“Adaptive entity- identifier generation for IMD emergency access,” in Proceedings of the First Workshop on Cryptography and Security in Computing Systems - CS2’14, 2014, pp. 41-44.

[11] M. Rostami, A. Juels, and F. Koushanfar,“Heart-to-Heart (H2H): Authentication for Implanted Medical Devices,” Proc. 2013 ACM SIGSAC Conf. Comput.

Commun. Secur. - CCS’13, pp. 1099-1112, 2013.

[12] C. C. Y. Poon, Y. T. Zhang, and S. Di Bao,“A novel biometrics method to

secure wireless body area sensor networks for telemedicine and M-health,” IEEE Commun. Mag. , vol. 44, no. 4, pp. 73-81, 2006.

[13] T. Hong, S. Di Bao, Y. T. Zhang, Y. Li, and P. Yang,“An improved scheme of IPI-based entity identifier generation for securing body sensor networks,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2011, pp. 1519-1522.

[14] K. Cho and D. H. Lee,“Biometric based secure communications without pre deployed key for biosensor implanted in body sensor networks,” in Lecture Notes in Computer Science ( including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics ), 2012, vol. 7115 LNCS, pp. 203-218.

[15] S. Di Bao, C. C. Y. Poon, Y. T. Zhang, and L. F. Shen,“Using the timing

information of heartbeats as an entity identifier to secure body sensor network,” IEEE Trans. Inf. Technol. Biomed. , vol. 12, no. 6, pp. 772-779, 2008.

[16] S.-D. Bao, Y.-T. Zhang, and L.-F. Shen,“Physiological signal based entity

authentication for body area sensor networks and mobile healthcare systems.,” Conf. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. IEEE Eng. Med. Biol.

Soc. Annu. Conf , vol. 3, pp. 2455-8, 2005.

[17] Cohen, N. Nissim, and Y. Elovici,“Novel set of general descriptive features for enhanced detection of malicious emails using machine learning methods,” Expert Syst. Appl , vol. 110, pp. 143-169, Nov. 2018.

[18] M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf,“Support vector machines,” IEEE Intell. Syst. their Appl. , vol. 13, no. 4, pp. 18-28, Jul. 1998.

[19] G. Ratsch, B. Scholkopf, S. Mika, and K.-R. Miiller,“SVM and Boosting: One Class,” 2000.

[20] M. Kintzlinger, N. Nissim " Keep an eye on your personal belongings! The

security of personal medical devices and their ecosystems", Journal of

Biomedical Informatics Volume 95, July 2019, 103233