SYSTEMS AND METHODS FOR PREDICTING CARDIOTOXICITY OF MOLECULAR PARAMETERS OF A COMPOUND BASED ON MACHINE LEARNING ALGORITHMS

1. CROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of priority of U.S. Provisional Application No. 62/181, 115, filed June 17, 2015, the content of which is hereby incorporated by reference in its entirety.

2. TECHNICAL FIELD

[0002] This application generally relates to predicting cardiotoxicity of a compound.

3. BACKGROUND

[0003J A given compound that is administered to a subject may be intended to interact with a desired target, e.g., a protein involved in a particular pathology that the compound is intended to treat, but potentially can interact with one or more unintended targets, e.g., proteins that are not involved in the particular pathology that the compound is intended to treat. Such interactions with unintended targets potentially can cause severe side effects. A prominent example is the hERG (human ether-a-go-go related gene) potassium channel, which is responsible for the repolarization of the cardiac action potential. In the late 1990s, numerous drugs had to be removed from the market because such drugs unintentionally blocked the hERG potassium channel, resulting in a prolongation of the QT-interval of the action potential and causing the subject to experience life threatening arrhythmia. Since then, each compound entering the market or assessed for clinical trials is assessed for safety with respect to the hERG potassium channel. Compounds that interfere with cardiac ion protein channels or other normal activity of the heart can be referred to as being "cardiotoxic" or as having "torsadogenic activity." Cardiotoxic compounds can cause Torsades de Pointes.

[0004] Computer models for predicting the side effects of a compound with respect to the hERG potassium channel potentially can be helpful so as to sort out high risk compounds even before those compounds are synthesized. For example, receptor-based approaches utilize three-dimensional (3D) structural data available for intended and unintended targets, e.g., proteins. However, such approaches can be relatively expensive, which can make it prohibitive to use such approaches to analyze large datasets, e.g., large numbers of compounds. Additionally, such approaches can be limited to studies of molecules with available parameters (e.g., force-fields). Therefore, molecular simulations targeting protein- drug complexes are filling the niche for the "designer's" approach to pre-clinical studies where the dataset is already curated and a relatively small sub-set of the potentially-toxic compounds identified. Other examples for receptor-based models use molecular docking, all- atom molecular dynamics, and Free Energy simulations

[0005] Exemplary alternatives are ligand based models, which can be collectively categorized as structure-activity relationship (SAR) models, and which can be less expensive to use than are receptor-based approaches, and can be relatively computationally fast and reasonably accurate. In a SAR model, the chemical structure of a compound is known, such that the compound can be characterized using a set of parameters, for example, the compound's solubility, the compound's weight, or the number of rotatable bonds in the compound. Such so called "molecular parameters" then are used as input for machine learning algorithms to explore relationships within the data and to train models. Some molecular parameters are strongly correlated with one another. As one example, the number of rings in a compound, the number of atoms in a compound, and the molecular weight of the compound may be correlated with one another. Such correlations potentially can lead to high variance in the models, thus reducing the robustness of the model. The general recommendation to deal with covariant data is linearization, e.g., with principle component analysis (PCA) or feature selection. Alternatively, distance based methods like iso-mapping can be applied to learn the underlying structure of the data and train the model based on that structure. Another possibility is to use certain types of non-linear models that perform an internal feature selection. Finally, the selected set of features, which can include specific parameters, linear combinations or otherwise transformed collective coordinates, then can be fed into a machine learning algorithm.

[0006] Models to predict hERG affinities have been published for at least a decade. Some of them have been made publicly available. However, it appears that accuracies of some models can be overestimated due to a variety of reasons. For example, an apparently high correlation between assayed activity and in-silico predictions can arise from using a relatively limited training set and/or test set. For example, the generation of a dataset of active and inactive compounds is usually not a randomized and representative sample. Splitting a dataset up into training, test and validation set can therefore lead to artifacts. In machine learning a training set is used to train a model. If sufficient data is available, the remaining data is spilt up into a test set for the selection of the best model so as to avoid or inhibit over-fitting, and a validation set so as to estimate the off-sample accuracy/error, e.g., the prediction error in a sample that has not been used to build or select the model. If the data is not randomized, confounding variables can exist in the dataset which are not representative for the population of all data, which can lead to high off-sample accuracies in validation sets originating from the same sample as the training set.

[0007] Some studies measure the quality of the prediction based on the true positive rate, however this approach can overestimate the performance of the model. For example, based on a model classifying all compounds as "active," then the true positive rate will be optimal (meaning one), but the resulting model cannot distinguish between the different classes (e.g., active or inactive). Therefore, other metrics must be used like the prediction accuracy (PA) or Kohen's kappa (KK) or the Fl score. PA takes both classes into account and can lead to accurate estimations as long as both classes (active and inactive) are equally represented in the validated set. KK takes random correct classification into account and Fl is designed to account for unbalanced dataset similar to KK. Other metrics can be used to estimate the ranking quality of a model, such as the area under the receiver-operator-characteristic curve (AROC). For real number estimates metrics like the Pearson correlation coefficient (R) and the coefficient of determination (R2) or the root mean squared error (RMSE) are used. While r measures the coupling between two variables, R2 and RMSE measure the absolute agreement of two variables.

4. SUMMARY

[0008] Embodiments of the present invention provide systems and methods for predicting cardiotoxicity of molecular parameters of a compound based on machine learning algorithms.

[0009] Under one aspect, a computer-implemented method of predicting cardiotoxicity of molecular parameters of a compound is provided. The method includes, by a computer, providing as input to a machine learning algorithm the molecular parameters of the compound. The molecular parameters can include at least structural information about the compound, and the machine learning algorithm can have been trained using respective molecular parameters of compounds known to have cardiotoxicity and of compounds known not to have cardiotoxicity. The method also can include, by the computer, receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the compound.

[0010] In some embodiments, the representation of the predicted cardiotoxicity includes, for each molecular parameter of at least the subset of the molecular parameters of the compound, a numerical value representing the predicted cardiotoxicity of that molecular parameter.

[0011] Some embodiments further include redesigning the compound so as not to include at least one of the molecular parameters of at least the subset. For example, the method can include, by the computer, providing as input to the machine learning algorithm the molecular parameters of the redesigned compound; and by the computer, receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the redesigned compound.

[0012] In some embodiments, the representation includes a value representative of a prediction that the molecular parameter of at least the subset will cause the compound to block two or more cardiac ion protein channels. For example, the two or more cardiac ion protein channels can be selected from the group consisting of: sodium ion channel proteins, calcium ion channel proteins, and potassium ion channel proteins. In some embodiments, the potassium ion channel protein hERGl, the sodium ion channel protein is hNa _v 1.5, or the calcium channel protein is hCa _v l .2.

[0013] Some embodiments further include, by the computer, providing as input to the machine learning algorithm, respective molecular parameters of a plurality of compounds of which the previously recited compound is a member. Some embodiments further include, by the computer, receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds. Some embodiments further include, by the computer, selecting a compound of the plurality of compounds based on the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds.

[0014] In some embodiments, the compounds known to have cardiotoxicity and the compounds known not to have cardiotoxicity are selected based on a statistical analysis of the molecular parameters of those compounds. [0015] In some embodiments, the machine learning algorithm is selected from the group consisting of: a naive Bayes model, a naive Bayes bitvectors model, a decision tree model, a random forest model, a LogReg model, and a boosting model. In some embodiments, the boosting model includes the XGBoost algorithm.

[0016] In some embodiments, the molecular parameters further include one or more of physical information about the compound, and chemical information about the compound.

[0017] Under another aspect, a computer system for predicting cardiotoxicity of molecular parameters of a compound is provided. The computer system includes a processor; and at least one computer-readable medium. The medium can store the molecular parameters of the compound, the molecular parameters including at least structural information about the compound. The medium also can store a machine learning algorithm having been trained using respective molecular parameters of compounds known to have cardiotoxicity and of compounds known not to have cardiotoxicity. The medium also can store instructions for causing the processor to perform steps including: providing as input to the machine learning algorithm the molecular parameters of the compound; and receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molec ular parameter of at least a subset of the molecular parameters of the compound.

[0018] In some embodiments, the representation of the predicted cardiotoxicity includes, for each molecular parameter of at least a subset of the molecular parameters of the compound, a numerical value representing the predicted cardiotoxicity of that molecular parameter.

[0019] In some embodiments, the at least one computer-readable medium further stores instructions for causing the processor to redesign the compound so as not to include at least one of the molecular parameters of at least the subset.

[0020] In some embodiments, the at least one computer-readable medium further stores instructions for causing the processor to: provide as input to the machine learning algorithm the molecular parameters of the redesigned compound; and receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the redesigned compound.

[0021] In some embodiments, the representation includes a value representative of a prediction that the molecular parameter of at least the subset will cause the compound to block two or more cardiac ion protein channels. In some embodiments, the two or more cardiac ion protein channels are selected from the group consisting of: sodium ion channel proteins, calcium ion channel proteins, and potassium ion channel proteins. In some embodiments, the potassium ion channel protein hERGl, the sodium ion channel protein is hNa _v 1.5, or the calcium channel protein is hCa _v 1.2.

[0022] In some embodiments, the at least one computer-readable medium further stores instructions for causing the processor to: provide as input to the machine learning algorithm respective molecular parameters of a plurality of compounds of which the previously recited compound is a member; receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds; and select a compound of the plurality of compounds based on the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds.

[0023] In some embodiments, the compounds known to have cardiotoxicity and the compounds known not to have cardiotoxicity are selected based on a statistical analysis of the molecular parameters of those compounds.

[0024] In some embodiments, the machine learning algorithm is selected from the group consisting of: a naive Bayes model, a naive Bayes bitvectors model, a decision tree model, a random forest model, a LogReg model, and a boosting model. In some embodiments, the boosting model includes the XGBoost algorithm.

[0025] In some embodiments, the molecular parameters are selected from the group consisting of: structural information about the compound, physical information about the compound, and chemical information about the compound.

[0026] Under another aspect, at least one computer-readable medium for use in predicting cardiotoxicity of molecular parameters of a compound is provided. The at least one computer-readable medium stores: the molecular parameters of the compound, the molecular parameters including at least structural information about the compound; a machine learning algorithm having been trained using respective molecular parameters of compounds known to have cardiotoxicity and of compounds known not to have cardiotoxicity; and instructions for causing a processor to perform steps including: providing as input to the machine learning algorithm the molecular parameters of the compound; and receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the compound.

5. BRIEF DESCRIPTION OF THE FIGURES

[0027] FIG. 1 A illustrates steps in an exemplary method of predicting cardiotoxicity of molecular parameters of a compound, according to some embodiments of the present invention.

[0028] FIG. IB illustrates steps in an exemplary method of training a machine learning algorithm for predicting cardiotoxicity of molecular parameters of a compound, according to some embodiments of the present invention.

[0029] FIG. 2 illustrates an exemplary system for predicting cardiotoxicity of molecular parameters of a compound, according to some embodiments of the present invention.

[0030] FIG. 3A illustrates an exemplary probability distribution of mutual similarity among a plurality of compounds that are known to have cardiotoxicity and a plurality of compounds that are known not to have cardiotoxicity. Inset illustrates result of primary component analysis of such compounds.

[0031] FIG. 3B illustrates an exemplary probability distribution of mutual similarity among a subset of compounds that are known to have cardiotoxicity and a subset of compounds that are known not to have cardiotoxicity. Inset illustrates result of primary component analysis of such compounds.

[0032] FIGS. 4A-4B respectively illustrate exemplary IC50 and pIC50 values of an exemplary set of compounds, according to some embodiments of the present invention.

[0033] FIGS. 5A-5J illustrate ROC curves for an exemplary training set and test sets for exemplary machine learning algorithms, according to some embodiments of the present invention.

[0034] FIGS. 6A-6E illustrate exemplary performance measures of exemplary machine learning algorithms, according to some embodiments of the present invention.

[0035] FIGS. 7A-7C illustrate ROC curves for an exemplary training set, test set, and validation set for exemplary machine learning algorithms, according to some embodiments of the present invention. [0036] FIG. 8 illustrates exemplary prediction accuracies for an exemplary training set, test set, and validation set for exemplary machine learning algorithms, according to some embodiments of the present invention.

[0037] FIG. 9 illustrates histograms showing exemplary predicted or actual numbers of active (1.0 on x-axis) and inactive (0.0 on x-axis) compounds in an exemplary test set with respect to different exemplary machine learning algorithms, according to some embodiments of the present invention. The histogram activities shows the actual distribution.

[0038] FIGS . 1 OA- 10G illustrate exemplary performances of different exemplary machine learning algorithms with respect to an exemplary validation set, according to some embodiments of the present invention. Compounds w ith IC50 of less than or equal to 10 μΜ were considered "active." The left-most panels indicate an exemplary probability to be active, the middle panels indicate an exemplary corresponding classification over the experimental pIC50 values, and the right-most panels are corresponding ROC curves.

[0039] FIG. 1 1 illustrates an exemplary heatmap of the mutual correlation coefficients of all features in an exemplary training set, according to some embodiments of the present invention.

[0040] FIGS. 12A-12H illustrate exemplary ROC curves for an exemplary training set and test set for exemplary machine learning algorithms using isomapping, according to some embodiments of the present invention.

[0041] FIGS. 13 A- 13E illustrate exemplary performance measures of exemplary machine learning algorithms using isomapping, according to some embodiments of the present invention.

[0042] FIGS. 14A-14C illustrate ROC curves for false positives for an exemplary training set, test set, and validation set for exemplary machine learning algorithms, according to some embodiments of the present invention.

[0043] FIG. 15 illustrates ROC curves for compounds in an exemplary training set for a NULL machine learning algorithm, according to some embodiments of the present invention.

[0044] FIGS. 16A-16D illustrate performance of an exemplary 3C model for assessment of torsadogenic potential for a blinded set of blockers, according to some embodiments of the present invention. [0045] FIGS. 17A- 17J illustrate probabilities to be active and ROC curves for an exemplary validation set for different machine learning algorithms, according to some embodiments of the present invention.

[0046] FIGS. 18A-18B respectively illustrate probabilities to be active and ROC curves for an exemplary validation set for a consensus among different machine learning algorithms, according to some embodiments of the present invention.

[0047] FIGS. 19A-19D illustrate probabilities to be active on hERG (light grey) with respect to antifungal activity (dark grey) for an exemplary set of compounds, according to some embodiments of the present invention.

[0048] FIG. 20 illustrates the pIC50 values of an exemplary set of compounds, according to some embodiments of the present invention.

[0049] FIG. 21 A illustrates an example of the boosting model on a specific dimension of an input vector, according to some embodiments of the present invention.

[0050] FIG. 21 B illustrates an example of a pair of an active compound (left) and inactive (right) where a change of a chemical group leads to a shift in the class probability.

[0051] FIG. 22 illustrates illustrate an AROC histogram of the molecular descriptors, according to some embodiments of the present invention.

[0052] FIGS. 23A-D illustrate ROC curves of the most predictive molecular descri ptors (AROOO.55) (FIG. 23A), normal features (FIG. 23B), 2D-pharmacophore features (FIG. 23C) and similarity based features (FIG. 23D), according to some embodiments of the present invention.

[0053] FIG. 24 illustrates steps in an exemplary method of selecting a model of molecular parameters of a compound that can be used for predicting cardiotoxicity of the compound, according to some embodiments of the present invention.

[0054] FIGS. 25A-D illustrate mean cross-validated R2 (Q2) from 10 fold cross- validation (FIG. 24A), the mean cross-validated AROC (cvAROC) from 10 fold cross- validation (FIG. 24C) and the corresponding box plots (FIGS. 24B and 24D), according to some embodiments of the present invention.

[0055] FIGS. 26A-D illustrate learning curves for different numbers of iterations (N) (feature set; 8, parameter set: 5). The error bars indicate the standard deviation of five repetitions with randomly selected training and test sets, according to some embodiments of the present invention.

[0056] FIG. 27A illustrates the correlation of fitted data with experimental data, the dashed line shows perfect correlation, the vertical and horizontal dashed lines show the cutoffs used for class and classification (>5 active), according to some embodiments of the present invention.

[0057] FIG. 27B illustrates the corresponding ROC curve using same class criterion as illustrated in FIG. 27A, according to some embodiments of the present invention. The dashed line shows the random distribution and the shaded area shows expected variance of random prediction.

[0058] FIGS. 28A-C illustrate model performance for test set 1, according to some embodiments of the present invention. FIG. 27A illustrates correlation with experimental data, the dashed line shows perfect correlation, horizontal and vertical dashed lines show cutoffs used for class and classification (>5 active). FIG. 27B illustrates ROC curve using same class criterion. The dashed line shows random distribution, and the shaded area shows expected variance of random prediction. FIG. 27C illustrates error over the distance to the training set for each compound.

[0059] FIG. 29A illustrates the approximated distribution of pIC50 values in training set and test sets, according to some embodiments of the present invention.

[0060] FIG. 29B illustrates the approximated distribution of the maximum similarities to compounds in the training set for all test sets, according to some embodiments of the present invention. For the training set the similarity to the next most similar compound is shown.

[0061] FIGS. 30A-C illustrate model performance for test set 2, according to some embodiments of the present invention.

[0062] FIGS. 31 A-C illustrate model performance for test set 3, according to some embodiments of the present invention.

[0063] FIGS. 32 A-C illustrate model performance for test set 4, according to some embodiments of the present invention.

[0064] FIGS. 33A-C illustrate model performance for combined test sets, according to some embodiments of the present invention. [0065] FIGS. 34A and 34B illustrate the relationship, including AROC, R ² and r, between the minimal distance to the training set (MDT) by combining all test sets, according to some embodiments of the present invention.

[0066] FIG. 35 illustrates the top 50 relative feature scores for the final XGBoost model, according to some embodiments of the present invention.

[0067] FIG. 36 illustrates the accumulated number of compounds per publications and separation in training and test sets (Publications ranked by number of compounds), according to some embodiments of the present invention.

6. DETAILED DESCRIPTION

[0068] Embodiments of the present invention provide systems and methods for predicting cardiotoxicity of molecular parameters of a compound based on machine learning algorithms. For example, systems and methods are provided for predicting with improved accuracy, based on molecular parameters, whether a compound may block one or more, or even two or more, cardiac ion protein channels. For example, in some embodiments, activity of a given compound is predicted not only with respect to the hERG potassium channel, disclosed herein, but also with respect to one or more other cardiac ion protein channels, such as the hNa _v 1.5 sodium channel, disclosed herein, and the hCa _v 1.2 calcium channel, disclosed herein. As used herein, "active" compounds are considered to be compounds that have an IC50 value of lower than 10 μΜ with respect to blocking a cardiac ion protein channel, including but not limited to hERG, hNa _v 1.5, or hCa _v 1.2. The present systems and methods can facilitate accurate and rapid pre-clinical and pre-synthetic screening programs for pipelines of compounds. In comparison, other types of computational analysis can be so computationally time consuming as to preclude performing such analysis on practically useful numbers of compounds.

[0069] It should be appreciated the assessment of cardiotoxicity has become important for the approval of new compounds as pharmaceutical drugs. Demand for computational assessments of cardiotoxicity also is likely to increase. For many targets (including desired and unintended targets), extensive assay libraries are publicly available and are stored in databases such as ChEMBL (available online at www.ebi.ac.uk/chembl/ and operated by the European Molecular Biology Laboratory-European Bioinformatics Institute). Currently, there appears to be no crystal structure available for hERG. Therefore, structure based drug design has been done with homology models. Available models of hERG typically include the pore region of the potassium channel and in some cases the voltage sensing domains. hERG, however, has been studied extensively and a large amount of ligand affinities has been collected and is provided in databases which are publicly accessible. In the case of hERG, it has been suggested that molecules do not only bind to the inner cavity, but also target a binding site at the voltage sensors (NS 1643). It is also likely that residues in the binding site adopt different conformations for different ligands. By modeling docking with just a single structure potentially can neglect compounds that bind to a different conformation or a different binding site. Additionally, different stereoisomers can have different affinities. However, the present systems and methods need not necessarily depend on the particular structure of the target, nor on the conformation of the compound.

[0070] In some embodiments, "activity" can include a binary definition, e.g., a definition of a compound, as a whole, being either active or inactive. Additionally, the present systems and methods can provide finer binning, ranges of pIC50, or raw pIC50 values, and per-group decomposition with statistical weights corresponding to risk factors associated with the functional group or other molecular parameter. Probabilities for a compound to be active can be output. For example, the present systems and methods can correlate molecular parameters with experimental data, so that a user can be provided with an estimation about the affinity. Some implementations can use a linear regression model that directly predicts pIC50s. Other embodiments readily can be envisions based on the teachings herein.

6.1 Cardiac Ion Protein Channels

6.1.1 Human Ether-a-go-go Related Gene 1 (hERGl) Channel:

[0071] Cardiotoxicity is a leading cause of attrition in clinical studies and post-marketing withdrawal. The human Ether-a-go-go Related Gene 1 (hERGl) K ⁺ ion channel is implicated in cardiotoxicity, and the U.S. Food and Drug Administration (FDA) requires that candidate drugs be screened for activity against the hERGl channel. Recent investigations suggest that non-hERG cardiac ion channels are also implicated in cardiotoxicity. Therefore, screening of candidate drugs for activity against cardiac ion channels, including hERGl , is recommended.

[0072] The hERGl ion channel (also referred to as KCNH2 or Kvl 1.1) is a key element for the rapid component of the delayed rectified potassium currents (for) in cardiac myocytes, required for the normal repolarization phase of the cardiac action potential (Curran et ai, 1995, "A Molecular Basis for Cardiac- Arrhythmia; HERG Mutations Cause Long QT Syndrome," Cell, 80, 795-803; Tseng, 2001, "I(Kr): The hERG Channel," J. Mol. Cell. Cardiol, 33, 835-49; Vandenberg et al, 2001 , "HERG Kb Channels: Friend and Foe," Trends. Pharm. Sci. 22, 240-246). Loss of function mutations in hERGl cause increased duration of ventricular repolarization, which leads to prolongation of the time interval between Q and T waves of the body surface electrocardiogram (long QT syndrome-LQTS) (Vandenberg et al, 2001 ; Splawski et al, 2000, "Spectrum of Mutations in Long-QT Syndrome Genes KVLQT1, HERG, SCN5A, KCNE1, and KCNE2," Circulation, 102, 1178- 1185; Witchel et al, 2000, "Familial and Acquired Long QT Syndrome and the Cardiac Rapid Delayed Rectifier Potassium Current, Clin. Exp. Pharmacol. Physiol, 27, 753-766). LQTS leads to serious cardiovascular disorders, such as tachyarrhythmia and sudden cardiac death.

[0073] Diverse types of organic compounds used both in common cardiac and noncardiac medications, such as antibiotics, antihistamines, and antibacterials, can reduce the repolarizing current Vkr (i.e., with binding to the central cavity of the pore domain of hERGl) and lead to ventricular arrhythmia (Lees-Miller et al, 2000, "Novel Gain-of-Function Mechanism in Kb Channel-Related Long-QT Syndrome: Altered Gating and Selectivity in the HERGl N629D Mutant," Circ. Res., 86, 507-513; Mitcheson et al, 2005, "Structural Determinants for High-affinity Block of hERG Potassium Channels," Novarlis Found. Symp. 266, 136-150; Lees-Miller et al, 2000, "Molecular Determinant of High- Affinity Dofetilide Binding to HERGl Expressed in Xenopus Oocytes: Involvement of S6 Sites," Mol.

Pharmacol, 57, 367-374). Therefore, several approved drugs (i.e., terfenadine, cisapride, astemizole, and grepafloxin) have been withdrawn from the market, whereas several drugs, such as thioridazine, haloperidol, sertindole, and pimozide, are restricted in their use because of their effects on QT interval prolongation (Du et al, 2009, "Interactions between hERG Potassium Channel and Blockers," Curr. Top. Med. Chem., 9, 330-338; Sanguinetti et al, 2006, "hERG Potassium Channels and Cardiac Arrhythmia," Nature, 440, 463-469).

[0074] Accordingly, in some embodiments of the systems and methods disclosed herein, the cardiac ion protein channel is the Human Ether-a-go-go Related Gene 1 (hERGl) channel. The DNA and amino acid sequences for hERGl are provided as SEQ ID NO: 1 and SEQ ID NO: 2, respectively. Without being limited by any theory, in one aspect of the disclosure, the blocking of the central pore cavity or channel of hERG by a drug is a predictor of the cardiotoxicity of the drug. Undesired drug blockade of K ⁺ ion flux in hERGl can lead to long QT syndrome, eventually inducing fibrillation and arrhythmia. hERGl blockade is a significant problem experienced during the course of many drug discovery programs. 6.1.2 Human Navl.5 Voltage Gated Sodium Channel:

[0075] The Na _v l .5 voltage gated sodium channel (VGSC) is responsible for initiating the myocardial action potential and blocking Na _v 1.5 through either mutations or its interactions with small molecule drugs or toxins have been associated with a wide range of cardiac diseases. These diseases include long QT syndrome 3 (LQT3), Brugada syndrome 1 (BRGDA1 ) and sudden infant death syndrome (SIDS).

[0076] Accordingly, in other embodiments of the systems and methods disclosed herein, the cardiac ion protein channel is the hNa _v 1.5 voltage gated sodium channel. The DNA and amino acid sequences for hNavl .5 are provided as SEQ ID NO: 3 and SEQ ID NO: 4, respectively.

[0077] Without being limited by any theory, in one aspect of the disclosure, the blocking of the central pore cavity or channel of hNa _v 1.5 by a drug is a predictor of the cardiotoxicity of the drug. Undesired drug blockade of Na ⁺ ion flux in hNa _v l .5 can lead to long QT syndrome, eventually inducing fibrillation and arrhythmia. Blockage of hNa _v 1.5 is a significant problem experienced during the course of many drug discovery programs. For example, ranolazine is understood to block only the slowly inactivating component of the sodium current.

6.1.3 Human Cavl.2 Voltage Gated Calcium Channel:

[0078] The Ca _v l .2 voltage gated calcium channel is also responsible for mediating the entry of calcium ions into excitable cells and blocking Ca _v 1.2 through either mutations or its interactions with small molecule drugs or toxins have been associated with a wide range of cardiac diseases. These diseases include long QT syndrome 3 (LQT3); Brugada syndrome 1 (BRGDA1); inherited neuronal ion channelopathies such as described in Catterall et al., "Inherited neuronal channelopathies: New windows on complex neurological diseases," J. Neurosci. 28(46): 1 1768-11777 (2008)," the entire contents of which are incorporated by reference herein; and atrial fibrillation, which can have a genetic component, such as described in Christophersen et al., "Genetics of atrial fibrillation: From families to genomes," J Hum. Genet. 2015 May 21, doi: 10.1038/jhg.2015.44 (epub ahead of print), the entire contents of which are incorporated by reference herein.

[0079] Accordingly, in still other embodiments of the systems and methods disclosed herein, the cardiac ion protein channel is the hCa _v 1.2 voltage gated calcium channel. The DNA and amino acid sequences for hCavl.2 are provided as SEQ ID NO: 5 and SEQ ID NO: 6, respectively.

[0080] Without being limited by any theory, in one aspect of the disclosure, the blocking of the central pore cavity or channel of hCa _v 1.2 by a drug is a predictor of the cardiotoxicity of the drug. Undesired drug blockade of Ca ⁺² ion flux in hCa _v 1.2 can lead to long QT syndrome, eventually inducing fibrillation and arrhythmia. Blockage of hCa _v 1.2 is a significant problem experienced during the course of many drug discovery programs.

6.2 Compounds

[0081] In some embodiments of the systems and methods disclosed herein, the compound is selected from a list of compounds that have failed in clinical trials, or were halted in clinical trials due to cardiotoxicity. Such compounds could benefit from a structural prediction of the molecular parameter or subset of molecular parameters that may be responsible for blocking two or more of the cardiac ion protein channels disclosed herein.

[0082] Accordingly, in some embodiments, the compound is selected from Table 1 , below:

[0083] In other embodiments, the compound is an anticancer agent, such as an anthracycline, mitoxantrone, cyclophosphamide, fluorouracil, capecitabine and trastuzumab. In some embodiments, the compound is an immunomodulating drug, such as interferon- alpha-2, interleukin-2, infliximab and etanercept. In some embodiments, the compound is an antidiabetic drug, such as rosiglitazone, pioglitazone and troglitazone. In some embodiments, the compound is an antimigraine drug, such as ergotamine and methysergide. In some embodiments, the compound is an appetite suppressant, such as fenfulramine,

dexfenfluramine and phentermine. In some embodiments, the compound is a tricyclic antidepressants. In some embodiments, the compound is an antipsychotic drug, such as clozapine. In some embodiments, the compound is an antiparkinsonian drug, such as pergolide and cabergoline. In some embodiments, the compound is a glucocorticoid. In some embodiments, the compound is an antifungal drugs such as itraconazole and amphotericin B. In some embodiments, the compound is an NSAID, including selective cyclo-oxygenase (COX)-2 inhibitors.

[0084] In still other embodiments, the compound is an antihistamine, an antiarrhythmic, an antianginal, an antipsychotic, an anticholinergic, an antitussive, an antibiotic, an antispasmodic, a calcium antagonist, an inotrope, an ACE inhibitor, an antihypertensive, a beta-blocker, an antiepileptic, a gastroprokinetic agent, an alphal -blocker, an antidepressant, an aldosterone antagonist, an opiate, an anesthetic, an antiviral, a PDE inhibitor, an antifungal, a serotonin antagonist, an antiestrogen, or a diuretic.

[0085] In still other embodiments, the compound is an active ingredient in a natural product.

[0086] In still other embodiments, the compound is a toxin or environmental pollutant.

[0087] In still other embodiments, the compound is an antiviral agent. For example, in some embodiments, the compound is selected from the group consisting of a protease inhibitor, an integrase inhibitor, a chemokine inhibitor, a nucleoside or nucleotide reverse transcriptase inhibitor, a non-nucleoside reverse transcriptase inhibitor, and an entry inhibitor.

[0088] In still other embodiments, the compound is capable of inhibiting hepatitis C virus (HCV) infection. For example, in some embodiments, the compound is an inhibitor of HCV NS3/4A serine protease. In some embodiments, the compound is an inhibitor of HCV NS5B RNA dependent RNA polymerase. In some embodiments, the compound is an inhibitor of HC V NS5A monomer protein.

[0089] In still other embodiments, the compounds is selected from the group consisting of Abacavir, Aciclovir, Acyclovir, Adefovir, Amantadine, Amprenavir, Ampligen, Arbidol, Atazanavir, Balavir, Boceprevirertet, Cidofovir, Darunavir, Delavirdine, Didanosine.

Docosanol, Edoxudine, Efavirenz, Emtricitabine, Enfuvirtide, Entecavir, Famciclovir, Fomivirsen, Fosamprenavir, Foscarnet, Fosfonet, Ganciclovir, Ibacitabine, Imunovir, Idoxuridine, Imiquimod, Indinavir, Inosine, Interferon type III, Interferon type II, Interferon type I, Interferon, Lamivudine, Lopinavir, Loviride, Maraviroc, Moroxydine, Methisazone, Nelfinavir, Nevirapine, Nexavir, Oseltamivir (Tamiflu), Peginterferon alfa-2a, Penciclovir, Peramivir, Pleconaril, Podophyllotoxin, Raltegravir, Ribavirin, Rimantadine, Ritonavir, Pyramidine, Saquinavir, Sofosbuvir, Stavudine, Telaprevir, Tenofovir, Tenofovir disoproxil, Tipranavir, Trifluridine, Trizivir, Tromantadine, Truvada, Valaciclovir (Valtrex),

Valganciclovir, Vicriviroc, Vidarabine, Viramidine, Zalcitabine, Zanamivir (Relenza), and Zidovudine.

6.3 Systems and Methods

[0090] In some embodiments, the present systems and methods encompass preprocessing of data regarding compounds available for use in preparing training sets, test sets, and validation sets respectively for use in training, testing, or validating a machine learning algorithm. For example, higher accuracies have been observed in validation set that was generated from the same population of compounds as the training set, but lower accuracies have been observed for independent validation sets that were not generated from the same population as the data set. The apparently exaggerated accuracy in the circumstance where the validation set was generated from the same population of compounds as the training set is believed to arise from confounding, e.g., correlation with third variables. Confounding typically is reduced via randomization or stratification of the population of compounds. However, to stratify a dataset of compounds can be relatively difficult for high dimensional data and even may remove the actual information that distinguishes active from inactive compounds. Regarding randomization, the samples of active an inactive compounds in a database such as ChEMBL can be biased by the nature of experiments which were performed in preparing the database. For example, experiments can be designed to find and report active compounds and can focus on derivatives that share high structural similarity.

Randomization can reduce such a bias, but can sacrifice instances that can be valuable for the model building process. Accordingly, without randomization or stratification, the reported errors in validation sets should be interpreted with special care. Therefore, it is recommended to at least one alternative validation set that originates from a different population than the training and test sets. Additionally, as provided in greater detail herein, the present systems and methods, the compounds used to prepare the training set, test set, and validation set can be selected based on a statistical analysis of the molecular parameters of those compounds.

[0091] Note that although compounds can be considered "active" based on their activity, the data in databases such as ChEMBL can include different assays in different cell lines, and therefore the apparent activity of a compound can be based, in part, upon the particular assay or cell line used, rather than being based on the interaction between the compound and the target. Accordingly, it can be desirable to include in the data set only compounds for which activity was assessed using a single specific assay, because the IC50 depends on the environment in the cell (e.g., ion concentration, pH, and the like). However, such a strategy may exclude too many compounds, thus yielding a training set, test set, and validation set that are too small to accurately train and assess a machine learning algorithm. In some embodiments, the machine-learning based systems and methods provided herein include creating training and validation sets based on one or more single-source cell lines that overexpress hERG and NaV1.5 channels. Additionally, potential variability of data from various cell lines can be accounted for based on literature and data mining on available data from clinical trials and reported torsadogenic activity for large panels of compounds as an additional training descriptor correlated to molecular parameters such as can be used herein.

[0092] The present systems and methods are believed to be useful for targets with or without structural model, and for which binding assays have been performed for a hundred compounds or more, or on the order of hundreds of compounds (or more). The present systems and methods are believed to facilitate the process of drug optimization and can serve as a warning system for structures which are likely to reveal interactions between a compound and an unintended target, particularly, a cardiac ion protein channel. Therefore, the present systems and methods can support both positive design (optimization of affinity against target) and negative design (increase of specificity for target). An exemplary advantage of providing predictions of cardiotoxicity based on molecular parameters is the relatively short computational time, thus facilitating rapid screening of a large number of compounds. [0093] Drug induced QT prolongation is known to be a multi-channel phenomenon in which hERG blockage contributes, noting that there are FDA approved drugs that block hERG but do not prolong the QT interval. The present systems and methods optionally encompass multi-target approaches to predict the occurrence of drug induced QT prolongation more accurately by predicting a compound's interactions with multiple ion channels. In one nonlimiting example, a polynomial regression models based machine learning algorithm received as input various molecular parameters (e.g.. solubility, lipophilicity, molecular weight, number of specific atoms, molecular fingerprints and other molecular and structural properties, such as distances between atoms and groups of atoms or groups with distinct functions such as hydrogen donors or aceptors) for compounds in an exemplary dataset of established blockers of hERGl , Ca _v 1.2 and Na _v 1.5 channels with reported dysrhythmic activity and electrophysiological data. In one nonlimiting example, described in greater detail below in the "Examples" section, the Pearson correlation coefficients in the validation set between experimental and predicted pIC50 of

liERG/Na _v 1.5/Ca _v 2.1 model are 0.78/0.6/0.51 respectively (with saquinavir as clear outlier in all datasets) with blinded predictive power in torsadogenic activity of ~70 % for identification of true-positives (torsadogenic) and 49 % of true-negatives (non-torsadogenic). Therefore, the preliminary model described in greater detail below with reference to FIG! 8 can provide enhanced accuracy relative to single-channel based predictive platforms. For example, various models have been generated to predict hERG blockade with prediction accuracies above 80% for relatively small and curated data sets. The models can be distributed independently and optionally can be combined into a multi-target prediction system, that allows one to optimize drugs with respect to various targets simultaneously. Optionally, structure-based results, e.g., from molecular docking, in principle can be integrated into the model design, as soon as structural models of the targets are available. For example, the binding affinities to different conformational states of hERGl channel are known to correlate strongly with the blocker's efficacy. The analysis of different molecular parameters, e.g., molecular group decomposition, from the present systems and methods can aid results of receptor-drug modeling, thus facilitating identification of potentially dangerous moieties (e.g., chemical groups) in the assessed groups of the compounds. Characteristic features of large data sets of compounds, which can have varying chemical scaffolds, then can be developed or identified. [0094] In one nonlimiting example, several thousands of compounds can be estimated on a single CPU in a few minutes and the approach can be easily run in parallel. Optionally, based on the predicted cardiotoxicity of certain molecular parameters, a compound can be redesigned.

[0095] FIG. 1 A illustrates steps in an exemplary method of predicting cardiotoxicity of molecular parameters of a compound, according to some embodiments of the present invention. Method 100 illustrated in FIG. 1 A includes providing as input to a machine learning algorithm respective molecular parameters of one or more compound (step 101). In some embodiments, the molecular parameters can include at least structural information about the compound(s) (step 101 ). By "structural information" it is meant information regarding the presence and relative arrangement of atoms within the compound, e.g., within different portions of the compound. Other exemplary molecular parameters include, but are not limited to, physical information about the compound(s) and chemical information about the compoLind(s). Physical information about the compound(s) can include, but is not limited to, molecular weight, number of atoms and rings, and molecular volume. Chemical information about the compound(s) can include, but is not limited to, polarity, and the number of certain chemical groups. Other examples of physical and chemical infomiation about the compound(s) are provided elsewhere herein. In one example, a suitably programmed computer such as described below with reference to FIG. 2 suitably can obtain the molecular parameters via a user interface, via the network, or from a local or remote computer-readable medium, such as the ChEMBL database. The computer can store the molecular parameters in any suitable computer-readable medium. In one nonlimiting example, the computer obtains the molecular parameters for each compound in the form of a SMILES (simplified molecular-input line-entry system) file such known in the art.

[0096] In some embodiments, the machine learning algorithm has been trained using respective molecular parameters of compounds known to have cardiotoxicity and of compounds known not to have cardiotoxicity (step 101). An exemplary method of training such a machine learning algorithm is provided below with reference to FIG. IB. Exemplary machine learning algorithms include, but are not limited to, a naive Bayes model, a naive Bayes bitvectors model, a decision tree model, a random forest model, a LogReg model, and a boosting model. In some embodiments, the boosting model includes the XGBoost algorithm. In one nonlimiting example, the machine learning algorithm is stored on the same computer-readable medium as are the molecular parameters. In another nonlimiting example, the machine learning algorithm is stored in a different computer-readable medium than are the molecular parameters. The machine learning algorithm can receive as input the molecular parameters of the compound from the computer in any suitable manner. For example, in some embodiments, the same computer can obtain the molecular parameters and also can execute the machine learning algorithm, providing the molecular parameters to that algorithm. In other embodiments, a first computer can obtain the molecular parameters and can transmit the molecular parameters via any suitable wired or wireless communication channel to a second computer that can execute the machine learning algorithm, receiving the molecular parameters as input.

[0097] Method 100 illustrated in FIG. 1A further includes receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the one or more compounds (102). The machine learning algorithm can provide the output to the computer in any suitable manner. For example, in some embodiments, the same computer can execute the machine learning algorithm and receive the output from that algorithm. In other embodiments, a first computer can execute the machine learning algorithm and can transmit the output via any suitable wired or wireless communication channel to a second computer that can receive the output. The computer can store the output in any suitable computer-readable medium. In one nonlimiting example, the machine learning algorithm is stored on the same computer- readable medium as is the output. In another nonlimiting example, the machine learning algorithm is stored in a different computer-readable medium than is the output.

[0098] In one nonlimiting example, the representation of the predicted cardiotoxicity includes, for each molecular parameter of at least the subset of the molecular parameters of the compound, a numerical value representing the predicted cardiotoxicity of that molecular parameter. Table 2 illustrates an exemplary output including such a representation. In the examples provided in Table 2, it can be seen that "fr_piperidine" is associated with the greatest risk of cardiotoxicity, "TPSA" is associated with the next highest risk of cardiotoxicity, and so on. Table 2. Exemplary Output

[0099] In some embodiments, the representation provided as output in step 102 includes a value representative of a prediction that the molecular parameter of at least the subset will cause the compound to block two or more cardiac ion protein channels. In some embodiments, the two or more cardiac ion protein channels include two or more of sodium ion channel proteins, calcium ion channel proteins, and potassium ion channel proteins. Illustratively, the potassium ion channel protein can be bJERGl , the sodium ion channel protein can be hNa _v 1.5, and the calcium channel protein is hCa _v l .2. As noted elsewhere herein, the present systems and methods optionally can predict the cardiotoxicity of molecular parameters with respect to multiple targets. For example, the output provided in step 102 can include a plurality of predictions that the molecular descriptor will cause the compound to block a corresponding plurality of cardiac ion protein channels, e.g., two or more of hERGl , hNa _v 1.5, and hCa _v 1.2. In one example, the information from relative blockade of 3 major cardiac currents can be used as a safety assessment score generated from Rudy model of cardiac currents generating torsadogenicity metrics.

[00100] Optionally, method 100 illustrated in FIG. 1 A further can include redesigning the compound so as not to include at least one of the molecular parameters of at least the subset. For example, based on the output of step 102, the computer can identify one or more molecular parameters that are predicted to be relatively cardiotoxic, and can modify such molecular parameter(s) so as to provide a compound having reduced predicted cardiotoxicity. For example, the computer can obtain the molecular parameters of one of the original compounds of step 101, and can "redesign" the compound by appropriately adjusting the value of one or more of such molecular parameters that are predicted to be relatively cardiotoxic. The computer then can execute steps 101 and 102 of method 100 based on the redesigned compound. For example, the computer can provide as input to the machine learning algorithm the molecular parameters of the redesigned compound in a manner analogous to that described above with reference to step 101 ; and can receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the redesigned compound in a manner analogous to that described above with reference to step 102.

Optionally, such compound redesign and re-analysis can be repeated any suitable number of times. Optionally, following one or more such redesign steps, the compound can be synthesized in a laboratory and evaluated for effectiveness with respect to the desired target, as well as for cardiotoxicity.

[00101] Additionally, note that method 100 optionally can be executed for any desired number of compounds. For example, method 100 further can include, by the computer, providing as input to the machine learning algorithm, respective molecular parameters of a plurality of compounds (of which the compound described above is a member), and can receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds. Additionally, method 100 optionally includes, by the computer, selecting a compound of the plurality of compounds based on the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds.

[00102] The machine learning algorithm used in method 100 can be trained using any suitable technique. In some embodiments, the compounds known to have cardiotoxicity and the compounds known not to have cardiotoxicity, upon which the machine learning algorithm is trained, can be selected based on a statistical analysis of the molecular parameters of those compounds. For example, FIG. IB illustrates steps in an exemplary method of training a machine learning algorithm for predicting cardiotoxicity of molecular parameters of a compound, according to some embodiments of the present invention.

[00103] Method 110 illustrated in FIG. IB includes obtaining respective molecular parameters of a plurality of compounds known to have cardiotoxicity and a plurality of compounds known not to have cardiotoxicity (step 1 11). In one example, a suitably programmed computer such as described below with reference to FIG. 2 suitably can obtain the molecular parameters via a user interface, via the network, or from a local or remote computer-readable medium. The computer can store the molecular parameters in any suitable computer-readable medium. In one nonlimiting example, the computer obtains the molecular parameters for each compound in the form of a SMILES (simplified molecular- input line-entry system) file such known in the art. In some embodiments, the molecular parameters can be obtained from a publically accessible compound database such as described below with reference to FIG. 2. The compounds for which molecular parameters are obtained in step 1 11 can have a distribution of activities, e.g., can have a distribution of IC50's or (dimensionless) pIC50's such as respectively illustrated in FIGS. 3A-3B, wherein "active" compounds can be considered those with an IC50 of less than 10 μΜ.

[00104] Method 1 10 illustrated in FIG. IB further includes, based on a statistical analysis of the respective molecular parameters, assigning to a training set a subset of compounds known to have cardiotoxicity and a subset of compounds known not to have cardiotoxicity (step 112). As one example, principal component analysis (PC A) can be used so as to identify, and to reduce or eliminate, mutual similarity among a plurality of compounds known to have cardiotoxicity and a plurality of compounds known not to have cardiotoxicity, but selecting only a subset of each such plurality. For example, FIG. 3A illustrates an exemplary probability distribution of mutual similarity among a plurality of compounds that are known to have cardiotoxicity ("active"), a plurality of compounds that are known not to have cardiotoxicity ("inactive"), and similarity between active and inactive compounds ("act- inact"). Compounds can be considered to have molecular parameters that are similar to one another based upon such compounds having a dice similarity of greater than 0.15. It can be seen in FIG. 3 A that the pluralities of compounds include a relatively wide range of molecular parameters. The inset to FIG. 3 A illustrates the result of PCA of such compounds. The yellow boxes that appear along the diagonal correspond to clusters of compounds that have similar molecular parameters as one another. Such clusters represent redundancy within the pluralities of compounds, e.g., sets of compounds that have similar molecular parameters as one another and thus potentially can skew the training of the machine learning algorithm.

[00105] PCA or other suitable technique can be used so as to curate the pluralities of compounds, e.g., so as to assign to a training set a subset of compounds known to have cardiotoxicity or to a subset of compounds known not to have cardiotoxicity. For example, PCA can be used to generate a linear independent set of input features. The features under consideration can be standardized, e.g., by converting the features to Z-score = (x- MEAN)/STD, where x is the value of a specific feature and MEAN and STD respectively are the average and standard deviation of all values of that feature in the dataset. The PCA is applied to the standardized values. The output of the PCA can include linear independent linear combinations of features, in other words, collective coordinates that describe the largest variances in the dataset. The PCA vectors are useful to reduce the number of independent features used to train models.

[00106] For example, FIG. 3B illustrates an exemplary probability distribution of mutual similarity among a subset of compounds that are known to have cardiotoxicity ("active"), a subset of compounds that are known not to have cardiotoxicity ("inactive"), and similarity between active and inactive compounds ("act-inact"), wherein the subsets were selected using PCA. It can be seen in FIG. 3B that the pluralities of compounds again include a relatively wide range of molecular parameters. The inset to FIG. 3B illustrates the result of PCA of such compounds, in which it can be seen that substantially no yellow boxes appear along the diagonal that would correspond to clusters of compounds that have similar molecular parameters as one another, as they did in the inset to FIG. 3B. Accordingly, in some embodiments, the statistical analysis of step 112 of method 1 10 provides that the subsets of compounds known to have cardiotoxicity or known not to have cardiotoxicity include substantially no clusters representing redundancy within the pluralities of compounds, that otherwise potentially can skew the training of the machine learning algorithm.

[00107] Referring again to FIG. IB, method 110 further includes executing a machine learning algorithm using the training set. Exemplary machine learning algorithms include, but are not limited to, a naive Bayes model, a naive Bayes bitvectors model, a decision tree model, a random forest model, a LogReg model, and a boosting model. In some embodiments, the boosting model includes the XGBoost algorithm. In one nonlimiting example, the machine learning algorithm is stored on the same computer-readable medium as are the molecular parameters. In another nonlimiting example, the machine learning algorithm is stored in a different computer-readable medium than are the molecular parameters. The machine learning algorithm can receive as input the molecular parameters of the compound from the computer in any suitable manner. For example, in some embodiments, the same computer can obtain the molecular parameters and also can execute the machine learning algorithm, providing the molecular parameters to that algorithm. In other embodiments, a first computer can obtain the molecular parameters and can transmit the molecular parameters via any suitable wired or wireless communication channel to a second computer that can execute the machine learning algorithm, receiving the molecular parameters as input. The resulting trained machine learning algorithm can be used, for example, in any suitable method for predicting cardiotoxicity of molecular parameters of a compound, including but not limited to method 100 described above with reference to FIG. 1A.

[00108] It should be noted that methods 100 and 110 can be executed using any suitable combination of hardware and software. For example, FIG. 2 illustrates an exemplary system for predicting cardiotoxicity of molecular parameters of a compound, according to some embodiments of the present invention. The computer-based architecture illustrated in FIG. 2 includes system 200 that is configured to implement one or both of methods 100 and 1 10; one or more compound databases 230 that are configured to store molecular parameters for compounds known to be cardiotoxic or known not to be cardiotoxic, such as ChEMBL, that are configured to communicate with system 200 via the Internet or other network 220; and a plurality of remote clients 250 that are configured to communicate with system 200 via the Internet or other network 220, are configured to receive user queries requesting predictions of cardiotoxicity of molecular parameters of one or more compounds, to submit such queries to system 200, to receive the results of such queries from system 200, and to output the results of such queries to the user. Alternatively, information within one or more of remote data sources 230 can be converted to local storage within system 200. It will be appreciated that remote compound databases 230 can be operated by an independent entity and need not necessarily be considered to be part of the present invention; accordingly, the architectural details of such data sources 230 are omitted from FIG. 2 for simplicity.

[00109] As illustrated in FIG. 2, system 200 includes one or more processing units (CPU's) 201 (e.g., processing means), a network or other communications interface (NIC) 202 (e.g., networking means), one or more non-volatile, non-transitory, computer readable memory devices or media such as magnetic disk storage or persistent devices 203 (e.g., memory means or storage means) optionally accessed by one or more controllers 204, a user interface 205 including a display 206 and a keyboard 207 or other suitable device for accepting user input, a memory 210 (e.g., memory means or storage means), one or more communication busses 208 for interconnecting the aforementioned components, and a power supply 209 for powering the aforementioned components. Data in memory 210 can be seamlessly shared with non-volatile memory 203 using known computing techniques such as caching. Memory 210 or memory 203 can include mass storage that is remotely located with respect to the central processing unit(s) 201. In other words, some data stored in memory 210 or memory 203 can be hosted on computers that are external to system 200 but that can be electronically accessed by system 200 over an Internet, intranet, or other form of network or electronic cable using network interface 202. In one illustrative embodiment, system 200 is a personal computer. Of course, the present methods equivalently can be performed using commercially available or custom hardware with dozens or more processors connected in parallel, at even greater speed.

[00110] Memory 203 can store one or more databases that store molecular parameters of one or more compounds. Preferably, such database(s) respond appropriately to queries from various modules that can be stored within memory 210, such as described further below. Memory 210 preferably stores an operating system 21 1 that is configured to handle various basic system services and to perform hardware dependent tasks, and a network

communications module 212 that is configured to connect system 200 to various other computers such as remote curated data sources 230 and to clients 250 via one or more communication networks 120, such as the Internet, other wide area networks, local area networks (e.g., a local wired or wireless network can connect the system 200 to the remote client 250), metropolitan area networks, and so on.

[00111] Memory 210 also can store a cardiotoxicity prediction module 213 that includes a plurality of modules configured to cause processing unit 201 to execute the various steps of one or both of methods 100 and 1 10. For example, cardiotoxicity prediction module 213 can include a molecular descriptor module 214 configured to cause processing unit 201 to obtain molecular descriptors for one or more compounds from data source 230, from memory 203, or from a remote client 250, such as described above with reference to step 101 of method 100 or step 1 11 of method 1 10. In some embodiments, the molecular descriptors include at least structural information about the one or more compounds. As noted herein, in one illustrative embodiment the molecular descriptors are in the form of SMILES files, although other suitable formats can be used. Molecular descriptor module 214 also can be configured to cause processing unit 201 to store molecular descriptors within a database in memory 203, such as described above with reference to step 101 of method 100 or step 111 of method 110. Molecular descriptor module 214 also can be configured to cause processing unit 201 to assign to a training set, based on a statistical analysis of respective molecular parameters, a subset of compounds known to have cardiotoxicity and a subset of compounds known not to have cardiotoxicity, such as described above with reference to step 112 of method 110.

[00112] Cardiotoxicity prediction module 213 illustrated in FIG. 2 also includes a machine learning module 215 configured to cause processing unit 201 to train a machine learning algorithm in a manner such as described above with reference to step 113 of method 1 10, or to provide as input to a machine learning algorithm the respective molecular parameters of one or more compounds, where the machine learning algorithm has been trained using respective molecular parameters of compounds known to have cardiotoxicity and compounds known not to have cardiotoxicity, in a manner such as described above with reference to step 101 of method 100. For example, machine learning module 215 can include instructions for causing processing unit 201 to input into the trained machine learning algorithm the molecular descriptors of one or more compounds. In some embodiments, processing unit 201 also executes such a machine learning algorithm.

[00113] In one nonlimiting example, molecular descriptor module 214 need not necessarily require a user to input or define specific molecular parameters or machine learning algorithms to be used, and instead automatically can obtain and train different machine learning algorithms so as to generate best guess/scoring. In one nonlimiting example, molecular parameters can include any of the following molecular parameters available in RDKit:

[00114] fr_C_0_noCOO, PEOE_VSA3 , Chi4v, fr_Ar_COO, fr_SH, Chi4n,

SMRJVSA10, fr_para_hydroxyIation, fr barbitur, fr_Ar_NH, fr halogen,

fr_dihydropyridine, fr_priamide, SlogP_VSA4, fr_guanido, MinPartialCharge, fr_furan, frjnorpholine, fr_nitroso, SlogP_VSA6, fr_C002, fr_amidine, SMR VSA7,

fr benzodiazepine, ExactMolWt, fr jmine, MolWt, fr_hdrzine, fr_urea, NumAromaticRings, fr_quatN, NumSaturatedHeterocycles, NumAliphaticHeterocycles, fr_benzene, fr_phos_acid, fr_sulfone, VSA EStatelO, fr aniline, fr_N_0, fr sulfonamd, frjhiazole, TPSA,

SMR_VSA5, PEOE_VSA14, PEOE_VSAl 3, PEOE_VSA12, PEOE_VSAl 1 ,

PEOE_VSA10, BalabanJ, frjactone, fr_Al_COO, EState_VSA10, EStat_VSAl l,

Heavy AtomMolWt, fr_nitro_arom, ChiO, Chi 1 , NumAliphaticRings, MolLogP, fr_nitro, fr_Al_OH, fr_azo, NumAliphaticCarbocycles, fr_C_0, fr_ether, fr_phenol_noOrthoHbond, fr_alkyl_halide, NumVaienceElectrons, fr aryljnethyl, fr_Ndealkylation2, MinEStatelndex, fr_term_acetylene, HallKierAIpha, fr_C_S, fr_thiocyan, fr_ketone_Topliss, VSA_EState4, VSA_EState5, VSA_EState6, VSA_EState7, NumHDonors, VSA EState2, EStateJVSA9, frJHOCCN, fr_phos_ester, MaxAbsEStatelndex, SlogP_VSA12, VSA_EState9,

SlogP_VSA10, SlogP_VSAl 1, fr_COO, NHOHCount, fr_unbrch_alkane,

NumSaturatedRings, MaxPartialCharge, fr_methoxy, fr_tliiophene, SlogP_VSA8,

SlogP_VSA9, MinAbsPartialCharge, SlogP_VSA5, NumAromaticCarbocycles, SlogP_VSA7, SlogP_VSAl, SlogP_VSA2, SlogP_VSA3, NumRadicalElectrons, fr_NH2, fr_piperzine, frjiitrile, NumHeteroatoms, fr_NHl, fr_NH0, BertzCT, LabuteASA, fr_amide, Chi3n, fr_imidazole, SMR_VSA3, SMR_VSA2, SMR VSA1, Chi3v, SMR_VSA6, EState_VSA8, SMR_VSA4, EState_VSA6, EState_VSA7, EState_VSA4, SMR_VSA8, EState_VSA2, EState_VSA3, fr_Ndealkylationl, EState_VSAl , fr_ketone, Kappa3, ChiOn, fr_diazo, Kappa2, fr_Ar_N, fr_Nhpyrrole, fr_ester, SMR_VSA9, VSA_EStatel, fr_prisulfonamd, fr oxime, EState_VSA5, VSA_EState3, fr_isocyan, Chi2n, Chi2v, Heavy AtomCount, fr_azide, NumHAcceptors, frjactam, fr_allylic_oxid, VSA_EState8, fr_oxazole, fr_piperdine, fr_Ar_OH, fr_sulfide, fr alkyl carbamate, NOCount,

PEOE_VSA9, PEOE_VSA8, PEOE_VSA7, PEOE_VSA6, PEOE_VSA5, PEOE_VSA4, MaxEStatelndex, PEOE_VSA2, PE0E_VSA1, NumSaturatedCarbocycles, frjmide, FractionCSP3, Chilv, fr Al OH noTert, fr epoxide, fr hdrzone, fr_isothiocyan,

NumAromaticHeterocycles, fr_bicyclic, Kappal, MinAbsEStatelndex, fr_phenol, MolMR, Chi In, fr_aldehyde, fr_pyridine, fr_tetrazole, RingCount, fr_nitro_arom_nonortho, ChiOv, fr ArN, NumRotatableBonds, or MaxAbsPartialCharge.

[00115] In one nonlimiting example, any of such molecular parameters can be calculated based on a SMILES file, e.g., a SMILES string such as 'CCC Molecular parameter module 214 can build a molecule object which is then standardized. Afterwards, the molecular parameters are calculated. For each machine learning algorithm, molecular parameters that do not carry any information can be removed.

[00116] In some embodiments, molecular parameters include chemical features with topological (2D) distances between them that produce 2D pharmacophore or 2D fingerprint features. For example, 2D pharmacophore features include the feature definitions from Gobbi and Poppinger (Gobbi and Poppinger 1998) as implemented in RDKit. In some embodiments, the compounds are converted to 2D fingerprints represented as bit-vectors. Each element of the bitvectors serves as a feature for the machine learning algorithm, while keeping bits that were activated at least 100 times.

[00117] Cardiotoxicity prediction module 213 illustrated in FIG. 2 also includes a query module 216 configured to cause processing unit 201 to receive a query term identifying one or more compounds for which cardiotoxicity is to be predicted in a manner such as described above with reference to step 101 of FIG. 1 A. In some embodiments, query module 216 causes processing unit 201 to cause display 206 to display a graphical user interface (GUI) that allows the user to readily define the query term. For example, the GUI can include a list of compounds that are available for analysis and a mechanism configured to permit the user to select from the list, e.g., by presenting check boxes or radio buttons adjacent the compounds that the user can select, or by allowing the user to highlight within the list the compounds of interest, using keyboard 207 or other suitable user interface device coupled to system 200. The GUI also can be configured to facilitate the user's selection of a particular operation to be perform on the selected compounds, such as by allowing the user to redesign compounds by identifying one or more molecular parameters to be altered. For example, the GUI can present the user with output representing the predicted cardiotoxicity of each molecular descriptor of at least a subset of the molecular descriptors of a compound, and the GUI can permit the user to adjust one or more of such molecular descriptors and to run a new prediction in a manner such as described above with reference to method 100. Additionally, as noted below, query module 216 can cause processing unit 201 to accept query terms that are defined remotely, e.g., at remote client 250.

[00118] Query module 216 also causes processing unit 201 to provide as input to the trained machine learning algorithm the molecular descriptors of the one or more compounds, in a manner such as described above with reference to step 101 of method 100. Based on the machine learning algorithm's response, query module 216 causes processing unit 201 to generate an output that represents the predicted cardiotoxicity of each molecular descriptor of at least a subset of the molecular descriptors of the compound, in a manner such as described above with reference to step 102 of method 100. Exemplary suitable outputs are described herein, and others readily can be envisioned. For example, query module 216 can cause processing unit 201 to cause display 206 to display, for each molecular descriptor of at least a subset of molecular descriptors of the compound, a numerical value representing the representing the predicted cardiotoxicity of that molecular parameter. Alternatively, query module 216 can cause processing unit 201 to generate a signal for transmission via a suitable communication channel to remote client 250. Query module 216 further can cause processing unit 201 to cause such an output to be stored in memory 203, to be printed on an associated printer (not illustrated), or otherwise provided to the user. Exemplary outputs are described in greater detail below with reference to Example 2.

[00119] Optionally, system 200 is connected via a network such as the Internet 220 to one or more remote clients 250, which permit users who are remote from system 200 to submit and receive the results of queries to system 200. Typically, remote client 250 can include one or more processing units (CPUs) 251 ; a network or other communications interface (NIC) 252; one or more magnetic disk storage and/or persistent storage devices 253 that are accessed by one or more controllers 254; a user interface 255 including a display 256 and a keyboard 257 or other suitable device configured to accept user input; a memory 260; one or more communication busses 258 for interconnecting the aforementioned components; and a power supply 259 for powering the aforementioned components. In some embodiments, data in memory 260 can be seamlessly shared with non-volatile memory 253 using known computing techniques such as caching.

[00120J The memory 260 preferably stores an operating system 261 configured to handle various basic system services and to perform hardware dependent tasks; and a network communication module 262 that is configured to connect remote client 250 to other computers such as system 200. The memory 260 preferably also stores compound analysis module 263 that is configured to cause processing unit 251 to receive user input defining query terms in a manner analogous to query module 216 of system 200, and to transmit such query terms to query module 216 for use in predicting cardiotoxicity of molecular parameters of a compound. Compound analysis module 263 can cause processing unit 251 to receive a response from query module 216 based on the query terms, and to output such response in a manner analogous to that described above, e.g., can cause display 256 to display a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the compound.

[00121] Note that memories 203 and 210 of system 200 and memories 253 and 260 of remote client 250 can include any suitable internal or external memory device, such as FLASH, RAM, ROM, EPROM, EEPROM, or a magnetic or optical disk or tape.

[00122] Accordingly, embodiments of the present invention provide a computer system for predicting cardiotoxicity of molecular parameters of a compound. The computer system can include a processor (e.g., processing unit 201 of system 200 or processing unit 251 of remote client 250), and at least one computer-readable medium (e.g., memory 203, memory 210, memory 253, memory 260, compound database(s) 230, or any suitable combination thereof). The memory can store the molecular parameters of the compound, the molecular parameters including at least structural information about the compound. The memory also can store a machine learning algorithm having been trained using respective molecular parameters of compounds known to have cardiotoxicity and of compounds known not to have

cardiotoxicity (e.g., machine learning module 215). The memory also can include instructions for causing the processor to perform a step including providing as input to the machine learning algorithm the molecular parameters of the compound (e.g., molecular descriptor module 214, query module 216, compound analysis module 263, or any suitable combination thereof). The memory also can include instructions for causing the processor to receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the compound (e.g., query module 216, compound analysis module 263, or any suitable combination thereof).

[00123] In some embodiments, the representation of the predicted cardiotoxicity includes, for each molecular parameter of at least a subset of the molecular parameters of the compound, a numerical value representing the predicted cardiotoxicity of that molecular parameter. In some embodiments, the at least one computer-readable medium further stores instructions for causing the processor to redesign the compound so as not to include at least one of the molecular parameters of at least the subset. In some embodiments, the at least one computer-readable medium further storing instructions for causing the processor to provide as input to the machine learning algorithm the molecular parameters of the redesigned compound; and receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the redesigned compound.

[00124] In some embodiments, the representation includes a value representative of a prediction that the molecular parameter of at least the subset will cause the compound to block two or more cardiac ion protein channels. In some embodiments, the two or more cardiac ion protein channels are selected from the group consisting of: sodium ion channel proteins, calcium ion channel proteins, and potassium ion channel proteins. In some embodiments, the potassium ion channel protein is hERGl, the sodium ion channel protein is hNa _v 1.5, or the calcium channel protein is hCa _v 1.2.

[00125] In some embodiments, the at least one computer-readable medium further stores instructions for causing the processor to provide as input to the machine learning algorithm respective molecular parameters of a plurality of compounds of which the previously recited compound is a member; receive as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds; and select a compound of the plurality of compounds based on the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of each of the compounds of the plurality of compounds.

[00126] In some embodiments, the compounds known to have cardiotoxicity and the compounds known not to have cardiotoxicity are selected based on a statistical analysis of the molecular parameters of those compounds.

[00127] In some embodiments, the machine learning algorithm is selected from the group consisting of: a naive Bayes model, a naive Bayes bitvectors model, a decision tree model, a random forest model, a LogReg model, and a boosting model. In some embodiments, the boosting model includes the XGBoost algorithm. In some embodiments, the molecular parameters are selected from the group consisting of: structural information about the compound, physical information about the compound, and chemical information about the compound.

[00128] Embodiments of the present invention further provide at least one computer- readable medium for use in predicting cardiotoxicity of molecular parameters of a compound (e.g., any suitable combination of memory 203, memory 211, compound database(s) 230, memory 253, and memory 260. The at least one computer-readable medium stores the molecular parameters of the compound, the molecular parameters including at least structural information about the compound. The at least one computer-readable medium further stores a machine learning algorithm having been trained using respective molecular parameters of compounds known to have cardiotoxicity and of compounds known not to have

cardiotoxicity. The at least one computer-readable medium further stores instructions for causing a processor (e.g., processing unit 201 or processing unit 251 , or any suitable combination thereof) to perform steps including: providing as input to the machine learning algorithm the molecular parameters of the compound; and receiving as output from the machine learning algorithm a representation of the predicted cardiotoxicity of each molecular parameter of at least a subset of the molecular parameters of the compound.

6.4 EXAMPLES

[00129] The following examples are intended to be purely exemplary, and not limiting of the present invention.

6.4.1 Example 1

[00130] In a first example, the present systems and methods were implemented in the Python 2.7 programming language (available from the Python Software Foundation at www.python.org) and the IPython notebook web application (available from the IPython development team at ipython.org). The scikit-learn machine learning in Python library was used for the machine learning algorithms (available from the scikit-learn Project at scikit- learn.org). The calculation of the molecular parameters was done using RDkit Open-Source Cheminformatics Software (available at www.rdkit.org). For the molecular fingerprints, a bitlength of 1024 bits and a depth of 2 was used. Compounds included those listed in Kramer et al., "MICE models: Superior to the HERG model in predicting Torsade de Pointes," Scientific Reports 3: 2100, pages 1-7 (2013) and in the Supplementary Material thereto, the entire contents of which are incorporated by reference herein, exemplary compounds of which are described below in Table 5. Similarities among compounds were calculated based on fingerprint comparison using the Dice similarity score. A molecular fingerprint can be expressed as a bitvector (a vector with only 0 and 1 as components) that is based on the structure of the molecule. The fingerprints can be encoded descriptions of the molecular topology (e.g., atom types and connectivity). There may not be a straightforward connection between the molecular fingerprint and molecular parameters such as used herein, unless the bits of the fingerprint happen to represent specific structural components or other molecular parameters of the compound.

[00131] Compound structures and bioassays were taken from the ChEMBL database. Entries for the ChEMBL target ID: CHEMBL240 with the assay description 'Inhibition of human ERG' and bioactivity type 'IC50' were included. As actives served all compounds with values below 10000 nM. The values were converted to dimensionless pIC50 values. Inactive compounds were compounds were the assay description contained 'Not Active'. No decoys structures were included. Duplicates and ambiguously labeled compounds were removed from the dataset. Compounds assayed recently were saved for final validations and removed from the dataset. These compounds are referred to as second validation set V2. The final dataset contained 1083 active and 910 inactive compounds. The dataset was randomly split up in train, test, and validation sets. The training set contained 60% of the active compounds and 60% of the inactive compounds. 182 of each active and inactive compounds served as training set. The remaining 434 compounds were defined as the first validation set VI .

[00132] The training set was used to train several machine learning algorithms. First, NULL-model based machine learning algorithms were built based on a single molecular parameter. For each molecular parameter the compounds were ranked according to the parameter values. Then, the area under the receiver-operator-characteristic curve (AROC) values were calculated using the roc_score function in scikit-learn. The parameters were sorted according to the AROC values in ascending order and feed successively into the model building algorithms, except for the naive Bayes BitVector (NBB V) model, where the molecular fingerprints were used exclusively as input. The standard parameters for other machine learning algorithms were used as implemented in the scikit-learn Python library except for the following options. For the decision tree machine learning algorithm and the random forest machine learning algorithm, max_features was set to the number of input features. For each set of features, the following machine learning algorithms were executed and applied to the test set: logistic regression (LR), naive Bayes (NB), decision tree (DT), random forest (RF), boosting (BO), and XGBoost. The machine learning algorithm with the highest accuracy was selected for further evaluation. The selected machine learning algorithm then was applied to the validation set VI and the second validation set V2.

[00133] The second validation set V2 contained detailed IC50 values for hERG.

Compounds with pIC50 values above 5 (corresponding to IC50 of 10 μΜ) were labeled as active and with pIC50 values below 5 were labeled as inactive. The quality of the prediction was evaluated in terms of prediction accuracy (AC), true-positive rate (TPR), false-positive rate (FPR), true-negative rate (TPN), false-negative rate (FPN), Kohen's Kappa (KK), the Fl score (F 1 ), the AROC, the correlation of the predicted class probability to be active with the pIC50 values, sensitivity, and specificity. Sensitivity can be expressed as TP/(TP+FN) and specificity can be expressed as TN/(FP+TN), where TP is the number of true positives, FP is the number of false positives, TN is the number of true negatives, and FN is the number of false negatives. Such performance metrics are well known in the art.

[00134] FIGS. 5A-5J illustrate ROC curves for an exemplary training set and test sets for exemplary machine learning algorithms, according to some embodiments of the present invention. More specifically, FIG. 5A illustrates ROC curves for a naive Bayes machine learning algorithm for the training set of Example 1 , and FIG. 5B illustrates ROC curves for that naive Bayes machine learning algorithm for the test set of Example 1. FIG. 5C illustrates ROC curves for a naive Bayes bitvectors machine learning algorithm for the training set of Example 1, and FIG. 5D illustrates ROC curves for that naive Bayes bitvectors machine learning algorithm for the test set of Example 1. FIG. 5E illustrates ROC curves for a decision tree machine learning algorithm for the training set of Example 1 , and FIG. 5F illustrates ROC curves for that decision tree machine learning algorithm for the test set of Example 1. FIG. 5G illustrates ROC curves for a random forest machine learning algorithm for the training set of Example 1 , and FIG. 5H illustrates ROC curves for that random forest machine learning algorithm for the test set of Example 1. FIG. 51 illustrates ROC curves for a boosting machine learning algorithm for the training set of Example 1, and FIG. 5 J illustrates ROC curves for that boosting machine learning algorithm for the test set of Example 1. Based on FIGS. 5A-5I, it can be understood that based on a given set of "actives" and "inactives," the ROC curve can express how well the two groups are separated from each other in respect to a continuous number, e.g., a predicted probability. A random number generator would be expected to produce a line along the diagonal of an ROC plot, indicating a mixture of active and inactive compounds. A perfect separation between active and inactive compounds would be expected to produce a line that extends from the lower left corner to the upper left corner to the upper right corner. Thus, based on FIGS. 5A-5I, it can be seen that the class probability leads to a significant separation of active and inactive compounds in the analyzed dataset.

[00135] FIGS. 6A-6E illustrate exemplary performance measures of exemplary machine learning algorithms, according to some embodiments of the present invention. More specifically, FIGS. 6A-6E illustrate prediction accuracy (AC), true-positive rate (TPR), true- negative rate (TNR), Kohen's Kappa (KK), sensitivity, and specificity for the following respective machine learning algorithms using the training set of Example 1 : logistic regression, naive Bayes, decision tree, random forest, and boosting. Table 3 lists different performance measures of these machine learning algorithms (MLAs), ordered by prediction accuracy (AC), for the validation set of Example 1. The quality of a classification can be measured by different metrics that provide information about different aspects of the classification. For example, the area under the ROC curve can be based on the class probability that underlies a classification (assigning a compound to a predicted class), whereas AC, sensitivity, and specificity are based on a classification. For example, a low AC combined with a high AROC can indicate that a different cutoff can be used for the classification. Table 3. Performance Measures for Machine Learning Algorithms

[00136] FIGS. 7A-7C illustrate ROC curves for an exemplary training set, test set, and validation set for exemplary machine learning algorithms, according to some embodiments of the present invention. More specifically, FIG. 7A illustrates respective ROC curves for the following machine learning algorithms for the training set of Example 1 : logistic regression (LogReg), naive Bayes, decision tree (D-Tree), random forest, boosting, and naive Bayes bitvector (NB-BitVect). FIG. 7B illustrates respective ROC curves for those same machine learning algorithms for the test set of Example 1. FIG. 7C illustrates respective ROC curves for those same machine learning algorithms for the validation set of Example 1.

[00137] FIG. 8 illustrates exemplary prediction accuracies for an exemplary training set, test set, and validation set for exemplary machine learning algorithms, according to some embodiments of the present invention. More specifically, FIG. 8 illustrates the respective accuracies of the following machine learning algorithms for the training set, test set, and validation set of Example 1 : logistic regression (LogReg), naive Bayes, decision tree (D- Tree), random forest, boosting, and naive Bayes bitvector (NB-BitVect). Based on FIG. 8, it can be understood that different models perform well in the validation set. The graphs compare the quality of the fit and off-sample performances (test and validation) of different models. [00138] FIG. 9 illustrates histograms showing exemplary predicted or actual numbers of active (1.0 on x-axis) and inactive (0.0 on x-axis) compounds in an exemplary test set with respect to different exemplary machine learning algorithms, according to some embodiments of the present invention. The histogram activities shows the actual distribution. More specifically, FIG. 9 illustrates histograms showing exemplary predicted numbers of active and inactive compounds for the test set of Example 1 for the following machine learning algorithms: boosting, decision tree (D-Tree), logistic regression (LogReg), naive Bayes bitvector (NB-BitVect), naive Bayes, and random forest. Additionally, the lower left plot of FIG. 9 shows the actual activities of the compounds for the test set of Example 1. Similar conclusions as from FIG. 7 plus the cutoff that has been applied (0.5) leads to classifications with accuracies of above 0.8 in the validation set. FIG. 9 shows the raw number of compounds respectively classified as actives and inactives as compared to the actual number of actives and inactives in the dataset ('activity'). Random forest and decision tree can be seen to predict the correct numbers.

[00139] FIGS. 10A-10G illustrate exemplary performances of different exemplary machine learning algorithms with respect to an exemplary validation set, according to some embodiments of the present invention. Compounds with IC50 of less than or equal to 10 μΜ were considered "active." The left-most panels indicate an exemplary probability to be active, the middle panels indicate an exemplary corresponding classification over the experimental pIC50 values, and the right-most panels are corresponding ROC curves. FIG. 10A provides such information for the logistic regression (LogReg) machine learning algorithm. FIG. 10B provides such information for the naive Bayes machine learning algorithm. FIG. IOC provides such information for the decision tree (D-Tree) machine learning algorithm. FIG. 10D provides such information for the random forest machine learning algorithm. FIG. 10E provides such information for the boosting machine learning algorithm. FIG. 10F provides such information for the naive Bayes bitvector (NB-BitVect) machine learning algorithm. FIG. 10F provides such information for the Consensus Scoring (CS) machine learning algorithm, which aims to gain more robust and more accurate results for off-sample instances (compounds that have not been used in training and test sets).

[00140] FIG. 1 1 illustrates an exemplary heatmap of the mutual correlation coefficients of all features in an exemplary training set, more specifically, the training set of Example 1, according to some embodiments of the present invention. From FIG. 1 1 , it can be seen that some of the molecular parameters of the training set are strongly correlated with each other and a feature selection or linearization may be useful to be applied.

[00141] FIGS. 12A-12H illustrate exemplary ROC curves for an exemplary training set and test set for exemplary machine learning algorithms using isomapping, according to some embodiments of the present invention. Isomapping is a distance based method that learns and simplifies the structure of the input data. Isomapping aims to conserve the distances of instances in a high dimensional space by using a smaller number of dimensions. The isomap vectors can be used an input for machine learning. For example,, FIG. 12A illustrates ROC curves for a naive Bayes machine learning algorithm using isomapping to modify the training set of Example 1 , and FIG. 12B illustrates ROC curves for that naive Bayes machine learning algorithm using isomapping to modify the test set of Example 1. FIG. 12C illustrates ROC curves for a decision tree machine learning algorithm using isomapping to modify the training set of Example 1 , and FIG. 12D illustrates ROC curves for that decision tree machine learning algorithm using isomapping to modify the test set of Example 1. FIG. 12E illustrates ROC curves for a random forest machine learning algorithm using isomapping to modify the training set of Example 1 , and FIG. 12F illustrates ROC curves for that random forest machine learning algorithm using isomapping to modify the test set of Example 1. FIG. 12G illustrates ROC curves for a boosting machine learning algorithm using isomapping to modify the training set of Example 1, and FIG. 12H illustrates ROC curves for that boosting machine learning algorithm using isomapping to modify the test set of Example 1. The plots visualize the fit of the training data and the performance in the test set. With more features used as input better fits are and predictions are possible in general.

[00142] FIGS. 13A-13E illustrate exemplary performance measures of exemplary machine learning algorithms using isomapping, according to some embodiments of the present invention. The blue background spans minimum and maximum, mean (black x) and median (white +). More specifically, FIGS. 13A- 13E illustrate prediction accuracy (AC), true-positive rate (TPR), true-negative rate (TNR), Kohen's Kappa (KK), sensitivity, and specificity for the following respective machine learning algorithms using isomapping to modify the training set of Example 1 : logistic regression, naive Bayes, decision tree, random forest, and boosting. From these plots, it can be understood that isomapping can lead to similar predictions as using the raw features, but potentially with enhanced accuracy. Table 4 lists different performance measures of these machine learning algorithms (MLAs) using isomapping, ordered by prediction accuracy (AC), for the validation set of Example 1 modified using isomapping.

Table 4. Performance Measures of Machine Learning Algorithms Using Isomapping

[00143] FIGS. 14A-14C illustrate ROC curves for false positives for an exemplary training set, test set, and validation set for exemplary machine learning algorithms, according to some embodiments of the present invention, without using isomapping. More specifically, FIG. 14A illustrates respective ROC curves for false positives for the following machine learning algorithms for the training set of Example 1 : logistic regression (LogReg), naive Bayes, decision tree (D-Tree), random forest, boosting, and naive Bayes bitvector (NB-BitVect). FIG. 14B illustrates respective ROC curves for false positives for those same machine learning algorithms for the test set of Example 1. FIG. 14C illustrates respective ROC curves for false positives for those same machine learning algorithms for the validation set of Example 1.

[00144] FIG. 15 illustrates ROC curves for compounds in an exemplary training set for a NULL machine learning algorithm, according to some embodiments of the present invention. More specifically, FIG. 15 illustrates ROC curves that were generated by sorting compounds according to the values of individual molecular parameters, such as LogP, molecular weight, and the like, in ascending order (descending when AROC was negative). From these plots, it can be understood that single molecular parameters can have predictive power. Models that are built on a plurality of such molecular parameters (e.g., machine learning algorithms that are trained on a plurality of such molecular parameters) thus can have improved performance relative to those that are built on or trained on a single one of such molecular parameters. [00145] FIGS. 16A-16D illustrate performance of an exemplary 3C model for assessment of torsadogenic potential for a blinded set of blockers, according to some embodiments of the present invention. FIGS. 16A-16C respectively illustrate scatter plots of experimental and predicted pIC50 values for (A) hERGl, (B), Na _v l .5, and (C) Ca _v l .2 for the training set (+,□) and validation set (ο,·) of Example 1. An exemplary selection of compounds is highlighted. Experimental data (IC50 values for hERGl and Na _v 1.5 converted to pIC50) for the training set and validation set were adapted from Kramer et al., "MICE models: Superior to the HERG model in predicting Torsade de Pointes," Scientific Reports 3: 2100, pages 1-7 (2013) and in the Supplementary Material thereto, the entire contents of which are incorporated by reference herein. FIG. 16D illustrates exemplary performance of logistic regression models in terms of true positive rate (+TdP) and true negative rate (-TdP). Evaluation was based on 9 random selections of training sets for +TdP and 16 random selections of training sets for - TdP. The error bars in FIG. 16D indicate the standard deviations. Random-generation predictor set is shown for comparison. Y-axis displays percentage for true predictions of torsadogenic blockers and X-axis for "neutral" or -TdP blockers. From these plots, it can be understood that using hERG in combination with other channels, e.g., NaV and CaV channels, can lead to significantly improved predictions of cardiotoxicity.

6.4.2 Example 2

[00146] Using the software packages described above, standard evocations and modules were instantiated using the following code:

'Set working directory'

import os,sys

PATH="/home/swacker/Documents/Notebooks/ModeIing/016-hERG-mo del- publication"

os.chdir("%(PATH)s/301 -Validation" %vars())

sys.path.append("%(PATH)s/lib" %vars()) from modeling import *

import pickle

%pylab inline

plt.style.use('ggplot') #Set seeds for random number generator.

np.random.seed( 12345)

random.seed(12345)

#Options

SaveFigOpt={'prefix ^l :'hERG-Validation-','path':'./figures'}

#PlotDemo()

[00147] Additionally, the interactive namespace was populated from numpy and matplotlib.

[00148] The compounds to be analyzed (which also can be referred to as ligands) were loaded from respective SMILES (smi) file, that contains SMILES codes and unique IDs for each compound. For example, the SMILES (.smi) files were read and converted to a pandas (python module) DataFrame instance which is a table like object. Then molecular parameters for those compounds were calculated and included into a table such as partially reproduced in Table 5. These molecular parameters were later used by the machine learning algorithms to classify the compounds. This example uses the same dataset of compounds to validate the hERG machine learning algorithms as was used in Example 1, e.g., compounds from Kramer et al., "MICE models: Superior to the HERG model in predicting Torsade de Pointes," Scientific Reports 3: 2100, pages 1-7 (2013) and in the Supplementary Material thereto, the entire contents of which are incorporated by reference herein. The pIC50 values of the compounds used in this Example are shown in FIG. 20. Additionally, machine learning algorithms such as described above in Example 1 were loaded.

Table 5. Compounds

[00150] The following code was used to add the molecular parameters present in rdkit to the information about the molecules:

ligands = AddMolProp(ligands) #Adds all molecular properties present in rdkit to the dataframe

ligands

[00151] Exemplary molecular parameters for the compounds are listed in Table 6. Table 7 below lists the meaning of the molecular parameter of Table 6.

[00152] Table 8 lists additional molecular parameters that may or may not appear in Table 6, and also or alternatively can be used. Certain information in Table 8 adapted from Wicker et al., "Will it crystallize? Predicting crystallinity of molecular materials," CrystEngComm 17: 1927-1934 and supporting information, DOI: 10.1039/C4CE01912A (2014), the entire contents of which are incorporated by reference herein.

[00153] The following code was used to load the following machine algorithms (models): boosting (BO), decision tree (DT), logistic regression (LR), naive bayes (LB), and random forest (RF):

models = LoadModels('../201-Model-AllFeatures/out/*.p') #Loads models models #Contains vector of features required by the model, a unique ID of the model, an info string and the acctual model.

#all this information is actually stored in the model as attributes model.ID, model.type, model.X, model.info.

../201-Model-AllFeatures/out/hERG-Model-AllFeatures-BO-model .p

../201-Model-AllFeatures/out/hERG-Model-AllFeatures-DT-model .p .7201 -Model- AllFeatures/out/hERG-Model-AllFeatures-LR-model.p

.7201 -Model- AllFeatures/out/hERG-Model-AHFeatures-NB-model.p

.7201-Model-AllFeatures/out/hERG-Model-AllFeatures-RF-model. p

[00154] The models need a DataFrame with the columns listed in the attribute X. The function AddMolProp() adds all molecular parameters present in the RDKit python package. The current molecular parameters used in RDKit are provided elsewhere herein.

[00155] Applying each of the machine learning algorithms (models) to the dataframe (molecular parameters of compounds such as listed in Table 5) outputs a prediction containing the class probability to be active, the predicted classification, and the compound ID in the index and as a separate column. The output also can include an indication of which model was used, a unique model-ID (e.g., 6c61f5e5-5378-4bbe-835b-05fDcddb4742 for the boosting algorithm), and an information string, e.g. the target(s) for which the machine learning algorithm was trained. Table 9 lists exemplary outputs of the boosting machine learning algorithm. The function ScoreModels() applies all models to the prepared DataFrame and returns a DataFrame with the classification and the corresponding class probabilities such as shown in Table 9.

Table 9. Outputs for Boosting Machine Learning Algorithm

[00156] Such predictions were validated by comparing the classification and the class probabilities from the models with actual experimental data. Various metrics were to assess different aspects of the quality of the predictions. The following code was used to load experimental data that was in the form of a CSV file, although it should be understood that any other suitable file format can be used:

EXPDATA=pd.io.parsers.read_csv("../data/Nature-Brown2013-Act ivities.csv") #Read csv file with experimental data

EXPDATA.head()

[00157] Exemplary experimental data for certain compounds is listed in Table 10.

Table 10. Exemplary Experimental Data

[00158] The pIC50 of each compound was calculated based on data such as shown in Table 10 using the following code, and the calculated pIC50s are shown in Table 11. For example, in order to compare the classification, experimental values are loaded from a comma separated values (cvs) file which are provided as pIC50s.

EXPDATA['pIC50 ^, ]=IC50_to_pIC50(EXPDATA['HERG-IC50']) #It is better to work with pIC50 values instead of IC50 EXPDATA.index = EXPDATA['ID']

pIC50s=EXPDATA[['ID7pIC50']]

pIC50s #The index must contain the unique compound ID as workaround for the bug in pd.DataFrame.join()

Table 11. pIC50s.

[00159] The validation and evaluation of the quality of the prediction can be performed by analyzing the correlation of the class probabilities with the experimental values, the receiver- operator-characteristic (ROC) curve. The class probabilities can be used to classify the compounds. In some embodiments, a class probability of less than 0.5 is labeled as 'inactive' against a given target and larger than or equal to 0.5 as 'active.' Furthermore, multiple metrics can used to measure the quality of the classification.

[00160] An example of an analysis is shown in FIGS. 17A-17J. The scatter plots (left plots in FIGS. 17A-17J) show the class probability to be active over the experimentally validated pIC50 values. The value for the Pearson correlation coefficient is shown in the figure (CC). The graphs (left) show receiver operator characteristic curves according to different cutoffs applied to the experimental data, that defines 'active' and 'inactive' compounds. For the model building a cutoff of 5 (pIC50) has been used. For the classification always a cutoff of 0.5 (class probability) has been applied so far. For example, a compound with the class probability 0.8 can be classified as 'active'. The final dataframe contains the predictive power of the models according to the individual cutoffs and different metrics:

[00161] The following code was used to validate the predictions:

[00162] FIGS. 17A-17J illustrate probabilities to be active and ROC curves for an exemplary validation set for different machine learning algorithms, according to some embodiments of the present invention. More specifically, FIG. 17A illustrates probabilities to be active using the boosting machine learning algorithm, and FIG. 17B illustrates ROC curves for the boosting machine learning algorithm for different cutoffs. FIG. 17C illustrates probabilities to be active using the decision tree machine learning algorithm, and FIG. 17D illustrates ROC curves for the decision tree machine learning algorithm for different cutoffs. FIG. 17E illustrates probabilities to be active using the logistic regression machine learning algorithm, and FIG. 17F illustrates ROC curves for the logistic regression machine learning algorithm for different cutoffs. FIG. 17G illustrates probabilities to be active using the native Bayes machine learning algorithm, and FIG. 17H illustrates ROC curves for the native Bayes machine learning algorithm for different cutoffs. FIG. 171 illustrates probabilities to be active using the random forest machine learning algorithm, and FIG. 17H illustrates ROC curves for the random forest machine learning algorithm for different cutoffs. The scatter plots (FIGS. 17A, 17C, 17E, 17G, and 171) show the class probability to be active over the experimentally validated pIC50 values. The value for the pearson correlation coefficient is shown in the figure (CC). The ROC graphs (FIGS. 17B, 17D, 17F, 17H, and 17J) show receiver operator characteristic curves according to different cutoffs applied to the experimental data, that defines 'active' and 'inactive' compounds. For the model building a cutoff of 5.5 has been used. For the classification always a cutoff of 0.5 has been applied so far. In one example, a compound with the class probability 0.8 is classified as 'active'. In other examples, cutoffs of 3, 4, and 5 can be used.

[00163] Table 12 includes performance measurements of the various machine learning algorithms (MLAs) for different cutoffs.

Table 12. Performance Measurements.

[00164] The predictions of the different models can be combined to obtain a consensus- score CS which can be superior over individual scores in many applications. In this example, the CS is simply the average of the class probabilities, which was calculated using the following code:

CS = pd.concat(predictions).groupby('ID').mean()

CS['Classification']=Classify(CS['Probability'],0.5)

def

Validate_prediction(pIC50s,probabilities,cutoffs=[5.5],inver se=False,pIC50colNa me- pIC50',title=None):

This function validates the prediction according to a set of cutoffs used for defining 'active' and 'inactive' compounds.

pIC50s and prediction are pandas. DataFrames containing the values and the compounds-IDs import pandas as pd

from sklearn.metrics import roc_curve,confusion_matrix,roc_auc_score

Colors = get_colors(len(cutoffs)

results=[]

plt.figure().set_size_inches((8,4))

^Convert cutoffs to iterable list in case a float/int cutoff has been used as input #if not isiterable(cutoffs): cutoffs = [cutoffs]

for i,cutoff in enumerate(cutoffs):

color = Colors.next()

data = Join(probabilities,pIC50s)#.dropna()

data['Class'] = Classify(data[pIC50colName],cutoff)

auroc = roc_auc_score(data['Class'],data['Probability'])

plt.subplot(121)

[00165] FIGS. 18A-18B respectively illustrate probabilities to be active and ROC curves for an exemplary validation set for a consensus among different machine learning algorithms, namely a consensus prepared using the above code, according to some embodiments of the present invention. [00166] Table 13 includes performance measurements of a consensus the various machine learning algorithm for different cutoffs, prepared using the above code.

[00167] This example also shows how a model (trained machine learning algorithm) can be visualized and interpreted. In general, the interpretation of the models is not straightforward because many input parameters are used which are not necessarily linearly independent. An example of the boosting model on a specific dimension of the input vector is shown in FIG. 21 A. However, similar input structures can be compared with small structural changes that lead to a comparably large shift in the class probability. FIG. 21B shows an example of a pair of an active compound (left) and inactive (right) where a change of a chemical group lead to a shift of 0.287 in the class probability. The comparison of the structures shows that an exchange of a aromatic ring with a carboxyl group has been replaced by a an aliphatic ring with two nitrogens. Reasoning how this structural change causes a change in the activity is beyond the scope of the current models. However, this analysis identifies pairs of compounds that can be studied with structure based methods like docking or molecular dynamics simulations.

Table 13. Performance Measurements for Consensus.

6.4.3 Example 3

[00168] Some embodiments of the present systems and method s provide for

"rehabilitation" or "redesign" of compounds that are predicted to include molecular parameters that are likely to be cardiotoxic, e.g., are likely to block the hERGl channel or two or more of the channels disclosed herein. As used herein, "rehabilitation" can mean reducing one or more side effects (e.g., decreasing a hERGl blocking affinity) while maintaining efficient binding to the desired target. In one non-limiting example, azole-based antifungal drugs can be at least partially rehabilitated while at least partially retaining their efficacy. For example, the systems and methods provided herein can be used to perform one or more of the following steps:

[00169] 1) Predict the likelihood that a compound will block one or more, two or more, or all three of hERG, N av or Cav ion protein channels,

[00170] 2) Identify one or more therapeutic effect sites responsible for the desired target activity;

[00171] 3) Re-design one or more molecular parameters, e.g., compound moieties, predicted to be responsible for blocking one or more, two or more, or all three of hERG, N _a v or Cav ion protein channels, and that also are predicted to dispensable (e.g., not necessary) for binding to the desired target,

[00172] 4) Perform differential analysis for on- and off- target interactions and perform fragment-based drug modification.

[00173] For example, the present systems and methods can be used to predict interactions between compounds and one or more, two or more, or all three of hERG, Ca _v 1.2 and Na _v l .5 channels. As provided elsewhere herein, such systems and methods can facilitate rapid and accurate evaluation of drug candidates, significantly reducing potential risks in drug development, e.g., based on molecular parameters of those compounds (e.g., solubility, lipophilicity, molecular weight, number of specific atoms, molecular fingerprints and other molecular and structural properties). Such molecular parameters can serve as variables used for supervised learning to against experimental activity data (e.g., ChEMBL database) to create predictive models. The Pearson correlation coefficients in the validation set between experimental and predicted pIC50 of hERG/Na _v l .5/Ca _v 2.1 model are 0.78/0.6/0.51 respectively (with saquinavir as clear outlier in all datasets) with blinded predictive power in torsadogenic activity of ~70 % for identification of true-positives (torsadogenic) and 49 % of true-negatives (non-torsadogenic). Therefore, the preliminary model shown in FIGS. 16A- 16D, discussed elsewhere herein, already supersedes single-channel based predictive platforms.

[00174] The introduction of azoles and derivatives revolutionized the treatment of fungal and trypanosomosis infections. For example, miconazole, an imidazole antifungal agent, is directly associated with acquired QT prolongation and ventricular arrhythmias. For further details, see Kikuchi et al., "Blockade of HERG cardiac K+ current by antifungal drug miconazole," British Journal of Pharmacology 144: 840-848 (2005), the entire contents of which are incorporated by reference herein. This is one of many examples of antifungal agents that are in clinical use despite apparent risks to patients. Our preliminary data from the previous grant on hERG l blockade supported by the Heart and Stroke Foundation (Alberta, N WT) demonstrate that it possible to have compounds based on the structure of miconazole with substantially reduces effects on APs in neonatal mice cardiomyocytes Some of these compounds have greater or comparable antifungal activity than miconazole itself, thus substantially reducing potential risks to patients. Azole derivatives are often prescribed in cases of the systemic fungal infection with IV delivery route posing substantial risks because of hERG, Na _v and likely Ca _v blockade.

[00175] For example, FIGS. 19A- 19D illustrate probabilities to be active on hERG (light grey) with respect to antifungal activity (dark grey) for an exemplary set of compounds, according to some embodiments of the present invention. FIGS. 19A- 19D were prepared using steps analogous to those described above in Examples 1 and 2, but for an exemplary set of antifungal compounds.

[00176] With respect to the rehabilitation of established compounds that display unwanted interactions with cardiac targets, the compound can be as a starting material and different molecular parameters that are predicted to be cardiotoxic, including but not limited to structural features, can be blocked, cleaved or otherwise altered and the resulting modified compound then re-analyzed as provided herein so as to predict cardiotoxicity. Optionally, the compound or the modified compound, or both, can be chemically synthesized and assayed, e.g., tested for desired activity, such as antifungal activity.

6.4.4 Example 4

[00177] Appendix A attached hereto is incorporated by reference herein and forms part of the present disclosure, and relates to an example for predicting cardiotoxicity with respect to the hERG channel, using as input the same test set as described above with reference to Examples 1 and 2. More specifically, Appendix A includes hERG python code, with relevant inputs, example of SMILES input for a training set, and comparison to two other programs - Schrodinger Inc. and web-based server for QSAR hERG prediction. In this example, developer's bits of explicit coding are omitted, and outputs are provided.

6.4.5 Example 5

[00178] Appendix B attached hereto is incorporated by reference herein and forms part of the present disclosure, and relates to an example for predicting cardiotoxicity with respect to the Navl .5 channel, using as input the same test set as described above with reference to Examples 1 and 2. More specifically, Appendix B includes Navl .5 python code, with relevant inputs, and example of SMILES input for a training set. In this example, developer's bits of explicit coding are omitted, and outputs are provided.

6.4.6 Example 6

[00179] Appendix C attached hereto is incorporated by reference herein and forms part of the present disclosure, and relates to an example for predicting cardiotoxicity with respect to the Cavl .2 channel, using as input the same test set as described above with reference to Examples 1 and 2. More specifically, Appendix C includes Cavl.2 python code, with relevant inputs, and example of SMILES input for a training set. In this example, developer's bits of explicit coding are omitted, and outputs are provided.

6.4.7 Example 7

[00180] The following example relates to a quantitative structure activity relationship (QSAR) model for the voltage gated potassium channel, known as hERG. The model is based on the XGBoost algorithm and trained on publicly available data from the ChEMBL database. The model performs well on compounds that are similar to the training set with a coefficient of determination of up to R ² =0.8 and allows to quantitatively estimate the potential of novel chemical scaffolds to block hERG. The example employs a boosting tree algorithm for the machine learning algorithm to build the QSAR model. The purpose of the model is to quantitatively estimate the potential of novel chemical scaffolds in respect to hERG. In alternative embodiments, the methods presented below are applicable to other supervised learning tasks.

We used the currently most advanced publicly available boosting tree algorithms, which is known as extreme gradient boosting (XGBoost) to build a predictive model based on the content of the ChEMBL database. XGBoost is a parallel tree learning algorithm to build boosting tree classification, regression and ranking models (Chen et al., 2016, "XGBoost: A Scalable Tree Boosting System," arXiv:1603.02754 [cs.LG], available at

https://arxiv.org/abs/1603.02754). In recent years, ensemble methods such as random forest have been successfully applied to both classification and regression problems. Ensemble tree models use tree-like structures to classify instances or fit arbitrary functions. The fit is done based on attributes of the instances called features. An instance can be a chemical compound and the features may be molecular descriptors such as the weight, number of hydrogen bond donors. Each tree consists of a number of branches where the dataset is split corresponding to a chosen feature and a split value. The number of splits is often set to a smaller number what limits the ability of a single tree to fit a function accurately. But if many trees (ensembles) are combined very accurate classifiers can be designed. In boosting algorithms each tree aims to fit the instances better that are missed by the previous trees.

[00181] Our model is based on open source software and trained on publicly available data from ChEMBL database (Gaulton et al., 2012, "ChEMBL: a large-scale bioactivity database for drug discovery," Nucleic Acids Res., 40, Dl 100-7). We used the open source toolkit for cheminformatics RDKit (Landrum, "RDKit: Open-source cheminformatics" available at http://www.rdkit.org) to handle chemical structures, reactions and transformations and to calculate molecular descriptors and similarities.

[00182] The ChEMBL database (Dec 2015) was queried for bioactivities for the target 'CHEMBL240' the potassium voltage-gated channel subfamily H member 2 (hERG) using the python chembl-webresource-client. The ChEMBL database at the date of evaluation contained single protein bioactivities for 6018 different targets. hERG ranked 54 for the number of available bioactivities and 1 st when the query was restricted to IC50 values. Assay descriptions that were included were: 'Inhibition of human ERG', 'Binding affinity to human ERG', 'Inhibition of human ERG at 10 uM' and 'Inhibition of human ERG channel'. Furthermore, the query was restricted to the bioactivity type 'IC50', the assay type 'Β', the operator '=' and the target confidence '9'. Then the dataset was restricted to assays that contain at least 6 activities for different compounds in order to increase the consistency of experimental data in the training set. Only items with a specified IC50 value were selected.

[00183] Duplicate compounds in the dataset were identified based on compound similarities. In case the same value was reported multiple times the items were grouped. If multiple items with different IC50 values remained the record with the minimal IC50 value was selected after checking the values in the original publications. In some cases the reported 1C50 values was transcribed wrongly into ChEMBL. In this cases the values were corrected. In the following the four different test sets are described.

[00184] From the curated ChEMBL dataset that contained 699 compounds 100 compounds were selected randomly and saved as first external test set (Testl). This set contained experimental values obtained by the same 40 assay-IDs as the training set. Test set 2 (Test2) contained 55 compounds with IC50 values measured with the same protocol and were reported by Kramer et al. (Kramer et al., 2013, "MICE models: superior to the HERG model in predicting Torsade de Pointes," Sci Rep., 3, 2100). Smiles codes for the compounds were generated with "molconvert" from ChemAxon's MarvinSketch. The records belonging to assays that contain between 2 and 5 compounds were defined as test set 3 (Test3), in total 155 compounds. The remaining 73 entries were defined as test set 4 (Test4). Every record in Test4 had a unique ChEMBL assay ID. Compounds with a pIC50 > 5 were defined as 'active' compounds.

6.4. 7. 1 Feature generation and feature sets

[00185] The smiles codes for the compounds were obtained from the ChEMBL database. The codes were used to generate RDKit molecule objects (RDKit version: 2015.03.1) and standardized using molvs (version 0.03) package. Hydrogen atoms were added. The resulting structures were used to calculate all molecular descriptors that were implemented in RDKit.

6.4. 7.2 Predictive Features

[00186] All molecular descriptors with zero variance were removed. The remaining features were filtered, so that no two features had a mutual absolute pearson correlation above 0.99 based on data in the training set. The final set contained 150 descriptors. As mentioned above we used a cutoff of pIC50 > 5 to define 'active' compounds. Based on this assignment we analysed the predictive power of individual the molecular descriptors using the receiver-operator-characteristic (ROC) and the corresponding area under the ROC (AROC) for each descriptor. The molecular descriptors with an AROC > 0.55 are referred to as 'predictive features' comprising 55 features.

6.4.7.3 Normal Features

[00187] The predictive features were subjected to a test for normality. Features that met the criterion for normality were selected. The 'normal features' are a subset of the predictive features. More information can be found in the supplementary information. The normal features comprised 48 features.

6.4.7.4 Similarity based Features

[00188] The compound similarities were calculated with RDkit. We used morgan fingerprints (nBits=1024, radius=2) and the Tanimoto similarity score of the standardized molecules. For each compound the four most similar compounds in the training set were identified. The similarities as well as the pIC50 values of the corresponding compounds were added to the database as features. The similarity of the most similar compounds is named 'simO', the second best 'siml ' and so on. The corresponding pIC50 values 'valueO', 'valuel ', and so on. These features were based on the K-nearest neighbors algorithm and the observation that similar compounds have higher probability to have similar pIC50 values and used to predict the activity of a compound based on the activity of the most similar compounds, wherein the eight features are referred to as similarity based features.

6.4.7.5 2D Pharmacophore Features

[00189] A set of chemical features with topological (2D) distances between them produce 2D pharmacophore features. We used the feature definitions from Gobbi and Poppinger (Gobbi et al., 1998, "Genetic optimization of combinatorial libraries," Biotechnol. Bioeiig., 61 , 47-54) as implemented in RDKit. The compounds in the training set were converted to 2D fingerprints represented as bit-vectors. Each element of the bitvectors served as a feature for the machine learning algorithm, while keeping bits that were activated at least 100 times. 6.4.7.6 Feature sets

[00190] Different sets of features were evaluated. The following list indicates each set and assigns a number to serve as reference: 1 : PredictiveFeatures, 2: NormalFeatures, 3:

SimilarityFeatures, 4: Pharm2DFeatures, 5: PredictiveFeatures + SimilarityFeatures, 6: PredictiveFeatures+Pharm2DFeatures, 7: PredictiveFeatures + Pharm2DFeatures +

SimilarityFeatures, 8: NormalFeatures + Pharm2DFeatures.

6.4.7.7 Model selection

[00191] For use XGBoost algorithm, a stepwise protocol was used. First, cross-validation was used to identify the most dominant features and parameters. Learning curves were generated to characterize the dependency of the model performance in respect to the size of the dataset as well as to identify the optimal number of iterations for the generation of the final model.

[00192] Since XGBoost can be quite sensitive to the parameters used to fit the model, we performed a grid search to find a combination of parameters and features with best cross- validated performance. The parameters and corresponding values for the grid search were: 'colsample_bytree' : 0.3/0.5/0.8/0.9, 'subsample': 0.2/0.5/0.8/0.9, 'max_depth' : 3/4/5/6/8 and 'eta': 0.001/0.01/0.05/0.1. The dataset was shuffled and afterwards divided into 10 mutually exclusive groups. For each fold, a model was trained using compounds from 9 groups and the remaining group was used for validation. The RMSE for training and validation set was monitored. Once the validation error did not increase within 1000 iterations, the fitting was stopped. The results of the best iteration were stored for the validation set. The stored data was used to calculate the cross-validated coefficient of determination Q2, AROC, and other metrics. This procedure was repeated for each combination of features and parameters as set forth above.

[00193] For the selected parameters and features, learning curves were generated. The learning curve monitors the on- and off-sample performance over the size of the training set. The learning curves were generated by averaging the performance of 20 repetitions. Random samples of the complete training served as validation set and fractions of the remaining compound were used to fit the model. In some embodiments, the size of the validation set was 10% of the complete training set. The number of regression trees was controlled for each curve and varied between 200 and 1,500. [00194] The final model was trained using the complete training set and a predefined number of iterations. The final model trained using 900 iterations, which was slightly higher than the optimal number identified with the learning curves. This was done to compensate for the slightly higher number of training samples when using the full dataset. The parameters for the final model were: colsample_bytree: 0.9, eta: 0.01 , max_depth: 3, subsample: 0.2.

6.4.7.8 Y -random ization

[00195] The Y-randomization test prevents high model performance due to chance correlation. We used y-randomization, i.e. randomly shuffle the target variable and repeat the model selection and fitting procedure. In some embodiments, Y-randomization was used to eliminate chance correlation among the generated model. In some embodiments, no Y- randomization was used.

[00196] A stepwise protocol was used to select the model parameters and features. 10-fold cross-validation was used to identify the most dominant feature sets (Fl -8) and hyper- parameters. The optimal number of iterations was estimated based the shape of learning curves. Afterwards, the model was applied to pre-defined test sets to estimate the model's ability to generalize and to estimate the off-sample performance. The method of model selection

6.4.7.9 Predictive power of individual features

[00197] We performed a ranking of all features by calculating the area under the ROC- curve (AROC). We applied defined positive class as compounds with pIC50 > 5 otherwise the compounds belong to the negative class. The sign of the feature was changed when the AROC was negative, ensuring a ranking according to the actual discriminative power. A histogram of the AROC values for the molecular descriptors is shown in FIG. 22. Only five descriptors generated a curve with an AROC above 0.65. The highest value (0.67) was obtained by the molecular LogP value (MolLogP), wherein in some embodiments, a valid model has at least have an AROC of 0.67. We analyzed the predictive power of the remaining features by calculating the area under the ROC curves (AROC).

[00198] The ROC curves of all groups of features are shown in FIGS. 23A-D The most predictive molecular descriptors was the molecular logP value. The valueX features, i.e prediction based on the pIC50 values of the most similar compounds, generate a highly significant ROC-curve. In contrast the simX features are essentially random predictions. This is expected since the maximal similarity to the training set does not carry information about activity per se. The simX features are still meaningful, since these feature can serve as weighting factors and contribute to a better prediction for models that are limited to the similarity based features.

6.4.7.10 Model selection

[00199] Steps of the method of selecting a model for predicting cardiotoxicity of a compound are shown in FIG. 24, according to some embodiments of the present invention.

[00200] We tested 320 different sets of parameters times 8 different sets of features. The best set of features according to both the mean cross-validated R2 (Q2) score and cross- validated AROC (cvAROC) was feature set 6, the combination of the predictive molecular descriptors and the 2D pharmacophore features. This combination performed best for a broad range of parameters on both metrics (FIGS. 25A-D). The median Q2 of feature set 6 was 0.66 followed by feature set 8 with 0.65. The worst set was feature set 3 with 0.49. The maximum value was 0.67 by feature set 6. From feature sets F1 -F4 the 2D-pharmacophore features (F4) did perform best.

[00201] The individual similarity based features (value0-4) showed the highest AROC values without resulting in over-all improvement. The maximum AROC using only the similarity based features was 0.8 compared to 0.78 gained by the feature 'valueO' only. The median Q2 values was 0.49 with a standard deviation of 0.018. Together, with the predictive features (F5) the median Q2 value increased to 0.63 compared to 0.61 using only the predictive features (Fl), while adding them to the predictive features and the pharmacophore features decreased the performance (F7).

[00202] As shown in FIGS. 26A-D train and test error had similar values for less iterations. For more iterations both curve decreased to lower values due to a better fit of the training data. For all curves the validation curve had a negative slope, indicating that the predictive power increased with the size of the training set.

[00203] The off-sample error stopped decreasing at around 800 iterations. For this set of parameters and the dataset at hand, the optimal number of iterations was around 800. Based on this results we fitted the final model using 900 iterations taking into account the slightly bigger size of the training set when using all compounds. FIGS. 27A-B shows the fitted training data and the corresponding ROC curve. The RMSE was around 0.5 units and the AROC was 0.90. 6.4.7.11 Model evaluation

[00204] After identification of the best features and parameters we trained the model using the complete training set of around 700 compounds. Afterwards, we evaluated the off- sample performance based on 4 different test sets. The sets differed in their origin, size and composition. Test set 1 (Testl) was a randomly chosen subset removed from the training set before modeling. Test set 2 (Test2) comprised 55 compounds with experimental data from the same protocol (Kramer et al., 2013, "MICE models: superior to the HERG model in predicting Torsade de Pointes," Sci Rep., 3, 2100). Initially, test sets 3 (Test3) and 4 (Test4) were not dedicated as test sets because, especially for Test4, the number of assays and therefore the expected inconsistency of experimental factors is high. However, these sets are still valuable to probe the limitations of our model and useful to define a reasonable applicability domain.

[00205] For Testl the final model demonstrated reasonable performance to estimate a compound's pIC50 value. The correlation between the predicted (score) and the experimental pIC50 values is 0.84 the R ² score is 0.7, as shown in FIGS. 28A-C. The relative ranking performance was quantified with the AROC which was 0.88. The root mean squared error (RMSE) was 0.7. The similarity between the compounds in Testl and the training set was high compared to the other test sets FIG 29B. Although, as shown in FIGS. 30A-C, with 0.91 the AROC for to Test2 higher as for Testl . Also the RMSE was higher (0.95).

[00206] For Test3 the AROC was 0.76, as shown in FIGS. 31 A-C. Interestingly, in two cases compounds that were structurally identical to compounds in the training set showed significant deviances from the experimental data. This was because the separation of the experimental data in training set and test sets was done using assay identification numbers in the ChEMBL database. Within each set duplicates were removed, but not duplicates that appear in different sets. The disagreement of the predicted value with the experimental value demonstrates how different reported pIC50 values from different assays can be. The related compounds are CHEMBL41, CHEMBL549. CHEMBL41 is also known as Prozac it is identical to CHEMBL 1201082 which is the corresponding salt. CHEMBL549 is also known as Cilatopram.

[00207] Finally, for Test 4 the AROC dropped to 0.52 which can be considered a random prediction. As shown in FIGS. 32 A-C, the ROC curve was within the area that is expected for random predictions. Both the average distance to the training set and the experimental diversity was highest for Test4. Also the range of pIC50 values that the test set spans was lower. This made it challenging to predict the pIC50 values. For the majority of compounds, a smaller distance to the training set went along with lower RMSE values.

[00208] We investigated the relationship between the RMSE and the minimal distance to the training set (MDT) by combining all test sets, as shown in FIGS. 33A-C. Then performed a threshold analysis by neglecting all compounds with MDT above/below the threshold value and analysed the performance on the remaining compounds. For this analysis, we removed all compounds with MDT=0. The results for AROC, R ² and r are shown in FIG. 34A-B. We observed a clear drop of the performance when considering compound with a MDT larger than 0.6. The AROC dropped to values 0.8, the R ² went down to 0.4. When considering compounds with MDT larger than the threshold, all values decreased even further. When exclusively considering the performance for compounds with MDT > 0.6, the AROC stayed above 0.7, which is a significant result.

6.4.7.12 Model interpretation

[00209] The frequencies of the features used in the final model were evaluated with the fscore function implemented in XGBoost. The feature that was most frequently used by the model was the molecular logP value, followed by MOE like descriptor (PEOE_VSA6), Kappa3 and the topological polar surface area. The top ranks were dominated by molecular descriptors. 2D-Pharmacophore features had lower scores. FIG. 35 shows that the model frequently uses a variety of features. The model allows estimating the pIC50 value within a range of around one pIC50 units for compounds that are similar to the training set.

[00210] The distribution of pIC50 values in the training set had a maximum at 5 which is used as the classification cutoff to define active and inactive classes. This means low and highly active compounds were overrepresented in Test3 compared the other sets, which made it easier for the model to distinguish both classes, as shown in FIGS. 29A-B and FIG. 36. The per compound similarity between Test3 and the training set was rather low compared to Testl. Test4 was less predictive, because the set had a comparably narrow range of pIC50 values in addition to the compounds being very dissimilar to the training set. As shown in FIGS. 29A, Test4 contained almost no compounds with a maximum similarity above 0.5. The distributions of both the pI50 values as well as the maximum similarity of Testl was most similar to training set. This was expected because both sets were drawn randomly from the same pool of compounds, while exhibiting some differences as well, which are based on by random fluctuations and the limited size of 700 compounds in total. The distribution of pIC50 values of Test 3 and 4 is very similar to each other, as illustrated in FIG. 29A.

[00211] We found AROC values around 0.9 and R ² above 0.8 for compounds that were similar to the training set (MDT < 0.5). For subsets of less similar compounds, the performance dropped. For the least similar subset (MDT > 0.7), the result was still better than random, while the best performance was observed for compounds that were similar to the compounds in the training set.

[00212] The performance on the least similar subset was still better than the performance on Test4, indicating that the diversity among the experimental methods used for compounds in Test4 reduces the predictive power of the model. In Test4, every compound had a different ChEMBL assay ID. The model performed well on Test2 with the majority of compounds in Test2 having an MDT above 0.6, which is based on the compounds in Test2 sharing the same experimental conditions, thus displaying a uniform distribution of pIC50 values.

[00213] For Testl, a simple random selection of 100 compounds, the R ² value was 0.7 and the correlation coefficient was 0.84. The differences in performance shows that it is important to take the test set composition into account when estimating the off-sample performance in addition to capturing time dependence, which was not considered in this current model.

[00214] None of the compounds in the test sets was used during the training and model selection process, except for two compounds that were present in Test3. These compounds had different experimental values in both sets and therefore, did result in an overestimation of the performance of the model, while being representative of the diversity of the data. In some cases, different pIC50 were measured for different isoforms.

[00215] The present model was based on 0D, ID and 2D descriptors without taking into account the conformation of a compound. An improvement of the model would include capturing effects that depend on 3D conformations like stereoisomerism.

[00216] Even for the training set the model had an RMSE above 0.4 pIC50 units and at least 0.51 for the test sets. If pIC50 values for a group of new compounds is known, the data of these compounds can easily be included in the training set to retrain the model. Since structurally closely related compounds are more likely to have similar activities, the model is able to estimate the affinities of such derivative compounds fairly accurate. By generating thousands of derivatives of a single compound, the model can be used to rank and prioritize these derivatives.

[00217] Besides fscore as "feature importance," the addition to features with low scores significantly increased the performance of the model. As shown in FIGS. 26A-D, incorporating the 2D-pharmacophore features boost model performance, while none of these features had a relative score above 10% (FIG. 35). In some cases, the values provided by the fscore function resulted in a more pronounced boost of performance in comparison to the other model parameters.

[00218] After the tuning process, the final model was trained on the complete dataset using in total 120 features to ensure all available information was used to build the model. The gradient boosting regression tree algorithm performs internal feature evaluation and only uses features that are important to improve the fit. Using a small learning rate and randomly chosen subsamples reduced the risk for overfitting, since for every step the algorithm evaluated a different sample of the dataset.

[00219] The learning curve allows visualizing how the model behaviour changes with varying the size of the training set. The learning curve is also used to analyze whether the model suffers from high bias or variance, and to decide whether the inclusion of more data would improve the model. If the learning curve indicates that the regression model suffers from high variance in the dataset, adding more reliable features and consistent data likely improves the model. Addition of 3D features will likely boost the performance of the model, since for some compounds we observed a steady decrease of the validation RMSE when increasing the training set size.

[00220] Comparing curves with different number of trees allowed us to determine when the model started to overfit the training data. Overfitting occurs, when the error in the test set stopped decreasing and eventually started increasing with more iterations indicating that the model generalizes. One of the advantages of tree based models is their robustness against overfitting. The error-bars in the plots were the calculated using standard deviations, expecting lower RMSE values for the training set (Train) as compared to the validation/test sets (Test). Feature selection and parameter tuning was done as described in detail above.

[00221] Different stereoisomers were not distinguishable in our modelling approach. Most compounds come as mixtures between all isomers and the effective IC50 is an average dominated by the stereoisomer with the lowest IC50 value. Stereoisomerism is not captured in the less than 3D descriptor space of RDKit molecular descriptors or 2D-pharmacophore features. An option for the model includes to add 3D features, which is motivated by the fact that, for example, terfenadine and its metabolite fexofenadine undergo a change in their 3D equilibrium structure due to the formation of an intramolecular hydrogen bond, which prevents hERG blocking. Addition of 3D features would require identifying a finite number of relevant 3D conformers among infinitely many possible confirmations of a particular compound. The model of this example was based on molecular properties that do not depend on the 3D confonnation of neither the ligand nor a receptor, providing a baseline for structure based virtual screening of compounds against hERG.

Alternative Embodiments

[00222] It should be understood that the examples and code sections provided above are intended to be purely exemplary and not limiting of the present invention.

[00223] Additionally, it should be noted that the systems and methods can be implemented on various types of data processor environments (e.g., on one or more data processors) which execute instructions (e.g., software instructions) to perform operations disclosed herein. Non-limiting examples include implementation on a single general purpose computer or workstation, or on a networked system, or in a client-server configuration, or in an application service provider configuration. For example, the methods and systems described herein can be implemented on many different types of processing devices by program code comprising program instructions that are executable by the device processing subsystem. The software program instructions can include source code, object code, machine code, or any other stored data that is operable to cause a processing system to perform the methods and operations described herein. Other implementations can also be used, however, such as firmware or even appropriately designed hardware configured to carry out the methods and systems described herein. For example, a computer can be programmed with instructions to perform the various steps of one or both of the flowcharts shown in FIGS. 1 A-1B.

[00224] It is further noted that the systems and methods can include data signals conveyed via networks (e.g., local area network, wide area network, internet, combinations thereof, etc.), fiber optic medium, carrier waves, wireless networks, etc. for communication with one or more data processing devices. The data signals can carry any or all of the data disclosed herein that is provided to or from a device. [00225] The systems' and methods' data (e.g., associations, mappings, data input, data output, intermediate data results, final data results, etc.) can be stored and implemented in one or more different types of computer-implemented data stores, such as different types of storage devices and programming constructs (e.g., RAM, ROM, Flash memory, flat files, databases, programming data structures, programming variables, IF-THEN (or similar type) statement constructs, etc.). It is noted that data structures describe formats for use in organizing and storing data in databases, programs, memory, or other computer-readable media for use by a computer program.

[00226] The systems and methods further can be provided on many different types of computer-readable storage media including computer storage mechanisms (e.g., non- transitory media, such as CD-ROM, diskette, RAM, flash memory, computer's hard drive, etc.) that contain instructions (e.g., software) for use in execution by a processor to perform the methods' operations and implement the systems described herein.

[00227] Moreover, the computer components, software modules, functions, data stores and data structures described herein can be connected directly or indirectly to each other in order to allow the flow of data needed for their operations. It is also noted that a module or processor includes but is not limited to a unit of code that performs a software operation, and can be implemented for example as a subroutine unit of code, or as a software function unit of code, or as an object (as in an object-oriented paradigm), or as an applet, or in a computer script language, or as another type of computer code. The software components and/or functionality can be located on a single computer or distributed across multiple computers depending upon the situation at hand.

[00228] It should be understood that as used in the description herein and throughout the claims that follow, the meaning of "a," "an," and "the" includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise. Finally, as used in the description herein and throughout the claims that follow, the meanings of "and" and "or" include both the conjunctive and disjunctive and can be used interchangeably unless the context expressly dictates otherwise; the phrase

"exclusive or" can be used to indicate situation where only the disjunctive meaning can apply. Other Alternative Embodiments

[00229] While various illustrative embodiments of the invention are described above, it will be apparent to one skilled in the art that various changes and modifications may be made therein without departing from the invention. The appended claims are intended to cover all such changes and modifications that fall within the true spirit and scope of the invention.

APPENDIX A

hERG activity prediction.

Can we use molecular structures, related properties and similiarities to distinguish between hERG active and inactive compounds?

• desired accuracy +90%

RDkit : Convert smiles codes into molecular descriptors

Out[16]:

3 rows x 96 columns

In [17]:

Data acquisition

In [18]: # REC

REC_ID='CHEMBL240'

# Get compound record...

In [19]:

'Get set of active compound IDs from chEMBL database'

REC_bioact = pd.DataFrame(targets.bioactivities(REC_ID))[

['parent_cmpd_chemblid',\

'value'A

'units'A

'activity_comment',\

'bioactivity_type',\

'assay_description',\

'name_in_reference',\

'reference']]

print 'Number of entries before filtering:', len(REC_bioact) '"Compounds with IC50 activities (actives and inactives)'"

REC_bioact=REC_bioact[\

(RECjDioact.value != 'Unspecified') &\

(REC_bioact.bioactivity_type == 'IC50') &\

(REC_bioact. reference != 'Unspecified') &\

(REC_bioact.assay_description == 'Inhibition of human ERG ¹ ) ]

'FILTER maximum activity 20000nM'

REC_bioact=REC_bioact[(REC_bioact.value.astype(float) <=10000)]

'FILTER activity not exactly l OOOOnM, because most likely an experimental artifact'

REC_bioact=REC_bioact[(REC_bioact.value.astype(float) !=10000)]

REC_bioact=REC_bioact[(REC_bioact.value.astype(float) !=20000)]

REC_bioact=REC_bioact[(REC_bioact.value.astype(float) !=30000)]

REC_bioact=REC_bioact[(REC_bioact.value.astype(float) !=25000)]

'FILTER for duplicates'

REC_bioact=REC_bioact.drop_duplicates(subset='parent_cmpd_ch emblid') 'Convert value to float and calculate plC50 = -log10(X/nM)'

REC_bioact.value=REC_bioact.value.astype(float)

REC_bioact['logval']=-1 *np.log10(REC_bioact.value/1000000000.) 'Add activity column'

REC_bioact['activity']=[ act(i) for i in REC_bioact.value ]

REC_bioact=REC_bioact.sort('value')

print 'Number of entries after filtering: ',len(REC_bioact)

REC_bioact.hist()

Number of entries before filtering: 14056

Number of entries after filtering: 416

Out[19]:

The data reveals several plateaus e.g. at lOOOOnM and 30000nM. These might experimental artifacts. The data spans six orders of magnitude. In this figures compounds without a value (the inactive set) is omitted.

In [27]:

'Get SMILES structures' print "Database connection:", compounds.status()

'Workaround a problem with compounds.get() returning empty list with certain IDs is the querry'

Database connection: True

Found structures of 415 compounds

Defining decoys

In [31]:

Found structures of 485 compounds

Build training, test and validation set

In [32]:

act=REC_data[REC_data.act

Found 144 actives and 341 inactive compounds. 485 CHEMBL168366 1

CHEMBL2334620 1

CHEMBL195936 1

CHEMBL49537 1

CHEMBL226140 1

dtype: int64 In [33]:

n_a+n_a_t:N_a], inact_sample[n_i+n_i_t:N_i]])

print "Training set contains ", len(train), " compounds", len(train[train.activity==1]), len(train[train.activity==0])

print "Test set contains ", len(test), " compounds", len(test[test.activity==1]), len(test[test.activity==0])

print "Validation set contains ", len(valid), " compounds" , len(valid[valid.activity==1 ]) , len(valid[valid.activity==0])

print "Alltogether: ", len(train)+len(test)+len(valid)

print "Checking overlap (should be 0!)"

print 'b in A',len(train[train.chemblld.isin( test.chemblld )]) print 'a in B',len(test [test.chemblld. isin( train.chemblld )]) Training set contains 290 compounds 86 204 Test set contains 96 compounds 28 68 Validation set contains 99 compounds 30 69

Altogether: 485

Checking overlap (should be 0!) b in A 0

a in B 0

Exploratory Data Analysis

In [34]:

Dataset of ⁷ active' compounds

Index

0.43 9400.0

In [35]:

'Drawing compounds from the active set'

Draw.MolsToGridlmage([Chem.MolFromSmiles(x) for x in act.smiles.head(9)],sublmgSize=(200,200))

Out[35]:

In [36]:

'Drawing compounds from the inactive set'

Draw.MolsToGridlmage([Chem.MolFromSmiles(x) for x in inact.smiles.head(9)],sublmgSize=(200,200))

Out[36]:

Similarity

In [37]:

In [40]:

hh = heatmap(Similarities(act_fps,inact_fps)) plt.savefigfheatmap-similarities- act_inact.png",dpi=300) hh = heatmap(Similarities(inact_fps,inact_fps)) plt.savefig("heatmap-similarities-inact.png",dpi=300)

95

Heatmap of the active and inactive sample, based on Dice similarity score:

S Dice =2p+2p+q+d

• P number of positive in both

• q number of positive in A but not B

• d number of positive in B but not A

• Note that aggreements are weighted twice

Dissimilarty is black and similarity is white. The maps shows there are some clusters of similar ligands in the active set. Less so in the inactives set. There are no high similarities between active and inactive samples.

The histogram show the probability of mutual similarities. The distribution for the inactive set is shifted to lower values, indicating a higher diversity as compared to the active sample. Due to the diversity filtering the similarity distributions are very similar.

Molecular Properties Output

—Actives—

— Inactives—

Out[44]:

We see some general tendencies. For all properties the deviation is smaller in the active set. Interesting is that the active compounds are bound to positive logP values. Both, the fraction of hydrogen donors and acceptors seems to be smaller. So the topological polar surface area, which makes sense because this area depends on the donors and acceptors. Furthermore, there seems to be at least one aromatic ring in the actives.

Molecular Properties: Interrelations

In [45]:

The interrelations of the properties in the active (red) and the inactive (blue) samples look very similar. However, we have to keep in mind that the inactives are random decoys and no proven inactives. The plots show that the actives occupy a subspace of the inactive compounds. Every region occupied by actives also has some inactives. Notably, the actives are bound to positive values in the logP values, the molecular weight (MW). The fraction of hydrogen donors (fNHDon) is lower. Chemical groups analysis

The analysis of the mean of the counts of chemical groups shows that the chemical moieties correlate strongly in both sets active and inactive. A high number of a particular moiety in one group usually means a higher number in the other group too. For all groups the means are within the mutual standard deviations. Therefore, we cannot

conclude that the occurrence of a certain moiety has special explanatory power, in the first place. However, we should look at the p-values. Also interesting would be to look at combinations. In [49]: fr_C_O -4.0670 +0.0001

fr_COO2 -5.6422 +0.0000 fr_COO -5.6422 +0.0000 fr_AI_COO -5.2128 +0.0000 fr_piperdine +3.8616 +0.0002 fr_NH0 +3.7160 +0.0003

NULL Models

In [51 ]:

Set X_names=features to include the chemical groups.

Set X_names=properties to only use the other molecular descriptors. print features

print properties X_names = features y_name = 'activity'

print len(X_names), pow(2,len(X_names))

['CSP3', 'LogP', 'MW, 'NAromRing', 'NHAt', 'NOCount', 'NRings', 'NRotB', TPSA', 'fNHAcc', 'fNHDon', 'fr_C_O', 'fr_C002', 'fr_COO', 'fr_AI_COO', 'fr_piperdine', 'fr_NH0'] ['CSP3', 'LogP', 'MW, 'NAromRing', 'NHAt', 'NOCount', 'NRings', 'NRotB', 'TPSA', 'fNHAcc', 'fNHDon']

17 131072

In [52]:

GENERATION OF NULL MODELS

X auroc fNHDon 0.645862 fr_NH0 0.637654 TPSA 0.631840 fr_piperdine 0.630557 fr_C_O 0.596472 fNHAcc 0.596187 CSP3 0.595218 LogP 0.589746 fr_COO2 0.578431 fr_COO 0.578431 fr_AI_COO 0.568627 NRings 0.555147 NOCount 0.551841 NRotB 0.545913 2 MW 0.542921

4 NHAt 0.536594

3 NAromRing 0.510944

The shape of the ROCs shows that the variables are relatively weak predictors.

Model building

Logistic Regression

In [54]:

Naive Bayes

"Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes' theorem with the "naive" assumption of independence between every pair of features. Naive Bayes learners and classifiers can be extremely fast compared to more sophisticated methods. [...] [Although] naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability outputs from predict_proba are not to be taken too seriously." scikit-learn 0. 15.2 documentation In [55]:

Naive Bayes on Bit-Vectors

A new approach which works with Morgan fingerprints and no molecular features. Could be a good candidate for consensus scoring.

In [56]:

Decision Tree Module

In [57]:

Random Forest Module

In [58]:

Boosting Module

In [59]:

Model analysis

Prediction Accuracy over Number of Features

In [60]:

We would expect an initially increasing accuracy which drops and decreases with a certain model complexity. The drop indicate over-fitting. We see that holds for the maximum accuracy, but not for the mean or the median. The accuracies are averaged over the number of features used in the model. Maybe, the average accuracy should not be used to do this check.

Model Selection

The models are selected according to the the highest out-of-sample accuracy (or the lowest out-of-sample error). In [61]:

feat='AC N_X=14

SelectModelScheme = 1

if SelectModelScheme == 0:

LR_model=LR_models_test[LR_models_test['N_X']==N_X].sort(fea t,ascending=Fa lse).model.values[0]

NB_model=NB_models_test[NB_modelsJest['N_X']==N_X].sort(feat ,ascending=F alse).model.values[0]

DT_model=DT_models_test[DT_models_test['N_X']==N_X].sort(fea t,ascending=F alse).model.values[0]

RF_model=RF_models_test[RF_models_test['N_X']==N_X].sort(fea t,ascending=F alse).model.values[0]

BO_model=BO_models_test[BO_models_test['N_X']==N_X].sort(fea t,ascending=F alse).model.values[0]

else:

LR_model=LR_models_test.sort(feat,ascending=False).model.val ues[0]

NB_model=NB_models_test.sort(feat,ascending=False).model.val ues[0]

DT_model=DT_models_test.sort(feat,ascending=False).model.val ues[0]

RF_model=RF_models_test.sort(feat,ascending=False).model.val ues[0]

BO_model=BO_models_test.sort(feat,ascending=False).model.val ues[0]

NBBV_model=NBBV_models_test.sort(feat,ascending=False).mo del.values[0] BestModels_byAC=[]

for i, model in zip(['LogReg', 'Naive Bayes','D-Tree','Random

Forest', ^, Boosting ^, , ^, NB_BitVect'],\

[LR_model,NB_model,DT_model,RF_model,BO_model,NBBV_model]): model. Iabel=i

BestModels_byAC.append({'moder:model,'Family':i})

BestModels_byAC=pd.DataFrame(BestModels_byAC)

In [62]:

print "Family Features #Features\n "

for i in BestModels_byAC. model:

print i. label, i.X , len(i.X) Family Features #Features

LogReg ['fNHDon', fr_NH0', 'TPSA', 'fr_piperdine', 'fr_C_O', 'fNHAcc', 'CSP3', 'LogP', 'fr_C002'] 9

Naive Bayes ['fNHDon', 'fr_NH0', TPSA', 'fr_piperdine', fr_C_0', 'fNHAcc', 'CSP3', 'LogP'] 8

D-Tree ['fNHDon', 'fr_NH0', 'TPSA ¹ , 'fr_piperdine', 'fr_C_0', 'fNHAcc', 'CSP3'] 7 Random Forest ['fNHDon ¹ , 'fr_NH0', 'TPSA', fr_piperdine', fr_C_0', 'fNHAcc', 'CSP3 ¹ , 'LogP', 'fr_C002', 'fr_COO', 'fr_AI_COO'] 1 1

Boosting ['fNHDon', fr_NH0', 'TPSA', 'fr_piperdine', 'fr_C_0' _f 'fNHAcc', 'CSP3', 'LogP', 'fr_C002', 'fr_COO', fr_AI_COO', 'NRings', 'NOCount', 'NRotB', 'MW'] 15

NB_BitVect ['FP'] 1

Model Performance

Training set

In [64]:

Result Interpretation

The Random Forest and the Boosting models usually have the highest prediction accuracies (AC). The false negative rate (FN) is very high in all models. The ROCs show that the calculated class probabilites the models can be used to rank a list of compounds. Now, it remains to be seen, whether the models can predict hERG active and inactive compounds.

Quantitative Prediction

A try to fit the plC50 values in a linear regression model. Cross-validated with k- means crossvalidation.

In [71 ]:

Application of Machine Learning Algorithms for Test-Set of Blockers published by Barakat et al, Toxicology Letters, 2014, doi: 10.1016/j.toxlet.2014.08.007. pIC50 outputs shows R correlation coefficients to experimentally available data from Barakat et al used as a test set for model validation.

89 compounds . QSAR platform from http://labmol.farmacia.ufg.br/predherg/.

Relevant citation: Braga, R. C; Alves, V. M.; Silva, M. F. B.; Muratov, E.; Fourches, D.; Tropsha, A.; Andrade, C. H. Tuning hERG out: Antitarget QSAR Models for Drug

Development. Curr. Top. Med. Chem. 2014, 14, 1399-1415.

3 rows x 96 columns

In [7]:

'"Miscelaneous"'

from I Python. display import Image

Data acquisition

In [8]:

# REC

REC_ID='CHEMBL1980'

# Get compound record...

target = targets.get(RECJD)

Number of entries before filtering: 695 Number of entries after filtering: 240

The data reveals several plateaus e.g. at lOOOOnM. These might be experimental artefacts. The data spans six orders of magnitude. In this figures compounds without a value (the inactive set) is omitted. In [18]:

Get structures'

print "Database connection:", compounds.status()

structures=compounds.get(list(REC_bioact.parent_cmpd_chem blid))

#Get the compound structures from the ChEMBL database

a=[i for i in structures if type(i)!=int]

REC_cmpds=pd.DataFrame(a)[['chemblld', 'smiles']]

REC_cmpds=REC_cmpds.dropna()

print 'Found structures of %d compounds' %(len(REC_cmpds))

REC_cmps_properties=pd.DataFrame(list(MolecularProperties (REC_cmpds.smiles))) df 1 =AddMolProp(REC_cmpds)

df1.count()

REC_data=pd.merge(REC_bioact.rename(columns={'parent_cmpd_ch emblid':'chemb lld'}),df1 ,on='chemblld')

REC_data=REC_data[REC_data.NHAt>10]

print 'Found structures of %d compounds' %(len(REC_data))

Database connection: True

Found structures of 240 compounds Found structures of 240 compounds In [19]: REC_data['activity'] Out[20]:

#de\ decoys

'"Decoys'"

'"Filtering with pandas is a PITA'"

'Filter out the entries with empthy smiles strings' 'Filter out smiles with two or more molecules'

'check for smiles entries that contain multiple molecules' 'Data Transformation'

Doublicates 0

952 Build training, test and validation set

Found 92 actives and 860 inactive compounds. 952 CHEMBL569085 1

CHEMBL123454 1

CHEMBL1 1 1622 1

CHEMBL335768 1

CHEMBL458520 1

dtype: int64 In [25]:

Ex loratory Data Analysis

Heatmap of the randomly selected active (left) and inactive (top) compounds, based on Dice similarity score:

S Dice = 2p+q+d

Where

P number of positive in both,

Q number of positive in

A but not B

J

d number of positive in B but not A

. Note that aggreements are weighted twice

Results

• The similarity of the inactive compounds is much higher than expected, if only the REC inactive records in CHEMBLDB are used.

• therefore -800 decoys were added randomly selected from CHEMBLDB

• the dissimilarity is OK

In [28]:

Draw.MolsToGridlmage([Chem.MolFromSmiles(x) for x in act.smiles.head(9)],sublmgSize=(200,200))

Out[29]:

In [30]:

Draw.MolsToGridlmage([Chem.MolFromSmiles(x) for x in inact.smiles.head(9)],sublmgSize=(200,200))

In [31 ]:

The similarity distribution with the actives and the inactive sets do overlap as shown in the figure. The plot reveals, there are significantly more similar compounds with similarity scores above 0.6 in the actives set. What is the actual fraction of very similar compounds? We should add a filtering step before generating training, test and validation sets

Activity Bar-Graph

Chemical groups based predictions

In [35]: list(Data. index)

Out[36]: In [37]:

The analysis of the mean of the counts of chemical groups shows that the chemical moieties correlate strongly in both sets active and inactive. A high number of a particular moiety in one group usually means a higher number in the other group too. For all groups the means are within the mutual standard deviations. Therefore, we cannot conclude that the occurrence of a certain moiety has special explanatory power, in the first place. However, we should look at the p-values. Also interesting would be to look at combinations.

In [38]: NULL Models

The shape of the ROCs shows that the deviation of the active compounds is lower as compared to the decoys with a similar mean as seen in the

histograms. Can these variables be called weak predictors? Are these properties sufficient? In fact, there is no predictive power in the variables. In [41]:

Model building Logistic Regression

In [42]:

2035

Naive Bayes

"Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes' theorem with the "naive" assumption of independence between every pair of features. Naive Bayes learners and classifiers can be extremely fast compared to more sophisticated methods. [...] [Although] naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability outputs from

predict_proba are not to be taken too seriously." scikit-learn 0.15.2 documentation In [43]:

 Select Best Models

In [48]:

The finally selected Boosting model contains only 4 variables (NAromRing, fr_ALCOO, fNHDon and TPSA). It is interesting that the models often use different variables.

In [50]:

A=EvalModels(BestModels_byAC,train,labels=BestModels_byAC .Family,plot=True,lin ewidth=2)

<matplotlib.text.Text at 0x67ca7d90>

Out[51]:

In [52]:

In [56]:

pd.scatter_matrix(Result_test,f igsize=(10, 10)) plt.show()

Error estimation

In [57]:

in [59]:

The classification accuracy of the models is between 0.55 for the LR model and and 0.75 for the Naive Bayes model. The true negative rate of the Naive Bayes model is best with 0.88. The FP rate of the RF and LR model are smallest. The RF model has a very high FN rate, although the accuracy is pretty high. Altogether, the NB model seems to be the most robust model with a sensitivity of -0.85 and specificity of -0.7. However, the ROC shows that the RF model can perform much better. No literature data on machine learning in Nav1.5 available. The test set of experimental plC50 is to be generated by Achlys Inc cell lines.

In [60]:

The Brown set data points were used as a validation set.

APPENDIX C

3 rows x 96 columns In [10]:

Number of entries before filtering: 254

Number of entries after filtering: 27

The data reveals several plateaus e.g. at l OOOOnM and 30000nM. These might be experimental artifacts. The data spans six orders of magnitude. In this figures compounds without a value (the inactive set) is omitted.

In [14]:

'Get structures'

print "Database connection:", compounds.status()

structures=compounds.get(list(REC_bioact.parent_cmpd_chembli d))

#Get the compound structures from the ChEMBL database

Database connection: True

Found structures of 27 compounds

In [15]:

Heatmap of the randomly selected active (left) and inactive (top) compounds, based on Dice similarity score:

S Dice= 2p+q+d

Where

P number of positive in both,

Q number of positive in

A but not B

d number of positive in

B but not A

Results

• The similarity of the inactive compounds is much higher than expected, if only the REC inactive records in CHEMBLDB are used.

• therefore -800 decoys were added randomly selected from CHEMBLDB

• the dissimilarity is OK

In [20]:

In [21 ]:

Draw.MolsToGridlmage([Chem.MolFromSmiles(x) for x in act.smiles.head(9)],sublmgSize=(200,200))

In [22]:

Draw.MolsToGridlmage([Chem.MolFromSmiles(x) for x in inact.smiles.head(9)],sublmgSize=(200,200))

Out[22]:

In [23]:

The similarity distribution with the actives and the inactive sets do overlap as shown in the figure. The plot reveals, there are a view more similar compounds with similarity scores above 0.8 in the actives set. The inactive set has a view compounds wich are rather dissimilar from the rest. However, the fraction of both have only minor impact on the modeling.

In [25]:

Chemical groups based predictions

In [27]:

In [28]:

The analysis of the mean of the counts of chemical groups shows that the chemical moieties correlate strongly in both sets active and inactive. A high number of a particular moiety in one group usually means a higher number in the other group too. For all groups the means are within the mutual standard deviations. Therefore, we cannot conclude that the occurrence of a certain moiety has special explanatory power, in the first place. However, we should look at the p-values. Also interesting would be to look at combinations.

In [30]:

NULL Models

In [33]:

Naive Bayes

"Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes' theorem with the "naive" assumption of independence between every pair of features. Naive Bayes learners and classifiers can be extremely fast compared to more sophisticated methods. [...] [Although] naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability outputs from

predict_proba are not to be taken too seriously." scikit-learn 0. 15.2 documentation In [35]:

models={}

Decision Tree

In [36]:

Random Forest

In [37]:

<matplotlib.text.Text at 0x29f45910>

Boosting

In [38]:

random.seed(34345)

Number of Features and Prediction Accuracy

In [52]:

Type Markdown and LaTeX: a

2

Select Best Models

In [40]:

Out[46]:

Type Markdown and LaTeX:

a

2

In [47]:

Result_test={}

for name, model in zip(BestModels_byAC.Family,BestModels_byAC. model):

X= model.X

Result_test[name]=model.predict_proba(test[X]).T[1 ]

Result_test=pd.DataFrame(Result_test)

Result_test.index=test[['activity']]. index

Result_test=Result_test.join(test[['activity']])

Result_test.boxplot(by='activity',figsize=(10,10))

plt.show()

Result_test.hist(figsize=(10, 10),normed=True,bins=10)

plt.suptitle('Histogram of predicted probabilitiesViTestset')

plt.showQ

224 Error estimation

In [49]:

The classification accuracy of the models is between 0.55 for the LR model and and 0.75 for the Naive Bayes model. The true negative rate of the Naive Bayes model is best with 0.88. The FP rate of the RF and LR model are smallest. The RF model has a very high FN rate, although the accuracy is pretty high. Altogether, the NB model seems to be the most robust model with a sensitivity of -0.85 and specificity of -0.7. However, the ROC shows that the RF model can perform much better.

• How is the classification done with model.predict()?

Sensitivity