INCONSISTENCY

TECHNICAL FIELD

[0001] The present disclosure relates to data inconsistency detection and remediation in Internet of Things (loT) data.

BACKGROUND

[0002] Internet of Things (loT) is considered as a stepping-stone in building modem wireless telecommunication systems. loT devices are heterogeneous and use various protocols to serve the different applications hosted in edge clouds. The quality of services offered by these applications highly rely on the quality of the data collected from the loT systems. However, while collecting such data, many problems can occur and cause data inconsistency. Data inconsistency refers to data values in one data set not being consistent with values in another data set. This means that the collected data may be changed differently or may not be synchronized between the source and destination in the loT system. One reason behind such inconsistency could be the sampling rate of loT devices: there can exist a high number of loT devices serving an application, and hence changing sampling rate for many devices is a tedious task and is not efficient. In addition, the loT devices could serve more than one application (e.g., virtual sensors). In that case, changing the sampling rate for a single application might affect the performance of the other applications. Furthermore, loT devices are heterogeneous and use various application protocols e.g., Constrained Application Protocol (CoAP), Message Queuing Telemetry Transport (MQTT), Extensible Messaging and Presence Protocol (XMPP), etc. These devices may serve different providers and may experience different communication conditions (e.g., different network delays).

SUMMARY

[0003] It is therefore of interest to design an automated system/architecture to dynamically identify reasons behind data inconsistency in loT systems and hence early identify such events to avoid their impacts on loT applications hosted in edge cloud. [0004] There is provided a computer implemented method for predicting Internet of Things (loT) data inconsistency. The method comprises obtaining labelled loT data, the loT data being collected from a plurality of loT devices by a monitoring system. The method comprises analyzing characteristics of the labelled loT data and identifying features of loT data inconsistency. The method comprises training, using the labelled loT devices data and the features of loT data inconsistency, a ML model to predict the loT data inconsistency. The method comprises generating, using the labelled loT data and the features of loT data inconsistency, a set of inconsistency rules to be applied to live loT data predicted as inconsistent by the ML model.

[0005] There is provided an apparatus for predicting Internet of Things (loT) data inconsistency. The apparatus comprises processing circuits and a memory. The memory contains instructions executable by the processing circuits whereby the apparatus is operative to obtain labelled loT data, the loT data being collected from a plurality of loT devices by a monitoring system. The apparatus is operative to analyze characteristics of the labelled loT data and identify features of loT data inconsistency. The apparatus is operative to train, using the labelled loT devices data and the features of loT data inconsistency, a ML model to predict loT data inconsistency. The apparatus is operative to generate, using the labelled loT data and the features of loT data inconsistency, a set of inconsistency rules to be applied to live loT data predicted as inconsistent by the ML model.

[0006] There is provided a non-transitory computer readable media having stored thereon instructions for predicting Internet of Things (loT) data inconsistency, the instructions may comprise any of the steps described herein.

[0007] The method and apparatus provided herein present improvements to the way data inconsistency detection and remediation in loT data operate.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure l is a block diagram illustrating deployment options for the proposed loT data inconsistency predictor.

[0009] Figure 2 is a schematic high-level view of the proposed system.

[0010] Figure 3 is a flowchart of a method for the proposed loT data inconsistency prediction.

[0011] Figure 4 is the first part of a flowchart of an example of loT data inconsistency prediction.

[0012] Figure 5 is the second part of the flowchart of the example of loT data inconsistency prediction. [0013] Figure 6 is a flowchart of a method for predicting Internet of Things (loT) data inconsistency.

[0014] Figure 7 is a schematic illustration of an apparatus in which steps of the method described herein can be executed.

[0015] Figure 8 is a schematic illustration of a virtualization environment in which the different method(s) and hardware components described herein can be deployed.

DETAILED DESCRIPTION

[0016] Various features will now be described with reference to the drawings to fully convey the scope of the disclosure to those skilled in the art.

[0017] Sequences of actions or functions may be used within this disclosure. It should be recognized that some functions or actions, in some contexts, could be performed by specialized circuits, by program instructions being executed by one or more processors, or by a combination of both.

[0018] Further, computer readable carrier or carrier wave may contain an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein.

[0019] The functions/actions described herein may occur out of the order noted in the sequence of actions or simultaneously. Furthermore, in some illustrations, some blocks, functions or actions may be optional and may or may not be executed; these are generally illustrated with dashed lines.

[0020] At least some aspects of the techniques described herein may be implemented using artificial intelligence, which comprises a variety of techniques as would be apparent to a person skilled in the art, including machine learning techniques.

Machine learning techniques include Neural Network (NN), or Artificial Neural Network (ANN), and both terms may be used interchangeably herein.

[0021] Herein, a solution for predicting data inconsistency in loT systems is provided as an extension to inconsistency detection and resolution solutions. The proposed solution takes as input data collected from loT systems (historical and live data) along with characteristics of these loT systems (type, location, sampling rate, requirements, etc.). It builds machine learning (ML)-based model (e.g., Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN) models, etc.) to identify data inconsistency with loT data early. It generates as output the main factors contributing to data inconsistency, prediction results, and a set of inconsistency rules to handle predicted data inconsistency.

[0022] Traditional “resolution” components solve the inconsistencies after they happen, e.g., they remove inconsistent data. Therefore, the existing “resolution” component is also extended, to be able to avoid inconsistency before its occurrence. The main factors contributing to data inconsistency and the inconsistency rules are added to refine the resolution actions (e.g., update data collection sources).

[0023] The proposed system allows to detect data inconsistency with loT systems early and take appropriate actions to prevent data inconsistency events in the future. It allows building inconsistency rules that can be used by both inconsistency prediction and detection solutions. It allows to identify the main contributing factors contributing to data inconsistency: which can be used to refine/improve the existing inconsistency detection models in terms of accuracy, relevance, etc. It also can be applied to other types of data inconsistency that could be detected when collecting data such as routing packets, database, etc.

[0024] Referring to Figure 1, the proposed loT data inconsistency predictor system 5 is described. Figure 1 presents the way the proposed system intersects or resides in existing loT architecture layers. The proposed system can be deployed in the application layer 10 or the network layer 15 (i.e., loT gateways). This depends on whether an existing inconsistency detector 20 solution is deployed or not. In the case of the existence of an inconsistency detector 20, the proposed solution can extend it to provide additional functionalities and improve the existing functionalities. If an inconsistency detector 20 solution does not exist, then the proposed system can be deployed either in the application layer 10 or the network layer 15 according to the provider preference.

[0025] Figure 2 shows the proposed loT data inconsistency predictor system 5. [0026] The system takes as input the data from loT devices along with the loT devices characteristics such as their type (e.g., sensor, actuator), location, destination, sampling rate, etc. The system also takes the application requirements (e.g., service level agreement (SLA)) and profiles, and the network conditions into consideration. [0027] The main functionality of the loT data inconsistency predictor 5 is to predict and identify loT data inconsistency early, using historical data collected from the loT devices and the profiles of the applications provided by the users/application provider. To do this, the loT data inconsistency predictor 5 first identifies the causing factors that contributed the most to the data inconsistency. These factors are then used to build ML-based prediction models to identify data inconsistency on the live data collected from the loT devices. These factors are also used to build inconsistency prediction rules that assist in inconsistency detection/prediction.

[0028] After predicting loT data inconsistency, the system generates as output the prediction results and the main factors contributing to the data inconsistency. It also builds a set of inconsistency rules using the prediction results to assist in inconsistency prediction and avoidance.

[0029] Figure 3 shows the flowchart of the proposed method for loT data inconsistency prediction. The flowchart contains and integrates two parts: the inconsistency detector 20 (boxes 31, 37 and 38) and the inconsistency predictor 5 (boxes 32-36). The loT data inconsistency predictor 5 works as follows.

[0030] The method 30 starts by collecting, step 31, historical input data (labeled) from a monitoring system.

[0031] Then, it identifies, step 32, the contributing factors of data inconsistency using labeled historical data and user profiles. The user profiles could include user-specified information on the data, for instance, where to collect data from, required sampling rate, application specifications (e.g., latency-sensitive, data-dependent), etc. The historical data are labeled data that includes the loT devices' data, characteristics (e.g., type, location, destination, sampling rate), and their network conditions and whether the data was consistent or not.

[0032] The method measures the degree of correlation between all these factors, which are also called features. The Spearman’s correlation coefficient (Bishara, Anthony J., and James B. Hittner. "Testing the significance of a correlation with nonnormal data: comparison of Pearson, Spearman, transformation, and resampling approaches." Psychological methods 17.3 (2012): 399.) can be used between all these identified features.

[0033] The resulting correlation coefficient would be a matrix with N rows and N columns such that N defines the number of obtained feature elements. The obtained correlation matrix could be used as an indication of the degree of relationship between the feature elements in the obtained matrix. The method 30 uses this matrix to select the most relevant features that are inter-correlated with the data inconsistency.

[0034] According to the defined contributing causes, prediction models are built, step 33. The output of this step is prediction model(s). These models are used to predict data inconsistency on the live data collected from the loT devices. For this step, different machine learning models can be used such as LSTM, CNN, etc.

[0035] Using the contributing factors, inconsistency detection/prediction rules are also built, step 34. This step is complemented by using the user Profiles. These rules are saved in the rules repository 39 which helps in the prevention and the avoidance of loT data inconsistency.

[0036] In step 35, the method performs data inconsistency prediction on the live data. [0037] If an inconsistency is predicted, data refinement actions are generated, step 36, using the rules in the rules repository 39. These actions are sent to step 31 (i.e., collect data) to guide the live data collection process such that future inconsistencies are avoided or reduced as much as possible.

[0038] In step 37, the method analyzes the collected live data to see if there is any inconsistency. Different methods are known for that purpose such as performing consistency checks, hop counts for routing tables, comparisons based on thresholds, etc. In the present method, it is proposed to use user-profiles and ML techniques. [0039] Once an inconsistency is detected, the method proposes, step 38, a solution to solve it. This is an improvement over known method which, to solve inconsistency, simply remove the inconsistent data. In the solution presented herein, the contributing factors for data inconsistency are identified, such as the sampling rate, then the rules repository that contains a set of rules is used to find the best candidate solution to be applied considering the identified contributing factor, to recover from the detected inconsistency.

[0040] Figures 4 and 5 illustrate an example of the proposed loT data inconsistency predictor 5. The purpose of this example is to illustrate the different steps of Figure 3 in a simplified manner.

[0041] In Figure 4, the steps 31-34, performed on the historical data are shown. In this part, the historical data (labeled) is collected, step 31, first from the loT devices. [0042] The method then identifies, step 32, the most relevant features or factors, contributing factors, that have an impact on the consistency of the loT devices' data. In this example, it is assumed that the sampling rate was identified as the most contributing factor to the loT data inconsistency.

[0043] According to the contributing factors, inconsistency rules are built, step 34. At this step, the application/user profiles are used to assist in building meaningful inconsistency rules. An example of an inconsistency rule, Rule 1, is shown in Figure 4. The rule says that the collected data for app 1 should have a low sampling rate since this app is a critical application and it requires the latest collected data, for example within the latest 1 hour. The method then updates the rules repository 39 with the new rule that is built.

[0044] The method also builds, step 33, inconsistency prediction models using the contributing factors. For this step, the method obtains different data parameters, characterizes the collected data, and builds prediction models to predict expected data inconsistency.

[0045] To build the prediction models, for example, LSTM can be used, and the following steps can be applied. First, historical data is collected. Then, contributing factors to data inconsistency are identified, this step can be done using the Spearman correlation coefficient. The ML model (e.g., LSTM) is then trained using this data. The historical data can be splitted into training and testing data to prevent the model from overfitting. Cross validation or K-fold cross validation can be used for data splitting, where a model is trained and evaluated “k” times on different samples. The model is trained to classify the future live data into two categories: consistent or inconsistent. The trained prediction model is used later on the live data to detect if there is any consistency. Its output indicates either “yes” meaning there is inconsistency or “no” indicating no inconsistency.

[0046] Finally, the method outputs the prediction model that is built to be used for inconsistency prediction on the live data.

[0047] In Figure 5, the steps performed on the live data are shown. In this part, the method collects, step 31, live data from the loT devices. This can include the raw data, the loT devices' characteristics (such as sampling rate, their location, target destination, etc.), and their network conditions.

[0048] It then performs data inconsistency prediction, step 35, using the prediction model generated in Figure 4.

[0049] If data inconsistency is predicted, it generates, step 36, data refinement actions using the rules in the rules repository 39. For instance, following the rule (Rule 1) generated in Figure 4, the method suggests filtering the collected data by keeping only the data collected from sensors 3 and 4 (i.e., S3 and S4) since these sensors have a low sampling rate and satisfy the inconsistency rule in the rules repository, and remove the data collected from SI and S2. [0050] Finally, it sends data refinement actions to the entity that collects live data, such that data inconsistency can be avoided in the future.

[0051] The rules identified by the method can also assist in the data inconsistency detection procedure on the live data. The identified rules can predict the inconsistency before it happens. However, the same rules can be applied by the data inconsistency detection system, where an inconsistency is detected after it happens. When used to predict inconsistency, the system can take same preemptive actions to avoid inconsistency. For instance, following the example above, the data inconsistency detection can utilize this sampling rate rule to avoid collecting data from specific sensors (identified previously with a high sampling rate). In this way, the system can avoid data inconsistency and collects less amount of data to be analyzed.

[0052] It should be noted that methods and steps described herein are, generally, computer implemented methods and steps. The term computer may be interpreted as having different meanings, such apparatus, gateway, etc.

[0053] Figure 6 illustrates a computer implemented method 60 for predicting Internet of Things (loT) data inconsistency. The method comprises obtaining, step 61, labelled loT data, the loT data being collected from a plurality of loT devices by a monitoring system. The method comprises analyzing, step 62, characteristics of the labelled loT data and identifying features of loT data inconsistency. The method comprises training, step 63, using the labelled loT devices data and the features of loT data inconsistency, a ML model to predict the loT data inconsistency. The method comprises generating, step 64, using the labelled loT data and the features of loT data inconsistency, a set of inconsistency rules to be applied to live loT data predicted as inconsistent by the ML model.

[0054] The labelled loT data may comprise the loT data of a plurality of loT devices and a plurality of features for each of the loT data including at least one of a type of loT data, a type of loT device, a location of acquisition of the loT data, a time of acquisition of the loT data, a destination for the loT data, a sampling rate of the loT data, a latency-sensitiveness of the loT data, a network condition when the loT data was acquired, and an indication whether the loT data is consistent or not.

[0055] It should be noted that the same loT data can serve both latency-sensitive and non-sensitive application. The latency-sensitive application may require the data quickly. [0056] Analyzing the characteristics of the labelled loT data may comprise measuring a degree of correlation between features of the labelled loT data. The degree of correlation may be measured using Spearman’s correlation coefficients between the features of the labelled loT data. The resulting correlation coefficients may form a matrix with N rows and N columns, where N defines a number of feature elements. Values of the matrix may indicate a degree of relationship between the feature elements. The feature elements having a higher degree of relationship with the loT data inconsistency may be selected as the features of loT data inconsistency.

[0057] The set of inconsistency rules may further be used for refining the loT data collected from the plurality of loT devices for continuous training of the ML model.

[0058] Training the ML model may comprise training a plurality of ML models to address different types of performance in terms of accuracy, precision, recall and Flscore. The plurality of ML models may be selected among Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN). A person skilled in the art would understand that other equivalent models, not listed by name, could be alternatively used.

[0059] The method may further comprise performing data inconsistency prediction on live loT data, using the ML model. The method may further comprise, if a data inconsistency of live loT data is predicted, generating refinement actions using the set of inconsistency rules. The method may further comprise using the refinement actions to guide live data collection to avoid future inconsistencies. The method may further comprise performing a data inconsistency verification on live loT data, using the ML model and information in the labeled loT data. The method may further comprise, if a data inconsistency is verified, generating a resolution action using the set of inconsistency rules, to fix the loT data inconsistency.

[0060] Referring to Figure 7, there is provided an apparatus (HW) 71, in which functions and steps described herein can be implemented. Alternatively, the functions and steps described herein can be implemented in a virtual hardware instance, in an application or any equivalent, e.g. in the application domain.

[0061] The apparatus 71 may be an loT gateway, server, network node, radio base station, or other computing device which may be part of a cloud computing system, edge computing system, or which may be a standalone device.

[0062] The apparatus may reside in the gateway domain or in the application domain. The apparatus (physical or virtual) may be used in conjunction with an existing inconsistency detector to improve its operation and performance. The apparatus may be used to detect and predict loT data inconsistencies as well as to remedy such inconsistencies, for example by modifying the way future loT data is collected.

[0063] The apparatus 71 comprises processing circuitry 73 and memory 75. The memory 75 can contain instructions executable by the processing circuitry 73 whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0064] The apparatus 71 may also include non-transitory, persistent, machine-readable storage media 77 having stored therein software and/or instruction 79 executable by the processing circuitry 73 to execute functions and steps described herein. The apparatus may also include network interface(s) and a power source.

[0065] The instructions 79 may include a computer program for configuring the processing circuitry 73. The computer program may be stored in a physical memory local to the device, which can be removable, or it could alternatively, or in part, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium. [0066] Referring to figure 8, there is provided a virtualization environment 80 in which functions and steps described herein can be implemented.

[0067] The virtualization environment 80 (which may go beyond what is illustrated in figure 8), may comprise systems, networks, servers, nodes, devices, etc., that are in communication with each other either through wire or wirelessly, e.g. through a network interface component (NIC) comprising physical network interface(s). Some or all of the functions and steps described herein may be implemented as one or more virtual components (e.g., via one or more applications, components, functions, virtual machines, containers, etc.) executing on one or more physical apparatus in one or more networks, systems, environment, etc.

[0068] A virtualization environment provides hardware 81 comprising processing circuitry 83 and memory 85. The memory 85 can contain instructions executable by the processing circuitry 83 whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0069] The hardware 81 may also include non-transitory, persistent, machine readable storage media 87 having stored therein software and/or instruction 89 executable by the processing circuitry 83 to execute functions and steps described herein. [0070] The instructions 89 may include a computer program for configuring the processing circuitry 83. The computer program may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program may be stored in a physical memory local to the hardware 81, which can be removable, or it could alternatively, or in part, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0071] Referring to figures 7 and 8, there is provided an apparatus, 71, 81 for predicting Internet of Things (loT) data inconsistency. The apparatus comprises processing circuits 73, 83 and a memory 75, 85, the memory containing instructions executable by the processing circuits whereby the apparatus is operative to obtain labelled loT data, the loT data being collected from a plurality of loT devices by a monitoring system. The apparatus is operative to analyze characteristics of the labelled loT data and identify features of loT data inconsistency. The apparatus is operative to train, using the labelled loT devices data and the features of loT data inconsistency, a ML model to predict loT data inconsistency. The apparatus is operative to generate, using the labelled loT data and the features of loT data inconsistency, a set of inconsistency rules to be applied to live loT data predicted as inconsistent by the ML model. The apparatus may be a data inconsistency resolution system. The apparatus may be an loT gateway. The apparatus may be further operative to execute any of the steps described herein.

[0072] Still referring to figures 7 and 8, there is provided a non-transitory computer readable media 77, 87 having stored thereon instructions 79, 89 for predicting Internet of Things (loT) data inconsistency. The instructions may comprise any of the steps described herein.

[0073] Modifications will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that modifications, such as specific forms other than those described above, are intended to be included within the scope of this disclosure. The previous description is merely illustrative and should not be considered restrictive in any way. The scope sought is given by the appended claims, rather than the preceding description, and all variations and equivalents that fall within the range of the claims are intended to be embraced therein. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.