Technical Field

[0001] This disclosure relates generally to monitoring assets on a smart grid system and personnel accessing those assets.

Background

[0002] The traditional electrical power grid is evolving into a smart grid. Smart grid integrates the traditional electrical power grid with information and communication technologies. Such integration empowers the electrical utilities providers and consumers, improves the efficiency and the availability of the power system while constantly monitoring, controlling and managing the demands of customers. A smart grid is a complex network composed of millions of devices and entities including substations, transformers, voltage regulators, switches, meters and other equipment connected on a network. Such a massive network comes with many security concerns and vulnerabilities.

[0003] A smart grid network includes many components and most of them are located in remote premises thai, are usually unattended which makes them vulnerable to physical access. Additionally the network used for communication between devices on the smart grid may have limited data capacity.

[0004] A security system is needed that provides security of any premises on the smart grid making use of the communication network already deployed to reach devices on the grid.

Summary

[0005] Certain embodiments described herein provide methods and systems for monitoring assets such as components, devices, or equipment at premises throughout a smart grid. One exemplary computer implemented method comprises receiving image data by an onsite system located at a particular location or at one of the premises of a utility grid. The image data includes a face recorded by one or more cameras located on the same premises. Face print information is extracted from image data of the face. The face print information is less than an amount of data comprising the image data of the face.

[0006] Asset management information from an asset management system is also received by the onsite system. The asset management information includes usage and event data from devices on the network throughout the smart grid. The asset management information includes information from devices on the premises that may be connected using a narrowband network. The method involves generating a security alert by correlating the asset management information with the face print information and transmitting in real time the security alert to a monitoring center on the smart grid. The security alert is transmitted using any existing network deployed to communicate with devices on the smart grid. The existing network may be a narrow band network with very low bandwidth. The security alert generated would be of a size and format necessary for transmission on such a network.

[0007] Another exemplary embodiment further comprises receiving multiple image data from various cameras. First, image data which contains a face is received by the onsite system. A first location is determined associated with the image data. A second image data of the face is subsequently received and a second location is determined associated with the second image data. A vector is determined that corresponds to a trajectory from the first location to the second location and the vector information is included in the security alert. As the security alert is transmitted via an existing network that may have low bandwidth, the security alert generated is of a size and format necessary for transmission on such a network. [0008] Another exemplary embodiment comprises receiving current image data of a visual output of a device located on the premises. Abnormal operation of the device may be determined by comparing the current image data to stored image data for a visual output of the device reflecting normal operation. A security alert is generated comprising the results of the comparison wherein an amount of data for the results of the comparison is less than an amount of data for the image data of the visual output. The security alert may be transmitted in real time to a monitoring center. The security alert is to be transmitted on an existing network and therefore needs to meet the requirement of size and format for transmission on such a network.

[0009] In other embodiments, a computer-readable medium (such as, for example, random access memory or a computer disk) comprises code for carrying out these methods.

[0010] These embodiments are mentioned not to limit or define the disclosure, but to provide examples of embodiments to aid understanding thereof. Embodiments are discussed in the Detailed Description, and further description is provided there. Advantages offered by the various embodiments may be further understood by examining this specification.

Brief Description of the Figures

[0011] These and other features, aspects, and advantages of the present disclosure are better understood when the following Detailed Description is read with reference to the accompanying drawings, where:

[0012] Figure 1 is a diagram illustrating an operating environment for monitoring personnel according to certain embodiments;

[0013] Figure 2 is a diagram illustrating an operating environment for monitoring visual output of a device;

[0014] Figure 3 is a flow chart illustrating a method for generating a security alert according to certain embodiments; and [0015] Figure 4 is a flow chart illustrating a method for generating a security alert for visual output of a device;

Detailed Description

[0016] Smart grids provide a resource such as electricity from centralized distribution generation stations to customers through transmission and distribution systems. These transmission and distribution systems can be located at various sites and contain assets such as meters, routers, collectors and any other equipment that requires surveillance. A local onsite system operates at a site that houses the equipment required for a transmission system or a distribution system to operate. Cameras connected to the onsite system monitor the equipment located throughout the premises of the site. Video or images are captured by the cameras as image data and are provided to the onsite system. The image data is used by the onsite system to detect, identify, or track movement of personnel within the site. In another embodiment, the image data is used by the onsite system to determine abnormal operations of a device.

[0017] In one embodiment, the onsite system analyzes the image data to detect faces. If a face is detected, the onsite system extracts face print information from any face detected in the image data. The face print information comprises a digitally captured representation of a face that can be used for security purposes because it is as individual as a person's fingerprints. The digital representation of the face may be obtained by measuring nodal points of the face and determining a digital code which represents the face print. The nodal points may be, for example, the distance between the eyes, width of the nose, depth of eye sockets, length of jaw bone, etc. The face print information comprises an amount of data less than the image data. The onsite system may determine the identity of the face recorded by comparing the face print information extracted to a database of known faces. Identity of the face determines whether the face is of a known person. If the face is known, the database of known faces may provide identity information such as name, authorization and contact information. [0018] The onsite system may also analyze image data to track movement of personnel within the site. Image data may be provided from one or more cameras to the onsite system. When the onsite system receives multiple image data that differs in location or time and the multiple image data displays the same face, the onsite system determines the location associated with each image data to track movement. A vector is determined that corresponds to the trajectory of movement of a person based on the different locations identified for each image data received. The vector provides information on the movement of the person. If the movement is determined to be towards assets that are monitored, a security alert may be generated to notify that a person is approaching monitored assets.

[0019] In another embodiment the onsite system may analyze image data of a visual output device (i.e. a screen, a meter panel, etc.) that is located on the premises. The onsite system stores image data of the visual output device that reflects normal operation. A camera provides current image data of the output device to the onsite system. The onsite system compares the current image data received to stored image data for the visual device to determine the operating condition of the device.

[0020] In addition to receiving image data from cameras, the onsite system also collects or receives asset management information. Asset management information includes asset management information such as status and event information for various assets throughout the smart grid including the assets on the premises. The asset management information may be received from assets on the premises. For example the assets on the premises may be on a local area network and may relay to the onsite system any status or event information. Alternatively or in addition to, asset management information may be received from an asset management system. The asset management system may be housed on the premises with the onsite system or may be located remotely.

[0021] The onsite system correlates the face print information and/or the vector information with asset management information and generates a security alert. The security alert is transmitted in real time to a monitoring center via an existing network that is used by the devices on the smart grid. The existing network may be a narrow band network and therefore the security alert must be generated by the onsite system to be a size appropriate for transmission on the band width of narrow band network.

[0022] As defined herein "assets" includes any device, component, or piece of equipment capable of performing functions related to the distribution and monitoring of a utility resource. An asset can be a meter or meters located at a facility that measures the consumption of a utility such as gas, water, or electric power. Other examples of assets include routers, collectors, host computers, hubs or any other device that facilitates the distribution and monitoring of a utility resource.

[0023] As defined herein "premises" includes any facility or property that houses assets that are under camera surveillance. Such premises may be a substation housing equipment for the transmission and distribution of a resource.

[0024] Security Alerts from Face Print Information or Trajectory of Movement

[0025] Referring now to the drawings, FIG. 1 depicts an operating environment of premises under surveillance. Cameras 101A-101D located on the premises monitor areas such as entryway 102 to assets. Cameras 101A-101D may be capturing image data at all times or may be enabled in response to a command or detection of activity at the premises.

[0026] Cameras 101A-101D capture video as image data that is transmitted to onsite system 1 10. Onsite system 1 10 may be located on the premises. Along with the image data, cameras 101A-101D may transmit the time the image data was recorded and the location of the camera. FIG. 1 shows camera 101 A monitoring entryway 102 so that the image of person 103 is captured as image data 104A as the person approaches entryway 102. Cameras 101B, 101C, and 101D, capture image data of person 105 as the person moves in the direction shown by arrow 102. Camera 101 A sends to onsite system 1 10 image data 104 A. Likewise camera 10 IB sends to onsite system 1 10 image data 104B, camera 101 C sends to onsite system 1 10 image data 104C, and camera 101D sends to onsite system 1 10 image data 104D. [0027] Onsite system 1 10 comprises image data module 1 1 1 , asset security management information module 1 12, image extraction module 1 14, security alert module 1 13, and memory 1 15. As shown in FIG. 1 , camera 101 A monitors entry way 102 and sends the captured image data 104A to onsite system 1 10. The image data module 1 1 1 of onsite system 1 10 receives the image data 104A and may store it in memory 1 15 as image data 121. The image data module 1 1 1 analyzes the image data 104A received to detect faces of any person on image data 104A.

[0028] If a face is detected in the image data 104A by image data module 1 1 1 , the image extraction module 1 14 extracts face print information 122 from the image data 104A received from camera 101 A. The face print information 122 comprises a digital representation of the face. The digital representation of the face may be obtained by measuring nodal points of the face and determining a digital code which represents the face print. The face print information 122 uniquely identifies the individual on the image data 104A received from camera 101 A. The face print information 122 comprises less than the amount of data comprising the image data of the face 104A or 121.

[0029] The image data module 1 1 1 may also detect and track movement of personnel from image data received. In FIG. 1 , onsite system receives image data 104B- 104D from cameras 101B- 101D, respectively. When the image data module 1 1 1 receives multiple image data from one or more cameras, the image data module may compare the image data received to determine if the same face was captured by the different cameras. If the image data module 1 1 1 determines that the same face is captured in the different image data 104B-104D, the image data module 1 1 1 determines the location associated with each image data. The image data module 1 1 1 uses the locations to determine a vector that corresponds to a trajectory of movement of the person 105 as shown by the image data 104B-104D received. The recognition that the same face appears in the multiple image data may be determined by comparing the image data of the face from one image data to the image data of the face of another image data. Alternatively, the recognition of the same face may be performed by the image extraction module extrapolating face print information from each of the image data 104B- 104D and comparing the face print information to determine if the same face appears in the multiple image data 104B-104D.

[0030] In another embodiment, the image extraction module 1 14 may identify a face on the image data, thereby identifying the person 103 or 105. The image extraction module may use the image data 121 or the face print information 122 and a face database 123 to identify the person 103 or 105. The face database 123 may contain image data of faces with identity information associated with each face. Instead or in addition to containing image data of faces, face database 123 may contain face print information of faces along with associated identity information for each face print information. By comparing the image data of the face to stored image data in face data base 123 or by comparing the face print information 122 to stored face print information in a face data base 123, the identity of the person associated with the face may be determined. Therefore, the onsite system 1 10 would have the face print information 122 and any information identifying the face such as name, position, contact information, authorization, etc.

[0031] The asset security management information module 1 12 of onsite system

1 10 receives asset management information regarding the status of any asset or equipment on the network that is associated with any location under surveillance by onsite system 1 10. Status information for any asset may comprise usage and event data specifying the usage of the resource provided by the asset and/or any event reported to the asset management system. Additionally, status information may include the time the usage or event information was gathered by the asset and/or the time it was received by the asset security management information module 1 12. Events may include but are not limited to power loss, suspected tampering, network time synchronization issues, meter failure, etc. Asset security management information may be received from any of the assets on the premises or may be received from asset management system 130. An asset management system 130 may be a head-end system or a meter data management system that controls and monitors a metering network. The asset management system 130 may be located at the premises with the onsite system or at a remote location. [0032] The security alert module 1 13 of onsite system 1 10 correlates the face print information with asset management information. The correlation identifies associations between the face print information 122 and the asset management information. Associations may be made based on time, location, type of event received, identity of personnel, or any other factor that would correlate the face print information to the asset management information received. For example, asset security management information module 1 12 would receive asset management information gathered at time T for any assets located behind entry way 102. Asset management information received may indicate that a meter is not communicating at time T and intervention by personnel is required. The asset management information (meter outage) may be associated with a face print extracted by image extraction module from image data recorded at time T by camera 101 A which monitors entry to assets behind entryway 102. Similarly, if asset management information received for assets behind entryway 102 do not disclose that any intervention by personnel is required and face print information from image data 1 15 recorded from camera 101 A is received that reveals that a potential unauthorized entry by personnel 103 may be occurring, a correlation is made between the face print information and asset management information that would disclose a potential unauthorized entry to assets behind entryway 102.

[0033] Security alert module 1 13 generates a security alert 125 based on the correlation of the asset management information with the face print information 122. Additionally, if movement of personnel was tracked and a vector of the trajectory of movement was determined by image data module 1 1 1 , the vector information 126 is included in the security alert 125 generated. The security alert 125 may include any information pertaining to the correlation. The information included in the security alert may include but is not limited to time, location, face print information, or asset management information. For example, security alert 125 may include the time the image data was recorded, the location or identity of the camera that recorded the image data, the location or identity of the premises under surveillance, the face print information along with any identity of the individual identified by the extracted face print information, and any usage or event data from any of the assets under surveillance. If movement of personnel was detected, the vector information 126 may also be included in security alert 125.

[0034] The security alert is transmitted by the security alert module 1 13 of onsite system 1 10 to a monitoring system 140. The security alert is transmitted via network 145 which is the communication network already deployed to reach devices on the smart grid. This communication network may be a narrowband network and the security alert would need to be configured in a size appropriate for transmission in the bandwidth of such a network.

[0035] Note that a utility network such as an Advanced Metering Infrastructure

(AMI) network offers limited bandwidth for exchange of data. This limited bandwidth is most often incompatible with the transmission of video, photographs, full resolution images, etc. Typically the transfer of large files requires networks with bandwidth in excess of 1MBPS. A utility network may be capable of peak exchanges at several tens or low hundreds of kbps. Representing images of faces or assets by a reduced number of points such as with face print information allows relevant information such as identification of personnel, movement of personnel, abnormal operation of a device, or any information relevant to the security of assets to be conveyed using less amount of data.

[0036] Security Alerts from Visual Output of Assets

[0037] In another embodiment, shown in FIG. 2, camera 201 transmits the captured visual output of asset 210 as image data 202 to image data module 1 1 1 of onsite system 1 10. The onsite system 1 10 stores image data of the visual output of assets reflecting normal operations in visual output database 124. Therefore, visual output database stores image data for the visual output of device 210 that reflects normal operations of device 210. To determine abnormal operation, the image data module 1 1 1 compares the current visual output image data 202 received (visual output of device 210) to the stored image data reflecting normal operations of device 210 stored in visual output database 124. A threshold of normal operating conditions may be defined to determine the status of operation of device 210. The threshold may define by how much the visual output image data may deviate from stored visual output image data to determine abnormal or normal operation of device 210.

[0038] For example, in FIG. 2, the device 210 may be a utility meter providing a resource such as electricity. The stored image data stored in visual output database 124 for device 210 may include information about normal usage values for voltage, current, or frequency of the resource provided. The current visual output image data 202 received may also include information reflecting the current operating conditions for voltage, current, or frequency at the current time. Determining the abnormal operations of device 210 comprises comparing the visual output image data 202 to the stored image data in visual output database 124 based on thresholds defining tolerance levels for voltage, current, or frequency of the resource provided. The values defining tolerance levels may be represented numerically or by locations of meter gauges that reflect normal or abnormal operation.

[0039] If abnormal operation is determined, security alert 125 is generated by security alert module 1 13 comprising the results of the comparison. The amount of data for the results of the comparison is less than an amount of data for the image data 202 of the visual output of device 210. The security alert 125 generated from the results of the comparison is transmitted by the security alert module 1 13 of onsite system 1 10 to a monitoring system 140. The security alert is transmitted via network 145 which is the communication network already deployed to reach devices on the smart grid. This communication network may be a narrowband network and the security alert would be configured in a size appropriate for transmission in the bandwidth of such a network.

[0040] Alternatively or additionally, in visual output database 1 18, onsite system

1 10 may store data points of image data of the visual output of assets reflecting normal operations. The data points stored would reflect the minimal amount of information required to determine normal operation. The amount of data for data points being less than the amount of data for the image data 202 of the visual output of device 210. Image extraction module 1 14 would extract data points from image data 202 of the visual output of device 210. The data points extracted would be the minimal amount of information required to determine the status of operation reflected by image data 202. The amount of data for the data points is less than the amount of data for the image data 202. The data points extracted would be compared to data points stored in visual output database 124 for device 210 to determine abnormal operation. The security alert 125 would be generated from the results. The size of the security alert appropriate to be transmitted in the bandwidth of any network already deployed to reach devices on the smart grid.

[0041] Referring next to FIG. 3, shown is a flowchart that provides one example of the generation of security alerts from the correlation of asset management information with face print information. It is understood that the method 300 shown by the flowchart of FIG. 3 provides an example of the different types of functional arrangements that may be employed to implement the generation of a security alert from face print information.

[0042] Beginning with block 305, the online system receives image data from one or more cameras located on the premises. The cameras located on the premises capture video or images of a location on the premises. The captured video or image is received by the onsite system as image data. In addition to image data, the onsite system may also receive location and time information associated with the image data. The time information may reflect the time the image data was captured. The location information may reflect the coordinates or identify a location where the image data was captured.

[0043] Continuing with block 310, the online system determines if a face is detected in the image data. If a face is detected in the image data, the onsite system extracts face print information from the image data of the face at block 315. The face print information comprising information less than the amount of data comprising the image data of the face received.

[0044] At block 320, the onsite system may receive image data from another camera. This second image data is analyzed to determine if a face is detected in the second image data at block 325. At block 330, the onsite system extracts face print information from the second image data. The face print information extracted comprises less information than the amount of data comprising the image data of the face received. At block 335, the face print information extracted from the first image data at block 315 is compared to the second face print information extracted from the second image data at block 330. If the comparison of the face print information reveals that it is the same face, at block 340, a vector representing the trajectory of movement of the face is determined.

[0045] The face print information extracted may be compared against face prints in a database to determine the identity of the face. The identity may include, for example, name, position, contact information, authorization, etc. If no match is made between the face print extracted and the face print information in the database, the onsite system may determine that an unauthorized personnel is on the premises. This information may be included when the security alert is generated at block 350

[0046] Continuing with block 345, the onsite system receives asset management information from an asset management system. Asset management information comprises any event, status, or usage information from any of the assets associated with any location being monitored by the onsite system.

[0047] At block 350, the online system generates a security alert. If the various face print information that have been extracted do not match and therefore no trajectory has been determined, a security alert is generated based on the correlation of the face print information and the asset management information received. The security alert may also include any time or location information associated with the face print information. Additionally, the onsite system may compare the face print information to a face database to determine an identity for the face, such as name, title, contact info, etc. The identity information may also be included in the security alert.

[0048] If a vector representing a trajectory has been determined at block 340, the vector information is correlated with asset management information to generate a security alert at block 350. The vector information would provide the direction of movement of a person. The vector information may be included in the security alert. However, once movement of a person is determined towards monitored assets, the security alert my contain information disclosing the movement of the person but does not necessarily have to include the vector. In this embodiment, the security alert would include a notification of movement by a person towards monitored assets. The security alert may also include time and location information. Additionally any identity identified for the face by comparing the face print information to face prints in a database may also be included in the security alert.

[0049] Continuing with block 360, the security alert is transmitted via a network that is already deployed to communicate with devices. This network may be a narrow band network and the security alert generated would be of a size and format necessary for transmission on such a network.

[0050] Referring next to FIG. 4, shown is a flowchart that provides one example of the generation of security alerts from the visual output of a device. It is understood that the method 400 shown by the flowchart of FIG. 4 provides an example of the different types of functional arrangements that may be employed to implement the generation of a security alert from visual output information.

[0051] Starting at block 405, a camera monitoring a device sends image data of the visual output of the device to the onsite system. The onsite system at block 410 determines that the image data is visual output of a device.

[0052] Continuing with block 415, the image data is compared to stored image data for the device that reflects normal operations. If the comparison reveals that the image data reflects normal operation of the device, no action is taken by the onsite system as shown in block 430. If the comparison reveals that the image data reflects abnormal operation of the device monitored, a security alert is generated at block 425 including the results of the comparison. The amount of data for the results of the comparison is less than the amount of data for the image data received. This allows a security alert to be transmitted on a network with much less bandwidth.

[0053] Continuing with block 430, the security alert is transmitted on a network that is already deployed by the onsite system to communicate with devices on the grid. This network may be a narrow band network and the security alert generated would be of a size and format necessary for transmission on such a network. General Considerations

[0054] These examples given are only for illustrative purposes and not meant to limit the invention to these devices. While the present subject matter has been described in detail with respect to specific aspects thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily produce alterations to, variations of, and equivalents to such aspects. Accordingly, it should be understood that the present disclosure has been presented for purposes of example rather than limitation and does not preclude inclusion of such modifications, variations, and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. In particular, the steps performed to generate a security alert do not necessarily have to be performed in the order specified. For example, the extrapolation of face print and the receiving of asset management information may occur in any order. The heading used are for organizational purposes only and are not meant to be used to limit the scope of the description. As used throughout this application, the word "may" is used in a permissive sense, meaning "having the potential to" rather than "must". Similarly, the words, "include", "including", and "includes" mean "including, but not limited to."