Data processing unit and method, and surveillance module

Field of the invention

The present invention relates to a data processing unit for processing audiovisual input data from a data source, such as a digital video camera, comprising a plurality of data encoders connected to the data source. In a further aspect, the present invention relates to a surveillance module comprising a data processing unit. In an even further aspect the present invention relates to a data processing method for processing audiovisual input data from a data source, such as a digital video camera.

Prior art

US patent application US2003/0197629 describes a multiple codec imager system and method for compressing data. Multiple encoders are included on a single chip, and allow the same input data to be compressed in parallel. In the implementation described, mutual subsystems may be used by more than one encoder.

Summary of the invention

The present invention seeks to provide a method and system for parallel encoding and/or compression of audiovisual input data, which is particularly suited for camera surveillance applications. In camera surveillance applications, two usually mutually excluding requirements exist, in that it must be possible to monitor a scenery (or multiple sceneries) continuously, but also to use surveillance camera data for detailed image analysis, e.g. identification of persons.

According to the present invention, a data processing unit according to the preamble defined above is provided, in which the plurality of data encoders operate on the audiovisual input data in parallel, and in that a first one of the plurality of data encoders operates to provide an encoded lossless data stream, and a second one of the plurality of data encoders operates to provide a real time data stream with a reduced bit rate compared to a bit rate of the encoded lossless data stream. Such a hybrid data processing unit allows to use the same source data for different objectives, e.g. as mentioned above for monitoring a scenery and for detailed image analysis. Furthermore, the real time data stream with reduced bit rate will more easily fit in a given transmission channel (e.g. using a GPRS or UMTS network), allowing to

transmit (and view) the scenery image in a continuous manner, which is suitable for monitoring a scenery for surveillance.

The first one of the plurality of data encoders is in an embodiment of the present invention arranged to provide a high quality video data output stream. The data output stream may e.g. be encoded using the MPEG2 or MPEG4 formats, which are well known to the skilled person. Advantageously, this high quality data output stream is at full spatial and temporal resolution of the data source, e.g. at 25 frames/sec video, and an 640x480 image size. The bit rate of such a data stream may be as high as 20 Mbit/sec. The second one of the plurality of data encoders provides in a further embodiment a real time data stream at a reduced quality compared to the encoded lossless data stream. This output stream is e.g. encoded using the well known H.264 protocol, e.g. at four or eight frames/sec and a reduced image size, resulting in an output data stream bit rate of 60 kbit/sec. In a further embodiment, the first one and second one of the plurality of data encoders are arranged to provide synchronous time stamps in both the encoded lossless data stream and the real time data stream. This is particularly useful for later retrieval of data associated with the scenery under surveillance.

In a further aspect, the present invention relates to a surveillance module comprising a data processing unit according to the present invention. Furthermore, the surveillance module comprises a data input interface connected to the data processing unit, and a transceiver module connected to the output of the data processing unit for transmission of the encoded lossless data stream, the real time data stream, or both. The transmission mode may be dependent on the structure of the total surveillance system. In a further embodiment, the surveillance module further comprises a local storage device connected to the output of the first one of the plurality of data encoders, in which the data processing unit is further arranged to store the encoded lossless data stream in the local storage device. This ensures that all information available from the data source is stored. The amount of data to be stored thus depends on the data source bit rate, and the desired time period for which data needs to be stored. The local storage device may be incorporated in the surveillance module itself, or in another unit in the total surveillance system, e.g. a nearby surveillance module, a repeater, etc.

The surveillance module may be further arranged to remotely address stored encoded lossless data stream for retrieval by a remote receiver device. This may be accomplished using a suitable transmission channel, e.g. using a Wireless Local Loop (WLL) network. In a further embodiment, in which the real time data stream has a lower frame rate than the encoded lossless data stream, the data processing unit is further arranged to retrieve, e.g. on request from the remote receiver device, intermediate data from the encoded lossless data stream associated with intermediate frames from a predetermined time period of the real time data stream, and to provide the intermediate data to one of the plurality of data encoders. This allows to obtain intermediate frames originally not present in the real time data stream to be retrieved at a later time, e.g. in thumbnail format or in a resolution equal to the real time data stream frames using the second encoder. The intermediate frames then allow to better interpret a scenery or to select (a part of) one of the intermediate frames for retrieval at full resolution. A further aspect of the present invention relates to a remote receiver device comprising a transceiver unit arranged to interface with one or more surveillance modules according to an embodiment of the present invention, e.g. using a wireless transmission channel (UMTS, GPRS,...), and a central storage unit connected to the transceiver unit, the central storage unit being arranged to store all data received from the one or more surveillance modules. This enhances the capabilities of a surveillance system to review scenery registered by a number of surveillance modules afterwards. Also, the intermediate frames or detailed imagery selected by an operator and received from one of the surveillance modules is stored on the central storage unit.

In an even further aspect, the present invention relates to a data processing method for processing audiovisual input data from a data source, such as a digital video camera, in which the data processing method comprises providing in parallel an encoded lossless data stream and a real time data stream with a reduced bit rate compared to a bit rate of the encoded lossless data stream. The encoded lossless data stream may be a high quality video data output stream, and the real time data stream may have a reduced quality compared to the encoded lossless data stream. The data stream may be provided with (synchronous) time stamps, to enable and facilitate later retrieval. The method may further comprise transmission of the encoded lossless data stream, the real time data stream, or both. Also, the encoded lossless data stream may

be locally stored, and in a further embodiment, the stored encoded lossless data stream may be retrieved by remote addressing. In a further embodiment, the real time data stream has a lower frame rate than the encoded lossless data stream, and further comprising retrieving, on request, intermediate data from the encoded lossless data stream associated with intermediate frames from a predetermined time period of the real time data stream, and encoding the intermediate data for transmission. All transmitted data may be stored at a central location.

Short description of drawings The present invention will be discussed in more detail below, using a number of exemplary embodiments, with reference to the attached drawings, in which

Fig. 1 shows a schematic view of a surveillance module according to an embodiment of the present invention;

Fig. 2 shows a schematic view of a receiver to be used in co-operation with the surveillance module of Fig. 1;

Fig. 3 shows a schematic view of a surveillance system in which a surveillance module according to the present invention is used; and

Fig. 4 shows a schematic view of a further surveillance system set-up in which the present invention is used.

Detailed description of exemplary embodiments

Figure 1 schematically shows a surveillance module 1, in which a data processing unit 15 according to an embodiment of the present invention is used. The surveillance module 1 comprises a controller 13 which is connected to all elements of the surveillance module 1 as discussed below, for controlling the functioning of the surveillance module 1. A data source 19 is provided in the surveillance module 1, e.g. in the form of a digital video camera or an input interface for receiving data from one or more digital video cameras. The data source 19 is connected to a data processing unit 15. As an alternative, the data source 19 provides an analogue audio/video signal which is digitized in the data processing unit 15.

The data processing unit 15 comprises two encoders 20, 21 in this embodiment, which receive a data stream from the data source 19, in which a first encoder (high quality encoder 20) provides a first output data stream 22, and in which a second

encoder (real time encoder 21), provides a second output data stream 23. The high quality encoder 20 is a lossless encoder, as a result of which all information available at the data source 19 is comprised in the first output data stream 22. The high quality encoder 20 may be arranged to provide an MPEG-2 output data stream 22 at e.g. 20 Mbit/s. The real time encoder 21 is an encoder which is adapted to the rest of the system as described below. This real time encoder 21 is arranged to provide a second data stream 23 at a much lower bit rate (e.g. 60 kbit/s), which ensures that scenery from the data source 11 is available for monitoring purposes, even when a low capacity transmission channel 2 is used in the surveillance system. The first output stream 22 and second output stream 23 are connected to a transceiver 16 for transmission to further elements of a surveillance system as will be discussed below. The transceiver may e.g. be arranged to provide a wireless transmission of data via antenna 17 and a wireless transmission channel 2. As an alternative, the transmission channel 2 may be a hard- wired connection to other elements of the surveillance system. It will also be apparent to the skilled person that the transceiver 16 may be provided separately for each output data stream 22, 23, together with an associated antenna 17.

More in particular, the high quality encoder 20 may be arranged to provide an output data stream in television or firm quality (i.e. 25 or 50 frames/sec at maximum resolution), and the real time encoder 21 may be arranged to provide an output data stream at a lower frame rate, e.g. eight or four frames per second, at the same or a lower image resolution. The low bit rate output data stream 23 (e.g. at 60 kbit/s) can be transmitted at a very low capacity transmission channel 2, e.g. using a GPRS or UMTS network. As an example, the real time encoder 21 may encode an output data stream according to the H.264 protocol. The high quality output data stream 22 may also be transmitted in (near) real time when a transmission channel 2 with sufficient capacity is available, e.g. using a Wireless LAN or Wireless Local Loop (WLL) network.

Furthermore, the surveillance module 1 comprises a local storage device 12, e.g. in the form of mass data storage, such as, but not limited to, magnetic or optical disc storage, or electronic components, such as RAM, PROM, etc. The local storage device 12 is connected to the output of the first encoder 20, and under control of controller 13, all or part of the first output data stream 22 may be recorded on local storage device 12.

This assures that all information conveyed in the data stream from the data source 19 is available in the surveillance module for later processing or distribution.

Figure 2 shows a schematic diagram of a remote receiver device 3 co-operating with the surveillance module 1. The remote receiver device 3 may be positioned remote from the surveillance module 1 at a distance depending on the transmission and receiving characteristics. The receiver device 3 comprises receiver a transceiver unit 8, a central storage unit, for example a hard disk drive or a semiconductor memory, such as RAM (Random Access Memory), and a display device 10, both connected to the transceiver unit 8. In the normal mode of operation, the transceiver unit 8 receives the data stream from the transmitter 1 and stores it in the central storage unit 9 from where it can be retrieved immediately for real-time viewing on the display device 10 or at a later time to view a recorded event.

In a further embodiment of the present invention several transmitter-receiver pairs or repeaters could be cascaded in order to be able to bridge a certain transmission distance between the surveillance module 10 and the remote receiver device 3.

Figure 3 schematically shows an exemplary embodiment of a surveillance system comprising several surveillance modules 1 (connected to surveillance camera's 6) and a single remote receiver device 3 in a star configuration. In this configuration, the data streams from surveillance modules 1 to the receiver device 3 could be managed utilizing time division multiplexing, in which transmission time slots are dedicated to the different transmitters 16 of each surveillance module 1. The time slots may be synchronized so that the packets from the surveillance modules 1 do not interfere with each other at the receiver device 3. The transceiver unit 8 (see Fig. 2) could be designed to manage the different elements of the surveillance system and e.g. provide an interface to facilitate operations and management of the surveillance system. In a further embodiment of the present invention, a central controller 11 is provided, connected to the remote receiver device 3 and designed for managing the surveillance system. It is e.g. possible to set the priority levels of the different data streams, to assign the time slots for the different transmitters 16, perform software upgrades, etc. The controller 13 of the surveillance module 1 (see Fig. 1) is arranged to control the operation of transceiver 16. In normal operation, e.g. only the low bit rate, real time output data stream 23 is transmitted by the transceiver 16. This allows the remote receiver device 3 to monitor a complete scenery related to the data input source 19 in

real time, even when the transmission channel 2 only allows a limited data rate to be transmitted. The controller 13 of the surveillance module 1, and the central controller 11 of receiver device 3 may interoperate to set further modes of operation of the surveillance module 1 (e.g. using the same transmission channel 2, or a separate control channel). When a particular event is observed on the display device 10 of the receiver device 3, it may de desirable to obtain the high quality data associated with that event. The controller 13 may then be instructed to retrieve the relevant high quality data from the local storage device 12 and transmit it using the transceiver 16.

Both the real time output data stream 23 and the high quality output data stream 22 are advantageously provided with markers or time stamps. This allows to unambiguously retrieve parts of the high quality data stream 22 by using the same markers or (synchronous) time stamps of the real time output data stream 23.

The controller 13 may also be arranged to provide the high quality output data stream 22 in real time to the transceiver 16 when sufficient bandwidth on the transmission channel 2 is available. This may be advantageous when an event

(burglary, car theft, etc.) is in progress, and the observer at the display device 10 wants to follow this event in real time with sufficient detail to obtain certain details of the scenery (description of persons, directions, etc.).

Figure 4 schematically shows a further exemplary application of the present invention in which a mobile unit 12 such as a car is used. The mobile unit 12 comprises a transceiver unit 8 and a display 10 for receiving and inspecting the data streams from the surveillance modules 1. Depending on the available bandwidth of the transmission channel between surveillance module 1 and the mobile unit 12, the transmitted data stream from the surveillance module 1 could comprise the full frame rate and resolution data from the data sources 19 (high quality output stream 22 in real time), e.g. when a high capacity data network is available (e.g. a Wireless Local Lan (WLL) network). Also, in a further mode, previously captured high quality data may be retrieved from the local storage device 12 (recovery or play back mode). When only a lower bandwidth network is available (e.g. using a GPRS or UMTS network when driving towards the site under surveillance, before entering the WLL network area), reduced frame rate and/or frame resolution data (real time output data stream 23) may be received and monitored. This embodiment allows e.g. a guard to drive to a building

under surveillance, and monitor various cameras in the building before entering the building.

The remote receiver device 3 may also be used in a further mode of operation. In normal mode, the remote receiver device 3 only receives, stores and displays the low data rate real time data stream 23 from a surveillance module 1. When a particular event happens, or has happened, it may be desired to review the data streams. At the remote receiver device 3, only the real time data stream 23 is available as stored data, e.g. at four frames per second. When reviewing, a time period of interest may be zoomed in to, by requesting from the associated surveillance module 1, the intermediate frames between two available real time frames. When receiving such a request, the surveillance module 1 retrieves the associated lossless data from that time period from the local storage device 12 (e.g. at 25 frames per second), and provides it to one of the available encoders (20, 21). The selected encoder encodes the associated lossless data into the desired quality (e.g. thumbnail quality for a very low amount of data, or in the same quality as the real time data stream) and provides it to the transceiver 16.

Normally, the same transmission channel 2 is used for this reviewing data, so it may take some time to send the intermediate frames. The intermediate frames, once received by the remote receiver device 3, are shown on the display device 10, and preferably also stored on the central storage unit 9. As the intermediate frames are now available, the operator can more precisely review the past scenery.

Also, the operator may now select one of the available (low quality) frames, and have the remote receiver device 3 send a request for the complete full quality data associated with the selected frame. This will allow to obtain detailed information from the scenery, e.g. a number plate of a car, or the face of a person. Even more, the remote receiver device 3 may be arranged to allow to select only a part of interest of a single frame, and to transmit a request to the associated surveillance module 1 for only the associated data in full quality, in order to reduce the amount of data and time necessary for the surveillance module 1 to encode and transmit the data to the remote receiver device 3.

The additional data received by the remote receiver device 3 as a result from the operator requests (i.e. low resolution data of intermediate frames, and full quality data of (parts of) selected frames), is stored in the central storage unit 9. When another

operator, possibly at another location, but with (high speed) access to the central storage unit 9, reviews the same scenery, the additional intermediate frame data and full quality data will be helpful as it indicates that another operator has already reviewed that scenery part and may be of interest.