BACKGROUND [0002] As crimes, for instance violence, thefts or robberies, become more common, the use of various safety camera systems has increased mark- edly. In a typical system, surveillance cameras are placed to monitor various targets. Successive images taken by the cameras, typically video images, can be transmitted to a central control room, where a person observes the images of the different cameras in different monitors and tries to identify a dangerous situation. Successive images taken by a camera can also be stored for in- stance on tape, and later, as a routine procedure or after a crime has taken place, a person views the tape and tries to search the images for the crime or the people responsible for it.

[0003] A problem in such systems is the need for much manpower and, in addition, that a person can make mistakes, particularly during the dark hours at night, i. e. a crime may remain unnoticed or be noticed too late to be stopped. Furthermore, motion is difficult to distinguish in images shot during the dark time. A solution is to illuminate the targets to be monitored, but it in- creases costs in the form of lighting devices and electricity consumed.

[0004] A solution to the problems is to use motion detectors placed in the area to be monitored. The problem is the limited coverage of the detec- tors, wherefore quite a large number of them have to be placed in the areas monitored, which causes extra costs.

BRIEF DESCRIPTION [0005] The object of the invention is to provide an improved method of identifying motion in successive images and an improved device for identify- ing motion in successive images.

[0006] According to an aspect of the invention, a method is pro- vided, comprising: generating difference data descriptive of the difference in size between each luminance pixel of the previous image and the correspond- ing luminance pixel of the present image; adding the generated difference data

to a cumulative motion register, in which motion register a data element corre- sponds to each luminance pixel of an image; and identifying such data ele- ments as motion whose value exceeds a predetermined threshold value.

[0007] According to an aspect of the invention, a device is provided, comprising: means for generating difference data descriptive of the difference in size between each luminance pixel of the previous image and the corre- sponding luminance pixel of the present image; means for adding the gener- ated difference data to a cumulative motion register, in which motion register a data element corresponds to each luminance pixel of an image; and means for identifying such data elements as motion whose value exceeds a predeter- mined threshold value.

[0008] According to an aspect of the invention, a device is provided that is configured to: generate difference data descriptive of the difference in size between each luminance pixel of the previous image and the correspond- ing luminance pixel of the present image; add the generated difference data to a cumulative motion register, in which motion register a data element corre- sponds to each luminance pixel of an image; and identify such data elements as motion whose value exceeds a predetermined threshold value.

[0009] Preferred embodiments of the invention are described in the dependent claims.

[0010] The invention is based on analyzing successive images generated by a surveillance camera by analyzing the differences between their luminance pixels, enabling automatic identification of motion. The procedure slightly resembles motion estimation used in video coding, but the motion reg- ister employed is cumulative, allowing a person who moves a short time and then stops to also be detected in the image. Motion identification performed in this manner is also significantly faster than traditional motion estimation, since the source of the motion is not interesting; instead, the interest in the invention is only in motion identification.

[0011] The method and device of the invention provide a plurality of advantages. The method enables the implementation of a safety camera sys- tem, whose control room requires less manpower than previously. Further- more, the work of a person working in the control room is facilitated because the system automatically detects motion, and the system is also able to auto- matically give an alarm.

[0012] The method also enables the analysis of tapes recorded by a

monitoring camera, the aim being to identify the interesting points in the tapes, i. e. the points including motion. The embodiments of the method also have other advantages, which will be described in detail below.

LIST OF FIGURES [0013] In the following, the invention will be described in detail in connection with preferred embodiments with reference to the accompanying drawings, in which Figure 1 is a block diagram of a device for identifying motion in suc- cessive images; Figure 2 is a flow diagram illustrating a method of identifying motion in successive images ; Figure 3 illustrates the processing of a luminance pixel in a motion register ; Figures 4A, 5A, 6A and 7A show four images selected from a se- quence of successive images; Figures 4B, 5B, 6B and 7B show the motion identified in Figures 4A, 5A, 6A and 7A as a black-and-white image; Figures 4C, 5C, 6C and 7C show an image zoomed to the motion identified in Figures 4A, 5A, 6A and 7A; Figure 8 illustrates the effect of moving a camera; Figure 9 illustrates the effect of the use of an optical zoom in a camera; and Figure 10 illustrates the processing of a luminance pixel in a motion register in the example of Figures 4A, 5A, 6A and 7A.

DESCRIPTION OF THE EMBODIMENTS [0014] The device for identifying motion in successive images will be described with reference to Figure 1. The basic principle of the device is simple; it is configured to generate difference data by subtracting, from each luminance pixel in the previous image, the corresponding luminance pixel in the present image. The device is also configured to add the generated differ- ence data to a cumulative motion register. In the cumulative motion register, a data element corresponds to each luminance pixel in the image. The device is configured to identify such data elements as motion whose values exceed a predetermined threshold value.

[0015] Let us now study the device shown in Figure 1 more closely.

The device is able to communicate with one or more image sources, in our ex- ample with two video cameras 158,160. Any known device generating suc- cessive images 100A, 100B can be used as the image source. Since the in- vention analyzes changes in the luminance pixels of successive images, the information obtained from the image source has to contain luminance informa- tion, either as separate luminance pixels or otherwise enable the separation or conversion of the luminance pixels from the information coming from the image source. The cameras 158,160 can be polled one at a time until motion is de- tected somewhere. If motion is detected in the images of several cameras, the image to be transmitted may be selected for instance based on the motion area, or images can be sent from these cameras alternately in a sequence of a given length.

[0016] Several speeds may be used in image processing. For ex- ample, if no motion is detected, the speed 1 image/second is used, whereas 15 images/second are used during pre-alarm and alarm. When motion is de- tected, the read speed is increased, i. e. a shift occurs to what is known as a pre-alarm state. If the motion is continuous, a shift occurs to an alarm state, and the system starts to send images and the zoom may also then be turned on. Images of the pre-alarm state can be stored and sent if the alarm state has to be entered. When the motion stops, the system returns to the pre-alarm state, where it remains for a predetermined time. If still no motion is detected, the alarm can be switched off and the read speed dropped to minimum.

[0017] Coding successive images, for instance a video image, is used to reduce the amount of data, allowing it to be stored more efficiently in some memory means or to be transferred using a telecommunication connec- tion. An example of a video coding standard is MPEG-4 (Moving Pictures Ex- pert Group). The image sizes employed vary, e. g. the cif size is 352 x 288 pix- els and the qcif size is 176 x 144 pixels. Typically, a single image is divided into blocks including information on brightness, colour and location. The data in the blocks are compressed by blocks using the desired coding method.

Compression is based on deleting less significant data. Compression methods are mainly divided into three different categories: spectral redundancy reduc- tion, spatial redundancy reduction, and temporal redundancy reduction. Typi- cally, different combinations of these methods are used for compression.

[0018] In spectral redundancy reduction, the YUV colour model is applied, for example. The YUV model utilizes the fact that the human eye is

more sensitive to changes in luminance, i. e. brightness, than to changes in chrominance, i. e. colour. The YUV model includes one luminance component (Y) and two chrominance components (U and V, or Cb and Cr). For example, a luminance block according to the H. 263 video coding standard is 16 x 16 pix- els, and both chrominance blocks, which cover the same area as the lumi- nance block, 8 x 8 pixels. The combination of one luminance block and two chrominance blocks is called a macro block. Each pixel, both in the luminance and chrominance blocks, can receive a value between 0 and 255, i. e. eight bits are needed to present one pixel. For example, luminance pixel value 0 means black and value 255 white.

[0019] Successive images 100A, 100B coming to a device may thus conform to the YUV model, wherein luminance pixels constitute a separate component, for instance 352 x 288 luminance pixels in a cif-sized image.

[0020] Accordingly, there may be several image sources, and thus the device has to have an image source selector 102 for selecting the informa- tion stream to be processed with the device. No image source selector 102 is needed if there is only one image source. In addition, in our example, refer- ence number 102 also denotes the control circuit of an image source, for in- stance a camera.

[0021] The image 104 of the selected image source is applied to a frame buffer 106, in which the luminance and chrominance components of the image are stored.

[0022] The luminance component of the previous image in the frame buffer 106 is applied 108 to a second frame buffer 110.

[0023] Next, in block 116, difference data descriptive of the differ- ence in size between each luminance pixel of the previous image and the cor- responding luminance pixel of the present image are generated. This differ- ence data may be generated for instance by subtracting, from each luminance pixel in a previous image 114, the corresponding luminance pixel of a present image 112, or by some other mathematical operation giving the same result.

[0024] The difference data 118 generated are then added to a cu- mulative motion register 126. The cumulative motion register 126 is a memory whose size is the height of the image times the width of the image. A feasible read area is [-255,255] if an image in the YUV format is used. The memory may also be smaller, but in that case the data stored therein have to be pre- processed.

[0025] In an embodiment, the device is configured to reduce the values of the data elements in the motion register that are unequal to zero to- wards zero by a predetermined amount in association with each addition of dif- ference data. In the example of Figure 1, this is represented by block 120, to which the generated difference data 118 are applied, and in which a predeter- mined number is subtracted from each data element of the difference data ap- plied 125 from the motion register 126, and the obtained difference, to which the difference data 124 are added, is then applied to the motion register 126 as the new value of the data element. The predetermined number may be for instance one or another number greater than one. The subtraction may natu- rally also be carried out by adding the difference data to the motion register 126, in which the predetermined number is then subtracted from each data element that deviates from zero. The device includes a control block 136 for controlling the operation of the different parts of the device. A predetermined number 122 is applied to block 120 from the control block 136, the number be- ing subtracted from each data element. The control block 136 also controls 140 the image source selector 102.

[0026] In case of change of image sources, the content of the mo- tion register 126 is zeroed always before difference data on a new image source is added to the motion register 126.

[0027] Once the previous image and the present image are proc- essed in the above described manner, the control block 136 scans 128 the motion register 126 in a predetermined manner, for instance by rows or col- umns. Such data elements 126 in the motion register whose values exceed a predetermined threshold value are then identified as motion in the control block 136. In an embodiment, the control block identifies motion by blocks, for instance by luminance macro blocks. The blocks may also partially overlap. In an embodiment, motion is identified in a block of an image only if a predeter- mined number of data elements corresponding to said block in the motion reg- ister 126 exceeds the predetermined threshold value.

[0028] Figure 3 illustrates the processing of a luminance pixel in the motion register 126. The example is based on prototypes and experiments made by the applicant. The basis of the example is a series of events shot with a video camera, the series showing a light table. A person arrives, and leaves a black briefcase on the table. The horizontal axis in Figure 3 shows numbers 1 to 150 of successive images, and the vertical axis the value range of both

the luminance pixel and the data element in the motion register. Changes in the value of the luminance pixel under study are shown by curve 300. The variation in the value of the data element corresponding to said luminance pixel in the motion register 126 is shown by curve 302.

[0029] In images 1 to 13, the luminance pixel selected for study is part of the light table. The slight variation in the pixel value is noise. In images 14 to 18, the luminance pixel is part of the black briefcase that is placed on the table. Similarly, in images 19 to 150 the luminance pixel is part of the black briefcase left on the table.

[0030] As one sees by studying curve 302, our example uses an embodiment, wherein the values of the data elements stored in the motion reg- ister 126 and deviating from zero are reduced towards zero by a predeter- mined amount in connection with each addition of difference data. In our ex- ample, this predetermined amount is one. A reduction in the value of a data element results in that in image 137 the entire briefcase has disappeared from the motion register 126, i. e. merged into the background.

[0031] Since such data elements whose values exceed a predeter- mined threshold value are identified as motion in the device, it is interesting to consider what the suitable threshold value would be in the example of Figure 3. A suitable threshold value for removing noise, but not motion, could be for instance between 10 and 15, in which case motion would be detected about at image 130.

[0032] In an embodiment, if an image contains detected motion, the control block 136 controls 144 the frame buffer 106 to apply the present image to a zooming block 132 in the area 130 to be zoomed. The control data 144 contains information about which area to be zoomed in the image is sent as image 130 to the zooming block 132. From the control block 136, the control data 134 are applied to the zooming block 132, which control data indicate the ratio between the sizes of images received and transmitted. Said ratio is 1: 1 if zooming is not used. Automatic zooming to motion is an optional function, which is turned on via a user interface in the device (not shown in Figure 1).

Thus, the zooming block 132 zooms to the detected motion, for instance by in- terpolation of the zoomed area to the size of the original image by using known interpolation methods.

[0033] In motion identification, an area where there is motion is de- tected. This area is outlined such that the ratio of height and width always re-

mains the same compared with the original image. In addition, the area to be zoomed should be slightly larger than the area where there is motion. The enlargement should also have a maximum, which depends on the image size used and the accuracy of the camera. For example, a 100-fold enlargement on a qcif-sized image is no longer reasonable.

[0034] In an embodiment, the control block 136 and/or the zooming block 132 stores the zooming point in memory in order to control the change in the area to be zoomed. Information on the zooming point can be utilized by al- lowing only a change of a predetermined size in the zooming point between two successive images. In addition to or instead of the zooming point, the con- trol block 136 and/or the zooming block 132 may store the zooming ratio in memory. Information on the zooming point can be utilized by allowing only a change of a predetermined size in the zooming point between two successive images. By limiting the change in the zooming point and/or the variation in the zooming ratio, the image becomes more pleasant to look at, since the image is not changed too rapidly based on motion; instead, in steps of a given size, for example. Adjusting the speed of change is an optimization task between the monitoring speed of detected motion, image quality, and information content.

[0035] In addition, it is recommendable to let the motion area move freely inside the zooming area within given limits. These methods result in a smoothly moving image.

[0036] A zoomed or non-zoomed image 142 is then applied to an optional encoding block 150, with which the image may be encoded, if desired.

Encoding a zoomed image brings about the advantage that the result of the encoding improves, since redundant information is removed from the image, whereby for example a burglar in the zoomed image, can be identified more easily.

[0037] The encoding block 150 may be for instance some known video encoder, e. g. an mpeg4 encoder. The image 142 may also be applied for storage to a memory means included in the device or to a memory means connected to the device, e. g. a computer hard disk. The image 142 may also be applied to a viewing apparatus, e. g. a monitor; in this case, depending on its presentation format, the image may have to be converted.

[0038] In an embodiment, the device also includes a block 146 for generating a black-and-white image 148 from the data elements included in the motion register 126. In the manner described in Figure 1, the data ele-

ments can be applied 144 to block 146 from the control block 136, since the control block 136 reads 128 the data elements from the motion register 126, but block 146 could naturally read them also directly from the motion register 126. A black-and-white image may be implemented for instance by presenting the data elements exceeding a predetermined threshold value in the black- and-white image in white and the other data elements in black, or by present- ing the data elements exceeding a predetermined threshold value in black and the other data elements in white. This embodiment brings about the advantage that the motion detected is much more easy to detect in a black-and-white im- age 148 presenting motion than in the original image 100A, 100B. To still briefly revert to the example of Figure 3, it may be stated that if the threshold value used for generating a black-and-white image is the same value as for motion identification, i. e. for instance 10 to 15, then the person's motion and the remaining of the briefcase on the table are detected in the black-and-white image about up to image 130. Naturally, if desired, a smaller number can be set as the threshold value of a black-and-white image, whereby motion is visi- ble longer in the black-and-white image generated.

[0039] In an embodiment, the predetermined threshold value used in motion identification is adjusted to adjust the sensitivity of motion identifica- tion. Generally this can be indicated by generating a threshold value for the present image in the control block 136 by using a magnitude descriptive of the average value of the luminance pixels of the present image and the threshold value of the previous image. For example, for image k, the average of the lu- minance pixels can be calculated by wherein pij is a luminance pixel, and n x m is the size of the image.

All luminance pixels of the image do not have to be used in the calculation of the average.

[0040] Then a threshold value for image k tk = paA + q, and (2) = (+,)/ (), (3) wherein tk 1 is the threshold value of the previous image. Formula 2 uses a linear function, wherein p and q are constants, but a non-linear function may also be used. In accordance with Formula 3, the threshold value may also be weighted, r and s being constants, and by changing them the threshold

value, and thus the sensitivity of the device to large variations in brightness, can be adjusted. The user interface of the device may include a controller for stepless control of the threshold value to achieve the desired sensitivity.

Weighting is worthwhile in order for the system not to react to large rapid varia- tions in brightness, for instance a change in brightness caused by a lightning or a cloud. The average does not have to be used in Formulae 2 and 3, but some other statistical magnitude descriptive of the average value can also be used. Noise can also be measured from random pixels, and adjust the sensitiv- ity accordingly.

[0041] In an embodiment, the device is configured, for instance by the control block 136 being commanded, to move the luminance pixels of the previous image to the luminance pixels showing the same image area in the present image, to move the corresponding data elements in the motion register 126 to said same image area, and to zero the data elements corresponding to the luminance pixels in the present image only in the motion register 126, if the image area in the present image has moved relative to the image area of the previous image, i. e. the camera 158, 160 that generated the pictures has been moved.

[0042] In an embodiment, the device is configured, for instance by the control block 136 being commanded, to modify the image area in the pre- vious image to correspond to the image area of the present image by interpo- lating therein the missing luminance pixels corresponding to the image area of the present image, and to modify the image area of the motion register 126 to correspond to the image area of the present image by interpolating therein the missing data elements corresponding to the image area of the present image, if the image area in the present image is zoomed optically relative to the image area of the previous image, i. e. optical zooming is used in the camera 158,160 that generated the pictures.

[0043] In an embodiment, the device is configured, for instance by the control block 136 commanding 140 the selection block 102, to control 138 the camera 160 that generated the images to move in the direction of the mo- tion identified in the image. In this case, the camera 160 includes for instance an electric motor for turning the camera 160 in the direction of the motion. The control command 138 indicates the magnitude of the required motion for in- stance as degrees.

[0044] In an embodiment, the device is configured, for instance by

the control block 136 commanding 140 the selection block 102, to control 138 the camera 160 that generated the images to zoom optically to the motion identified in the image. In this case, the camera 160 includes an electrically controlled optical zoom for zooming the image generated by the camera 160 to the motion. The control command 138 indicates the required change in the zooming ratio.

[0045] In an embodiment, the device also includes a block 154 for giving an alarm if motion is identified. In Figure 1, reference number 156 de- notes an alarm. The same reference numbers 154,156 also describe an em- bodiment, wherein the device includes a block 154 for transmitting successive images, a black-and-white image generated or a zoomed image using a tele- communication connection 156. Images may be transmitted with or without an alarm. An image to be transmitted may be an image originating from the origi- nal image source, a generated black-and-white image or a zoomed image. The camera that generated the images may also have been moved or optically zoomed in the direction of the motion identified in the image. The telecommu- nication connection uses known solutions, e. g. a wireless radio link, a local area network, the Internet, a fixed dedicated line, or some other way to transfer data between two points. The device may also be configured to send only an image wherein motion is detected via the telecommunication connection 156. It may also be automatically concluded from the sensitivity selected by the sys- tem if it is worth to transmit an image coming from a camera or a black-and- white image of an image in the motion register. A black-and-white image may be transmitted if the brightness of the area to be shot drops below a given level. In this way, the device operates in nearly the dark such that a person looking at the image finds it easier to detect motion.

[0046] The device blocks shown in Figure 1 can be implemented as one or several application-specific integrated circuits (ASIC). Other implemen- tations are also feasible, e. g. a circuit constructed from separate logics com- ponents, or a processor and software. A hybrid of these implementations is also feasible. When selecting the implementation, a person skilled in the art takes into account for instance the requirements set on the size and power consumption of the device, the required processing power, manufacturing costs and production volumes. It should be noted that Figure 1 mainly de- scribes functional entities, whereby the parts of a practical hardware implementation may deviate from what is described, since the final question is integration degree: how to implement the device for identifying motion in

gration degree: how to implement the device for identifying motion in succes- sive images to implement the requested functionality as efficiently as possible and at reasonable cost in said application.

[0047] The above device is applicable to a plurality of purposes. It may be used to study stored image material, and search the material auto- matically for points where there is motion. The device may be placed in con- nection with a surveillance camera or in a central control room. A very inex- pensive version can be made of the device. In an inexpensive version, the de- vice also includes an inexpensive camera and can be coupled to the mains current. The device also includes parts enabling it to operate as a subscriber terminal in a mobile network. This allows the device to be used to guard for in- stance its owner's country house, and the device sends image and/or an alarm to the owner's subscriber terminal as soon as it detects motion. The device thus operates as a burglar alarm or a surveillance device for checking the situation at the country house. Naturally, other sensors, for instance a fire alarm, can be coupled to the device. A person skilled in the art is able to think of also other applications for the basic device described.

[0048] Next, the implementation of some operations of the device is described by a pseudo code conforming to the syntax of the C programming language.

Header #define mmin (a, b) ( (a) < (b) ? (a) : (b))/* macro for a minimum of two values typedef struct { int um ; int **cb ; int **cr ; } data ; typedef struct I int mid [2] ;/* midpoint */ int hor ;/* horizontal magnitude int ver ;/* vertical magnitude */

} mot ; typedef struct { int lines ;/* height of image int columns ;/* width of image */ mot motion ;/* area where motion is */ mot zoom ;/* area where zoom is */ data previous ;/* previous image */ data present ;/* present image */ int **movement ;/* motion register */ int **shape ;/* filtered motion register */ int sensitivity ;/* sensitivity */ } secur ;/* variable defined in main function */ Motion register function [0049] Adds the difference between images to the motion register and reduces the data in the register pixel by pixel by one. int Add motion (secur *frame) { int i, j ;/* counters */ int r = frame->lines ;/* lines in image */ int c = frame->columns ;/* columns in image */ for (i=0 ; i<r ; i++) { for (j=0 ; j<c ; j++) { frame->movement [i] [j] += ((frame->present). lum [i] D]-(frame->previous). lum [i] [j]) ; if (frame->movement [i] D] >0) frame->movement [i] [j]-= 1 ; if (frame->movement [i] D] <0) frame->movement [i] [j] += 1 ; }

return (1) ; } Motion check function int Motion check (secur *frame) { int i, j, k, l ;/* counters */ int r = frame->lines ;/* lines in image */ int c = frame->columns ;/* columns in image */ int found=0 ;/* motion found or not */ int sum ;/* number of motion pixels in block*/ int min k, max k, min I, max I ;/* variables of detected motion */ int sens = frame->sensitivity ;/* sensitivity */ for (i=0 ; i<r ; i++)/* copy and filter motion register */ { for (j=0 ; j<c ; j++) { frame->shape [i] 0] =frame->movement [i] D])/sens ; } min_k = c ;/* initialize minimums and maximums */ max k = 0 ; mins = r ; maxi = 0 ; for (k=0 ; k<c-7 ; k+=4)/* search for motion and its area in 8x8 blocks */ {/* blocks overlap for (l=0 ; I<r-7 ; I+=4) { sum=0 ; for (i=0 ; i<64 ; i++) { if (frame->shape [l+i/8] [k+i% 8] ! =0)

sum++ ; I if (sum>sens) { found=1 ; if (k<min_k) mink=k ; if (k>maxk) maxk=k+3 ; if (l<min_l) min-1=1 ; if (l>max_l) maux-1=1+3 ; } } } if (found==1)/* if motion, calculate its width, height and midpoint */ { (frame->motion). mid [0] = (mink+maxk)/2 ; (frame->motion). mid [1] = (min I+max I)/2 ; (frame->motion). hor = maxk-mink ; (frame->motion). ver = maxi-mini ; } else/* if no motion, width, height and midpoint are as in the image */ { (frame->motion). hor = c ; (frame->motion). ver = r ; (frame->motion). mid [0] += mmax ( ( (c/2)- (frame->motion). mid [0])/8, 1) ; (frame->motion). mid [1] += mmax ( ( (r/2)- (frame->motion). mid [1])/8, 1) ; } return (1) ; I

Zooming function int Zoom control (secur *frame) f int r frame lines ;/* lines in image */ int c = frame-> columns ;/* columns in image */ int left, right, up, down ;/* edges of zoom */ inti ;/* counter */ /* calculate new zoom height and width (compared with motion) */ if ( (frame->motion). hor < (c* (frame->motion). ver)/r) { if ( (frame->zoom). ver >= (frame->motion). ver +r/6) { (frame->zoom). ver-= mmin (abs ( ( (frame->zoom). ver- (frame->motion). ver-r/6)/4), 4) ; (frame->zoom). ver = mmax (mmin ( (frame->zoom). ver, r), r/2) ; (frame->zoom). hor = (c* (frame->zoom). ver)/r ; (frame->zoom). hor = mmax (mmin ( (frame->zoom). hor, c), c/2) ; (frame->zoom). hor = ( (frame->zoom). hor/2) *2 ; (frame->zoom). ver = ( (frame->zoom). ver/2) *2 ; if ( (frame->zoom). ver < (frame->motion). ver + r/10) f (frame->zoom). ver += mmin (abs ( ( (frame->zoom). ver- (frame->motion). ver-r/10)/4), 8) ; (frame->zoom). ver = mmax (mmin ( (frame->zoom). ver, r), r/2) ; (frame->zoom). hor = (c* (frame->zoom). ver)/r ; (frame->zoom). hor = mmax (mmin ( (frame->zoom). hor, c), c/2) ; (frame->zoom). hor = ( (frame->zoom). hor/2) *2 ; (frame->zoom). ver = ( (frame->zoom). ver/2) *2 ;

I }/* end of if statement */ else { if ( (frame->zoom). hor >= (frame->motion). hor + c/6) { (frame->zoom). hor-= mmin (abs ( ( (frame->zoom). hor- (frame->motion). hor-c/6)/4), 4) ; (frame->zoom). hor = mmax (mmin ( (frame->zoom). hor, c), c/2) ; (frame->zoom). ver = (r* (frame->zoom). hor)/c ; (frame->zoom). ver = mmax (mmin ( (frame->zoom). ver, r), r/2) ; (frame->zoom). hor = ( (frame->zoom). hor/2) *2 ; (frame->zoom). ver = ( (frame->zoom). ver/2) *2 ; I if ( (frame->zoom). hor < (frame->motion). hor + c/10) { (frame->zoom). hor += mmin (abs ( ( (frame->zoom). hor- (frame->motion). hor-c/10)/4), 8) ; (frame->zoom). hor = mmax (mmin ( (frame->zoom). hor, c), c/2) ; (frame->zoom). ver = (r* (frame->zoom). hor)/c ; (frame->zoom). ver = mmax (mmin ( (frame->zoom). ver, r), r/2) ; (frame->zoom). hor = ( (frame->zoom). hor/2) *2 ; (frame->zoom). ver = ((frame->zoom). ver/2) *2 ; } }/* end of else statement */ /* move zoom midpoint towards midpoint of motion */ for (i=0 ; i<2 ; i++) { if ( (frame->zoom). mid [i] < (frame->motion). mid [i]-4)

(frame->zoom). mid [i] +=mmin (abs ( ( (frame->zoom). mid [i]- (frame->motion). mid [i])/4), 8) ; I if ( (frame->zoom). mid [i] > (frame->motion). mid [i] +4) { (frame->zoom). mid [i] -=mmin (abs ( ( (frame->zoom). mid [i]- (frame->motion). mid [i]))/4, 8) ; } /* keep zoom inside image */ (frame->zoom). mid [0] = mmin (mmax ( (frame->zoom). mid [0], (frame->zoom). hor/2), c- (frame- >zoom). hor/2) ; (frame->zoom). mid [1] = mmin (mmax ( (frame->zoom). mid [1], (frame->zoom). ver/2), r- (frame- >zoom). ver/2) ; left = (frame->zoom). mid [0]- (frame->zoom). hor/2 ; right = (frame->zoom). mid [0] + (frame->zoom). hor/2 ; up = (frame->zoom). mid [1]- (frame->zoom). ver/2 ; down = (frame->zoom). mid [1] + (frame->zoom). ver/2 ; Convert (frame, left, right, up, down) ;/* Call interpolation function */ return (1) ; } [0050] Next, a method of identifying motion in successive images is described with reference to the flow diagram in Figure 2. The method starts at 200. Next, in 202, the image source to be used is selected, e. g. one camera out of several cameras. If there is only one camera, 202 need not be executed.

[0051] Once the camera is selected, an image is read from the camera in 204. The image should be such that the luminance component can be read directly from it, or such that the luminance component can be gener- ated based on information included in the image. In 206, the luminance com-

ponent is stored in memory.

[0052] Next, in 208, the following image is read from the camera.

The sensitivity of the system is adjusted in 210 in the above described manner.

[0053] Next 212 is entered, where difference data descriptive of the difference in size between each luminance pixel in the previous image and the corresponding luminance pixel in the present image is generated. In the same 212, the difference data generated are added to a cumulative motion register, in which motion register a data element corresponds to each luminance pixel in the image.

[0054] Next, in 214, such data elements are identified as motion whose value exceeds a predetermined threshold value. If desired, an alarm may be given of the motion identified.

[0055] The existence of an alarm is checked in 216. If an alarm is on, 230 is entered, otherwise 222.

[0056] Optional zooming to the motion is carried out in 230 in the manner described previously. After optional zooming, 232 is entered, where the zoomed image is applied to an encoder. In an embodiment, the camera may be movable or enable optical zooming, and the existence of these proper- ties is checked in 234.

[0057] If these properties do not exist, then 208 is entered, where the next image is read from the camera. Should said properties be in use, 236 is entered, where the camera is controlled in the desired manner, for instance in the above manner by turning the camera towards the motion identified or zooming optically to the motion. In 240, the values of the previous image and the motion register are modified in the above manner in order for motion identi- fication to operate correctly, so that the motion caused by the camera moving or the optical zooming would not be identified as motion. The process then continues from 240 to 208, where the following image is read from the camera.

[0058] In 222, a check is made to see if the camera employed needs to be changed. If it does not have to be changed, 234 is entered, from where the operation continues in the above-described manner. If the camera is changed to another, 228 is entered, where the motion register is emptied, so that the motion register values of the previous camera do not confuse the op- eration in the new camera.

[0059] Figure 2 does not show the end of the execution of the method, since, in principle, it can be ended at any point. A natural ending point

is when the desire is not to study any more successive images. The device of the above type is applicable to the execution of the method, but also other kinds of devices are suited to the execution of the method. The attached de- pendent method claims describe the preferred embodiments of the method.

Their operation was described above in connection with the device, and the description is therefore not repeated herein.

[0060] Next, a broad example of image processing will be described to fully make clear the operational principles of the method and device for iden- tifying motion in successive images.

[0061] Figures 4A, 5A, 6A and 7A shows four images, selected from a sequence of successive images. Since all images of the image sequence cannot be presented herein, four representative images have been selected for this purpose.

[0062] A camera shoots a table in the corner of a room. As Figure 4A shows, there is a radio cassette recorder on the table.

[0063] Next, in Figure 5A, a later image in the sequence is shown, where a person has walked to the table.

[0064] In Figure 6A, the person has taken the radio cassette re- corder and is holding it.

[0065] In Figure 7A, the person and the radio cassette recorder have exited the image, which only shows the corner of a room and an empty table.

[0066] Figure 10 illustrates the processing of a luminance pixel in the motion register in connection with the example of Figures 4A, 5A, 6A and 7A. Figure 10 is drawn up using the same principle as Figure 3, i. e. the hori- zontal axis shows numbers 1 to 30 of successive images and the vertical axis the value range of the luminance pixel and the data element in the motion reg- ister. The variations in the value of the luminance pixel under study are shown by curve 1000. The variation in the value of the data element corresponding to said luminance pixel in the motion register is shown by curve 1002.

[0067] In images 1 to 13, the luminance pixel selected for study is part of the radio cassette recorder. The slight variation in the pixel value is noise. In images 14 to 15, the luminance pixel is part of the person's hand. In images 16 to 30, the luminance pixel is part of the table.

[0068] As is detected by studying curve 1002, an embodiment is in use in our example, wherein the values of data elements included in the mo-

tion register and deviating from zero are reduced towards zero by a predeter- mined amount in connection with each addition of difference data. In our ex- ample, this predetermined amount is five.

[0069] Figures 4B, 5B, 6B and 7B show how the motion detected in Figures 4A, 5A, 6A and 7A looks like as a black-and-white image. The selec- tion in our example is that motion is presented in black and immobility in white.

[0070] Since Figure 4A shows no motion, Figure 4B is entirely white.

[0071] Motion is detected in Figure 5A and is shown in Figure 5B as a black figure presenting a person.

[0072] Figure 6B shows that the person is holding the radio cas- sette recorder. As is also shown in Figure 6B, the removal of the radio cassette recorder is visible as a motion at the point where it was. As is seen in Figure 7B, after the person's motion has disappeared, the removal of the radio cas- sette recorder is still visible. In the above-described manner, the threshold value for motion identification, the threshold value for generation of a black- and-white image and the predetermined amount to be used in the reduction in connection with the addition of difference data can be adjusted to adjust the time during which the motion is visible in the image. If the entire black-and- white image sequence is thought of as a moving image, then it shows the per- son's motion as a black figure that approaches the table, grabs the radio cas- sette recorder, and exits the corner of the room from the image area shot by the camera. A reduction in the value of a data element results in the motion identified in the image finally disappearing from the black-and-white image, i. e. when the values of the data elements included in the motion register are suffi- ciently reduced, the black radio cassette recorder also disappears from the black-and-white image.

[0073] Figures 4C, 5C, 6C and 7C show how an image zoomed to the motion identified in Figures 4A, 5A, 6A and 7A looks like.

[0074] A comparison of Figures 4A and 4C shows that in Figure 4C, no zooming is yet used on the motion, since, in accordance with Figure 4B, no motion is yet identified therein.

[0075] Figure 5C shows how the image is zoomed to the motion identified in the image, i. e. towards the person who walked into the image.

[0076] In accordance with Figure 6C, zooming has been continued.

[0077] In accordance with Figure 7C, zooming is directed towards

the removed radio cassette recorder, since, in accordance with Figure 7B, the removal remained visible as motion.

[0078] Once no more motion is identified in an image, the image area can be restored to normal, for instance by stepless zooming back to the original image area.

[0079] Next, the effect of moving the camera will be illustrated in Figure 8. Using the previously described embodiment, the camera that gener- ated the images is guided to move in the direction of the motion identified in the image. The previous image is outlined by frame 800, and the present im- age by frame 802. The obliquely checked area 804 is the old image area that is omitted in the study performed to identify motion. The horizontally checked area 806 is an area that exists only in the new image, for which area only the data elements corresponding to the luminance pixels in the present image are zeroed. A motion vector 808 between the previous image 800 and the present image 802 determines the direction in which the camera was moved. The common area 810 of the previous image 800 and the present image 802 is an area, for which the luminance pixels of the previous image 800 are transferred to the point of the luminance pixels included in the present image 802 and de- scribing the same image area, and, similarly, the corresponding data elements in the motion register are transferred to the point of said same image area. In practice, the data elements in the motion register and the luminance pixels of the previous image 800 in the frame buffer are moved a distance specified by the motion vector 808 in the opposite direction.

[0080] Finally, Figure 9 illustrates the effect of the use of an optical zoom in a camera in the above methods. Using the previously described em- bodiment, the camera that generated the images is directed to zoom optically in the direction of the motion identified in the image. The previous image is out- lined by frame 900, and the present image by frame 902. The checked area 904 is the area that is visible only in the previous image 900. The common part of the previous image 900 and the present image 902 is area 906. The image area 900 in the previous image is modified to correspond to the image area in the present image 902 by omitting area 904 from it, and by interpolating the missing luminance pixels that correspond to the image area in the present im- age 902 therein, and the image area in the motion register is modified to cor- respond to the image area in the present image 902 by interpolating the miss- ing data elements corresponding to the image area in the present image 902

therein.

[0081] Although the invention is described above with reference to the example according to the accompanying drawings, it is apparent that the invention is not limited thereto, but can be modified in a variety of ways within the scope of the inventive idea disclosed in the attached claims.