Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
INTERACTIVE APPARATUS AND METHOD FOR MONITORING PERFORMANCE OF A PHYSICAL TASK
Document Type and Number:
WIPO Patent Application WO/2004/019272
Kind Code:
A2
Abstract:
An interactive apparatus for monitoring and evaluating level of skill of at least one person or object performing a physical task in at least one area of activity (2), and a method for the same. The apparatus comprises: at least one remote input device for capturing images of at least one zone (6, 8, 10, 11) of the at least one area of activity (2); means to monitor changes in the captured images associated with performance of the physical task by the at least one person or object and to evaluate the changes against one or more predetermined criteria established for the physical task; and means to indicate level of success of the person or object in meeting the one or more predetermined criteria.

Inventors:
MANSFIELD RICHARD LOUIS (GB)
FLOWERS NICHOLAS JOHN (GB)
BAIR JACQUES ANDREW CLOUTIER (US)
Application Number:
PCT/GB2003/003681
Publication Date:
March 04, 2004
Filing Date:
August 21, 2003
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
SCYRON LTD (GB)
MANSFIELD RICHARD LOUIS (GB)
FLOWERS NICHOLAS JOHN (GB)
BAIR JACQUES ANDREW CLOUTIER (US)
International Classes:
G06T7/20; (IPC1-7): G06T1/00
Domestic Patent References:
WO1996006368A11996-02-29
Foreign References:
GB2293772A1996-04-10
US20020055383A12002-05-09
GB2337146A1999-11-10
Other References:
Patent Abstracts of Korea, YANG YOUNG SEOK : " Dance game system using image recognition" & KR 2002 004 435 A (YANG YOUNG SEOK) 16 January 2002 (2002-01-16) XP002263823
Attorney, Agent or Firm:
Jackson, Derek Charles (The Old Yard Lower Town, Claine, Worcester Worcestershire WR3 7RY, GB)
Download PDF:
Claims:
CLAIMS
1. An interactive apparatus for monitoring and evaluating level of skill of at least one person or object performing a physical task in at least one area of activity (2), characterised in that the apparatus comprises: at least one remote input device (4) for capturing images of at least one zone (6,8, 10,11) of the at least one area of activity (2) ; means to monitor changes in the captured images associated with performance of the physical task by the at least one person or object and to evaluate the changes against one or more predetermined criteria established for the physical task; and means to indicate level of success of the person or object in meeting the one or more predetermined criteria.
2. An apparatus as claimed in claim 1, characterised in that the at least one zone (6,8, 10,11) is a physical zone.
3. An apparatus as claimed in claim 1, characterised in that the at least one zone (6,8, 10,11) is a virtual zone.
4. An apparatus as claimed in claim 1,2 or 3, characterised in that the at least one remote input device (4) comprises a device responsive in the electromagnetic spectrum.
5. An apparatus as claimed in claim 4, characterised in that the device responsive in the electromagnetic spectrum is at least one camera means (4) adapted to capture digital images of the at least one area of activity (2).
6. An apparatus as claimed in claim 4, characterised in that the device responsive in the electromagnetic spectrum is at least one camera means (4) adapted to capture analogue images of the at least one area of activity (2).
7. An apparatus as claimed in claim 5 or 6, characterised in that the at least one camera means (4) is adapted and arranged to respond to visible radiation.
8. An apparatus as claimed in claim 5,6 or 7, characterised in that the at least one camera means (4) is adapted and arranged to respond to infrared radiation.
9. An apparatus as claimed in any preceding claim, characterised in that the means to monitor changes in the captured images comprises means to form at least one reference image (18) of the at least one zone (6,8, 10, 11) of the at least one area of activity (2) and to compare current captured images (20) with the at least one reference image (18).
10. An apparatus as claimed in claim 9, characterised in that the at least one reference image (18) incorporates, or has associated therewith, the one or more predetermined criteria against which the current captured images (20) are evaluated.
11. An apparatus as claimed in claim 9 or 10, characterised in that the current image (20) and the reference image (18) comprise a plurality of cells and in that cell data is formed based on the current images (20) and the at least one reference image (18).
12. An apparatus as claimed in claim 11, characterised in that statistical analysis of the cell data is effected to evaluate the monitored changes in the captured images.
13. An apparatus as claimed in claim 12, characterised in that the statistical analysis of the cell data is by subtraction between the current captured images (20) and the at least one reference image (18).
14. An apparatus as claimed in any one of claims 9 to 13, characterised in that the at least one reference image (18) is formed by a sampling technique.
15. An apparatus as claimed in any one of claims 9 to 14, characterised in that the at least one reference image (18) is formed by a filtering technique.
16. An apparatus as claimed in any preceding claim, characterised in that evaluation of the monitored changes in the captured images is in respect of size and/or shape and/or direction and/or speed and/or duration and/or temperature and/or quality of movement of the person or object performing the physical task.
17. An apparatus as claimed in any preceding claim, characterised in that evaluation of the monitored changes is effected by computer means (22) and associated software.
18. An apparatus as claimed in any preceding claim, characterised in that detected and monitored movement or physical change of the person or object is displayed on a screen.
19. An apparatus as claimed in claim 18, characterised in that the detected and monitored movement or physical change of the person or object is displayed on the screen in a selective manner.
20. An apparatus as claimed in claim 18 or 19, characterised in that different levels of sensitivity to detection of movement of the person or object are provided whereby some, but not all, movement of the person or object is arranged to be detected.
21. An apparatus as claimed in any preceding claim, characterised in that the means to indicate the level of success of the person or object in meeting the one or more predetermined criteria comprises visual means.
22. An apparatus as claimed in any preceding claim, characterised in that the means to indicate the level of success of the person or object in meeting the one or more predetermined criteria comprises audible means.
23. An apparatus as claimed in any preceding claim, characterised in that reward or award means are associated with the means to indicate the level of success.
24. An apparatus as claimed in any preceding claim, characterised in that the at least one zone (6,8, 10, 11) of the at least one area of activity (2) is defined to provide varying sensitivity therein to detection of movement therein of the person or object.
25. An apparatus as claimed in any preceding claim, characterised in that the one zone (6,8, 10,11) is adapted to be changed in size and/or shape and/or position and/or sensitivity with time.
26. An apparatus as claimed in any preceding claim, characterised in that the level of indicated success is in proportion to extent of interaction by the person or object with the at least one zone (6,8, 10,11).
27. An apparatus as claimed in any preceding claim, characterised in that a plurality of zones (6,8, 10,11) are provided, each having one or more of the predetermined criteria established in relation thereto, and in which interaction therewith by the person or object results in a particular level of success being indicated.
28. An apparatus as claimed in claim 27, characterised in that such plurality of zones (6,8, 10,11) are distinguished in respect of being rewarded, penalised, permitted, forbidden or restricted for entry by the person or object and interaction therewith by the person or object results in a correspondingly appropriate indication being provided.
29. An apparatus as claimed in claim 28, characterised in that the corresponding appropriate indication is an appropriate sound effect.
30. An interactive method of monitoring and evaluating level of skill of at least one person or object performing a physical task in at least one area of activity (2), the method comprising: providing at least one remote input device (4) for capturing images of at least one zone (6,8, 10,11) of the at least one area of activity (2); providing means to monitor changes in the captured images associated with performance of the physical task by the at least one person or object and to evaluate the changes against one or more predetermined criteria established for the physical task; and providing means to indicate level of success of the person or object in meeting the one or more predetermined criteria.
31. A method according to claim 30, characterised in that the at least one zone (6,8, 10,11) is a physical zone.
32. A method according to claim 30, characterised in that the at least one zone (6,8, 10,11) is a virtual zone.
33. A method according to claim 30,31 or 32, characterised in that the at least one remote input device (4) comprises a device responsive in the electromagnetic spectrum.
34. A method according to claim 33, characterised in that the device responsive in the electromagnetic spectrum is at least one camera means (4) adapted to capture digital images of the at least one area of activity (2).
35. A method according to claim 33, characterised in that the device responsive in the electromagnetic spectrum is at least one camera means (4) adapted to capture analogue images of the at least one area of activity (2).
36. A method according to claim 34 or 35, characterised in that the at least one camera means (4) is adapted and arranged to respond to visible radiation.
37. A method according to claim 34,35 or 36, characterised in that the at least one camera means (4) is adapted and arranged to respond to infrared radiation.
38. A method according to any one of claims 30 to 37, characterised in that the means to monitor changes in the captured images comprises means to form at least one reference image (18) of the at least one zone (6,8, 10, 11) of the at least one area of activity (2) and to compare current captured images (20) with the at least one reference image (18).
39. A method according to claim 38, characterised in that the at least one reference image (18) incorporates, or has associated therewith, the one or more predetermined criteria against which the current captured images (20) are evaluated.
40. A method according to claim 38 or 39, characterised in that the current image (20) and the reference image (18) comprise a plurality of cells and in that cell data is formed based on the current images (20) and the at least one reference image (18).
41. A method according to claim 40, characterised in that statistical analysis of the cell data is effected to evaluate the monitored changes in the captured images.
42. A method according to claim 41, characterised in that the statistical analysis of the cell data is by subtraction between the current captured images (20) and the at least one reference image (18).
43. A method according to any one of claims 38 to 42, characterised in that the at least one reference image (18) is formed by a sampling technique.
44. A method according to any one of claims 38 to 43, characterised in that the at least one reference image (18) is formed by a filtering technique.
45. A method according to one of claims 30 to 44, characterised in that evaluation of the monitored changes in the captured images is in respect of size and/or shape and/or direction and/or speed and/or duration and/or temperature and/or quality of movement of the person or object performing the physical task.
46. A method according to any one of claims 30 to 45, characterised in that evaluation of the monitored changes is effected by computer means (22) and associated software.
47. A method according to any one of claims 30 to 46, characterised in that detected and monitored movement or physical change of the person or object is displayed on a screen.
48. A method according to claim 47, characterised in that the detected and monitored movement or physical change of the person or object is displayed on the screen in a selective manner.
49. A method according to claim 47 or 48, characterised in that different levels of sensitivity to detection of movement of the person or object are provided whereby some, but not all, movement of the person or object is arranged to be detected.
50. A method according to any one of claims 30 to 49, characterised in that the means to indicate the level of success of the person or object in meeting the one or more predetermined criteria comprises visual means.
51. A method according to any one of claims 30 to 50, characterised in that the means to indicate the level of success of the person or object in meeting the one or more predetermined criteria comprises audible means.
52. A method according to any one of claims 30 to 51, characterised in that reward or award means are associated with the means to indicate the level of success.
53. A method according to any one of claims 30 to 52, characterised in that the at least one zone (6,8, 10, 11) of the at least one area of activity (2) is defined to provide varying sensitivity therein to detection of movement therein of the person or object.
54. A method according to any one of claims 30 to 53, characterised in that the one zone (6,8, 10,11) is adapted to be changed in size and/or shape and/or position and/or sensitivity with time.
55. A method according to any one of claims 30 to 54, characterised in that the level of indicated success is in proportion to extent of interaction by the person or object with the at least one zone (6,8, 10,11).
56. A method according to any one of claims 30 to 55, characterised in that a plurality of zones (6,8, 10,11) are provided, each having one or more of the predetermined criteria established in relation thereto, and in which interaction therewith by the person or object results in a particular level of success being indicated.
57. A method according to claim 56, characterised in that such plurality of zones (6,8, 10,11) are distinguished in respect of being rewarded, penalised, permitted, forbidden or restricted for entry by the person or object and interaction therewith by the person or object results in a correspondingly appropriate indication being provided.
58. A method according to claim 57, characterised in that the corresponding appropriate indication is an appropriate sound effect.
Description:
INTERACTIVE APPARATUS AND METHOD FOR MONITORING PERFORMANCE OF A PHYSICAL TASK This invention relates to an interactive apparatus and method for monitoring and evaluating level of skill of at least one person or object performing a physical task in at least one area of activity. The invention is applicable in the fields of entertainment (for example games), training and education, and employs computerised photo-analytical techniques to directly sense position and/or movement of part or all of a person or object within an area of activity, such as a scene, which may be an area of activity of a game, or a training or educational area of activity.

According to one aspect of the present invention there is provided interactive apparatus for monitoring and evaluating level of skill of at least one person or object performing a physical task in at least one area of activity, the apparatus comprising: at least one remote input device for capturing images of at least one zone of the at least one area of activity ; means to monitor changes in the captured images associated with performance of the physical task by the at least one person or object and to evaluate the changes against one or more predetermined criteria established for the physical task ; and means to indicate level of success of the person or object in meeting the one or more predetermined criteria.

According to a further aspect of the present invention there is provided an interactive method of monitoring and evaluating level of skill of at least one person or object performing a physical task in at least one area of activity, the method comprising: providing at least one

remote input device for capturing images of at least one zone of the at least one area of activity; providing means to monitor changes in the captured images associated with performance of the physical task by the at least one person or object and to evaluate the changes against one or more predetermined criteria established for the physical task ; and providing means to indicate level of success of the person or object in meeting the one or more predetermined criteria.

By"remote input device"there is meant herein a device that is not physically connected to a person or object performing the physical task.

The at least one zone may or may not be physically present or visible to a person. That is, a zone may be physical or virtual.

The at least one remote input device may comprise a device responsive in the electromagnetic spectrum, for example at least one camera means which may be adapted to capture digital or analogue images of the at least one area of activity.

The at least one camera means may be adapted and arranged to respond to visible and/or infrared radiation.

The means to monitor changes in the captured images may comprise means to form at least one reference image of the at least one zone of the at least one area of activity and to compare current captured images with the at least one reference image. The at least one reference image may incorporate, or have associated therewith, the one or more predetermined criteria against which the current captured images are evaluated.

The current image and the reference image may comprise a plurality of cells and cell data may be formed, based on the current images and the at least one reference image.

Statistical analysis of the cell data may be effected to evaluate the monitored changes in the captured images, and such as by subtraction between the current captured images and the at least one reference image.

The at least one reference image may be formed by a sampling and/or filtering technique.

Evaluation of the monitored changes in the captured images may be in respect of size and/or shape and/or direction and/or speed and/or duration and/or temperature and/or quality of movement of the person or object performing the physical task.

Evaluation of the monitored changes may be effected by computer means and associated software.

Detected and monitored movement or physical change of the person or object may be displayed, such as in selective manner, on a screen.

Different levels of sensitivity to detection of movement of the person or object may be provided whereby some, but not all, movement of the person or object may be arranged to be detected.

The means to indicate the level of success of the person or object in meeting the one or more predetermined criteria may comprise visual and/or audible means.

Reward or award means may be associated with the means to indicate the level of success.

The at least one zone of the at least one area of activity may be defined to provide varying sensitivity therein to detection of movement therein of the person or object. The at least one zone may be adapted to be changed in size and/or shape and/or position and/or sensitivity with time.

The level of indicated success may be in proportion to extent of interaction by the person or object with the at least one zone.

A plurality of zones may be provided, each having one or more of the predetermined criteria established in relation thereto, and in which interaction therewith by the person or object results in a particular level of success being indicated. Such plurality of zones may be distinguished in respect of being rewarded, penalised, permitted, forbidden or restricted for entry by the person or object and interaction therewith by the person or object may result in a correspondingly appropriate indication being provided, such as an appropriate sound effect.

The apparatus and method of the present invention may be particularly applied in the entertainment (for example games), training and education fields.

In the entertainment field, levels of success in meeting a goal may be indicated in dependence upon extent or nature of interaction, by one or more competing persons, with one or more monitored zones in one or more areas or rooms of activity. Success at one particular level of activity may result in one or more appropriate rewards and progression to a higher level of activity.

Similarly, failure of appropriate interaction may result in penalty.

In a simple form of game, a person may be asked to cross a playing field within specific criteria. This may be with single or multiple zones and with or without application of speed criteria. If the person manages to cross the field within the criteria, success is indicated. Points may be scored for every successful crossing and the points may be graded according to the level of difficulty. In competitive forms of such a game, players may compete against each other as they progress through different levels.

In the training field, carrying out of sequential processes by a person, such as in a manufacturing, testing or processing environment, which may include interaction with hazardous zones, may be monitored and evaluated. For example, a goal may be to train a person for an off-site task, such as to carefully enter a nuclear reactor and alter the position of an object without disturbing the environment.

In the education field, the apparatus and method of the present invention may, for example, be used for teaching controlled movement and/or bio-feedback, for performance control. Learning of a specific task by a person, such as involving a controlled muscle movement, or the learning of keyboard skills, may be monitored and compared with reference image data incorporating predetermined criteria. Success in meeting the criteria may result in reward and/or subsequent direction of the person to a next stage of task to be learned.

The following examples will provide a better understanding of the present invention and show more clearly how it may be carried into effect, reference being made to the accompanying drawings in which: Figure 1 is a schematic representation of an embodiment of apparatus according to the present invention; Figure 2 is a schematic plan view of an area of activity with zones in the apparatus of Figure 1 ; and Figure 3 is a schematic representation of a further embodiment of apparatus according to the present invention.

EXAMPLE 1 Referring to Figures 1 and 2, an entertainment game comprises an area of activity 2, such as at least one room of a building, or an outside area, which is monitored by at least one video camera 4. The area of activity 2 contains one or more zones 6,8, 10,11, which may or may not be monitored by the video camera or cameras 4.

A player 12 of the game starts at, and optionally returns to, a position 14 in the area of activity 2 and has to reach, and possibly return from, a goal 16, while being monitored by the camera or cameras 4 and while being assessed against certain predetermined criteria, such as established in relation to the zones 6,8, 10,11.

The camera or cameras 4 is or are arranged to capture digital images of the area of activity 2 and, if required, may be arranged to respond to infrared

radiation if the game is required to be played in conditions of darkness.

At least one reference image 18 (having a plurality of cells or pixels) of the area of activity 2, in the absence of the player 12, is captured by the camera or cameras 4 and stored. Such reference image 18 is arranged to incorporate, or have associated therewith, data appropriate to the criteria established in respect of the one or more zones 6,8, 10,11.

Current digital images 20 (having a plurality of cells or pixels) of the area of activity 2 are also captured by the camera or cameras 4 during movement of the player 12 across the area of activity 2. Such movement of the player results in changes to the current image 20 compared with the reference image 18.

Cell (pixel) data is formed by the apparatus based on the current image 20 and the reference image 18. Statistical analysis of the cell data is effected, the current image 20 being subtracted from the reference image 18 on a cell by cell basis, by computer-based means 22, to form a difference image 24.

The techniques for processing the current and reference images to form the difference image are known and described for example in PCT/GB 02/00762 (the disclosure of which is incorporated herein by reference), which also describes a sampling and/or filtering technique suitable for use in forming the reference image 18.

Based on the difference image 24 and having regard to the criteria specified in respect of the one or more zones 6, 8,10, 11, an indication is provided to the player 12,

such as by visual or audible alarm 26, of his or her level of success in moving across the area of activity 2 towards the goal 16. Detected and monitored movement of the player 12 may be displayed, such as in a selected manner, on a well-known form of screen (not shown).

The game may involve various levels of difficulty. At a basic level, discrimination may be set low in the apparatus so that perhaps only the speed at which the player 12 crosses the area of activity 2 to reach the goal 16 may be monitored and evaluated. Such goal 16 may, for example, comprise simulated valuable items, such as jewels, which are to be stolen by the player 12.

Provided the player 12 reaches the goal 16 within a predetermined specified time period criterion, the alarm 26 will not be activated.

At higher levels of the game, various criteria are introduced as elements of interference in one or more of the zones 6,8, 10,11, which are associated with the reference image 18 and which the player 12 has to anticipate and, if necessary, avoid to prevent being penalised and the alarm 26 being activated.

The zones 6,8, 10,11 may, for example, operate as follows. One or more"safety"zones 11 may be provided, in which the player 12 escapes surveillance by the camera or cameras 4 and can relax.

One or more"penalty"zones 8 may be provided in which the at least one camera 4 recognises the presence of the player 12 and the alarm 26 indicates that he or she has been detected, with a possible resultant penalty.

One or more"kill"zones 10 may be provided. If the player 12 enters such a zone 10, he or she is detected by the camera or cameras 4 and the alarm 26 is activated to indicate that the player 12 has failed at least that particular level of the game.

One or more"reward"zones 6 may also be provided which, if entered by the player 12, may result in the player 12 being provided with specific powers over other zones 8, 10.

Contours of safety 28 may be provided in which the level of wa-rning around the zones 6,8, 10 is graded 30,32, whereby early warning is provided to the player that a particular zone is being approached.

The apparatus of the present invention may be adapted whereby the positions of the various monitored zones 6, 8,10, 11 are arranged to change from time to time to increase the difficulty of the game.

The functions of the zones 6,8, 10,11 may be arranged to change over a period of time such that, for example, "kill"zone 10 becomes a"safety"zone, or"safety"zone 11 becomes a"penalty"zone, and further adding to the complexity of the game.

The game may be played by a single player 12 or by a competing plurality of players 12, who may be marked to assist identification by the camera (s) 4.

The apparatus may incorporate memory means (not shown) for storing information relating to performance by the player 12 in present and/or previous games.

EXAMPLE 2 Apparatus similar to that described in Example 1 is used for training personnel in a task, such as entering a nuclear reactor and crossing a hazardous area (area of activity 2), having regard to various zones (zones 6,8, 10,11) of different hazard levels, in order to reach, and possibly perform an operation on, an object (goal 16), such as for altering the position of the object without disturbing the environment.

EXAMPLE 3 Referring to Figure 3, apparatus according to the present invention is applied for educating a person 12 and providing training, such as for behavioural modification.

The person 12 is, for example, learning keyboard skills at a keyboard 34 and is, for example, learning to type a sequence of letters.

In similar manner to the apparatus of Figure 1, camera 4 captures current digital images 20 of the sequence being typed and also stores a reference image 18 appropriate to the correct sequence. The difference image data 24 formed by the computer-based means 22 is used to provide a measure of the typing accuracy and/or speed of the person 12 and an appropriate indication of success level is provided by the indicating means 26.

Different levels of difficulty may be introduced as training proceeds. These may include changes in time limits allowed for the task, more complex tasks, and possibly solitary tasks to allow training alone. Memory means (not shown) may also be included for storing

information relating to the player's performance in present and/or previous exercises.

In an alternative training programme, one or more muscles of the participating person 12 is or are arranged to be moved in a controlled manner as defined by the person 12 and such movement is visually monitored and evaluated against reference image data by the apparatus of the present invention.

Although specific examples of the present invention have been provided hereinbefore, the present invention is not limited to such examples and other applications of the apparatus and method of the present invention will become apparent to the skilled person, within the scope of the invention.