The Technolo y Field

This invention relates to a method for producing tones based on the scanned position of bodies in space. The invention further relates to a device for performing this method.

Present State-of-the-Art

In the Automotive, Engineering, Plastics and Food industries, camera systems are currently used primarily for control and measurement purposes; like code and character readings, print control, fault-finding, colour identification, position and product recognition, as well as counting and completeness checking. It is used especially where 100 % quality is required. It is one of the most important ways to achieve high quality products. For example, in the Entertainment industry, there is a known solution where the camera vision of a camera system is used to express the modern way of digital drawing by means of body movements. In terms of generating sounds for example, a method using tapes located on the floor is used. Their subsequent weighting by means of the human body - (a dancer), generates a tone. The method of creating tones based on sensing the position of bodies in space, based on the principle of the machine- vision of a camera system is however, not yet known.

Disclosure of the Invention

The nature of the Tone Generation method - based on the scanned position of bodies in space according to the invention, resides in the fact that the objects of interest are scanned in space and the height of the generated tones, their colour and loudness, are assigned according to the spatial coordinates of the objects. The coordinates of the object are determined by the machine-vision principle of a camera system when at least one camera senses a scene in which there are objects of interest that are different from the surrounding environment, and other objects with their characteristic colour, shape and/or dimensions. Each single snapshot is pre-processed by segmenting the image based on the occurrence of a given colour, which is given by the user's setting of the acceptable range of individual colour coordinates. On the resulting binary image, a Gaussian filter and morphological operations are applied to suppress noise - and to find contours that clearly bound the objects of interest. Each object of interest is determined by its moment characteristics, which determine the area of the object in the image, and two spatial coordinates of the centre of gravity. The third spatial coordinate of the centre of gravity of an object is determined, in the case of one-camera capture, after initial calibration according to the size of the area of the interest object in the image. In the case of capture by at least two cameras, the coordinate is determined by the superposition of the images from different cameras, taken at the same time.

The height of the tone is determined by the assigned spatial coordinate such that the image is, in the vertical direction, imaginatively divided into a set of, each corresponding to a certain tone height, and particularly - 1 to 88 sections of the corresponding piano keyboard.

The tone colour is determined by the additional assigned spatial coordinate so that the image is divided horizontally into 1 to 128 sections that agree with the General MIDI standard. Each section corresponds to the tone colour of a particular instrument from a group consisting of pianos, tuned drums, an organ, guitars, basses, strings, the sounds of the ensemble, a jigsaw, slips, whistles, electronic solo sounds, electronic accompanying sounds, electronic sound effects, ethnic, percussion and other sound effects.

The loudness of the tone is determined by the additional assigned spatial coordinate such that the closer the camera's object, the louder the sound is.

The device for carrying out the method - to the invention, includes a camera system consisting of one or more cameras for sensing the objects of interest in the scene; the output of the camera system is connected to the input of the tone generator, where its output is then connected - in the final stage, to a loudspeaker.

Thus, this Optoelectronic device - according to the invention, generates sounds or tones based on the spatial coordinates of bodies, objects or marks, or interest objects - that are then assigned pitch, tone and tone colour, or respectively, the type of musical instrument. The beginning and the end of the tone is controlled by uncovering and covering the interest object. These interest objects can be characterised by their colour, shape or dimensions. Based on the selected differences, a search for these interest objects is initiated, then their co-ordinates in space are determined, and the corresponding sound is generated.

In the case of the detection of multiple interest objects in an image, several tones differing in height, loudness, colour, and instrument type are periodically played back. Assignment of a given height, loudness, and tone colour to spatial coordinates of the interest objects can be altered to meet the user's needs or requirements. Clarify the Drawing

For closer explanations of the invention's nature, the accompanying drawings are included where they represent:

Fig. 1 - A scheme of the method of how to create tones based on the position of one body in space

Fig. 2 - A scheme of the method of how to create tones based on the position of three bodies in space

Fig. 3 - A scheme of the method of how to create tones based on the position of three bodies in a space with two cameras

Invention Implementation Examples

Example 1

In the exemplar embodiment of the tone generation method, based on the scanned position of bodies in space according to the invention (see Fig. 1), the object of interest - namely the colour mark on the dancer costume is scanned in space and, according to its spatial coordinates, height, colour and loudness factors are assigned to the generated tone. The coordinates of the object are determined by means of the machine-vision principle of Camera System 1, when one camera scans a scene with a dancer. The scanning frequency of the bodies is 5x per second. The height of the created tone is determined by the x coordinate; the image is in the vertical direction is fictively divided into 88 sections, which corresponds to a piano keyboard.

The tone colour is determined by the y coordinate, and the image is divided horizontally into 128 sections, and their number corresponds to the quantity of instruments chosen by the user. The set of available instruments agrees with the General MIDI standard (pianos, tuned drums, an organ, guitars, basses, strings, the sounds of ensemble, jigsaw, slips, whistles, electronic solo sounds, electronic accompanying sounds, electronic sound effects, ethnic, percussion and sound effects), in total it contains 128 instruments. The use of sound libraries is another possibility.

The loudness of the tone is determined by the third spatial coordinate, such that the closer the camera is to the object, the louder the sound is. The third spatial coordinate of the centre of gravity of the object is, in this case - (the use of a single one camera), related to the size of the interest object's area in the image, and requires initial calibration. The assignment of a given height, loudness and tone colour to spatial coordinates of interest objects can be altered according to the user's needs or requirements.

The device for carrying out the method - according to the invention in the exemplary embodiment ... (see Fig. 1), includes a Camera System (1) - consisting of a single camera for scanning the object of interest (2) in the scene; the output of the camera system is then connected to the input of the Tone Generator (3), where its output is then connected in the final stage to the Loudspeaker (4).

Example 2

In the course of using the methods and the device corresponding to Example 1, individual dancers of an ensemble generate sounds by the positioning of marks on their costumes (see Fig. 2), or on their hands in the dancing area. The tone's duration is generated by opening and closing their hands. By lifting or dropping their hand, the instrument type changes. The tone pitch changes according to the dancer's movements to the sides; while the tone loudness changes according to their moving forwards and backwards. In this way, the ensemble creates a complex dynamic sound according to their dance movements.

In an alternative embodiment - (see Fig. 3), the Camera System (1) comprised of a pair of cameras for scanning the position of interest objects (2) on the scene. In this case, the third spatial coordinate of the centre of gravity of the object is determined by the superposition of the images from different cameras taken at the same time.

Example 3

In the course of using this method and the device corresponding to Example 1, a child playing with a ball produces sounds that lead to their calming down, or for their entertainment.