Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
ACOUSTIC ANALYSATION OF AN OPERATIONAL STATE OF PROCESS MACHINERY
Document Type and Number:
WIPO Patent Application WO/2017/191362
Kind Code:
A1
Abstract:
The invention relates to a method, comprising recording acoustic data of an environment (200) of a process machinery (201 -206) by a first sound acquisition unit (210-220) comprising a first acoustic sensor, transmitting recorded acoustic data to a data processing device (250), forming a first audio profile for recorded acoustic data by determining an acoustic spectrum for acoustic data received from the first acoustic sensor (210-220) and comparing it to pre-stored operational state audio in the memory of the data processing device (250). If the first audio profile corresponds to an audio profile of a malfunction operational state, the method further comprises locating the malfunction of the process machinery (201 -206) on the basis of a location of the first sound acquisition unit (210-220), determining at least one image sensor (230-238) at the location of the malfunction and transmitting a trigger signal by the data processing device (250) for triggering said at least one image sensor (230-238) for capturing one or more images and for transmitting the captured image data to said data processing device (250) for further analysis. The invention further relates to an operational state monitoring system and a computer program product performing the method.

Inventors:
VALKONEN MIKA (FI)
SEPPÄNEN MIKKO (FI)
Application Number:
PCT/FI2017/050324
Publication Date:
November 09, 2017
Filing Date:
April 28, 2017
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
PROCEMEX OY LTD (FI)
International Classes:
G05B23/02; G01H3/00; G01M13/00; G01S3/802; H04N7/18
Foreign References:
JP2002084530A2002-03-22
JP2667404B21997-10-27
JP2004085455A2004-03-18
JPH10294933A1998-11-04
JPH07261834A1995-10-13
JP2011027687A2011-02-10
US20160163168A12016-06-09
Other References:
"Acoustic source localization", WIKIPEDIA, 15 April 2016 (2016-04-15), XP055435815, Retrieved from the Internet [retrieved on 20170815]
Attorney, Agent or Firm:
BERGGREN OY (FI)
Download PDF:
Claims:
Claims:

1. A method, comprising: recording acoustic data of an environment of a process machinery by a first sound acquisition unit comprising a first acoustic sensor; transmitting recorded acoustic data to a data processing device; forming a first audio profile for recorded acoustic data by determining an acoustic spectrum for acoustic data received from the first acoustic sensor; comparing the first audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device; if the first audio profile corresponds to an audio profile of a malfunction operational state the method further comprises: locating the malfunction of the process machinery on the basis of a location of the first sound acquisition unit; determining at least one image sensor at the location of the malfunction; and transmitting a trigger signal by the data processing device for triggering said at least one image sensor for capturing one or more images and for transmitting the captured image data to said data processing device for further analysis.

2. A method according to claim 1 , wherein the method further comprises: recording acoustic data of the environment of the process machinery by at least a second sound acquisition unit comprising an acoustic sensor; transmitting recorded acoustic data to the data processing device; forming a second audio profile for acoustic data recorded by the second sound acquisition unit; comparing the second audio profile to pre-stored operational state audio profiles; and if at least the first and the second audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise, locating a malfunction of the process machinery on the basis of locations of the first and the second sound acquisition units, and intensity of abnormal noise in the first and the second audio profiles. 3. A method according to claim 2, wherein the method further comprises: recording acoustic data of the environment of the process machinery by at least a third sound acquisition unit comprising an acoustic sensor; transmitting recorded acoustic data to the data processing device; forming a third audio profile for acoustic data recorded by the third sound acquisition unit; comparing the third audio profile to pre-stored operational state audio profiles; and if at least the first, the second and the third audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise, locating a malfunction of the process machinery on the basis of locations of the first, the second and the third sound acquisition units, and intensity of abnormal noise in the first, the second and the third audio profiles.

4. A method according to claim 3, wherein the location on of the malfunction of the process machinery is determined on the basis of triangulation. 5. An operational state monitoring system comprising a first sound acquisition unit comprising a first acoustic sensor and a data processing device and wherein the operational state monitoring system is arranged to monitor an acoustic environment of a process machinery, and wherein the first acoustic sensor is arranged to record acoustic data of the environment of the process machinery and the first sound acquisition unit is arranged to transmit the recorded acoustic data to the data processing device, and wherein the data processing device is arranged to form a first audio profile for recorded acoustic data by determining an acoustic spectrum for acoustic data received from the first acoustic sensor, and to compare the first audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device and if the first audio profile corresponds to an audio profile of a malfunction operational state to locate the malfunction of the process machinery on the basis of a location of the first sound acquisition unit, determine at least one image sensor at the location of the malfunction; and transmit a trigger signal for triggering said at least one image sensor for capturing one or more images and for transmitting the captured image data to said data processing device for further analysis.

6. An operational state monitoring system according to claim 5, wherein the operational state monitoring system further comprises a second sound acquisition unit comprising a second acoustic sensor and wherein the second acoustic sensor is arranged to record acoustic data of the environment of the process machinery and the second sound acquisition unit is arranged to transmit the recorded acoustic data to the data processing device, and wherein the data processing device is arranged to form a second audio profile for recorded acoustic data, compare the second audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device, and wherein if at least the first and the second audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise, to locate a malfunction of the process machinery on the basis of locations of the first and the second sound acquisition, and intensity of abnormal noise in the first and the second audio profiles. 7. An operational state monitoring system according to claim 6, wherein the operational state monitoring system further comprises a third sound acquisition unit comprising a third acoustic sensor and wherein the third acoustic sensor is arranged to record acoustic data of the environment of the process machinery and the third sound acquisition unit is arranged to transmit the recorded acoustic data to the data processing device, and wherein the data processing device is arranged to form a third audio profile for recorded acoustic data, compare the third audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device, and wherein if at least the first, the second and the third audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise, to locate a malfunction of the process machinery on the basis of locations of the first, the second and the third sound acquisition units, and intensity of abnormal noise in the first, the second and the third audio profiles.

8. An operational state monitoring system according to claim 7, wherein the location of the malfunction of the process machinery is determined on the basis of triangulation.

9. A computer program product, stored on a computer readable medium and executable in a computing device, wherein the computer program product comprises instructions to perform a method according to any of the claims 1 to 4.

Description:
Acoustic analysation of an operational state of process machinery Technical field The present invention relates to a method for operational state monitoring of continuous manufacturing processes and process machinery, in which method at least acoustic monitoring is used for monitoring the operational state of machines. The invention also relates to an operational state monitoring system and a computer program product causing an apparatus to carry out the operational state monitoring method.

Background

In continuous manufacturing processes, there are materials or products constantly running through the machine. The condition of the machine itself can be monitored in order to detect possible machine malfunctions or process deviations that may cause those deviations to products or web breaks, but which could also cause unscheduled or planned downtimes of a machine/machinery or damage the machine itself. A machine operator could walk around the process machinery environment and listen to the operational state of machinery in order to hear if there are mechanical problems with machinery or if something else is wrong.

Summary

Now there has been invented an improved method and technical equipment implementing the method. Various aspects of the invention include a method, a monitoring system for acoustically monitoring the operational state of machinery by using at least one acoustic sensor, and a computer readable medium comprising a computer program stored therein, which are characterized by what is stated in the independent claims. Various embodiments of the invention are disclosed in the dependent claims. According to a first aspect of the invention, there is provided a method comprising: recording acoustic data of an environment of a process machinery by a first sound acquisition unit comprising a first acoustic sensor, transmitting recorded acoustic data to a data processing device, forming a first audio profile for recorded acoustic data, comparing the first audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device, and locating the malfunction of the process machinery on the basis of a location of the first sound acquisition unit if the first audio profile corresponds to an audio profile of a malfunction operational state.

According to an embodiment, the method further comprises: recording acoustic data of the environment of the process machinery by at least a second sound acquisition unit comprising an acoustic sensor, transmitting recorded acoustic data to the data processing device, forming a second audio profile for acoustic data recorded by the second sound acquisition unit, comparing the second audio profile to pre-stored operational state audio profiles, and if at least the first and the second audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise: locating a malfunction of the process machinery on the basis of locations of the first and the second sound acquisition units, and intensity of abnormal noise in the first and the second audio profiles. According to an embodiment, the method further comprises: recording acoustic data of the environment of the process machinery by at least a third sound acquisition unit comprising an acoustic sensor, transmitting recorded acoustic data to the data processing device, forming a third audio profile for acoustic data recorded by the third sound acquisition unit, comparing the third audio profile to pre-stored operational state audio profiles, and if at least the first, the second and the third audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise: locating a malfunction of the process machinery on the basis of locations of the first, the second and the third sound acquisition units, and intensity of abnormal noise in the first, the second and the third audio profiles. According to an embodiment, the location of the malfunction of the process machinery is determined on the basis of triangulation. According to an embodiment, the method further comprises: determining at least one image sensor at the location of the malfunction, and transmitting a trigger signal for triggering said at least one image sensor for capturing one or more images and to transmit the captured image data to said data processing device for further analysis.

According to a second aspect of the invention, there is provided an operational state monitoring system comprising a first sound acquisition unit comprising a first acoustic sensor and a data processing device wherein the operational state monitoring system is arranged to monitor an acoustic environment of a process machinery, and wherein the first acoustic sensor is arranged to record acoustic data of the environment of the process machinery and the first sound acquisition unit is arranged to transmit the recorded acoustic data to the data processing device, and wherein the data processing device is arranged to form a first audio profile for recorded acoustic data, compare the first audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device and locate the malfunction of the process machinery on the basis of a location of the first sound acquisition unit if the first audio profile corresponds to an audio profile of a malfunction operational state.

According to an embodiment, the operational state monitoring system further comprises a second sound acquisition unit comprising a second acoustic sensor wherein the second acoustic sensor is arranged to record acoustic data of the environment of the process machinery and the second sound acquisition unit is arranged to transmit the recorded acoustic data to the data processing device, and wherein the data processing device is arranged to form a second audio profile for recorded acoustic data, compare the second audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device, and wherein if at least the first and the second audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise, to locate a malfunction of the process machinery on the basis of locations of the first and the second sound acquisition, and intensity of abnormal noise in the first and the second audio profiles. According to an embodiment, the operational state monitoring system further comprises a third sound acquisition unit comprising a third acoustic sensor wherein the third acoustic sensor is arranged to record acoustic data of the environment of the process machinery and the third sound acquisition unit is arranged to transmit the recorded acoustic data to the data processing device, and wherein the data processing device is arranged to form a third audio profile for recorded acoustic data, compare the third audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device, and wherein if at least the first, the second and the third audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise, to locate a malfunction of the process machinery on the basis of locations of the first, the second and the third sound acquisition units, and intensity of abnormal noise in the first, the second and the third audio profiles. According to an embodiment, the location of the malfunction of the process machinery is determined on the basis of triangulation. According to an embodiment, the operational state monitoring system further comprises at least one image sensor wherein said data processing device is arranged to determine at least one image sensor at the location of the malfunction and to transmit a trigger signal for triggering said at least one image sensor for capturing one or more images and to transmit the captured image data to said data processing device for further analysis. According to a third aspect of the invention, there is provided a computer program product, stored on a computer readable medium and executable in a computing device, wherein the computer program product comprises instructions to perform a method comprising: recording acoustic data of an environment of a process machinery by a first sound acquisition unit comprising a first acoustic sensor, transmitting recorded acoustic data to a data processing device, forming a first audio profile for recorded acoustic data, comparing the first audio profile to pre-stored operational state audio profiles stored in the memory of the data processing device, and locating the malfunction of the process machinery on the basis of a location of the first sound acquisition unit if the first audio profile corresponds to an audio profile of a malfunction operational state. According to an embodiment, the method further comprises: recording acoustic data of the environment of the process machinery by at least a second sound acquisition unit comprising an acoustic sensor, transmitting recorded acoustic data to the data processing device, forming a second audio profile for acoustic data recorded by the second sound acquisition unit, comparing the second audio profile to pre-stored operational state audio profiles, and if at least the first and the second audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise: locating a malfunction of the process machinery on the basis of locations of the first and the second sound acquisition units, and intensity of abnormal noise in the first and the second audio profiles. According to an embodiment, the method further comprises: recording acoustic data of the environment of the process machinery by at least a third sound acquisition unit comprising an acoustic sensor, transmitting recorded acoustic data to the data processing device, forming a third audio profile for acoustic data recorded by the third sound acquisition unit, comparing the third audio profile to pre-stored operational state audio profiles, and if at least the first, the second and the third audio profiles correspond to an audio profile of a malfunction operational state and comprise an abnormal noise: locating a malfunction of the process machinery on the basis of locations of the first, the second and the third sound acquisition units, and intensity of abnormal noise in the first, the second and the third audio profiles. According to an embodiment, the location of the malfunction of the process machinery is determined on the basis of triangulation. According to an embodiment, the method further comprises: determining at least one image sensor at the location of the malfunction, and transmitting a trigger signal for triggering said at least one image sensor for capturing one or more images and to transmit the captured image data to said data processing device for further analysis.

Brief description of the drawings

In the following, various embodiments of the invention will be described in more detail with reference to the appended drawings, in which Fig. 1 shows an operational state monitoring system according to an example embodiment;

Fig. 2 shows a process machinery environment comprising an operational state monitoring system according to an example embodiment; and

Fig. 3 shows an acoustic monitoring method of an operational state monitoring system according to an example embodiment.

Detailed description

Previously a machine operator was able to walk around the process machinery environment and hear the operational state of machinery and/or whether there were mechanical problems with machinery or if something was wrong. He could perform this listening even while performing his usual duties. Nowadays the machine operator usually works in an at least partially soundproof process control room wherein he is not able to hear or he might not even have a skill to hear the state of the machinery even if it is possible in the control room. The machine operator usually visually monitors the state of the process machinery in monitoring devices, for example, in screen displays or he may get warnings, for example, alarming signals or other indications, when the operational state is changed from normal to abnormal.

The present invention enables the acoustic monitoring operational state of process machinery even if machine operators are in at least partially soundproof process control rooms. The present invention may also enable acoustic monitoring of such deviations which are not possible to be visually monitored by cameras. The operational state monitoring system using acoustic monitoring according to the invention may also be used for complementing visual monitoring performed by a machine vision system or a visual monitoring performed by a machine vision system may be used for complementing acoustic operational state monitoring system thus enabling more comprehensive and more accurate monitoring system for the machinery and processes. The present invention relates to an operational state monitoring system comprising a plurality of sound acquisition units comprising at least one acoustic sensor used for acoustically monitoring the operational state of process machinery. The sound acquisition unit continuously transmits the environmental sound of the process environment as acoustic data to a data processing device for analysis. For transmitting, the sound acquisition unit comprises a transceiver or a transmitter. In some cases, the sound acquisition unit according to an example embodiment may also comprise one or more camera sensors in addition to the at least one acoustic sensor. The term "operational state" refers in this context to any normal or abnormal state of process machinery detectable by one or more acoustic sensors of sound acquisition units from acoustic environment around the process machinery. Operational state of machinery may be determined, for example, on the basis of a sound of an engine, operating frequency of an engine, volume level, a sound provided by a web movement, abnormal noise etc. The operational state monitoring system according to an embodiment of the invention comprises, as already mentioned above, at least one, but possibly a plurality of sound acquisition units, each comprising one or more acoustic sensors. The acoustic sensor of the sound acquisition unit of acoustic systems according to embodiments of the invention may be, for example, a microphone recording acoustic environment i.e. audio signal around the process machinery and converting the recorded acoustic signals into electrical signals that are transmitted as acoustic data to a data processing device for analysis. Received acoustic data is analysed by the data processing device. The data processing device may determine an acoustic spectrum for acoustic data received from each acoustic sensor in order to form an audio profile for each recorded acoustic data. Audio profiles provide information that may comprise information also about volume levels in addition to frequencies. The data processing device may comprise at least one memory comprising pre-stored audio profiles for different operational states of the process machinery. Pre-stored audio profiles may comprise audio profiles for different operational states relating, for example, to normal and/or abnormal operational states of different machines of process machinery and/or for the whole machinery. Pre- stored audio profiles may comprise profiles, for example, for a normal state, a start-up state, a shut-down state, an idle state or a malfunction state etc. The data processing device may define which pre-stored audio profile correspond the audio profile of received acoustic data and on the basis of that correspondence an operational state of the machinery may be determined.

There is a plurality of sound acquisition units around, inside and/or in the vicinity the process machinery. Sound acquisition units are stationary and because locations of sound acquisition units are known, it is possible to locate, on the basis of location information of sound acquisition units, which part of the machinery the acoustic data comprising abnormal noise that indicates malfunction state originates from. If there are more than one sound acquisition units where sound profiles of transmitted acoustic data indicates malfunction state, the location of malfunction i.e. abnormal noise source is possible to determine, for example, by using triangulation.

If at least two sound acquisition units of a plurality of sound acquisition units arranged in the process machinery environment transmits data where the acoustic spectrum has the same or at least partly the same kind of deviation from a normal state acoustic spectrum, it is possible to locate the source that causes the deviation on the basis of intensity of the deviation of each acoustic spectrum. For example, if the data processing device detects that at least two sound acquisition units have corresponding changes in the acoustic spectrum, the probability of the location of the source causing the change can be calculated on the basis of the location of those at least two sound acquisition units having corresponding changes and intensity of the change. Probability of the location of the source causing the change being near the sound acquisition unit having the most intense change in the acoustic spectrum is higher. Whereas, one or more sound acquisition units having no or at least no corresponding changes in the acoustic spectrum indicates that the source causing the change may probably not be near those sound acquisition units.

However, it is also possible that there is only one sound acquisition unit around, inside and/or in the vicinity the process machinery that is arranged to record acoustic environment around a process machinery and convert the recorded acoustic signals into electrical signals that are transmitted as acoustic data to a data processing device for analysis. The data processing device may then compare this acoustic spectrum of acoustic data to acoustic data profiles in order to find if the process machinery is working normally or abnormally. If only one sound acquisition unit is used, it may be more difficult to locate the source of the malfunction, but it may still be possible. The acoustic operation state of the machinery monitoring system may further comprise a machine vision system comprising at least one image sensor used for detecting deviations in process machinery. The term "deviation" refers in this context to any deviation visually detectable from the machinery or process environment, for example, a malfunction or fault or a defect or deviation in mechanical structure of a machine, or a defect or deviation in some other part of a process. An image sensor is used for capturing images of a machine or other part of the process that is a monitoring target of that image sensor. The image sensor of the machine vision system may be, for example, a camera, for example, a c-mos or ccd camera, a matrix or line scan camera, a black and white or colour camera, a regular or smart camera, or any suitable camera. Targets arranged to be monitored may be illuminated for imaging. Image sensors may be arranged in connection with acoustic sensors and/or acoustic sensors and image sensors may be integrated into the same sound acquisition unit and/or image sensors may be arranged around the machinery as separate units. Also locations of image sensors are known. Acoustic sensors and/or sound acquisition units may be arranged, in or in connection with monitoring beams or monitoring rails and possible one or more lights or they may be fixed to machines or some other structures surrounding the process machinery. Acoustic sensors and/or sound acquisition units may also be integrated to image sensor(s). On the basis of the located noise source causing abnormal noise i.e. a malfunction or change in acoustic data detected by comparing an audio profile of acoustic data to pre-stored audio profiles, a data processing device may determine one or more image sensors for capturing one or more images. Those one or more image sensors are in such location(s) that the located noise source or the area comprising the located noise source could be imaged and thus visually monitored. It is possible that those one or more image sensors are not imaging at that moment or that those one or more image sensors are already imaging the area comprising the noise source by sequential images with low frequency, but in some cases the noise source of source area must be monitored more carefully if rarely captured sequential images of the noise source do not provide enough information. This means that more images of the noise source area may be needed in order to enable visual detection of possible machine/machinery malfunction(s) or process deviations that cannot be detected from images captured with low frequency. This more accurate monitoring may need a plurality of sequential images with high frequency. This monitoring can be performed, for example, as follows. If a data processing device detects that at least one audio profile formed for acoustic data includes abnormal noise indicating a malfunction exists, it may determine on the basis of the location of those sound acquisition units that transmit that acoustic data the location of the abnormal noise source. If there is more than one audio profile including abnormal noise, the data processing device may also use triangulation for determining the abnormal noise source. After the abnormal noise source is determined, the data processing device may indicate the location comprising the malfunction for a machine operator and/ or the type of the malfunction. Alternatively or in addition to indicating, the data processing device may trigger one or more image sensors, which are determined to be in such location(s) that the abnormal noise source or the area of the abnormal noise source could be imaged and visually monitored. At least one image sensor may be configured by a trigger signal to capture one or more images or at least one image burst. The term "image burst" means in this context a period of time during which an image sensor is in a high speed image shooting mode. Images captured during the image burst may also comprise higher resolution. The data processing device may, for example, by a trigger signal determine for the image sensor the image capturing frequency and/or a resolution of an image burst to be captured and/or the length of time of the image burst.

During image bursts one or more triggered image sensors are in a so called second image capturing frequency mode. The image capturing frequency of the second image capturing frequency mode is higher than the image capturing frequency of the first image capturing frequency mode, which is the normal imaging frequency outside the image bursts. During the first image capturing frequency mode, an image sensor may capture, for example, 50 - 100 images per second and during the second image capturing frequency mode the image sensor may capture, for example, 500 - 1000 images per second. Usually, an image burst takes a relatively short time, for example, 0.5 - 1 second, because it may produce so much image data for analysing. Furthermore, images of the image burst may have a higher resolution than images captured outside the image burst i.e. during the first image capturing frequency mode. An image sensor suitable for capturing and also transmitting image burst data comprising a plurality of images, possibly high resolution images, needs to have sufficient processing power. The image sensor capturing the image burst may store image burst data in its memory before it transmits the image burst data for analysing to a data processing device that is wirelessly or via a wired connection connected to the image sensor. This way, the data transmission rate may not form a limitation for image burst data transmission. Figure 1 shows an embodiment of the invention, in which an operational state monitoring system 100 is disclosed in conjunction with process machinery comprising machines 108a, 108b. The operational state monitoring system 100 comprises two sound acquisition units 101 , 102, a camera sensor 103 and a data processing device 106. The first sound acquisition unit 101 comprises an acoustic sensor 104, an image sensor 105 and means for transmitting and receiving data 1 10. The means for transmitting and receiving data 1 10 may be, for example, a transceiver, or a receiver and a transmitter arranged to transmit acoustic data and image data wirelessly or via a wired connection from the sensors 104, 105 to the data processing device

106 and to receive wirelessly or via a wired connection trigger signals indicating that image(s) or image burst(s) should be captured or when image(s) or image burst(s) should be captured. The acoustic sensor 104 is arranged to record environmental sound and the image sensor 105 is arranged to capture images. That recorded environmental sound is arranged to be transmitted as acoustic data to the data processing device 106 for analysis. Further, captured images are arranged to be transmitted as image data to the data processing device 106, for example, regularly or after an image, images or an image burst is requested.

The second sound acquisition unit 102 comprises an acoustic sensor

107 and a transmitter 1 1 1. The acoustic sensor 107 is arranged to record environmental sound and to transmit that acoustic data to the data processing device 106.

The acoustic sensors 104, 107 may be, for example, microphones. The image sensor 103 comprises also means for transmitting and receiving data 1 12. The means for transmitting and receiving data 1 12 may be, for example, a transceiver, or a receiver and a transmitter. The image sensor 103 is arranged to capture images and to transmit image data to the data processing device 106 and to receive trigger signals indicating that image(s) or image burst(s) should be captured or when image(s) or image burst(s) should be captured.

The data processing device 106 comprises at least one processor, at least one memory including computer program code for one or more program units and pre-stored audio profiles for different operational states of the process machinery and means for receiving acoustic data and image data wirelessly or via a wired connection from the sensors 104, 105, 107, 109. Pre-stored audio profiles may comprise audio profiles for different operational states of the process machinery relating, for example, to normal and/or abnormal operational states of different machines of the process machinery and/or for one of the machines 108a, 108b. Pre-stored audio profiles may comprise, for example, for a normal state, a start-up state, a shut-down state, an idle state or a malfunction state etc. Means for receiving acoustic data and image data may be, for example, a receiver or a transceiver, and means for transmitting trigger signals to an image sensor or sensors 101 , 103 wirelessly or via a wired connection may be, for example, a transmitter or a transceiver. There may be multiple processors e.g. a general purpose processor and a graphics processor and a DSP processor and/or multiple different memories e.g. volatile memory for storing data and programs at run-time and nonvolatile memory such as a hard disk for permanently storing data and programs. The data processing device 106 may be any computing device that is suitable for handling acoustic data and possibly also image data, such as a computer. The data processing device 106 is in electronic communication with sensors 102, 103, 104, 105 via signal lines. The data processing device 106 may also include a video controller and an audio controller for generating signals that can be produced for the user with computer accessories. The data processing device 106 may produce output to the user through output means. The video controller may be connected to a display. The display may be e.g. a flat panel display or a projector for producing a larger image. The audio controller may be connected to a sound source, such as loudspeakers or earphones. The data processing device 106 is configured to receive acoustic data from the acoustic sensors 102, 104 that record environmental acoustic signals and convert acoustic signals into electrical acoustic data and transmit the acoustic data to the data processing device 106. The data processing device 106 is arranged to determine an acoustic spectrum for received acoustic data transmitted by acoustic sensors 102, 104 and form an audio profile for acoustic sensors 102, 104. After forming of audio profiles, the data processing device 106 compares formed audio profiles to pre-stored audio profiles of different operational states. On the basis of the comparison, the data processing device 106 may determine an operational state of the machinery around each acoustic sensor. If it determines that one or more formed audio profiles are equivalent to an operational state that indicates malfunction of the machinery, the data processing device 106 may determine the location of the malfunction on the basis of the location of the sensor(s) that transmits that acoustic data and on the basis of intensity of malfunction noise. The location of the malfunction may also be called a location of an abnormal noise source.

After the abnormal noise source is located, the data processing device 106 may indicate the location comprising the malfunction for the machine operator. Alternatively or in addition to indicating, the data processing device 106 may determine at least one image sensor suitable for imaging or one that is currently imaging that location determined to comprise the abnormal noise source. Then the data processing device 106 may transmit a trigger signal to that at least one image sensor(s) requesting that an image burst or one or more images should be captured and the image data should be transmitted to the data processing device 106 for further analysis. In other words, the data processing device 106 requests one or more images or an image burst from at least one image sensor imaging the location comprising the malfunction. The data processing device 106 may also define for the image sensor(s) in the trigger signal the number of images to be captured or duration of the image burst or those may be predefined for the image sensor(s). After the image sensor(s) have captured the requested image(s) or the image burst, the image sensor(s) may start to capture images at the lower image capturing frequency or it/they may stop imaging until the next trigger signal is received.

For example, if an abnormal noise i.e. noise indicating a malfunction is detected in the acoustic data transmitted by the acoustic sensor 102 and the abnormal noise source is located in the area of a machine 108a, the data processing device 106 transmits a trigger signal to the camera sensor 103 requesting that an image burst should be captured and the image burst data should be transmitted to the data processing device 106 for further analysis. The data processing device 106 may further be arranged to notify the machine operator of the machines 108a, 108b about the malfunction state. Whereas, if the abnormal noise is detected in the acoustic data transmitted by the acoustic sensor 104 and the abnormal noise source is located in the area of a machine 108b, the data processing device 106 transmits a trigger signal to the camera sensor 105 requesting that an image burst should be captured and the image burst data should be transmitted to the data processing device 106 for further analysis.

Figure 2 shows a process machinery environment 200 according to an embodiment of the invention, in which an operational state monitoring system is disclosed in conjunction with a plurality of machines 201 -206. A plurality of sound acquisition units 210-220 comprising at least one acoustic sensor and a plurality of camera sensors 230-238 are placed around the process machinery environment 200. Imaging direction of camera sensors 230-238 are indicated by arrows. Each sound acquisition unit 210-220 and camera sensor 230-238 has a stationary location in the process machinery environment 299. Location of each sound acquisition unit 210-220 is different when compared to some other sound acquisition unit 210-220. Acoustic sensors of sound acquisition units 210-220 are arranged to record environmental sound of the process machinery environment 200 around them and to transmit the recorded environmental sound as acoustic data to a data processing device 250 that is wirelessly, or via a wired connection, connected to the sound acquisition units 210-220 and also to camera sensors 230-238. The data processing device 250 performs a spectre analysis for each acoustic data received from sound acquisition units 210-220 for forming audio profile for each audio signal. After that the data processing device 250 compares formed audio profiles to pre- stored operational state audio profiles. If one formed audio profile corresponds to a pre-stored operational state audio profile indicating malfunction, the data processing device 250 may determine the location of the malfunction transmitting the abnormal noise on the basis of the location of that sound acquisition unit that has transmitted the acoustic signal where the audio profile corresponds to the pre-stored operational state audio profile indicating malfunction.

If at least two formed audio profiles correspond to a pre-stored operational state audio profile indicating malfunction, the data processing device 250 may use the location of those sound acquisition units recorded, those audio profiles and intensity of abnormal noise to locate the malfunction transmitting the abnormal noise. When the abnormal noise location is determined, the data processing device 250 may indicate the machine operator about the malfunctioned operational state and/or its location of the malfunction and/or the data processing device 250 may transmit a trigger signal to one or more camera sensor(s), which are in such location(s) that the abnormal noise source or an area wherein the abnormal noise source could be imaged and visually monitored. By the trigger signal the data processing device 250 may request that one or more images or an image burst should be captured and transmitted to the data processing device 250. After receiving the image data, the data processing device 250 may analyse the received image data in order to find a cause of the malfunction/abnormal noise or in order to find more information about the malfunction causing the abnormal noise.

For example, if the data processing device 250 has located the abnormal noise source on the basis of abnormal noise intensity and locations of acoustic sensors of sound acquisition units 210-220 that recorded the abnormal noise in the area of the machine 205, the data processing device 250 may trigger camera sensors 236, 237 that are imaging the device 205 to perform an image burst and to transmit the image burst data to the data processing device 250 for analysis.

Figure 3 shows a monitoring method 300 of an operational state monitoring system according to an example embodiment. The operational state monitoring system comprises a first sound acquisition unit comprising a first acoustic sensor and a data processing device. In step 310, the operational state monitoring system is recording acoustic data of an environment of a process machinery by a first sound acquisition unit comprising a first acoustic sensor. In step 320, the operational state monitoring system is transmitting recorded acoustic data to a data processing device. In step 330 a first audio profile is formed for recorded acoustic data. In step 340, the first audio profile is compared to pre-stored operational state audio profiles stored in the memory of the data processing device. In step 350, if the first audio profile corresponds to an audio profile of a malfunction operational state, the malfunction of the process machinery is located on the basis of a location of the first sound acquisition unit. It should be noted that if at least three audio profiles formed for acoustic data recorded in an environment of a process machinery correspond to an audio profile of a malfunction operational state, the location of the malfunction can be calculated by using triangulation in addition to intensity of abnormal noise in those at least three formed audio profiles.

The various embodiments of the invention can be implemented with the help of computer program code that resides in a memory and causes an apparatus to carry out the invention. For example, the apparatus that is a computing device, for example, a data processing device may comprise circuitry and electronics for analysing, receiving and transmitting data, a computer program code in a memory, and a processor which, when running the computer program code, causes the apparatus to carry out the features of an embodiment. The processor, when running the computer program code, may carry out the steps of the following method: receiving environmental acoustic data of process machinery from an acoustic sensor of a sound acquisition unit, for example, a microphone, analysing the received acoustic data, forming audio profile for the received acoustic data, and if audio profiles of said received acoustic data correspond to pre-stored operational state audio profiles indicating malfunction or some other interesting noise, said data processing device is arranged to determine a location of the malfunction transmitting abnormal noise i.e. abnormal noise location and to transmit an indication to a machine operation and/or to transmit by a trigger signal to at least one image sensor in the malfunction area to capture an image(s) or an image burst that is imaging at a high image capturing frequency during a relatively short period of time and to transmit the image data for further analysis.

Sound acquisition units according to the invention comprise at least circuitry and electronics for recording environmental acoustic sound, converting acoustic signals into electrical signals and transmitting this acoustic data. Camera sensors according to the invention comprise circuitry and electronics for capturing images at low frequency and/or high frequency (image burst), and receiving triggering signals and transmitting image data. If sound acquisition units comprise a camera sensor, the sound acquisition units further comprise circuitry and electronics for capturing images at low frequency and/or high frequency (image burst), and receiving triggering signals and transmitting image data.

It is obvious that the present invention is not limited solely to the above- presented embodiments, but it can be modified within the scope of the appended claims.