SYSTEM

TECHNICAL AREA OF THE INVENTION:

The current invention concerns monolith systems for technical inspection of the parts, machinery, natural processes and events using the software structure of photogrammetry and integrating it with the available information in the cloud space or specific servers and also combining with the photogrammetry of standard parts and preparing the required content for performing technical inspection such as technical details, dimensions, used materials, final quality, performance of the part under inspection, production or application flaws, installation and operation method or replacement and repair and practical usage while executing these operations and also involves mechanisms for transmitting information to the experts and a method to be used in group decision in the inspection structures including the visual inspection and correct decision and also a system and structure for performing remote inspection operations with the help of inspectors and experts. PRIOR ART:

With the developments in the science and different industries it has been always an important issue to be able to control while the process is in place, after the production process and also to perform the inspections after production and for the purpose of maintenance and repair and periodical inspection to ensure the security and quality of the products usually the inspection process industries constitute an essential part to such industry and always experts have been trained and put into similar operations such as process control. Development of the industries and the need for experienced and experts and specialized people in inspection with knowledge and expertise has been increasing day by day and there is sometimes need to inspect a part or machine or a mechanism in the industry which may require experts to be dispatched from very long distances to the site to install and commission the machinery or inspect it. This matter has always wasted lots of time and human assets and material capitals. More recently, the industries have used some structures to transmit the information to the experts as in images which would be exploited and along with the knowledge of these experts to fix the problems. However, lack of accurate measurements in such images and also the fact that the inspection standards cannot be used in such scenarios makes such inspection processes unacceptable to international inspection communities. The photogrammetry process has turned into a robotic measurement structure based process with the utmost accuracy. In this regard one can mention the invention no. US6166811A patented in United States of America titled as“Robot-based gauging system for determining three-dimensional measurement data”. The current invention is a measurement system based on robot to determine the measured 3D data of an object. The robot based measurement system comprises of at least a mobile part and a robot controller for controlling the movement of the mobile part in respect to surface of an object. A sensor is attached to the surface of the object with no contact to the mobile part of the robot to record the visual data and reports the location of the reporter as the representative of location date of the remote sensor. A synchronous signal generator is connected to each one of the contact sensors and location report device. The measurement system of the robot including a visual system consistent for recovery of visual data from the contact sensor and location data from the location reporting device in which the visual system synchronizes the visual data with the location date to determine the 3D measurement date for the object. Also the invention no. US5251156A registered in USA titled: “Method and apparatus for non-contact measurement of object surfaces”. Images of the measured object taken from different angles or locations are saved and according to the coordinates of the intended features on the object surface are evaluated by the photogrammetry method and then the measured values of the location provided by the coordinates measurement device and the angular values provided by joint head are used to determine the coordinates of each of the points on the object surface. But such information is as in data and the experts have no control on the performed operations during inspection process. In order to fix the issue, the data extracted from the photogrammetry structure is transmitted by the common structures and via internet which has been a novel approach to improve the results of data transmission and reception platform integration system with photogrammetric structure. Also one could mention the invention“SYSTEM FOR INSPECTING OBJECTS USING AUGMENTED REALITY” no. US 9,633,481 in which the inspection is performed based on the dimensions and coordinates of the markers on the sample for comparison with the model created from the map which is defined using the virtual reality capabilities. In this method, the information recorded from a main sample is transmitted to the camera and the symptoms and natural specifications of the sample is also combined with the information recorded by the camera and finally, the monitor displays the previous pre-prepared images and the intended subject simultaneously. This process consists of a series of digital instruments and photogrammetric measurement systems along with an interactive network of servers and users which have been defined all together as a network. Also in the invention “UTILIZING AUGMENTED REALITY-BASED TECHNOLOGIES TO PROVIDE SITUATION-RELATED ASSISTANCE TO A SKILLED OPERATOR FROM REMOTE EXPERTS” with no. US20020067372A1 enables the experts or professional users to interactively get in touch while performing an introduced activity and also in the paper “Augmented and Virtual Reality for Inspection and Maintenance Processes in the Aviation Industry Procedia Manufacturing 19 (2018) 156-163” a limited application of augmented reality has been introduced in the technical inspection based on the pre-designed models.

Considering the continuous growth and introduction of systems which connect the actual and virtual reality and the augmented reality would be the most important ring of this loop which will supply the virtual components for completion of the real world information. The augmented reality is capable of creating views of the reality with higher levels of common information details which enables the user with the basic or fundamental information to have more accurate assessments about the quantitative and qualitative technical parameters. This access is created by exploiting the digital equipment such as smart phones, tablets, augmented reality cameras and also using the information available at the servers. This feature enables an accurate inspection in place by the user with general experience and using the least number of samples and repeat. THE PROBLEMS OF PRESENT INVENTIONS:

In the existing activities and inventions, the use and ability to use the virtual reality method in the basic and minor inspection and repair processes are mentioned. These methods mainly rely on proper design condition and accept during inspection (Ok) based on using the standard content. But this method is generally a dual comparative method in which the subject is compared only with a standard content. Also one of the issues which is not considered in the previous technologies was the possibility of direct inspection by the expert in real circumstances which requires the digital content to be prepared at the site from the sample under inspection and being compared with standards and subsequently consulted with the expert and deciding about the inspection result could be just a simple comparative survey but there was no way to compare the inspected materials and the previous inspection results and also real standard matters based on an original sample, also there has been no direct involvement of the experts in the inspection process so far. Moreover, due to the need of inspection for the heavy parts or parts inside the devices, it is usually decided based on the visual inspection, a determinant decision based on which the need for replacement or complete suspension of the devices indicates the probability of suspension and should the inspector make any mistakes, a severe loss will forced to the system. GOALS OF THE INVENTION:

This invention aims to provide a structure and system for online and/or offline communication between the expert and skilled operator during inspection of component and machineries. This communication is needed to exchange correct information of the subject under inspection and to preprocess the data and integrate the standard matter data and also previously examined matters and also the documents and information required in the augmented reality. This invention emphasizes on the problem of creating virtual content from the sample under inspection, especially in the inspections in which quick and accurate decisions are difficult, for example qualitative inspection and visual inspection which are more difficult than quantitative inspection. The current invention by creating a photogrammetric based system for inspection and providing accurate profile with the capability of making decision based on digital content, comparison with the witness samples at the qualitative inspections and also applying modifications or corrections in the sample based on such content.

DESCRIPTION OF THE INVENTION:

The current invention is an integrated structure in the augmented reality space using the photogrammetry and comparative analysis using the pre-made photogrammetry from the standard subject under inspection along with the available documentations and information and results of previous inspections. In this inventions, using the predefined servers or taking advantage of the possibility of saving information in the cloud space, the inspection results and photogrammetry of the standard sample of the component and machinery which may be used as the probable subject and in the future, and also by adding up the available documentations and information including the sketches, a correct format of the information structure is created in the augmented reality space which could be exploited by the inspector during the real inspection of the component, machine or process which could be an individual with moderate level of information in this field. This invention provides the user with an online and/or offline combination of all the information and images related to photogrammetry who can be any expert person from anywhere in the world. This way, the inspection process could also be evaluated by one or more experts and analyzed using the analytic softwares and lead to a stronger and more correct assessment.

In this invention, a photogrammetric augmented reality based inspection is performed with the process of creating the content based on the minimum necessary levels, as depicted in figure (1). This process includes creating the virtual content of inspection from the subject (101), a database which could be saved on a server, computer or on the cloud space (102), the information resulted from the photogrammetry is stored in itself and also it can provide them in an online manner to the integration structure. On the other hand, a predefined virtual content exists for the subject under inspection which could be mounted on the common databases as the virtual content of the inspection (103) and called during inspection. This information could be anywhere on the server space (102) or anywhere else. Creating the virtual content of inspection using 3D images with the help of photogrammetry robot for the sample with specific dimensions including height and diameter which depends on the circumstances of the photogrammetry is performed in the first stage. The 3D photogrammetric virtual content is completed by the technical information (104), dimensional data (105), operational conditions (106), results of failure probability analysis (107) and records of previous inspections.

The virtual content of component inspection in this stage contains all technical - operational details of the device enabling also to observe all the available record including the latest version of technical maps, explosion map, operational tolerances and adjustments map, combinational maps with other devices, results of pre-installation inspection, chemical analysis, metallography results, images of electron microscope (SEM - TEM), and other information required as per the subject that the definition of such information takes place based on need and inspection protocol. This virtual content of component inspection is collected as the component standard in the inspection servers or cloud space 108 and would be presented to the inspectors as per needed. The access tools to this virtual content of component inspection include smart phones, tablet, laptop, desktop computers or augmented reality cameras 109 which works under augmented reality software.

The inspection process takes place by adaptation of virtual content of component inspection with the component (figure 2) which is in the form of examination of interactive effect of standard content and the content accessible during the component inspection 201. This real content of component is available using the camera communication tool 202 or creating photogrammetric content with photogrammetry robot located on inspection site 203 if necessary. The comparison results of standard content with the real content is presentable in the form of figurative comparison in the display tools of communication instruments 204.

The inspection diagram based on the photogrammetric augmented reality is depicted in figure 3. The inspection process of component virtual content starts based on the international, regional, national, industrial, corporate or requirements of component standard or process 301. The decision making diagram based on stages of sharing 302, results of component virtual content inspection 303. Results of comparison of photogrammetric profile of standard device with the actual device under inspection 304, the commands and procedures, and protocols towards final inspection result 306 and reportable decision.

In order to make sure of the quality of component virtual content inspection, communication with the expert 305 is also added to the augmented reality communication tool. This communication includes controlling the documents of the experts in the similar matters, relevant feedbacks, consult with an expert or a team of experts and the results of experts group, or action commands. The basis of creating the device virtual content inspection is built on open source standard bases or by the design of specialized AR software and communication with the tool Such inspection process enables appropriate decision in the first and most important stage on inspection of part within heavy and probably expensive device which requires more accurate inspection or replacement or repair. All stages of such inspection could be recorded and saved as in virtual files.

The innovative feature of the invention: The main parts improved or problems fixed by this invention include:

- In the current inspection processes, the most important stage of component inspection in the complex sets is performed based on controlling some simple and fundamental inspection items and just relying on the inspector’s decision in the actual location of the sets, but in this invention it is also possible to examine and take into account the comments of other inspectors and experts as well.

- It is also possible to make documents during photogrammetry by integrating inspection protocols and information, standards and inspection results.

- In this invention it is also possible for other inspectors to contribute from far distances and for them to be responsible for their comments and decisions.

- Also in the previous procedures there is more error probability due to ignorance of the inspector such as device and complex failure risks and even extra caution which brings about the limits caused by long term suspension of the work or unnecessary repair or replacement all of which are reduced due to the fast decision making and remote supervision of the experts.

- Considering deployment of portable photogrammetric structures in the vehicles with the specifications according to the environmental conditions or helicopters, drones, quadcopters and integration of their photogrammetric content with other information and maps and the photogrammetries performed in the standard circumstances, the photogrammetry process could be accelerated.

- Photogrammetric augmented virtual reality based inspection provides decision making capability in the initial component inspection in a complex and simultaneous comparison with the best rejected component or the worst approved one (witness sample).

- The witness sample might have been made based on a predefined standard as in photogrammetry or defined and presented based on operational circumstances including the weather, dust, environmental pollution and other issues.

- Photogrammetric virtual reality based inspection enables simultaneous operational, dimensional, assembling, morphology, corrosion, decay, and procedural items all together.

- This invention also provides the means to introduce component inspection items after identification by the inspector and reporting the result in the form of augmented reality.

- Photogrammetric virtual reality based inspection determines the order of inspection items based on the importance in the operational circumstances which could be updated via server or shared cloud of information.

- Using this invention, it is also possible for the expert or experts team to be uninterruptedly and effectively present beside the actual component inspector enabling feedback and action plan. BRIEF DESCRIPTION OF THE DRAWINGS:

Figure 1 depicts the performance of the system which includes:

101- Creation of component inspection virtual content (object)

102- database/ server of inspection virtual content 103- 3D standard photogrammetric virtual content and device standard in the inspection servers of cloud space of information

104- Technical information

105- Dimensional information

106- Information of operational circumstances 107- Records of previous inspections, results of probable failure analyses.

108- Communication of inspection virtual content and creation of inspection virtual content

109- Access tools to component inspection virtual content using smart phones, tablets, laptop, desktop computers or augmented reality cameras Figure 2 demonstrates the interactive effect of camera information and photogrammetry robot to create the content which includes: 201- The standard content data and the content accessible during component inspection

202- Real content data of the component using the communication tool camera

203- Photogrammetry content data using the photogrammetry robot located at the inspection site

204- Providing the standard content at the same time with the real content in form of figurative comparison in display tool of communication equipment

Figure 3 illustrates the decision making diagram based on the information and the accesses by the current invention which includes: 301- Inspection instruction

302- Consult with an expert or team of experts and results of expert group

303- Decision making based on the inspection

304- Virtual content of the component based on the standard definitions

305- Contact with experts, controlling the documents of experts, relevant feedbacks, action commands

306- Final result of inspection