Industrial Inspection Using Augmented Reality

Technical Field

[0001] This invention relates generally to an industrial inspection method and system and more particularly to use of visual imaging and visualization in industrial inspection.

Background Art

[0002] There are several new and advanced methods of industrial inspection that are being used in manufacturing and production facilities. Some of the latest methods of industrial inspection include:

[0003] 1 . Artificial Intelligence (Al) - Al and Machine Learning (ML) are being used to automate the inspection process and improve accuracy.

[0004] 2. 3D Scanning - This technology is used to create a digital representation of a physical object. 3D scanning can be used to compare the digital model with the original object to identify any deviations or defects.

[0005] 3. Infrared Thermography - This inspection technique uses thermal imaging to detect temperature changes in machinery. It can detect any issues such as misalignments or overheating that may result in system failure.

[0006] 4. High-Speed Cameras - High-speed cameras can capture thousands of frames per second and are used to inspect moving parts or high-speed processes.

[0007] 5. Optical Coherence Tomography (OCT) - This non-destructive inspection technique uses light waves to create a detailed image of the internal structure of an object. It is applicable for inspecting small parts or surfaces that are difficult to access. [0008] Overall, the increasing use of automation and Al in industrial inspection is leading to more efficient, accurate and cost-effective inspections for manufacturing and production facilities.

[0009] US pat. No.10908585 teaches an equipment inspection system and method, which are capable of preventing an operator from skipping inspection. The equipment inspection system includes: a host computer, at least one information medium attached to the industrial machine, storing information matching to an inspection part of the industrial machine, a reading device for reading the information medium; and at least one portable terminal.

[0010] US application NO.20100299179 A1 discloses an automated inspection tour includes a sequence and location of multiple inspection points marked with a unique identifier. User performs inspection tours by proceeding to each inspection point in the sequence indicated, and operating a portable data terminal (PDT) to read each unique identifier.

Summary of Invention

[0011 ] The summary of disclosed invention is intended to provide an overview of the subject matter of the invention, and is not intended to identify essential elements or key elements of the subject matter, nor is it intended to be used to determine the scope of the claimed implementations. The proper scope of the present disclosure may be ascertained from the claims set forth below in view of the detailed description below.

[0012] The present invention provides an innovative and effective inspection method and system for industrial environments. It comprises a mobile inspection device, multiple user interfaces, and a controller to monitor the inspection process. The invention also allows inspectors to view relevant notes during inspection, providing inspection records, specific instructions, and other necessary information. These features make it a valuable tool for investigators in various settings.

Technical Problem

[0013] Traditional inspection systems in industrial environments can be challenging and time-consuming, requiring specialized equipment and expertise. They often lack flexibility and customization, leading to errors and inefficiencies in notetaking and documentation.

Advantageous Effects of Invention

[0014] This invention offers an efficient, easy-to-use, and customizable inspection method and software for industrial environments. Its user interfaces are accessible to a wide range of users and recognize various inputs, including check boxes, hand gestures, finger touch, and voice recognition. Viewing relevant notes during inspection improves accuracy and efficiency.

Brief Description of Drawings

[0015] Fig.1 shows a block diagram of an inspection environment in an industrial environment.

[0016] Fig.2 shows an example of a graphical user interface screen that displays inspection results with detected swipe motion.

[0017] Fig. 3 shows an example of a graphical user interface screen that inputs inspection results by detecting hand and finger gestures

[0018] Fig.4 shows an example of a GUI screen that enters inspection results through checkboxes during inspection.

[0019] Fig. 5 is an illustration of notes displayed in a graphical user interface according to an illustrative embodiment.

[0020] Fig. 6 illustrates physical hardware system including one or more data processing systems including computers, server computers, tablets as well as mobile inspection system and inspection control system.

[0021 ] Fig. 7 illustrates the flow chart of the control part of the system.]

Description of Embodiments

[0022] The invention disclosed in present document provides an inspection method and system for industrial environments, including petrochemical or oil and gas plants and other industrial plants. This method involves a human inspector who uses a positioning system to identify any non-conformities or hazards in the environment. Examples of inconsistencies include leaks, smoke, breakage, spills, or other hazards.

[0023] The inspector uses a mobile inspection device, such as a mobile phone, head-mounted display, mixed reality smart glasses, or tablet, which has a touch screen and a positioning system. The device provides a user interface (Ul) through a mobile application. The Ul receives input from an input system, which may include a gesture recognition system, camera, touch input device, microphone, or other input devices. The invention's user interface module uses real-time metrics to gather information about inconsistencies and receive user input, controlled by a physical hardware system, such as gesture recognition, camera, touch input device, or microphone. The invention's innovative features and functions can be achieved independently in various embodiments or combined in other embodiments.

[0024] The invention includes the Global Positioning System (GPS), accelerometer, inertial navigation system, which is integrated into the mobile inspection device or connected to it. The augmented reality module works with the positioning system to display more information and notes related to the devices for the inspector.

[0025] The invention allows inspectors to select devices for inspection through either positioning or scanning barcodes or RFID tags. As the inspector moves in the environment, data stored in the inspection device is displayed in augmented reality and through positioning. The GUI displays a checklist of criteria that may indicate a standard or specification to which an item must conform. The graphical indicator may indicate the selected status of risk severity, the type of handling involved, or other types, represented by colors or symbols.

[0026] Control module: Control means control of inputs that are placed from a controller in a mobile inspection device or in a remote computer. The controller is configured to receive user input based on the location of the mobile inspection device. In addition, a controller can request additional information on the user's position about one or more conditions. The controller can leave a note and request new information from the user or create a task for another user. The controller may create notes for a group of locations or devices. [0027] Augmented reality module: A graphical indicator using augmented reality may be displayed on or near the faulty or damaged device (or location) so that a human operator interprets the note for the item.

[0028] Computer system is a physical hardware system and comprises one or more data processing systems. The data processing systems may be selected from at least one of computers, server computers, tablets, or any other suitable type of data processing system.

[0029] A camera system is a hardware system that includes one or more cameras that are used to generate images for display on a display system as well as an augmented reality display.

[0030] The graphic indicator may have different forms. For example, the graphical indicator may include at least one of an icon, an image, text, an animation, color, or other type of indicator that may be displayed in the graphical user interface to indicate a communication. The server is located in the computer system and receives the note from the controller.

[0031] Illustrative embodiments, as well as preferred manner of use, purposes and further features thereof, are best understood by reference to the following detailed description of an illustrative embodiment of the present disclosure when read in conjunction with the accompanying drawings:

[0032] Figure 1 shows a block diagram of an inspection environment in an industrial setting such as oil and gas plants. The inspection system is performed by human operators using a mobile inspection(20) device with a positioning system. The mobile inspection device includes a touch screen with a mobile application providing a user interface (Ul) and an input system consisting of a gesture recognition system, camera, touch input device, microphone, or other input devices.

[0033] Figure 2 depicts an example of a graphical interface that allows the human operator to perform a group of gestures recognized by the input system to generate user input to create a note. The group of movements may include finger movements that are recorded by the device as things such as a quick alarm or recording the operator's location. [0034] Figure 3 shows an example of the inspector passing devices through a test window or declaring a failure or safety problem based on the test results. As an example of inspection or checking function, a swipe to the left confirms the device, and a swipe to the right rejects the device.

[0035] Figure 4 shows an illustration of items displayed in a graphical user interface(20), which provides an experience of adding hazard type and severity to the human operator performing the inspection. The graphical interface displays a group of criteria to the inspector in the form of a checklist, indicating the standard or specification that the item must conform to. The Ul can also indicate other functions, such as risk severity, the type of handling involved, and the current status of the inspection, using different colors, symbols, or other graphical indicators. This process detects the position of an inspection device and displays a checklist with a note at a location where the note is automatically assigned to the display system. A user input is then received, wherein the user input includes a group of checkmarks assigned by a human operator relative to an item in a group of items in the field of view of the mobile inspection device.

[0036] Figure 5 provides an augmented reality experience for the human operator performing the inspection. Based on the notes, the type of related risks, the device report, etc. are displayed in relation to the item in the field of view in the graphical user interface in a display system for the mobile inspection device. The positioning system comprises the Global Positioning System (GPS), accelerometer, inertial navigation system that is integrated as part of the mobile inspection device or may be connected

[0037] Figure 6 shows a computer system(10). The server(30) receives the note from the controller. The computer system is a physical hardware system that includes one or more peripheral systems. The data processing systems may include computers, server computers, tablets, or any other suitable type of data processing system. The display system is a hardware system that includes one or more display devices. The camera system is a hardware system that also includes one or more cameras used to generate images for display on a display system as well as an augmented reality display. The user's position can be identified using the Global Positioning System (GPS), accelerometer, inertial navigation system, and other such technologies. The server stores the data in the storage system, which can take different forms such as a hard disk drive, a solid- state drive, a tape drive, an optical drive, or any other suitable storage device.

[0038] Figure 7 depicts a Control module, which is responsible for managing the inputs received from a controller in a mobile inspection device or a remote computer. The controller is designed to receive user input based on the location of the mobile inspection device and can request additional information on the user's position regarding one or more conditions. The controller can leave a note and request new information from the user or create a task for another user. It can also create notes for a group of locations or devices.

[0039] The main function of control part comprising the following steps:

[0040] If environment/ machine has defined, select the desired machine category, automatically determines machine location, if there needs new data/note related to machine in database add new data/note in database otherwise connect machine to data center, assign each note to related inspector as tack by controller, show machines and equipment that selected by controller and show information and new notes of controller and finish Inspector mission, i