TECHNICAL FIELD

[0001] Described embodiments generally relate to auditing precious metal storage locations, such as a gold vault. In particular, embodiments relate to auditing precious metal storage locations using networked computer systems and camera systems.

BACKGROUND

[0002] Precious metal storage locations such as vaults may have obligations and/or incentives to have audits conducted to assess details pertaining to stored precious metal at the storage location. The audits may verify claimed holdings at the precious metal storage location. In order for there to be confidence in the audit outcomes, adequate audit procedures should be adopted to conduct the audit.

[0003] It is desirable to provide improved systems and methods for auditing of precious metal storage that allow a high degree of confidence in the audit outcomes, or to at least provide a useful alternative to prior systems and methods.

[0004] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each claim of this application.

[0005] Throughout this specification the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.

SUMMARY

[0006] Some embodiments relate to a system for auditing precious metal inventory across multiple storage locations, the system including: a server system including at least one processor and memory, the server system being in communication over a network with multiple client computing devices associated with respective storage location, each storage location being configured to store a part of the precious metal inventory; wherein the memory stores executable program code that, when executed by the at least one processor, causes the at least one processor to: maintain a precious metal inventory record for each of the multiple storage locations; receive audit data from each of the multiple client computing devices in relation to an audit process conducted at the respective storage location, the audit data including for each client computing device: at least one image purported to be of precious metal stored at the respective storage location, and quantitative and qualitative data describing the part of the precious metal inventory; for each client computing device: process the at least one image to detect image irregularities, and optionally transmit a supplemental information request to the client computing device for a further at least one image purported to be of the precious metal stored at the respective storage location when image irregularities are detected in the at least one image.

[0007] The audit data may further include: personnel identification information of a person recorded at the client computing device as responsible for conducting the audit process at the storage location. The audit data may further include: location information defining a geographic location of the respective client computing device.

[0008] Processing the at least one image to detect image irregularities may include determining whether the at least one image is substantially the same as at least one image previously received in audit data in relation to a previous audit process for the respective storage location.

[0009] Processing the at least one image to detect image irregularities may include comparing time and location metadata of the at least one image to expected or stored time and location information for the audit process at the respective storage location. The at least one image may include at least one still image and at least one video image.

[0010] The memory may store executable program code that, when executed by the at least one processor, causes the at least one processor to: process the at least one image to determine an audited amount of precious metal; transmit the supplemental information request when the audited amount does not match an expected amount of the part of the precious metal inventory based on the precious metal inventory record for the respective storage location.

[0011] Processing the at least one image to determine an audited amount of precious metal may include using an object detection framework to determine precious metal objects in the at least one image and to determine a number, size and kind of the precious metal objects in the at least one image. Using the object detection framework may include using one of: a deep neural network, a scale-invariant feature transform, or a Viola-Jones object detection framework.

[0012] The memory may further store executable program code that, when executed by the at least one processor, causes the at least one processor to: communicate with each client computing device to initiate and record the audit process over a non-zero duration of time up to 30 minutes. For example, the duration may be between about 1 minute and about 30 minutes, or a different duration, such as about 5 minutes to about 30 or 40 minutes or about 5 minutes to less than about 1 hour.

[0013] The memory may further store executable program code that, when executed by the at least one processor, causes the at least one processor to: transmit an audit initiation prompt to a selected one or more of the client computing devices to begin the audit process.

[0014] The audit initiation prompt may be transmitted in response to an audit trigger event detected by the server system in relation to a respective one of the client computing devices or in relation to the storage location associated with the respective client computing device.

[0015] The memory may further store executable program code that, when executed by the at least one processor, causes the at least one processor to: receive further audit data in response to the supplemental information request, the further audit data including the further at least one image; process the further at least one image to detect image irregularities, and transmit a further supplemental information request to the client computing device for an additional at least one image purported to be of the precious metal stored at the respective storage location when image irregularities are detected in the further at least one image.

[0016] The system may include the client computing devices, wherein each of the client computing devices includes at least one client processor and client memory storing executable program code that, when executed by the at least one client processor, causes the at least one client processor to implement a client application to conduct the audit process. [0017] The client application may be configured to capture the audit data and transmit the audit data to the server system.

[0018] Some embodiments relate to a system for auditing precious metal inventory, the system including: a client computing device located at a storage location, the storage location including a secure storage structure to store precious metal, the client computing device including at least one processor and memory accessible to the at least one processor and being configured to communicate with a remote server over a network; wherein the client computing device includes or is in communication with at least one camera positioned or positionable to capture images of the secure storage structure, and wherein the client computing device is configured to execute program code stored in the memory to implement an audit application to: capture personnel identification information of personnel performing an audit process using the audit application; receive qualitative and quantitative data describing the precious metal stored in the secure storage structure at a time of the audit process; determine a geographic location of the client computing device during the audit process; determine an expected amount of the precious metal present in the secure storage structure; receive from the at least one camera at least one image purported to be of the precious metal within the secure storage structure during the audit process; transmit to the remote server audit data for the audit process, the audit data including the personnel identification information, location information defining the geographic location, the at least one image and the qualitative and quantitative data.

[0019] The at least one camera may include a first camera that is integrated with the client computing device. The at least one camera may additionally include a second camera in a fixed location at the storage location that is positioned to capture images of an area that at least partially includes the secure storage structure.

[0020] The audit application may be configured to restrict predetermined software functions of the client computing device until the audit application receives a message from the remote server that the audit process has been determined by the remote server to be complete.

[0021] The audit application may be configured to display a prompt to capture and upload a further at least one image purported to be of the precious metal in response to receiving a request from the remote server for the further at least one image.

[0022] The audit data may include metadata indicating a time and location of each at least one image transmitted to the remote server as part of the audit data.

[0023] The audit application may be configured to generate a prompt to conduct the audit process at least once per business day or other time period. Other time periods may include: every second business day, every 12 hours, every 6 hours, or every 4 hours, for example. The audit application may be configured to generate a prompt, for example at the client computing device or an associated computing device, to conduct an additional audit process in response to a trigger event. The trigger event may include at least one of: receipt of an audit prompt message from the remote server; a change in personnel within a predetermined time period; or expiry of a predetermined time period since a previous audit. [0024] The client computing device may be configured to authenticate an identity of personnel conducting the audit process using biometric data received by the client computing device prior to or during initiation of the audit process.

[0025] The biometric data may include face images of the personnel and wherein the client computing device is configured to perform a facial recognition process to at least one of: determine whether the personnel is authorised to conduct the audit process; or uniquely identify the personnel.

[0026] Some embodiments relate to a method for auditing precious metal inventory across multiple storage locations, the system including: providing a server system including at least one processor and memory, the server system being in communication over a network with multiple client computing devices associated with respective storage location, each storage location being configured to store a part of the precious metal inventory; maintaining a precious metal inventory record for each of the multiple storage locations; receiving audit data from each of the multiple client computing devices in relation to an audit process conducted at the respective storage location, the audit data including for each client computing device: at least one image purported to be of precious metal stored at the respective storage location, and quantitative and qualitative data describing the part of the precious metal inventory; and for each client computing device, processing the at least one image to detect image irregularities. [0027] The method may further include, for each client computing device, transmitting a supplemental information request to the client computing device for a further at least one image purported to be of the precious metal stored at the respective storage location when image irregularities are detected in the at least one image.

[0028] Some embodiments relate to a method for auditing precious metal inventory using a client computing device located at a storage location, the storage location including a secure storage structure to store precious metal, the client computing device including at least one processor and memory accessible to the at least one processor and being configured to communicate with a remote server over a network, wherein the client computing device includes or is in communication with at least one camera positioned or positionable to capture images of the secure storage structure, the method including: capturing personnel identification information of personnel performing an audit process using the audit application; receiving qualitative and quantitative data describing the precious metal stored in the secure storage structure at a time of the audit process; determining a geographic location of the client computing device during the audit process; determining an expected amount of the precious metal present in the secure storage structure; receiving from the at least one camera at least one image purported to be of the precious metal within the secure storage structure during the audit process; transmitting to the remote server audit data for the audit process, the audit data including the personnel identification information, location information defining the geographic location, the at least one image and the qualitative and quantitative data. [0029] Some embodiments relate to computer readable storage storing executable program instructions that, when executed by at least one computing processor of a computing device, cause the computing device to perform any of the methods described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Embodiments are described in further detail below, by way of example, with reference to the accompanying drawings.

[0031] Figure 1 is a block diagram of a system for auditing precious metal storage locations according to some embodiments;

[0032] Figure 2 is a schematic diagram of a vault location during an audit process according to some embodiments;

[0033] Figure 3 is an example screen display of a client vault computing device during the audit process according to some embodiments;

[0034] Figure 4 is a block diagram of two precious metal bullions which do not match listed precious metal holdings;

[0035] Figure 5 is a block diagram of two precious metal bullions which do not match listed precious metal holdings;

[0036] Figure 6 is a block diagram of two precious metal bullions which do not match the listed precious metal holdings;

[0037] Figure 7 is a block diagram of two precious metal bullions which match the listed precious metal holdings;

[0038] Figure 8 is a flowchart of a method of server system operation for managing audits of precious metal storage locations; and [0039] Figure 9 is a flowchart of an error detection process of the method of server system operation for managing audits of precious metal storage locations.

DETAILED DESCRIPTION

[0040] Described embodiments generally relate to auditing precious metal storage locations, such as a gold vault. In particular, embodiments relate to auditing precious metal inventory across multiple storage locations using networked computer systems and camera systems.

[0041] Embodiments described herein facilitate regular and accurate auditing of precious metal inventories in specific precious metal storage locations, as well as across multiple such location or a network of such locations. This can be useful for accurate inventory tracking of precious metals stored in various separate physical locations, particularly where the precious metals are the subject of a transferrable digital certificate for that precious metal, for example. Embodiments described herein allow accurate information capture during local audit processes at the storage locations and include measures to automatically detect indications of fraud. For example, automatic image processing of images captured at the storage locations can identify a possible fraudulent re-use of imagery of the precious metal. In another example, location determination of the computing device that is used to conduct the audit can identify possible fraud if the computing device is not in or close to an expected physical location of the stored precious metal.

[0042] Figure 1 is a block diagram of a system 100 for auditing precious metal storage locations. System 100 includes a network 150, a server 120, a data store 130 accessible to server 120 and at least one or multiple client computing devices. Network 150 may comprise direct connections between hosts, enterprise networks, Internet, local area networks or any other networks both wired or wireless.

[0043] In some embodiments, system 100 may also comprise other systems, such as a reserve management server 180 and a transaction management server 190. The other systems, such as reserve management server 180 and transaction management server 190, may be communicatively coupled via network 150 to server 120 and to each other. Reserve management server 180, transaction management server 190 and server 120 may share data with each other through transmitting and receiving messages, or periodic reporting. Reserve management server 180, transaction management server 190, and server 120 may each have their own APIs to share data.

[0044] Server 120 may be a single server, a server system, a cloud-based server or server system, or other physical or virtual computing device or system providing centralised server functions and resources to client computing devices. Such client computing devices may include audit request devices 110 or vault client computing devices 147, for example.

[0045] Server 120 comprises at least one computer processor 121 and memory 122 accessible to the at least one computer processor 121. The at least one computer processor 121 may be or include multiple physical and/or virtual processing units that together function as a processor, and which is described as server processor 121 for convenient reference herein.

[0046] Server 120 further comprises a communications module 128. Communications module 128 may facilitate communication between server 120 and other devices via network 150, and/or via a wired communication protocol, such as USB or Ethernet, and/or via a wireless communication protocol, such as Wi-Fi or Bluetooth, for example.

[0047] Memory 122 may comprise one or more volatile or non-volatile memory types, such as RAM, ROM, EEPROM, or flash, for example. Memory 122 may be configured to store sets, groups or other forms of structured or organised executable program code to implement notifications module 123, inventory module 124, vault session module 125, image analysis module 126, data analysis module 127 and transaction tracking module 129. Server processor 121 may be configured to access memory 122 to read and execute notifications module 123, inventory module 124, vault session module 125, image analysis module 126, data analysis module 127, and transaction tracking module 129 stored in memory 122; to read and load stored data; and to perform processes specified in notifications module 123, inventory module 124, vault session module 125, image analysis module 126, data analysis module 127 and transaction tracking module 129, to process stored data.

[0048] Data store 130 may comprise one or more of flash memory, external hard drives, cloud storage or any other data storage medium external to server 120. Data store 130 may be configured to store audit request records 131 and vault details data 137. Audit request records 131 may comprise inventory data 132, video recording data 133, image data 134, temporal data 135, and location data 136, for example. Data store 130 may also comprise a communications module 138. Communications module 138 may facilitate communication between data store 130 and other devices, such as server 120, via a network, via a wired communication protocol, such as USB or Ethernet, or via a wireless communication protocol, such as Wi-Fi or Bluetooth, for example.

[0049] System 100 further comprises vault location 140 and audit request device 110. Audit request device 110 may be configured to use communications module 116 to communicate via network 150 to external or remote devices, such as server 120.

[0050] Vault location 140 may be a storage location for precious metal. Vault location 140 may comprise a facility or facilities for securely storing precious metals in a vault housing. Vault location 140 may comprise a client vault computing device 147, an entrance sensor system 165, one or more secure storage structures 170 and a video camera system 160 located therein. Vault location 140 may also comprise a vault computer system 175. Vault computer system 175 may be at the same building location as the secure storage structure 170 to interface with the video camera system 160, the client vault computing device 147 and/or the entrance sensor system 165. There may be multiple (and possibly many) vault locations 140 within system 100. Each vault location 140 may comprise a client vault computing device 147, one or more secure storage structures 170 and video camera system 160 located therein. The one or more secure storage structures may include existing structures for precious metal storage, such as a safe, a small vault or a large (e.g. walk-in) vault, for example. The secure storage structures 170 may be different in number, physical structure or form at each different vault location 140.

[0051] Vault computing device 147 may be a laptop, tablet computing device, desktop computer or other form of client computing device. Vault computing device 147 comprises a processor 141 and memory 142 that is accessible to processor 141. Processor 141 may comprise one or more microprocessors, central processing units (CPUs), application specific instruction set processors (ASIPs), or other processors capable of reading and executing instruction code.

[0052] Memory 142 may comprise one or more volatile or non-volatile memory types, such as RAM, ROM, EEPROM, or flash, for example. Memory 142 may be configured to store executable program code to implement a client vault audit application 143 and data 144. Processor 141 may be configured to access memory 142 to read and execute client vault audit application 143 stored in memory 142, to read and load stored data 144, and to perform processes specified in the code of client vault application 143 to process stored data 144.

[0053] Vault computing device 147 may further comprise vault input and output 145, vault location module 146, vault camera 148, and vault communications module 149. Processor 141 may be configured to communicate with vault input and output 145, camera 148, and vault communications module 149.

[0054] Vault input and output 145 may comprise one or more of an output display screen, an input mouse, an input keyboard or other I/O devices.

[0055] Vault location module 146 may comprise sensors, circuitry and/or code in memory 142 to communicate with one or more positioning information devices. Positioning information devices are devices external to vault device 147 that may communicate with the vault location module 146 to send positioning information of the vault device 147 to the vault location module 146. Positioning information devices may comprise global navigation satellite system (GNSS) satellites, mobile network elements or servers providing IP geolocation services, for example.

[0056] Vault camera 148 may comprise hardware and software for capturing still images and/or video images. In some embodiments, vault camera 148 may be separate to vault computing device 147 and images may be uploaded from the vault camera 148 to the vault computing device 147 via a wired or wireless connection.

[0057] Vault communications module 149 may facilitate communication via a wired communication protocol, such as USB or Ethernet, or via a wireless communication protocol, such as Wi-Fi, Bluetooth or NFC, for example.

[0058] The one or more secure storage structures 170 may be located within vault location 140. One or more secure storage structures 170 for convenient reference may be referred to as secure storage structure 170 herein. The secure storage structure 170 may comprise one or more vault safes containing precious metal 210. In some embodiments, the secure storage structure 170 may include one or more secure rooms containing precious metal 210.

[0059] Video camera system 160 may comprise equipment for capturing, storing and transmitting images and videos. For example in some embodiments, as seen in Figure 2, video camera system 160 comprises one or more overhead cameras 422. In some embodiments, video camera system may communicate and transfer data via a wired communication protocol, such as USB or Ethernet, or via a wireless communication protocol, such as Wi-Fi, Bluetooth or NFC, for example.

[0060] Entrance sensor system 165 may comprise equipment, sensors and hardware for detecting room entrance movement for an entrance to vault location 140, and transmitting captured sensor data. Entrance sensor system 165 may also comprise memory to store captured sensor data. Entrance sensor system 165 may include one or more magnetic contact sensors, pneumatic sensors, push button sensors, break beam sensors, and/or PIR sensors, for example. [0061] Vault computer system 175 may be a laptop, tablet computing device, desktop computer or other form of client computing device. Vault computer system 175 may be communicatively coupled to receive data or trigger capturing of data from at least one of vault computing device 147, entrance sensor system 165, and video camera system 160. Vault computer system 175 may be communicatively coupled to send and receive data via network 150 to server 120. Vault computer system 175 may comprise some or all of the functionality of vault device 147 in carrying out method 800 of Figure 8, and/or function as an intermediary of communications between vault device 147 and server 120.

[0062] System 100 may include a client audit request device 110. Audit request device 110 may be a laptop, tablet computing device, smartphone, desktop computer or other form of client computing device. Audit request device 110 comprises a processor 111 and memory 112 that is accessible to processor 111. Processor 111 may comprise one or more microprocessors, central processing units (CPUs), application specific instruction set processors (ASIPs), or other processors capable of reading and executing instruction code.

[0063] Memory 112 may comprise one or more volatile or non-volatile memory types, such as RAM, ROM, EEPROM, or flash, for example. Memory 112 may be configured to store a client requester application 113 and data 114. Processor 111 may be configured to access memory 112 to read and execute the client requester application 113 stored in memory 112, to read and load stored data 114, and to perform processes specified in the client requester application 113 to process stored data 114.

[0064] Audit request device 110 may further comprise requester input and output 115 and communications module 116. Processor 111 may be configured to communicate with requester input and output 115 and communications module 116. There may be multiple audit request devices 110 in system 100.

[0065] Communications module 116 may facilitate communication via network 150 to external or remote devices such as server 120, via a wired communication protocol, such as USB or Ethernet, or via a wireless communication protocol, such as Wi-Fi or Bluetooth, for example.

[0066] Requester input and output 115 may comprise one or more of an output display screen, an input mouse, an input keyboard or other I/O devices.

[0067] The reserve management server 180 may implement computing functionality to make sale or purchasing decisions and effect sale or purchase transactions of precious metal based on at least a reserve of precious metal at vault storage locations 140. Data regarding precious metal inventory in the reserve management server 180 may be accessible by or provided to server 120 through one or more Application Programming Interface (API) calls to the reserve management server 180.

[0068] The reserve management server 180 may implement computing functionality to make sale or purchasing decisions and effect sale or purchase transactions of precious metal based on at least a reserve of precious metal at vault locations 140.

[0069] The transaction management server 190 may implement computing functionality for performing precious metal digital asset sale or purchase transactions on behalf of retail customers. The precious metal digital assets that are the subject of the transactions each correspond to a fixed quantum of precious metal stored at vault locations 140.

[0070] Data from server 120 and data store 130 regarding precious metal inventory may be made accessible to the reserve management server 180 and transaction management server 190 through one or more API calls to the server 12O.Figure 2 is a side view of the vault location 140 during an audit process method according to some embodiments.

[0071] Vault location 140 may comprise an auditor 205, secure storage structure 170, a client vault device 147, video camera system 160 and precious metal 210 as shown in Figure 2. [0072] Auditor 205 may be a person, individual, or system for conducting audit operations within the vault location 140. The auditor 205 may be an employee of a business conducted at the vault location, for example. The auditor 205 may have a unique personnel identification number. That unique personnel identification number may be associated in vault details data 137 with a unique identifier of the vault location. In some embodiments, conducting audit operations comprises recognising precious metal within the vault location 140, completing a checklist to validate precious metal holdings in the vault location 140, and positioning a camera to take images of precious metal at the vault location 140.

[0073] Vault device 147 may be or include a tablet device as shown in Figure 2. Vault device 147 may include vault camera 148.

[0074] Auditor 205 may position vault device 147 to capture one or more (and possibly many) images using vault camera 148 of precious metal 210. Precious metal 210 may comprise gold, silver or platinum bullion ingots, for example, as shown in Figure 2. Images taken of precious metal 210 may include overhead views, views of different sides of the precious metal, views with multiple pieces of precious metal 210, and views with single pieces of precious metal in an image.

[0075] Secure storage structure may be a safe as shown in Figure 2. Images taken of precious metal 210 may be taken when precious metal 210 is located on or in secure storage structure 170, or on an extendible drawer extending from within the secure storage structure 170, as shown in Figure 2, for example.

[0076] In some other embodiments, when vault camera 148 is separate from vault device 147, vault device 147 does not need to be present within the vault location 140 when an audit is undertaken, but rather nearby to vault location 140. Instead auditor 205 may only need to position vault camera 148 to take images of precious metal 210. Then subsequently auditor 205 may upload the images of the precious metal 210 to the nearby vault device 147 using a wired or wireless connection from the vault device camera 148. The upload process may be conducted by a display on the vault device 147, the display being generated from the execution of client vault audit application 143 of vault device 147.

[0077] Video camera system 160 may comprise a video camera positioned with an overview of the room and the audit operations. Video camera system 160 may be continuously recording video footage; or video camera system 160 may be activated during the presence of auditor 205 or during audit operations. Video camera system 160 may be connected to the vault input output 145 of vault device 147, and the camera system 160 may send still or video images to vault device 147 at times including when a precious metal audit is conducted at vault location 140. In some embodiments, the software operating video camera system 160 is stored in memory 142, and operated and controlled by the processor 141 in vault device 147.

[0078] Figure 3 is an display of the client vault computing device 147 generated by the client vault audit application 143 during the audit process shown in Figure 2 according to some embodiments.

[0079] As described in Figure 2, the vault computing device 147 may be a tablet computing device or other computing device with a touch- sensitive display and input interface and having camera 149 integrated therein.

[0080] The vault computing device 147 may have a display 310 which may be visible on a touchscreen user interface and display of the vault computing device 147. Display 310 may also comprise interactive images loaded by processor 141 running client vault audit application 143. In some embodiments, the interactive images may show the vault camera 148 view and allow the auditor to trigger capture of images by providing an image capture input to the user interface.

[0081] The interactive displays of vault computing device 147 may show an interactive inventory list as shown in Figure 3. The interactive inventory list may comprise the listed precious metal holdings 325 for the vault location 140 shown in Figure 2. In some embodiments, some or all of the data displayed in the listed precious metal holdings 325 may be initially blank and have input functionality to allow the auditor 205 to input information pertaining to the precious metals 210. In some embodiments, the input functionality may comprise an extendable list. In an example embodiment, only the item numbers are displayed in the listed precious metal holdings 325. In some embodiments, this may be because only a portion of inventory list data is sent at step 815.

[0082] In the example display of Figure 3, the auditor 205 has completed inputting data pertaining to two inventory items of the precious metals 210. In some embodiments, there may be any integer number of inventory items of precious metals that may be inputted by auditor 205. In some embodiments, the listed precious metal holdings 325 are displayed as quantitative and/or qualitative data describing a list of the expected precious metals contained within the vault location 140. The listed precious metal holdings 325 may be displayed in the form of a table, wherein quantitative and/or qualitative data categories used to distinguish pieces of precious metal from within a group of precious metals may be displayed as column headers, and the different precious metals expected to be held are listed within each row. As shown in Figure 3, the column headers may include item number, item identifier (ID), metal, purity, weight and form. In some embodiments, there may be fewer or more column headers which may necessitate data entry for different, the same, or similar quantitative or qualitative descriptors for precious metal.

[0083] “Item number” is an integer number for listing and counting the precious metals 210 at a vault location 140. “ID” refers to any inscribed identification on precious metal piece. “Metal” refers to the precious metal element contained in the precious metal piece. “Purity” refers to the quantity of the precious metal element within the piece of precious metal as a percentage of the content of the entire precious metal piece. “Weight” refers to the mass of the precious metal piece. “Form” refers to the form or shape of the precious metal piece, such as coin or ingot.

[0084] The first entry at row 1 of list 325 refers to one of the pieces of the precious metal 210 held within the vault location 140. The first entry is listed as item number “1”. Item “1” has an id of “adz3”, comprises the metal “gold”, has a mass of 50 grams, a purity of 50%, and is in the form of an ingot, for example. Meanwhile the second entry refers to a different piece of precious metal to item “1”. The second entry is listed as item “2”. Item “2” has an id of “adz4”, comprises the metal “gold”, has a mass of 100 grams, a purity of 50%, and is in the form of an ingot, for example.

[0085] Thus the similarities between item “1” and “2” are that they are both made from gold, have a purity of 50% and are in the form of an ingot. The differences between item “1” and “2” are their inscribed identifications and their mass.

[0086] In some embodiments, some or all of the data displayed in the listed precious metal holdings 325 may be blank and the client vault audit application 147 provides input functionality to allow the auditor 205 to input information pertaining to the precious metals 210. In an example alternative embodiment, only the item numbers are displayed in the listed precious metal holdings 325. In some embodiments, this may be because only a portion of inventory list data is sent at step 815.

[0087] The interactive inventory list may also comprise an auditor check and attach section 315. The auditor check and attach section 315 may comprise interactive entries under the column headings. The interactive column headings indicate additional categories of the listed precious metal holdings 325 displayed on the display 310 of vault device 147. As shown in Figure 3, the auditor check and attach section 315 may include availability element 340 and attach element 345.

[0088] The availability element 340 may include a selectable option on display 310, caused by the execution of a script from client vault audit application 143. The availability element 340 may include an interactive check box GUI element.

Availability element 340 may be presented on display 310 to be used or interacted with by auditor 205. Auditor 205 may interact with availability element 340 to confirm whether a metal defined in a row in listed precious metal holdings 325 is present at the vault location 140. As shown in Figure 3, the availability element 340 may be labelled with text “availability”, but is not limited thereto. In Figure 3, the auditor 205 may be presented with text “Y / N” within the availability element 340 associated with a listed precious metal holdings 325. The auditor 205 may then select either “Y” to confirm the availability or “N” to confirm unavailability of the precious metal.

[0089] The attach element 345 may include a selectable option on display 310, caused by the execution of a script from client vault audit application 143. The attach element 345 may include an interactive checkbox GUI element. The attach element 345 may be presented on display 310 to be used or interacted with by auditor 205. As shown in Figure 3, the attach element 345 may be labelled with text “attach”, but is not limited thereto. Auditor 205 may interact with attach element 345 to open an application to cause or allow vault camera 148 to capture at least one image, such as a colour photo, of a precious metal 210 in the vault location 140. After capturing the image, “Attach” may allow for the user to attach the image as image data to the interactive inventory list of the client audit vault application 143. After capture of the one or more images, the client audit vault application 143 subsequently allows closure of the camera application to return to the display shown in Figure 3. In some embodiments, the attach element 345 is visually modified on the display after capturing the one or more images.

[0090] In some embodiments, there may be a display of a feedback information section 330 as shown in Figure 3. In some embodiments, the vault device processor 141 running client vault audit application 143 performs a comparison to verify the inventory list data received from server 120 after the server has sent the information at step 815 in Figure 8 with the data input into the precious metal holdings 325. In some embodiments, the verification of the inventory list data with the input data is caused when a row entry in the precious metal holdings 325 and/or check and attach section 315 is completed. In some embodiments, if the data matches, the feedback information section 330 may display a message confirming that the list items at the row entry completed matches, as shown in Figure 3 where the Item number 2 has been checked and verified. Display 310 may also include information pertaining to whether or not the audit list is complete as shown in Figure 3 where there is an indication message that the audit is complete. In some embodiments, when one or more entries in the precious metal holdings does not match there may be a message displayed within the feedback information section 330 describing that there is a mismatch, and it may also specify the mismatch details such as the particular item number, the column header, and the expected data from the inventory list data received from server 120.

[0091] In some embodiments, there may also be a submit form button 335 on display 310 which when selected by the vault auditor 205 may cause vault device 147 to send audit data to server 120. In some embodiments, the audit data comprises data from the precious metal holdings 325, and optionally data derived from the check and attach section 315 in the form of inventory list data. In some embodiments, the audit data may also comprise one or more still or video images recorded by video camera system 160 and/or vault camera 148. The sent audit data may be received by server 120 at step 820 in Figure 8.

[0092] Figure 8 shows a flowchart of a method of server system operation for managing audits of precious metal storage locations.

[0093] Method 800 commences when the server 120 receives an external or internal prompt to commence an audit at a vault location 140 at step 805 of Figure 8. In some embodiments, server 120 may execute a software process to commence an audit at multiple vault locations 140 at step 805. In order to facilitate an audit for each of multiple precious metal storage locations, the server processor 121 may execute all the steps described in method 800 for each location in sequence, or alternatively some or all the method 800 steps may be executed by server processor 121 in parallel to each other for each location. In some embodiments, commencement of the audit process may be prompted by an internal monitoring process of server 120 or may be prompted by an audit trigger event, such as an interaction between server 120 and a client vault audit application 143. For example, when the client vault audit application 143 is launched for the first time each day or for the first time after the vault device 147 is powered on, it may communicate with the server 120 to check whether an audit process is required to be performed at that time. In some embodiments, the audit trigger event may include an entrance notification from vault device 147 to server 120. The vault device 147 may generate and then send the entrance notification following receipt from entrance sensor system 165 of a notification regarding a recent sensor event detected at the entrance of vault location 140. The recent sensor event detected at the entrance of vault location 140 may be the latest time of the entrance of vault location 140 being opened and/or closed.

[0094] In some embodiments, the server 120 may receive a prompt signal sent from the audit request device 110 whose processor 111 is executing the audit requester application 113. The prompt signal is received by the notifications module 123 via the communications module 128 of the server 120. In an alternative embodiment, the prompt may be a signal sent from the vault device 147 whose processor 141 is executing the client vault audit application 143. The client vault audit application 143 may be configured to generate the prompt to conduct the audit process at least once per business day or other time period. Other time periods may include: every second business day, every 12 hours, every 6 hours, or every 4 hours, for example. In another alternative embodiment, the prompt signal may be sent from server processor 121 after server processor 121 executes inventory module 124 and determines from inventory data 132 and vault details data 137 whether there are inventory balance changes over a threshold for one or more vault locations 140, or discrepancies with prior audits in audit request records 131 with vault inventory levels recorded in vault details data 137.

[0095] Upon receiving a prompt at step 805 the server processor 121 executing notifications module 123 will undertake the next step 810.

[0096] In some other embodiments, the server processor 121 executes notifications module 123, which contains code to periodically prompt the server processor 121 to search vault details data 136 to read an audit schedule of a vault, wherein the audit schedule defines time stamps for audits to be undertaken at a vault location 140. Upon finding a time stamp within or exceeding a predetermined time threshold for conducting an audit, the server processor 121 may execute notifications module 123 to trigger the next step 810. [0097] Method 800 continues with the server processor 121 loading and then executing inventory module 124, wherein inventory module 124 contains code to cause server processor 121 to read inventory data 132 from data store 130 in order to find a precious metal inventory record for vault location 140 at step 810 in Figure 8. In some other embodiments, inventory module 124 contains code to cause server processor to make an API request to retrieve a precious metal inventory record for vault location 140 from a data store 185 accessible by reserve management server 180. In some embodiments, the precious metal inventory record for a vault location 140 may be or include inventory list data. In some embodiments, there may be inventory list data for multiple vault locations 140 stored in inventory data 132, wherein inventory list data for each vault location 140 forms a part of data describing an entire precious metal inventory. In some embodiments, the inventory list data may be contained in vault details data 137. Inventory data 132 may contain data pertaining to precious metal inventory stored at a vault location 140. In some embodiments, the inventory data 132 may be or include data pertaining to the precious metal inventory from the latest conducted audit. In some embodiments, the data within inventory data 132 and vault details data 137 pertaining to precious metal inventory stored at a vault location 140 may have been updated since the latest conducted audit by other systems or server 120. In some embodiments, updates to inventory data 132 and vault details data 137 may be caused by transaction updates made by server processor 121 executing code from transaction tracking module 129. In some embodiments, transaction updates may comprise indications of withdrawals or additions of precious metal in the vault location 140 due to transactions, or transfers of a quantity of the precious metal to other locations. In some embodiments, the precious metal inventory stored at a vault location 140 may also or instead be stored in vault details data 137, and thus server processor 121 may instead read the inventory list data from vault details data 137 at step 810 in Figure 8.

[0098] Once the server processor 121 has read the inventory list data for vault location 140, method 800 continues with the server processor 121 executing notifications module 123, wherein notifications module 123 contains code to cause server processor 121 to send an audit commencement notification and the inventory list data to vault device 147 at step 815 in Figure 8. The audit commencement notification may comprise a message or command to vault device 147 to initiate the audit process. The audit commencement notification may comprise data to indicate to vault device 147 to record the audit process over a specified duration. In some embodiments, the specified duration is a non-zero duration of time up to 30 or 60 minutes. For example, the duration may be between about 1 minute and about 60 minutes, or a different duration, such as about 5 minutes to about 30 or 40 minutes or about 5 minutes to less than about 1 hour. The audit commencement notification may also comprise data indicating the latest time of the entrance of vault location 140 being opened and/or closed from the received entrance notification,

[0099] Once the server processor 121 has sent the audit commencement notification and the inventory list data, method 800 continues with the server processor 121, executing notifications module 123, receiving an audit commencement confirmation notification and (optionally) location information from vault device 147 at step 816 in Figure 8. In some embodiments, the location information comprises geographical coordinate data, wherein the geographical coordinate data defines or indicates the physical location of vault location 140, or a physical location in proximity to vault location 140. In some embodiments, the geographical coordinate data is determined by the location module 146 in the vault device 147. In some embodiments, the geographical coordinate data is determined from a GPS receiver (or other form of GNSS receiver) contained within vault device 147. In some embodiments, the geographical coordinate data is determined from vault device 147 receiving data from a mobile network. In some embodiments, the location information comprises internet address data, wherein the internet address data may, when processed, pertain to or at least partially define the physical location of vault location 140, or a physical location in proximity to vault location 140. In some embodiments, the location information comprises an address, such as a postage, business or residential address. In some embodiments, the server processor 121 executing notifications module 123 further stores the location information in location data 136. [0100] In some embodiments, the server processor 121 may also receive from vault device 147 data of personnel identification information. In some embodiments, data of personnel identification information is generated from a personnel identification check to validate the identity of the auditor 205. The personal identification check may be performed by the auditor 205 interacting with vault device 147 whose processor 141 is executing client vault application 143. . In some embodiments, one or more factors of personnel identification comprises identification data such as numbers, names, passwords, passcodes, fingerprint, voice audio, face image, or data derived from a secure token or NFC card, for example. In some embodiments, the client vault application 143 invokes a facial recognition process as part of a personnel authentication and/or identification process. The facial recognition process requires the personnel logging into the client vault application 143 to record multiple images of the person’s face, which in most embodiments will be done using a camera integrated with the vault client computing device 147. Those multiple facial images are then input to a facial recognition engine (executing on the vault client computing device 147) to process the images and compare them to facial images of the personnel stored in stored data 144 and/or in data store 130. The client vault application 143 may use a commercially available facial recognition engine, such as that provided by Jumio of Palo Alto, CA, USA. The facial recognition engine may have a feature to detect whether the recorded facial images received at the vault client computing device 147 are from a live feed and to reject authentication or identification where the facial recognition engine detects that the recorded facial images are not from a live feed.

[0101] In some embodiments, the server processor 121, upon receiving data of personnel identification, executes vault session module 125. Vault session module 125 contains code to cause server processor 121 to read and determine whether there is a match of the data of one or more factors of personnel identification with the records in vault details data 137. If there is no match, the server processor 121 may execute notifications module 123 which contains code to send a notification message to vault device 147 notifying that there is an identification problem, and server processor 121 may not proceed to step 817. In an alternative embodiment, if there is no match, the server processor 121 executing vault session module 125 may store the personnel identification information (or a corresponding identifier) in vault details data 137, and store data in audit request records 131 pertaining to the identity of auditor 205 in relation to the particular audit request. If there is a match, the server processor 121 executing vault session module 125 may store identity data read from the matching entry in vault details data 137 and store in audit request records 131 an entry for the particular audit request, wherein the data pertains to the identity of auditor 205.

[0102] Once server processor 121 has received location information defining a geographic location of vault device 147, the server processor 121 may load and execute data analysis module 127, wherein data analysis module 127 contains code to cause server processor 121 to process the geographic location indicator to determine whether the geographic location indicator is the correct location at decision step 817 in Figure 8. In some embodiments, the server processor 121 executing data analysis module 127 may read a set vault location data from vault details data 137, wherein the set vault location data comprises data accurately defining the geographic location of vault location 140, for example in longitude and latitude to an accuracy of 10 to 50 metres. In some embodiments, the data analysis module 127 then compares the geographic location indicator to the set vault location data, to check for a matching location. If the geographic location indicator and the set vault location data do not exactly match, the server processor 121 (executing data analysis module 127) may calculate a difference in the distance between the two location data, and then determine if the distance is within a predetermined distance threshold. In some embodiments, if the calculated difference between the two location data is within a predetermined threshold, the server processor 121 may determine that the two location data are an approximate positive match. Otherwise, the server processor 121 will determine that there is a negative match (or no match) of the geographic location indicator to the set vault location data.

[0103] The location match may be determined based on a location margin of error, vicinity threshold or within a predetermined location accuracy, for example. The vicinity threshold or location margin of error may be a distance of around 20 metres to around 100 metres, or around 30 metres to around 50 metres, for example. For example, if the received location information of the vault device 147 indicates that a location of the vault device 147 is within the vicinity threshold or location margin of error (e.g. around 50 metres or yards) of the set vault location (i.e. a location previously stored in vault details data 137 as the location of the secure storage structure), then the server processor 121 determines a positive location match. Otherwise, if the received location information of the vault device 147 indicates that a location of the vault device 147 cannot be determined or is not within the vicinity threshold or location margin of error of the set vault location, then the server processor 121 determines a negative location match.

[0104] In some embodiments, the server processor 121 executing data analysis module 127, prior to determining whether the two location data match, may convert the location data form of the geographic location indicator and/or the set vault location data. Converting the location data form may comprise calculating an approximate location data of another form. For example, the geographic location indicator may be an IP (internet protocol) address while the set vault location data may be a physical business address. The server processor 121 may then convert both location data to geographical coordinate data prior to determining whether the two location data match at decision step 817. In some embodiments, the server processor 121 may then store the geographic location indicator, the set vault location data, the difference calculation, or the calculated difference in distance in location data 136.

[0105] In some embodiments, decision step 817 is not necessarily undertaken during method 800, and therefore also step 818 would not be undertaken in method 800. In this case, location information may instead be stored for future reference. In some embodiments, in the aforementioned case, the location information is not even sent by vault device 147, or is sent by vault device 147 but not stored at the server 120.

[0106] In further embodiments, the location data of the vault device 147 is repeatedly or continuously determined and sent by vault device 147 to the server 120 during the audit session as part of the audit data. Server processor 121 executing data analysis module 127 determines a likelihood that an average of the location data is at or within a threshold vicinity of vault location 140. Server processor 121 may execute the decision steps 817 and step 818 later in method 800, such as after step 840, to account for inaccuracies in initial or particular location data.

[0107] In some embodiments, if the geographic location data and the set vault location data are determined by server processor 121 to not match at step 817, the server processor 121 executing data analysis module 127 may initiate an incorrect geographic location procedure at step 818 in Figure 8. According to some embodiments, step 818 comprises server processor 121, executing notifications module 123, sending a notification to audit request device 110 informing the audit request device 110 with details pertaining to the calculated mismatch at step 817. In some embodiments, step 818 comprises server processor 121, executing notifications module 123, sending a notification to vault request device 147 in order to cause vault device 147 to send geographic location indicator data of the same form previously sent or geographic location indicator data of a form not previously sent, thus causing the server 120 executing method 800 to then undertake the method 800 from step 816 again.

[0108] In some embodiments, if the geographic location data and the set vault location data are determined by server processor 121 to match at step 817, the server processor 121 will load and execute vault session module 125 and begin a secure information sharing session with vault device 147 at step 819 in Figure 8. The server processor 121 executing vault session module 125 may then receive audit data from vault device 147 in step 820 in Figure 8.

[0109] In some embodiments, a location match of the vault device 147 with the set vault location may not be required in order to proceed with the audit process. In such embodiments, the geographic location data of the vault device 147 is recorded as part of the audit session data captured through vault device 147 and sent to the server 120. Discrepancies in geographic location of the vault device 147 (as compared to the location of vault locationl40) may be the subject of a separate audit process, such as may be performed by audit device 110, for example. [0110] In an alternative embodiment, the server processor 121 executing vault session module 125 instead sends a notification to vault device 147 at step 819 without beginning a secure information sharing session. In some embodiments, the server processor 121 executing session module 125 may be connected to a session by vault device 147, and then receive audit data comprising one or more still or video images, and inventory list data from vault device 147 in step 820 in Figure 8. In some embodiments, the one or more still or video images may comprise an image purported to be of precious metal stored at a vault location 140. In some embodiments, the one or more still or video images may comprise footage, for example as captured by camera system 160, of the audit being undertaken by the auditor 205 at the vault location 140.

[0111] In some embodiments, step 819 and 820 may be implemented using a file sharing protocol, for example SFTP, FTPS or SMB.

[0112] In some embodiments, the server processor 121 may then load and execute image analysis module 126, wherein the image analysis module 126 contains code to cause the server processor 121 to determine whether an error exists within audit data sent from vault device 147 in step 825 shown in Figure 8. In some embodiments, server processor 121 may instead load and execute data analysis module 127, wherein data analysis module 127 contains code to cause server processor 121 to determine whether an error exists within audit data sent from vault device 147 in step 825 shown in Figure 8. In some embodiments, step 825 may comprise steps 926, 928, 930 and 932 as shown in Figure 9, wherein steps 926 and 932 are completed by server processor 121 executing data analysis module 127, and steps 928 and 930 are completed by server processor 121 executing image analysis module 126. In some embodiments, the server processor 121 executing either image analysis module 126 or data analysis module 127 may then determine if the error is within a predetermined threshold at step 830 in Figure 8. In some embodiments, step 830 comprises steps 927, 929, 931 and 933 as shown in Figure 9, wherein steps 927 and 933 are completed by server processor 121 executing data analysis module 127, and steps 929 and 931 are completed by server processor 121 executing image analysis module 126. [0113] In some embodiments, when an error is detected at step 830, the server processor 121 executing notifications module 123 may send a supplemental audit request notification to vault device 147 as shown in step 835 in Figure 8, wherein the supplemental audit request notification indicates that supplemental information is required such as new still or videos images, or inventory list data. In some embodiments, at step 835, the server processor 121 executing vault session module 125 may again receive audit data (in the form of supplemental audit data) from vault device 147 in step 820 shown in Figure 8. In some embodiments, once step 835 has been executed a predetermined number of times for a particular commenced audit, the server processor does not send a new audit request notification to vault device 147 when an error is detected at step 830.

[0114] If no errors are detected at step 830, the server processor 121 executing the vault session module 125 stores the audit data in audit request records 131 in step 840. Alternatively, the server processor 121 executing the vault session module 125 may instead store the audit data in audit request records 131 in steps 820, 825, 830 and 835 when applicable information is received by server processor 121. In some embodiments, received and/or stored audit data comprises any of: audit images, audit videos, inventory list data, timestamps, results of processing from server processor 121 executing data analysis module 127, results of processing from server processor 121 executing image analysis module 126, location data, messages from vault device 147, and notifications data, for example.

[0115] In some embodiments, the server processor 121 executing notifications module 123 may send a notification of a completed audit to interested devices in step 845 shown in Figure 8. In some embodiments, interested devices comprises vault device 147 and audit request device 110. In some embodiments, server processor 121 executing notifications module 123 may also store the sent notifications and any responses of the sent notifications to audit request records 131 ending the method 800 at step 850 as shown in Figure 8. [0116] Figure 9 shows a flowchart of the error detection within the method of server system operation for managing audits of precious metal storage locations.

[0117] In some embodiments, after the server processor 121 executing the vault session module 125 receives the audit data comprising one or more still or video images, and inventory list data from vault device 147 in step 820, the server processor 121 may execute data analysis module 127 in order to compare received inventory list data from step 820 with the sent inventory list data from step 815 in step 926 shown in Figure 9. In some embodiments, the received inventory list data is stored in inventory data 132. In some embodiments, the sent inventory list data is stored in inventory data 132 or vault details data 137. In some embodiments, comparing the inventory list data comprises comparing the file sizes of the sent inventory list data and the received inventory list data. In some embodiments, comparing the inventory list data comprises identifying any differences between both the sent inventory list data and the received inventory list data. In some embodiments, identifying any differences comprises identifying differences in number of items and identifying differences in characteristics of corresponding items. In some embodiments, identifying any differences comprises identifying a number of confirmations in the received list data and comparing to the number of items in the sent inventory list data. In some embodiments, step 825 comprises step 926 as described for Figure 8.

[0118] In some embodiments, the server processor 121 executing data analysis module 127 then determines whether or not the sent inventory list data and the received inventory list data match at decision step 927. In some embodiments, predetermined thresholds are used to determine acceptable differences between the two inventory list data, wherein an acceptable difference is a difference of a number or characteristic having a magnitude above or below a predetermined threshold, and therefore the acceptable difference will be deemed to not cause the lists to be unmatched. In some embodiments, the threshold applied to the difference of the number is zero or is nonzero. In some embodiments, step 830 comprises step 927 as described for Figure 8. [0119] In some embodiments, if the server processor 121 determines that the lists match at step 927, then the server processor 121 may execute image analysis module 126 and check for any image irregularities, for example by making an image duplication check at step 928 shown in Figure 9. An image duplication check may comprise the server processor 121 executing image analysis module 126 reading image data from audit request records 131, such as from image data 134, in order to find one or more prior images. The image duplication check may comprise the server processor 121 executing image analysis module 126 reading video data from audit request records 131, such as from video recording data 133, in order to find one or more prior video recordings. The image duplication check may further comprise server processor 121 executing image analysis module 126 to determine whether one or more prior images matches any images received from step 820.

[0120] In some embodiments, determining whether one or more prior images matches another image received comprises in part determining one or more probabilities of a match between the one or more prior images and the image received, wherein each probability of a match is associated with each prior image. An image duplication check may further comprise server processor 121 executing image analysis module 126 to determine whether one or more prior video recordings matches any video recordings received from step 820, wherein each probability of a match is associated with each prior video recording. In some embodiments, determining whether one or more prior images matches another image received comprises in part determining one or more probabilities of a match between the one or more prior video recordings and the video recording received.

[0121] In some embodiments, the server processor 121 executing image analysis module 126 then determines if the images match in decision step 929 shown in Figure 9. In some embodiments, server processor 121 executing image analysis module 126 generates or has already stored a predetermined threshold of the likelihood of a matching image. In some embodiments, each of the probabilities of a match between the one or more prior images and the image received is compared with the predetermined threshold of the likelihood of a matching image, wherein if any of the probabilities of a match between the one or more prior images and the image received exceeds the predetermined threshold of the likelihood of a matching image then server processor 121 executing image analysis module 126 determines that the image received is a duplicate. The predetermined threshold may help to detect whether one or more prior images has been altered, modified or doctored to produce the image received, for example. In some embodiments, server processor 121 executing image analysis module 126 generates or has already stored a predetermined threshold of the likelihood of a matching video recording. In some embodiments, each of the probabilities of a match between the one or more prior video recordings and the video recording received is compared with the predetermined threshold of the likelihood of a matching video recording. If any of the probabilities of a match between the one or more prior video recordings and the video recording received exceeds the predetermined threshold of the likelihood of a matching video recording then server processor 121 executing image analysis module 126 determines that the video recording received is a duplicate.

[0122] In some embodiments, if the server processor 121 determines that the images and video recordings are not a duplicate at step 929, then the server processor 121 may execute image analysis module 126 and make a bullion check at step 930 shown in Figure 9. In some embodiments, the bullion check comprises the server processor 121 executing image analysis module 126 applying one or more image recognition models on the images or video recordings received from step 820. In some embodiments, the one or more image recognition models comprise a model which detects images or video recordings received which contain no recognisable images of precious metal. Images or video recordings received which are detected to contain no recognisable images of precious metal may then not require any more processing using image recognition models in step 930. In some embodiments, the one or more image recognition models are built using machine learning techniques. In some embodiments, the one or more image recognition models are used for feature detection or character recognition in an image or video recording as shown in Figure 4. In some embodiments, the one or more image recognition models are used for recognising the size or shape of a bullion in an image or video recording as shown in Figure 5. In some embodiments, the one or more image recognition models are used for recognising the precious metal element or quality of the bullion in an image or video recording as shown in Figure 6.

[0123] In some embodiments, the image analysis module 126 comprises program code to perform object detection to detect one or more bullion objects in the images and video recordings received at step 820. Step 930 of checking bullion in the received images and videos may be performed using various object detection techniques including object detection by trained deep neural networks, use of scale-invariant feature transform (SIFT), or use of a Viola-Jones object detection framework.

[0124] Use of deep neural networks may comprise a pre-training of a deep neural network using a large number of training images comprising a known number and types of bullion objects with a boundary information identifying each bullion object in each training image. Once a deep neural network is trained on a robust training dataset, the trained deep neural network may be used by the image analysis module 126 to determine a number and type of bullion objects present in the images and video recordings received at step 820. The various types of deep neural networks suitable for object detection include: REGION BASED concurrent neural networks (R-CNN, Fast R-CNN, Faster R-CNN), Single Shot MultiBox Detector (SSD), You Only Look Once (YOLO), for example.

[0125] Use of a scale-invariant feature transform (SIFT) comprises transformation of an image into a large collection of feature vectors, each of which is invariant to image translation, scaling, and rotation, partially invariant to illumination changes and robust to local geometric distortion. A predefined set of feature key point vectors corresponding to each class of objects are used for comparison with each of the transformed feature vectors. A matching object class may be identified based on Euclidean distances between the feature key point vectors and the transformed feature vectors. Use of the Viola-Jones object detection framework comprises a similar feature extraction and comparison technique to identify the bullion objects in the images and video recordings received at step 820. [0126] In some embodiments, the server processor 121 executing image analysis module 126 then determines if there is an incorrect bullion in the images at step 931 shown in Figure 9. In some embodiments, to determine an incorrect bullion the server processor 121 executing image analysis module 126 initially reads received inventory list data stored in inventory data 132 from step 820. In some embodiments, the server processor 121 executing image analysis module 126 then determines if the images of precious metal bullions analysed in step 930 match the received inventory list data. In some embodiments, matching comprises matching the features, characters, size, shape, elements and quality of the precious metals shown in the images of the precious metal bullions with the features, characters, size, shape, elements and quality listed in the received inventory list data. In some embodiments, server processor 121 executing image analysis module 126 calculates a probability of a match between an image of a precious metal bullion and an inventory list item. In some embodiments, server processor 121 executing image analysis module 126 then applies a predetermined threshold probability, wherein if the probability of a match between and image of a precious metal bullion and an inventory list item exceeds the predetermined threshold probability, then the image of a precious metal bullion may be determined by server processor 121 to match the inventory list item. In some embodiments, the server processor 121 instead reads and applies the sent inventory list data rather than the received inventory list data for step 931.

[0127] In some embodiments, if the server processor 121 determines that there isn’t an incorrect bullion in the images at step 931, then the server processor 121 may execute data analysis module 127 and check for any image or data irregularities by making a metadata check at step 932 shown in Figure 9. In some embodiments, a metadata check may comprise server processor 121 executing data analysis module 127 reads the metadata of the received data from step 820 wherein the received data comprises received images, video recordings and inventory list data stored in audit request records 131. In some embodiments, metadata may comprise timestamps or location information. In some embodiments, the metadata check may comprise server processor 121 executing data analysis module 127 reads timestamps of when the session commenced in step 819 from audit request records 131. The metadata may comprise data from the entrance notification pertaining to the latest time of the entrance of vault location 140 being opened and/or closed.

[0128] In some embodiments, the server processor executing data analysis module 127 then determines if there is incorrect metadata or a data irregularity at step 933, as shown in Figure 9. In some embodiments, the timestamp metadata check may comprise server processor 121 executing data analysis module 127 to determine if the timestamp metadata of the received audit data were after the session commencement time and before the commencement of step 933. If the data analysis module identifies that the received timestamp data is before the session commencement time or after commencement of step 933, data analysis module 127 may identify that there is incorrect metadata or a data irregularity. In some embodiments, the location metadata check may comprise server processor 121 executing data analysis module 127 to determine if the location metadata of the received audit data is within or close to the vault location 140. If the data analysis module identifies that the location metadata of the received audit data is not within the vault location, data analysis module 127 may identify that there is incorrect metadata or a data irregularity. In some other embodiments, if the data analysis module 127 identifies that the received audit data is not within close enough proximity to the vault location 140, based on a proximity threshold, data analysis module 127 may identify there is incorrect metadata or a data irregularity. In some embodiments, the data analysis module 127 determines if the time passed since the latest time of the entrance of vault location 140 being opened and/or closed has exceeded a predetermined entrance time threshold. If the determined time passed exceeds the predetermined entrance time threshold, the data analysis module 127 may consequently identify that there is incorrect metadata or a data irregularity. Where incorrect data or a data irregularity is identified as part of 933, then the server processor 121 may send a request to the vault device to perform a supplemental audit at 835.

[0129] In some embodiments, if the inventory lists don’t match at step 927, or duplicate images are detected at step 929, or incorrect bullions are detected at step 931, or there are incorrect metadata or a data irregularity is determined at step 933, the server processor 121 executing image analysis module 126 or data analysis module 127 may transmit a request for a supplemental audit to vault device 147 at step 835 as described for Figure 8 and shown in Figures 8 and 9. In an alternative embodiment, rather than requesting a new audit at step 835, the server 120 may instead initiate a financial transaction for the determined difference in precious metal inventory by communicating with a financial transaction system via network 150. In some embodiments, the initiation of the financial transaction may occur when the inventory lists do not match at step 929, for example the received inventory list from vault device 147 is missing precious metal when compared with the sent list data for the vault location 140.

[0130] In some embodiments, if no incorrect metadata are determined at step 933, the server processor 121 stores audit data at step 840 as described for Figure 8 and shown in Figures 8 and 9.

[0131] Figure 4, 5, 6 and 7 show diagrams of two precious metal bullion 210 wherein each figure shows an alternative combination of characteristics of the bullion captured in an image by device vault device 147 as shown in Figure 2. In Figure 3 the auditor 205 is presented with listed precious metal holdings 325 wherein there are two items listed which are meant to represent two precious metal bullion 210 stored at the vault location 140. The auditor 205 may then interact with the check and attach section 315 which may cause a camera interface to appear on display 310 of vault device 147, and allow the auditor 205 to take images of the two precious metal bullion at the vault location, and to confirm the precious metal holdings.

[0132] Figure 4 shows a diagram of two precious metal bullion which don’t match the listed precious metal holdings 325. In Figure 4, the top precious metal bullion shown has an id of adz3, it has the form of an ingot, its metal is gold, it has the weight of 50g and purity of 50%. The top metal bullion appears to match item number 1 in listed precious metal holdings 325. In Figure 4 the bottom precious metal bullion listed has an id of adz5, it has the form of an ingot, its metal is gold, it has the weight of 100g and purity of 50%. The bottom metal bullion does not appear to match the remaining item number 2 in listed precious metal holdings 325, due to having a different id to the item number 2.

[0133] Figure 5 shows a diagram of two precious metal bullion which don’t match the listed precious metal holdings 325. In Figure 5, the top precious metal bullion shown has an id of adz3, it has the form of an ingot, its metal is gold, it has the weight of 50g and purity of 50%. The top metal bullion appears to match item number 1 in listed precious metal holdings 325. In Figure 5 the bottom precious metal bullion listed has an id of adz4, it has the form of an ingot, its metal is gold, it has the weight of 50g and purity of 50%. The bottom metal bullion does not appear to match the remaining item number 2 in listed precious metal holdings 325, due to having a different weight to the item number 2. The bottom bullion in Figure 5 has the same size as the top bullion which is used to indicate that the top and bottom bullions have the same weight.

[0134] Figure 6 shows a diagram of two precious metal bullion which don’t match the listed precious metal holdings 325. In Figure 6, the top precious metal bullion shown has an id of adz3, it has the form of an ingot, its metal is silver, it has the weight of 50g and purity of 75%. In Figure 6, the bottom precious metal bullion listed has an id of adz4, it has the form of an ingot, its metal is gold, it has the weight of 100g and purity of 50%. The bottom metal bullion appears to match item number 2 in listed precious metal holdings 325. The top metal bullion does not appear to match the remaining item number 1 in listed precious metal holdings 325, due to having a different element and purity to the item number 2. The top bullion in Figure 6 contains a different pattern to the bottom bullion which represents that the two bullion have different precious metal elements and purity.

[0135] When inspecting the bullion of Figures 4, 5 or 6 while viewing precious metal holdings 325 in Figure 3, the auditor 205 may identify this discrepancy when interacting with the check and attach section 315, and may send this information in the inventory list data via vault device 147 to server 120. In some embodiments, the server processor 121 may then check for an inventory match at steps 926 and 927 and, if identifying a mismatch, may request to vault device 147 that a supplemental audit be undertaken in step 835 or notify another system for resolution. In some embodiments, the auditor 205 may not identify this discrepancy when interacting with the check and attach section 315, and may send this information in the inventory list data via vault device 147 to server 120. In some embodiments, the server processor 121 may then check for an inventory match at steps 926 and 927 and would not identify a mismatch. However, the server processor 121 may detect an incorrect bullion in the image or images attached with the inventory list data in steps 930 and 931. The server processor 121 may consequently request to vault device 147 that a supplemental audit be undertaken in step 835 or notify another system for resolution.

[0136] Figure 7 shows a diagram of two precious metal bullion which match the listed precious metal holdings 325. In Figure 7, the top precious metal bullion shown has an id of adz3, it has the form of an ingot, its metal is gold, it has the weight of 50g and purity of 50%. In Figure 7, the bottom precious metal bullion listed has an id of adz4, it has the form of an ingot, its metal is gold, it has the weight of 100g and purity of 50%.

[0137] When inspecting the bullion of Figures 7 while viewing precious metal holdings 325 in Figure 3, the auditor 205 may incorrectly identify a discrepancy when interacting with the check and attach section 315, and may send this information in the inventory list data via vault device 147 to server 120. In some embodiments, the server processor 121 may then check for an inventory match at steps 926 and 927 and, if identifying a mismatch, may transmit a request to vault device 147 that supplemental information is required in step 835 or notify another system for resolution. In some embodiments, the auditor 205 may correctly identify that there are no discrepancies when interacting with the check and attach section 315, and may send this information in the inventory list data via vault device 147 to server 120. In some embodiments, the server processor 121 may then check for an inventory match at steps 926 and 927 and would not identify a mismatch. In some embodiments, the server processor will conclude the remaining steps of method 800 without undertaking step 835. However in some other embodiments, the server processor may incorrectly detect an incorrect bullion in the image or images attached with the inventory list data in steps 930 and 931. The server processor 121 may consequently request to vault device 147 that supplemental images and/or audit data are required in step 835 or notify another system for resolution.

[0138] FIG. 10 illustrates an example computer system 1000. In particular embodiments, one or more computer systems 1000 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 1000 provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems 1000 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 1000. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate. The server 120, vault device 147, vault computer system 175, entrance sensor system 165, video camera system 160, reserve management server 180, transaction management server 190, and audit request device 110 may incorporate a subset or all of the computing components described with reference to the computer system 1000 to provide the functionality described in this specification.

[0139] This disclosure contemplates any suitable number of computer systems 1000 to implement each of the server 120, vault device 147, vault computer system 175, entrance sensor system 165, video camera system 160, reserve management server 180, transaction management server 190, and audit request device 110. Computer system 1000 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. Where appropriate, computer system 1000 may include one or more computer systems 1000; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 1000 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example and not by way of limitation, one or more computer systems 1000 may perform in real-time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 1000 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

[0140] In particular embodiments, computer system 1000 includes a processor 1002, memory 1004, storage 1006, an input/output (I/O) interface 1008, a communication interface 1010, and a bus 1012. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0141] In particular embodiments, processor 1002 includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, processor 1002 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1004, or storage 1006; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 1004, or storage 1006. In particular embodiments, processor 1002 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 1002 including any suitable number of any suitable internal caches, where appropriate. As an example and not by way of limitation, processor 1002 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 1004 or storage 1006, and the instruction caches may speed up retrieval of those instructions by processor 1002. Data in the data caches may be copies of data in memory 1004 or storage 1006 for instructions executing at processor 1002 to operate on; the results of previous instructions executed at processor 1002 for access by subsequent instructions executing at processor 1002 or for writing to memory 1004 or storage 1006; or other suitable data. The data caches may speed up read or write operations by processor 1002. The TLBs may speed up virtual-address translation for processor 1002. In particular embodiments, processor 1002 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 1002 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 1002 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 1002. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0142] In particular embodiments, memory 1004 includes main memory for storing instructions for processor 1002 to execute or data for processor 1002 to operate on. As an example and not by way of limitation, computer system 1000 may load instructions from storage 1006 or another source (such as, for example, another computer system 1000) to memory 1004. Processor 1002 may then load the instructions from memory 1004 to an internal register or internal cache. To execute the instructions, processor 1002 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 1002 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor 1002 may then write one or more of those results to memory 1004. In particular embodiments, processor 1002 executes only instructions in one or more internal registers or internal caches or in memory 1004 (as opposed to storage 1006 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 1004 (as opposed to storage 1006 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple processor 1002 to memory 1004. Bus 1012 may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor 1002 and memory 1004 and facilitate accesses to memory 1004 requested by processor 1002. In particular embodiments, memory 1004 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 1004 may include one or more memories 1004, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0143] In particular embodiments, storage 1006 includes mass storage for data or instructions. As an example and not by way of limitation, storage 1006 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magnetooptical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage 1006 may include removable or non-removable (or fixed) media, where appropriate. Storage 1006 may be internal or external to computer system 1000, where appropriate. In particular embodiments, storage 1006 is non-volatile, solid-state memory. In particular embodiments, storage 1006 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 1006 taking any suitable physical form. Storage 1006 may include one or more storage control units facilitating communication between processor 1002 and storage 1006, where appropriate. Where appropriate, storage 1006 may include one or more storages 1006. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0144] In particular embodiments, I/O interface 1008 includes hardware, software, or both, providing one or more interfaces for communication between computer system 1000 and one or more I/O devices. Computer system 1000 may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system 1000. As an example and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces 1008 for them. Where appropriate, I/O interface 1008 may include one or more device or software drivers enabling processor 1002 to drive one or more of these I/O devices. I/O interface 1008 may include one or more I/O interfaces 1008, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.

[0145] In particular embodiments, communication interface 1010 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 1000 and one or more other computer systems 1000 or one or more networks. As an example and not by way of limitation, communication interface 1010 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wirebased network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 1010 for it. As an example and not by way of limitation, computer system 1000 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 1000 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WLMAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system 1000 may include any suitable communication interface 1010 for any of these networks, where appropriate. Communication interface 1010 may include one or more communication interfaces 1010, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface. [0146] In particular embodiments, bus 1012 includes hardware, software, or both coupling components of computer system 1000 to each other. As an example and not by way of limitation, bus 1012 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a frontside bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 1012 may include one or more buses 1012, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application- specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

[0147] It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.