Related Applications

This Application claims the benefit of priority of Israeli Patent Application 286,420, filed September 14, 2022, entitled "Smart Aiming Device with Built-In Training System for Marksmanship and Firearm Operation," the contents of which are incorporated by reference as if fully set forth herein.

Field of the Invention

The present disclosure, in some embodiments, concerns a training system for marksmanship and firearm operation, and more specifically, but not exclusively, to a smart aiming device for a firearm that provides feedback to a user regarding one or more techniques of firearm operation based on analysis of sensor data collected during operation of the firearm.

Background of the Invention

Marksmanship training refers to the process of teaching a user how to safely and effectively use a firearm. Marksmanship training may include instruction as to how to aim the firearm, how to hold the firearm steadily, and how to actuate the trigger.

Historically, an operator of a firearm would train in the field. The training would commence with basic training at a shooting range, followed by more complex scenarios such as squad trainings. The marksmanship training would usually be done with live fire, at large facilities, and under supervision of paid instructors. This form of training was necessarily expensive, and also did not necessarily provide training of satisfactory quality.

In later years, simulators were introduced to the world of training, allowing simulation of various environments for a limited number of trainees. However, the simulators were expensive to build and maintain, and often incorporated different equipment than what the user would actually have in combat.

Recently, some manufacturers have implemented marksmanship training systems that are attachable to firearms. The system includes a small device, containing sensors, that is attachable to the rail, or a different portion, of the firearm. The sensors record information about the movements and orientation of the firearm prior to and during actuation of the trigger. Based on these movements, the system conveys a report to the user regarding how to improve his or her firing techniques.

Separately, certain firearm manufacturers have developed fire control systems (FCS) that apply image recognition technology in order to assist firearm users in aiming. A fire control system may include an integrated image sensor and processor. The processor may include image processing software that detects potential targets within a scene, allows "locking" onto these targets, tracks them, and calculates the proper aim point that will ensure a hit on a target. The fire control system may assist in timing the discharge of a bullet by the processor, to a moment when the firearm is properly aimed. An exemplary fire control system is disclosed in U.S. Patent 10,097,764, the contents of which are incorporated by reference as if fully set forth herein.

Summary of the Invention

Marksmanship trainers that have been developed to date typically require installation of new hardware. For example, many marksmanship trainers contain sensors that are installed on side of the barrel, near the muzzle. These sensors are installed solely for the purpose of collecting data for the marksmanship training, and are removed during regular use of the firearm. This is counterproductive. The user trains on a firearm that has a different weight and feel than the firearm he is using during a real combat operation. The user will struggle to adapt his or her training to the real world. In particular, this does not correlates with the "Fight as you train, train as you fight" principle that has been a core component of training in many armies, including the United States Army.

Accordingly, there exists a need for a marksmanship training aid that is implemented on a firearm using the exact same hardware components that will be installed on the firearm during regular usage of the firearm.

The present disclosure, in certain embodiments, relates to a smart aiming device that includes a built-in marksmanship training system. The same hardware components that are used to provide inputs and outputs for the smart aiming device during operational use are also used to provide inputs and outputs for the marksmanship training system. As a result, the marksmanship training system does not add any weight to the firearm or affect the user experience of firing the firearm. The training provided by the marksmanship training system may thus be easily and effectively applied by the user.

According to a first aspect, a smart aiming device for a firearm is disclosed. The device includes a processor, a memory, and a plurality of sensors including an inertial measurement unit (I MU) for measuring movement of the firearm; a trigger sensor for monitoring a depression of a trigger of the firearm; and an image sensor configured to capture images of a target region. During operation of the firearm, the sensors collect data and provide said data to the processor and memory for analysis and storage. The device further includes a sight including an optical window for viewing a target therethrough via at least one of an arrangement of optical lenses and a micro-display. The memory includes a non-transitory computer-readable medium storing therein instructions, that, when executed by the processor, causes the processor to provide feedback to a user regarding one or more techniques of firearm operation, based on analysis of the data collected by the plurality of sensors during operation of the firearm.

In another implementation according to the first aspect, the feedback includes feedback on how far an actual point of aim of the firearm was from the desired point of aim during actuation of the trigger.

In another implementation according to the first aspect, the feedback includes feedback on technique of trigger actuation.

In another implementation according to the first aspect, the feedback includes feedback on barrel roll.

In another implementation according to the first aspect, the feedback includes instruction on how to compensate aiming of the firearm based on at least one of type of ballistics, movement of the target, range to the target and wind.

In another implementation according to the first aspect, the firearm is adjustable between an unlocked mode, in which the firearm fires whenever the trigger is actuated, and a locked fire mode, in which the processor is configured to time a firing of the firearm following actuation of the trigger based on data collected by the plurality of sensors, so that the firearm fires when the processor determines that a locked-on target will be hit in a desired hit area.

Optionally, the feedback includes feedback on how to operate the firearm in the locked fire mode.

Optionally, the feedback includes feedback on a process of locking onto a target.

Optionally, the feedback includes feedback on trigger technique in the locked fire mode.

In another implementation according to the first aspect, the processor is configured to provide the feedback in within an operational mode, in which the processor provides feedback in response to a user request following firing of the firearm.

In another implementation according to the first aspect, the processor is configured to provide the feedback in a non-intrusive manner within an operational mode, by presenting at least one of an icon or text on a perimeter of a field of display and one or more small graphical markings near the target region.

In another implementation according to the first aspect, the processor is configured to provide the feedback in a dedicated training mode, in which the processor automatically provides one or more predetermined categories of feedback to the user following a series of actuations of the firearm.

Optionally, the feedback includes feedback regarding consistency of at least one of the user's accuracy or timing during the series of actuations.

Optionally, the device is operable in both a live fire mode and a dry fire mode, and the device is configured to provide identical feedback following operation during the dry fire mode as during the live fire mode. Optionally, the device is operable in a virtual reality mode in which the processor presents a virtual reality including one or more virtual reality targets to a user via the micro-display, and the user is capable of simulating firing on the one or more virtual reality targets, and the processor provides feedback to the user in the dedicated training mode regarding the one or more techniques performed within the virtual reality mode. Optionally, the device is operable in an augmented reality mode in which the processor presents one or more augmented reality targets super-imposed over a view of a target obtained from the optical lens, and the user is capable of simulating firing on the one or more augmented reality targets, and the processor provides feedback to the user in the dedicated training mode regarding the one or more techniques performed within the augmented reality mode.

Optionally, the device is configured to perform preconfigured training sets that include shooting at least one of from predetermined positions, on predetermined targets and in a limited amount of time.

In another implementation according to the first aspect, each of the plurality of sensors is used in operation of the firearm, such that no additional hardware components are required for providing the feedback.

In another implementation according to the first aspect, a system includes the device and at least one reactive target, and the device is configured to detect a hit of the reactive target by detecting a fall of the reactive target or any other visual signal that the reactive target provides when the reactive target is hit.

In another implementation according to the first aspect, a system includes the device and at least one communicative target, and the device is configured to provide feedback based on information communicated to the processor by the at least one communicative target.

According to a second aspect, a method of providing training for a firearm with a smart aiming device is disclosed. The method includes, during operation of the firearm, collecting data from a plurality of sensors including a trigger sensor for monitoring a depression of a trigger of the firearm; and an image sensor configured to capture images of a target region, and providing said data to a processor and memory for analysis and storage. The method further includes providing feedback to a user regarding one or more techniques of firearm operation, based on analysis of the data collected by the plurality of sensors. The providing step is performed by projecting the feedback from a micro-display into an optical window of a sight of the firearm. In another implementation according to the second aspect, the step of providing feedback includes providing the feedback within an operational mode in response to a user request following firing of the firearm.

In another implementation according to the second aspect, the step of providing feedback includes providing the feedback in a non- intrusive manner within an operational mode, by presenting at least one of an icon or text on a perimeter of a field of display and one or more small graphical markings near the target region.

In another implementation according to the second aspect, the step of providing the feedback comprises providing the feedback automatically in a dedicated training mode following a series of actuations of the firearm.

Brief Description of the Drawings

In the drawings:

FIG. 1 is a schematic block diagram illustrating the uses of hardware of a smart aiming device for both operation of the firearm and for providing feedback, according to embodiments of the present disclosure;

FIG. 2 is a schematic diagram illustrating main components of a smart aiming device, according to embodiments of the present disclosure;

FIG. 3 depicts steps of a method for providing feedback regarding marksmanship and firearm operation, according to embodiments of the present disclosure;

FIG. 4A depicts an example of feedback provided by the smart aiming device following a single shot in operational mode, according to embodiments of the present disclosure;

FIG. 4B depicts an example of feedback provided by the smart aiming device following a single shot in dedicated training mode, according to embodiments of the present disclosure;

FIG. 5 depicts an example of feedback provided by the smart aiming device following a series of shots, according to embodiments of the present disclosure; and FIG. 6 depicts a second example of feedback provided by the smart aiming device following a series of shots, according to embodiments of the present disclosure.

Detailed Description of the Invention

The present disclosure, in some embodiments, concerns a training system for marksmanship and firearm operation, and more specifically, but not exclusively, to a smart aiming device for a firearm that provides feedback to a user regarding one or more techniques of firearm operation based on analysis of sensor data collected during operation of the firearm.

Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and/or methods set forth in the following description and/or illustrated in the drawings and/or the Examples. The invention is capable of other embodiments or of being practiced or carried out in various ways.

FIG. 1 illustrates a schematic block diagram of a smart aiming device 100 having hardware components suitable for functioning of an operational mode 110, for providing feedback 120 within the operational mode, and additionally or alternatively for providing feedback in a dedicated training mode 130, according to embodiments of the present disclosure.

The operational mode 110, feedback 120 and dedicated training mode 130, each utilize data collected by a plurality of sensors 140. These sensors 140 include trigger sensor 142 and image sensor 144.

Trigger sensor 142 is configured to detect depression of the trigger of the firearm. The trigger sensor 142 may incorporate any technology known to those of skill in the art for detecting actuation of a trigger. For example, trigger sensor 142 may be a movement sensor, such as magnetic sensor or a Hall Effect sensor. Alternatively, the trigger sensor 142 may be a trigger pressure sensor.

Image sensor 144 may include any type of image sensor that is known or that may become known, such as a CMOS sensor, a CCD sensor, a forward-looking infra- red sensor, a multispectral or hyper-spectral camera, or any other sensor suitable for the functions described herein.

Optionally, sensors 140 include an inertial measurement unit (IMU) 146. Inertial measurement unit (IMU) 146 detects linear acceleration of the firearm using one or more accelerometers, as well as rotational rate of the firearm using one or more gyroscopes. The inertial measurement unit 146 may optionally include a magnetometer. The IMU 146 is used to detect movement of the firearm and change of orientation of the firearm. Optionally, sensors 140 may include other types of sensors. Without limitation, such sensors may include: a microphone; inclinometer; accelerometer/inertial sensor; compass; GPS, Laser Range Finder (LRF), temperature measurement device (e.g. thermometer, thermocouple); barometer; and anemometer. Such components may improve the accuracy of smart aiming device 100, and compensate for environmental factors that affect firing accuracy. Certain of the above-listed sensors may also optionally be used to provide intelligence, e.g. a geospatial information system (GIS) and GIS data base, which may include capability for determining user location.

Smart aiming device 100 further includes a processor 150. Processor 150 is integrated with a non-transitory computer-readable medium having stored thereon software instructions, that when executed by the processor, causes the processor to perform various functions, including the provision of feedback, as set forth herein. In addition to storing the software instructions, the memory may have information stored thereon regarding a type of ballistics loaded in the firearm. Optionally, the processor 150 includes wireless communication hardware, for transmitting data back and forth between a user device such as a smartphone or tablet. Optionally, processor 150 includes a barrel tracking module. The barrel tracking module detects a change of orientation of images captured by the image sensor 146, and uses this change of orientation to track movements of the barrel of the firearm, even without input from IMU 142.

In addition, smart aiming device 100 includes a microdisplay 160. Microdisplay 160 may comprise, for example, an OLED or LCD screen. The microdisplay 160 may be used to project images or instructions to a user. The projected images or instructions may be either overlaid on a real world view, or may be an independent video display.

The hardware components of smart aiming device 100 may be used during operation of the firearm, in operational mode 110. In particular, various of the hardware components described herein function together as a sight. These include lenses 164, 166 (described in connection with FIG. 2); the processor 150; and the microdisplay 160.

During operation of the firearm, the firearm is convertible between unlocked fire mode 112 and locked fire mode 114. In the unlocked fire mode 112, the firearm fires whenever the trigger is actuated, just as with any standard weapon. In the locked fire mode 114, the processor is configured to time a firing of the firearm following actuation of the trigger based on data collected by sensors 140, so that the firearm fires when processor 150 determines that a locked-on target will be hit in a desired area.

In particular, when operating in the locked fire mode 114, smart aiming device 100 functions as a fire control system. As used in the present disclosure, the term "fire control system" refers to a system that controls when a firearm is discharged. Such a system detects and locks onto a target and tracks the point of aim of the firearm relative to the target. Once the target is acquired and locked onto, the system waits for the firearm to be correctly oriented and positioned (for example, in direction or elevation) before allowing the firearm to discharge. In exemplary embodiments, even when a user holds down the trigger, the weapon will only discharge when the weapon is pointing in the right direction. The processor 150 takes into account not only the relative position of the firearm to the target, but also factors such as speed of the target, distance, slope (ballistics), roll, wind, air pressure, and involuntary movement of the firearm caused by the user. In particular, when the memory has information stored regarding the ballistics parameters of the firearm and the ammunition, the processor 150 may be configured to time the firing based on the type of ballistics.

In exemplary embodiments, the processor 150 detects a specific target zone, or "epsilon," at a central location on the target. For example, if the target is a human, the target zone may be defined as the human's chest. The processor 150 may also include a fire timing module that determines when to allow a bullet discharge, calculated using the relative position between a point of aim of the firearm and the target zone.

The operation of the locked fire mode and fire control system may be substantially similar to that described in U.S. Patent 10,097,764, entitled "Firearm, Aiming System Therefor, Method of Operating the Firearm and Method of Reducing the Probability of Missing a Target," and Israeli patent application No. 281842, filed March 25, 2021, entitled "Telescopic Rifle Sight", each of which is assigned to the assignee of the pending application, and the contents of which are incorporated by reference as if fully set forth herein.

In addition, while in the operational mode 110, the firearm may be operable in a zeroing mode 116, for zeroing a sight that is built in to the smart aiming device 100. As used in the present disclosure, the term "zeroing" refers to a process of aligning a reticle of a sight of a firearm so that the point of aim as indicated by the reticle aligns with the true point of aim of the firearm at a specific range. An exemplary description of the use of a smart aiming device for zeroing a firearm is disclosed in Israeli Patent Application 283793, filed June 7, 2021, entitled "System And Method for Zeroing of Smart Aiming Device," which is assigned to the assignee of the pending application, and the contents of which are incorporated by reference as if fully set forth herein.

As discussed above, the processor 150 includes a module configured to provide feedback 120. Specifically, the module is a computer program product, stored in a non-transitory memory of the processor 150, that, when executed by the processor 150, causes the processor to provide feedback to the user. When providing feedback 120, the smart aiming device 100 uses the same hardware components (e.g., sensors 140, processor 150, and microdisplay 160) that were used in the operational mode 110. However, in the provision of feedback 120, instead of using the sensors only for improved functioning of the firearm, the smart aiming device 100 also analyzes user performance and may instruct users how to improve their marksmanship and operational techniques. The feedback 120 is generated based on the data gathered with the sensors and analyzed with the controller. The feedback 120 may be provided to the user while the smart aiming device 100 is within the operational mode 110, as indicated by arrow 127. While within the operational mode, the user requests the feedback 120 following firing of the firearm. The feedback 120 may include, for example, analysis of how much the actual point of aim deviated from the intended point of aim.

In addition, smart aiming device 100 may be operated in a dedicated training mode 130. The dedicated training mode 130 is used to provide feedback to the user regarding firing techniques during a series of one or more shots. The user enters the dedicated training mode 130 prior to shooting a series of one or more shots, and device 100 automatically provides feedback following completion of the series of shots, without further prompting.

The feedback provided in the dedicated training mode 130 may be more robust than that provided in the operational mode 110. For example, the dedicated training mode 130 may utilize the entire micro-display 160 to provide the feedback, whereas the feedback within the operational mode 110 may include feedback only on a small portion of the micro-display 160. This limited display of the feedback permits simultaneous operation of the firearm. In addition to the different modes of display, the dedicated training mode 130 may be used to provide additional types of feedback, as will be set forth herein.

In both the operational mode and the dedicated training mode 130, the smart aiming device 100 may provide feedback 122 to a user regarding aiming or firing technique. Such feedback may be applied regardless of whether the user was previously using the firearm in an unlocked mode 112 (which generally does not require use of the sensors 140 or processor 150) or in locked fire mode 114. The sensors 140 continuously collect data during operation of the firearm, even in the unlocked fire mode 112. This data is stored in the memory of processor 150 and analyzed in order to provide feedback. Examples of this feedback will be provided further herein. In addition, in both the operational mode and the dedicated training mode 130, the smart aiming device 100 may also provide instruction 124 on the use of the locked fire function 114. For example, the smart aiming device may provide feedback regarding the manner in which the user locked onto a target, or performed a trigger pull while the aiming device 100 was locked onto a target. Examples of this feedback will also be provided further herein.

In addition, in the dedicated training mode 130, the smart aiming device 100 may be used to provide simulations 132. During these simulations, at least one virtual target is projected through microdisplay 160. Examples of such virtual targets include an augmented reality target (i.e., a target overlaid onto a real-world image) and a virtual reality target (i.e., a target within a scene that is entirely projected by the micro-display). These simulations will also be discussed in greater depth further herein.

In addition, in the dedicated training mode 130, the device 100 provides feedback about a series of shots 134. The feedback may include statistics about accuracy or consistency of the timing of the shots throughout the entire firing session.

Advantageously, the same hardware components that are used in the operational mode 110 are used to provide inputs for the feedback, whether in the operational mode or in the dedicated training mode 130. As a result, the user is able to perform the training on the exact same gear installed on the firearm that he or she will use during live fire situations, without changing the look or feel of the firearm.

FIG. 2 schematically depicts hardware components of smart aiming device 100, according to embodiments of the present disclosure. In addition, typical functioning of the hardware components in the operational mode is described.

Smart aiming device 100 is affixed onto firearm 101, and generally aligned with bore 102 of the firearm. Bore 102 is also known as a barrel. Firearm 101 is depicted as a rifle; however, smart aiming device 100 may be applied onto any suitable firearm, such as a pistol, a handgun, a shotgun, a machine gun, a carbine, a revolver, a grenade launcher, or a rocket launcher. Smart aiming device 100 includes housing 162, which may be substantially tubular, and which may be made of any suitable material. Ocular 164 is arranged at the rear of device 100, closer to the stock 103 and grip 104 of the firearm 101. Ocular 164 may be an arrangement of optical lenses. Ocular 164 is also referred to herein as an optical window. Objective lens 166 is arranged at the front of device 100, closer to the barrel 102 of the firearm 101. Objective lens 166 is also referred to herein as a lens assembly. Objective lens 166 is depicted schematically as a single lens; however, objective lens 166 may consist of multiple lenses. Objective lens 166 may be adjustable, to enable focusing and/or zooming of the smart aiming device 100 on targets at different distances from the shooter. Optionally, objective lens 166 includes one or more filters or apertures that may be adjusted by the user, or an adjustable focus control, in order to change the focus of objective lens 166.

Smart aiming device 100 may optionally include additional lens assemblies within housing 102, in addition to ocular 164 and objective 166. These additional lenses may also be used for adjusting the focus of the view of the smart aiming device 100.

Smart aiming device 100 includes a beam splitter 168. Beam splitter 168 may be constructed in any manner that is known to those of skill in the art. For example, beam splitter 168 may be made of two triangular glass prisms which are glued together at their base using polyester, epoxy, or urethane-based adhesives. Incident light reflected off of target 161 enters lens assembly 166 as light beam 163. When light beam 163 reaches beam splitter 168, the light beam 163 is split into light beam 165 and light beam 167.

Light beam 165 reaches image sensor 146. Image sensor 146 is used to capture an image of the target 161. The image sensor 146 is integrated with processor 150. Additional sensors 140 are schematically indicated as near the processor 150, and are integrated with the processor as well.

Microdisplay 160 is also integrated with processor 150. In operational mode, microdisplay 160 may project a digital reticle along light path 169. As used in the present disclosure, the term "digital reticle" refers to any electronically created image of a reticle. The digital reticle may be displayed in any shape or color, such as a circle or cross-hairs. Optionally, the microdisplay 160 is configured to change the shape of the display of the digital reticle according to instructions from the fire control system, for example, depending on whether the firearm 101 is properly aimed at the target zone within the target. Beam splitter 170 refracts at least a portion of the light emitted by the microdisplay toward light path 171. Light path 171 thus includes both light from the lens assembly 166 and light from microdisplay 160, combined into a single view. User 173 views the light through optical window 164. Accordingly, the user 173 sees an image of the digital reticle superimposed over an image of the target 161.

In addition to the digital reticle, the microdisplay 160 may display other types of display items, in the operational mode. For example, microdisplay 160 may display boundaries of a target region that are calculated by a fire control system. These boundaries may be projected to overlay the image of the target that is viewed by the user through ocular 164, similarly to the projection of the reticle. In addition, the microdisplay 160 may project the entire scene that is captured by image sensor 146. This is especially useful at times of low visibility, such as at night, when the naked eye is unable to discern significant details in a landscape. Because the image sensor 146 may be far more sensitive than the human eye, especially in the near-infrared range, the image projected by the microdisplay may be much more useful than the image obtained directly from ocular 176. In order to prevent interference of the view with the image from micro-display, a mechanical shutter (not shown) may be configured to block light path 167, when desired.

The foregoing description of the physical configuration of system 100 with firearm 101 is merely exemplary, and other implementations may be used, without departing from the scope of the present disclosure. For example, instead of a single optical path 171 for light coming from lens assembly 166 and light coming from microdisplay 160, there may be two separate optical paths. In addition, instead of two beam splitters 168, 170, there may be only a single beam splitter.

Referring now to FIG. 3, an exemplary method 200 of providing training is disclosed. Optionally, at step 201, the smart aiming device 100 receives an instruction to enter the dedicated training mode 130. In such cases, the subsequent steps are performed within the dedicated training mode 130. When no instruction to enter the dedicated training mode 130 is provided, the subsequent steps are still performed, with the ability to provide feedback in the operational mode.

At step 202, the smart aiming device collects sensor data while the user is operating the firearm. Optionally, the user may operate the firearm to fire on a real- world target. In addition or in the alternative, the user may operate the firearm on a virtual reality or augmented reality target. The advantages of operation with virtual reality and augmented reality will be described in detail further herein. The firing may also be in live fire or dry fire mode. The sensors provide this data to the controller, including the processor and memory, for analysis and storage.

At step 203, the processor analyzes the sensor data, to thereby determine feedback to provide about the operation of the firearm.

At step 204, the smart aiming device 100 provides feedback to the user regarding one or more techniques of firearm operation, based on the analysis of the data collected by the sensors. This feedback may be provided upon request by the user on data collected in operational mode, or may be provided automatically in the dedicated training mode, as discussed above.

Preferably, the memory has information stored therein regarding a type of ballistics loaded in the firearm, and the feedback is provided specific to the firearm and ballistics that are loaded in the firearm.

The feedback is provided to the user by display of the feedback on the microdisplay, which is viewed by the user through the optical window 164 of the sight. The feedback may be provided in a textual format, in a pictorial format, or in a video format. In the textual format, a message is displayed on the micro-display, such as "Grip hold is too firm" or "Compress trigger more slowly." The text may also include data regarding information such as timing of the trigger actuation, direction of the trigger actuation, or whether the target was successfully hit. In the pictorial format, the feedback may include, for example, an image of the scene captured by the image sensor at the time of the shot, including a mark indicating the target, and another mark indicating the true point of aim during shooting. The video feedback may include a playback of the sequence of aiming at the target and shooting at the target, with commentary or instructions included therein as textual and / or audio components. The foregoing are merely examples, and the specific mode of feedback may be tailored to the type of instruction that is being provided, as is understood to those of skill in the art.

In preferred embodiments of performance of method 200, the sensors whose data is used to determine the feedback are the same sensors that are used in operation of the smart aiming device 100. Advantageously, in such embodiments, no additional sensors are required for provision of this feedback, enabling the smart aiming device 100 to be manufactured compactly, and also enabling the training to be performed on a firearm that is identical in all respects to those used in live fire scenarios.

The following is a non-exhaustive list of categories of feedback that may be provided by the smart aiming device 100:

Aiming Feedback. The feedback may include feedback on how far an actual point of aim of the firearm was from the desired point of aim, during actuation of the trigger. For example, the fire control system may lock onto a target, even when the fire control system is not used for controlling the firing, but rather for the sole purpose of identifying the user's intended point of aim. The system may recognize the desired aim point, on known or unknown targets, and optionally present this aim point to the user. In addition to automatically determining the intended aim point, during a play-back of a video of a firing event, the user may pinpoint the desired aim point, using system controls. If user's point of aim is off, the system, based on all available data, may give one more possible causes to why such a deviation happened, such as improper weapon hold, shooting position, etc.

Trigger Actuation. The feedback may include feedback regarding technique of trigger actuation. Proper trigger pull is a very important technique to master in marksmanship. If not done properly, for example if done too fast or in a jerking motion, the trigger pull may cause undesired movements of the barrel prior to the bullet leaving the barrel. As a result, the bullet will have an incorrect point of impact. The trigger sensor 144 enables the entire system to evaluate whether the trigger pull was executed properly. Information about the trigger pull may also be combined with information about the deviation of the point of aim, in order to evaluate whether the deviation of the point of aim was caused by the incorrect trigger pull.

Barrel Roll. Some users, especially inexperienced ones, unintentionally angle their weapons. This angling introduces an error known as "cant error" which affects the eventual point of impact, despite proper aimpoint. The feedback regarding angling of the weapons is derived from data obtained by the IMU sensor 142, and specifically an accelerometer.

FIGS. 4A and 4B illustrate exemplary displays of the feedback for the abovedescribed techniques. In FIG. 4A, a limited feedback is provided in a non-obtrusive manner on the display of the target region, while the firearm continues to operate in operational mode. Display 300 includes a display of circular target 304, with dot-style reticle 306 at a center thereof. A locked-on ellipse 310 is displayed over a portion of a human target 308. Other features of the scene surrounding the target region, such as trees 316, are also visible, just as in standard operational mode. There are two additions in the view of display 300 compared to a display in the standard operational mode. First, the location where the previous shot hit relative to the dotstyle reticle is marked with "x" 312. Any other small graphical marking may similarly be overlaid near the target region. Second, a textual indicator 314 at the perimeter of the display 300 provides some textual feedback to the user, regarding how to improve on the previous shot. An icon may also be displayed in addition to or instead of the text, with the presence of the icon indicating that the previous shot included one or more of the above-described errors.

In the view of display 400 in FIG. 4B, the feedback is provided in the dedicated training mode. The entire display, as it appeared at the time of the shot, is reproduced in display 400. This display may optionally be made smaller and cropped relative to the display 300, to permit more room on the screen for feedback. The display of the circular target 404, dot-style reticle 406, human target 408, ellipse 410, and trees 416 are all reproductions of the view from when the firearm was shot. In the illustrated example, the user rolled the barrel at a high roll angle when firing the shot; this barrel roll is indicated through a tilting of display 400. The display 400 further includes one or more icons indicating particular errors associated with the previous firing. For example, icon 418 is displayed to indicate the presence of barrel roll, and icon 420 is displayed to indicate presence of trigger jerk during trigger depression. Of course, the feedback may also include textual feedback, and may also address other matters such as aiming error, trigger speed, and wind.

Consistency of Marksmanship Technique. In addition to providing feedback about an individual actuation of the trigger, the smart aiming device 100 may also provide feedback about consistency of marksmanship technique during a series of shots, in the dedicated training mode. The feedback may relate to deviation in point of aim, trigger actuation, barrel roll, and any combination thereof, as well as any other parameters discussed below.

Training Sessions. Relatedly, the smart aiming device 100 may provide an infrastructure for building training sessions, particularly in the dedicated training mode 130. For example, the device 100 may start a training sequence that requires the trainee to shoot ten rounds on a target. The system may provide feedback on parameters such as the time taken from an initial buzzer to firing of the first round, time taken to complete all rounds, and the mean time taken for each round.

Exemplary feedback for the training sessions is illustrated in FIGS. 5 and 6. FIG. 5 is a graphic user interface 500 for feedback provided following a series of shots at a target 501. The target 501 is depicted with a series of concentric circles 503, with each circle assigned a point value from 6-10, depending on how close the circle 503 is to a center of the target 501. Each point of impact of a bullet is marked by a crosshairs 502. The user interface 500 further includes a cumulative score 504 for the entire shooting session, which is calculated based on the point values of each of the points of impact. The user interface may further include a time indicator 505, which indicates the average time spent in between two consecutive shots.

FIG. 6 illustrates another embodiment of graphic user interface 600. Interface 600, like interface 500, includes a virtual hitmap of a target 601. In this case, the target 601 is shaped like a human upper body. Each point of impact is depicted as an "X" 602. A counter 603 lists the number of hits on target as compared to the total number of shots taken.

The user interfaces 500, 600 thus simulate the physical hitmaps that a user might see if he fired a series of shots at a target range, and subsequently walked up to the target. Advantageously, this simulation is provided without any need for additional equipment, and without even requiring the user to travel to the firing range.

Compensation for Ballistics or for Movement of Target. As is known to those of skill in the art, different ballistics within a firearm have different trajectories upon firing. Likewise, when the target is moving during the firing, it is necessary to compensate the aim of the bullet accordingly. Compensating for ballistics or target movement is, in prior art systems, a specialized skill that often is not taught to the regular soldier. However, the fire control system 120 of smart aiming device 100 includes a locked fire mode, which is able to lock onto and track the target even when the target moves. Likewise, the fire control system is configured to take into account the effect of ballistics when calculating the trajectory of the bullet. As a result, these tracking features may also be used, particularly in the dedicated training mode 130, to train the operator to properly compensate for ballistics and leading the target. Notably, this training may be used even with respect to operation of the firearm in the unlocked fire mode. Likewise, these tracking features may be provided even when the user is using a simple red dot or mechanical reticle, without relying on a reticle projected by microdisplay 160.

Wind Correction. In the same way that the smart aiming device 100 may calculate expected deviations based on ballistics or a moving target, the smart aiming device 100 may also calculate expected deviations based on wind direction and speed. The device 100 may communicate this deviation to the user, so that the user may learn how he or she should have adjusted his or her aim due to the wind.

Operation of Locked Fire Mode. In certain embodiments, the feedback includes feedback on how to operate the firearm in the locked fire mode. In general, the locked fire mode 120 helps prevent most user mistakes when operating a firearm. However, like any system, it is important for the user to train how to use the locked firing mode properly to further enhance its effectiveness.

For example, the feedback may include feedback on the process of locking on to a target, even before attempting to fire a shot. Likewise, a user may unintentionally perform an unneeded action that causes the aiming device to release the lock on the target and seek another target. Sometimes, a user may intentionally desire to move the lock from one target to another target. In addition, the user may wish to transition from the locked fire mode to the unlocked fire mode, and vice versa. In all these instances, the system may provide the user with standard sequences of training and so to provide feedback to the user, via instructions displayed in the microdisplay.

Dry Fire Mode. The training of the training mode is effective regardless of whether the firearm was operated in a live fire mode or a dry fire mode. As used in the present disclosure, a dry fire mode is a mode in which the user simulates firing a bullet by actuating the trigger, but no bullet is fired. The sensors are able to accurately determine the time of either a live shot, or a dry shot, as well as the aiming direction of the weapon during that time. As a result, the feedback may be substantially identical.

Reactive Target. The smart aiming device may be used as part of a system including reactive targets. As used in the present disclosure, a reactive target has one or more elements configured to interact with projectiles contacting the targets. When the projectiles contact the target, one or more electrical or mechanical properties of the reactive elements may change. This, in turn, causes a reaction in the reactive target. For example, the reactive target may drop to the ground, or may emit a visual signal. The system may detect this reaction, for example, with the image sensor.

Optionally, the reactive target may be a "communicative target" that includes a memory and communications circuitry, for storing information about when and where it was hit, and for communicating this information to a central processor. As used in the present disclosure, the term "communicative target" refers to a target that provides such information regarding when and where it was hit to a centralized system integrated with smart aiming device 100, such as processor 150. The device analyzes this data and may give live feedback about a hit to a user. The device may also measure features such as time to first hit, and range to the target (based on a time-of-flight calculation).

Returning now to the subject of virtual reality and augmented reality, it is easier to train on some target, even an imaginary one, than without any target. While basics of trigger work may be improved without a target, perfection of aiming typically requires a target. Typically, obtaining a target may be done only in a standalone environment such as a shooting range or a simulator. These environments are expensive to provide.

In order to enable practice of aiming without a physical target and without a simulator, smart aiming device 100 may also be used in a virtual reality mode. The processor 150 presents a virtual reality display including one or more virtual reality targets via the micro-display 160. The virtual reality display is a video or still image that represents a training scenario. This video may be saved in system memory, downloaded from an external device, or even may be generated during use of the smart aiming device 100. When the user moves the smart aiming device 100, he or she sees different parts of the virtual reality display, just like he or she would have seen in a real world display. All of the marksmanship training and locked fire mode feedback discussed above may be provided in the virtual reality mode, except that typically, for safety purposes, firing while in the virtual reality mode is enabled only in a dry fire mode or while using blanks ammunition.

Smart aiming device 100 may also be operated in an augmented reality mode. The processor 150 presents one or more augmented reality targets superimposed over a view obtained from the optical lens 164. This superimposition is similar to the superimposition of a digital reticle over the view from the optical lens, described above in connection with FIG. 3. The device 100 may superimpose images of a single target, multiple targets, or additional scene elements. In certain embodiments, the virtual reality or augmented reality targets are moving targets or airborne targets such as drones. The user simulates firing on the one or more augmented reality targets. As in the virtual reality mode, nearly all of the feedback that is available following regular operation is available following operation in the augmented reality mode. As a safety precaution, when working in the augmented reality mode, the smart aiming device 100 may alert the user if it detects a human in the view of optical lens 164. This is especially important if the augmented reality training is performed with live ammunition.