SYSTEM, DATA CARRIER

The invention relates to a surveillance method, a drone, a mobile device, a surveillance system and a data carrier.

Here and there, people feel unsafe or uncomfortable in certain situations. For a situation at home, many surveillance methods and systems are known for detecting and deterring burglars from entering a house or home and/or for producing alarms upon such incidents.

The situation is far less developed when thinking about mobile contexts, particularly when foot-walks in remote or dim areas must be made. Communities provide lighting to such paths for deterring malicious people, and occasionally video monitoring is provided for having some kind of deterring effect and documentation of what is going on. But there are many

locations where no monitoring at all is made. Providing lighting alone is in many cases not felt deterring by

malicious people.

Similar concerns exist regarding vehicles, be they trucks or passenger cars or ships. They are subject to hijacking or theft even while in use. So, generally speaking, mobile targets (persons or vehicles) or their operators or owners feel a lack of surveillance and deterring measures. It is the object of the invention to provide technology for rendering mobile target surveillance with documenting and possibly deterring effect.

This object is accomplished by the features of the independent claims. Dependent claims are directed on preferred embodiments of the invention.

A surveillance method has the steps of launching, by a mobile target such as a person or a vehicle or a ship in a scenery, a drone equipped with a camera that has or establishes a

wireless tracking link to the mobile target, tracking the target in the scenery by the drone in accordance with

information obtained from the tracking link, taking one or more images of the scenery by the drones camera and generating corresponding image data, and transmitting the image data away from the drone, preferably through a wireless link, towards a first predetermined destination.

Thus, a drone tracks a mobile target that wishes this be done and sends image data directly or indirectly to a predetermined destination through at least one wireless link or channel. This tracking, imaging and image sending may start well before any critical situation develops and may be made such that it is well recognizable that it is made so that it has a

deterring effect to malicious people because they understand that they are detected, photographed and documented before being able to stop this.

The tracking link can be established before launching or while launching or after launching the drone or when switching the drone on. The drone may direct the cameras field of view towards the mobile target.

The drone may have a wireless link to a mobile electronic communication device carried by the mobile target, such as a persons mobile phone or a built-in communication unit of a vehicle or ship or vessel, and the image data are sent automatically upon generation from the drone to the mobile device and are forwarded from the device preferably

automatically upon receipt to a first predetermined

destination.

Thus, a drone can use a built-in communication unit of a vehicle or a persons mobile device (a smartphone, PDA, mobile phone, tablet, ...) as a relay for forwarding the taken images. Then, the drone does not need access to commercial networks, but has access only to the communication unit or mobile device of the mobile target. But, vice versa, if the drone (or components thereof) have access to commercial

telecommunication networks, the drone may send taken images directly thereto, i. e. to a base station of a communication network.

The drone and/or a linked wireless device of the mobile target may send audible and/or visual signals into the scenery, wherein the audible signals may comprise an alarm sound and/or prerecorded voice messages, preferably in plural languages and sequentially sent out, preferably upon incidence of a first condition, and wherein the visual signals may include the display of image date received from the drone on a display provided at the mobile target, such as a target persons mobile device. The first condition may for example be that this signaling is separately commanded, i.e. switched on.

The drone may be remote controllable from the site of the mobile target, e.g. by a target person or a driver/operator of the vehicle, and/or by a remote person, wherein particularly the field of view of the camera and/or the flight path may be remote-controllable .

A start message may be sent from the system (drone or linked wireless device) to a second predetermined destination when the system is activated or the drone is launched. This may be made through one or two or more channels in parallel.

A drone has a flying mechanism for flying the drone, a

tracking mechanism for tracking, while flying, a mobile target (person, vehicle, ship) on the ground based on information from said mobile target and interacting with the flying mechanism, and a camera for taking one or more images of a scenery on the ground and generating related image data, wherein the drone is adapted to establish a first

communication link towards at least a first predetermined destination, and has means for sending said image data

automatically upon generation towards said first predetermined destination .

Thus, following the desire of the mobile target (or its operating person) in the scenery, the drone has a tracking link to said mobile target and uses it for tracking the target, i.e. following the target in the air as the target passes on on the ground. While tracking the mobile target, the drone takes images/videos and sends them to a predetermined person or destination.

The drone may have a directing mechanism for directing the cameras field of view in accordance with information from the tracking mechanism and/or other information. The drone may be adapted to establish a second communication link towards at least a second predetermined destination, and having means for sending at least one message towards said second destination.

The drone may have signaling means for releasing visible and/or audible signals. It may be built as a helicopter or a quadrocopter or as a fixed wing device.

A mobile electronic communication device may be built-in into a vehicle or may be portable and has first communication means adapted to establish a first wireless link to a drone and adapted for receiving image data from the drone, and second communication means adapted to establish a second wireless link to a predetermined final destination for automatically forwarding the image data received from the drone to said final destination. It is a goal of the invention to send surveillance data

(images, videos, audio) to a destination remote from the scenery so that existence of the data cannot be controlled in the scenery. In one embodiment, the drone can send its image data/video data/audio data to a mobile communication device carried by the mobile target (person, vehicle, ...), which in turn, and preferably automatically, forwards the received data to a predetermined final destination. The predetermined destination may be some kind of private destination (e.g. a family member) or a public destination (police) or may be the destination of a commercial service provider for ancillary services there.

The mobile device may have means for sending out data acquired by the mobile device itself, preferably to the first or final destination, preferably combined with or integrated into the data received from the drone before being forwarded. Such data may be or comprise location data and/or ID data and/or time data and/or audio data and/or image/video data.

The mobile device may be adapted to establish a wireless link to the drone allowing the drone to track the mobile device.

A documentation system has a drone and a linked mobile device as described above. A data carrier holds software that, when running on a mobile device, renders a mobile electronic communication device as described above.

Embodiments of the invention are described in the following with reference to the attached drawings, in which Figure 1 is a schematic representation of a surveillance system,

Figure 2 is a schematic representation of a drone used in the surveillance system,

Figure 3 is a schematic representation of control logics used in the drone,

Figure 4 is a schematic representation of a portable mobile communication device used in the surveillance system,

Figure 5 is a plan view of explaining an inventory

monitoring system and method,

Figure 6 is a schematic representation of a drone used for such a method, and

Figure 7 is a schematic representation of control logics in such a drone .

In the following specification, features shall be deemed combinable with each other also if such a combination is not expressly mentioned, as far as the combination is not excluded by technical reasons. Descriptions of methods and method steps shall be understood also as description of apparatuses and apparatus components implementing the respective method

(steps), and vice versa. The same applies to software aspects in relation to method aspects and apparatus aspects. In Figure 1, numeral 1 generally indicates the scenery. For example, one many assume that it is a footpath between a train station and a private home. 2 symbolizes a target person that wishes to use the footpath, and 3 symbolizes another person primarily unrelated to the target person 2. But likewise, 2 may be a vehicle driving along a road, or a ship. In the following, reference is made predominantly to a target person 2. But vehicles and ships as the subject of surveillance shall be meant correspondingly. As far as activities of a person are addressed, this person may be the target person 2 or the operator or driver or responsible person of a target vehicle or ship.

Generally, in the scenery 1 the target person 2 or operator of the target vehicle/ship may feel unsafe, for example in view of recognizable other persons 3, or because of other knowledge or assumptions. When the target person 2 nevertheless wants to enter the scenery 1, it may decide to use a surveillance method and system so that the situation is less anonymous and is thus felt to be safer.

The surveillance system comprises a drone 11 that is launched by the target person 2 when it deems this advisable. Launching can, for example, be made when a situation starts to appear unsafe and may be made well before a particular threat is recognized. The target person 2 then passes the scenery 1, for example follows a footpath towards the desired target, followed/tracked by the drone in the air.

The drone 11 has a tracking link 16 towards the target person 2 and tracks the person, i.e. substantially follows it after being launched. While doing this, the drone uses a built-in camera and takes images of the scenery, particularly of the target person and/or of other scenery portions, possibly selected or adjusted according to certain criteria.

The images taken by the camera are sent out by the drone after they have been taken, preferably automatically, such as periodically, and preferably soon after they were taken (less than 100 or 50 or 20 or 10 seconds or 5 seconds or 2 seconds or 1 second after they were taken or immediately after they were taken) . The image data are sent to a predetermined destination which may be identified by a technical address used in a technical communication channel, preferably formed in accordance with a certain standard.

The first destination may be a predefined destination

identified by a predefined technical address. It may have been input and stored in the drone or in the mobile device by the target person upon system initiation well before being used. Depending on the configuration of the drone, the channel used for sending out the image data may be a channel that has to be paid for when being used or may be a free channel. The channel to be paid may be a channel as used by mobile phones

identified and defined by a SIM structure (subscriber identity module structure) . This is symbolized by arrow 18 in Figure 1, going directly to a stationary node or base station 14

connected through wires to grounded structures 15. This channel 18 may involve AAA activities (authentication,

authorization, accounting) . It may be used for serving higher level applications such as email, WhatsApp, streaming

technology, FTP, HTTP or the like

But instead of such a channel 18 or in addition thereto, free channels 17 may be used which do, at least, not require accounting. In Figure 1, it is symbolized as stretching between the drone (or a component thereof) and a mobile portable communication device 12 ("mobile device" in the following) held by the target person 2. Such a channel may be a WiFi/Wlan channel or a Bluetooth channel and may use higher level applications such as email, WhatsApp, streaming

technology, FTP, HTTP or the like.

In this case (i.e. when the drone 11 sends the image data through channel 17 to the mobile device 12 of the target person 2), it is preferred that the mobile device 12, upon receipt of such data, is configured to immediately forward them away from the scenery. This is symbolized by arrow 19 in Figure 1. It may be the regular attachment of a mobile device 12 to a base station 14 under the usual AAA requirements.

In this relayed approach (11 → 17 → 12 → 19 → 14 → ...) the drone has a predetermined channel 17 towards the mobile device 12 served in accordance with first technical

requirements/standards, and the mobile node has or establishes another channel 19 towards remote technical structures

according to second technical requirements/standard that may be different from the first one. The technical address used by the drone 11 for accessing the mobile device 12 for sending data to the mobile device is usually different from the technical address used by the mobile device 12 for forwarding the data through base station 14 to their final destination. But both of them are preferably predefined and are thus used automatically for getting the image or video stream from drone 11 via mobile node 12 to base station 14 and beyond. From the drones point of view, the first predetermined destination may then be the mobile device 12. But the final destination of the data is not the mobile device, but rather a remote first predetermined final destination defined by the address used by the mobile phone for forwarding the image data received from the drone 11. The final destination may be a technical target such as a web server receiving the image data, dentified, for example, by a (sub) domain or an IP address or of a related service, possibly sent through FTP, or an email address or a Whattsapp terminal, or any other kind of technical address of technical structures.

The drone 11 may have further a tracking link 16 towards the target person 2. It may be an active link in that components of the drone communicate unidirectional (towards drone or away from drone) or bidirectional with a component 13 carried for the purpose of tracking by the target person 2. In another, not shown embodiment, this active tracking channel 16 can be between drone 11 and the portable device 12 of the target person so that a separate device 13 is superfluous. But the tracking link may also be passive in the sense that it uses, for example, image recognition in the drone for tracking the target person imaged through the camera of the drone, and so following it.

The system adjustments may be such that after launch of the drone by the target person 2, the required communication channels are established. Likewise, tracking is commenced. Then, the drone periodically or continuously takes images through its camera and sends them away from the scenery either directly to base station 18 or relayed by the mobile device 12 of the target person 2, as explained earlier. The purpose is to get surveillance data away from the scenery before somebody else intercepts and prevents this. For tracking, the drone may fly orientation path portions, such as loops, e.g. for detecting relative-direction-dependent signal strength of signals exchanged on the tracking link, for finding the direction towards the target. Another goal is that other persons are aware of the

surveillance and thus refrain from unwanted activities. For achieving this, the system can be designed such that it sends recognizable signals into the scenery, either from the drone 11 or from another system component, such as the mobile device 12 of the target person 2 or operator of a target vehicle. The signals can be audible signals and/or visual signals. They may be released upon incidence of a certain first predetermined condition, for example when commanded by the target person 2 or operator of a target vehicle through the mobile device 12.

Figure 2 shows a schematic representation of a drone used in the described method and system. It is depicted as a "copter" with rotary wings. It may be a quadrocopter or may have a higher or lower number of rotors. But it can also be a fixed wing drone similar to a regular plane. The overall size may be as small as less than 20 cm or less than 15 cm or less than 10 cm overall extension.

20a symbolizes a housing or frame of the drone. 20b symbolizes the drive mechanism, shown as blades of a rotor. 20c

symbolizes the drive which may be an electric motor or a combustion engine. 20d is a power supply and may, thus, be a battery or a fuel tank.

20e is a camera directed with its optics (lens, ...)

substantially downward in a standard orientation of the drone. The camera's field of view may, however, be adjustable through means 26 for controlling the field of view. These means 26 may control the posture of the camera 20e relative to the

frame/housing 20a of the drone and/or may control the overall posture of the drone in space so that the camera' s field of view is controlled together with it, and/or thy may control the angular opening width of the cameras field of view.

20g symbolizes a tracking module for tracking a target to be followed on the ground. The target may be the target person 2 in Figure 1 or a target vehicle or a device 12, 13 carried by it. There may be an active connection (symbolized by 16) between the drone and the target person (a device ported by the target person 2) . The tracking module 20g can in this case be some kind of radio connection which may be unidirectional

(for example from a hand held device 13 of the target person 2 or from the mobile device 12 of the target person 2) to the drone 11, particularly through its interface 20g or 21 to the tracking module 20g, or it may be bidirectional amongst these components. But likewise, tracking may be passive in the sense that the tracking module 20g uses the camera 20e and performs image recognition on images acquired by the camera for finding the target person 2 and tracking it.

Tracking may be made, controlled, determined and commanded according to known technologies. It may happen autonomously controlled without requiring continuous intervention from remote. It involves that the drone in the air follows the target person 2 physically as the target person 2 moves through the scenery 1. The drone may position itself at a certain altitude above ground. The altitude may be predefined or may be remote controlled by the target person 2 or may be variable according to certain strategies or procedures. The altitude should be such that the drone is out of the physical reach of persons on the ground, such as at least 2 or 5 meters above ground.

For tracking/following the target person 2, various patterns may be selectable and adjustable for the drone, such as

"follow" (the drone being behind the target person 2 in movement direction) , "proceed" (the drone being ahead of the target person in moving direction) , "hover" (above the target person) , "left" or "right" (left or right of the target person) or "circle" (the drone circling around the target person) . Such settings or selections may be made by the user in accordance with circumstances in the scenery. They can be set into the drone for enabling related autonomous control there .

20h symbolizes general sensors of the drone, used for flying the drone. They may be proximity sensors directed downward and/or may comprise speed sensors and/or translational and/or rotational acceleration sensors and/or signal strength sensors and/or a compass. Such sensor signals may be used for general flight control, collision avoidance, altitude control, position control, launch and landing control and the like.

21 and 22 symbolize wireless interfaces of preferably

different quality. One of them may, for example, be a

WiFi/WLAN interface and/or a Bluetooth interface. Another may be a regular wireless interface as used by mobile phones for connecting to a base station 14. It is pointed out that interfaces 20g, 21 and 22 need not be present all together. They may be provided as required by the various communication requirements .

If a WLAN/Wi-Fi interface 21 is present, it may form a channel 17 towards a mobile device, particularly the mobile device 12 of the target person 2. An interface 22 may form a channel 18 towards a base station 14. 20f symbolizes a general controller that receives signals and sends out signals. Many of them may be digital signals. The controller 20f may be or comprise a CPU, a memory, an

interconnecting bus and other such components. It may be or comprise a microcomputer. Through various tasks, the

controller 20f may implement the various components and modules required, on the one hand side, for the regular flight operation, and, on the other hand side for the particular steps of the described surveillance method including imaging and tracking.

The regular flying tasks may include the general flying control (altitude, speed, posture, drive control, collision avoidance and the like) . It may also control the overlaying tracking control for implementing the tracking as described earlier .

It may comprise a remote controlling interface 28 for allowing remote control of the drone from outside. It may receive control input and may output image/video data towards the remote controlling site.

And it comprises components 27 for implementing the

particulars of the mentioned surveillance. These components 27 are symbolically depicted in Figure 3. It may comprise an imaging management and forwarding function or module 31. It is pointed out in this context that "image" may generally mean still images or may also mean video images. The image forwarding may comprise steps of or means for receiving image/video data from the camera 20e, possibly modifying and/or augmenting and/or formatting them, and forwarding them to a first predetermined destination by using one of the provided interfaces 21/17 or 22/18, depending on circumstances, system layout and settings. The image

forwarding function or module 31 forwards images in desired or preset quality, data volume, resolution, color code and frequency as described above. Functionality thereof may be remote controllable, for example through the mobile device 12 of the target person 2.

The component 27 may also have a remote control management interface or functionality or module 33. It may be responsive to a mobile device 12 of the target person 2 communicating with the drone through channel 17 and interface 21. But it may also be open to "very" remote control from another person, possibly through interface 22 (if provided) and channel 18. Setting may, for example, be such that drone control is handed over to a remote person when it is found in the drone that such a person is technically accessible and other conditions are met, e.g. release by the target person. Then, not only image forwarding happens autonomously, but also drone control is made independent of the target person 2.

The drone may have optical signaling means 23 and/or

acoustical signaling means 24. It may have a related local messaging control step or module 32. Optical signaling may, for example, include some kind of flashing light and/or some kind of scenery illumination by powerful light. Acoustical signaling may include the reproduction of some kind of alarm sounds, for example similar to that of police vehicles or the like. Likewise, it may include playing voice messages through one or more speakers included in the acoustical signaling means 24 of the drone. Such kind of messaging may include playing voice messages successively in different languages, such as "Surveillance is running (pause) Oberwachung lauft (pause) Purpose of this aspect of the invention is to promote awareness for other persons 3 in the scenery that they are already monitored. Component 27 may also have camera control means/steps for controlling the field of view of the camera. As said earlier, this may use a dedicated mechanism 26 for setting the cameras orientation relative to the drone's frame 20a, for example by setting independently azimuth and elevation or at least one of them, for example elevation (angle of the line of side in a vertical plain) . The azimuth (angle in plain view) may be set by the drones overall attitude in space. The field of view control 34 may be such that a constant field of view is adjusted or that it constantly changes for grasping the scenery as complete as possible, or it may follow more complex routines, such as based on image recognition.

It is pointed out insofar that it needs not necessarily be the primarily desire to direct the field of view of the camera 20e towards the target person 2. Rather, other persons 3 in the scenery may be, at least intermittently, the subject of interest, and there may be provided appropriate means for temporarily or continuously directing the camera' s field of view towards these other persons 3, for example image

recognition and linking this with the camera orientation module .

Control part 27 may also have remote messaging module or procedures 36 for sending data or image data or messages to remote addresses such as a second predetermined destination, either relayed through the mobile device 12 or directly to a base station 14. This may be provided in addition or in parallel to the image forwarding mechanism 31 and may include information data such as time stamp, possibly geolocation information (from GPS and/or other localization technology), ID data or the like. Messaging format may be same or similar as that of component 31 email or sms or ftp or Whattsapp or the like.

The drone 11 may have release means 25 for releasing a

substance into the scenery, for example pepper spray. The control portion 27 may have a related release procedure or module 35. It may operate remote controlled for example in accordance with control input from the target person' s mobile device 12 or through the remote control module 33 from another remote controller. Settings may be such that the substance is released in a certain preferred direction. This direction may be the field of view of the camera. The release mechanism 25 may thus be coupled with the camera 20e, if it is displaceable relative to the drone frame 20a.

Further, an interface 37 to other distributed surveillance or monitoring structures may be provided. For examples, there exist distributed monitoring structures built by apps running on mobile phones. These apps search for same apps on nearby other phones and maintain some kind of connection. If

triggered by the holder of the mobile phone, an alarm is sent to these other phones so that the holder can appropriately react. The interface 37 may have procedures to access such other surveillance systems or monitoring structures,

preferably upon incidence of a second predetermined condition, such as a related input command from the mobile device 12 of the target person 2.

The controller 20f of the drone may have general managing functions such as acquiring and recording preset data, storing and retrieving such data and image data, making settings particularly for data settings and the like. This management input may be made at the drone 11 itself through appropriate means or may be made through a link to another device, for example the mobile device 12 of the target person 2.

Further, the drone may have a start recognition procedure or module 38 for recognizing commencement of operation. It may operate autonomously and may recognize through appropriate sensors a launch situation. Such a launch situation may, for example, be a start by simply throwing the drone by the target person up into the air. This can be recognized by an

acceleration sensor and signal evaluation for finding certain signal patterns (first strong acceleration for less than 1 second while throwing, but thereafter very low or practically 0 acceleration) . Upon recognizing this, the drone may start flight operation, i.e. start the drive mechanism, and may autonomously make initializing activities such as establishing the tracking link and/or the communication link and the starts with image taking and image forwarding. The purpose of this feature is to allow a quick start without cumbersome and time consuming operations of the user.

The drone may have a function or module 39 for handling loss of the tracking link. The functionality may involve that the drone flies away from the scenery for a predetermined time or a predetermined distance when loss of the tracking link is detected. Goal is again to get the drone out of the reach of other persons 3. The same may happen when remaining power drops below a certain threshold and the user does not prevent the drone from leaving the scenery.

The tracking link to the tracked target may be established before launching the drone or after launching the drone or when switching it on. The same may apply to the communication link .

In its flying control, the tracking drone may have a function or module for changing its position and particularly distance from the tracked person autonomously, i.e. decreasing it and increasing it again, possibly repeatedly or periodically. The goal is to obtain best possible images of details, on the hand side, and good overview images, on the other hand side. Figure 4 shows schematically a mobile device 12 as an example of a portable electronic communication device held by a target person 2. It may be a tablet or a PDA or a smartphone or regular mobile phone or some other apparatus with wireless communication capabilities. It may comprise a short range and middle range communication interface 40b for wireless

communication. This may be Bluetooth or Wi-Fi/WLAN or the like. Further, it comprises an interface 40c for regular wireless connection to a base station 14. It further comprises a display 40a and a general controller 40d that controls, amongst others, the usual functionalities of a mobile device.

The controller 40d may have an initialization function or module 41 involving the establishment of the required wireless links, particularly a tracking link 16 as far as actively acquired from the mobile device 12, a wireless link 17 to the drone 11 for data exchange (receiving image data) and possibly for remote controlling the drone, and a data link 19 to a base station 14. The initialization module 41 may establish these links as required autonomously without requiring user

interaction. The start routine may be triggered by a signal from the drone, for example upon launching the drone as described above. Thus, the entire system may be made such that it is launched and put into operation by simply launching the drone . The mobile device 12 may have an image forwarding function or module 42. It may receive image/video data from drone 11 through link 17 and interface 40b and may automatically forward them through interface 40c and link 19 to base station 14 and beyond, from where it goes to a predetermined

destination defined earlier by settings in the mobile device 12. Image forwarding can be made under known procedures such as FTP, HTTP, as email attachment or as stream or the like. Image forwarding may be made to more than one predetermined address. For example, it may be made to a private address and can, in addition, be made to a related service provider or to some kind of official prosecution authorities.

The mobile phone may also comprise a module 43 and/or function for sending out a start message to a second predetermined destination, informing about the fact that the system has been put into operation. Such a message may be sent out through plural channels in parallel, for example WhatsApp, email, SMS and the like. The goal of this feature is to inform somebody else about start of the system so that somebody else is alerted on the situation. The system layout may further be such that this alerted person is authorized to take remote control of the drone through its remote control module 33.

Image forwarding may be made continuously over time and may thus involve the sending out of the plural images in a certain frequency, for example one every second or of higher or lower frequency . The mobile phone may further have an auto display function or module 44 that displays image data received from the drone on the mobile phone's display 40a. Purpose of this is to enable the target person to demonstrate to other persons 3 that they are monitored so that they refrain from unwanted activities. The auto display functionality may include also displaying other information than the images taken by the drone, for example printed text messages in various languages or the like.

The mobile device 12 may further have a remote control function or module 45 for remote controlling the drone 11. This may involve the regular functions such as flight

settings, tracking settings and the like, but may also involve the control of specific features, for example setting the camera' s field of view manually, for example onto a person 3 in the scenery. System layout may be such that either the camera's field of view and/or entire drone tracking is switched over to such a person 3. The tracking link 16 may then be made through image recognition on the images taken by camera 20e of drone 11.

The mobile device 12 may further have a module 46 or function for sending out data generated by the mobile phone itself.

This may be made through separate messaging or by augmenting the data received from the drone before forwarding them. Such data may include time stamp data and/or GPS (geo position) data, possibly voice recordings, identity data and the like. Insofar, the initialization module 41 may have a functionality of starting self-localization of the mobile device 12.

A part of the described method and system may be a start mechanism for the described system components, i. e. the drone and possible other components. The start mechanism may be adapted to accomplish physical and administrative functions, such as

1. switching on the drone and possibly other components such as a mobile device, if not already "on",

2. physically launching the drone,

3. loading required settings in the drone and/or in an

associated mobile device,

4. establishing/opening the various channels for image

sending and forwarding and possibly other messages at the drone and/or the associated mobile device,

5. starting the target tracking function,

6. starting the imaging function with the drones camera,

7. starting the image sending function,

8. sending out a start message,

9. starting the sending of signals from the drone into the scenery . The start mechanism may be designed such that a single

operation or operation sequence at a single device is

sufficient for accomplishing many or all of the above items. It may be an operation sequence on a touch panel or operating hardware buttons at an appropriate location, for example more or less integrated into, or vice versa separate from, other switch mechanisms. It may be made on the drone or on an associated mobile communication device or on a launch

structure (see below) . Or it may come from the mentioned start recognition procedure or module 38 for recognizing

commencement of operation.

The start mechanism may comprise a drone launch structure. If carried by a target vehicle or, for example, in a back-pack of a target person, it may comprise an upward directable

guidance, for example a rod or rails or a tube-like structure holding the drone and open or easily openable at or on its upward end. The mechanism may comprise a compression spring. The drone may be spring-biased from underneath so that upon start it is mechanically released and ejected in guidance direction away from or out of the launch structure.

The launch structure may be a portable or hand-held device. A target person may carry it in a back-pack. A vehicle may mount it somewhere outside of the passenger compartment. But likewise, it may be designed such that it lies ready to use somewhere near the operator and can, for launch, manually be held out of the window. The drone may be owned by the individual desiring the

surveillance. But likewise, it may be owned by a related service provider (commercial, community, gated community, ...) which provides for some kind of temporary rental one or more drones parked at appropriate locations "ready for use" for example at a docking station for battery charging. The drone may then be called by the individual desiring the surveillance through appropriate measures. The call may be made, e.g., via a web site of a related service provider or via a pre-agreed code sent through a predefined channel such as SMS, Whatsapp or the like. The service provider may then start the drone, which flies then to the calling mobile target either along already possible tracking or based on received information such as position of the base station that received the call. Upon arrival at the calling mobile target, the drone and overall system operates as described. The various

initializations may then be made. Required data may then be input or may already be available from an earlier registration at the service provider who stored these data in the drone or sends them to the drone. Upon termination, the drone may fly back to its docking station. This kind of service may be subject to some kind of debiting. Disarming the system may be made commanded by the target person or operator of the target vehicle. It may involve the input of certain commands and/or of a certain code at the associated mobile device, possibly checking the code, and commanding the drone into a landing sequence for physically landing the drone. This may still happen under tracking the mobile target. Landing of the drone may be made autonomously by the drone without requiring remote control for doing so. While or after landing, also many of the start activities may be reversed, such as shutting the communication channels, stopping the camera etc.. Tracking may persist as long as required. But disarming may also be made under remote control. Landing a drone that tracks a vehicle may be made when the vehicle is parked somewhere.

Prior to the actual use of the system there may be some initiation steps, such as · Installing one or more programs or apps on a mobile

device. The programs/apps may be for integrating the mobile device into the system, such as being a data relay between drone 11 and a remote destination and providing other functionalities or modules as described above, and/or for interfacing to the drone for making settings in the drone, and/or for making the mobile device

trackable by the drone,

• setting data in the mobile device 12 and for particular apps therein, such as communication path definitions to the drone and/or to external, preference/default

settings, defining the first and second predetermined destination,

• setting data in the drone, either directly at a hardware interface thereof or through an associated/connected mobile device 12, the data including the first and second predetermined destination in terms of technical data (IP addresses, email addresses, phone numbers, logins (user name, password), ...) required for reaching the

respective destinations, possibly preference or default settings, remote control settings and the like,

• registering at a service provider for ancillary services or providing a drone for temporary rental as mentioned above including related data input,

• Setting a method of disarming the activated system, such as remote-commanded disarming from a predetermined destination, or local by entering simple commands and/or a code such as a PIN (personal identification number) code .

Once these settings are made, the system is ready for use. System components may also include a remote start module or a remote start request module so that a drone can be started or requested to be started by someone else than the mobile target. Upon request, the target person or operator of target vehicle may decide to launch the system or might reject it. Police or other authorities may use this for requesting everybody in a certain area to launch their devices/drones to get as many 'eyes in the air' as possible for recording in order to survey a situation.

Likewise, in vehicles system components may also include an auto start module that activates, starts and launches the system upon detection of an accident (recognizable from transient unusual sensor signal patterns, such as high

acceleration values) . The drone is then automatically released to fly up and take picture of the scenery and send them to the first predetermined destination or to first responders.

Alternatively, the drone may fly around using autonomous navigation techniques (AI - artificial intelligence) to film passengers possibly injured in the car and send these data to first responders as said first predetermined destination. The drone may further have a module and functionality to fly back the earlier path to alert incoming traffic about the accident to make them speed down. Insofar, a feature of the invention is that the first predetermined final destination may be different depending on certain conditions, such as the launch circumstances. If launched willfully, it may be one certain destination. If launched automatically, it may be another destination.

The drone may be adapted to temporarily carry, also during flight, a mobile device such as a smartphone or the like, to which it is connected by cables (e.g. USB) and/or wireless (e.g. WiFi, Bluetooth or the like) . A related app or program may be running on the mobile device for interfacing towards matching structures of the drone itself. The mobile device may then take over several or many or most or all of the above mentioned functions. It may also act as controller of the drone and interfaces towards the native drone components, such as flight control, in appropriate manner by one or more related interfaces. The drone may use the functionalities of the mobile device, such as computing capabilities, GPS, camera, communication and the like, for the purposes as described above. The drone may have a mechanical holder for holding the mobile device in a certain relative posture, particularly for appropriately directing the cameras field of view. The drone may have a charging structure and port to which a potentially carried mobile device can connect for charging a carried mobile device. A further aspect of the invention is an inventory monitoring method, system and drone/vehicle.

The presence of inventory may be difficult to monitor, particularly when it is spread out across large areas or is at remote locations. Examples for this may be large car parks with a high number of vehicles, construction sites or, for example, cattle on grazing land. When the presence of a high number of items is of interest, it is increasingly cumbersome to check the presence of each of the items, depending on the number of items. Generally, such distributed inventory is subject to theft and other diminution effects. The longer such effects remain undetected, the more difficulty it may be to counteract them or reverse them.

Other fields of use for this invention may be to detect unexploded ordnance in live fire ranges as well as in post combat or war zones. Further to this end the invention may also be used, say in bow hunting for detecting and tracking a wounded animal. Finally, the invention may be used in

conjunction with search and rescue of people buried, say after an avalanche.

An inventory monitoring method comprises the steps of

launching a drone or another suitable autonomous moving vehicle, having the drone fly a path or vehicle drive a path, checking, during flying or driving, for presence of items on the ground, and storing data in accordance with said presence check and/or communicating such data, preferably automatically and immediately through a wireless link from the drone or vehicle to a predetermined destination.

Presence checking may comprise interrogating, by suitable components in said drone or vehicle, wireless readable

structrures such as RFID (radio frequency identification) tags provided on the items on the ground, by an RFID reader in the drone or vehicle, preferably a long range RFID reader.

But likewise, for checking presence of items, the drone may take images - in multiple wavebands including but not limited to infra-red - and perform image recognition on them for finding items. Then, instead of RFID, or in addition thereto, optical codes such as QR codes for optical recognition may be used, depending on circumstances. The results of image

recognition may also or instead be used for one or more of navigating the drone along a certain path, navigating the drone towards an object assumed to carry a readable RFID tag to be read.

The drone, when flying, may follow a predetermined flight path, and/or may follow a flight path generated or modified ad hoc, and/or may be remote-controlled. A drone used for presence checking has flying means for making the drone fly, navigation means for controlling the drone to fly a path, checking means for checking, during flying, for presence of items on the ground, and communicating means for communicating data in accordance with said presence check, preferably automatically and immediately through a wireless link . The drone may have an RFID reader, preferably a long range RFID reader, for interrogating, from said drone, RFID tags provided on the items on the ground, as said presence check.

The drone may have a camera for imaging the ground below the drone, and may have image recognition means for evaluating the images obtained by the camera.

It is thus an object of the invention to provide an inventory monitoring method and a drone capable of efficiently

monitoring the presence of items of an inventory.

This object is accomplished by the features of the independent items . Another aspect of the invention relates to inventory

monitoring. As set earlier, such inventory may be livestock on grazing land or may be carparks with a high number of cars. Another example are construction sites on which valuables are kept, such as construction machines or materials. A method for monitoring the presence of such inventory comprises the steps of launching a drone and making the drone fly a certain path. During flying, the drone checks for the presence of items on the ground and stores or communicates data obtained in response to such presence checks.

The presence check may comprise image recognition on images taken by the drone during flying or may comprise reading of RFID tags attached to the respective items on the ground. Image recognition may be targeted to the appearance of the inventory itself or to optical targets attached thereto, e.g. on the roof top of vehicles.

The drone may fly along a predetermined path. The drone may also be adapted to autonomously determine its path or to modify a predetermined path autonomously in accordance with certain criteria.

A drone for inventory monitoring has flying means for making the drone fly, navigation means for controlling the drone to fly path, checking means for checking, during flying of the drone, for presence of items on the ground, and communicating means or storing means for communicating or starring data in accordance with said presence check.

In the above and the following, the invention is described with reference to a flying drone. But instead, an autonomously driving and/or remote-controllable ground vehicle or rover may be used instead, if the environment (pavement) and other system parameters (speed, distances, ...) allow it. Insofar, references above and in the following to a drone shall also be considered as relating in the same way to a ground vehicle, ships and vessels included, used for example in fish farming.

Figure 6 shows schematically a certain scenery. 51 is a drone on its fly. 52 may be a pre-known area subject to monitoring. For example, it be fenced in grassland or a construction site or the like. The drone is made to fly a certain path 57. The path may be designed such that it scans a certain territory, for example 52 and Figure 5. It may meandering as shown. The Figure shows various items 56 on the monitored area. 54a, b, are assumed to be individual vehicles, for example

construction machines or car rental vehicles on a parking lot or the like. 53a, b, c, is assumed to be construction

material. As an example, one may assume products of semi valuable materials, for example a plurality of copper cable coils. 55a, b symbolizes livestock, such as grazing cattle.

All items are assumed to have individual RFID tags, i.e. three tags 56a, b, c, for the construction material items 53a, b, c two RFID tags 56d, e for vehicles 54a, b and RFID tags 56f, g in earmarks of livestock 55a, b. As said earlier, also other items may be considered here, such as the use for finding unexploded ammunition/ordnance carrying an RFID tag, finding people carrying such tags in rugged areas or possibly buried by an avalanche, or finding deer hit in bow hunting from an arrow carrying an RFID tag.

The drone may scan the given territory 52 and may make

presence check during such a scan. This includes, in the shown example, RFID reading. For doing so, the drone 51 is equipped with a related reader. During a scanning flight, the drone reads RFID tags nor optical tags and thus collects information from said tags. The operation of the drone includes the known measures for RFID reading, amongst them, depending on system layout, sending signals towards assumed RFID tags for powering them up and/or for logically activating them. The drone may store received and read information and/or may communicate it to a predetermined technical address. For example, the

collected information may finally comprise a list of IDs obtained from reading the various RFIDs. This list can then be evaluated. For example, it can be checked against an expected list or an earlier list. The drone is schematically shown in Figure 6. Flying means are symbolized by 52 and are shown as a propeller. The drone may be a helicopter or a Quattro copter or may be a regular fixed wing device, the latter being preferable for power efficiency. 53 symbolizes a motor, 54 a power supply which may be a battery for an electric motor 53 or may be a fuel tank for a combustion rotor. 58 is a camera that may be provided on the drone. 57 is an RFID reader, preferably a long range RFID reader. 56a, b, c are communication modules to be explained later. 55 is a general controller for controlling the various functions and tasks in the drone. The controller 55 may control the usual items such as flying, navigation and the like. But it may also control the features specific for the invention such as RFID reading, compiling read information and storing and/or communicating such information.

Numerals 56 may be various communication modules for allowing the drone to communicate externally. One or more of them (as shown) may be provided. One of them may be a local

communication module, such as for WLAN/Wi-Fi or Bluetooth

(56a) . Another 56b of them may be for long range communication towards mobile communication base stations requiring payment and AAA (authentication, authorization, accounting) measures. 56c may be some kind of tracking mechanism if tracking is required. The tracking mechanism may be an active one with data exchange from and/or towards a target to be tracked. Or it may be passive in the sense that no active signaling is involved. Then, it may be based on image recognition. On images provided by camera 58 and runs in the controller 55. Presence checking may also be or comprise taking images of a scenery on the ground and evaluating these images or

forwarding these images to a predetermined destination.

Forwarding images may be made in parallel to RFID reading and forwarding data relating to RFID reading. But likewise, instead of, or in addition to, forwarding image data and/or RFID data, such data can be stored in the drone and can be read out through a suitable interface later.

The drone may follow a predetermined path or may, at least, fly from a start point to a predetermined destination. At a certain destination, it may scan a certain area (such as 52 in Figure 5) according to preset strategies or according to a preset path. But likewise, it may decide the path for scanning an area 52 autonomously in accordance with searching criteria. Such criteria may be image recognition. It is assumed that livestock is to be monitored. Livestock may be somewhere in a certain area. Thus, the drone flies to the area and then decides its flight path autonomously. For accomplishing this, it may take images and evaluate it for finding livestock. It then may determine the path portions (such as meanders) to cover the locations of livestock. So, overall, the flight path determination may be a mixture between predetermined flying and autonomously decided flying, the latter based on image recognition or other input. Such other input may be RFID or QR reading itself. For

example, animals may have crowded up so that in a certain territory 52 they are crowded at a certain spot (assume, for example, in Figure 5 the lower left corner) . There, many RFID readings will be obtained. Possibly, there may be too many of them for properly reading and evaluating all of them. Then, the flight strategy might be decided to be changed such that proper RFID reading can be made. For example, flight altitude can be changed/lowered or flight speed may be changed/lowered to receive fewer RFID readings per time. A certain area may be scanned plural times. Double detections of same RFIDs may later be eliminated by related checks.

The returning RFID signals (i.e. from tag to reader) may also be used for navigation, i.e. deciding the drones flight path, for example towards the tag.

In a kind of "subscriber model" or "service provider model", a drone may steer sequentially to plural sites 52 for making individual and independent presence checks in each of these sites. The drone may further be adapted to decide its flight path and schedule in accordance with power considerations. Particularly, the drone may decide autonomously to steer towards a predetermined point in accordance with power

considerations. For example, it may decide to fly to its home base or to some kind of docking station. At the home base or at a docking station, refilling/charging may be made, and data may be handed over away from the drone to some kind of recipient. This may be made wireless (WLAN/Wi-Fi, Bluetooth or the like) or may be made through attached cables. The drone may have a GPS functionality for locating itself continuously and in real time. The drone may supplement the presence check data by data generated by itself, such as location data, time stamp data and the like. Figure 7 symbolizes the central controller 55 of the drone. It comprises a module for flight control 71, a module 72 for path deciding and path guidance, a module 73 for possibly tracking a target, i.e. finding and following a tracking goal and interfacing towards path module 72 and flight control module 71 for accomplishing tracking by the drone. 74 is a module for presence check and may include the control of an RFID reader. It may further comprise data assembling and formatting

structures for compiling the presence check data obtained in response to the checks. Particularly, it may compile RFID responses in some kind of list, data file or the like. 75 may be a communication controller for holding one or more predetermined addresses (communication addresses) , for

controlling communication modules and communication activities on the sending side and on the receiving side. 76 may be a camera control module. It may control taking images and/or video streams. Further, it may control the field of view of the camera both regarding opening angle and direction. The cameras field of view may be controlled by adjusting the camera relative to the frame/body 51 of the drone, or by appropriately orienting the entire drone.

The drone may have remote control capabilities. The controller 55 may have a remote control module 77 rendering related external communication and internal control and signaling. Remote control may be used, for example, for scanning a certain target area 52 by controlling the flight path of the drone such that it covers the desired area. Under such a strategy, the drone may fly autonomously to the target area 52 and may then be taken over to remote control.

An image recognition module 78 may be provided, if not already contained in some other modules. Image recognition may be used for presence checking and/or for flight path deciding. The mentioned modules may be implemented in parts by hardware components and possibly in other parts by software components rendering applications running on a microcomputer provided in the drone. The controller 55 may be or comprise such a

microcontroller.

It is pointed out that components 71 to 77 of the controller 55 need not necessarily be provided in combination. Rather, one or plural of them may be provided as required.

Feature combinations for defining the matter for which

protection is sought in terms of the technical features of the invention may be itemized as follows:

A. An inventory monitoring method comprising the steps of launching a drone or starting a vehicle,

having the drone fly or vehicle drive a path,

checking, during flying or driving, for presence of items on the ground, and

storing data in accordance with said presence check and/or communicating such data, preferably automatically and immediately through a wireless link.

B. The method according to item A, wherein checking

comprises interrogating, by said drone, RFID tags

provided on the items on the ground, by an RFID reader, preferably a long range RFID reader. The method according to item A or B, wherein the drone, when flying, follows a predetermined flight path.

The method according to one of the items A to C, wherein the drone, when flying, follows a flight path generated or modified ad hoc.

The method according to one of the items A to D, wherein the drone takes images and performs image recognition on them, the results of image recognition being used for one or more of

navigating the drone along a desired path,

navigating the drone towards an object assumed to carry an RFID tag to be read,

detecting the presence of an item on the ground A drone or vehicle having

flying or driving means for making the drone fly or the vehicle drive,

navigation means for controlling the drone to fly or vehicle to drive a path

checking means for checking, during flying or driving, for presence of items on the ground, and

communicating means for communicating data in accordance with said presence check, preferably automatically and immediately through a wireless link. The drone or vehicle of item G, having an RFID reader, preferably a long range RFID reader, for interrogating, from said drone or vehicle, RFID tags provided on the items on the ground, as said presence check.

The drone or vehicle of item G, having a camera for imaging the ground below the drone, and

image recognition means for evaluating the images obtained by the camera.