ENFORCEMENT

CROSS-REFERENCE TO RELATED APPLICATIONS

[001] This application claims the benefit of U.S. Provisional Application No. 62/221 ,629 filed on September 22, 2015, the contents of which are hereby incorporated by reference.

TECHNICAL FIELD

[002] The present disclosure relates generally to parking monitoring systems, and more specifically to parking monitoring based on image processing.

BACKGROUND

[003] Camera-based traffic monitoring systems are deployed by law enforcement agencies and municipalities to enforce traffic laws (including parking laws) in an attempt to detect unsafe driving behavior and illegal parking.

[004] Existing solutions for camera-based traffic monitoring are typically based on video or stills footage used for prosecution of drivers in violation of traffic laws. Studies show that most existing solutions successfully identify a low percentage of all traffic violations due to the complexity involved in identifying certain violations. For example, existing solutions face particular challenges in identifying parking violations such as blocking a bus stop, parking in a place that obstructs view of a junction, and otherwise not complying with parking regulations.

[005] One of the reasons for the deficiencies of existing solutions is that such solutions perform only static analyses of traffic violations. In a typical analysis utilized by existing solutions, the cameras generate a single or a few frames of the vehicle being present in the alleged state of the violation. The low number of frames may not be sufficient to determine if the vehicle is in violation.

[006] Additionally, existing solutions face challenges in accurately identifying parking violations when different areas within a parking complex have different sets of parking rules or have different parking authorizations. For example, some areas may include handicapped spots, some areas may only allow compact cars, different areas may have different allowable parking time limits, some areas may be open to the public generally while others are available only to residents or parking permit owners, some areas may be accessible only to owners of vehicles having particular license plate numbers, and the like. Even further, such authorizations may vary based on, for example, date and time.

[007] It would therefore be advantageous to provide a solution that would overcome the deficiencies of the prior art.

SUMMARY

[008] A summary of several example embodiments of the disclosure follows. This summary is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not wholly define the breadth of the disclosure. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later. For convenience, the term "some embodiments" may be used herein to refer to a single embodiment or multiple embodiments of the disclosure.

[009] Certain embodiments disclosed herein include a method for enforcing parking rules based on visual content. The method comprises: obtaining a first visual content captured by a first capturing device, the first visual content showing a vehicle at a first time; analyzing, via an image processor, the first visual content; identifying, based on the analysis, a license plate character string of the first visual content; identifying, based on the license plate character string, parking authorization information associated with the vehicle; and determining, based on the parking authorization information and the first visual content, a parking legality status of the vehicle.

[0010] Certain embodiments disclosed herein also include a non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method, the method comprising: obtaining a first visual content captured by a first capturing device, the first visual content showing a vehicle at a first time; analyzing, via an image processor, the first visual content; identifying, based on the analysis, a license plate character string of the first visual content; identifying, based on the license plate character string, parking authorization information associated with the vehicle; and determining, based on the parking authorization information and the first visual content, a parking legality status of the vehicle.

[0011] Certain embodiments disclosed herein also include a system for enforcing parking rules based on visual content. The system comprises: an image processor configured to at least identify characters in visual content; a processing circuitry; and a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to: obtain a first visual content captured by a first capturing device, the first visual content showing a vehicle at a first time; analyze, via the image processor, the first visual content; identify, based on the analysis, a license plate character string of the first visual content; identify, based on the license plate character string, parking authorization information associated with the vehicle; and determine, based on the parking authorization information and the first visual content, a parking legality status of the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the disclosed embodiments will be apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0013] Figure 1 is a network diagram utilized to describe the various disclosed embodiments.

[0014] Figure 2 is a flowchart illustrating a method for enforcing parking rules based on visual content according to an embodiment.

[0015] Figure 3 is a flowchart for determining whether a parking violation has occurred according to an embodiment.

[0016] Figure 4 is a block diagram of a parking enforcement system according to an embodiment.

[0017] Figure 5 is a visual representation of a parking area.

DETAILED DESCRIPTION

[0018] It is important to note that the embodiments disclosed herein are only examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed embodiments. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality. In the drawings, like numerals refer to like parts through several views.

[0019] The various disclosed embodiments include a method and system for enforcing parking rules based on analysis of visual content. A first visual content showing a vehicle at a first time is obtained. The first visual content is analyzed to identify a license plate number of the shown vehicle. Based on the identified license plate number, parking authorization information of the shown vehicle is determined. It is determined, based on the parking authorization information and the first visual content, whether the vehicle is parked legally. In some embodiments, determining whether the vehicle is parked legally is based further on a second visual content showing the vehicle at a second time.

[0020] Fig. 1 shows an example network diagram 100 utilized to describe the various disclosed embodiments. The example network diagram 100 includes a network 120 communicatively connected to a plurality of capturing devices (CDs) 1 10-1 through 1 10- N (hereinafter referred to individually as a capturing device 1 10 and collectively as capturing devices 1 10, merely for simplicity purposes), a parking enforcement system (PES) 130, and a database 140. The network 120 may be, but is not limited to, a wireless, cellular or wired network, a local area network (LAN), a wide area network (WAN), a metro area network (MAN), the Internet, the worldwide web (WWW), similar networks, and any combination thereof.

[0021] Each of the capturing devices 1 10 may be, but is not limited to, a personal computer, a laptop, a tablet computer, a smartphone, a portable terminal device, a wearable computing device, or any other device capable of capturing, storing, and sending visual content (e.g., images or videos). In another embodiment, any of the capturing devices 1 10 may be a stationary or otherwise non-portable device physically located in particular portions of a parking area (e.g., the entrance, the exit, or both). The parking area is a location designated for parking, and may include a plurality of parking spots. The capturing devices 1 10 may be further configured to generate metadata associated with captured visual content. Such metadata may indicate, for example, a time or geographic location of the captured visual content.

[0022] If the capturing device 1 10 is a mobile or otherwise portable device, it may be utilized by, e.g., parking area operators patrolling the parking area. In such a case, visual content may be captured by the capturing device 1 10 in response to user input (e.g., a user gesture) by the user of the capturing device 1 10. A location of the visual content may also be determined based on GPS signals of the capturing device 1 10. Such mobile capturing may be useful when, for example, the patrolling user monitors multiple parking areas or portions thereof, particularly when at least some of the parking areas or portions thereof do not have capturing devices located nearby.

[0023] Each capturing device 1 10 further includes at least one sensor 1 15. The sensor 1 15 is configured to capture visual content such as, but not limited to, images, video, or both. Each sensor 1 15 may be, but is not limited to, a still image camera, a video camera, and the like. The capturing devices 1 10 may be physically located so as to allow capturing of visual content of different portions of a parking area, at different angles, or a combination thereof.

[0024] Each capturing device 1 10 typically includes a processing circuitry (not shown) and a memory (not shown), the memory containing instructions for sending the captured visual content to the parking enforcement system 130 via the network 120. Each capturing device 1 10 may further include a motion sensor (not shown) configured to detect movement of vehicles near the capturing device 1 10 and to cause the capturing device 1 10 to capture visual content in response to detected vehicle movement.

[0025] The database 140 includes information related to vehicles with respect to characters on license plates of the vehicles. Such information may include parking rules for determining parking legality status of vehicles such as, but is not limited to, whether each vehicle is allowed to park in particular portions of the parking area (e.g., in handicap spots or in spots with reserved or permit parking), restrictions on parking (e.g., particular days, times, or both), parking pricing (e.g., a driver of a particular vehicle may be entitled to discounted parking), combinations thereof, and the like.

[0026] The database 140 may further store information related to predetermined parking requirements. The parking rules may be further based on the predetermined parking requirements. As a non-limiting example, the parking rules may indicate that a parking spot designated for train passengers may only be utilized by vehicle owners who purchased a train ticket and only for a limited time (e.g., a known duration of time of the round trip of the train or a predetermined amount of time such as 24 hours). In such an example, the parking legality status may be determined as legal only if the database 140 has stored thereon data indicating purchase of a train ticket such as, but not limited to, a receipt or other scanned information (for example, a QR code), information from a data source (not shown) associated with the train service, and the like.

[0027] In an embodiment, the parking enforcement system 130 is configured to determine parking violations in parking areas. Parking violations may be, but are not limited to, incomplete payments, parking in prohibited areas, parking without a permit, and the like. To this end, in a further embodiment, the parking enforcement system 130 is configured to obtain a first visual content captured by a first capturing device of the capturing devices 1 10. The first visual content shows a vehicle at a first time. The first visual content may be further associated with a first time pointer indicating the first time. The first time pointer may be, e.g., included in metadata associated with the first visual content.

[0028] The parking enforcement system 130 is configured to analyze, using image processing techniques, the first visual content to at least identify a license plate character string (e.g., "A1 B 2C3") associated with the vehicle. Based on the identified license plate character string, the parking enforcement system 130 is configured to identify a parking authorization information or other parking information related to the vehicle. Identifying the parking authorization information may include, but is not limited to, searching the database 140 using the identified license plate character string. The parking authorization information may include, but is not limited to, permits (e.g., handicap or resident permits), prepaid parking fare, restrictions on parking (e.g., limits on times, days, or areas of parking), and the like.

[0029] In a further embodiment, the parking enforcement system 130 is configured to obtain a second visual content from a second capturing device. The second visual content shows the vehicle at a second time. In an embodiment, the second capturing device and the first capturing device may be the same capturing device 1 10. In another embodiment, the first visual content and the second visual content may show different portions of the parking area (e.g., an entrance and an exit of a parking lot). The second visual content may be further associated with a second time pointer indicating the second time. The second time pointer may be, e.g., included in metadata associated with the second visual content

[0030] Based on the first visual content and the parking authorization information, the parking enforcement system is configured to determine a parking legality status (i.e., whether the vehicle is parked legally). In a further embodiment, the parking legality status may be further determined based on the second visual content. In yet a further embodiment, the parking legality status may be further determined based on the first time pointer and the second time pointer. Such a time pointer-based determination may be utilized when, for example, parking authorizations for the vehicle indicate that the owner of the vehicle has only paid for a limited amount of time parking or that the vehicle is restricted to parking at specific times or for a limited duration of time.

[0031] In an embodiment, the parking legality status may be determined based on at least one parking rule. To this end, in a further embodiment, the parking enforcement system 130 may be configured to retrieve the at least one parking rule from, e.g., the database 140.

[0032] In yet a further embodiment, different parking rules may be applied for different parking areas or portions of a parking area. As a non-limiting example, some parking spots may allow parking at any time, while other parking spots allow parking only between 8 AM and 8 PM. To this end, in an embodiment, the parking enforcement system 130 may be configured to determine which parking rules to retrieve based on a location of the first visual content. The location of the first visual content may be determined based on, but not limited to, GPS signals of the first capturing device (e.g., if the first capturing device is a mobile device), a predetermined location of the first capturing device (e.g., if the first capturing device is stationary), image processing of the first visual content, a combination thereof, and the like.

[0033] The parking legality status may be determined as illegal if, for example, the vehicle is parked in a prohibited portion of the parking area (either generally or specifically for the vehicle), if the vehicle is parked above a threshold duration of time, if the vehicle is parked on a prohibited day or at a prohibited time, if the vehicle is parked for a period of time associated with a parking fare greater than a prepaid fare for the vehicle, and the like.

[0034] In a further embodiment, when a parking legality status of the vehicle is determined as illegal, the parking enforcement system 130 may be configured to prevent the vehicle from leaving the parking area until, for example, a fine or overdue fare is paid. To this end, the parking enforcement system 130 may further include or be communicatively connected to a controller (not shown), wherein the controller is configured to control a gate or other barrier (not shown) to entry or exit from the parking area. In yet a further embodiment, the parking enforcement system 130 may be configured to identify the license plate character set in the second visual content and, upon determining that the license plate character set is associated with a vehicle for which the parking legality status is illegal, automatically cause the controller to prevent opening, raising, or otherwise removing the barrier from the vehicle's path.

[0035] In another embodiment, a parking legality status of the vehicle is determined as illegal, the parking enforcement system 130 may be configured to send a notification indicating the illegal parking. The notification may be sent to, for example, a device utilized by a parking area operator such as, but not limited to, the capturing device 1 10 or any other device (not shown) owned or utilized by the parking area operator. The notification may indicate to the parking area operator that the operator should inspect the vehicle.

[0036] In a further embodiment, the notification may include a visual illustration of a parking legality status of each of a group of vehicles in the parking area. Fig. 5 is an example visual illustration 500 showing parking legality status of vehicles occupying multiple parking spots 1 -32 in a parking area. The shaded parking spots 1 -2, 507, 9, 12, 14-15, 17, 19-24, and 27-29 illustrate parking spots occupied by illegally parked vehicles. The visual illustration may be utilized by, e.g., a parking area operator delivering tickets or checks for illegal parking.

[0037] In yet another embodiment, the parking legality status may be stored in the database 140 for subsequent use. Such subsequent use may include, for example, retrieval by a parking area operator (e.g., via the capturing device 1 10 or another device utilized by the parking area operator) to confirm parking legality of manually inspected vehicles. [0038] It should be noted that the embodiments described herein above with respect to Fig. 1 are described with respect to a single vehicle in the first visual content merely for simplicity purposes and without limitations on the disclosed embodiments. Multiple vehicles may be shown in the first visual content, and a license plate character set of each vehicle may be identified without departing from the scope of the disclosure. Further, a second visual content may be obtained for each vehicle, with each second visual content showing one of the vehicles. Thus, the disclosed embodiments may be equally utilized to determine parking violations for a plurality of vehicles.

[0039] It should be noted that any or all of the data stored in the database 140 may be restricted data that is stored in a designated portion of the database 140 that is not directly accessible via the network 120. An example of restricted data may be license plate character strings. In such an embodiment, the parking enforcement system 130 may be configured to obtain the restricted data via communication with one or more proxy servers (not shown) communicatively connected to the server over the network 120.

[0040] It should further be noted that different portions of a parking area (e.g., different spots or other subsections of the parking area) may be subject to different parking rules without departing from the scope of the disclosure.

[0041] Fig. 2 is an example flowchart 200 illustrating a method for enforcing parking rules based on visual content processing according to an embodiment. In an embodiment, the method may be performed by a parking enforcement system (e.g., the parking enforcement system 130).

[0042] At S210, a first visual content captured by a first capturing device is obtained. The first visual content may be an image or video, and may be received from the first capturing device (e.g., the capturing device 1 10), or retrieved from a storage. The first visual content shows a vehicle at a first time. In an embodiment, S210 may further include obtaining metadata associated with the first visual content. The metadata may indicate, for example, a time or location of the first visual content.

[0043] At S220, the first visual content is analyzed, using image processing, to identify a license plate character string associated with the vehicle shown in the first visual content. The license plate character string may include, but is not limited to, letters, numbers, and symbols shown in the license plate that are utilized to uniquely identify the vehicle. [0044] At S230, based on the identified license plate character string, parking authorization information of the vehicle is identified. In an embodiment, S230 includes searching, based on the license plate character string, a storage (e.g., the database 140), for associated parking authorization information. The parking authorization information may include, but is not limited to, permits (e.g., handicap or resident permits), prepaid parking fare, restrictions on parking (e.g., limits on times, days, or areas of parking), and the like.

[0045] At S240, a second visual content captured by a second capturing device may be obtained. The second visual content shows the vehicle at a second time. In an embodiment, S240 may further include obtaining metadata associated with the second visual content. The metadata may indicate, for example, a time of the second visual content. In another embodiment, the second capturing device and the first capturing device may be the same capturing device. In yet another embodiment, S240 may only be performed if the parking authorization information indicates that the vehicle is authorized for parking only for particular times, durations of time, and the like.

[0046] At S250, based on the first visual content and the parking authorization information, a parking legality status of the vehicle shown in the first visual content is determined. In an embodiment, S250 may further include retrieving (e.g., from a database) parking rules associated with the location of the vehicle as shown in the first visual content. Such an embodiment may be performed when, for example, different rules apply to different parking areas or portions thereof monitored by the parking enforcement system. In another embodiment, the parking legality status may be determined further based on the second visual content and, in particular, the time of the second visual content in relation to the time of the first visual content. Such a time-based parking legality status determination may be utilized when the parking rules or the parking authorization information indicates that the vehicle is only permitted to park for limited periods of time or at particular days or times.

[0047]At optional S260, the determined parking legality status may be sent as a notification, stored in a database (e.g., the database 140), or both. In a further embodiment, S260 may further include automatically issuing a parking ticket or check if the parking legality status is determined as illegal. [0048] Figure 3 is an example flowchart 300 illustrating a method for enforcing parking rules based on visual content processing according to another embodiment. In an embodiment, the method may be performed by a parking enforcement system (e.g., the parking enforcement system 130).

[0049] At S310, a first visual content captured by a first capturing device is obtained. The first visual content may be an image or video, and may be received from the first capturing device (e.g., the capturing device 1 10), or retrieved from a storage. The first visual content shows a vehicle at a first time. In an embodiment, S310 may further include obtaining metadata associated with the first visual content. The metadata may indicate, for example, a time or location of the first visual content.

[0050] At S320, the first visual content is analyzed, using image processing, to identify a license plate character string associated with the vehicle shown in the first visual content. The license plate character string may include, but is not limited to, letters, numbers, and symbols shown in the license plate that are utilized to uniquely identify the vehicle.

[0051] At S330, based on the identified license plate character string, parking authorization information of the vehicle is identified. In an embodiment, S330 includes searching, based on the license plate character string, a storage (e.g., the database 140), for associated parking authorization information. The parking authorization information may include, but is not limited to, permits (e.g., handicap or resident permits), prepaid parking fare, restrictions on parking (e.g., limits on times, days, or areas of parking), and the like.

[0052] At S340, a location of the vehicle shown in the first visual content is determined. The location may be determined based on, but not limited to, the analysis of the first visual content. As a non-limiting example, the location may be determined based on one or more visual indicators (e.g., a sign or known marker indicating the location of the parking spots or portions shown in the first visual content). In a further embodiment, the location may be determined further based on GPS signals received by the first capturing device, the obtained metadata, or both.

[0053] At S350, based on the determined location, the first visual content, and the parking authorization information, a parking legality status of the vehicle shown in the first visual content is determined. In an embodiment, S340 may further include retrieving (e.g., from a database) parking rules associated with the location of the vehicle as shown in the first visual content. Such an embodiment may be performed when, for example, different rules apply to different parking areas or portions thereof monitored by the parking enforcement system. In another embodiment, the parking legality status may be determined further based on the second visual content and, in particular, the time of the second visual content in relation to the time of the first visual content. Such a time-based parking legality status determination may be utilized when the parking rules or the parking authorization information indicates that the vehicle is only permitted to park for limited periods of time or at particular days or times.

[0054] At optional S360, the determined parking legality status may be sent as a notification, stored in a database (e.g., the database 140), or both. In a further embodiment, S360 may further include automatically issuing a parking ticket or check if the parking legality status is determined as illegal.

[0055] Fig. 4 shows an example block diagram of the parking enforcement system 130 implemented according to an embodiment. The parking enforcement system 130 includes a processing system 410 coupled to a memory 415, a storage 420, an image processor 430, and a network interface 440. In an embodiment, the components of the parking enforcement system 130 may be communicatively connected via a bus 450.

[0056] The processing system 410 may be realized as one or more hardware logic components and circuits. For example, and without limitation, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), Application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), general-purpose microprocessors, microcontrollers, digital signal processors (DSPs), and the like, or any other hardware logic components that can perform calculations or other manipulations of information.

[0057]The memory 415 may be volatile (e.g., RAM, etc.), non-volatile (e.g., ROM, flash memory, etc.), or a combination thereof. In one configuration, computer readable instructions to implement one or more embodiments disclosed herein may be stored in the storage 420.

[0058] In another embodiment, the memory 415 is configured to store software. Software shall be construed broadly to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Instructions may include code (e.g., in source code format, binary code format, executable code format, or any other suitable format of code). The instructions, when executed by the one or more processors, cause the processing system 410 to perform the various processes described herein. Specifically, the instructions, when executed, cause the processing system 410 to perform an on-demand authorization of access to protected resources, as discussed hereinabove.

[0059] The storage 420 may be magnetic storage, optical storage, and the like, and may be realized, for example, as flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVDs), or any other medium which can be used to store the desired information.

[0060] The image processor 430 may include, but is not limited to, a recognition unit (RU) 435 configured to identify patterns, features, or both, in unstructured data sets. Specifically, in an embodiment, the image processor 430 is configured to identify at least characters in the unstructured data.

[0061] The storage 420 may also store metadata generated based on analyses of unstructured data by the image processor 430. In a further embodiment, the storage 420 may further store queries generated based on the metadata.

[0062] The network interface 440 allows the parking enforcement system 130 to communicate with the database 140 for the purpose of, for example, retrieving parking authorization information and parking rules. Additionally, the network interface 440 allows the parking enforcement system 130 to communicate with the capturing device 1 10 in order to receive visual content, metadata, and GPS signals.

[0063] It should be understood that the embodiments described herein are not limited to the specific architecture illustrated in Fig. 4, and other architectures may be equally used without departing from the scope of the disclosed embodiments.

[0064] It should be noted that the various disclosed embodiments are described with respect to legality of parking merely for simplicity purposes and without limitation on the disclosed embodiments. Parking does not need to be prohibited by law to be illegal pursuant to the disclosed embodiments. Any parking that is not permitted (e.g., as indicated by parking rules of a private parking area operator) may be determined to be illegal for purposes of the disclosed embodiments without departing from the scope of the disclosure.

[0065] As used herein, the phrase "at least one of" followed by a listing of items means that any of the listed items can be utilized individually, or any combination of two or more of the listed items can be utilized. For example, if a system is described as including "at least one of A, B, and C," the system can include A alone; B alone; C alone; A and B in combination; B and C in combination; A and C in combination; or A, B, and C in combination.

[0066] The various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium consisting of parts, or of certain devices and/or a combination of devices. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units ("CPUs"), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit. Furthermore, a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal.

[0067] All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the disclosed embodiment and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosed embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.