Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
EVENT FRAME LOSSLESS COMPRESSION FRAMEWORKS, AND MEMORY-EFFICIENT REPRESENTATIONS FOR RANDOM ACCESS TO EVENT FRAME PIXELS
Document Type and Number:
WIPO Patent Application WO/2024/012685
Kind Code:
A1
Abstract:
In some examples, an imaging apparatus can comprise a sensor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged. The imaging apparatus can comprise a processor, a memory coupled to the processor, the memory configured to store program code executable by the processor, the program code comprising one or more instructions, whereby to cause the imaging apparatus to generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor, and encode the event frame to generate a bitstream, the bitstream comprising a binary sequence defining at least one of a number of events, a number of non-events, and event polarities associated with events of the event frame.

Inventors:
SCHIOPU IONUT (SE)
BILCU RADU CIPRIAN (SE)
Application Number:
PCT/EP2022/069755
Publication Date:
January 18, 2024
Filing Date:
July 14, 2022
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
HUAWEI TECH CO LTD (CN)
SCHIOPU IONUT (SE)
International Classes:
H04N19/93; G06T1/00; H03M7/00; H04N19/46
Domestic Patent References:
WO2019067732A12019-04-04
Other References:
KHAN NABEEL ET AL: "Time-Aggregation-Based Lossless Video Encoding for Neuromorphic Vision Sensor Data", IEEE INTERNET OF THINGS JOURNAL, IEEE, USA, vol. 8, no. 1, 8 July 2020 (2020-07-08), pages 596 - 609, XP011827528, DOI: 10.1109/JIOT.2020.3007866
SRUTARSHI BANERJEE ET AL: "Lossy Event Compression based on Image-derived Quad Trees and Poisson Disk Sampling", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 1 December 2020 (2020-12-01), XP081826665
YANG DAN ET AL: "Motion Aware Double Attention Network for Dynamic Scene Deblurring", 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), IEEE, 19 June 2022 (2022-06-19), pages 1112 - 1122, XP034173940, DOI: 10.1109/CVPRW56347.2022.00120
BENETTI MICHELE ET AL: "A Low-Power Vision System With Adaptive Background Subtraction and Image Segmentation for Unusual Event Detection", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: REGULAR PAPERS, IEEE, US, vol. 65, no. 11, 1 November 2018 (2018-11-01), pages 3842 - 3853, XP011691102, ISSN: 1549-8328, [retrieved on 20181005], DOI: 10.1109/TCSI.2018.2857562
MOHAMMAD MOSTAFAVI I S ET AL: "Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networks", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 20 November 2018 (2018-11-20), XP081784455, DOI: 10.1109/CVPR.2019.01032
Attorney, Agent or Firm:
KREUZ, Georg M. (DE)
Download PDF:
Claims:
CLAIMS

1. An imaging apparatus comprising a sensor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged, the imaging apparatus comprising: a processor; a memory coupled to the processor, the memory configured to store program code executable by the processor, the program code comprising one or more instructions, whereby to cause the imaging apparatus to: generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor; and encode the event frame to generate a bitstream, the bitstream comprising a binary sequence defining at least one of a number of events, a number of non-events, and event polarities associated with events of the event frame.

2. The imaging apparatus as claimed in claim 1, wherein the program code comprises one or more further instructions, whereby to cause the imaging apparatus to: generate a vector representation of the event frame; and encode the vector representation of the event frame.

3. The imaging apparatus as claimed in claim 2, wherein the program code comprises one or more further instructions, whereby to cause the imaging apparatus to use the vector representation of the event frame to: generate respective binary representations for a first component of the vector representation of the event frame, the first component comprising one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame.

4. The imaging apparatus as claimed in claim 3, wherein the program code comprises one or more further instructions, whereby to cause the imaging apparatus to use the vector representation of the event frame to: generate further respective binary representations for a second component of the vector representation of the event frame, the second component subsequent to the first component and comprising one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame.

5. The imaging apparatus as claimed in any preceding claim, wherein the program code comprises one or more further instructions, whereby to cause the imaging apparatus to: divide the event frame into an array of multiple portions, and for each one of the multiple portions: generate a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events, and header data comprising event polarities associated with events of the portion.

6. The imaging apparatus as claimed in any preceding claim, wherein the program code comprises one or more further instructions, whereby to cause the imaging apparatus to: for each pixel, assign a positive or negative value to each event occurring over the predetermined time interval on the basis of a change in the intensity, sum the values occurring over the predetermined time interval, and determine the sign of the sum of values.

7. The imaging apparatus as claimed in claim 6, wherein the program code comprises one or more further instructions, whereby to cause the imaging apparatus to: encode the sign of the sum of values using a ternary alphabet.

8. An event signal processor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged, the event signal processor configured to: generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor; and encode the event frame to generate a bitstream, the bitstream comprising one or more of a binary sequence defining at least one of a count of events, a count of non-events, and event polarities associated with events of the event frame.

9. The event signal processor as claimed in claim 8, further configured to: generate a vector representation of the event frame; and encode the vector representation of the event frame.

10. The event signal processor as claimed in claim 9, further configured to use the vector representation of the event frame to: generate respective binary representations for a first component of the vector representation of the event frame, the first component comprising one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame.

11. The event signal processor as claimed in claim 10, further configured to use the vector representation of the event frame to: generate further respective binary representations for a second component of the vector representation of the event frame, the second component subsequent to the first component and comprising one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame.

12. The event signal processor as claimed in any of claims 8 to 11, further configured to: divide the event frame into an array of multiple portions, and for each one of the multiple portions: generate a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events, and header data comprising event polarities associated with events of the portion.

13. The event signal processor as claimed in any of claims 8 to 12, further configured to: for each pixel, assign a positive or negative value to each event occurring over the predetermined time interval on the basis of a change in the intensity, sum the values occurring over the predetermined time interval, and determine the sign of the sum of values.

14. The event signal processor as claimed in claim 13, further configured to: encode the sign of the sum of values using a ternary alphabet.

15. A method for generating event data representing asynchronous changes in intensity of a scene or object to be imaged using an imaging apparatus comprising an event sensor, the method comprising: generating, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor; and encoding the event frame to generate a bitstream, the bitstream comprising a binary sequence defining at least one of a number of events, a number of non-events, and event polarities associated with events of the event frame.

16. The method as claimed in claim 15, further comprising: generating a vector representation of the event frame; and encoding the vector representation of the event frame.

17. The method as claimed in claim 16, further comprising, using the vector representation of the event frame, generating respective binary representations for a first component of the vector representation of the event frame, the first component comprising one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame.

18. The method as claimed in claim 17, further comprising, using the vector representation of the event frame, generating further respective binary representations for a second component of the vector representation of the event frame, the second component subsequent to the first component and comprising one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame.

19. The method as claimed in any of claims 15 to 18, further comprising: dividing the event frame into an array of multiple portions, and for each one of the multiple portions: generating a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events, and header data comprising event polarities associated with events of the portion.

20. The method as claimed in any of claims 15 to 19, further comprising: for each pixel, assigning a positive or negative value to each event occurring over the predetermined time interval on the basis of a change in the intensity, summing the values occurring over the predetermined time interval, and determining the sign of the sum of values.

21. The method as claimed in claim 20, further comprising: encoding the sign of the sum of values using a ternary alphabet.

22. A machine-readable storage medium encoded with instructions for encoding event data representing events generated using an event sensor, the instructions executable by a processor, whereby to cause the processor to: generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor; and encode the event frame to generate a bitstream, the bitstream comprising one or more of a binary sequence defining at least one of a count of events, a count of non-events, and event polarities associated with events of the event frame.

Description:
EVENT FRAME LOSSLESS COMPRESSION FRAMEWORKS, AND MEMORYEFFICIENT REPRESENTATIONS FOR RANDOM ACCESS TO EVENT FRAME

PIXELS

TECHNICAL FIELD

The present invention relates generally to event data generated by an event sensor. Aspects relate to low-complexity lossless coding frameworks for efficient memory representation of such event frames, and memory-efficient representations for fast and low computational complexity and random access to pixels of event frames.

BACKGROUND

An event camera is a device comprising an event sensor that is capable of generating data in response to relative changes in luminance in a scene or object being imaged. Specifically, an event camera sensor can report changes of incoming light intensity using asynchronous events triggered at the pixel level. Each pixel either reports a change in brightness (increase or decrease, typically in relation to a threshold) or remains silent when the light intensity does not change.

Event cameras have a high temporal resolution. Individual asynchronous events can be triggered at a timestamp distance of up to one microsecond (ms). As such, event cameras are widely used for Computer Vision tasks where the event-based approach can provide an improved performance compared with the more traditional RGB-based approach. For example, image deblurring, feature detection and tracking, optic flow estimation, 3D estimation, view synthesis, and many other applications can benefit from the use of event data acquired using the event sensor of an event camera. In general, these upper-level applications tend to consume the event data as images, in the form of event frames (EFs), rather than as individual asynchronous event sequences. As such, event sequences are typically accumulated into EFs having a 2 -bit per pixel representation.

EFs can be captured at a frame rate of up to one million (M) frames per second (fps), and some event cameras are able to provide a maximum bandwidth of up to a few hundred million events per second (Meps). Typically, event camera software encapsulates event information using 8 Bytes (B) per event. As such, very high rates of, e.g., a few GB of event data per second can easily be generated when capturing fast moving scenes.

Despite its utility, the encoding of asynchronous event sequences comprising such large amounts of data is a relatively nascent field. Once example of asynchronous event encoding is Spike Coding, in which the spatial and temporal characteristics of event location information are exploited. An input asynchronous event sequence is divided into macro-cubes, where the event spatial, time, and polarity information are encoded separately. Each macro cube has the full spatial resolution of the pixel array of the event sensor. A macro-cube is then spatially split into small event-cubes. The encoding process for the event-cube consists of event location coding, which is performed using address-prior (AP) mode and time-prior (TP) mode, and event polarity coding, which utilizes previous coding information as the reference to predict an imminent polarity. A computed residual error of the AP mode, TP mode, and polarity coding are then provided to an adaptive context-based entropy coder. In AP mode, a location histogram map and location histogram counts are encoded. The location histogram map indicates whether an event exists or not, while the location histogram counts the number of events that occurred for each pixel. In TP mode, offsets of the spatial location to a central point are encoded using an entropy coder.

Spike coding is, however, designed to deal only with raw asynchronous data rather than event frames, which, as noted above, are generally preferred. Furthermore, all raw events that occur in a time interval are encoded. However, for some applications not all events are required and a lossy solution may be more desirable, particularly when such large amounts of data as those noted above are under consideration.

Other encoding methods can encode spatial information, a number of events, and event polarity for event frames generated at specific time intervals. However, such methods are dependent on the performance of a video codec employed to encode a sequence of so-called superframes. For example, in the case of sensors with relatively high resolutions, e.g., 640x480, or 1280x720, captured events will tend to be sparsely disturbed inside frames. Therefore, superframes will not be efficiently encoded by, e.g., HEVC (which is typically used as the video codec) is designed to encode a different type of information. SUMMARY

An objective of the present disclosure is to provide a lossless event coding frameworks and efficient-memory representations that enable ransom access to event frame pixels. Such frameworks and representations can be used as part of an event signal processing apparatus.

The foregoing and other objectives are achieved by the features of the independent claims.

Further implementation forms are apparent from the dependent claims, the description and the Figures.

A first aspect of the present disclosure provides an imaging apparatus comprising a sensor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged, the imaging apparats comprising a processor, a memory coupled to the processor, the memory configured to store program code executable by the processor, the program code comprising one or more instructions, whereby to cause the imaging apparatus to generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor, and encode the event frame to generate a bitstream, the bitstream comprising a binary sequence defining at least one of a number of events, a number of non-events, and event polarities associated with events of the event frame.

A large number of event frames (e.g., of the order of several thousand) can be encoded quickly with minimal computational cost, thereby enabling implementation using relatively low- complexity hardware, such as network edge devices

In an implementation of the first aspect, the program code can comprise one or more further instructions, whereby to cause the imaging apparatus to generate a vector representation of the event frame, and encode the vector representation of the event frame. The imaging apparatus can use the vector representation of the event frame to generate respective binary representations for a first component of the vector representation of the event frame, the first component comprising one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame. In an example, the imaging apparatus can use the vector representation of the event frame to generate further respective binary representations for a second component of the vector representation of the event frame, the second component subsequent to the first component and comprising one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame. The program code can comprise one or more further instructions, whereby to cause the imaging apparatus to divide the event frame into an array of multiple portions, and for each one of the multiple portions, generate a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events, and header data comprising event polarities associated with events of the portion. The program code can comprise one or more further instructions, whereby to cause the imaging apparatus to for each pixel, assign a positive or negative value to each event occurring over the predetermined time interval on the basis of a change in the intensity, sum the values occurring over the predetermined time interval, and determine the sign of the sum of values. The program code can comprise one or more further instructions, whereby to cause the imaging apparatus to encode the sign of the sum of values using a ternary alphabet.

A second aspect of the present disclosure provides an event signal processor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged, the event signal processor configured to generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor, and encode the event frame to generate a bitstream, the bitstream comprising one or more of a binary sequence defining at least one of a count of events, a count of non- events, and event polarities associated with events of the event frame.

A memory-efficient representation codec is provided that can be used to encode one or more EFs, and provide random access to a group of pixels. It can offer a coding representation that is at least two times more efficient than an EF representation based on 2 -bits per pixel.

In an implementation of the second aspect, the event signal processor can generate a vector representation of the event frame, and encode the vector representation of the event frame. The event signal processor can use the vector representation of the event frame to generate respective binary representations for a first component of the vector representation of the event frame, the first component comprising one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame. The event signal processor can use the vector representation of the event frame to generate further respective binary representations for a second component of the vector representation of the event frame, the second component subsequent to the first component and comprising one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame.

In an example, the event signal processor can divide the event frame into an array of multiple portions, and for each one of the multiple portions, generate a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events, and header data comprising event polarities associated with events of the portion. The event signal processor can, for each pixel, assign a positive or negative value to each event occurring over the predetermined time interval on the basis of a change in the intensity, sum the values occurring over the predetermined time interval, and determine the sign of the sum of values. The event signal processor can encode the sign of the sum of values using a ternary alphabet.

A third aspect of the present disclosure provides a method for generating event data representing asynchronous changes in intensity of a scene or object to be imaged using an imaging apparatus comprising an event sensor, the method comprising generating, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor, and encoding the event frame to generate a bitstream, the bitstream comprising a binary sequence defining at least one of a number of events, a number of non- events, and event polarities associated with events of the event frame.

In an implementation of the third aspect, the method can further comprise generating a vector representation of the event frame, and encoding the vector representation of the event frame. The method can further comprise, using the vector representation of the event frame, generating respective binary representations for a first component of the vector representation of the event frame, the first component comprising one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame.

In an example, the method can further comprise, using the vector representation of the event frame, generating further respective binary representations for a second component of the vector representation of the event frame, the second component subsequent to the first component and comprising one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame. The method can further comprise dividing the event frame into an array of multiple portions, and for each one of the multiple portions, generating a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events, and header data comprising event polarities associated with events of the portion. The method can further comprise for each pixel, assigning a positive or negative value to each event occurring over the predetermined time interval on the basis of a change in the intensity, summing the values occurring over the predetermined time interval, and determining the sign of the sum of values. The method can further comprise encoding the sign of the sum of values using a ternary alphabet.

A fourth aspect of the present disclosure provides a machine-readable storage medium encoded with instructions for encoding event data representing events generated using an event sensor, the instructions executable by a processor, whereby to cause the processor to generate, for a predetermined time interval, an event frame using the event data, the event frame defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor, encode the event frame to generate a bitstream, the bitstream comprising one or more of a binary sequence defining at least one of a count of events, a count of non-events, and event polarities associated with events of the event frame.

BRIEF DESCRIPTION OF THE FIGURES

Embodiments will now be described by way of example only with reference to the figures, in which:

Figure l is a schematic representation of a machine according to an example; Figure 2 is a flow chart of a method according to an example;

Figure 3 is a flow chart of a method according to an example;

Figure 4 is a flow chart of a method according to an example; and

Figure 5 is a flow chart of a method according to an example.

DETAILED DESCRIPTION

Example embodiments are described below in sufficient detail to enable those of ordinary skill in the art to embody and implement the systems and processes herein described. It is important to understand that embodiments can be provided in many alternate forms and should not be construed as limited to the examples set forth herein.

Accordingly, while embodiments can be modified in various ways and take on various alternative forms, specific embodiments thereof are shown in the drawings and described in detail below as examples. There is no intent to limit to the particular forms disclosed. On the contrary, all modifications, equivalents, and alternatives falling within the scope of the appended claims should be included. Elements of the example embodiments are consistently denoted by the same reference numerals throughout the drawings and detailed description where appropriate.

The terminology used herein to describe embodiments is not intended to limit the scope. The articles “a,” “an,” and “the” are singular in that they have a single referent, however the use of the singular form in the present document should not preclude the presence of more than one referent. In other words, elements referred to in the singular can number one or more, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including,” when used herein, specify the presence of stated features, items, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, items, steps, operations, elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein are to be interpreted as is customary in the art. It will be further understood that terms in common usage should also be interpreted as is customary in the relevant art and not in an idealized or overly formal sense unless expressly so defined herein. An imaging apparatus in the form of an event camera can comprise a sensor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged. The generated event data can be used by one or more applications, such as those described above for example. According to an example, event data can be accumulated into event frame (EF) sequences. Lossless event coding frameworks and codecs are provided for hardware implementations in the form of, e.g., an event signal processing (ESP) apparatus.

In order to enable efficient pre-processing of the EFs, simple and fast event coding solutions are provided, which may be suitable for low-cost hardware implementations of event signal processing (ESP) devices such as ICs/ASICs and so on. According to an example, a lossless event coding framework is provided. The coding framework can be implemented as part of an ESP device.

In an example, a low-complexity codec for fast coding of large EF sets is provided, and a memory-efficient representation codec for fast and low computational complexity and random access (RA) to any group of EF pixels. In an example, a low-complexity lossless coding framework uses a run-length encoding (RLE) scheme for EF coding, and employs a coding technique, such as Elias Coding for example, to codify the frequency of the event/non-event symbols. A low-complexity codec for fast coding of large EF sets that uses an efficient representation is provided, in which an input event-volume is reconstructed using a Binary Map (BM) and a vector of concatenated event time-intervals. A memory-efficient representation codec for RA to any group of pixels is provided, in which each group is separately encoded in a bitstream and the information needed to extract any group bitstream from the output bitstream is provided in a header.

According to an example, an imaging apparatus comprises a sensor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged. The sensor has pixel resolution W x H. In an example, if the sensor detects a change of incoming light intensity larger than a predefined intensity threshold 8, then an asynchronous event e = (x, y, p, t) is triggered at pixel location (x, y), V x G [1, H], y 6 [1, W], at atime t, and with polarity p G {-1, 1 }, where p = 1 signals an intensity increase by 8 and p = -1 signals an intensity decrease by 8. An asynchronous event sequence {ei}, i = 1, 2, ..., N, of length N collects individual events triggered over a time period T. The sequence can be divided into spatio-temporal neighbourhoods (i.e., event-volumes) of time interval A (A < T) having size WxHxA. According to an example, for each event-volume an EF, denoted EF, is generated as follows: each position (x, y) accumulates the triggered events by summing their polarity and setting the new event polarity as the sign of the sum. An EF is set using the following 2 -bit representation: symbol 0 signals negative polarity events, symbol 1 signals non-event positions, and symbol 2 signals positive polarity events. Hence, EF contains symbols in the ternary alphabet A3={0, 1, 2}. This can be used to quantize the event time stamp (associated to each EF) and reduce the number of events.

According to an example, an RLE scheme is adapted for EF coding by counting the number of consecutive event or non-event symbols found in an EF, and codifying the counts using Elias Coding. In an example, a ternary vector V (with alphabet A3) can be encoded as a bitstream B, where V= vec(EF) and ‘vec’ is a vectorization operator. A variable k counts, in turn, the number of consecutive event symbols (0 or 2) or non-event symbols (1). A search key, 71, is used to signal the non-event symbol type using 71 = 0 and event symbol type using 71 = 1. Bitstream B can be either transmitted to a decoder, or stored on a storage device by, e.g., packing it into the byte sequence {<21)1=1,2 L by mapping every subset, S 1 = of B into an 8-bit symbol on, such that:

Accordingly, given an input vector V of symbols {-1, 0, +l} a bitstream B can be computed by, initially, setting N as the number of elements in V. The count, k, is set to 1, and the search key is initialised to 0 (i.e., always start the search with no event positions). It is first assumed that V does not contain only one symbol. As such, for i = 1, 2, . .., N, if V(i) = %, k is incremented, otherwise a new symbol has been detected. The count is represented using Elias Coding, as will be described in more detail below.

If the current symbol was an event the next polarity is encoded. That is, if IT = 1, polarities are stored using symbols “0” (for V(i)=-1) and “1” (for 7(i)=l). If k >= N (i.e., EF is either full of zeros or full of events), the a 1 is added to the bitstream at the first bit position thereof. If 71 = 0 (i.e., EF is full of zeros (no events detected), the second bit position of the bitstream is updated to a value of a 0. Otherwise (EF is full of events), the second bit position of the bitstream is updated to a value of a 1. At this point, as will be described in more detail below, V can be encoded as being full of polarity values.

In the case that k < N, and if it is 1, the count of the bitstream is represented using Elias coding and the count of polarities is represented using symbols 0 (for p=-l) and 1 (for p=l).

To avoid encoding the symbol type after each count, n is initialized as 7r«— 0, as EF is usually sparse. If F(l) is an event symbol, then the first count is ki = 0. Since Elias Coding codifies positive numbers, ki is changed to ki + 1 by initializing k = 1, while the following counts remain unchanged. Note that if V is a unique symbol sequence, then ki = N + 1 if EF is empty and ki = N if EF is full of events.

As noted above, a 1 can be added to the bitstream at the first bit position thereof (B(l)), which signals whether V is a unique symbol sequence. If B(l) is set, then B(2) signals the unique symbol type. In an example, an all non-event (empty) EF can be encoded as a ' 10' bitstream. The unlikely case of an all event (full) EF can be signalled using a ' 1 T bitstream, and a variation of the above process can be used to encode a vector V’= V » 1, where “»” is a bit-shifting operator which modifies symbols 0 and 2 into 0 and 1, respectively. It is noted that the last non- event symbol count can be ignored as V can be initialized at the decoder side with non-event symbols. Furthermore, according to an example, the count k can be codified either by employing Elias-Gamma Coding (EGC), which efficiently codifies small numbers, e.g., event symbol counts; or by employing Elias-Delta Coding (EDC), which is a more complex solution for encoding large numbers, e.g., non-event symbol counts. Note that count k > 0 can be codified by EGC using 2[log2 (k)J+l bits, and by EDC using [log2 (k)J+2[log2 ([log2 (k)J+l)J+l bits. The corresponding EGC (EDC) decoder first counts the number of consecutive ' O’ symbols to set N (L), and then decode k (N and k) using the following N+l (L+l and N+l) bits. Note that when EF has a large matrix-fill percentage, the non-event symbol counts decrease and, therefore, the EDC increased complexity may not be justified.

The approach described above has low complexity as the EF is traversed once and simple codifying methods are employed. An improved coding performance can be implemented by employing a more efficient compression method. However, such an approach would be more complex and have an increased hardware implementation cost. According to an example a larger EF set (e.g., up to a few thousand EFs) captured over a time period T can be encoded with a compression performance close to that of the state-of-the-art lossless coding methods. For example, consider F EFs in an input set of EFs, which form an EF Volume (EFV), denoted V, having size W X H X F. A Binary Map (BM), denoted B, can be used to represent the (x, y) positions where at least an event occurs in the EF stack, and a Concatenated Vector (CV), denoted C, can be used to collect event time-intervals C f of length F, corresponding to each event spatial position, (x, y).

If, in B there are N e positions set, then C is of length FN e . According to an example, to compress V , B and C are first generated and then encoded using the method described above. Since B and C are encoded in turn, the last non-event symbol count should not be ignored when encoding B, therefore, the event symbol condition for rr, can be removed. In an example, a corresponding process for decoding can decode B followed by C, traverse B to search for the N e positions for which B(i, j) = 1 and C in order to collect the set of vectors {(/}i =1 , 2 w e of length F , and then set each event time-interval as V(i,j, : ) = C t . Therefore, according to an example, to compress V, EFV symbols are remapped as follows: {0, 1, 2}— >{-l, 0, 1}. The BM (of size H*W) is initialised with zeros, and for each position (i,j) in EFV if EFV(i,j,l:F) contains at least one event then BM(ij) is set to a value of 1. A first bitstream can then be encoded using Elias Coding for example (EGC or EDC as described above). For each position (ij) in EFV, if BM(ij) = 1 the value of the CV is set such that CV <— EFV(ij,l :F), and a second bitstream can be encoded using Elias Coding for example. The first and second bitstreams can be concatenated to form a final bitstream.

According to an example, a process for memory-efficient representation (MER) of EFs can be used to encode one or more EFs, and provide random access (RA) to a group of pixels. In an example, a MER as described below can provide a representation at least two times more efficient than the EF representation of 2 -bits per pixel. A set of /EFs is divided into a set of blocks (volumes) of size wxh (w * h x f), which can be represented by a set of vectors multiple of w, and H is a multiple of h.

Each ternary vector Vi is then codified as bitstream Bi using the process described above. The header bitstream, Bh represents the EGC codification of each Bk length. The concatenated bitstream, B c , represents the bitstream set K- The output bitstream, B, concatenates

Bh with B c .

According to an example, a decoder can decode a set of K values (header information), and then access any group of pixels with index 1 by extracting Bi from B, decoding Bi to obtain Vi, and resizing Vi to block/volume size to obtain the group of pixels.

According to an example, MER can employ EGC for low-computational complexity, and provide RA to any group of pixels using the header information, which stores the length of all bitstreams, based on which any Bi can be extracted from B c , and then decoded the group of pixels so that it can be modified. A vector of only non-events symbols can be encoded using a single 'O' bit thanks to the MER feature of storing the length of each bitstream Bi.

Examples in the present disclosure can be provided as methods, systems or machine-readable instructions, such as any combination of software, hardware, firmware or the like. Such machine-readable instructions may be included on a computer readable storage medium (including but not limited to disc storage, CD-ROM, optical storage, etc.) having computer readable program codes therein or thereon.

The present disclosure is described with reference to flow charts and/or block diagrams of the method, devices and systems according to examples of the present disclosure. Although the flow diagrams described above show a specific order of execution, the order of execution may differ from that which is depicted. Blocks described in relation to one flow chart may be combined with those of another flow chart. In some examples, some blocks of the flow diagrams may not be necessary and/or additional blocks may be added. It shall be understood that each flow and/or block in the flow charts and/or block diagrams, as well as combinations of the flows and/or diagrams in the flow charts and/or block diagrams can be realized by machine readable instructions.

The machine-readable instructions may, for example, be executed by a machine such as a general-purpose computer, a platform comprising user equipment such as a smart device, e.g., a smart phone, a special purpose computer, an embedded processor or processors of other programmable data processing devices to realize the functions described in the description and diagrams. For example, machine-readable instructions may, for example, be executed by a machine such as an event signal processor configured to generate event data representing asynchronous changes in intensity of a scene or object to be imaged.

A processor or processing apparatus may execute the machine-readable instructions. Thus, modules of apparatus (for example, an encoder or decoder as described above) may be implemented by a processor executing machine readable instructions stored in a memory, or a processor operating in accordance with instructions embedded in logic circuitry. The term 'processor' is to be interpreted broadly to include a CPU, processing unit, ASIC, logic unit, programmable gate set, or event signal processor etc. The methods and modules may all be performed by a single processor or divided amongst several processors.

Such machine-readable instructions may also be stored in a computer readable storage that can guide the computer or other programmable data processing devices to operate in a specific mode. For example, the instructions may be provided on a non-transitory computer readable storage medium encoded with instructions, executable by a processor.

Figure 1 is a schematic representation of a machine according to an example. The machine 100 can be, e.g., an event signal processor, or an imaging apparatus comprising a sensor. The machine 100 comprises a processor 103, and a memory 105 to store instructions 107, executable by the processor 103. The machine comprises a storage 109 that can be used to store data representing event data, captured image data, encoded data, event frames and so on, as described above. The instructions 107, executable by the processor 103, can cause the machine 100 to encode event data 110 representing events generated using an event sensor 112. According to an example, the instructions 107 are executable by the processor 103, whereby to cause the processor 103 to generate, for a predetermined time interval, an event frame 114 using the event data 110, the event frame 114 defining a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor, and encode the event frame to generate a bitstream 116, the bitstream comprising one or more of a binary sequence defining at least one of a count of events, a count of non-events, and event polarities associated with events of the event frame. Accordingly, the machine 100 can implement a method for generating event data representing asynchronous changes in intensity of a scene or object to be imaged using an imaging apparatus comprising an event sensor.

Such machine-readable instructions may also be loaded onto a computer or other programmable data processing devices, so that the computer or other programmable data processing devices perform a series of operations to produce computer-implemented processing, thus the instructions executed on the computer or other programmable devices provide an operation for realizing functions specified by flow(s) in the flow charts and/or block(s) in the block diagrams.

Further, the teachings herein may be implemented in the form of a computer or software product, such as a non-transitory machine-readable storage medium, the computer software or product being stored in a storage medium and comprising a plurality of instructions, e.g., machine readable instructions, for making a computer device implement the methods recited in the examples of the present disclosure.

In some examples, some methods can be performed in a cloud-computing or network-based environment. Cloud-computing environments may provide various services and applications via the Internet. These cloud-based services (e.g., software as a service, platform as a service, infrastructure as a service, etc.) may be accessible through a web browser or other remote interface of the user equipment for example. Various functions described herein may be provided through a remote desktop environment or any other cloud-based computing environment.

While various embodiments have been described and/or illustrated herein in the context of fully functional computing systems, one or more of these exemplary embodiments may be distributed as a program product in a variety of forms, regardless of the particular type of computer- readable-storage media used to actually carry out the distribution. The embodiments disclosed herein may also be implemented using software modules that perform certain tasks. These software modules may include script, batch, or other executable files that may be stored on a computer-readable storage medium or in a computing system. In some embodiments, these software modules may configure a computing system to perform one or more of the exemplary embodiments disclosed herein. In addition, one or more of the modules described herein may transform data, physical devices, and/or representations of physical devices from one form to another.

Figure 2 is a flow chart of a method according to an example. In the example of figure 2, the method is suitable for generating event data representing asynchronous changes in intensity of a scene or object to be imaged using an imaging apparatus comprising an event sensor. In block 201, an event frame (over a predetermined time interval) is generated using the event data. In an example, the event frame defines a two-dimensional representation comprising multiple pixels, each pixel comprising a ternary value defining one or more characteristics of a set of events for an event volume defined by the predetermined time interval over the sensor. In block 203, the event frame is encoded to generate a bitstream. In an example, the bitstream comprises a binary sequence defining at least one of a number of events, a number of non-events, and event polarities associated with events of the event frame.

Figure 3 is a flow chart of a method according to an example. In the example of figure 3, data representing an event frame, such as the event frame described in relation to figure 2, is used to generate a vector representation of the event frame in block 301. The vector representation can be encoded. In an example, in block 303 the vector representation of the event frame is used to generate respective binary representations for a first component of the vector representation of the event frame. The first component comprises one or more of a first number of consecutive non-events of the event frame, a first number of consecutive events of the event frame, and a first polarity sequence for the first number of consecutive events of the event frame. In block 305, further binary representations for a second component of the vector representation of the event frame can be generated. In an example, the second component can be subsequent to the first component and can comprise one or more of a second number of consecutive non-events of the event frame, a second number of consecutive events of the event frame, and a second polarity sequence for the second number of consecutive events of the event frame.

Figure 4 is a flow chart of a method according to an example. In the example of figure 4, in block 401, the event frame is divided into an array of multiple portions. For each one of the multiple portions: a bitstream comprising a binary sequence defining at least one of a count of events, and a count of non-events is generated in block 403, and in block 405 header data comprising event polarities associated with events of the portion is generated. Figure 5 is a flow chart of a method according to an example. In the example of figure 5 for each pixel, a positive or negative value is assigned in block 501 to each event occurring over the predetermined time interval on the basis of a change in the intensity. In block 503, the values occurring over the predetermined time interval are summed. In block 505 the sign of the sum of values is determined. In block 507, the sign of the sum of values and set symbol of the ternary alphabet are determined.

The preceding description has been provided to enable others skilled in the art to best utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or to be limited to any precise form disclosed. Many modifications and variations are possible without departing from the spirit and scope of the instant disclosure. The embodiments disclosed herein should be considered in all respects illustrative and not restrictive. Reference should be made to the appended claims and their equivalents in determining the scope of the instant disclosure.