CROSS-REFERENCE

[001] This application claims priority to U.S. Application No. 17/336,258 filed on June 1, 2021, and U.S. Provisional Application No. 63/107,923 filed on October 30, 2020, each of which is incorporated herein by reference in its entirety for all purposes.

BACKGROUND

[002] The present disclosure relates to processing remote sensing data, such as data obtained from synthetic aperture radar ("SAR").

[003] Remote sensing data may be or comprise data indicative of one or more of (i) the reflectivity pattern of a zone as detected from returns to one or more sensors and (ii) the subsidence or surface relief of a zone as detected by one or more sensors.

[004] A SAR system can obtain data representing the topography of the surface of the Earth and the objects thereon. The data may comprise amplitude and phase measurements of microwave pulses that are generated by the SAR system, reflected by the surface of the Earth or other objects, and return to the SAR system. Elements in the zone targeted by the SAR system may generate different returns depending on their reflectivity characteristics.

[005] SAR and similar sensing systems generate data that is representative of the topography of a space. Traditionally, to provide some level of visual interpretability, and/or to enable an operator to easily review the sensed data, the sensed data can be transformed to greyscale or color images with different shades of grey or different colors representing different sensed altitudes, different distances from the sensor or different reflectivity characteristics. For example, a wooded region may have a different pattern of scattering than neighboring arable terrain. That difference can be sensed by remote sensing apparatus. The location of the wooded region may be represented in a light shade and the surrounding ground in a dark shade. This can be achieved by roughly mapping the values output by a SAR system (which may take as many as 2.4 billion values in some implementations) into buckets in the visible range. However, such images are nevertheless difficult to interpret. At best, visible imagery of this type only vaguely resembles what a visible color image of the same area might look like. Analysts may be tasked with annotating the SAR imagery to localize and identify objects, but this process may be time consuming and error-prone given the generally low-quality nature of SAR imagery, and artifacts that can be produced by active systems such as layover, foreshortening, or shadowing, when transferred to the visual space in the manner described above.

[006] It would be advantageous to have an improved way of transforming remote sensing data of the sort generated by a SAR system to a more readily interpreted color format.

SUMMARY

[007] According to one aspect there is provided a method for generating a visual color image from remote sensing data comprising: (a) obtaining said remote sensing data; (b) preprocessing said remote sensing data to generate pre-processed remote sensing data; and (c) processing said pre-processed remote sensing data with a generative neural network to generate said visual color image.

[008] In some embodiments, step (b) may comprise applying a clustering algorithm to said remote sensing data.

[009] In some embodiments, step (b) may comprise applying a dimensionality reduction algorithm to said remote sensing data.

[010] In some embodiments, step (b) may comprise normalizing said remote sensing data.

[OH] In some embodiments, the generative neural network may comprise an encoder and a decoder.

[012] In some embodiments, the encoder may comprise one or more fully connected layers.

[013] In some embodiments, the encoder may comprise one or more convolutional layers and one or more adaptive instance normalization (AdalN) layers.

[014] In some embodiments, the decoder may comprise one or more convolutional layers and one or more adaptive instance normalization (AdalN) layers.

[015] In some embodiments, the method may comprise, prior to step (c), training said generative neural network with a discriminative neural network.

[016] In some embodiments, the training may comprise: (1) training said discriminative neural network to classify an input image as a true visual color image or a predicted visual color image generated by said generative neural network; and (2) training said generative neural network to generate predicted visual color images that said discriminative neural network incorrectly classifies as true visual color images. [017] In some embodiments, step (2) may comprise providing a plurality of remote sensing training examples to said generative neural network to generate a plurality of predicted visual color images, providing said plurality of predicted visual color images to said discriminative neural network, and penalizing said generative neural network if said discriminative neural network correctly classifies said predicted visual color images.

[018] In some embodiments, the method may comprise training said generative neural network at least in part on coincident pairs of remote sensing data and visual color images.

[019] In some embodiments, the method may comprise training said generative neural network using self-consistency loss.

[020] In some embodiments, the method may comprise postprocessing said visual color image.

[021] In some embodiments, the method may comprise displaying said visual color image in a user interface. Steps (a) to (c) may be performed for a single input data set in less than 5 seconds.

[022] In some embodiments, the method may comprise providing said visual color image to an image processing pipeline for analysis.

[023] In some embodiments, the remote sensing data may comprise an indication of a location at which said remote sensing data was obtained.

[024] In some embodiments, the generative neural network may receive as input a representation of a desired output domain and a sampled vector from a truncated Gaussian distribution.

[025] According to a second aspect there is provided one or more non-transitory computer storage media storing instructions that are operable, when executed by one or more computers, to cause said one or more computers to perform operations comprising: obtaining remote sensing data; pre-processing said remote sensing data to generate pre- processed remote sensing data; and processing said pre-processed remote sensing data with a generative neural network to generate a visual color image.

[026] According to a third aspect there is provided a system, comprising: one or more computer processors; and memory comprising machine-executable instructions that, upon execution by said one or more computer processors, are configured to generate a visual color image from remote sensing data by: obtaining said remote sensing data; pre-processing said remote sensing data to generate pre-processed remote sensing data; and processing said pre-processed remote sensing data with a generative neural network to generate said visual color image.

[027] In some embodiments, the remote sensing data may be synthetic aperture radar (SAR) data and/or the visual color image may be a red-green-blue (RGB) image.

[028] In some embodiments, the remote sensing data may be or include topographic data. That may be absolute topographic data. That may be relative topographic data.

[029] According to a fourth aspect there is provided an image processing system for generating a visual color representation of a remote sensing data input, the image processing system comprising one or more processors configured to: receive a remote sensing data input representing the topography of a zone; receive an environment input indicating an environment; implement one or more trained neural networks, the network or the networks being trained to colorise a first input in dependence on a second input; and input the remote sensing data input and the environment input to the network(s) and thereby generate an output visual color representation.

[030] In some embodiments, the neural network(s) may comprise a third network adapted to generate the output visual color representation in dependence on a first input and a second input, the first input being a representation in a latent space of remote sensing data and the second input being a representation in the latent space of an environment.

[031] In some embodiments, the neural network(s) may comprise a first network adapted to receive remote sensing data and map it to the latent space.

[032] In some embodiments, the first network may be adapted to downsample the remote sensing data.

[033] In some embodiments, the neural network(s) may comprise a second network adapted to receive an indication of an environment and map it to the latent space.

[034] In some embodiments, the processor(s) may be configured to implement a user interface whereby an operator can designate the environmental input.

[035] In some embodiments, the user interface may comprise a user interface element indicating two or more environments and a multi-position selector movable by the user, the location of which designates the environmental input.

[036] According to a sixth aspect there is provided a method of training an image processing system for generating a visual color representation of a remote sensing data input, the method comprising: repeatedly inputting to one or more neural networks training data pairs, each training data pair comprising (i) a remote sensing data input representing the topography of a zone and (ii) an environment input indicating an environment; causing the one or more neural networks to form an output; inputting the output to a fourth neural network, the fourth neural network being a network trained to output a feedback signal representative of a realism of its input; and applying feedback to the said one or more neural networks in dependence on the feedback signal generated by the fourth neural network.

[037] In some embodiments, the method may comprise gathering the remote sensing data using a SAR unit. The remote sensing data may comprise absolute or relative altitude or topographic data.

[038] In some embodiments, the remote sensing data may be SAR data. The remote sensing data may be derived from a SAR sensing system. The remote sensing data may be or comprise topographic data. The topographic data may be relative topographic data. The topographic data may be relative topographic data. The topographic data may be relief texture data. The remote sensing data may be data indicative of the relief, topography or physical texture of a region of the Earth.

[039] The present disclosure further provides systems and methods for generating visible color images from remote sensing data such as synthetic aperture radar ("SAR") data. The visible color images may be red-green-blue ("RGB") images. These may contain red, green and blue visible elements and/or may be defined by red, green and blue components.

[040] In an aspect, the present disclosure provides a method for generating an RGB image from SAR data, comprising, (a) obtaining the SAR data; (b) pre-processing the SAR data to generate preprocessed SAR data; and (c) processing the pre-processed SAR data with a generative neural network to generate the RGB image. In some embodiments, (b) comprises applying a clustering algorithm to the SAR data. In some embodiments, (b) comprises applying a dimensionality reduction algorithm to the SAR data. In some embodiments, (b) comprises normalizing the SAR data. In some embodiments, the generative neural network comprises an encoder and a decoder. In some embodiments, the encoder comprises one or more fully connected layers. In some embodiments, the encoder comprises one or more convolutional layers and one or more adaptive instance normalization (AdalN) layers. In some embodiments, the decoder comprises one or more convolutional layers and one or more adaptive instance normalization (AdalN) layers. In some embodiments, prior to (c), training the generative neural network with a discriminative neural network. In some embodiments, the training comprises: (1) training the discriminative neural network to classify an input image as a true RGB image or a predicted RGB image generated by the generative neural network; and (2) training the generative neural network to generate predicted RGB images that the discriminative neural network incorrectly classifies as true RGB images. In some embodiments, (2) comprises providing a plurality of SAR training examples to the generative neural network to generate a plurality of predicted RGB images, providing the plurality of predicted RGB images to the discriminative neural network, and penalizing the generative neural network if the discriminative neural network correctly classifies the predicted RGB images. In some embodiments, the method further comprises training the generative neural network at least in part on coincident pairs of SAR data and RGB images. In some embodiments, the method further comprises training the generative neural network using self-consistency loss. In some embodiments, the method further comprises postprocessing the RGB image. In some embodiments, the method further comprises displaying the RGB image in a user interface, wherein (a) — (c) are performed in less than 5 seconds. In some embodiments, the method further comprises providing the RGB image to an image processing pipeline for analysis. In some embodiments, the SAR data comprises a location at which the SAR data was obtained. In some embodiments, the generative neural network receives as input a representation of a desired output domain and a sampled vector from a truncated Gaussian distribution.

[041] Another aspect of the present disclosure provides a non-transitory computer readable medium comprising machine executable code that, upon execution by one or more computer processors, implements any of the methods above or elsewhere herein.

[042] Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein.

[043] Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in this art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive. INCORPORATION BY REFERENCE

[044] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and/or take precedence over any such contradictory material.

BRIEF DESCRIPTION OF THE DRAWINGS

[045] The systems and methods of the present disclosure will now be described by way of example with reference to the accompanying drawings (also referred to herein as “Figure” or “FIG ”). In the drawings:

[046] Figure 1 schematically illustrates a system for generating red-green-blue ("RGB") images from synthetic aperture radar ("SAR") data, according to some embodiments of the present disclosure;

[047] Figure 2 is a flow chart of a process for training a generative adversarial network, according to some embodiments of the present disclosure;

[048] Figure 3 is a flow chart of a process for generating RGB images from SAR data, according to some embodiments of the present disclosure;

[049] Figure 4 shows a computer system that is programmed or otherwise configured to implement methods provided herein;

[050] Figure 5 schematically illustrates a generative network of the system of Figure 1, according to some embodiments of the present disclosure; and

[051] Figure 6 shows a flow diagram representing the processing of remote sensing data to generate an output image

DETAILED DESCRIPTION

[052] While various embodiments of the invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.

[053] Whenever the term "at least," "greater than" or "greater than or equal to", "no more than," "less than," or "less than or equal to" or similar terms precedes a series of two or more numerical values, that term applies independently to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2 or 3 means that the value in question may be greater than or equal to 1 or greater than or equal to 2 or greater than or equal to 3.

[054] The term "synthetic aperture radar" ("SAR"), as used herein, generally refers to a radar system that employs the motion of the system's transmitting and/or receiving antenna over a subject region. Such motion may provide better spatial resolution than a conventional radar system having one or more substantially fixed antennas. A SAR system may have an antenna. The SAR system may be mounted on a mobile platform such as an aircraft or satellite. The distance that the SAR system travels (e.g., on the aircraft or satellite) during the round-trip time of a radar pulse may be taken to define the system’s synthetic aperture. That may be the receiving cross-section of the antenna. The synthetic aperture may be much larger the antenna's physical aperture, for example more than 10 or 100 times the physical aperture. A relatively large synthetic aperture may allow the SAR system to generate relatively high-resolution data, for example in the form of a topographical map.

[055] The present disclosure provides systems and methods for generating visible multi-color images from SAR data. The images may be RGB images. A SAR system mounted on an aircraft or satellite can obtain SAR data representing the topography of the surface of the Earth and the objects thereon. The SAR data may comprise amplitude and phase measurements of microwave pulses that are generated by the SAR system, reflected by the surface of Earth (or other objects), and return to the SAR system. The reflected returns that are sensed by the SAR system may represent the topography of the subject zone by virtue of their being dependent on the reflectivity (in intensity and/or phase) of parts of the subject zone. The reflected returns that are sensed by the SAR system may represent the topography of the subject zone by virtue of their being dependent on the distance of parts of the subject zone from the antenna.

[056] The system may process the remote sensing data generated by (for example) a SAR system to generate a multicolor representation thereof. In one example, the multicolor representation may be a natural-color representation of objects represented in the remote sensing data. For example, instead of showing a tree represented in the remote sensing data in a color or shade that represents its reflectivity or height, the tree may be shown in a green that imitates how the tree might be expected to appear. This can help an analyst to interpret the remote sensing data. As will be discussed below, other color mappings are possible. For example, some elements may be colored according to their expected natural coloring and other elements may be given an a coloring that is expected to be unnatural, so as to highlight the latter elements.

[057] Figure 1 schematically illustrates a system 100 for generating a color image of remote sensing data. The system 100 may have a SAR system 110, a pre-processing module 120, a generative adversarial network (GAN) 130, and a postprocessing module 140.

[058] The SAR system 110 may be mounted on a mobile platform, for example an aerial vehicle such as an aircraft or satellite. The SAR system uses synthetic aperture radar to estimate the topography of a subject area. The SAR system 110 can obtain SAR data that represents the topography of a surface (e.g. the surface of the Earth) and objects on the surface. The SAR system 110 can obtain such data by transmitting successive pulses of radio waves and receiving and recording their reflections. The SAR system 110 can process the SAR data to generate a SAR image of the surface. The SAR image may be a map of radar reflectivity, including amplitude and optionally phase. The raw SAR image may be more difficult for a human to interpret than an RGB image. For example, although the raw SAR image may resemble an oblique photograph, it may not resemble a photograph taken from the location of the radar.

[059] Additionally, certain radar returns may be overlaid on each other, and objects in motion may appear to be in incorrect locations due to the Doppler effect. Therefore, it may be beneficial to convert SAR data directly into an RGB image that a human can more easily interpret.

[060] The preprocessing module 120 can receive the SAR data from the SAR system 110 and normalize it to a range of -1 to 1 by subtracting the mean of the SAR data and dividing by the standard deviation. The preprocessing module 120 may also perform clustering and/or dimensionality reduction on the SAR data to discover distributions of textures and patterns to inform the colorization process. Clustering may involve applying a clustering algorithm to the SAR data to generate groups of similar data (e.g. patterns) within the SAR data. The clustering algorithm may be, for example, a centroid-based clustering algorithm (e.g. k-means), a hierarchical clustering algorithm, a distributionbased clustering algorithm, a density-based clustering algorithm, or the like. [061] Dimensionality reduction may involve applying a dimensionality reduction algorithm to the SAR data to transform it from a high-dimensional space with many features to a low-dimensional space with fewer features, which may make the SAR data easier to process. The dimensionality reduction algorithm may be, for example, an encoder, t- distribution stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection (UMAP), or the like.

[062] In some circumstances, the preprocessing module 120 may also perform three- dimensional (3D) modelling and simulation on the SAR data to allow for the viewing of the SAR data from various angles to gain understandability about a scene.

[063] The GAN 130 can receive the normalized SAR data from the preprocessing module 120 and process it to generate an RGB image. The GAN 130 may have a generative network 132 and a discriminative network 134, as shown in Figure 1. The generative network 132 can generate the RGB image. The generative network 132 may be a neural network. The neural network may be a feedforward neural network, a convolutional neural network (CNN), or the like. In some embodiments, the generative network 132 is an autoencoder. The autoencoder may have an encoder that is configured to generate a latent space representation of the normalized SAR data. The autoencoder may also have a decoder that is configured to generate an RGB image from the latent space representation. The autoencoder may include feedforward, convolutional, or recurrent layers. In some embodiments, the autoencoder may be a fully convolutional autoencoder.

[064] The discriminative network 134 can train the generative network 132. The training process will be described in detail in reference to Figure 2. The discriminative network 134 may be a classifier. The discriminative network 134 may be a neural network.

[065] Figure 5 schematically illustrates an alternative embodiment of a generative network that may be used in the GAN 130 of Figure 1. The generative network of Figure 5 may have an encoder 510 and a network 520. The encoder 510 may receive a representation of the desired output domain of the network 520 (Input 1) and a sampled vector from a truncated Gaussian distribution (Input 2). The truncated Gaussian distribution may be a distribution of textures and patterns in the SAR data discovered during pre-processing. Inputs 1 and 2 may provide context about what the output of the network 520 should look like. In some embodiments, the encoder 510 may be a neural network with one or more fully connected layers. In other embodiments, the encoder 510 may be a neural network with one or more convolutional layers and one or more adaptive instance normalization (AdalN) layers.

[066] The network 520 may have a first sub-network 522 and a second sub-network 524. The first sub-network 522 may be an encoder. In some embodiments, the first sub-network 522 may have one or more convolutional layers and one or more AdalN layers. The first sub-network 522 may receive SAR data (Input 3) and generate a latent space representation of the SAR data. The latent space representation may then be appended to the output of the encoder 510. The second subnetwork 524 can process the latent space representation and the output of the encoder 522 to generate an RGB image. The second sub-network 524 may be a decoder. The second sub-network 524 may have one or more convolutional layers and one or more AdalN layers. In alternative embodiments, the network 520 instead receives RGB data and generates SAR data.

[067] The postprocessing module 140 can receive the RGB image from the generative network 134 and undo the effect of the preprocessing module 110, e.g. by multiplying the RGB image by the standard deviation of the SAR data and adding the result to the mean of the SAR data. The postprocessing module 140 can also undo any preprocessing performed by the preprocessing module 120 (e.g. clustering or dimensionality reduction). The resulting RGB image may be transmitted to an analyst in real-time or substantially real-time for review. Alternatively or additionally, the resulting RGB image may be provided to an image processing pipeline that is configured to classify objects in the RGB image.

[068] The system of Figure 1 and its components can be implemented on one or more computing devices. The computing devices can be servers, desktop or laptop computers, electronic tablets, mobile devices, or the like. The computing devices can be located in one or more locations. The computing devices can have general-purpose processors, graphics processing units (GPU), application-specific integrated circuits (ASIC), field-programmable gate-arrays (FPGA), or the like. The computing devices can additionally have memory, e.g. dynamic or static random-access memory, readonly memory, flash memory, hard drives, or the like. The memory can be configured to store instructions that, upon execution, cause the computing devices to implement the functionality of the components. The computing devices can additionally have network communication devices. The network communication devices can enable the computing devices to communicate with each other and with any number of user devices, over a network. The network can be a wired or wireless network. For example, the network can be a fiber optic network, Ethernet® network, a satellite network, a cellular network, a Wi-Fi® network, a Bluetooth® network, or the like. In other implementations, the computing devices can be several distributed computing devices that are accessible through the Internet. Such computing devices may be considered cloud computing devices. The system may comprise one or more memories storing in non-transient form code executable by one or more processors of the said computing device or devices to perform the functions described herein.

[069] A SAR system is an example of a remote sensing system. A remote sensing system may comprise a system of one or more sensors that can sense the nature of objects or terrain from distance: for example, from an aerial or space vehicle over the terrain. The distance between the sensor and the objects or terrain being sensed may be more than 500m or more than 1km, or more than 2km or more than 15,000km. RADAR and LIDAR are other examples of remote sensing systems. Data that can be captured by a remote sensing system and that can be used as input to the data interpretation system described herein includes topographic data. Topographic data describes the absolute or relative altitude of the terrain in a region. The terrain includes exposed ground and structures such as buildings and vegetation over the ground. When a remote sensing system carried by a platform such as a vehicle senses the distance between the platform and two neighbouring points on the terrain, that directly indicates the relative altitude of those points. Such a relative altitude - altitude relative to neighbouring terrain - can be derived irrespective of whether the absolute altitude of the platform is known. If the absolute altitude of the platform is known then the sensed relative altitudes can be converted to absolute altitudes. Relative altitude data can indicate features such as relief texture, which can be indicative of the nature of the ground or its cover. For example, the relief texture of a wooded area will typically comprise short-scale variation in altitude that differs from that of open ground or an urban area. Relative altitude features can also pick out boundaries of topographical features such as the edges of wooded areas or the path of a river.

[070] Figure 2 is a flow chart of process 200 for training the GAN 130 of Figure 1. The process can be performed by a system of one or more appropriately programmed computers in one or more locations.

[071] In an operation 210, the system can train the discriminative network 134 to classify inputs as being true RGB images (e.g. aerial images of the Earth taken by a digital photograph) or RGB images generated by the generative network 132. The training data may include both true RGB images and RGB images generated by the generative network 132. The true RGB images and generated RGB images may be labelled as such, enabling the system to train the discriminative network 134 through a supervised learning process. The loss function of the discriminative network 134 may be the error of the network in classifying the training inputs. The supervised learning process may involve computing the gradient of the loss function with respect to the weights of the network and updating those weights accordingly. During operation 210, the generative network 132 is not trained.

[072] In an operation 220, the system can train the generative network 132 to generate RGB images from SAR data. Training the generative network 132 may involve providing SAR data to the generative network 132 to produce an output, providing that output to the discriminative network 134 to classify it as a true RGB image or a generated RGB image and computing the loss of the generative network 132. The loss function of the generative network 1132 may penalize the generative network 132 for failing to fool the discriminative network 134. The system can compute the gradient of the loss function with respect to the weights of the generative network 132 via backpropagation through both networks. However, during operation 220, only the weights of the generative network 132 are actually updated. The loss function may be a mean-square error function or the like. The loss function may have a regularization (e.g. LI or L2 normalization) term.

[073] In some embodiments, training the GAN 130 may involve a supervised learning process in which SAR data is provided to the generative network 132 to generate a predicted RGB image and the predicted RGB image is compared to an actual RGB image of the same location as the SAR data. The error between the two can be incorporated into the loss function. The error may be computed using a perceptual loss function, mutual information loss function, a cross-correlation loss function, an LI loss function, an L2 loss function or the like. In some cases, however, coincident SAR data and RGB images may not be available. In such cases, self-consistency loss, such as cycle loss, may be incorporated into the loss function of the generative network 132. Self-consistency loss may be calculated by providing SAR data to the generative network 132 to generate a predicted RGB image and providing the predicted RGB image to a second generative network that is trained in tandem with the generative network 132. The second generative network may be configured to generate predicted SAR data from the predicted RGB image. The self-consistency loss of the generative network 132 may be the difference between the original SAR data and the predicted SAR data. The selfconsistency loss may be incorporated into the loss function of the generative network 132. In some embodiments, the generative network 132 may instead be trained to do predict SAR data from an RGB image and a second generative network is not needed. The system can repeat operations 210 and 220 several times, until convergence of the generative network 132.

[074] In some cases, the true RGB images and SAR data used in the training process described above may be tagged with metadata that indicates, for example, the location at which the images were captured, or the SAR data was obtained. Such metadata may improve the performance of the generative network 132 in generating realistic RGB images. For example, a generated RGB image for a given location may be more appropriately colored based on the landscape typically found in that location.

[075] Figure 3 is a flow chart of a process 300 for generating an RGB image from SAR data. The process can be performed by the system 100 of Figure 1. In an operation 310, the system can obtain SAR data. In an operation 320, the system can preprocess the SAR data. In one embodiment, preprocessing involves normalization. Normalization may involve subtracting a mean of the SAR data and dividing by a standard deviation of the SAR data. In an operation 330, the system can process the preprocessed SAR data with a generative network (e.g. the generative network 132) to generate an RGB image. In an operation 340, the system can postprocess the RGB image, e.g. by multiplying by the standard deviation of the SAR data and adding the mean of the SAR data to undo the preprocessing.

[076] As indicated above, the present system may be configured to receive remote sensing data indicative of the topography of a zone and may process that data to generate a visual color output. The remote sensing data may comprise an array of numeric data points. Each data point may be a pixel in a map representing the remote sensing data. The array and/or map may be two-dimensional. Each data point may represent the remote sensing nature of a corresponding location in the zone. The remote sensing nature may be indicative of that location’s reflectivity, height or distance from a transmitter and/or receiver. The remote sensing data may be generated by any suitable sensor system. Non-limiting examples of suitable sensor systems include SAR, conventional radar and sensors that sense ambient radiation in the non-visible spectrum. The visual color output is an image colored in a plurality of colors in the visible spectrum. The visual color output may comprise an array of pixels in two dimensions. Each pixel may be indicative of the color of the respective location in an output image. The color may be represented by RGB values or in any other suitable model or format, for example CMY (cyan/magenta/yellow), CMYK (cyan/magenta/yellow/black), HSL (hue, saturation, lightness) or HSV (hue, saturation, value).

[077] The visual color output may be generated in such a way that over at least part of its area it can be expected to represent a natural-color image of the subj ect of the remote sensing data. For instance, if a part of the remote sensing data represents an object whose natural color might be expected to be green then that part of the output may be represented in green. Some ways in which this can be done will now be described in more detail.

[078] Figure 6 shows steps in the processing of SAR data to generate a visible color image.

[079] At a high level, the system operates as follows. A received remote sensing image 600, such as a SAR image, is transformed by a first neural network 601 to a latent space of a third neural network 603. The output of the first neural network is a first input 610. A second neural network 602 generates a second input 611 in the latent space of the third neural network 603. The significance of the second neural network will be described further below. Then the third neural network transforms the first and second inputs to an output 612. The output 612 is a visible colored representation of the input image 600.

[080] The SAR image may be processed to reduce and/or filter the image prior to it being input to the first neural network. The first neural network is a network that has been trained in a manner to be described below to transform the SAR image to the latent space of the third neural network.

[081] The second neural network receives an input that is representative of a biome. Examples of biomes include desert, temperate forest, tropical forest and littoral. In different biomes objects of similar remote sensing characteristics may tend to have a different appearance. For example, a hill may be a buff color in a desert biome or dark green in a temperate forest biome. A raised feature the size of a house may have typically have a red roof in a European urban biome and a grey roof in an Asian urban biome. Input provided by an operator or an automatic control unit to the second neural network may take one of two forms. In one example it may be a visible color image of a location in a biome for which the output image is desired to be colored. The image may not or need not be of the same location as the input remote sensing image 600. The second neural network is trained to transform that image to coordinates in the latent space of the third neural network. In a second example, the input provided by an operator or an automatic control unit may be a value that indicates a desired biome. For example, a user may be presented with a user interface element such as a slider or a series of photographs which allows the user to designate a biome from a range of biomes. The photographs may depict different biomes and a user may designate one such biome using a user interface as input to the second neural network. Textual descriptors may be used instead of photographs. Or a user interface element 608 such as a one or two-dimensional slider may allow a user to select a biome in a range between two or more biomes that are represented on the user interface by images or text. For example, the slider may extend from a depicted desert biome to a depicted tropical rainforest biome and the user may designate a biome of a desired aridity or degree of vegetation by positioning the slider at a desired location between its endpoints. Other ranges of biomes may be presented on a slider such as a range of seasons, latitudes or altitudes. Using one or more such sliders a user can designate a desired biome as input to the second neural network. As discussed above, once a biome or a form or style of biome has been designated to the second neural network, the second neural network transforms that designation to coordinates in the latent space of the third neural network.

[082] The third neural network operates so as to transform its input that is dependent on the input remote sensing data in dependence on its input that is dependent on a designated biome or type of biome. It may be considered to color or style the input remote sensing data in dependence on the designated biome or type of biome.

[083] The first and second neural networks may reduce their respective input data to form their respective outputs. To achieve this, those networks may be trained in any suitable way. For example, they may be trained end-to-end with the third neural network, or they may be trained individually so as to form similar outputs for similar inputs. The first to third neural networks may be trained in combination by inputting training data comprising pairs of a remote sensing image and a visual image of a location in a biome where the remote sensing image was captured. For this purpose the visual image need not be of the same location depicted in the remote sensing image; or it may be of the same location but taken at a different time - for example in a different season or before changes to the environment had been made. This makes it easier to acquire training data. The visual image is conveniently an aerial photograph. Feedback to the weights of the first to third neural networks may be derived from the degree of difference between the (i) image output from the third neural network in dependence on the training data and (ii) a reference image. In one example, the reference image may be a visual aerial photograph of a location spatially coincident with the location depicted by the remote sensing data of the training data. In another example, the reference image may be a visual image of a location in the same biome as the location depicted by the remote sensing data of the training data. In another example, the feedback to the first to third neural networks may be generated by a fourth neural network 604. The fourth neural network is shown by a dashed line in Figure 6 to indicate that optionally it may only be used during the training phase.

[084] The fourth neural network is a discriminative network which can act in an adversarial way during training of the first to third neural networks. The fourth neural network is trained to generate an output that represents whether an image input to it has a realistic appearance. The fourth neural network may be trained on the reference image and the second neural network may be trained on a set of training data comprising images captured in the real world and also on real world images that have been visually transformed. The fourth neural network is trained to discriminate between the two sets of images. Then during training of the first to third neural networks, the fourth neural network can provide feedback that indicates whether the output of the third neural network looks realistic.

[085] The first and second neural networks may implement any suitable architecture. For example, they may implement a CycleGAN architecture. The third neural network may be implemented in any suitable way. One suitable implementation is as an autoencoder. The third neural network may implement a U-Net architecture. The third neural network may perform successive downsamplings of the data input to it that is dependent on the input remote sensing image, followed by successive upsamplings of the downsampled data, with weighted links between downsampling and upsampling stages.

[086] A remote sensing image, or remote sensing data, may be considered to be a representation of the topography of a geographical zone or area. It may be generated by a SAR unit or by other mechanisms such as LIDAR, other radar systems or photogrammetry.

[087] A visual image may be considered to be an image having detail in the visual spectrum. It may be a colored image or a black-and-white image or a grayscale image. Colors of the image may be designated using the RGB model or another suitable model. [088] In examples of the approaches described above, data gathered by remote sensing of a region of terrain can be transformed to a visual representation of that terrain by means of a model derived from terrain that is not adjacent to or surrounding the sensed region. The model can be derived from terrain that is remote from the sensed region. The visual output can provide a representation of a region of terrain. The visual representation of the whole region of the output can be derived from the remote sensing data. The process of forming the visual output may comprise a step of selecting a model by reference to which the visual representation may be formed. That may, for example be a model of a selected biome. That model may be selected manually. It may be selected without dependence on any visual representation of terrain adj acent to or surrounding the sensed region.

[089] Neural networks

[090] The present disclosure describes various types of neural networks. Neural networks may employ multiple layers of operations to predict one or more outputs from one or more inputs. Neural networks may have one or more hidden layers situated between an input layer and an output layer. The output of each layer may be the input to another layer, e.g. the next hidden layer or the output layer. Each layer of a neural network may specify one or more transformation operations to be performed on the input to the layer. Such transformation operations may be referred to as neurons. The output of a particular neuron may be a weighted sum of the inputs to the neuron, adjusted with a bias and multiplied by an activation function, e.g. a rectified linear unit (ReLU) or a sigmoid function. Neural networks of the type described herein may alternatively be known as weighted processing networks. A weighted processing network may store one or more weights at one or more levels and may apply those weights discriminatively to its inputs to form an output. The weights may be determined by an iterative training process.

[091] Training a neural network may involve providing inputs to the untrained or partially neural network to generate predicted outputs, comparing the predicted outputs to expected outputs, and updating the algorithm's weights and biases to account for the difference between the predicted outputs and the expected outputs. Specifically, a cost function may be used to calculate a difference between the predicted outputs and the expected outputs. By computing the derivative of the cost function with respect to the weights and biases of the network, the weights and biases may be iteratively adjusted over multiple cycles to minimize the cost function. Training may be complete when the predicted outputs satisfy a convergence condition, e.g. a small magnitude of calculated cost as determined by the cost function.

[092] CNNs are neural networks in which neurons in some layers, called convolutional layers, receive input from only a small portion of the input data set (e.g. a short time segment of a speech data). These small portions may be referred to as the neurons' receptive fields. Each neuron in such a convolutional layer can have the same weights. In this way, the convolutional layer can detect certain features in any portion of the input data set. CNNs may also have pooling layers that combine the outputs of neuron clusters in convolutional layers and fully connected layers that are similar to traditional layers in a feed-forward neural network.

[093] Computer systems

[094] The present disclosure provides computer systems that are programmed to implement methods of the disclosure. Figure 4 shows a computer system 401 that is programmed or otherwise configured to implement the system 100 or perform processes 200 or 300. [095] The computer system 401 includes a central processing unit (CPU, also "processor" and “computer processor" herein) 405, which can be a single core or multi core processor, or a plurality of processors for parallel processing. The computer system 401 also includes memory or memory location 410 (e.g. random-access memory, read-only memory, flash memory), electronic storage unit 415 (e.g. hard disk), communication interface 420 (e.g. network adapter) for communicating with one or more other systems, and peripheral devices 425, such as cache, other memory, data storage and/or electronic display adapters. The memory 410, storage unit 415, interface 420 and peripheral devices 425 are in communication with the CPU 405 through a communication bus (solid lines), such as a motherboard. The storage unit 415 can be a data storage unit (or data repository) for storing data. The computer system 401 can be operatively coupled to a computer network ("network") 430 with the aid of the communication interface 420. The network 430 can be the Internet, an internet and/or extranet, or an intranet and/or extranet that is in communication with the Internet. The network 430 in some cases is a telecommunication and/or data network. The network 430 can include one or more computer servers, which can enable distributed computing, such as cloud computing. The network 430, in some cases with the aid of the computer system 401, can implement a peer-to-peer network, which may enable devices coupled to the computer system 401 to behave as a client or a server. [096] The CPU 405 can execute a sequence of machine-readable instructions, which can be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 410. The instructions can be directed to the CPU 405, which can subsequently program or otherwise configure the CPU 405 to implement methods of the present disclosure. Examples of operations performed by the CPU 405 can include fetch, decode, execute, and writeback.

[097] The CPU 405 can be part of a circuit, such as an integrated circuit. One or more other components of the system 401 can be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).

[098] The storage unit 415 can store files, such as drivers, libraries and saved programs. The storage unit 415 can store user data, e.g. user preferences and user programs. The computer system 401 in some cases can include one or more additional data storage units that are external to the computer system 401, such as located on a remote server that is in communication with the computer system 401 through an intranet or the Internet.

[099] The computer system 401 can communicate with one or more remote computer systems through the network 430. For instance, the computer system 401 can communicate with a remote computer system of a user (e.g. a computer system with a user interface for reviewing RGB images). Examples of remote computer systems include personal computers (e.g. portable PCs), slate or tablet PCs (e.g. Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g. Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user can access the computer system 401 via the network 430.

[100] Methods as described herein can be implemented by way of machine (e.g. computer processor) executable code stored on an electronic storage location of the computer system 401, such as, for example, on the memory 410 or electronic storage unit 415. The machine executable or machine-readable code can be provided in the form of software. During use, the code can be executed by the processor 405. In some cases, the code can be retrieved from the storage unit 415 and stored on the memory 410 for ready access by the processor 405. In some situations, the electronic storage unit 415 can be precluded, and machine-executable instructions are stored on memory 410.

[101] The code can be pre-compiled and configured for use with a machine having a processer adapted to execute the code or can be compiled during runtime. The code can be supplied in a programming language that can be selected to enable the code to execute in a pre-compiled or as-compiled fashion

[102] Aspects of the systems and methods provided herein, such as the computer system 401 can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and/or associated data that is carried on or embodied in a type of machine readable medium. Machine-executable code can be stored on an electronic storage unit, such as memory (e.g. read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[103] Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution. 100491 The computer system 401 can include or be in communication with an electronic display 435 that comprises a user interface (UI) 440. Examples of UIs include, without limitation, a graphical user interface (GUI) and webbased user interface.

[104] Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 405. The algorithm can, for example, an algorithm for training the GAN 130.

[105] While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. It is therefore contemplated that the invention shall also cover any such alternatives, modifications, variations or equivalents. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.