Cross-Reference to Related Applications

[0001] The present application claims priority from Australian Provisional Patent Application No 2016901760 filed on 11 May 2016, the content of which is incorporated herein by reference.

Technical Field

[0002] This disclosure relates to predicting solar irradiance. For example, the predicted solar irradiance can be used for controlling solar systems.

Background

[0003] Solar irradiance is a natural resource that is commonly harvested using solar photovoltaics or solar thermal systems. This natural resource is highly intermittent mainly due to clouds occluding the sun. Current systems are often not at the optimal operation point because it is difficult to predict the solar irradiance.

[0004] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each claim of this application.

Summary

[0005] A method for predicting solar irradiance comprises:

capturing multiple training sky images by an image sensor;

receiving object location data in relation to the multiple training sky images, the object location data being indicative of a location in the training sky images of an object that may affect the solar irradiance; determining one or more parameter values based on the object location data, the parameter values being indicative of a relationship between the location of the object and the solar irradiance; and

determining a predicted solar irradiance value based on the parameter values.

[0006] Capturing the multiple training sky images may be at predetermined locations, orientations and lens configurations of the image sensor. The predetermined locations, orientations and lens configurations of the image sensor may be one constant location, orientation and lens configuration of the image sensor.

[0007] Receiving the object location data may comprise:

generating a display of the multiple training sky images; and

receiving the object location data as user input data in relation to the multiple training sky images, the user input data being indicative of a location in the training sky images of an object that may affect the solar irradiance.

[0008] Capturing multiple training sky images may comprise capturing multiple training sky images for different positions of the sun. The object that may affect the solar irradiance may be the sun and the object location data is indicative of a location of the sun in the multiple training sky images. The one or more parameter values may be indicative of a location, orientation and lens configuration of the image sensor.

Determining the predicted solar irradiance value may comprise determining a predicted position of the sun at a prediction time based on the parameter values and determining the predicted solar irradiance based on the predicted position of the sun.

[0009] It is an advantage that the parameter values can be determined using a small number of identified sun locations, allowing the fast and accurate setup of a model for a camera of arbitrary fisheye characteristics, tilt, and orientation.

[0010] The object that may affect the solar irradiation may bea cloud cover and the object location data maybe indicative of a classification of pixels of the multiple training sky images, the classification being in relation to a cloud cover in the sky images. The one or more parameter values may be parameter values of a supervised learning classifier based on the user input data. Determining the predicted solar irradiance value may comprise:

capturing a test sky image;

applying the supervised learning classifier to a pixel of the test sky image to determine a classification in relation to a cloud cover for the pixel , and

determining the predicted solar irradiance value based on the

classification.

[0011] It is an advantage that the supervised learning classifier classifies clouds accurately. In particular, the supervised learning classifier classifies cloud edges and thin and distant clouds accurately. The human-based training further allows the use of inexpensive cameras with low dynamic range, lossy compression and low resolution.

[0012] The classification in relation to the cloud cover may be based on a set of three labels including thick, medium and thin, or a set of two labels including cloud and clear sky.

[0013] The supervised learning classifier may be an ensemble classifier comprising multiple-sub classifiers, and determining the classification may comprise applying the ensemble classifier to a pixel of the test sky image to calculate for each of the multiple sub-classifiers an output classification and determining the classification for the pixel based on the output classification of each of the multiple sub-classifiers.

[0014] The ensemble classifier may bea decision forest and the sub-classifiers are decision trees.

[0015] Determining the ensemble classifier may comprise determining nodes and edges of the decision forest.

[0016] Determining the classification in relation to a cloud cover for the pixel may comprise determining the classification based on a Red-Blue Ratio classifier. [0017] It is an advantage that the Red-Blue Ratio classifier classifies overcast and dark clouds accurately. As a result, combining the Red-Blue Ratio classifier with the ensemble classifier leads to accurate classification of most clouds.

[0018] The method may further comprise:

capturing multiple test sky images; and

applying the supervised learning classifier to pixels of the multiple test sky images to determine a classification in relation to a cloud cover for each of the multiple test sky images;

wherein determining the predicted solar irradiance value comprises determining a movement of cloud pixels and determining the predicted solar irradiance value based on the movement of the cloud pixels.

[0019] The advantage of determining cloud movement is that future shading events can be predicted accurately.

[0020] Determining the movement may comprise determining a motion vector of the pixels classified as having a cloud cover that is above a predetermined threshold.

[0021] The method may further comprise determining a temporal extrapolation based on the motion vector, wherein determining the predicted performance comprises determining a predicted sun occlusion event based on the extrapolation.

[0022] Determining the predicted sun occlusion event may comprise determining a predicted time value or a predicted magnitude value or both for the predicted sun occlusion event.

[0023] Determining the motion vector of the pixels may comprise determining the motion vector for each pixel separately. [0024] The advantage of determining the motion vector for each pixel separately is that the method can capture any number of independently moving clouds/cloud layers and predict their impact on irradiance separately.

[0025] The method may further comprise:

capturing multiple test sky images; and

applying the supervised learning classifier to pixels of the multiple test sky images to determine a classification in relation to a cloud cover in the sky images for the pixels of each of the multiple test sky images,

wherein determining the predicted solar irradiance value comprises determining cloud formation based on the classification and determining the predicted solar irradiance value based on the cloud formation.

[0026] It is an advantage that shading events that are caused by newly formed clouds can be predicted accurately when the cloud formation warnings are available.

[0027] The method may further comprise:

determining a cloud altitude based on captured test sky images;

determining a characteristic value of a shadow cast by a cloud at the cloud altitude,

wherein determining the predicted solar irradiance value comprises determining the predicted solar irradiance value based on the shadow.

[0028] Determining the predicted solar irradiance value may comprise determining one or more of:

presence of clouds within a predetermined viewing angle of sun;

shading-time predictions; and

solar power generation.

[0029] A method for operating a network of multiple cameras to predict irradiance comprises:

performing the above method for each of the multiple cameras of the network; and

combining the classifications or predictions to generate location data indicative of a cloud cover and cloud characteristics for the area covered by the multiple cameras.

[0030] The method may further comprise determining a predicted performance of a photovoltaic system based on the predicted solar irradiance value.

[0031] The method may further comprise generating a control signal to control the photovoltaic system based on the predicted performance.

[0032] The method may further comprise:

receiving measurement data indicative of the actual performance of the photovoltaic system;

determining a difference between the predicted performance and the actual performance;

determining configuration parameters of the system to reduce the difference between the predicted performance and the actual performance.

[0033] The configuration parameters may comprise position and orientation of the image sensor.

[0034] Generating a control signal may comprise one or more of:

generating a control signal to control an inverter output to adjust ramp-rates in an electrical grid;

controlling energy storage charge and discharge rates and timing;

controlling the rate and timing of energy consumption of electrical devices; controlling electrical devices to provide voltage, frequency and network services to the electrical grid; and

controlling a generator to compensate for a reduction of performance caused by a cloud cover reducing irradiance of the photovoltaic system. [0035] It is a problem with existing methods that there is a delay between a change in performance of a photovoltaic system and the adaptation of the electrical grid in response to the change of performance. Therefore, it is an advantage that the overall stability and reliability of the grid can be increased when generators are started or inverters limited before the actual performance of the photovoltaic system changes. That is, the generators and inverters are controlled based on the predicted performance instead of the actual performance. Similarly, battery storage charge and discharge rates can be optimised, demand-response actions can be taken, and ancillary services can be provided to the market more effectively using the predicted solar performance.

[0036] Capturing the training sky images may comprise capturing multiple images at different exposure parameters and combining the multiple images into a single training sky image.

[0037] Determining a predicted performance of the photovoltaic system may comprise determining a predicted performance of the photovoltaic system based on historical performance data.

[0038] The image sensor may generate the training sky images or the test sky images or both with an analog to digital resolution of 8 bit or less.

[0039] It is an advantage that 8 bit cameras are less expensive than high dynamic range cameras and capturing training sky images to determine a predicted performance results in a satisfactory accuracy even at the low resolution of 8 bit or less. It is also an advantage that compressed images from 8 bit cameras are smaller in size and can be transmitted or stored more quickly and cheaply than images from high dynamic range cameras.

[0040] The image sensor may generate the training sky images or the test sky images or both with a spatial resolution of 1024x768 pixels or less. [0041] The method may further comprise repeating the steps of capturing multiple training sky images, generating a display, receiving user input and determining one or more parameter values for multiple image sensors located across a prediction region, wherein determining the predicted solar irradiance comprises determining the predicted solar irradiance at a prediction location within the prediction region based on the parameter values determined for each of the multiple image sensors.

[0042] The camera network is advantageous because it extends the single-camera forecast horizon of about 30 minutes, to several hours, and expands the area covered from a 10km radius to hundreds of square kilometres using multiple cameras, allowing the network to out-compete longer-term approaches such as satellite cloud prediction, for a fraction of the capital cost, but with higher spatial resolution and temporal frequency. Preferably, the camera network collects imagery every 1 second, at a 10m grid resolution, whereas satellites will typically provide images every 15 minutes at around a 10km grid.

[0043] Determining a predicted solar irradiance value may comprise determining a predicted direct normal irradiance for individual collectors comprising heliostats, linear Fresnel, parabolic trough, or dish collectors in a solar thermal facility, and the method may further comprise per collector selection and targeting control to optimise irradiance control and irradiance slew rate over a receiver surface.

[0044] Determining a predicted solar irradiance value may comprise determining a predicted direct normal irradiance for individual collectors comprising heliostats, linear Fresnel, parabolic trough, or dish collectors in a solar thermal facility, and the method may comprise automated or manual control to optimise plant output, short term operability or long term operability.

[0045] Software, when executed by a computer, causes the computer to perform the above method. [0046] A computer system for predicting solar irradiance comprises:

a first input port to receive from an image sensor captured multiple first training sky images;

a second input port to receive object location data in relation to the multiple first training sky images, the object location data being indicative of a location in the training sky images of an object that may affect the solar irradiance ;

a processor

to determine one or more parameter values based on the object location data, the parameter values being indicative of a relationship between the location of the object and the solar irradiance, and

to determine a predicted solar irradiance value based on the parameter values.

[0047] The computer system may further comprise a display device to generate a display of the multiple first training sky images, wherein the second input port is to receive the object location data as user input data in relation to the multiple first training sky images, the user input data being indicative of a location in the training sky images of an object that may affect the solar irradiance.

[0048] The image sensor may be configured to generate the training sky images or the test sky images or both with an analog to digital resolution of 8 bit or less.

[0049] The image sensor maybe configured to generate the training sky images or the test sky images or both with a spatial resolution of 1024x768 pixels or less.

[0050] The image sensor may beconfigured to generate images using lossy compression.

[0051] A camera is used for capturing training sky images to determine a predicted solar irradiance value, wherein the camera is configured to generate the training sky images with an analog to digital resolution of 8 bit or less. [0052] A camera that is configured to generate images with an analog to digital resolution of 8 bit or less is used for capturing training sky images and of a display and input device for receiving user input data in relation to the training sky images, the user input data being indicative of a location in the training sky images of an object that may affect the solar irradiance to determine a predicted solar irradiance value based on the multiple training sky images and the user input data .

[0053] The number of captured training sky images may be such that the accuracy of the predicted performance is sufficient for generating a control signal that compensates for a difference between a power demand and the predicted photovoltaic performance.

[0054] A camera that is configured to generate images with an analog to digital resolution of 8 bit or less is used for capturing sky images that serve as a test input to a supervised learning classifier to predict solar irradiance.

[0055] The supervised learning classifier may be an ensemble classifier.

[0056] The camera may be further configured to generate images with a spatial resolution of 1024x768 pixels or less.

[0057] The camera may be configured to generate images using lossy compression.

[0058] Multiple cameras may be used as stated above in a network of cameras located across a prediction region to determine a predicted solar irradiance value for a location around the prediction region.

[0059] Preferably, the network comprises more than 5 cameras. More preferably, the network comprises more than 10 cameras. Even more preferably, the network comprises between 10 and 20 cameras covering a visible ground-area of several hundred square kilometres or more.

[0060] The camera may be configured to generate the training sky images with one or more of: an analog to digital resolution of 8 bit or less;

a pixel resolution of 1024x768 or less; and

use of a lossy compresion.

[0061] A method for determining a network of image sensors to estimate a performance of a photovoltaic system comprises:

performing the above method to determine a predicted solar irradiance;

determining a predicted performance of the photovoltaic system based on the predicted solar irradiance;

receiving measurement data indicative of the actual performance of the photovoltaic system;

determining a difference between the predicted performance and the actual performance; and

determining the network of image sensors by reducing the difference between the predicted performance and the actual performance.

[0062] Determining the network of image sensors may comprise determining the number and location of image sensors that reduces the difference between the predicted performance and the actual performance.

[0063] A method for solar irradiance prediction comprises:

capturing by a camera training sky images; and

determining a predicted solar irradiance value based on the training sky images,

wherein the camera has an analog to digital resolution of 8 bit or less.

[0064] A method for solar irradiance prediction comprises:

capturing multiple training sky images by a camera with an analog to digital resolution of 8 bit or less; receiving from a display and input device user input data in relation to the multiple training sky images, the user input data being indicative of a location in the training sky images of an object that may affect the solar irradiance; and

determining a predicted solar irradiance value based on the multiple training sky images and the user input data.

[0065] A method for solar irradiance prediction comprises:

capturing by a camera with an analog to digital resolution of 8 bit or less sky images; and

applying the sky images to a test input of a supervised learning classifier to predict solar irradiance.

[0066] A method for estimating a performance of a solar power generation system comprises:

capturing multiple first training sky images by an image sensor for different positions of the sun;

generating a display of the multiple first training sky images;

receiving user input data in relation to the multiple first training sky images, the user input data being indicative of a location of the sun in the multiple first training sky images;

determining one or more parameter values based on the user input data, the parameter values being indicative of a location, orientation and lens configuration of the image sensor;

determining a predicted position of the sun at a prediction time based on the parameter values;

determining a predicted performance of the solar power generation system based on the predicted position of the sun; and

generating a control signal to control the photovoltaic system based on the predicted performance.

[0067] A solar power generation system comprises:

a supervised training module of a localised sun location model, a prediction module of the sun's position

a prediction module of a performance of the solar power generation system; and

a control module of an output of the solar power generation system.

[0068] A method for estimating a performance of a solar power generation system comprises:

capturing of multiple second training sky images by the image sensor for different cloud cover;

generating a display of multiple second training sky images;

receiving user input data indicative of a classification of pixels of the multiple second training sky images, the classification being in relation to a cloud cover in the sky images;

determining an ensemble classifier based on the user input data, the ensemble classifier comprising multiple sub-classifiers;

capturing a test sky image;

applying the ensemble classifier to a pixel of the test sky image to calculate for each of the multiple sub-classifiers an output classification;

determining a classification as cloud or clear sky for the pixel based on the output classification of each of the multiple sub-classifiers;

determining a predicted performance of the solar power generation system based on the classification as cloud or clear sky; and

generating a control signal to control the photovoltaic system based on the predicted performance.

[0069] A solar power generation system comprises:

a supervised training module of a cloud pixel classification model using an ensemble classifier; and

a prediction module of performance based on the classification

[0070] Training sky images are a subset of the sky images taken by a camera which are used for training and calibrating various models and parameters which allow the forecasting system to operate or forecast more accurately. For example, training sky images can be displayed to allow a user to manually identify objects of interest, so a fully automatic object identifier can be trained to identify similar objects in other images.

[0071] Test sky images are images which are processed automatically by the system to provide forecasts and other information, using the data and models obtained from the training process.

[0072] Optional features described above in relation to one or more of the aspects of method, system, software and use are also optional features of the other aspects.

[0073] Throughout this specification the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.

Brief Description of Drawings

[0074] An example will now be described with reference to:

Fig. la illustrates a solar system.

Fig. lb illustrates a photovoltaics system.

Fig. 2 illustrates the controller from Figs, la and 2b in more detail.

Fig. 3 illustrates a method for predicting solar irradiance.

Fig. 4 illustrates an example display.

Fig. 5a illustrates an unprocessed sky test image.

Fig. 5b illustrates an overlay over the sky test image in Fig. 5a where areas that are classified as cloud by a Red-Blue Ratio (RBR) classifier are shaded.

Fig. 5c illustrates an overlay over the sky test image in Fig. 5a where areas that are classified as cloud by a Random Forest (RF) classifier are shaded.

Fig. 5d illustrates a combined RBR and RF classifier where shading indicates area where both models agree while black indicates areas where only one model detects clouds.

Fig. 6 illustrates an output chart.

Fig. 7 illustrates an output histogram chart.

Fig. 8a illustrates a test sky image for motion vector estimation.

Fig. 8b illustrates an overlay over the test sky image to show the cloud's projected future path.

Fig. 8c illustrates output charts.

Fig. 9 illustrates a chart and sky image of a cloud front timing prediction for high cirrus type clouds.

Figs. 10a, 10b, 10c and lOd illustrate a sequence of test sky images.

Figs. 1 la and 1 lb illustrate camera networks covering Canberra and

Newcastle (Australia), respectively.

Description of Embodiments

[0075] This disclosure provides sun location models and optimisation algorithms, enabling the sun's location to be accurately tracked in a sky image. The model can be trained and optimised using a small number of human identified or computer identified sun locations, allowing the fast and accurate setup of a model for a camera of arbitrary fisheye characteristics, tilt, and orientation. This disclosure further provides lens distortion and calibration algorithms, modelling location and movement in a distorted camera image.

[0076] This disclosure further provides a cloud classification model allows the identification of clouds in a sky image using a supervised machine learning model which is trained on a small number of human-identified cloud and sky pixels. This model may be combined with a standard pixel red-blue-ratio model to improve the classification accuracy beyond the performance of either model separately, especially on low resolution, lossily-compressed, limited dynamic range images, such as from the inexpensive camera hardware, such as hardware comprising a colour resolution of 8-bit including JPEG cameras and other formats.. [0077] Cloud presence evaluation aggregates cloud pixel classification data and provides features which indicate the presence or absence of clouds and estimates the likelihood of shade events to occur in the near future.

[0078] Cloud movement identification allows cloud motion vectors to be estimated from a sequence of images. This may use optical flow algorithms and dense optical flow algorithms to compare the position of clouds between two frames of a sky image and estimate their movement.

[0079] Cloud projection algorithms allow predictions of a cloud's future position and velocity to be made. These custom algorithms use the cloud motion vectors (from previous step) to efficiently estimate the future position of every pixel in the image at a given time in the future, across the distortion-correct fisheye space. It identifies cloudy- classified pixels which will pass within a given distance of the sun, and are therefore likely to cause shading, and extracts a number of forecast features from them. These features are stored and used in future steps to provide irradiance and shade-timing forecasts.

[0080] Shade-event timing warnings, which provide a binary forecast of upcoming shade events and times. These warnings can be given with a range of adjustable biases, to allow a range of forecasts, from very conservative (avoiding shade-event misses), to very aggressive (only predicting shade events with high probability).

[0081] Irradiance predictions may forecast both the timing and magnitude of irradiance decrease, during a shading event.

[0082] Cloud formation (advection) detection uses computer vision techniques to detect clouds forming (ie appearing from a clear sky rather than moving in from across the image) near the sun. Probabilistic warnings of cloud formation can be provided using this information. These warnings are an important addition to the cloud movement vector information, which generally can't predict shade events caused by clouds which form from a clear sky. [0083] Cloud height stereography uses concurrent images from two or more cameras spaced an appropriate distance apart, to match cloud features in all images and determine their height using stereography.

[0084] Wide-area shadow projection and forecasting uses cloud height measurements to build a 3D model of cloud locations to be constructed, and ray-tracing techniques to be used to extrapolate the position of shadows on the ground.

[0085] Multi-camera forecast combination combines the above techniques, and applies them to multiple concurrent, real-time images from a network of skycams spaced across a region (like a city) to provide a seamless region-wide forecast, providing very high resolution forecasts with much longer time-horizons and higher accuracy than a single camera.

[0086] Photovoltaic inverter output shaping and ramp rate limiting may use skycam observations and forecasts, that is predicted irradiance values, to control a number of solar inverters, limiting their output to minimise ramp-rates in an electrical grid.

[0087] Provided herein is a tool which evaluates performance profiles of single and multi-camera irradiance forecasts and allows optimal placement and spacing of cameras in a multi-camera network to ensure the area is adequately covered in all cloud conditions, and that forecast errors in chosen areas can be evaluated.

[0088] The tool may take shading, irradiance and/or solar power forecasts, and optionally, load forecasts, and use them determine the optimal amount of spinning reserve (additional capacity of base-load generation to cover possible unforeseen changes in load and intermittent generation) needed in an electrical grid or mini-grid.

[0089] Some techniques minimise the effect of sun-flare and make clouds near the sun more visible in an all-sky image by combining multiple images at different exposures from the same camera, or from multiple cameras with different exposures situated close to each other. [0090] Some algorithms detect the formation and dissolution of clouds, specifically with an aim to detecting clouds forming from a clear sky near the sun which have a high probability of shading the sun in the near future.

[0091] Supervised learning techniques may use pixel data and cloud movement forecast information to predict solar irradiance and/or solar power production. These models are trained on historic sky-images and data from the same, or a different, location.

[0092] Potential uses of the technology include:

Remote Area Power System with photovoltaics (PV) and fossil fuel backup

Distributed PV - Residential/commercial rooftop PV

Large grid-connected PV - PV Solar farms

Large concentrating solar-thermal - heliostat field solar farms

Small solar-thermal - residential solar hot water, solar air conditioning

Energy markets - generator dispatch and maintenance planning

Commercial/Industrial building peak charge avoidance

Electrical network management and planning

Solar resource assessment and forecast validation -

Data sales to weather forecasting agencies

Home security

[0093] Fig. la illustrates a solar system 100 comprising the sun 101 shining through clouds 102 onto a mirror 103. The mirror 103 reflects the sunlight onto solar tower 104, where the sunlight is converted to heat, which eventually generates electric power. Controller 105 moves mirror 103 to follow the sun 101 to maximise the amount of sunlight that reaches tower 104. When insufficient sunlight reaches the tower 104, controller 105 can start a diesel generator 106 or other alternative electrical generators to compensate for the reduction of power generated by tower 104 due to occlusion of the sun 101 by clouds 102. It is noted that in most practical applications there would be a large number of mirrors, such as 173,500 mirrors in the case of the Ivanpah Solar Electric Generating System. Each of these mirrors may be individually controlled by controller 105.

[0094] Controller 105 is faced with the difficulty that the system of mirror 103, tower 104 and diesel generator 106 presents a significant time lag between a control decision and the effect of the control. For example, once cloud 102 occludes the sun 101, the generated power drops faster than the start-up time of the generator 106. In another example, multiple mirrors are used and the aim is to maintain a constant power output, which means that at some times of high irradiation, some mirrors need to direct the sunlight elsewhere. Measuring the temperature in tower 104 may be too slow to perform an efficient control of the mirrors.

[0095] In order to address the time lag in the system, controller 105 predicts the solar irradiance. For example, controller 105 predicts the solar irradiance for the entire field of mirrors or for each mirror separately. Particularly for large arrays of mirrors, some mirrors may be shaded by cloud 102 while others are fully illuminated. By predicting the solar irradiance, controller 105 can determine more accurately, how to control the system including, when to start-up generator 106, whether to stop generator 106 or keep it running for the next occlusion or whether to adjust the orientation of mirror 103.

[0096] Three cameras 107, 108 and 109 support the prediction of solar irradiance by controller 105. In particular, the cameras 107, 108 and 109 capture sky images and present those image to a user. A user assists controller 105 by indicating on the sky images objects that may affect the solar irradiance, such as clouds 102, sun 101 or manmade objects including, planes, helicopters and blimps. This means the user identifies objects that would affect solar irradiance if they were at the right location, such as occluding the sun. However, in order to learn the appearance of those objects, the currently displayed objects may not actually occlude the sun but similar objects of the same type may occlude the sun at a later time. Based on these classification aids, controller 105 learns to classify further images and predicts the solar irradiation at any location within the area covered by cameras 107, 108 and 109. [0097] Fig. lb illustrates a photovoltaics system where the mirrors, such as mirror 103 in Fig. la, are replaced by an array of photovoltaic solar collectors 110 that generate electricity directly without first converting the irradiation to thermal energy. Similar to Fig. la, controller 105 controls the individual photovoltaic solar collectors 110 and generator 106 as described herein.

[0098] Fig. 2 illustrates the controller 105 in more detail. The controller 105 is basically a computer system that comprises a processor 201 connected to a program memory 202, a data memory 203, a communication port 204 and a user port 204. The program memory 202 is a non-transitory computer readable medium, such as a hard drive, a solid state disk or CD-ROM. Software, that is, an executable program stored on program memory 202 causes the processor 201 to perform the method in Fig. 3, that is, processor 201 captures sky images by receiving image data from camera 107 through communication port 204. Processor 201 receives object location data being indicative of a location in the training sky images of an object that may affect the solar irradiance. This object location data may be generated by a computer vision algorithm that automatically detects the object in the image. Processor 201 may also generate on screen 106 a display 207 to receive user input indicative of objects that may affect solar irradiance from user 218. Processor 201 then determines a relationship between the location of the object and the solar irradiance and predicts the solar irradiance based on the relationship.

[0099] The processor 201 may then store the predicted solar irradiance value on data store 203, such as on RAM or a processor register. Processor 201 may also send the determined predicted solar irradiance value via communication port 204 to a server 220, such as a weather service provider.

[0100] The processor 102 may receive data, such as sky image data, from data memory 203 as well as from the communications port 204 and the user port 205, which is connected to a display 206 that shows a visual representation 207 of the test sky images as well as potentially the predicted solar irradiance to a user 116. In one example, the processor 201 receives the image data from cameral 107, 108 and 109 via communications port 204, such as by using a Wi-Fi network according to IEEE 802.11. The Wi-Fi network may be a decentralised ad-hoc network, such that no dedicated management infrastructure, such as a router, is required or a centralised network with a router or access point managing the network.

[0101] In one example, the processor 201 receives and processes the test sky images in real time. This means that the processor 102 determines the predicted solar irradiance value every time image data is received from one or more of the cameras 107, 108 and 109 and completes this calculation before the cameras 107, 108 and 109 send the next image data update.

[0102] Although communications port 204 and user port 205 are shown as distinct entities, it is to be understood that any kind of data port may be used to receive data, such as a network connection, a memory interface, a pin of the chip package of processor 201, or logical ports, such as IP sockets or parameters of functions stored on program memory 202 and executed by processor 201. These parameters may be stored on data memory 203 and may be handled by-value or by-reference, that is, as a pointer, in the source code.

[0103] The processor 201 may receive data through all these interfaces, which includes memory access of volatile memory, such as cache or RAM, or non-volatile memory, such as an optical disk drive, hard disk drive, storage server or cloud storage. The controller 105 may further be implemented within a cloud computing environment, such as a managed group of interconnected servers hosting a dynamic number of virtual machines.

[0104] It is to be understood that any receiving step may be preceded by the processor 201 determining or computing the data that is later received. For example, the processor 201 determines an image by pre-filtering or other image processing and stores the image in data memory 203, such as RAM or a processor register. The processor 201 then requests the data from the data memory 203, such as by providing a read signal together with a memory address. The data memory 203 provides the data as a voltage signal on a physical bit line and the processor 201 receives the image data via a memory interface.

[0105] It is to be understood that throughout this disclosure unless stated otherwise, nodes, edges, graphs, solutions, variables, values and the like refer to data structures, which are physically stored on data memory 203 or processed by processor 201.

Further, for the sake of brevity when reference is made to particular variable names, such as "period of time" or "solar irradiance" this is to be understood to refer to values of variables stored as physical data in controller 105.

[0106] Fig. 3 illustrates a method 300 as performed by processor 201 for predicting solar irradiance. Fig. 3 is to be understood as a blueprint for the software program and may be implemented step-by-step, such that each step in Fig. 3 is represented by a function in a programming language, such as C++ or Java. The resulting source code is then compiled and stored as computer executable instructions on program memory 202.

[0107] Method 300 commences by processor 201 capturing 301 multiple training sky images by an image sensor, such as by cameras 107, 108 and 109. Capturing images may comprise requesting (pull) image data from the cameras 107, 108 and 109 or receiving unrequested (push) image data. Capturing may also comprise the actual process performed by the semiconductor device while keeping an aperture open while light irradiates the semiconductor to cause electrons to move in a detectable way, including CMOS and CCD principles. During the capturing of the multiple training images, the image sensor may remain at a constant location, orientation and lens configuration. Location includes latitude, longitude, altitude or other coordinates, orientation includes angles that define the line of sight of the image sensor. Lens configuration includes focal length. Constant location, orientation and lens

configuration means that these parameters do not change substantially between capturing the multiple training images. In other words, locations of objects that do not move appear at the same pixel location for all of the multiple training images.

Consequentially, shutters and apertures are not part of the lens configuration and therefore, shutter speed and aperture size may vary between different training images to adapt to different lighting conditions without influencing the pixel location of objects in the images.

[0108] In other examples, the location, orientation and lens configuration is predetermined. For example, the orientation of the image sensor may change with the seasons according to the different azimuth of the sun. This change is predefined in the sense that the processor 201 can retrieve the current value from data memory to take the current value into account when determining the position of the sun and the clouds. In yet another example, the current position and orientation may be measured by sensors including inertial sensors in cases where the sensor platform is not stable, such as on vehicles, ships, drones, etc.

[0109] Next, processor 201 generates a display of the multiple training sky images. Fig. 4 illustrates an example display, where one of the multiple training sky images is displayed. The display shows the sun 401 and clouds 402 and 403. However, it is difficult to accurately detect the sun and the clouds automatically. Therefore, user 208 taps on the touch screen 206 to classify certain pixels as cloud, clear sky or sun.

[0110] Processor 201 receives this user input data in relation to the multiple training sky images. The user input data is object location data indicative of a location in the training sky images of an object that may affect the solar irradiance. Objects include clouds and the sun. For example, the user input data comprises a pair of values (150,100) that indicates that the user 208 has classified a pixel at column 150 and row 100 as cloud. In this example, processor 201 receives the data via the activation of an interrupt from the touch screen controller over a device bus. In other examples, device 206 is external to the controller 105 and processor 201 receives the user input data over the internet or other data connections including Wifi and Bluetooth.

[0111] Alternatively, or in addition to the above, processor 201 may receive the object location data by performing a computer vision algorithm to automatically detect the objects, storing the result on data memory 203 and receive the object location data from data memory 203. For example, processor 201 may automatically identify position of the sun in clear-sky conditions by searching for highly saturated pixels that form a roughly circular region in the image.

[0112] Based on the object location data, processor 201 determines one or more parameter values. The parameter values are indicative of a relationship between the location of the object and the solar irradiance.

[0113] In the case of the sun 401, processor 201 may calculate the orientation of the camera 107, for example, since the orientation is indicative of the relationship between the pixel location of the sun in the image and the solar irradiance. By using multiple training images and the time difference between those images were taken, processor 201 can determine the fisheye lens geometry, position and orientation of the image sensor. Since these parameters are indicative of the relationship between the location of the sun and the solar irradiance, with this information processor 201 can predict the movement of the sun across the image. In one example, processor 201 uses the Omnidirectional Camera Calibration Toolbox for Matlab (OCamLib) from the University of Zurich with multiple photos of a checkerboard pattern. After calculating this calibrated fisheye model, processor 201 may use a standard sun azimuth/zenith model, to calculate the sun's rough angle in the sky, and use the fisheye model to determine which fisheye-image pixel that corresponds to. Unless the camera is perfectly flat and true-north facing, this location may be inaccurate, so processor 201 performs an optimisation to fit a model of the user-input true sun location to the calculated one, allowing rotational and translational errors. Once, 10-images, for example, have been processed like this, processor 201 may have calculated an accurate estimate of the pixel coordinate of the sun for any other timestamps, in images from this camera (assuming a fixed position and fixed lens configuration).

[0114] For example, using a sun location model that determines azimuth and zenith for a known latitude, longitude and time, processor 201 can calculate, for a given instant in time, an error between a) the sun's calculated azimuth and zenith angles and b) a user- input sun coordinate 401 in the image. Using a number of such calculations from images taken at different times on a clear-sky day, the standard sun location model can be corrected for the specific camera orientation, by fitting a model to the standard model which minimises the location errors. This corrected model allows the highly accurate prediction of sun location in a sky image from this camera for any time.

[0115] At one time instance t processor 201 has one pair of sun locations in the two images, say (v v[) , and at another time instance t ₂ processor 201 has another pair of sun locations in the two images, say (v ₂ , v ₂ ' ) . Note that each of these vectors are in the

3D space. Processor 201 may calculate the following two steps closed-form solution to obtain the relative rotation information based on the two pairs of matched sun locations: firstly estimating the rotation axis and then estimating the rotation angle around this axis.

[0116] Rotation Axis

[0117] For the two vectors (v v[) , processor 201 finds a plane π _χ which is perpendicular to the plane formed by Vj and v[ , and bisects Vj and v[ . Similarly processor 201 finds a plane π ₂ for (v ₂ , v ₂ ' ) . Then, processor 201 finds the intersection line for plane π _χ and plane π ₂ . The direction of this intersection line is the rotation axis that processor 201 is to obtain and is denoted as v _axis .

[0118] Rotation Angle

[0119] Once the rotation axis is obtained, the rotation angle around this axis can be obtained. For a vector v , it can be decomposed into two components: one is parallel to v _axis and the other is perpendicular or normal to v _axis . In this example, v _n represents the normal component of v . Similarly, represents the normal component of V . The rotation angle Θ is the angle between the two vectors v _n and which can be obtained by Θ = cos ^'1 ,, ^V , , ^V . ,, · [0120] The rotation matrix R can then be constructed based on the following formula using the rotation axis and rotation angle information:

R = I + (sin Θ)Η + (1 - cos Θ)Η ²

where I is the identity matrix and H is a skew- symmetric matrix composed of the elements of the rotation axis.

[0121] In the case of the clouds 402 and 403, processor 201 retrieves the pixel colour and movement values and textural information from user-identified cloud and sky areas and trains a classifier. The classifier can then be applied to test images to determine the location of clouds in the test images. Over multiple test images, processor 201 can determine the movement of the clouds, such as speed, and predict when the clouds will occlude the sun at the predicted sun location. This way, processor 201 determines a predicted solar irradiance value based on the parameter values. In one example, the outcome is binary. That is, the result is either indicative of either; a level of shading; or of clear-sky irradiance. In another example, the result is a continuous irradiance value (eg. from 0 to 1000 W/m2).

[0122] The user input data is indicative of a classification of pixels of the multiple training sky images, the classification being in relation to a cloud cover in the sky images. For example, the user interface 400 may comprise instructions to the user, such as "Please tap on clouds". Each time the user taps on a pixel of the image, processor 201 receives the RGB colour values of a group of pixels. Processor 201 uses this RGB colour value as a training sample that is classified positively as 'cloud' (TRUE). User interface 400 may change the displayed instructions to "Please tap on clear sky". Processor 201 may then use the received RGB values as training samples classified negatively, that is, blue sky (FALSE). In another example, the user may be asked to classify thin and thick clouds separately in a series of images.

[0123] Processor 201 repeats the steps of receiving the training samples until processor 201 learns a model that is sufficiently accurate. In other words, processor 201 calculates parameter values based on the user input data and these parameter values are parameter values of a cloud model and therefore, indicative of the relationship between the location of the cloud and the solar irradiance. While in some examples processor 201 calculates the current position of the clouds 402 and 403, repeating this calculation can lead to movement estimation and therefore, the position is indicative of a relationship between the location of the clouds 402 and 403 and the solar irradiance.

[0124] In one example, the one or more parameter values are parameter values of a supervised learning classifier based on the user input data. In that case, processor 201 can use a pixel of a later captured test sky images as an input to evaluate the classifier that outputs a classification as cloud or clear sky (or further classes as described herein). Processor 201 then determines the predicted solar irradiance value based on the classification and projected movement of all pixels, relative to the sun' s location. In particular, processor 201 repeats the classification steps for subsequent test images to determine movement of the clouds and when the movement intersects with the movement of the sun.

[0125] In one example, the classifier is a supervised learning classifier and preferably, is an ensemble classifier comprising multiple-sub classifiers. Processor 201 determines the classification by applying the ensemble classifier to a pixel of the test sky image to calculate for each of the multiple sub-classifiers an output classification. Processor 201 then determines the classification for the pixel based on the output classification of each of the multiple sub-classifiers. For the example of a Bayes optimal classifier, processor calculates y = argmax _{c eC} P(c I h _t )P(T I h _t )P(h _i ) where y is the predicted class, C is the set of all possible classes, such as "cloud" and "clearsky" or more classes in between. H is the hypothesis space, P refers to a probability, and T is the training data, that is, the training pixel values labelled by the user from the training sky images.

[0126] In another example, the ensemble classifier is a random forest and the sub- classifiers are decision trees as described in Breiman, Leo (2001). "Random Forests". Machine Learning 45 (1): 5-32, which is included herein by reference. Processor 201 trains the random forest by using tree learners and applying bootstrap aggregating, or bagging, that is, processor 201 selects a random sample with replacement of the training pixel values. Processor 201 fits trees to these samples:

For = 1, ..., B:

Sample, with replacement, n training examples from X, Y; call these ¾, F _¾ .

Train a decision or regression tree _¾ on ¾, F _¾ .

[0127] Processor 201 then applies the trained classifier to pixels of the test images and averages the predictions from all the individual regression trees on x' for each pixel: f = ^∑f _b (x')

[0128] Processor 201 may further randomly select subset of features for each decision tree. The features considered in the decision tree may comprise a weighted sum of the change in these values from a number of previous images. In another example the features may comprise texture information from the surrounding pixels, Red, Green, Blue (RGB) channels. In another example the features may comprise the distance from the pixel to the sun's location in the image, Hue, Saturation, Brightness (HSV) channels, a weighted sum of the change in these values from a number of previous images, texture information from the surrounding pixels. In another example the features may comprise the distance from the pixel to the sun's location in the image, Hue, Saturation, Brightness (HSV) channels, a weighted sum of the change in these values from a number of previous images, texture information from the surrounding pixels, Red:Blue ratio (RBR), Red, Green, Blue (RGB) channels, Red-Blue difference (RBD),an estimated amount of movement from a number of previous images, or the sun's current azimuth and zenith angles. In another example the features may comprise texture information from the surrounding pixels, an estimated amount of movement from a number of previous images. In another example the features may comprise Red-Blue difference (RBD),Hue, Saturation, Brightness (HSV) channels, the sun's azimuth and zenith angles, texture information from the surrounding pixels, Red, Green, Blue (RGB) channels, a weighted sum of the change in these values from a number of previous images, keypoint location, viola-jones rectangular features. In another example the features may comprise the sun's azimuth and zenith angles, viola- jones rectangular features, Red, Green, Blue (RGB) channels. In another example the features may comprise the distance from the pixel to the sun's location in the image, Hue, Saturation, Brightness (HSV) channels, scale-invariant feature transform (SIFT) features. In another example the features may comprise the distance from the pixel to the sun's location in the image, Hue, Saturation, Brightness (HSV) channels, Red:Blue ratio (RBR), viola-jones rectangular features, Red, Green, Blue (RGB) channels, an estimated amount of movement from a number of previous images. In another example the features may comprise the distance from the pixel to the sun's location in the image, viola-jones rectangular features, Hue, Saturation, Brightness (HSV) channels, Red-Blue difference (RBD), keypoint location, Red:Blue ratio (RBR), the sun's azimuth and zenith angles, Red, Green, Blue (RGB) channels, scale-invariant feature transform (SIFT) features. In another example the features may comprise the distance from the pixel to the sun's location in the image, a weighted sum of the change in these values from a number of previous images, Red:Blue ratio (RBR), keypoint location, an estimated amount of movement from a number of previous images, texture information from the surrounding pixels, Red-Blue difference (RBD). In another example the features may comprise Red:Blue ratio (RBR), a weighted sum of the change in these values from a number of previous images, Red-Blue difference (RBD), texture information from the surrounding pixels, scale-invariant feature transform (SIFT) features, Hue, Saturation, Brightness (HSV) channels. In another example the features may comprise keypoint location, an estimated amount of movement from a number of previous images, Hue, Saturation, Brightness (HSV) channels, a weighted sum of the change in these values from a number of previous images, Red, Green, Blue (RGB) channels, viola-jones rectangular features, scale-invariant feature transform (SIFT) features.

[0129] In this sense, during training based on the training images, processor 201 determines the ensemble classifier by determining nodes and edges of the random forest. That is, processor 201 learns that certain features (nodes) are connected (edges) if these features are indicative of a classification (leafs/terminal nodes). In other words, a path from the top node to the classification output indicates which features contribute to that classification. For example, a high value in the brightness channel (V) of a pixel is likely to contribute to a 'cloud' classification because clouds are generally brighter (white) pixels.

[0130] In one example, processor 201 also classifies each image pixel using a Red- Blue Ratio. That is, if the Red-Blue Ratio is above a predetermined threshold, processor 201 classifies that pixel as cloud. In one example, processor 201 classifies as a cloud if RBR> 0.95. In other examples, the threshold test is RBR>1.0, which is particularly useful if the sensor or JPG compression produce a lot of image noise. If the Red-Blue Ratio is below the threshold, processor 201 evaluates the ensemble classifier and classifies the pixel as cloud if the ensemble classifier outputs a cloud classification. In other words, the final cloud pixels are a union set of the cloud pixels from the Red-Blue Ratio threshold and the cloud pixels from the ensemble classifier.

[0131] As mentioned above, processor 201 may capture multiple test sky images and apply the supervised learning classifier to pixels of the multiple test sky images.

Processor 201 then determines a classification in relation to a cloud cover for each of the multiple test sky images. Processor 201 can then determine the predicted solar irradiance value by determining a movement of cloud pixels and determining the predicted solar irradiance value based on the movement of the cloud pixels. For example, processor 201 may determine a motion vector of pixels classified as having a cloud cover that is above a predetermined threshold, such as by following the

H.264/MPEG-4 AVC standard. In another example, processor 201 uses the Fameback algorithm. In this sense, processor 201 calculates the difference of the location of cloud pixels between subsequent images and uses this difference to determine for each cloud pixel a movement direction and speed.

[0132] In one example, processor 201 divides the test sky image into blocks and minimises an error of block-matching between subsequent images. In another example, processor 201 uses a continuous area of cloud pixels as one object and displaces that cloud object in an earlier frame until the displaced object minimises a difference to a later frame. The amount of direction of the displacement is then identical to the movement of the cloud between the two images. [0133] Similarly, processor 201 may apply transformation other than displacements to the cloud object. For example, processor 201 increases the size of the cloud object until a difference to a later image is minimised. This allows the processor 201 to consider cloud formation, that is, the generation of new clouds or growing clouds. In this sense, processor 201 determines the predicted solar irradiance value by

determining cloud formation based on the classification and determining the predicted solar irradiance value based on the cloud formation.

[0134] Processor 201 can then use the motion vector to calculate a predicted time value or a predicted magnitude value or both for the predicted sun occlusion event. That is, processor 201 calculates how long it will take the cloud to reach the sun based on the current speed and direction of the cloud and the distance to the sun.

[0135] In another example, processor 201 determines a cloud altitude, such as by further sensors including laser, radar, satellite or stereovision from two separate cameras. Based on the altitude, processor 201 can then calculate how the clouds will cast a shadow on the earth by applying raytracing or other shadow construction techniques, or using geometric transforms of the classified cloud image from a camera with known lens geometry. Processor 201 may also take topograhical data into account when determining shadow locations. A characteristic value of the shadow may be an outline of the shadow or a location of the shadow. Processor 201 may have stored a map or other representation of the forecast area, such as a grid of points on the earth's surface. Processor 201 may determine a degree of shadowing for each point of the grid which represents the predicted irradiance value at that point of the grid. Processor 201 may use predictive features and cloud-type classifications extracted from one or more concurrent cloud images and corresponding past irradiance or power measurements from several locations to model and predict the degree of shadowing at each grid point. Once a 3 -dimensional model of the cloud layers is constructed, processor 201 can determine the timing of the shadowing by examining clouds predicted to intersect a line between the sun and the ground locations of interest. For example, a thin cloud, such as a alto-cirrus cloud, may only cast a weak shadow, which reduces the irradiation to 90% of the clear-sky value. In contrast, a thick, dark grey cumulus cloud may cast a strong shadow that reduces irradiation to 10% of the clear sky value.

[0136] The points of the grid may also be non-regular but instead, the locations of individual collectors, such as heliostats, linear Fresnel, parabolic trough, or dish collectors in a solar thermal facility method. As a result, processor 201 determines a solar irradiance value, such as a direct normal irradiance for each collector. This allows per collector selection and targeting control to optimise irradiance control and irradiance slew rate over a receiver surface, such as tower 104. For example, processor 201 maintains a list of collectors that is sorted in descending order by irradiation and selects collectors from the top of the list until a required amount of total solar irradiation at the collector is reached. In this sense, processor 201 performs automated control to optimise plant output. Since the receiver surface is utilised more efficiently, the proposed controls also optimise short term operability and/or long term operability.

[0137] With the above data, processor 201 may determine one or more of:

presence of clouds within a predetermined viewing angle of sun;

shading-time predictions; and

photovoltaic power generation.

[0138] As shown in Fig. 1, processor 201 may receive captured test sky images from multiple cameras 107, 108 and 109 forming a camera network. The camera network may comprise more than 5 cameras, more than 10 camera, or between 10 and 20 cameras. In other examples, the camera network comprises over 100 cameras or even thousands of cameras, which is financially possible since the disclosed methods allow the use of low-cost cameras and low quality lossily compressed images which cost less to transfer and store.

[0139] Processor 201 can therefore combine the classifications of the individual camera pixels to generate location data indicative of a cloud cover and cloud characteristics for the area covered by the multiple cameras. For example, processor 201 may have stored information on an overlap of images from adjacent cameras and can therefore determine which clouds are seen by more than one camera. Processor 201 may also transform the fisheye images to linear images, that is, perform a correction of lens distortion and then stitch the images together to obtain one single image for the entire area covered by the camera network. Processor 201 can then apply the above methods on the single image to predict the solar irradiation and may predict photovoltaic performance, such as electrical energy generation in kW/h.

[0140] Over time, processor 201 may collect measurement data of actually produced energy and compare the measurement data to the predicted data. Processor 201 may then perform an optimisation method to adjust the configuration parameters of the image sensors. For example, processor 201 may calculate an optimised location of the image sensors, such that the difference between the prediction and the measurement is minimised.

[0141] Below is a description of the multi-camera orientation algorithm. The alignment (tilt and orientation) between all the cameras in a camera network, so that their respective images can be aligned accurately, which is important for estimating cloud height via triangulation, sun tracking, 3D cloud modelling and ground-shadow projection:

[0142] Multi-camera auto-orientation calibration

[0143] A closed-form solution for estimating the relative rotation parameters for any two cameras is disclosed. The algorithm only uses two or more pairs of sun location information from simultaneous images taken by two or more cameras at two different time instances. It does not need to use the absolute Sun location information which can be obtained by using the time and locations of the cameras. However, if the absolute sun location information is available, it can also be used. The algorithm is simple, direct, and non-iterative. This obtained rotation parameters can be used to generate undistorted images or to generate virtual fisheye images at a different viewing angle. [0144] Once the rotation parameters are obtained, the translation parameters between the two cameras can also be obtained using linear algorithms from matched image feature points. The separation of the estimation for the rotation and translation parameters is helpful in achieving better parameter estimates. After the rotation and translation parameters are obtained, cloud heights can be measured using stereographic image feature matches from the two cameras.

[0145] The algorithm can be applied to multiple cameras which are installed in a city for obtaining all the orientation parameters for these cameras. An optional master camera can be selected as the reference point. Further processing in robust estimation of the parameters can also be carried out to remove bad image matches. Nonlinear parameter optimisation can also be used for parameter refinements.

[0146] The same algorithm can also be used to estimate the relative orientation between the fisheye camera and the absolute positioning of the Earth or the Sun using the Sun orientation relative to the earth and the pixel location of the Sun in the image. A minimal of two pairs of such measurements are needed. Once this relative orientation between the camera and the Earth/Sun is obtained, a correction on the image, or an artificial/synthetic fisheye image, can be made. Sometimes gross errors and mismatches can occur in the feature matching or Sun location estimation process. In order to improve the parameter estimation process, robust estimation approaches, such as random sample consensus (RANSAC), least median squares, or other methods, can be used to remove the gross errors or bad matches. Only the good data are to be used for parameter estimations. The parameters obtained from the closed-form solution, either using or not using the robust estimation step, can be further improved using a nonlinear optimisation process based on the initial parameters obtained from the closed-form solution. This optimisation process may include all the good data points (after the robust estimation process if this process is included).

[0147] In one example, the image sensors apply exposure bracketing to provide multiple sky images for different exposures, such as five images over a range from +3 to -3 f-stops. This allows processor 201 to combine these images into a single HDR image, which addresses the problem of limited dynamic range from which low-cost image sensors typically suffer.

[0148] Cameras used in this disclosure may be configured to generate images with a digital resolution of 8 bit or less. Digital resolution may refer to the actual number of output bits of the A/D converters in the chip or the number of bits in the output file written on the camera's memory. This means the camera may employ a higher internal resolution or dynamic range and reduces the resolution or dynamic range of the data when it is written on memory, which is also referred to as downsampling. The use of 8 bit allows the use of inexpensive cameras while the above classification methods provide an accurate prediction even based on inferior image data. Some low cost cameras are further configured to generate image data with a pixel resolution of 1024x768 pixels or less and again, the above methods achieve good accuracy despite this low amount of image information. For example, the camera may comprise an image processor that receives the raw sensor data from a 7 megapixel CCD or CMOS sensor with 12 bit dynamic range, performs image processing, such as de-bayering, de- noising, colour balancing, and optionally downsamples the pixel resolution to

1024X768 and the dynamic range to 8 bit. The image processor then stores the resulting image as a file on memory, such as a JPEG image file, or sends the file to the controller 105.

[0149] The number of captured training sky images is such that the accuracy of the predicted performance is sufficient for generating a control signal that compensates for a difference between a power demand and the predicted photovoltaic performance. In one example, two sequential images may be sufficient at a minimum to give a forecast of irradiance or shading. Four or five images (at 10 second intervals) may give better performance as the noise in the motion vectors can be smoothed or filtered out.

[0150] The camera network is useful because it extends the single-camera forecast horizon of about 30 minutes, to several hours, and expands the area covered from about a 10km radius to hundreds of square kilometres using multiple cameras, allowing the network to out-compete longer-term approaches such as satellite cloud prediction, for a fraction of the capital cost, but with much higher spatial resolution and temporal frequency. For example, a camera network could collect imagery every 1 second, at a 10m grid resolution, whereas satellites will typically provide images every 15 minutes at around a 10km grid.

[0151] The high spatial and temporal resolution forecast data this network provides has numerous applications in the fields of weather and power forecasting, energy network optimisation and control, electrical demand management, electrical storage management. Secondarily, the imagery collected may also be used for other

applications like bushfire detection and location, home/business security, storm and lightning tracking & insurance, plane tracking and so on.

[0152] Fig. 5a illustrates an unprocessed sky test image.

[0153] Fig. 5b illustrates an overlay over the sky test image in Fig. 5a where areas that are classified as cloud by a Red-Blue Ratio (RBR) classifier are shaded. This shows good performance for overcast sky and dark clouds. However, the RBR classifier misclassifies near the sun as indicated at 511 and misses thin and near- horizon clouds as indicated at 512 and 513.

[0154] Fig. 5c illustrates an overlay over the sky test image in Fig. 5a where areas that are classified as cloud by a Random Forest (RF) classifier are shaded. This shows sensitivity to cloud edges and to thin and distant clouds. Fig. 3c also shows good near- sun performance. However, the RF classifier misses dark and non-textured cloud areas as indicated at 521 and 522.

[0155] Fig. 5d illustrates a combined RBR and RF classifier where shading indicates area where both models agree while black indicates areas where only one model detects clouds. This shows how the two model are complementary.

[0156] Fig. 6 illustrates a chart 600 that may be outputted by the proposed systems and methods. The chart comprises Direct Normal Irradiance (DNI) 601, Global Horizontal Irradiance (GHI) 602 and Photovoltaic (PV) Power 603. The current time is indicated at 604 and the values to the right hand side of the current time 604 are predicted values. This shows that the sharp increase of cloud cover leads to a sharp decrease of predicted irradiance indicated at 605. There are about 7 minutes warning time, which is sufficient to prepare the system.

[0157] Fig. 7 illustrates a histogram chart 700 that may be outputted by the proposed systems and methods. The histogram chart comprises bars that show cloud percentage in concentric rings around the sun. That is, the x-axis denotes the increasing distance from the sun. In one example, this cloud presence data may be used by a control system for starting a generator to compensate for the probably upcoming shading of a photovoltaic array.

[0158] Fig. 8a illustrates a test sky image 800 for motion vector estimation. Test sky image 800 comprises sun 801 and distant cloud 802. Cloud 802 moves towards sun 801 and the aim is to predict when cloud 802 will reduce the solar irradiation. Fig. 8b illustrates an overlay 810 over test sky image 800 to show the cloud's projected future path 811 where black areas indicate the likely path of the cloud 802 in the future based on predicted positions of the sun and the projection of the cloud pixels' current motion vectors. In this example, a simulation over the following half hour is run at 10- second time-steps, taking all the motion vectors extracted from all the current image's pixels, and extrapolating them across the fisheye image, to calculate probable paths 811 the cloudy pixels will take in future. If any of these paths cross into the region around the sun 801 during this simulation, the paths, and their originating cloud pixels 812, are marked for further analysis. This simulation results in estimates of the time until each of these cloudy pixels are likely to shade the sun, and features extracted from the originating cloud pixels 812 are used to estimate the amount of shading they are likely to cause. Because this approach operates on a per-pixel level, it can take into account an arbitrary number of cloud pixels, clouds and cloud layers moving with an arbitrary number of different velocities. Because the simulation is performed entirely in fisheye space, it is computationally efficient, and can consider every pixel individually. [0159] Fig. 8c illustrates charts 800 including DNI 801, GHI 822, PV power 833, effective cloud pixels 834, which is the number of pixels in shaded area 812 in Fig. 8b. Fig. 8c further shows the occlusion event at 835 caused by cloud 802 in Fig. 8a.

[0160] Fig. 9 illustrates a chart 900 and sky image 901 of a cloud front timing prediction for high cirrus type clouds. This example illustrates the accuracy of the predicted time to a shade event, based on the extrapolated movement vectors 902 of pixels 903 of an approaching cloud from near the horizon. The chart 900 shows the predicted time-to-shading (vertical axis) versus the actual measured time to shading (horizontal axis). The dotted line 910 represents a perfect forecast. The further lines 911, 912 and 913 show the 1 ^st , 2 ^nd and 5 ^th percentiles of the start of the cloud pixel histogram, respectively. These represent an estimated time until the front edge of the cloud enters the sun area 914. These estimates are calculated every 10 seconds from the time of the image shown, until the time of the beginning of the shade event. The chart 900 illustrates that the clouds were first detected about 33 minutes before the event, and that the forecast time-to- shading was very accurate (other curves are close to blue dotted line) from about 22 minutes, down to about 3 minutes before the event.

[0161]

[0162] Figs. 10a, 10b, 10c and lOd illustrate a sequence of test sky images. This sequence shows a cloud 1001, forming near the sun 1002 over 3 sequential frames (Figs. 10a, 10b and 10c), and then jumps 4 minutes to show the formed cloud 1001 shading the sun 1002 (Fig. lOd). This particular cloud 1001 appears to be moving away from the sun 1002 as it forms, so may not be detected by the motion vector-based detection methods. Its rate of formation toward the sun 1002, however, is faster than its movement away, so it actually causes a shading event. Warnings generated from this formation detection algorithm provide sufficient time to take action before the event, which may not be possible using just the motion vector-based approach.

[0163] Figs. 1 la and 1 lb illustrate camera networks covering Canberra and

Newcastle (Australia), respectively. The circles indicate a lower value of the field of view (with low cloud layers) of each camera in the network, positioned to provide overlapping coverage of the cities. The figures show that a small number of cameras can provide visual coverage of the entire city. In particular, 15 sites around Canberra and Newcastle are used for wide-area shading and irradiance forecasting.

[0164] There is provided a method for planning locations for network of cameras, taking into account:

• the cameras' spatial range,

• calculations of the spatial error functions of each camera,

• the ability to identify common cloud features for various cloud heights,

• the effect of topographical features on forecast accuracy and visible range,

• available physical locations for camera sites, and

• distribution of desired forecast locations

A person trains the cloud classification model by using a tool developed to manually identify clouds and sky in a small number of images (by colouring them in). They can also identify cloud types (on a number of levels from thin to thick).

[0165] The advantages of the disclosed methods are:

[0166] 1. Using an RBR model alone for accurate cloud classification may involve a clear-sky library (which estimates the sky colour without clouds or aerosols) which can be time-consuming to build (it needs a large number of clear- sky images in a variety of conditions, over a long period of time). Training the RF model only requires about 10 images to be manually classified for the trained model to automatically classify all clouds accurately.

[0167] 2. It performs much better than traditional approaches with low-quality images from the inexpensive hardware we're using. These images may be lossily- compressed, visually noisy, have low dynamic (brightness) range, have more lens and sun-flare artefacts, and are lower resolution than hardware that is typically used. [0168] In one example, processor 201 generates a control signal to control an inverter output to adjust ramp-rates in an electrical grid. Standard PV plants without storage or backup generators may exhibit power variations of up to 90% per minute, which is more than the typical compliance limit of 10%, for example. Ramp rate is essentially the speed at which a generator can increase (ramp up) or decrease (ramp down) generation. Generators have different characteristics, making some more suited to supplying certain needed functions. Since processor 201 predicts the solar irradiance, the ramp-up of the generator can be initiated earlier, which means more economical generators with lower ramp rates can be used. Processor 201 may also gradually reduce the output of solar power before a shading event to match the ramp-up of the generator. This can be achieved by processor 201 controlling an inverter that converts DC power from a solar panel to AC power for the grid.

[0169] Processor 201 may also generate the control signal to control energy storage charge and discharge rates and timing. For example, when processor 201 predicts high solar irradiation for the future, the energy storage (such as batteries) can be discharged more deeply beforehand. Conversely, if a longer period of low solar irradiation is predicted, processor 201 controls the system to maintain sufficient charge in energy storage.

[0170] Processor 201 may generate the control signal to control the rate and timing of energy consumption of electrical devices. For example, processor 201 may send control signals over the internet to electrical devices that may be not time critical, such as washing machines, central water heaters, pool pumps, electric car chargers and the like. This controls energy consumption of these devices such that energy consumption increases for times of predicted high solar irradiation and reduces energy consumption for times of predicted low solar irradiation.

[0171] Processor 201 may further control electrical devices to provide voltage, frequency and network services to the electrical grid, such as reactive power compensation. Finally, as mentioned above, processor 201 may control a generator to compensate for a reduction of performance caused by a cloud cover reducing irradiance of the photovoltaic system (ramp-up).

[0172] It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the specific embodiments without departing from the scope as defined in the claims.

[0173] It should be understood that the techniques of the present disclosure might be implemented using a variety of technologies. For example, the methods described herein may be implemented by a series of computer executable instructions residing on a suitable computer readable medium. Suitable computer readable media may include volatile (e.g. RAM) and/or non-volatile (e.g. ROM, disk) memory, carrier waves and transmission media. Exemplary carrier waves may take the form of electrical, electromagnetic or optical signals conveying digital data steams along a local network or a publically accessible network such as the internet.

[0174] It should also be understood that, unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "estimating" or "processing" or "computing" or "calculating", "optimizing" or "determining" or "displaying" or "maximising" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that processes and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0175] The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.