The subject-matter of the invention is a suspension of a photosensitive image sensor and a method for repeatably controlling and compensating for a temperature drift of an image of the photosensitive image sensor, in particular of a digital camera.

In most cases, it is assumed that after calibrating the camera, its imaging parameters (intrinsic, extrinsic parameters and distortion correction parameters) remain unchanged. However, in reality, due to the exposure of the camera to changing environmental conditions, these may change. One of the parameters that affect the camera calibration parameters is temperature. The influence of temperature on the imaging process in the camera manifests itself in two ways, i.e. there is observed an image drift associated with the warming up process of the camera and a temperature drift of the image associated with the change of the camera’s ambient temperature.

So far, cameras (recording devices with a photosensitive image sensor) mostly have not at all compensated for the temperature drift recorded on the image sensor when operating in variable ambient temperature conditions. Said drift is visible in the shifting and deforming (depending on the temperature) of the image position per photosensitive pixel(s) and this shifting can be up to 1.5 - 2 pixels from the expected position on the photosensitive image sensor, while for large cameras, the experimentally observed temperature drifts were as high as up to several pixels, which may translate to physical shifts of the object being recorded by the camera even by several centimetres (depending on the lens used).

Until now, in the mechanical design of digital cameras using a CMOS or CCD sensor, no connection was used between the sensor (or the PCB electronic plate with the sensor) and the housing (often referred to as the camera base, base plate, c-mount base or lens adapter), which left chaotic, i.e. non-repeatable thermal degrees of freedom for the sensor being deformed (and more precisely, the temperature-sensitive aluminium plate or the temperature-sensitive mounting of the PCB plate with the sensor to the camera base). Consequently, the recorded thermal drift of the image was random (both for the warming up and cooling of camera elements at a variable ambient temperature) and resulted in the mathematical adjustment, and also the compensation model, being impossible, and so the temperature drift of the image has so far been ignored.

Holder Handel, in a series of three publications “Compensation of thermal errors in vision based measurement systems using a system identification approach” 9th International Conference on Signal Processing, pp. 1329-1333, 2008, “Analyzing the influence of camera temperature on the image acquisition process”. SPIE, vol. 6805, pp. 1-8, 2008, “Analyzing the influences of camera warm-up effects on image acquisition”. Computer Vision - ACCV 2007, pp. 258-268, 2007, described how to compensate for the effect of temperature on camera calibration. The solutions presented by him are about how to compensate for the drift associated to both the warming up of the camera and the change of the external temperature. The presented compensation approach assumes the use of a pinhole camera model, described by an equation that does not take into account distortion correction. The compensation is achieved by adjusting the linear model parameterized only with respect to the external parameters of the camera. The author assumes that the internal parameters of the camera do not change under the influence of temperature. Of the ten parameters describing the equation, the author uses six to create a linear compensation model. In fact, this is a considerable simplification that does not take into account all aspects of the influence of temperature on the calibration of the camera. As a result of thermal deformation of the camera housing, sensor, lens mount and the lens itself, the internal coefficients of the camera also change.

Patent description WO2021164058A1 discloses a method and system for calibrating a temperature drift for a ToF camera. The method comprises adjusting a temperature by means of a temperature control jig to change the environment temperature of ToF camera samples in ToF cameras to be calibrated; acquiring the environment temperature of the ToF camera samples by means of a temperature sensor, and acquiring temperature drift coefficients of each ToF camera therein to obtain temperature drift coefficient sets; acquiring, from the temperature drift coefficient sets, a temperature drift coefficient set for calibration, wherein temperature drift coefficients in the temperature drift coefficient set for calibration enable a predetermined proportion of ToF cameras in the ToF camera samples to satisfy a preset measurement precision; acquiring the measurement precision, under each temperature drift coefficient in the temperature drift coefficient set for calibration, of the ToF cameras to be calibrated; and selecting a temperature drift coefficient enabling the measurement precision of the ToF cameras to satisfy the preset measurement precision, and calibrating the ToF cameras. Patent description CN110798593A discloses an industrial camera temperature drift elimination method. The industrial camera temperature drift elimination method comprises the following steps: step 1, an image output by the industrial camera under the action of temperature is formed by superposing P (n, m) on a pixel value of a corresponding position of the image and a temperature drift value of the image, wherein the two adjacent original images are considered to be two identical images, the temperature drift value of the corresponding position of the previous image can be approximate to the temperature drift value of the corresponding position of the current image, and the approximate temperature drift value is the pixel rising average value, namely the average pixel temperature drift value (T), of the original image under the influence of the temperature due to the fact that each photosensitive unit of the industrial camera is influenced by the same temperature. The beneficial effects of the invention are that the method can enable the industrial camera to be used normally under the condition that the temperature calibration is not needed through the comparison of two adjacent images, controls the complex factors of the environment, enables the camera to work more stably, can enable the images to be more precise, reduces the time consumed by the camera calibration to a certain degree, and improves the work efficiency.

Patent description CN112270712A discloses a method and system for calibrating a temperature drift based on the depth camera module. The calibration method comprises the steps of adjusting the temperature of the environment where the depth camera module is located to a preset environment temperature value; acquiring a measurement depth value, a light source real-time temperature value and a sensor real-time temperature value of the depth camera module for the calibration plate under the preset environment temperature value; calculating a difference value between the measurement depth value and an actual depth value of the calibration plate to obtain a measurement error; constructing a fitting function, fitting the preset environment temperature value, the light source real-time temperature value, the sensor realtime temperature value and the measurement error by using the fitting function, and calculating an optimal solution of an undetermined coefficient in the fitting function; and taking the optimal solution of the undetermined coefficient as the temperature drift coefficient of the depth camera module.

Patent description CN112393808A discloses a temperature-sensitive camera temperature compensation method and system, and belongs to the technical field of temperature-sensitive cameras. Aiming at the problems of low precision and low measurement accuracy of a temperature-sensitive camera in the prior art, the invention provides the temperature compensation method and system for a temperature-sensitive camera. A temperature sensor is connected in front of the temperature-sensitive camera, and a black body is arranged, by calculating the functions of the temperature measured by the temperaturesensitive camera, the actual temperature of the temperature sensor, the black body temperature, the environment temperature and the measured entity distance, the influence brought by factors such as the drifting of the camera, the non-uniformity of the temperature-sensitive camera, the refractive index of the temperature sensor, the environment temperature and the measured distance is calibrated. According to the invention, the temperature compensation of the temperature-sensitive camera is realized in a low-cost and high-precision manner, the corresponding calibration is real-time, the temperature- sensitive camera can be used all the time after leaving a factory and only needs to be calibrated once, a black body does not need to be arranged during measurement.

Patent description CN111182240A discloses a temperature drift self-compensation method for an image sensor. Through linearization and real-time temperature pixel drift compensation of an acquired image, the problem of distortion of an output pixel value of the image sensor caused by relatively high environment temperature is solved, so that the image sensor can normally work in a high-temperature narrow space for along time, and the environment applicability of the image sensor is improved. The method has general applicability to any image sensor affected by temperature, and enlarges the application environment of the image sensor.

The aim of the invention is to develop a suspension structure for a photosensitive image sensor and a method to make the temperature drift of an image (on sensors, especially photosensitive image sensors, such as CCD or CMOS) repeatable, and thus controllable and compensable. The temperature drift of an image is very difficult to technically eliminate or limit, and this is extremely expensive. Without a hardware modification of the camera, repeatable thermal drift cannot be achieved and cannot be controlled.

The invention relates to a suspension structure of a photosensitive image sensor, in particular of a digital camera, comprising a PCB plate with an image sensor, characterized in that the PCB plate with an image sensor has at least two resiliently seated first mounting openings or the camera base plate has at least two resiliently seated second mounting openings.

It is preferred that the first mounting openings are seated on the first flexures.

It is preferred that the second mounting openings are seated on the second flexures.

It is preferred that the PCB plate with an image sensor has four first mounting openings. It is preferred that the base plate has four second mounting openings.

It is preferred that the PCB plate with an image sensor and the camera base plate are releasably connected.

The essence of the method for repeatably controlling and compensating for the temperature drift of an image in a photosensitive image sensor, in particular of a digital camera, according to the invention is that the camera is hardware modified, the temperature drift of the camera image is recorded in steps of at least 1°C for temperature in the range from -35°C to 100°C, wherein a temperature sensor in the camera is used to record the temperature, the recorded temperature drift of the image is used to calculate a polynomial compensation model, and the calculated polynomial compensation model is used to compensate for the temperature drift of the image recorded by the camera.

It is preferred that the steps are performed cyclically, every one second, or every camera- recorded image.

It is preferred that the steps are performed before image recording or during image recording or after image recording.

The advantages of the suspension of a photosensitive image sensor and of the method for repeatably controlling and compensating for the temperature drift of an image of the photosensitive image sensor are that the non-repeatableness of the temperature drift observed in images recorded by digital cameras is eliminated or markedly reduced, production cost is low, implementation in existing products is easy and reliability is high.

Example implementations of the invention are presented in the drawing, where Fig. 1 shows a PCB plate with an image sensor mounted on the camera base and an attached lens, Fig. 2 shows a suspension structure of a PCB plate with an image sensor according to the invention in a version with Otl openings and Spl flexures, Fig. 3 shows the solution of Fig. 2 in a schematic view, Fig. 4 shows a suspension structure of a PCB plate with an image sensor according to the invention in a version with Ot2 openings and Sp2 flexures, Fig. 5 shows the solution of Fig. 4 in a schematic view.

Example 1

Fig. 2 and Fig. 3 show a suspension structure of a photosensitive image sensor comprising a PCB plate P with a camera image sensor M, the plate connected to the camera housing, i.e. a camera base B, such that the connection provides thermal degrees of freedom of the photosensitive image sensor, that is, makes it possible to freely and thus repeatably deform under the influence of a variable operating temperature of the camera. The hardware modification consists in changing the standard rigid way of mounting the PCB plate P with the image sensor M into mounting through four resiliently seated first mounting openings Otl. The first mounting openings Otl are seated on the first flexures Spl to form a frictionless and playless, flexible suspension of the PCB plate P with the image sensor M. The connection of the PCB plate P with an image sensor M to the camera base B is realized by means of screws.

Example 2

Fig. 4 and Fig. 5 show the construction of the camera base B with the lens O attached. The base B has four second mounting openings Ot2 seated on the second flexures Sp2. The PCB plate P with an image sensor M connects to the base B using the second mounting openings Ot2. This connection provides thermal degrees of freedom of the photosensitive image sensor, that is, it makes it possible to freely and thus repeatably deform under the influence of a variable operating temperature of the camera. The hardware modification consists in changing the standard rigid way of mounting the PCB plate P with an image sensor M into mounting with the use of four resiliently seated first mounting openings Ot2. The second mounting openings Ot2 are seated on the second flexures Sp2 to form a frictionless and playless, flexible suspension for the PCB plate P with an image sensor M. The connection of the PCB plate P with an image sensor M to the camera base B is realized by means of screws.

Example Implementation of the Method

The method for repeatably controlling and compensating for the temperature drift of an image in a photosensitive image sensor, in particular of a digital camera, is that the camera is hardware modified according to Example 1 or 2, then, when mounted on a dedicated stand, the camera records the temperature drift of the image in steps of 1°C for temperature in the range of from -35°C to 100°C, and to record the temperature a temperature sensor in the camera is used. The recorded temperature drift of an image is used to calculate a compensation model using a mathematical adjustment. The calculated compensation model is used to correct the image recorded by the camera, and a polynomial model is used as the compensation model. The above steps are performed cyclically, every one second, or every camera-recorded image during the image recording.

Example implementations of the invention, in order to achieved the desired effect, make it possible to use only Example 1 or only Example 2 or simultaneously use Examples 1 and 2. In any case, the effect of repeatably controlling and compensating for the temperature drift of the image of the photosensitive image sensor will be achieved.

The solution according to the invention will be used especially in 3D scanners and 2D cameras (with a CCD and CMOS image sensor), where special attention is paid to the fidelity of the image obtained or a correct representation of the position and orientation of the camera in space (medical scans, satellite photos, documentation of crime scenes, documentation of cultural heritage, etc.) and where a photo or an image, in order to be correctly taken and used, must be based on a reliable image (without shifting in space in relation to the real position of the photographed object in space).