The present invention relates to an apparatus for obtaining sensor data related to at least one storage location on an arrangement of shelves. The present invention also relates to an arrangement of shelves and a system.

Automated storage and retrieval systems (ASRS) are a known solution with various possible sizes for automatically placing and retrieving objects from defined storage locations, such as shelves in warehouses, storage rooms or other storage facilities. In some cases, the ASRS includes a pick and place robot for improving the efficiency of moving single items. A pick and place robot can be useful for minimizing or even avoiding having part of a storage and retrieval process dependent on human performance. Moreover, a pick and place robot may allow storing items with an irregular placement over a shelf or within a container and then using the robot for correctly and efficiently picking an intended item from the irregularly placed items. Furthermore, a pick and place robot may be used for storing objects within a container in a planned manner so that the objects are stored in an organised manner, such as in a manner that minimizes empty spaces between stored objects.

A pick or place operation of a pick and place robot is typically divided into two stages. First, a planning stage is performed, in which at least one image of a surface or container is captured, the at least one captured image is processed for the calculation of points for picking an intended object or for the calculation of locations to place a picked object, and a motion path is calculated for the pick and place robot to pick up the intended object or to place the picked object. Secondly, an execution stage is performed in which the robot is controlled to perform the calculated motion path.

It can be challenging to increase the efficiency of a pick and place robot when the objects to be picked or placed are stored in shelves. A disadvantage of known approaches is requiring the robot to precede the stages of the pick or place operations with yet another stage that includes robotic actuations to position the surfaces or containers in front of a sensor such as an image sensor so that the planning stage may happen. Another disadvantage from known approaches is only being able to capture images for objects, surfaces or containers on a shelf that are visible from the external periphery of the shelf. A further disadvantage of known approaches relates to requiring a lot of image sensors, which can be expensive and increase the need for maintenance. Also, in general, it is difficult and expensive to scale known approaches to bigger storage systems.

Document US10926954B2 discloses an approach in which a robotic arm is used for picking goods stored on a shelf rack. The robotic arm starts by moving a shelf or a container out of the shelf rack and onto a “removal position”, which parks the shelf/container at least partly in front of the shelf rack. At a position above the “removal position”, a fixed camera captures an image of the removed shelf/container and that image is then used for calculating picking points for an intended object and a motion path in order for the robotic arm to successfully pick the object. The robotic arm can then be controlled to pick the object from the removed shelf/container. A disadvantage of this approach is that it necessarily requires first taking the shelf/container out of the shelf rack and parking it on the “removal position” every time at least one stored object from that shelf/container is to be picked. This constraint substantially decreases the efficiency of the pick and place operations. Another disadvantage of this approach is its dependency on a pick and place robot that must also be able to move the entire shelf/container to and from the “removal position”. A further disadvantage is that this approach can be difficult and expensive to scale and apply to larger storage systems. Moreover, this approach is likely to allow only a low storage density due the space that is needed in front of the shelf rack in order to establish the “removal position” and have sufficient space for the robotic arm to move safely.

Another approach is disclosed in document CN107890243A. One embodiment described with reference to Figure 7 in the document includes a set of fixed cameras positioned at a distance in front of a shelf rack. The cameras are provided along column structures fixed on the floor. Images from the fixed cameras are used for performing the planning stage of pick and place operations, which are then carried out by a robot movable in front of the shelf rack. Another embodiment disclosed with reference to the same Figure 7 in the document provides that the cameras are vertically movable along the column structures and rotatable around the vertical axes of the columns. These two embodiments disclosed with reference to Figure 7 of the document, although not requiring the pick and place operations to be preceded with motions for positioning the objects in front of the cameras, have the disadvantage that the only motion paths that can be calculated are motion paths for picking objects that are visible from the periphery of a shelf. Thus, the approach disclosed by these embodiments is not applicable for calculating motion paths to pick objects that are not visible to the cameras, such as objects behind other objects when observed from the position of the column structures. Also, this approach is likely to allow only a low storage density due the empty space that is needed between the shelf rack and the columns.

A further known approach is disclosed in the same document CN107890243A. The embodiments described with reference to Figures 1 to 6 in the document include a set of fixed cameras installed on the shelfs so that images of the objects on the shelf may be captured. This approach has several disadvantages. First, it is strongly dependent on the viewing angle of the cameras being used in order to capture images of all objects storable on a shelf. Cameras with a smaller viewing angle may require longer subject distances, which may in turn make it necessary to provide empty space between the camera and the objects. Also, cameras with smaller viewing angles may make it necessary to provide more cameras. A high number of cameras may also be needed independently of the viewing angle of the camera. Moreover, a careful planning of the camera positions must be performed in order to ensure that all storage shelf locations are visible to at least one camera. Secondly, this solution requires sufficient computing power to multiplex and/or process data from all the cameras being used. Thirdly, this approach may be expensive due to the high number of cameras needed, effort to study camera positions, installation of the cameras, cables and control unit(s), and maintenance. Fourthly, the disadvantages of this system make it difficult to scale the solution to bigger storage systems.

The invention will now be disclosed and has for its object to remedy or to reduce at least one of the drawbacks of the prior art, or at least provide a useful alternative to the prior art. The object is achieved through features, which are specified in the description below and in the claims that follow. The invention is defined by the independent patent claims, and the dependent claims define advantageous embodiments of the invention.

According to a first aspect of the invention, there is provided an apparatus for obtaining sensor data related to at least one storage location on an arrangement of shelves, the apparatus being installable on the arrangement of shelves. The apparatus is adapted to manoeuvre at least one image sensor for generating the sensor data related to the at least one storage location, and the manoeuvring of the at least one image sensor is performable along at least one degree of freedom relative to a shelf on which the at least one storage location is located.

The inventors realised that an apparatus according to the first aspect of the invention has the advantage of using the same sensor for covering a support area on the shelf that is bigger than what would be captured from a fixed position. Also, the disadvantages of using fixed cameras as described above are solved. Moreover, the apparatus allows performing the planning stage (e.g. image capturing and processing of picking points or storing placements) while the pick and place robot is in the execution stage (e.g. performing picking and placing tasks) somewhere else on the arrangement of shelves, which a achieves an efficient dual stage of planning and executing in parallel. Furthermore, the apparatus can operate the sensors/cameras at low subject lengths, thus being less demanding in terms of space between shelves or other free space distances to be available on the shelves or around the stored objects. This, thus, contributes to an increased storage density. Also, it has been realised that an advantage of the apparatus is that the planning stage can be performed for all stored objects, even those that are not directly visible from the vicinity of the shelf. This overcomes the inefficiency in known prior art where the picking points are only calculated after taking the objects out of the shelf or where a storage placement for an object is only calculated after taking a container of a shelf out of the arrangement of shelves. It also overcomes the challenges in other known prior in which the stored objects are only scannable if they are directly visible from the vicinity of the shelf.

The manoeuvring of the at least one image sensor may comprise moving the at least one image sensor along at least one degree of freedom. This possibility allows requiring less space to be provided between shelves, which is advantageous for achieving a high storage density. Also, it becomes possible to provide the same subject distance along a shelf when scanning a shelf, which can simplify subsequent image processing. In one embodiment, the apparatus may comprise a cartesian coordinate robot for moving the at least one image sensor.

The manoeuvring of the at least one image sensor may comprise rotating the at least one sensor along at least one degree of freedom. This possibility allows scanning the same stored object or shelf location from multiple perspectives to get better overview of available picking points. Also, this possibility of having additional perspectives can be useful to increase the quality of the processed picking points by computing multiple pick points and then rank them by level of confidence that a robot will be successful on picking them. Moreover, the possibility of rotating can be advantageous in obtaining data (e.g. capturing images) from the at least one sensor in an inclined manner relative to the shelf and/or perpendicular relative to a shelf with a different inclination than the apparatus’ moving axes. In one embodiment, the apparatus comprises a gimbal actuator for rotating the at least one image sensor.

The apparatus may be adapted to be installed above the shelf and /or on an under side of another shelf of the arrangement of shelves.

In one embodiment, the manoeuvring of the at least one image sensor may be further performable along at least one degree of freedom for obtaining sensor data related to at least one storage location external to the arrangement of shelves. An example of such an external storage location can be a conveyor belt or a container placed in front of the arrangement of shelves.

According to a second aspect of the invention, there is provided an arrangement of shelves comprising:

- at least one storage location for storing objects;

- at least one apparatus according to the first aspect of the invention; and

- at least one control unit for controlling the at least one apparatus, each control unit comprising a processing unit.

The control unit may be adapted to access a database relating each of the at least one storage location with an apparatus of the arrangement of shelves. Also, the control unit may be configured to carry out the steps of:

- receiving a request for sensor data related to a storage location;

- accessing the database and identifying an apparatus related to the requested storage location;

- controlling the identified apparatus to manoeuvre the respective at least one image sensor and obtain sensor data related to the requested storage location;

- responding to the received request with a response comprising the sensor data obtained from the at least one image sensor. The adaptation of the control unit to access the database may include:

- providing the control unit with a memory for storing the database locally, and /or

- providing the control unit with a communication interface to communicate with a remote device provided with a memory for storing the database.

The arrangement of shelves may comprise at least two apparatus installed on a same shelf. Also, the arrangement of shelves may comprise an apparatus installed on at least two shelves. In this embodiment, the apparatus may be adapted to manoeuvre the at least one image sensor relative to the at least two shelves, so that a same image sensor is used for generating sensor data about storage locations along at least two shelves.

According to a third aspect of the invention, there is provided a system comprising:

- at least one arrangement of shelves according to the second aspect of the invention; and

- a master control unit for controlling the system, the master control unit comprising a processing unit, a memory and a communication interface for communicating with each control unit of the at least one arrangement of shelves.

The memory of the master control unit is configured with at least one relation between each storage location of the at least one arrangement of shelves and a control unit of an arrangement of shelves.

The master control unit may be configured to carry out the steps of:

- receiving a request to obtain sensor data related to a storage location;

- using the configured at least one relation, identifying a control unit of an arrangement of shelves;

- communicating with the identified control unit to obtain sensor data related to the storage location;

- responding to the received request with a response comprising the sensor data obtained from the identified control unit.

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 is a schematic perspective view of an apparatus embodiment including a cartesian coordinate robot for moving a sensor along two degrees of freedom;

Figure 2 is a schematic perspective view of another apparatus embodiment including a plurality of cartesian coordinate robots, each robot moving a sensor separately along a degree of freedom;

Figure 3 is a schematic partial perspective view of a further apparatus embodiment including a gimbal actuator for rotating a sensor along at least one degree of freedom;

Figure 4 is a schematic partial perspective view of another apparatus embodiment including a cartesian coordinate robot for moving a gimbal actuator along at least one degree of freedom, the gimbal actuator being provided for rotating a sensor along at least one degree of freedom;

Figure 5a is a schematic elevation view of an arrangement of shelves embodiment with three shelves;

Figure 5b is a schematic view of a system embodiment including the arrangement of shelves shown in Figure 5a and a pick and place robot embodiment;

Figure 6 is a schematic elevation view of another arrangement of shelves embodiment including an apparatus embodiment adapted to obtain sensor data related to a storage container provided on a conveyor belt in front of the arrangement of shelves.

Turning now to Figure 1 , it shows a shelf 210 on which there are sixteen storage containers 400 stored. Each container 400 is used for storing objects 410 and defines a storage location. In order to simplify the illustration shown in Figure 1 , only one container 400 is shown containing objects 410. The objects 410 may be stored and retrieved from the containers 400 on the shelf 210 using a pick and place robot (not shown) in many ways. Also, the shelf 210 is part of an arrangement of shelves 200 not shown in Figure 1. Moreover, the skilled person will see throughout the following description that the apparatus is useful for other types of shelves capable of storing different numbers and arrangements of containers 400. For example, a row of containers 400 may be provided over a shelf 210 with space for only the single row of containers 400. In another example, the arrangement of containers 400 may be more complex and have a higher number of containers 400 than the ones shown in the Figures. In a further example, the shelf 210 may be inclined and the containers 400 may be biased to slide or move towards one side of the shelf 210.

Above the shelf 210, there is provided an apparatus embodiment 100 for obtaining sensor data related to the storage locations, i.e. the containers 400, located on the shelf 210.. The apparatus 100 is installed on the arrangement of shelves 200, for example on the underside of another shelf not shown in Figure 1 , and it includes a moveable image sensor 1 10 for obtaining the sensor data.

The apparatus 100 includes a cartesian coordinate robot 120 for moving the sensor 110 along two perpendicular axes parallel to the shelf 210. The bidirectional arrows shown in Figure 1 illustrate the available manoeuvrability along the axes. Thus, the apparatus embodiment 100 shown in Figure 1 is adapted to move the sensor 1 10 along two degrees of freedom relative to the shelf 210. With the ability to move the sensor 110 in a controlled manner above the shelf 210, the sensor 1 10 can be positioned above any of the containers 400 stored on the shelf, which allows using the image sensor 110 to obtain sensor data about the objects 410 stored within any of the containers 400 on the shelf 210. In other words, the apparatus 100 can obtain sensor data related to any of the storage locations located on the shelf 210.

The image sensor 1 10 can be, for example, an image sensor for generating image or video data which can then be processed. The skilled person will know that there are image processing methods available for processing image or video data from an image sensor, such as for calculating picking points or calculating storage positions for an object 410 to be placed within a container 400. Also, the sensor 1 10 can be replaced by a three- dimensional camera or by more than one image sensor, which allows obtaining three- dimensional data about a storage location, such as the interior of a container 400. It can be observed that the same sensor 1 10 in Figure 1 can be used for covering the entirety of the shelf 210, even if the sensor 110 would be unable to capture the entire shelf 210 in its field of view when installed at a fixed position. Thus, the apparatus 100 is efficient in reducing the number of sensors 1 10 that are needed.

It can also be observed that the apparatus 100 is capable of moving the sensor 110 over a container 400 that may not be directly visible from the vicinity of the shelf 210, for example one of the containers 400 in the central region of the shelf 210 which are surrounded by containers 400 on all sides. This ability may enable further efficiency gains when obtaining data with the sensor 1 10.

Furthermore, it can also be seen that the sensor 110 can be positioned at a short distance, i.e. low height in the specific embodiment of Figure 1 , from a container 400. Thus, the apparatus 100 can be less demanding both in terms of free space needed above the shelf 210 as well as of the angle of view of the sensor 110 that is chosen to be purchased/used for implementing the apparatus 100.

Moreover, it can be observed that the apparatus 100 can be provided so that the moving axes for the sensor 110 are substantially parallel to the shelf 210, which will result in the distance from the sensor 110 to the shelf 210 to be uniform over all the containers 400. This possibility can be advantageous in that the quality of the data obtained with the sensor 1 10 is improved, because all the obtained data can be processed with the assumption of a uniform distance to the subject without requiring pre-processing related to correcting the orientation of the captured data. It can also be advantageous in making it more efficient to use artificial intelligence solutions, such as a neural network for identifying picking points, as the data will have less noise with respect to varying distances between the sensor 1 10 and the bottom of a container 400 stored on the shelf 210.

Figure 2 shows a similar situation as the one shown in Figure 1 , however the apparatus embodiment 100 in Figure 2 includes four cartesian coordinate robots 121-124, each for moving an image sensor 1 11-1 14 along one independent axis. Thus, each sensor 111 — 1 14 is maneuvered along one degree of freedom. The cartesian coordinate robots 121 — 124 are organised so that these move the sensors 1 11-1 14 in parallel columns above the containers 400. Although the number of sensors 11 1-114 used is higher in comparison to the apparatus embodiment 100 in Figure 1 , which may entail a higher cost than the cost of the embodiment in Figure 1 , the operational efficiency is also increased because the apparatus 100 can obtain data from several storage locations on the shelf 210 at same time. The apparatus embodiment 100 shown in Figure 2 is advantageous when used in processes with a high storage and retrieval rate and a high need for data related to the contents of the containers 400, i.e. storage locations, stored on the shelf 210.

Figure 3 shows an apparatus embodiment 100 including a rotatable sensor 115. The apparatus 100 provides the sensor 1 15 from a fixed position on a not shown arrangement of shelves 200, and the apparatus 100 includes a gimbal actuator 130 for rotating the sensor 115 along at least one degree of freedom. For example, embodiments of the gimbal actuator 130 may allow rotating the sensor 115 from side to side, upwards or downwards, and/or around a radial axis.

The apparatus 100 shown in Figure 3 has the advantage of being capable of obtaining data related to several storage locations on the shelf 210 which are visible from the apparatus 100 and requiring only the gimbal actuator 130 for manoeuvring the sensor 1 15. In comparison to the apparatus embodiments 100 shown in the preceding figures, the apparatus in Figure 3 may have a lower cost and require less maintenance due to the reduced number of moving parts.

Figure 4 shows part of an apparatus embodiment 100 including both a cartesian coordinate robot and gimbal actuator for manoeuvring an image sensor 1 16.

The image sensor 116 is illustrated, in an overlayed manner, at three positions 1 16a-1 16c for obtaining data related to the contents of the container 400. Using actuations from both the cartesian coordinate robot and the gimbal actuator, the sensor 116 is maneuvered sequentially through the three different sensor positions 116a-1 16c. This capability allows identifying picking points or storing positions from different perspectives, and thus it increases the quality of the picking points or storing positions that are calculated. The embodiment in Figure 4 can, for example, be used for obtaining a plurality of possible picking points for a stored object 410 and rank the obtained picking points by confidence that a picking robot will successfully pick the object 410. It can be used similarly for obtaining a plurality of possible storing positions for a picked object and rank the obtained positions by confidence that the picked object will fit. Also, it becomes possible to obtain data similarly to how a three-dimensional camera operates and thus obtain a three- dimensional view of the contents of a container 400.

Figures 5a and 5b show an arrangement of shelves 200 from an elevation view and from a perspective view, respectively. The arrangement of shelves 200 is schematically illustrated as a shelf rack, although the skilled person will see that many other types of arrangement of shelves 200 can be provided, such as wall mounted shelves, ceiling hanging shelves, carousel mounted shelves or other known arrangement types for at least one shelf. The arrangement of shelves 200 includes three shelves 211-213 supporting containers 400, three apparatus embodiments (illustrated only in Figure 5a) for obtaining data related to picking points for objects 410 stored on the containers 400 and a control unit 220 (shown only in Figure 5b) for controlling the three apparatus embodiments.

Figure 5a shows how the three apparatus embodiments are installed on the arrangement of shelves embodiment 200. Each shelf 211-213 is provided with an apparatus installed above the shelf 211-213. The skilled person will see that the three illustrated apparatus embodiments can be any of the apparatus embodiments shown in Figures 1 , 2 and 4. For the bottom shelf 211 and the middle shelf 212, the respective apparatus embodiment is installed on the underside of the respective above appearing shelf 212,213. For the top shelf 213, the apparatus embodiment is installed by including structural components for providing the apparatus at a plane above the top shelf 213.

It can be observed that the three apparatus embodiments shown in Figure 5a are advantageous in that they can be provided in arrangements of shelves 200 in which a high storage density is achieved. Each apparatus includes a cartesian coordinate robot 126— 128 and consumes only the vertical space necessary for manoeuvring the respective sensors 1 17-1 19 between shelves for obtaining sensor data about the contents of the containers 400.

The skilled person will also see that an apparatus embodiment according to the present invention can be installed in alternative ways on the arrangement of shelves 200 so that the manoeuvring of the image sensor is performed along at least one degree of freedom relative to the shelf. For example, the skilled person may prefer to install an apparatus embodiment directly on the shelf so that the sensor is maneuvered among the stored objects. In this example, the objects may be stored in at least one row around the positions that are reachable by the sensor so that the latter may obtain data related to the stored objects. Another option could be to install an apparatus for manoeuvring a sensor along planes substantially perpendicular to a shelf. Another option may be to install an apparatus adapted to manoeuvre a sensor along at least one degree of freedom relative to a plurality of shelves, in which the same sensor can be used for obtaining data about shelf locations on more than one shelf.

As mentioned above, the control unit 220 (shown only in Figure 5b) of the arrangement of shelves 200 shown in Figures 5a and 5b controls the three apparatus embodiments that are installed on the arrangement of shelves 200.

The control unit 220 is adapted to access a database relating each of the containers 400, i.e. storage locations, with one of the three apparatus embodiments installed on the arrangement of shelves 200. The access to the database allows the control unit 220 to identify which of the apparatus embodiments installed on the arrangement of shelves 200 relates to a storage location. Thus, the control unit 220 can be used as a controller for the arrangement of shelves 200.

The database is a data structure that allows identifying and relating storage locations (e.g. shelf location “A5” on the “third” shelf) with apparatus embodiments. The storage locations can be configured by the skilled person in accordance with possible container positions on the arrangement of shelves 200.

When a storage location is to be scanned, the control unit 220 can be configured to carry out the following steps:

- receiving a request for sensor data related to a storage location;

- accessing the database and identifying an apparatus related to the requested storage location;

- controlling the identified apparatus to manoeuvre the respective at least one image sensor 1 17-1 19 and obtain sensor data related to the requested storage location;

- responding to the received request with a response comprising the sensor data obtained from the at least one image sensor 117-1 19. Thus, the control unit 220 carries out the steps of controlling the appropriate apparatus embodiment on the arrangement of shelves 200 for obtaining sensor data, such as images or video, related to a storage location.

The access of the control unit 220 to the database can be implemented in several ways.

The control unit 220 may be provided with a memory for storing the database. Thus, the control unit 220 can access the database directly in a local memory. This approach has the advantage of keeping the database contained within the arrangement of shelves 200, which can be advantageous in minimizing the effort needed to replace, remove or add an arrangement of shelves 200 to a system 300 (as shown in Figure 5b). Keeping the database locally can thus be advantageous for the scalability of a system 300.

Alternatively or in addition, the control unit 220 may be provided with a communication interface to communicate with a master control unit 320 of a system 300 (as shown in Figure 5b). In this approach, the master control unit 320 operates as remove device. The communication interface can be provided so that the control unit 220 and the master control unit 320 work together in accordance with an organizational structure.

In one organizational structure, the master control unit 320 stores the digital representation and the communication interface of the control unit 220 is configured to communicate with the master control unit 320 to perform all operations related to the database. This approach may be advantageous in that the control unit 220 requires less resources in terms of computational power, which in turn lowers the cost and complexity of manufacturing the arrangement of shelves 200.

In another organizational scheme, the control unit 220 and the master control unit 320 collaborate in the storage of the database using a known decentralized database scheme, such as a replication or a sharding database scheme. This is particularly advantageous in bigger storage and retrieval systems in which there are several control units 220 connected to a master control unit 320.

In Figure 5b, a system embodiment 300 is shown. The system 300 includes the arrangement of shelves 200 for storing objects 410 within containers 400 supported on three shelves 211-213, a pick and place robot embodiment 310 for retrieving and storing objects 410 within the containers 400 in the arrangement of shelves 200, and a master control unit 320 configured to control the pick and place robot 310 and the arrangement of shelves 200. The three apparatus embodiments shown in Figure 5a have been hidden in Figure 5b, for the purpose of simplifying the schematic illustration. Also, the pick and place 310 is illustrated in Figure 5b as being at a distance from the arrangement of shelves 200, although the skilled person will see without requiring inventive skills that the pick and place robot 310 can be placed at a distance from the arrangement of shelves 200 that is suitable for picking and placing objects from/on the arrangement of shelves 200.

Different approaches may be implemented for the pick and place robot embodiment 310 to retrieve and store objects 410 from/on the containers 400.

An approach is to provide the arrangement of shelves 200 with sufficient vertical space between the shelves 211-213 so that the pick and place robot 310 may enter the space above the containers 400 and pick objects 410 directly from a container 400 stored on a shelf 21 1-213. This may be performed over several rows of containers 400, as long as the distance between shelves 211-213 of the arrangement of shelves 200 has been dimensioned appropriately to allow the pick and place robot 310 to enter the intended volume above the containers 400. In this approach, the pick and place robot 310 may be, for example, an embodiment of the pick and place robot shown in Figures 1 and 2 in WO 2020/067907 A1 , although the skilled person will see that other robot embodiments for picking and placing objects 410 could also be used, such as an industrial robotic arm provided with a tool suitable for picking and placing objects 410.

It may be advantageous to include, in the system 300, at least one robot for retrieving and storing containers 400 from/on a shelf 21 1-213 of the arrangement of shelves 200. This robot is not shown in the figures. This at least one robot for retrieving and storing containers 400 can be controlled by the master control unit 320 for moving a container 400 between two shelves of the arrangement of shelves 200 or between shelves of different arrangements of shelves. This allows shuffling and/or changing container positions on the shelves 211-213 of the arrangement of shelves 200 so that the pick and place robot 310 may then have an easier access to the contents of the stored containers 400. Another approach is to provide the arrangement of shelves 200 with minimal empty vertical spaces so that the storage density of the arrangement of shelves 200 is maximized. This approach may make it impractical to have the pick and place robot 310 enter the space above a container 400 while the latter is stored in an arrangement of shelves, as there will be insufficient space between shelves 21 1-213 to successfully pick an object 410 from a container 400 in this way. Thus, the pick and place robot embodiment 310 may be provided as disclosed in Figures 3 and 4 in WO 2020/067907 A1 , in which the pick and place robot embodiment includes a supporting frame for supporting a container 400 in a position outside of the arrangement of shelves 200 and picking or storing objects 410 from that position.

The system 300 includes a master control unit 320 for controlling the pick and place robot 310 and the arrangement of shelves 200. The memory of the master control unit 320 can be configured with at least one relation between each storage location of, i.e. each container 400 stored on, the at least one arrangement of shelves 200 and the control unit 220. Should the system 300 include more arrangements of shelves 200, the master control unit 320 is thus capable of relating the storage locations in the different arrangements of shelves 200 with the corresponding control unit 220.

The master control unit 320 is configured to carry out the following steps:

- receiving a request to obtain sensor data related to a storage location;

- using the configured at least one relation, identifying a control unit 220 of an arrangement of shelves 200;

- communicating with the identified control unit 220 to obtain sensor data related to the storage location;

- responding to the received request with a response comprising the sensor data obtained from the identified control unit 220.

It can be observed that the operation of the arrangement of shelves 200, which includes operating the three apparatus embodiments and the control unit 220, combined with the operation of the master control unit 320 and the pick and place robot 310 contributes to the efficient execution of retrieval and storage operations. This advantage results from the ability to perform the planning stage of a pick or place operation independently from the execution stage. The apparatus embodiments installed on the arrangement of shelves 200 do not have to wait for the pick and place robot 310 to complete operations before they can initiate the planning stage and start obtaining sensor data related to the contents of the containers 400 stored on the arrangement of shelves 200.

The skilled person will also see that the system 300 herein disclosed may be used for planning efficiently how an object 410 is to be stored within a container 400. The data obtained by an apparatus embodiment, besides being useful for processing picking points, can be used for processing storage spaces that are available within the container 400 for storing the object 410. This can also be achieved independently of the execution of operations by a pick and place robot, and therefore allows achieving a high rate of storage operations.

Figure 6 shows another arrangement of shelves embodiment 200 including two shelves 214-215 for storing containers 400 and two apparatus embodiments for obtaining sensor data related to the storage locations located on the shelves 214-215. It can be observed in Figure 6 that the shelves 620-621 are provided with an inclination, which causes the containers 400 stored on the shelves 214-215 to be biased to slide or move towards one side of the arrangement of shelves 200, specifically the right-hand side in Figure 6.

In front of the arrangement of shelves 200, i.e. on the right-hand side in Figure 6, there is provided a conveyor belt 330 for transporting containers 400 between locations in the place where the arrangement of shelves 200 is installed. Such a place can be a warehouse or a distribution centre, for example. Such a disposition of components is suitable for providing a robot to move containers 400 from the container belt 330 and store them on the arrangement of shelves 200, and vice-versa. Alternatively or in addition, a pick and place robot can be provided to pick objects stored in a container 400 provided on the conveyor belt 330 and place picked objects on a container 400 stored in the arrangement of shelves 200, and vice-versa.

In the arrangement of shelves 200, each apparatus is installed above each shelf 214-215 and comprises a cartesian coordinate robot 620-621 for moving an image sensor 610— 611. It can be observed that the apparatus installed for the bottom shelf 214 is adapted to obtain sensor data related to the storage container 400 provided on the conveyor belt 330 in front of the arrangement of shelves 200. This is advantageous in that it allows increasing the efficient of picking or placing objects from/into a container 400 on the conveyor belt 330. The planning stage for picking or placing operations related also to containers 400 on the conveyor belt 330 can be done in parallel to other execution stages. Thus, the arrangement of shelves 200 is advantageously suitable for an easier integration in a storage and retrieval system where there are containers 400 automatically moving to and from the arrangement of shelves 200.

The apparatus embodiment installed for the bottom shelf 214 can be adapted in a known way so that the image sensor 610 is capable of reaching the container 400 on the conveyor belt 330. For example, the apparatus can be provided with a linear actuator such as an hydraulic cylinder, a threaded rod electric actuator, a sliding electric actuator or a linear solenoid actuator.

Generally, the terms used in this description and claims are interpreted according to their ordinary meaning the technical field, unless explicitly defined otherwise. Notwithstanding, the terms “comprises” and “comprising” and variations thereof mean that the specified features, steps or integers are included. These terms are not interpreted to exclude the presence of other features, steps or integers. Furthermore, the indefinite article “a” or “an” is interpreted openly as introducing at least one instance of an entity, unless explicitly stated otherwise. An entity introduced by an indefinite article is not excluded from being interpreted as a plurality of the entity.

The features disclosed in the foregoing description, or in the following claims, or in the accompanying drawings, expressed in their specific forms or in terms of a means for performing the disclosed function, or a method or process for obtaining the disclosed results, as appropriate, may, separately, or in any combination of such features, be utilised for realising the invention in diverse forms thereof.

While the invention has been described in conjunction with the embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the embodiments of the invention set forth above are considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the scope of the claims.