Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
MODULAR DISPLAY APPARATUS AND METHOD FOR CONTROLLING THEREOF
Document Type and Number:
WIPO Patent Application WO/2022/228683
Kind Code:
A1
Abstract:
A modular display apparatus is provided. The modular display apparatus includes a plurality of display modules, a single connector on every display module for serial type connection to a single raster image processor configured to crop portions the image to be displayed into sub images, addressed to individual display modules. The exact position, orientation in space and unique address information of every individual display module is determined by digital optical recognition of a digital control image of the entire display module configuration after the plurality of display modules are mechanically assembled into a large screen and electrically connected.

Inventors:
PETKANCHIN LACHEZAR (BG)
Application Number:
PCT/EP2021/061287
Publication Date:
November 03, 2022
Filing Date:
April 29, 2021
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
NRG TECH LTD (BG)
International Classes:
G09G3/32; G06F3/14; G09F9/302
Foreign References:
US20200241828A12020-07-30
US20180113664A12018-04-26
US20160086582A12016-03-24
US20140193037A12014-07-10
US20200133615A12020-04-30
US20200348898A12020-11-05
Attorney, Agent or Firm:
PUCHBERGER & PARTNER PATENTANWÄLTE (AT)
Download PDF:
Claims:
Claims

1. A modular display apparatus, comprising a. a plurality of display modules (1 ), each with an electronic memory wherein an individual address number (3) is stored, and b. a raster image processor (7), which is connected to all display modules

(1) and configured to send an instruction to all display modules (1) to display its unique address number, their address number (3) and an orientation-sensitive set-up graphic (2), characterized in that c. it comprises a digital camera (9), which is configured to photograph all display modules (1) with their address numbers (3) and set-up graphics

(2) visualized and submit the obtained control image to the raster image processor (7), and d. the raster image processor (7) is configured to analyze the control image and to determine the spatial location, address number, and spatial orientation of every display module (1).

2. A modular display apparatus according to claim 1 , characterized in that the raster image processor (7) is further configured to store information describing all display modules (1 ) of the plurality of display modules with their address number, location and orientation in a set-up table.

3. A modular display apparatus according to claim 1 or 2, characterized in that the raster image processor (7) is configured to receive an input image to be displayed from an external apparatus, crop portions of the input image, and send those cropped portions with proper orientation to a particular display module (1) by addressing it with its address number.

4. A modular display apparatus according to claim 3, characterized in that the plurality of display modules (1) are configured to recognize their address numbers and only then to acquire the following cropped portions of the input image to be visualized and to visualize it.

5. A computer-implemented method to control a modular display apparatus with a plurality of display modules (1), each with an electronic memory wherein an individual address number (3) is stored, by a raster image processor (7), which is connected to all display modules (1), characterized in that a. the raster image processor (7) sends instructions to all display modules (1) to display its unique address number, their address number (3) and an orientation-sensitive set-up graphic (2), b. a digital camera (9) photographs all display modules (1 ) with their address numbers (3) and set-up graphics (2) visualized and submits the obtained control image to the raster image processor (7), and c. the raster image processor (7) analyzes the control image and to determines the spatial location, address number, and spatial orientation of every display module (1 ).

6. A computer-implemented method according to claim 5, characterized in that the raster image processor (7) stores information describing all display modules (1) of the plurality of display modules with their address number, location and orientation in a set-up table.

7. A computer-implemented method according to claim 6, characterized in that the raster image processor (7) receives an input image to be displayed from an external apparatus, crops portions of the input image into as many frames as there are display modules (1), and submits those cropped portions with the proper orientation and location taken from the set-up table to a particular display module (1) by addressing it with its address number, wherein the position of the display modules (1) in the modular display apparatus is the same as the position of the cropped portion in the input image.

8. A computer-implemented method according to claim 7, characterized in that the display modules (1) only acquire and visualize the cropped portions of the input image from the raster image processor (7) when they recognize their respective address number.

9. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 5 to 8.

10. A raster image processor adapted to be connected to a plurality of display modules (1) and further adapted to perform the steps of the method of claims 5 to 8.

Description:
Modular display apparatus and method for controlling thereof

This application is filed in the name of NRG Tech Ltd., j.k. Bokar bl. 43, ap. 1 , 1404 SOFIA (BG), VAT-No. BG201220396.

The invention relates to a modular display apparatus and a controlling method thereof. Particularly, the invention relates to a modular display apparatus with a screen that can be extended, and a controlling method thereof.

Technological advance in recent years gave rise to variety of multimedia devices, big and small. In particular, extra-large display devices, consisting of many identical modules put together to appear as one large multimedia screen are being developed.

Very often there is a need to alter both the size and the shape of such a modular large screen. For example, such a screen can be used in big concert events and in smaller venues like conference rooms. Assembly and disassembly should be quick and with minimal possibility for errors on the part of the technical team in charge.

Flowever, there are problems with such modular large screen devices in the related art. In some cases, the screen could not be resized after being assembled once. In other examples the screen needs a manual input from a skilled engineer after being resized.

More recent devices can automatically determine the position of every individual module of the screen by magnetic field sensors and electromagnets mounted on the sides of the display modules. By running a complicated algorithm to switch electromagnets on and off, and reading electromagnetic sensors, the exact topology of the entire screen can be determined. In most cases it is assumed that the large screen will have only a rectangular shape and all display modules need to be oriented vertically in strictly the same way. It is an object of the invention to overcome these shortcomings and provide a modular display apparatus with a screen that can be extended, and a controlling method thereof.

According to the invention, there is provided a modular display apparatus comprising a plurality of display modules, connected in arbitrary order, and a raster image processor connected to the to the plurality of display modules. The raster image processor is configured to send its unique address number to all display modules, and each display module is configured to display both its unique address number and the address number of the raster image processor received. There is further provided a digital camera, which is configured to photograph the display modules with their address numbers visualized and submit the obtained control image to the raster image processor.

The raster image processor is configured to analyze the control image by optical character recognition software and determine the orientation in space, the address number and the location of each display module relative to neighboring display modules, and record these data in a set-up table.

The raster image processor is further configured to extract a portion of an input image to be displayed and, based on the set-up table, send it to the specified display module with its unique address number and properly rotated so that the specified display module recognizes its address number and acquires the portion of the input image and displays it so that it appears on proper location with correct orientation. The raster image processor is configured to repeat the process for all of the remaining display modules.

Initially, the display modules are assembled together regardless of any order or orientation space. The only condition is to properly align the sides of neighboring display modules. Every display module of the plurality of display modules has its own address number stored in the memory inside it, together with other data, specific for the particular display module. Address number and other specific data is programmed into every display module memory during manufacturing. In a preferred embodiment the plurality of display modules are connected to the raster image processor in series. Star or wireless connections are also possible embodiments. No consideration is taken of the order of connection of the plurality of display modules to the raster image processor. No manual input is fed to the raster image processor of the number of display modules connected or the order of connection. Then initial set-up procedure is initiated by the raster image processor.

During the initial set-up procedure, the raster image processor sends over the serial connection line its address number and a universal command to all display modules of the plurality of display modules connected to visualize their set-up graphic. The set-up graphic is individual for every display of the plurality of display modules and comprises the display’s address number and other individual data. The set-up graphic may comprise symbols, figures and/or QR codes containing a unique address number for each display module, together with the address number of the raster image processor acquired over the serial connection and other data, related to the display module.

All display modules in the plurality of display modules recognize the universal command and visualize their set-up graphics. The composite image of all displays of the plurality of display modules with all their set-up graphics displayed on them is then photographed by a digital camera and this control image is fed back to the raster image processor. The control image contains the set-up graphics visualized by all displays of the plurality of display modules. The control image of the plurality of display modules is analyzed by the raster image processor and every display module’s location and orientation in space is determined by analyzing the set-up graphics displayed. In particular the set-up graphic may contain a figure which is orientation sensitive, like a right-angle triangle.

The set-up graphics are present in the control image taken of the plurality of display modules. A specialized algorithm running on the raster image processor identifies the set-up graphics in the control image, as well as their orientation in space, which indicates the orientation of every display module of the plurality of display modules. Specialized software on the raster image processor scans the control image and locates all orientation sensitive figures present in it and records their relative position and orientation in a set-up table. The position of every module is recorded in the set-up table in terms of relative coordinates, wherein every step is the length or width of one display module. This information is indicative of the position and orientation of all display modules of the plurality of display modules. The address numbers of the display modules are also visualized on the set-up graphic and therefore are also present in the control image of the plurality of display modules. This address number is displayed in a precise position relative to the orientation sensitive figure. Taking into account the orientation and position of all orientation sensitive figures, the set-up graphics of all address numbers and the address number of the raster image processor are cropped from the control image and fed to an optical character recognition (OCR) software which extracts exact address numbers of all display modules of the plurality of display modules, together with the address number the raster image processor.

For every display module of the plurality of display modules, a record is created in the set-up table, containing the address number, location coordinates, orientation in space as well as angle and other data of the display module. After all records, one for every display module, are added to the set-up table, the raster image processor runs a test procedure. Consecutively all display modules are addressed with their address numbers as from the filled set-up table followed by status request command. In the same sequence the display modules recognize their address numbers and reply with “ok” or an error message. If certain display module did not reply then the raster image processor issues an error message.

The raster image processor is further configured to crop an input image received from an external device, based on the identified locations of the display modules, and display each of the cropped image frames on each of the plurality of display modules based on the identified locations and the address number of the respective display modules. This enables to display a large image on a plurality of display modules which are located in arbitrary order and orientation. Every display module of the plurality of display modules is configured to receive packets of data from the raster image processor by recognizing its address number, which is attached to every packet of data. Every display module is further configured to visualize digital images received from the raster image processor in one or more data packets over the series connection.

According to the invention, there is provided a method for controlling a modular display apparatus, the method comprising: identifying the number of display modules in the plurality of display modules and locating the position, orientation and unique address number of every display module by extracting information from a control image of the plurality of display modules. The method may further comprise the step of cropping an input image, received from external apparatus, into as many image frames as there are display modules, rotating the image frame if necessary, and sending every cropped image frame to a particular display module, based on location, orientation and address number of the display modules, wherein the position of the display in the modular display apparatus is the same as the position of the cropped image in the input image.

The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

Fig. 1 is a diagram illustrating a display apparatus according to an embodiment;

Fig. 2 is a diagram illustrating a part of a display apparatus according to an embodiment;

Fig. 3 is a diagram illustrating a large-size display apparatus according to an embodiment;

Fig. 4 is an exemplary diagram of how display modules position in space is determined; Fig. 5 is a flow chart illustrating how the raster image processor crops portions of the input image and sends it to a display with particular address number, associated with its location and orientation in space;

Fig. 6 is an exemplary set-up table, containing description of all connected display modules with their address numbers, location as (X, Y) coordinates and their orientation as an angle; Fig. 7 is a diagram illustrating an example of display modules visualizing an exemplary input image;

Fig. 8a - 8f is a set of diagrams illustrating the images the raster image processor sends to individual display modules, so that the exemplary image from Fig. 7 appears correctly on the plurality of display modules;

Fig. 9 is a diagram illustrating an example of a configuration of display modules where there is a missing display module in the middle;

Fig. 10 is the set-up table, describing the configuration of display modules with one missing module in the middle;

While the embodiments of the disclosure will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, it should be noted that the embodiments are not for restricting or limiting the scope of the disclosure to a specific embodiment. hereinafter, the disclosure will be described in detail with reference to the accompanying drawings.

Figs. 1 - 2 are diagrams illustrating a single display module according to an embodiment. Referring to Figures 1 and 2 a display module 1 according to an embodiment may include a plurality of illuminated pixels 11 , may be implemented as an LED display module including light emitting diodes (LEDs). The LED of an LED display module according to an embodiment may be implemented as a micro-LED, representing one pixel 11 on the display module 1. However, an LED display module as described above is just an example, and a display module may also be implemented as an organic LED (OLED), an active-matrix OLED (AMOLED), a plasma display panel (PDP), and the like. Hereinafter, for the convenience of explanation, a display module according to an embodiment will be explained based on the assumption that it is an LED display module. In a preferred embodiment the number of pixels 11 horizontally equals the number of pixels 11 vertically and the front look of the display module 1 is square. Each side of the display module 1 can be aligned and mechanically attached to any side of a similar neighboring display module without consideration of the orientation of any of the attached display modules 1. Every pixel 11 is associated with portion of the video memory in the display module. By varying the content of the video memory, the pixels emit accordingly varying wave length light.

Fig. 3 illustrates an exemplary configuration of a modular display apparatus, comprising six display modules 1 , all connected by serial connection 8 to a raster image processor

7.

After the configuration is assembled and connected, the raster image processor 7 initiates the set-up procedure. The raster Image Processor 7 transmits over the serial connection 8 its unique address number followed by universal commands to the plurality of display modules 1 to visualize their set-up graphic. An exemplary set-up graphic comprises a right-angle triangle 2 and a nine-digit number 3. The right-angle triangles 2 indicate the orientation in space of each display module of the plurality of six display modules connected in the example configuration. The nine-digit numbers 3 comprises the address number of the raster image processor 7 received over the serial connection

8, concatenated with the address number of the display module which displays the particular set-up graphic.

In the exemplary configuration depicted in Fig. 3, the address number of the raster image processor 7 is 310. The remaining six digits of the nine-digit numbers 3 contain the individual address numbers of all display modules connected. In the exemplary configuration of Fig. 3, the address numbers of the connected display modules are:

010331 , 010483, 003063, 002172, 024191 , and 024332. The set-up graphics of the plurality of display modules 1 are photographed by a camera 9 and the control image from the camera 9 is sent to the raster image processor 7 for analysis. Fig. 4 visualizes an exemplary method to analyze the control image received from the camera 9. All pixels of the control image from the camera 9 are scanned horizontally from top to bottom. The line 6 indicates a horizontal line of pixels being scanned at a certain moment. Pixels of the control image from camera 9 are scanned in lines from left to right and top to bottom in two programming cycles - one for horizontal scan and one for vertical scan. The arrow 5 indicates the direction where the next horizontal lines of scanning will appear. The right-angle triangles 2a, 2b and the nine-digit numbers 3 are displayed in dark color on the plurality of display modules 1. Therefore, on the control image from camera 9 the right-angle triangles 2a, 2b will appear as a set of triangle shaped dark pixels.

When the line of scanning 6 crosses a triangle 2a, 2b, a section of dark pixels 4a, 4b is registered. In case the figure is a triangle the section of dark pixels 4a, 4b will increase or decrease in number of dark pixels registered with every new line of scan, but this increase or decrease will be linear. The increase or decrease of number of dark pixels 4a, 4b in the scanned line will be roughly the same with every new line of scan 6. This rule will not apply to the dark pixels of the nine-digit numbers 3.

In the left side of Fig. 4, the triangle 2a will register sections of dark pixels 4a, increasing linearly to the left until the line scan reaches the right-angle tip. With every line of scan the dark pixels will start earlier and will end roughly at the same number of pixels scanned. After that the section of dark pixels 4a will decrease linearly from the left. If such rule applies, the related display module 1 is registered as +90° rotated. The tips of the triangle are stored in a memory.

Similarly, in the right side of Fig. 4, the triangle 2b will register sections of dark pixels 4b, increasing linearly to the left and to the right until the line scan reaches the right-angle triangle hypotenuse. If such rule applies, the related display module 1 is registered as 0° rotated. Similar rules apply when display modules rotated at -90° and 180° are registered (not shown). The nine-digit numbers 3 are visualized in exemplary set-up graphics centered below the hypotenuse of the triangles. After determining orientation of every display module 1 of the plurality of display modules, the raster image processor 7 locates the pixels where every nine-digit number 3 is located on the control image from the camera 9. The images of the nine-digit numbers 3 are cropped from the control image of the camera 9 and fed to OCR (optical character recognition) software which recognizes every digit and all nine-digit numbers 3 as a sequence of decimal digits. The data is recorded in a set-up table as described with respect to Fig. 6 with the corresponding orientation of every display module 1 , its nine-digit number and position on the control image received from the camera 9.

Fig. 5 is a flowchart of the algorithm to analyze the control image received from the camera 9. As result, the set-up table as described in Fig. 6, describing the configuration of the plurality of display modules is filled in. The algorithm uses data extracted from the scan of the control image from the camera 9 and creates a simple (X, Y) coordinate system, where every display module 1 is assigned coordinates, such that the leftmost display modules have their X coordinate as 1 , and the lowermost display modules have their Y coordinate as 1. The maximum value of X is determined as the number of steps from the leftmost display module to the rightmost display module. Similarly, the maximum value for Y is determined as the number of steps from the lowermost display module to the uppermost display module.

Fig. 6 is an exemplary set-up table, derived from the exemplary configuration of display modules 1 shown in Fig. 3 after processing the control image from the camera 9 with the algorithm shown in the flowchart of Fig. 5

Fig. 7 is an example of an input image of six letters to be displayed on the plurality of display modules 1. The input image is received by the raster image processor 7 from an external source. After the input image is received by the raster image processor 7, it is scaled to fit into the total image area of the modular display apparatus and then divided according to the data in the set-up table, so that a separate portion of the image is prepared for every display module 1 in the modular display apparatus. This process involves properly rotating every portion of the image and breaking it down into bits of data to control every pixel on all display modules 1 in the modular display apparatus.

Figures 8a to 8f show how the desired input image from Fig. 7 is divided into 6 parts, one part for every display module 1 of the configuration as shown on Fig. 3. Each of those six parts is rotated according to the record in the set-up table from Fig. 6. Fig. 8a is the image to be sent to the display module with the coordinates (X=1 , Y=1). The display module has address 010331 and is rotated at 180° as recorded in the set-up table (Fig. 6). In other words, it is mounted upside down. Therefore, the image on Fig. 8a is rotated at 180°, to compensate for the way the display module had been rotated when mounted. Then it is sent to address 010331. The display module with coordinates (X=1 , Y=1 ) has exactly address 010331 and it acquires the image from Fig. 8a and visualizes it. As the orientation of the display module is compensated beforehand by the raster image processor 7, the image is visualized correctly as the letter O’.

Similarly, the image from Fig. 8b is rotated at 180° and sent to address 010483; the image from Fig. 8c is sent to address 003063 without rotation; the image from Fig. 8d is sent to address 002172 without rotation; the image from Fig. 8e is rotated at -90° and sent to address 024191 ; the image from Fig. 8f is rotated at +90° and sent to address 024191. As a result, the desired image will appear correctly on the modular display apparatus, regardless of the fact that individual display modules 1 are mounted in arbitrary fashion.

Fig. 9 is a diagram illustrating an example of a configuration of display modules where there is a missing display module 10 in the middle. Such configuration may be used in combination with a real object to be placed right in the middle of the modular display apparatus. Advertising is an exemplary application. Fig. 10 is the set-up table, generated after running the algorithm from Fig. 5 on the control image of the configuration on Fig. 9. The set-up table describes the configuration of display modules with one missing module in the middle, as shown on Fig. 9. The set - up table has a specific record, stating that the spot with coordinates (2,2) is empty.

The disclosed embodiment is not limited to the aforementioned specific to the embodiment, and it is apparent that various modifications can be made by those having ordinary skill in the art to which the disclosure belongs, without departing from the spirit and gist of the disclosure as claimed by the appended claims. Also, it is intended that such modifications are not to be interpreted independently from the technical idea or prospect of the disclosure.