METHOD FOR PROVIDING ELECTRONIC NOTE SERVICE BY USING DIGITAL PAPER AND PEN, SYSTEM AND COMPUTER-READABLE RECORDING MEDIUM WITH PROGRAM THEREFOR

[Technical Field]

The present invention relates to a method, a system, and a computer-readable recording medium recording a program for providing an electronic note service by using a digital pen and digital paper. More particularly, the present invention relates to a method, a system, and a computer- readable recording medium recording a program for providing an electronic note service by using digital paper, which consists of a number of pages having unit cell patterns, and a digital pen adapted to recognize the unit cell patterns on respective pages, wherein codes within a unit cell pattern are received to extract a coordinate value from the corresponding data, the coordinate value is converted into page-wise writing trajectory data, video/audio data is received from a separate input device and is combined with the page-wise writing trajectory data to create electronic note contents, and the created electronic note contents are played/edited by using the digital paper and the digital pen.

[Background Art] As generally known in the art, recent development of

technology in the information and communication fields (e.g. Internet, digital cameras, portable telephones) has enabled not only experts, but also general people to create various pieces of information that is faster and more meaningful than conventional media. Such a trend has caused the global spread of UCC (User Created Content) , which is also called UGC (User Generated Content), particularly in the U.S. The UCC refers to any non-commercial contents created by users and made public on line. For example, TIME, a weekly American newsmagazine, has selected "You" as the Person of the Year in December 2006 on the grounds of the increasing importance of anonymous contributors to blogs and online media. This selection has accelerated the global spread of UCC. The UCC is evolving from the entertainment UCC stage, which mainly provides simple writings and photographs, to the information UCC stage, which focuses on providing moving pictures .

Well-known UCC portals include YouTube, U.S., which provides more contents than conventional broadcasting stations. UCC portals based in Korea include Pandora TV, which records 1,600,000 visitors a month, Gom TV, Afreeca, Aura, Mgoon, Mncast, and Freechal Q.

Although users are creating more contents according to such a trend, conventional contents cannot reflect the

personality of respective users, because they are created by editing images with predetermined tools and inserting captions.

In other words, no technology has yet been developed which enables users to combine characters or pictures written or drawn by hand with images or voices to create contents reflecting the personality of respective users.

Among conventional technologies, digitizer technology can reproduce characters or pictures, which have been written or drawn by users with hand, on the monitor. As used herein, a digitizer refers to a device for detecting coordinates within a predetermined range of the screen and inputting pictures, characters, figure information, etc. as digital data. That is, digitizers convert analog data into digital data, and are used to read the coordinate of original input so that design drawings or figures are inputted into computers .

For example, a digitizer adapted to input the position of X and Y coordinates includes an electronic device having a large, flat square shape to extract the positional coordinate value with regard to the contact surface, and a cursor device equipped with a pen or button to be used on the electronic device. If the user moves the pen or cursor, the lower plate reads the coordinate information and

automatically transmits it to the screen memory place of the computer system. If the user presses the pen or button in a specific position, the corresponding instruction is executed. Such a digitizer is also used to decompose photographic images or the line of printed characters into electronic bits, memorize/store the bits in the computer, and retrieve them to reproduce the original image. Users of digitizers can also input figure data into computers and modify figures on the graphic display screen. However, conventional digitizers can just create or modify images, but cannot create contents combined with images .

[Disclosure] [Technical Solution]

Therefore, the present invention has been made in view of the above-mentioned problems, and the present invention provides a method, a system, and a computer-readable recording medium recording a program for providing an electronic note service by using digital paper, which consists of a number of pages having unit cell patterns, and a digital pen adapted to recognize the unit cell patterns on respective pages, wherein codes within a unit cell pattern are received to extract a coordinate value from the corresponding data, the coordinate value is converted into

page-wise writing trajectory data, video/audio data is received from a separate input device and is combined with the page-wise writing trajectory data to create electronic note contents, and the created electronic note contents are played/edited by using the digital paper and the digital pen.

In accordance with an aspect of the present invention, there is provided a system for providing an electronic note service by using a digital pen and digital paper, the system including digital paper having a unit cell pattern; a digital pen for reading a code of data within the unit cell pattern on the digital paper, the code being expressed based on a numeral system having a radix of 2 or larger; a video/audio input device for receiving an input of video information or audio information and creating video/audio data; and a user terminal equipped with an electronic note program to receive the at least one code, extract a coordinate value from corresponding data, convert the coordinate value into page-wise writing trajectory data, and combine the page-wise writing trajectory data with the video/audio data to create electronic note contents.

In accordance with another aspect of the present invention, there is provided a method for providing an electronic note service by a user terminal by using digital paper and a digital pen, the method including the steps of (a) receiving codes of data within a unit cell pattern

printed on the digital paper from the digital pen by using an installed electronic note program, the codes being expressed based on a numeral system having a radix of 2 or larger; (b) extracting a coordinate value from corresponding data after receiving the at least one code, and converting the coordinate value into page-wise writing trajectory data;

(c) receiving video/audio data from a video/audio input device; and (d) combining the video/audio data with the page-wise writing trajectory data to create electronic note contents.

In accordance with another aspect of the present invention, there is provided a computer-readable recording medium recording a program for providing an electronic note service by using a digital pen and digital paper, the program including a communication module for interworking with the digital pen and a video/audio input device; an information storage module for storing data necessary to drive the electronic note program; and a control module for controlling overall operation of the electronic note program, controlling the communication module to receive at least one code of data within a unit cell pattern printed on the digital paper from the digital pen, the code being expressed based on a numeral system having a radix of 2 or larger, extracting a coordinate value from corresponding data, converting the coordinate value into page-wise writing

trajectory data, receiving video/audio data from the video/audio input device, and combining the page-wise writing trajectory data with the video/audio data to create electronic note contents.

[Description of Drawings]

The foregoing and other objects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

FIG. 1 is a block diagram briefly showing a system for providing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention; FIG. 2 is a block diagram briefly showing a program for providing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention;

FIG. 3 is a flowchart showing a method for providing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention;

FIGs. 4 and 5 show exemplary interfaces of an electronic note program using a digital pen and digital paper according to an exemplary embodiment of the present

invention;

FIG. 6 shows an exemplary structure of electronic note contents according to an exemplary embodiment of the present invention; FIG. 7 shows a method for implementing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention;

FIG. 8 shows exemplary digital paper having unit cell patterns on its surface according to the present invention;

FIGs. 9-12 illustrate an exemplary method for assigning X coordinates of unit cell patterns according to the present invention;

FIGs. 13-16 illustrate an exemplary method for assigning Y coordinates of unit cell patterns according to the present invention;

FIG. 17 shows another exemplary digital paper having unit cell patterns on the surface created by combining the patterns shown in FIGs. 11 and 15 according to the present invention;

FIG. 18 shows exemplary construction of directional flags having no directive feature;

FIG. 19 shows the number of cases of direction flag cell distribution that can appear on a coordinator window; FIG. 20 shows exemplary encoding of data displayed at

first cells according to the present invention;

FIG. 21 shows X and Y coordinate values corresponding to meaning values of respective line segments shown in FIG. 20; FIG. 22 shows exemplary codes of data displayed at second cells according to the present invention;

FIG. 23 shows other exemplary codes of data displayed at second cells according to the present invention; and

FIG. 24 shows still other exemplary codes of data displayed at second cells according to the present invention.

[Mode for Invention]

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. In the following description and drawings, the same reference numerals are used to designate the same or similar components, and so repetition of the description on the same or similar components will be omitted. Furthermore, a detailed description of known functions and configurations incorporated herein is omitted to avoid making the subject matter of the present invention unclear.

FIG. 1 is a block diagram briefly showing a system for providing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of

the present invention.

The system for providing an electronic note service according to an exemplary embodiment of the present invention includes digital paper 110, a digital pen 120, a user terminal 130, a video/audio input device 140, and an electronic note program 150.

The digital paper 110 according to an exemplary embodiment of the present invention has data codes, which are based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on each page. The unit cell pattern 710 printed on the digital paper 100 will be described later in more detail with reference to FIGs. 8- 24.

Although it has been assumed that the digital paper 110 consists of more than one different pages, each of which has data codes (based on a numeral system having a radix of 2 or larger) within a unit cell pattern 710 printed thereon, this assumption is only an example for illustrating the technical idea of the present invention. In practice, various changes and modifications of the digital paper 110 are possible, including notebooks, diaries, housekeeping books, daily records, etc.

The digital pen 120 according to an exemplary embodiment of the present invention refers to a recognition means/device for recognizing data codes, which are expressed

based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on the digital paper 110 and transmitting the codes to the user terminal 130.

The user terminal 130 refers to a terminal adapted to receive various types of web page data via wired/wireless communication networks according to the user's key operation. The user terminal 130 may be a PC (Personal Computer) , a PDA (Personal Digital Assistant) , or a mobile communication terminal. The user terminal 130 includes a communication means 132, a central processing unit 134, a display means 136, and a storage means 138.

The communication means 132 of the user terminal 130 interworks with the digital pen 120 and the video/audio input device 140 to transmit/receive various types of data. The central processing unit 134 processes the overall operation of the user terminal 130, and drives the electronic note program 150 according to an exemplary embodiment of the present invention.

The display means 136 refers to a screen display means for receiving various types of data from the central processing unit 134 and outputting the data as characters, numbers, images, etc. The display means 136 displays various messages created while various programs stored in the storage means 138 are executed under the control of the central processing unit 134, and displays messages created

while the electronic note program 150 is being executed according to an embodiment of the present invention.

The storage means 138 refers to a device for storing an operating program for driving the user terminal 130 and various application programs.

The video/audio input device 140 refers to a device for receiving an input of video or audio information, and is positioned either inside or outside the user terminal 130. For example, the device for receiving video data may consist of a PC cam and/or a camera attached to the mobile communication terminal, and the device for receiving audio data may consist of a microphone attached to the PC or mobile communication terminal.

The electronic note program 150 according to an exemplary embodiment of the present invention is operated as follows: if recording is requested under the control of the central processing unit 134, the electronic note program 150 receives data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 from the digital pen 120. Then, the electronic note program 150 decodes the codes displayed at the first cells 701 within the unit cell pattern 710 into corresponding data, and combines the data to extract a coordinate value. The electronic note program 150 coverts the coordinate value into page-wise writing trajectory data

and outputs it. At a request for additional video/audio recording, the electronic note program 150 receives video/audio data from the video/audio input device 140. If requested to end the recording, the electronic note program 150 synchronizes the page-wise writing trajectory data and the video/audio data on the time axis in the order of input to create electronic note contents.

Particularly, the electronic note program 150 extracts a coordinate value from data corresponding to at least one code, which has been received from the digital pen 120, and converts the extracted coordinate value into digital-type page-wise writing trajectory data. The electronic note program 150 reads the page-wise writing trajectory data to output specific characters, images, figures, etc. The electronic note program 150 according to an exemplary embodiment of the present invention includes an electronic note contents editing module 152 for editing electronic note contents.

The electronic note contents editing module 152 is adapted to continuously list line objects, video objects (individual screen frames) , audio objects, image objects, and text objects in the order of storage time. The electronic note contents editing module 152 enables the user to edit (e.g. select, copy, paste, delete, merge, repeat) objects belonging to a specific time period as a batch.

Alternatively, respective objects are selectively deleted, copied, or pasted.

FIG. 2 is a block diagram briefly showing a program for providing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention.

The electronic note program 150 according to an exemplary embodiment of the present invention includes an information storage module 210, a control module 220, and a communication module 230.

The information storage module 210 is adapted to store various types of data necessary to drive the electronic note program 150.

The control module 220 controls the overall operation of the electronic note program 150 in the following manner: if recording is requested, the control module 220 receives data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 regarding a specific location on the digital paper 10 from the digital pen 120 by means of the communication module 230. Then, the control module 220 decodes the codes into corresponding data and combines the data to extract a coordinate value. The control module 220 converts the extracted coordinate value into digital-type page-wise writing trajectory data, and outputs the result of reading

the page-wise writing trajectory data. If additional video/audio recording is requested, the control module 220 receives video/audio data from the video/audio input device 140. If requested to end the recording, the control module 220 synchronizes the page-wise writing trajectory data and the video/audio data on the time axis in the order of input to create electronic note contents.

The communication module 230 is adapted to interwork and communicate with the digital pen 120 and the video/audio input device 140.

FIG. 3 is a flowchart showing a method for providing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention. The electronic note program 150 of the user terminal 130 checks if there is a request for recording (S310).

If it has been confirmed in step S310 that there is a request for recording, the electronic note program 150 of the user terminal 130 receives data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on the digital paper 110 from the digital pen 120. Then, the electronic note program 150 decodes the codes displayed at the first cells 701 within the unit cell pattern 710 into corresponding data and combines the data to extract a

coordinate value. The electronic note program 150 converts the coordinate value into digital-type page-wise writing trajectory data, and outputs the result of reading the page- wise writing trajectory data (S320) . Particularly, if the user writes character 'A' on the digital paper 110 with the digital pen 120, the digital pen 120 recognizes data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on the digital paper 110, and transmits the codes to the user terminal 130. Then, the electronic note program of the user terminal 130 decodes the codes displayed at the first cells 701 within the unit cell pattern 710 into corresponding data and combines the data to extract a coordinate value. The electronic note program converts the coordinate value into digital-type page-wise writing trajectory data, and outputs the results of reading the page-wise writing trajectory data, i.e. character 'A', on the display means 136.

The electronic note program 150 of the user terminal 130 checks if there is a request for video/audio recording (S330) .

If it has been confirmed in step S330 that there is no request for video/audio recording, the electronic note program 150 of the user terminal 130 creates contents only from the result of reading the page-wise writing trajectory

data (S332) .

In other words, the electronic note program 150 creates contents regardless of the order of time of input of the page-wise writing trajectory data. If it has been confirmed in step S330 that there is a request for video/audio recording, the electronic note program 150 of the user terminal 130 receives video/audio data from the video/audio input device 140 (S340) .

Particularly, if the user writes 'A' on the digital paper 110 with the digital pen 120, selects additional video/audio recording, and pronounces 'A ¹ , the video/audio input device 140 creates audio data corresponding to 'A' and transmits it to the electronic note program 150 of the user terminal 130. The electronic note program 150 of the user terminal

130 checks if there is a request for termination of the video/audio data (S350) .

If it has been confirmed in step S350 that there is a request for termination of the video/audio data, the electronic note program 150 of the user terminal 130 finishes receiving the video/audio data (S360) .

The electronic note program 150 of the user terminal

130 checks if there is a request for additional video/audio recording (S370) . If it has been confirmed in step S370 that there is a

request for additional video/audio recording, the electronic note program 150 of the user terminal 130 repeats steps S340 to S360.

If it has been confirmed in step S370 that there is no request for additional video/audio recording, the electronic note program 150 of the user terminal 130 synchronizes the page-wise writing trajectory data and the video/audio data on the time axis in the order of input to create electronic note contents (S380) . Particularly, if the user writes 'A' on the digital paper 110 with the digital pen 120, pronounces ¹ A', and selects termination, the electronic note program 150 of the user terminal 130 outputs the image 'A' and combines it with audio data corresponding to 'A' to create electronic note contents.

FIGs. 4 and 5 show exemplary interfaces of an electronic note program using a digital pen and digital paper according to an exemplary embodiment of the present invention. Referring to FIG. 4, the interface of the electronic note program 150 may include an electronic note toolbar 410, a track bar 420, a display image 430, an output 440 of page- wise writing trajectory data, an audio icon 450, and a multi-page distinction screen 460. Referring to FIG, 5, the electronic note toolbar 410

may include buttons for starting, pausing, and stopping recording, a button for turning on/off video and audio recording, a button for turning on/off audio recording, and buttons for starting, pausing, and stopping the playback of electronic note contents.

The track bar 420 may be adapted to display the proceeding status during recording and playback, as shown in FIG. 5.

The display image 430 is outputted on the electronic note program 150 as shown in FIG. 4 after the electronic note program 150 receives video data or audio data from the video/audio input device 140. The display image 430 shown in FIG. 4 is based on an assumption that video data has been received from the video/audio input device 140. The display image 430 can move in the region like a normal window. It is to be noted that, if audio data has been received from the video/audio input device 140, no display image 430 is separately displayed as shown in FIG. 4.

The output 440 of page-wise writing trajectory data is obtained in the following manner: the electronic note program 150 receives data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on the digital paper 110 from the digital pen 120. Then, the electronic note program 150 decodes the codes displayed at the first cells 701 within

the unit cell pattern 710 into corresponding data and combines the data to extract a coordinate value, which is converted into digital-type page-wise writing trajectory data. An exemplary result of reading the page-wise writing trajectory is displayed as the output 440.

Particularly, if the user writes 'HELLO ¹ on the digital paper 110 with the digital pen 120, the digital pen 120 recognizes data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on the digital paper 110 and transmits them to the user terminal 130. The electronic note program of the user terminal 130 decodes the codes displayed at the first cells 701 within the unit cell pattern 710 into corresponding data and combines the data to extract a coordinate value, which is converted into digital-type page- wise writing trajectory data. The result of reading the page-wise writing trajectory data is outputted as 'HELLO' on the display means 136 as shown in FIG. 4.

The audio icon 450 is adapted to inform that audio data is inserted.

The multi-page distinction screen 460 is adapted to distinguish between respective pages when the digital paper 110 consists of more than one different pages.

For example, if the user writes 'HELLO' on the first page of the digital paper 110 with the digital pen 120, the

digital pen 120 recognizes data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 printed on the first page of the digital paper 110 and transmits them to the user terminal 130. The electronic note program of the user terminal 130 decodes the codes displayed at the first cells 701 within the unit cell pattern 710 into corresponding data and combines the data to extract a coordinate value. The extracted coordinate value is converted into digital-type page-wise writing trajectory data, and the result of reading the page-wise writing trajectory data is outputted as 'HELLO' on the display means 136 as shown in FIG. 4. If the user inputs characters or images on the second or third page of the digital paper 110, they are outputted in the same manner, and the user can select each page from the multi- page distinction screen 460.

As such, the user can select a specific page of the digital paper 110 and input characters or images as desired. Then, the user can select the characters or images, which have been inputted into respective pages, from the multi- page distinction screen 460.

FIG. 6 shows an exemplary structure of electronic note contents according to an exemplary embodiment of the present invention. Referring to FIG. 6, the electronic note content

structure is divided into a header region 510, a page-wise object region 520, a video region 530, and an audio region 540.

The header region 510 contains basic information regarding the version and size of files, and an exemplary embodiment of the header region 510 is given in the following Table 1.

Header Region H σ

M

AVI(N) Name + Size WAVE(I) Name + Size WAVE(N) Name + Size (D H ¹

UPO Info AVI Info WAVE Info

—I

128kbyt e(XML)

_r [XML Structure ]

<?xml version=" 1.0' encoding="UTF-8' ?> [ Reference ]

<f> f : file

<iv="U i : info

<tk="l" n="" S="" /> v : version

<tk="2" n="" S=" '/> t:type

<tk="3" n="" S=" '/> k: kind

</f> 'T'-upO "2" - avi "3" - wav n : name s : size

The page-wise object region 520 contains lines, texts, images, and video information for each page. As used herein, the video information refers to information regarding the starting position of the image viewer and the name of video files. An exemplary embodiment of the page-wise object information 520 is given the following Table 2.

Page- Wise Object Region O ⁾ tr

CD f File Version ^λ Creator ID ^λ Total Page NumberTage At Time Of Storage^bject Time Lιst ^λ Total Recording Time [Reference] pO ^λ Page Name ^λ Skin File Path ^λ Lme Object Number f file K) pOIO ^λ Object Key ^λ Lme Creator ID ^λ Lme MoαV^Line Thickness ^λ Ijne Color^oordmate NumberXSiordinate List ^λ PIayback Starting Time ^λ Playback Ending Time P page pOtl ^λ Object Key ^λ Line Creator lD ^λ Line Mode ^λ Text To InputTont ^" NameTont Size ^λ Font StyleTont ColorText Region(Coordmate List) 1 line

^λ Tcxt Rcgion(Rcct Value) ^λ Playback Starting Timc ^λ Playback Ending Time t text p0i2 ^λ Object Key ^λ Line Creator lD ^λ Line Mode ^λ Tmage Region ^λ Playback Starting Time ^λ Playback Ending Time — > File Content(Bmary Info) image pOaS'Object Key ^λ Lme Creator ID ^λ Lme Mode ^λ Moving Picture File Name ^λ Movmg Picture Region(Final Position Info) ^λ Playback Starting Time ^λ Playback Ending Time m movie p0w4*Object Key ^λ Lme Creator ID ^λ Line Mode ^λ Audio File Name ^λ Audio Icon Region(Final Position Info) ^λ Playback Starting Time ^λ Playback Ending Time voice

Cn

Page- Wise Object Region

CU σ

I — ' f ^λ File Version ^λ Creator ID ^λ Total Page Mmnber ^λ Page At Time Of Storage ^λ Object Time List ^λ Total Recording Time φ H- >

O ¹ T ¹ file ω pO ^λ Page Name ^λ Skin File Path ^λ Line Object Number

0 "p" Means Page Info I Page Index Starts From 0 p0I0 ^λ θbject Key ^λ Llπe Creator ID ^λ Line Mode ^λ Line Thickness ^λ Line Color ^λ Coordinate Number ^λ Coordinate LisfPlayback Starting Time ^λ PIayback Ending Time

0 "p" Means Page Info

1 Page Index Starts From 0 α

2 "1" Means Line Info φ

3 Object Index Starts From 0 Cλ n

4 Object Key(Lme Key) yyyyMMddHHmmssff(16) + Random(4 Among 0-999, If Less Than Three Digits, Remainmgs Are Filled With 0)

5 Line Creator ID(UID) Used In Web Mainly Used By Smart Client H-

6 Line Mode(PEN = 0, PENCIL = 1 , BRUSH = 2, TEXT = 3, Fluorescent(Fluorescem Pen) = 4) T3 rt

7 Line Thickness H-

8 Line Color Based On RGB Value, 255,255,255 O

K) 9 Coordinate Number

10 Coordinate List- OIDlOO.lOO.Input Value.ActionTime.DrawTime^OO^OOJnputValu^ActionTime.DrawTime ,

O

Hi pOt1 ^λ Object Key ^λ Line Creator ID ^λ Line Mode ^λ Teτt To Input ^λ Font Name ^λ Font Size ^A Font Style ^λ Font Color ^λ Text Region(Coordinate List) ^λ Text Region(Rect Value) ^λ Playback Starting Time ^λ Playback Ending Time

0 "p ^«

1 Page Index σ

2 "t" φ

3 Object Index

4 Object Ke> M

5 Line Creator ID

6 Line Mode(PEN = 0, PENCIL = 1, BRUSH = 2, TEXT = 3, Fluorescent(Fluoresceat Pen) = 4) H-

7 Text To Input W

8 Font Name

9 Font Size £!

10 Font Style H-

11 Font Color < φ

12Text Region Coordinate Value "100,100,200,100,200,200,100,200" 13 Text Region rect Value-"x,y,width,height"

H- p0i2 ^λ θbject Key ^λ Line Creator ID ^λ Line Mode ^λ Image Region ^λ Playback Starting Time ^λ Playback Ending Time — > File Content(Binary Info) 3 p0a3 ^λ θbject Key ^λ Line Creator ID ^λ Line Mode ^λ Movmg Picture File Name ^λ Movmg Picture Region(Final Position Info) ^λ Playback Starting Time ^λ Playback Ending Time p0w4 ^λ θbject Key ^λ Line Creator lD ^λ Line Mode ^λ Audio File Name ^λ Audio Icon Region(Fmal Position Info) ^λ Playback Starting Time ^λ Play back Ending Time r+ φ

Particularly, the page-wise object region shown in Tables 2 and 3 includes coordinate vales obtained by receiving data codes, which are expressed based on a numeral system having a radix of 2 or larger, within a unit cell pattern 710 from the digital pen 120, decoding the codes displayed at the first cells 701 within the unit cell pattern 710 into corresponding data, and combining the data to extract the coordinate values; line objects consisting of color, thickness, etc; text objects inputted via the keyboard; image objects such as photographs or pictures; and video and audio objects inputted from the video/audio input device 140.

In order to play electronic note contents one after another, which have been obtained by synchronizing page-wise writing trajectory data and video/audio data on the time axis in the order of input, the page-wise object region may store respective objects together with the starting time and ending time with reference to the initial recording time.

The page-wise background image (skin) of the page-wise object region may be designated by the user by using an image file. The page-wise background image may be used as various business forms, lecture materials, photo files, page-wise images of various publications, etc. When the digital paper 110 is used to create actual forms, teaching materials, photographs, publications, etc., what has been

entered (characters and images) into the actual forms, teaching materials, photographs, publications, etc. with the electronic pen 120 can be outputted or reproduced in the same position on the electronic note contents. In the case of line objects contained in the page-wise object region, not only the starting and ending times of the lines, but also the value of time difference between each input coordinate value included in the corresponding line and the preceding coordinate value is stored so that the electronic note contents can be played more naturally.

The video region 530 contains file information regarding the size, file version, etc. of video data, as well as actual video data. An exemplary embodiment of the video region 530 is given in the following Table 4.

[Table 4]

Video Region

AVI File Info (1) AVI File Info (N)

The audio region 540 contains file information regarding the size, file version, etc. of audio data, as well as actual audio data. An exemplary embodiment of the audio region 540 is given in the following Table 5.

[Table 5]

Audio Region

WAVFileInfo(1) WAVFiIeMo(N)

FIG. 7 shows an exemplary a method for implementing an electronic note service by using a digital pen and digital paper according to an exemplary embodiment of the present invention.

The exemplary method for implementing an electronic note service by using a digital pen and digital paper is used to reproduce the output 440 of page-wise writing trajectory data more smoothly when the electronic note program 150 receives codes printed on the digital paper 110 from the electronic pen 120, decodes and combines the codes to extract coordinate values, and when it is determined that the distance between the extracted coordinate values is too large.

Referring to FIG. 7, the electronic note program 150 extracts arbitrary coordinates PO and Pl and, if it is determined that the distance between PO and Pl is at least five pixels, creates P ¹ O between PO and Pl so that the output 440 of page-wise writing trajectory data can be reproduced more smoothly. If the distance between PO and P'O is at least five pixels, P ¹ I is created between PO and P'O. Similarly, if the distance between P'O and Pl is at least

five pixels, P' 2 is created between P ¹ O and Pl.

In other words, when the electronic note program 150 receives codes printed on the digital paper 110 from the electronic pen 120, decodes and combines the codes to extract coordinate values, and when the distance between the extracted coordinate values is at least five pixels, as shown in FIG. 7, the electronic note program 150 creates intermediate coordinates and reproduces the output 440 of page-wise writing trajectory data. This prevents the electronic note program 150 from reproducing the output 440 of page-wise writing trajectory data in an unnatural manner.

Assuming with reference to FIG. 7 that section A spans

PO-P' 1, section B spans Pl-P ¹ O, section C spans PO-P2, and section D spans P'2-Pl, time information regarding sections A, B, C, and D can be created as defined by the following Table 6.

[Table β]

(Tl - TO)

T'l = (A Section)

4X1

(Tl - TO) TO = ^V _4χ2 (B Section)

(Tl - TO) T'2 = . _v , (C Section)

^{T1 =} ^ _X ^ C° ^Section)

Those skilled in the art can understand that FIG. 7 shows only an exemplary method of implementing an electronic note service by using a digital pen and digital paper, and that the method for reproducing the output 440 of page-wise writing trajectory data can be variously modified and adapted to without departing from the scope and spirit of the invention.

FIG. 8 shows an embodiment of digital paper having a unit cell pattern on the surface according to the present invention.

Referring to FIG. 8, a unit cell pattern 710 includes at least a predetermined number of first cells 701 displaying data codes, which are expressed based on a numeral system having a radix of 2 or larger, and at least a predetermined number of second cells 702 displaying data codes of a type distinguished from the first cells 701 or having no data.

As shown in FIG. 8, the first cells 701 display information of data, which is expressed based on a numeral system having a radix of 2 or larger, regarding X and Y coordinates, such as (0,0), (0,1), (1,0), or (1,1).

Although it has been assumed in the description of an embodiment of the present invention that binary data codes are displayed at respective first cells 701, the data is not limited to binary data. For example, the data may be based

on a numeral system having a radix larger than 2 (e.g. 3 or 4) depending on the number of code types.

According to the present embodiment shown in FIG. 8, a unit cell pattern 710 refers to a set of first and second cells 701 and 702 of NxM (4x4) size, which can be read as a whole by the sensing means (i.e. digital pen 210) and recognized separately. The unit cell pattern 710 is also called a window. Such a 4x4 unit cell pattern 710 consists of thirteen first cells 701 and three second cells 702. Although it has been assumed in the description of the present embodiment that N and M, which specify the size of the unit cell pattern 710, are identical, they may differ from each other.

The combination of data corresponding to thirteen first cells 701 indicates the X and Y coordinate values (also referred to as a window value) of the corresponding unit cell pattern 710. The three second cells 702 are placed in a predetermined position (i.e. lower right corner) within the unit cell pattern 710 so that the second cells 702 are joined by line segments in a predetermined shape (i.e. '-* '). Such a set of three second cells 702 placed in predetermined position and shape within the unit cell pattern 710 is referred to as a direction flag 703. The position of the direction flag 703 is determined so that the unit cell pattern 710 is distinguished from other adjacent unit cell

patterns, and the shape is determined to identify the direction of the digital paper 110 (e.g. paper), i.e. the degree of rotation of the digital paper 110, as will be described later in more detail. Although it has been assumed in the description of the present embodiment that the direction flag 703 has the shape of ^lJ ', the shape is not limited to that, and the direction flag 703 may have various shapes (e.g. '-η ', ^I~L| ) as long as it indicates the condition of rotation of the digital paper 110. A method for assigning X and Y coordinate values of a unit cell pattern according to an embodiment of the present invention will now be described in more detail with reference to FIGs. 7-14.

Method for assigning X coordinate in unit cell pattern FIG. 9 shows exemplary unit cell patterns 710 according to the present invention. Although respective first cells 701 contain data regarding both X and Y coordinates, as shown in FIG. 8, FIG. 9 separately shows data regarding the X coordinate for convenience of illustration.

Referring to FIG. 9, data regarding the X coordinate value of of unit cell patterns 710 gradually increases by 1 in the X direction (rightward direction) : OOOOOOOOOOOOO→OOOOOOOOOOOOl→OOOOOOOOOOOlO→OOOOOOOOOOOl l. The data regarding the X coordinate value corresponds to a

combination of data, which corresponds to thirteen first cells 701 within each unit cell pattern 710, in the order from 1 to 13 shown in FIG. 10. Such a gradual increase of the data regarding the X coordinate value of unit cell patterns 710 by one in the X direction (rightward direction) guarantees that, even if the coordinate window does not accurately coincide with the actual X and Y coordinate values, the actual X and Y coordinate values can be restored based on the regularity. As used herein, the coordinate window refers to a set of NxM cells actually read by the coordinate sensing means (i.e. digital pen 210). It is to be noted that, during actual application, images observed by the coordinate sensing means (digital pen 210) may be larger than the coordinate window. The unit cell patterns 710 in a row shown in FIG. 9 are repeatedly arranged in the Y direction (vertical direction) so that they span over a plurality of rows, as shown in FIG. 11.

FIG. 12 shows an embodiment alternative to that shown in FIG. 9. Referring to FIG. 10, the window value (X and Y coordinate values) does not necessarily begin from 0, but an arbitrary value can be the starting value. The digital pen 120 for reading the window value is informed of this fact. When X and Y coordinate values relative to the starting point are to be obtained, the arbitrary value is subtracted

from the currently read X and Y coordinate values. If the maximum value that can be displayed by all X and Y coordinate values is reached, the next X and Y coordinate values can be set to be 0. Method for assigning Y coordinate in unit cell pattern

FIG. 13 shows exemplary unit cell patterns 710 according to the present invention. Although respective cells contain data regarding both X and Y coordinate values, as shown in FIG. 8, FIG. 13 separately shows data regarding the Y coordinate value for convenience of illustration.

Referring to FIG. 13, data regarding the Y coordinate value of unit cell patterns 710 gradually increases by 1 in the Y direction (downward direction) : OOOOOOOOOOOOO→OOOOOOOOOOOOl→OOOOOOOOOOOlO→OOOOOOOOOOOl l. The data regarding the Y coordinate value corresponds to a combination of data, which corresponds to thirteen first cells 701 within each unit cell pattern 710, in the order from 1 to 13 shown in FIG. 14. Such a gradual increase of the data regarding the Y coordinate value the unit cell patterns 710 by one in the Y direction (downward direction) guarantees that, even if the coordinate window does not accurately coincide with the actual X and Y coordinate values, the actual X and Y coordinate values can be restored based on the regularity. Although respective first cells 701 are assigned X

coordinate values in the order shown in FIG. 10 and Y coordinate values in the order shown in FIG. 14, it is also possible to commonly apply the order shown in FIG. 10 or 14 to assign both X and Y coordinate values. Alternatively, the order shown in FIG. 10 is applied with regard to Y coordinate values, and the order shown in FIG. 14 is applied with regard to X coordinate values. Although not shown in the drawings, the positional number of data of respective first cells 701 may increase along a spiral curve starting from the center of the same unit cell pattern 710.

The unit cell patterns 710 in a column shown in FIG. 13 are repeatedly arranged in the X direction (horizontal direction) so that they span over a plurality of columns, as shown in FIG. 15. FIG. 16 shows an embodiment alternative to that shown in FIG. 13. Referring to FIG. 16, the X and Y coordinate values do not necessarily begin from 0, but an arbitrary value can be the starting value. The digital pen 120 for reading the X and Y coordinate values is informed of this fact. When the X and Y coordinate values relative to the starting point is to be obtained, the arbitrary value is subtracted from the currently read X and Y coordinate values. If the maximum value that can be displayed by all X and Y coordinate values is reached, the next X and Y coordinate values can be set to be 0.

Respective first cells 701 constituting a unit cell pattern 710 according to the present invention are assigned X coordinate values in the manner shown in FIG. 11, as well as Y coordinate values in the manner shown in FIG. 15. Then, the X and Y coordinate values (shown in FIGs. 11 and 15, respectively) in the same cell position are combined to obtain unit cell patterns as shown in FIG. 17.

A comparison between the unit cell patterns shown in FIGs. 8 and 17 reveals that the data values assigned to corresponding cells slightly differ. Such a difference results from the varying methods for assigning/combining data corresponding to second cells 402 within the unit cell pattern 710, as has been described with reference to FIGs. 10 and 14. Particularly, the unit cell pattern shown in FIG. 8 corresponds to an example of applying the order of assigning/combining data shown in FIG. 14 to both X and Y coordinate values. The unit cell pattern shown in FIG. 17 corresponds to an example of applying the order of assigning/combining data shown in FIG. 10 to X coordinate values and applying the order of assigning/combining data shown in FIG. 14 to Y coordinate values.

The direction flag 703 will be described in more detail with reference to FIG. 8.

Second cells 702 constituting a direction flag 703 must be arranged so that the direction flag itself has a

directive feature. This requires at least three second cells 702. If a smaller number of second cells 702 constitute a direction flag, it has at least two directive features no matter how the second cells 702 are arranged. However, if the direction flag 703 is not to be used for rotation sensing, but just for error correction, two or more cells may constitute the direction flag 703. Furthermore, if the direction flag 703 is simply used to distinguish between adjacent unit cell patterns, the direction flag 703 may consist of only one second cell 702.

When the direction flag 703 is not used for error correction, but solely for rotation sensing, the second cells 702 contain no information. Alternatively, the second cells 702 contain information displaying codes in a manner distinguished from the first cells 701 to make it known that they constitute a direction flag 703.

When the direction flag 703 is solely used for rotation sensing, it is enough to use three second cells 702 containing no information, and the three cells are preferably joined by line segments in the shape of '- ¹ ' .

Some arrays of three cells have no directive feature because the three cells are arranged along a straight line, as shown in FIG. 16, or because they have no center point, and are preferably excluded. Assuming that ' ^J '-shaped direction flags are used,

there are three possible types of distribution of second cells, as shown in FIG. 19a, FIG. 19b, and FIG. 19c, which can be considered on a coordinate window to find the angle of rotation (90 ^° , 180° , 270° ) of digital paper having a unit cell pattern printed thereon. Particularly, all cells are positioned next to one another on a coordinate window in the case of FIG. 19a, cells are divided into two groups in the case of FIG. 19b, and cells are divided into three groups in the case of FIG. 19c. In any case of FIG. 19a, FIG. 19b, and FIG. 19c, a cell C acting as the center point is located first, and it is recognized that other cells spaced relative to the cell C are positioned on the opposite side of the center point. The rotated direction flag is restored in this manner. If the restored direction flag has the shape of ' ^Ll , it indicates clockwise 90 ^° rotation. Similarly, ' r ¹ indicates 180° rotation, and 'η ' indicates counterclockwise 90° rotation. For example, the direction flag shown in FIG. 19a has not rotated; restoration of the direction flag shown in FIG. 19b gives the shape of ' ^Ll and indicates clockwise 90 ^° rotation; and restoration of the direction flag shown in FIG. 19c gives the shape of ' r' and indicates 180 ^° rotation. Such characteristics are used to obtain the angle of rotation of the digital paper, and make it possible to rotate a matrix of cells on the coordinate

window accordingly.

The correction of errors of a unit cell pattern 710 according to the present invention will now be described.

Error correction requires that 2-bit binary data be encoded and displayed at second cells 702 of the above- mentioned direction flag for distinguishing the unit cell pattern 710 in a manner different from that of first cells 701. The 2-bit values become the error correcting codes for X and Y coordinate values, respectively. The error correcting code for X coordinate values functions independent of that for Y coordinate values, but in the same manner. Therefore, the error correction code for X coordinate values will solely be described for clarity.

A 4x4 unit cell pattern 710 contains 13-bit information regarding X coordinate values. However, 2-bit error correction requires at least 4 redundant bits. The RS

(Reed-Solomon) code, which is the optimal algorithm for block codes, has the code type of (2 ^k -l, 2 ^k -l-k) . Therefore,

(2 ^k -l-k)-bit data can be reconstructed into a (2 ^k -l)-bit codeword to correct errors of up to 2 bits. In this case, k refers to the code number for error correction. If k=4, 12 of 16 cells are used for data, and the remaining 4 cells are used as error correcting codes. To this end, the number of second cells constituting the direction flag must be increased by 1 (i.e. a total of 4). Then, 11 of 12 cells are

subjected to error correction, and the remaining one cell is subjected to error processing by the upper layer.

A method for correcting bits other than the upper one bit will now be described. Second cells 702 constituting the direction flag are reconstructed so that they amount to 4. Respective second cells 702 are encoded by using a type of representation different from that of first cells 701 so that the second cells 702 can contain 2-bit information. The method for calculating the pattern position is the same as mentioned above even if the second cells 702 amount to 4. The only difference is that the number of first cells 701 within the unit cell pattern 710 is reduced by one. Then, RS coding (one of error correcting codes) is used to calculate (encode) error correcting codes with regard to the entire information (or a part of it) existing in the data cells, and the error correcting codes are encoded and displayed at the direction flag cells. If the unit cell pattern 710 is not read accurately, but obliquely, the error correcting codes cannot function properly. However, errors can be corrected by calculating the position according to the original position calculation method and conducting RS decoding. It is to be noted that the RS decoding is conducted with regard to estimated codewords, not the original RS-encoded codewords, and that the resulting

restoration may not be accurate.

A method for limitedly correcting the errors of lower bits only will now be described.

Among data constituting a unit cell pattern 710, bits of higher positional numbers are less likely to change than bits of lower positional numbers. This means that, if error correction is conducted with regard to bits of lower positional numbers, which undergo frequent change, the number of second cells 702 constituting a direction flag can be reduced. As a result, a larger number of different unit cell patterns 710 can be realized. As mentioned above, the number of cells constituting a direction flag 703 decreases, and the possibility that three second cells 702 may not be seen simultaneously is lower than the possibility that four second cells 702 may not be seen simultaneously. Therefore, there are more chances that the error correcting function will be conducted as desired.

Those skilled in the art can easily understand that, although it has been assumed in the above description that RS coding is used to correct errors of unit cell patterns 710, the method is not limited to that type.

Data codes displayed at the first and second cells 701 and 702 will now be described.

FIG. 20 shows data codes, which are expressed based on a numeral system having a radix of 2 or larger, displayed at

the first cells 701 according to the present invention. Particularly, assuming that virtual X and Y axes intersect with each other at the center point of each cell, FIG. 20a shows a first line segment lying on the X axis with the intersection point at its center, FIG. 20b shows a second line segment lying on the Y axis with the intersection point at its center, FIG. 20c shows a third line segment lying in the first and third quadrants with the intersection point at its center, and FIG. 2Od shows a fourth line segment lying in the second and fourth quadrants with the intersection point at its center. Each data code is displayed by one of the first to fourth line segments.

FIG. 21 shows a table enumerating the X and Y coordinate values corresponding to the meaning value of respective line segments shown in FIG. 20. Particularly, the first line segment shown in FIG. 20a has a meaning value of 0, and the corresponding X and Y coordinate values are (1,1). The second line segment shown in FIG. 20b has a meaning value of 1, and the corresponding X and Y coordinate values are (0,1). The third line segment shown in FIG. 20c has a meaning value of 2, and the corresponding X and Y coordinate values are (1,0). The fourth line segment shown in FIG. 2Od has a meaning value of 3, and the corresponding X and Y coordinate values are (0,0). Those skilled in the art can understand that the matching between the meaning values and

the X and Y coordinate values shown in FIG. 21 is only an example, and can be varied as desired.

FIG. 22 shows data codes, which are expressed based on a numeral system having a radix of 2 or larger, displayed at the second cells according to the present invention. Particularly, assuming that virtual X and Y axes intersect with each other at the center point of each cell, FIG. 22a shows a first line segment lying in the first and second quadrants in parallel with the X axis, FIG. 22b shows a second line segment lying in the third and fourth quadrants in parallel with the X axis, FIG. 22c shows a third line segment lying in the second and third quadrants in parallel with the Y axis, and FIG. 22d shows a fourth line segment lying in the first and fourth quadrants in parallel with the Y axis. Each data code is displayed by one of the first to fourth line segments.

Referring to FIG. 21, the first line segment shown in FIG. 22a has a meaning value of 0, and the corresponding X and Y coordinate values are (1,1). The second line segment shown in FIG. 22b has a meaning value of 1, and the corresponding X and Y coordinate values are (0,1). The third line segment shown in FIG. 22c has a meaning value of 2, and the corresponding X and Y coordinate values are (1,0) . The fourth line segment shown in FIG. 22d has a meaning value of 3, and the corresponding X and Y coordinate values are (0,0).

As another example of data codes, which are expressed based on a numeral system having a radix of 2 or larger, displayed at the second cells according to the present invention, line segments may be displayed in the same manner as the first to fourth line segments of first cells shown in FIGs. 20a to 2Od, except that they have different lengths to be distinguished from the line segments of the first cells.

FIG. 23 shows another example of data codes, which are expressed based on a numeral system having a radix of 2 or larger, displayed at the second cells according to the present invention. Particularly, assuming that virtual X and Y axes intersect with each other at the center point of each cell, FIG. 23a shows a first line segment extending from the intersection point to a point in the first quadrant, FIG. 23b shows a second line segment extending from the intersection point to a point in the second quadrant, FIG. 23c shows a third line segment extending from the intersection point to a point in the third quadrant, and FIG. 23d shows a fourth line segment extending from the intersection point to a point in the fourth quadrant. Each data code is displayed by one of the first to fourth line segments, each of which represents X and Y coordinate values selected from (0,0), (0,1), (1,0), and (1,1).

FIG. 24 shows another example of data codes, which are expressed based on a numeral system having a radix of 2 or

larger, displayed at the second cells according to the present invention. Particularly, assuming that virtual X and Y axes intersect with each other at the center point of each cell, FIG. 24a shows a first line segment lying on the positive X and Y axes with the intersection point at its center, FIG. 24b shows a second line segment lying on the negative X axis and on the positive Y axis with the intersection point at its center, FIG. 24c shows a third line segment lying on the negative X and Y axes with the intersection point at its center, and FIG. 24d shows a fourth line segment lying on the positive X axis and on the negative Y axis with the intersection point at its center. Each data code is displayed by one of the first to fourth line segments, each of which represents X and Y coordinate values selected from (0,0), (0,1), (1,0), and (1,1).

Various types of line segments have been described as means for encoding data, which are expressed based on a numeral system having a radix of 2 or larger, displayed at. first or second cells according to the present invention. Respective line segments, which have been defined as lines of a predetermined length, may also be expressed as a plurality of points constituting the same straight line. In order to prevent points belonging to a cell from constituting a line segment together with points belonging to an adjacent cell, the maximum distance between points

constituting a line segment must always be smaller than the distance between any point belonging to the corresponding cell and other points belonging to an adjacent cell.

Furthermore, although it has been assumed in the above description that binary data is encoded and displayed at respective cells according to the present invention, the data is not limited to binary data. For example, the data may be based on a numeral system having a radix larger than 2 (e.g. 3 or 4) depending on the number of code types. Although several exemplary embodiments of the present invention have been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

[industrial Applicability]

As mentioned above, according to the present invention, digital paper having a unit cell pattern printed thereon and a digital pen for recognizing the unit cell pattern on the digital paper are used to receive codes within a unit cell pattern, extract coordinate values from the corresponding data, convert the coordinate values into page-wise writing trajectory data, receive video/audio data from a separate input device, and combine the page-wise writing trajectory

data with the video/audio data to create contents.

The page-wise writing trajectory data and the video/audio data are synchronized on the time axis in the order of input to create contents. Therefore, the present invention is widely applicable to tours, accident spots, consolatory letters, love letters, diaries, lectures, conferences, etc.