FIELD

Embodiments described herein relate generally to augmented reality (AR) and, more particularly, to using augmented reality to identify articles that a person is wearing or carrying and to provide other information pertinent to the articles, such as purchase information.

BACKGROUND

This section is intended to introduce the reader to various aspects of art, which may be related to the embodiments described below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light.

In multicast and broadcast applications, data are transmitted from a server to multiple receivers over wired and/or wireless networks. A multicast system as used herein is a system in which a server transmits the same data to multiple receivers simultaneously, where multiple receivers form a subset of all the receivers up to and including all of the receivers. A broadcast system is a system in which a server transmits the same data to all of the receivers simultaneously. That is, a multicast system by definition can include a broadcast system. SUMMARY

This summary is provided to introduce a selection of concepts in a simplified form that are further described in the Detailed Description section below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. For purposes of an implementation, an AR device such as the HoloLens™ that

Microsoft produces can be used. The commercially available HoloLens™ includes an untethered, see-through head-mounted display (HMD) driven by an integrated computer. Such devices enable a person (user) to interact with 3D models displayed so as to appear to be in the user's world (that is, the models appear to be outside the HMD). HoloLens™ differs from Google Glass (an HMD historically marketed by Google), which merely superimposed a screen into the field-of-view of one eye. Still another AR device is a tablet computer, for example the iPad by Apple, which can be aimed at a scene by the user and which uses a camera to capture the scene and presents it on the display of the tablet with annotations graphically overlaid. In some cases, a smartphone can provide this same kind of interaction and by itself be considered an AR device.

The present invention can be used with any such AR device, Traditional mobile devices such as smartphones have short-range wireless capabilities such as Bluetooth™ (BT) or Near Field Communication (NFC) technology which may be used to communicate between the smartphone and an HMD, such as the HoloLens™. BT and NFC each include a set of communication protocols that enable two electronic devices, one of which is usually a portable device such as a smartphone, to establish communication. While BT can operate over a distance of several meters, NFC operates by bringing the two devices within 4 cm (2 in) of each other. The combination of the smartphone device and HMD can also include a cellular data phone connection that can be used to obtain information from a communication network such as the Internet using a cell phone connection (e.g., 3G/4G).

Various exemplary embodiments described herein may relate to, include, or take the form of a method for using augmented reality and articles accompanying a user. Examples, without limitation, of such accompanying articles include a purse, backpack, books, recent purchases (e.g., in shopping bags), or cellular telephone carried by a user, a shirt, shoes, hat, sunglasses, contact lenses, or other clothing or items worn by the user, makeup, such as lipstick worn by a user; a baby stroller or automobile being operated by the user, and pet or the pet's accessories (e.g., a leash or dog sweater). For the purpose of discussion, the example embodiments herein use "articles of clothing worn by a user", as an exemplary subset of "accompanying articles" and "accompanying articles" should be understood throughout. It should be noted that clothing (apparel) includes but is not limited to hats, caps, scarfs, ear muffs, sweaters, vests, sweatshirts, shirts, pullovers, pants, jeans, skirts, dresses, shoes, boots, slippers, coats, jackets, gloves, and eyewear. Further, clothing is not limited to apparel worn visibly, but also may include underwear such as undershirts, thermal underwear, briefs, boxers, panties, bras, lingerie.

Related exemplary embodiments described herein may relate to, include, or take the form of computer program product tangibly embodied in a non-transitory storage medium. The computer program product may include a first set of instructions stored in the non-transitory storage medium executable by a processing unit to determine articles of clothing worn by the user. The computer program product may further include a second set of instructions stored in the non-transitory storage medium executable by the processing unit to determine relationships among users. The computer program product may additionally include a third set of instructions stored in the non-transitory storage medium executable by the processing unit to use augmented reality with the identified articles clothing worn by the user and relationships among users to forward the list of articles worn by the user and related purchase information to some or all users with whom the user has a relationship. The computer program can be tangibly embodied in a non-transitory storage medium which may be included in a mobile device. A mobile device includes but is not limited to a cellular phone, smartphone, laptop computer, personal digital assistant (PDA), tablet computer, augmented reality glasses or any other equivalent device.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made to representative exemplary embodiments illustrated in the accompanying figures. It is understood that the following descriptions are not intended to limit the disclosure a particular embodiment or a set of particular embodiments. To the contrary, this disclosure is intended to cover alternatives, modifications, and equivalents as may be included within the scope of the described embodiments as defined by the appended claims and as illustrated in the accompanying figures. The proposed method and apparatus is best understood from the following detailed description when read in conjunction with the accompanying drawings. The drawings include the following figures briefly described below:

Fig. 1 shows an exemplary HMD with an integrated computer suitable for augmented reality. Fig. 2 shows an exemplary smartphone having wireless communication with the

HMD of Fig. 1.

Fig. 3A shows a male and a female, each having accompanying articles (here, clothing items) on or about their person.

Fig. 3B is a system overview showing a male and a female. Fig. 4 depicts some fabric care tags and an exemplary QR code.

Fig. 5 is a flowchart of the operation of an augmented reality system (e.g., smartphone or HMD and integrated computer) in accordance with the principles described herein.

Fig. 6 is a block diagram of a mobile device in accordance with the principles described herein.

Fig. 7 is a flowchart of the operation of a device of the second user.

It should be understood that the drawing(s) are for purposes of illustrating the concepts of the disclosure and is not necessarily the only possible configuration for illustrating the disclosure. DETAILED DESCRIPTION

The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope. All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage.

Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The disclosure as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.

Augmented reality is a live view (direct, as with an HMD, or indirect, as with the display of a tablet computer or smartphone) of a physical, real world space whose elements are augmented (or supplemented) by a computing device, where the augmentation is based, at least in part, on sensor input. Such augmentation may include visual information, such as, images, video, computer graphics and the like, presented on the HMD or display, or audio presented over a speaker or earphone. Such sensor input may include any of audio captured from a microphone, images or video captured by a camera, position information as determined by a localization service (e.g., GPS or wireless triangulation or ranging technique), and orientation as can be determined from one or more of an accelerometer, flux gate, gyroscope, and camera. For example, computing device generated visual information may be displayed on (and/or projected onto) the transparent screen of an HMD, through which a user can see a physical space. By way of another example, an electronic display of a tablet computer or smartphone may present live video of a physical space that is combined with additional computing device generated visual information. Thus, augmented reality may enhance a user's perception of a physical space, in contrast to virtual reality (VR), which replaces a physical space with a simulated space.

To define an inventory of accompanying articles (e.g., apparel on a person), a user's cell phone (smartphone) is configured with a program used to identify the articles, for example, what that user is wearing on a specific day. In one embodiment, the smartphone can read, using a camera, a Quick Response (QR) code provided on an article, a Universal Product Code (UPC) associated with the article, another type of barcode or machine- readable code associated with the article, or the smartphone can accept a manually entered (that is, typed in) description of the items, or provide another user interface (UI) that permits the user to provide information about the article. An example description of what a user is wearing on a specific day might be, "Today I am wearing my Chicago Bears cap, my Budweiser T-Shirt, and my Levis 501 jeans". The descriptive information may also indicate color, a logo, and/or a thumbnail image of the corresponding article. In some embodiments, the UI permits the user to select from a list of articles previously entered, to reduce burden on the user. In some embodiments, the user can also specify where a particular article is being worn (e.g., pants on the lower body, T-shirt on the upper body).

The manufacturer or retailer of an article may have the incentive to include a machine-readable tag (e.g., barcode or radio frequency identification (RFID)) that identifies the article as a means of advertising. The identifying tag can provide a link to, or otherwise include information about the clothing article to allow the wearer of the clothes to sync the clothes to the mobile device rather than manually entering information about the clothes. During or after syncing the clothes to the mobile device, the wearer of the clothes would have the option of blocking the emission of the clothes identifying information, except as allowed by the social networking rules mentioned elsewhere in this description.

Information representing the user's defined inventory of articles (e.g., apparel), can be stored within the memory of the mobile device so that the information can be made available to other individuals using a short range wireless technology. Bluetooth, NFC, Zigbee and WiFi are examples of short range wireless technologies that could form the communication channel between the clothing and the mobile device. NFC technology is characterized by a very short range on the order of several centimeters, which would only be suitable for very close encounters. It would be preferable to use a short range wireless technology or a personal area network (PAN). In some embodiments, the user can also define where an article was purchased, or how much was paid. For example, when specifying an inventory of what the user is wearing on a particular day, the user can also specify that this item is available from a specific retailer (or manufacturer) for purchase. Such associations (between items of apparel and information about the availability or source of the items of apparel) can be made by using a user interface where a person can make such associations by inputting information into a form/user interface/and the like. In some such embodiments, when a user has identified the retailer of an accompanying article (a particular item being worn today), the retailer can be notified, that is, the retailer can be informed that a particular (or anonymous) user has annotated a particular article (e.g., a shirt) as being available from that retailer. Such notification may be the basis for a reward program, or to track the success of certain fashion items, to detect trends, and otherwise keep the retailer better informed.

As a further option, visual recognition software/device can be used to recognize different items (clothing, apparel, attire) that a person may wear or put on for a day. For example, Amazon has a service where a user can take a picture of an item and information about the item (whether done through a live person via the Amazon Mechanical Turk service or through machine vision techniques) and the Amazon service resolves the picture to determine the attributes of the item(s). If multiple identifications of an item are returned (i.e., the result offers several different identifications for the same article), the alternatives can be provided to a user and the user can select which one is correct.

Optionally, the mobile device can be configured to remember the source from which a user has purchased specific items. Hence, when a user inputs that they are wearing a specific item, the device can reference memory to see if that item is listed on a purchase history log that resides within the device or remotely. If the device recognizes that the item was previously purchased from a retailer, the device can offer an option to a user to see if they want to make this information available to others using the short range wireless technology referenced above.

As a gatekeeper between determining whether or not a user's information is made available to others, relationships set up through a social networking service such as Facebook, Google+, Instagram, Snapchat, Linkedln, and the like to determine if two people are "friends" or have "friends" in common. Likewise, an address book or contact list, such as used for an email, could be used, such that finding another user in the address book or contact list indicates a relationship with that other user. In some cases, contact lists include the nature of a relationship for each entry, or the nature of a relationship might be indicated in general notes. In a simple case, two users can be linked together by one user sending another user an invitation to be linked together through a social networking platform. The user receiving the invitation must accept in order for the social networking to associate the two users as "friends", allowing them to be networked together. These associations can be stored in the local memory of a device, or be stored remotely on a server.

Continuing with the simple case, social networking sites can be pinged (checked) periodically from the service to generate a list of contacts that are associated with a particular user. This information can be downloaded to a mobile device of the particular user and can be referenced when another person approaches the particular user. In some embodiments, permission to share specific classes of information, e.g., description of an article, source for an article, date of purchase, or price paid for an article, may be set by independent user-selectable policies. For example, a user might select settings to allow any user to see a description of an article (e.g., "Levi's 501 jeans"), only users who are friends might learn where a user purchased the article (e.g., "gap.com", and perhaps only selected users (e.g., your sister) is allowed to see what you paid for the article (e.g., "$39.99") and when (e.g., "Spring Sale, March 2016"). Perhaps nobody is allowed to see the size (e.g., "size 14 slim"). In some embodiments, such policies may be compounded and may provide exclusions, as well: For example a policy setting might be directed to "friends, but not co-workers" so that while someone classified as a friend might be able to access certain information, nobody a user knows from work (even those classified as friends) would see such information.

In the following discussion, the first user is the one wearing or carrying articles that are registered and described in the inventory. The second user is the one employing AR techniques to observe descriptions of the articles being worn or carried by the first user. In an AR setting, the second user, having an AR-capable device based on any of an HMD, tablet computer, or smartphone and including a second mobile device approaches a first user having a first mobile device. The first mobile device has been previously supplied with the inventory of articles being worn or carried by the first user. The AR-capable second mobile device represents the "querying user", while the first mobile device represents the "queried user". Here, "querying" and "queried" are relative terms: As used herein, "querying user" or "querying device" refers to the user or mobile device that makes an inquiry to another (i.e., first) user and/or mobile device in proximity to the querying user. Correspondingly, as used herein, "queried user" or "queried device" refers to the user or mobile device that receives an inquiry from another (i.e., second) user and/or mobile device, the queried (first) user being in corresponding proximity to the querying (second) user. Generally, the querying user will have line-of-sight to the queried user, so that the AR-capable device is able display information in conjunction with the view of the queried user. The queried mobile device receives a query from the querying device of the second user in order to verify that there is a social connection between the first and second users. To accomplish this, a second user identifying information is transmitted by the querying mobile device to the queried mobile device. The existence of a social connection can be verified on the basis of the second user identifying information being found within the social information of the queried mobile device. In some embodiments, the nature of a relationship is also determined, for example whether the first and second users are friends, or friends of friends. The query from the second device to the first device may be facilitated through a short range wireless technology, or in other embodiments may rely on communication through the Internet.

Alternatively, the queried mobile device can verify the existence of a social relationship with the second user by consulting a remote server (e.g., a social networking service) using an account of the queried (first) user, to determine whether or not a relationship exists based on the second user identifying information. In such an embodiment, the relationship is verified by using a data communication channel to query a social service about whether or not the second user' s identifying information is associated within that social service with the first user's account (i.e., has a relationship to the first user). If a social connection between the queried (first) user and the querying (second) user is affirmed based on the second user's identifying information, a determination is made as to what information is to be returned to the querying user. This determination may depend upon the nature of the relationship between the two users. In one example embodiment, what information is to be returned to the querying user is based on the degrees of separation between the two: Friends (1 degree), friend of a friend (2 degrees) etc. and further based on policies regarding what information about articles of the queried user are to be communicated to other users based on their particular relationship(s). This can be defined in the listing on what is to be presented. Only the information that should be presented to the querying user is then transmitted from the queried mobile device to the querying device.

In some embodiments, the querying mobile device may supply parameters so as to be more selective about the information to be obtained regarding the queried user's inventory of articles. Such parameters may represent information directed to a particular article or articles of the inventory: For example, the location of the article on the queried user's body may be supplied, such that a querying mobile device might provide "upper torso" as a location parameter, so as to obtain further information about the queried user's shirt. A querying mobile device might request a particular kind of information about an article, e.g., information about where the item can be purchased. In combination, two such parameters may mean, "where can I buy that shirt?". The information to be returned may be determined, at least in part, from parameters received from the querying mobile device. Alternatively, based on the information about article(s) returned by the queried mobile device, the querying mobile device can request additional information about the article(s) from an online store or other resource over the Internet using a data connection, as when the response from the queried mobile device includes article-identifying information such as a UPC code, model number, or text description (e.g., "Levi's 501"), or the response includes a hyperlink (i.e., a URL).

Rendering of the descriptive information obtained about the article is performed so as to present the descriptive information to the querying (second) user in apparent proximity to the queried user, thereby creating an AR presentation. In this operation, the AR-capable device will use information from the previous step to render descriptive information (item name, price, and source of where purchased) in display positions relative to different locations of the first user. That is, the AR-capable device of the second user generically identifies the head of the first user, the arm or arms of the first user, the chest of the first user, and/or the leg or legs of the first user (many devices can do this using well known principles).

The first user can be identified before querying by noting the unique identifiers of the mobile devices that are in short range wireless communication and consulting a database of smartphone identifiers and the users known to be associated with those smartphones. For example, if the querying device wirelessly detects and recognizes a unique identifier (e.g., the Bluetooth address, i.e., the BD_ADDR, of a smartphone previously known to be associated with the first user), then the querying mobile device can prepare to find the first user in proximity. In this case, the unique identifier of the queried smartphone will correspond to a first user identification information. If identification of the first user has not been made before the first mobile device is queried, then the reply from the queried device can include the first user identification information. Receipt of the first user identification information by the querying mobile device can make recognition of the first user easier because the querying mobile device can be prepared to find the first user in proximity. With or without the first user identification information, the identity of the first user can be determined by identifying a head of some person in proximity and in the field of view of a camera of the AR-capable device, and then attempting to match the face of that person, as captured by the camera, by facial recognition. If the first user identification information has been determined, then the attempt at facial recognition is easier and faster, as it is performed looking for a match to a particular person on the first user identification information, but even without that simplifying parameter, facial recognition can be attempted by matching against all individuals related to the querying user on the social networking site. Detection of a successful facial recognition, that is finding a match to an associated reference facial information, results in the identification of the head in the field of view as being the head of the first user.

Once the first user is identified within the field of view of the AR-capable device, the device can then display the descriptive information about the articles so as to overlay or otherwise indicate their association with the first (queried) user, or more particularly, to the identified body regions of the queried user using AR techniques. For example, information about the first user's pants could be rendered over or near the first user's "legs" with pricing and information about where to purchase such pants. Alternatively, the pants information can be rendered to appear beside the first user with a callout line pointing to the first user, or the legs of the first user. Likewise, information about the shirt the queried user is wearing could be rendered over or near the first user's "chest" (or rendered with a line pointing to the first user's chest) using the techniques described above. Other areas and variations of what can be shown and what items can be represented are possible in accordance with the disclosed principles. In some embodiments, identification of body parts or articles by the AR-capable device may be informed by information associated with the articles. For example, if the first mobile device has replied to a query by returning an inventory that includes a pink shirt, then identification in the field-of-view of the camera of the AR-capable second mobile device can make use of that information by detecting pink regions. In an optional embodiment, information can be searched against an online commercial database such as Amazon, to determine if similar alternatives to an identified item are available. This can include alternatives that are less expensive, similar in size, color, shape, material, and the like. It is even possible to offer alternatives that are the "most selected" by others or most popular as selected via by ratings, crowdsourced data, or the like.

Fig. 1 shows a head-mounted display (HMD) as may be worn by a querying second user, wherein the HMD is suitable for augmented reality applications either independently or in conjunction with a querying second smartphone, where Fig. 2 shows an exemplary queried first smartphone, as might be carried by a queried first user, the queried smartphone having communication with either the HMD of Fig. 1 which may perform the querying function without need of the second smartphone, or with the second smartphone, wherein the first smartphone is the queried mobile device.

Fig. 3A shows a male and a female having various articles about their respective persons. For example, the male is wearing a shirt, pants and shoes (sneakers) and is carrying a book. The female is wearing a shirt (blouse), pants and shoes and is carrying several shopping bags. The articles (e.g., of clothing) may have QR or other machine-readable codes on the articles themselves or on tags attached to the articles, e.g., in the manner of fabric care tags such as shown in Fig. 4. In the alternative the articles may have, or may have had, UPC codes on tags or packaging.

Fig. 3B is an exemplary system overview having a male figure and a female figure. The male figure has a smartphone and is wearing an AR device. The male figure's smart phone is in communication with a store server through the cloud. The male figure is wearing a green shirt or jacket on his torso and blue pants from the waist down. The female figure has a smartphone in her red purse and is wearing a pink shirt or jacket on her torso and a brown skirt from the waist down. The female figure's smart phone is in communication with a social network server through the cloud. The male figure is interested in the female figure's brown skirt. Perhaps he would like to purchase on for his wife or girlfriend. The two smartphones communicate, the smartphone carried by the male provides and identification of the male to the female's smartphone so that the female's smartphone can determine (alone or with the social network server) the relationship between the male and female, if any. Based on that relationship, the proposed method and apparatus allow the male figure to determine that the skirt is a maxi-skirt and available from LongRiver.com. If the information about the skirt as supplied by the female's smartphone is only identification of the articles she has (and optionally the articles' placements and/or colors), then the male's smartphone can interrogate the remote store server for more details. Whether this or further information such as the cost are disclosed to the male figure, will depend on his relationship with the female figure and the corresponding permissions The AR device may display this information to the male in alignment with the male's view of the female. Fig. 5 is a flowchart of the operation of the queried (first) mobile device, e.g., a smartphone or AR-capable device (such as HoloLens™), in accordance with the principles of the proposed method. At 505 the smartphone or AR-capable device scans the first user's articles (e.g., clothing) and the corresponding description tags. This scanning of the first user's articles defines an inventory of articles to be worn or carried by the first user and is the preferred method by which the mobile device identifies the articles. For example, the smartphone can utilize QR codes that are on such articles (e.g., the first user's clothing) or UPCs associated with the articles. It is also possible for the mobile device, at this point, to accept a manually entered (typed in or, with voice recognition, dictated) description of the articles through a UI. Alternatively, the UI can accept a manual selection by the first user of the articles from a list of articles historically entered. The first user can also specify where an article is being worn or carried (pants on the lower body, T-shirt on an upper body, etc.). Optionally, the first user can also define where particular articles can be (or were) purchased. For example, when defining the inventory of what the first user is wearing on a particular day, the first user can also define that this item is available from a specific retailer for purchase. Such associations (between an article and information about the article's availability) can be made through a user interface where the first user makes such associations by inputting information into a form or other UI. As a further option, an image recognition module can be used to recognize different articles (e.g., each specific item of clothing) that a user may wear or carry today. For example, Amazon has a service where a customer can take a picture of an article and information about the article is returned. Whether done through a live person via the Amazon Mechanical Turk service or through machine vision techniques, the Amazon service resolves the picture to determine the identity and attributes of the article(s). When such a service returns multiple alternative articles as matching the image, the user can select which of the alternatives is correct. Optionally, a mobile device can be configured to remember (retrieve from memory) where the first user has purchased specific items, e.g., through an online retailer such as Amazon, or based on automatic or manual entry of a shopping receipt. Hence, when the first user inputs that they are wearing or carrying a specific article, the device can reference memory to see if that article is listed on a purchase history log that resides within the mobile device, or in remote storage accessible by the mobile device (e.g., server, gateway or cloud storage). If the mobile device recognizes that the article was previously purchased from a retailer, the mobile device can automatically import, or offer to the first user the option of importing, some or all of this information to be made available to others through the present invention, using the short range wireless technology referenced above. At 510 the inventory of articles is stored along with the associated description information. The inventory and corresponding descriptions can be stored within the memory accessible to the mobile device so that the information can be made available to other individuals using short range wireless technology.

At 515 the mobile device is queried by another mobile device regarding a social connection with a second user in close proximity of the first user, the second user typically having line of sight to the first user.

At 520 the mobile device performs a lookup and responds to the query. The mobile device acts as a gatekeeper to determine whether or not the first user's information is made available to other individuals (e.g., the second user). Relationships set up through a social networking service such as Facebook, Google+, Instagram, Snapchat, Linkedln, and the like are used to see if two people (the first and second users) have a relationship, e.g., that they are "friends" or have "friends" that are in common. Such relationships can be stored in the memory accessible to the mobile device, allowing the relationships to be stored in local memory, or on a remote server (e.g, "in the cloud"). This enables the mobile device to perform a lookup to determine whether a relationship exists between the first user and the second user. Social networking sites can be checked periodically by the mobile device to maintain a list of contacts that have a relationship with the first user. This information can be downloaded to a mobile device of the first user and can be referenced when a second user approaches or queries the mobile device through a short range wireless technology. Alternatively, the queried mobile device can verify a social relationship with a user identifying information that corresponds to the second user. The relationship of the users is verified by using a data communication channel to query a social networking service about whether or not the user identifying information corresponding to the second user is associated with the account corresponding to the first user on the social network service. The result of the lookup indicates whether there is a relationship between the two users, and if so, may further specify the nature of that relationship.

At 525 the mobile device determines what information can be disclosed and transmitted to the second mobile device (that of the second user), based on the results of the lookup. If it is affirmed that there is a social relationship between the first and second users (as verified by the social information), a determination is made as to what information is to be represented, that is, is allowed to be disclosed or transmitted to the second user. This determination may depend upon the nature of the relationship between the first and second users (for example, the degrees of separation between the two users: Friends (1 degree), friend of a friend (2 degrees) etc.) and what information about the articles present with the user are to be communicated to the other users based on the nature of the relationship between them. What is to be shared given a particular nature of relationship can be defined explicitly per article in the inventory or can be more generally represented as policies (e.g., only share pricing information about articles with friends, but not friends of friends).

The appropriate information of what can or should be disclosed (that is, returned) is then transmitted from the first user's mobile device to that of the second user at 530. The mobile device of the second user may perform a lookup and reference more information about articles in the inventory of the first user, including a name of the article, a location of the article on or about the first user's body (e.g., a shirt is worn on the upper torso), any historical purchase information about the article, and information about where the article can be purchased now or was purchased. This information can be wholly or partially determined from information received from the first user's mobile device. Alternatively, from some of the information received from the first user's mobile device, the AR-capable mobile device of the second user can request additional information about such items from a store or other online resource over the Internet using a data connection. In an embodiment, information can be searched against a commercial database such as an Amazon et al., to determine whether alternatives for (i.e., products similar to) an identified article are available. This can include alternatives that are less expensive, similar in size, color, shape, material, and the like. It is even possible to offer alternatives that are the "most selected" by others or most popular as selected via by ratings, review sites, or other crowdsourced data.

Fig. 6 is a block diagram of a mobile device in accordance with the principles described herein. The mobile device is an AR-capable device such as HoloLens™. The block diagram configuration includes a bus-oriented 650 configuration interconnecting a processor 620, and a memory 645. The configuration of Fig. 6 also includes a network interface 605 to a gateway, such as router or gateway. The router or gateway may utilize either a wired or a wireless interface to the mobile device. The network interface also provides a short range wireless interface. Processor 620 provides computation functions for the mobile device. The processor

620 can be any form of CPU or controller that utilizes communications between elements of the mobile device to control communication and computation processes. Those of skill in the art recognize that bus 650 provides a communication path between the various elements of the embodiment and that other point-to-point interconnection options (e.g. non- bus architecture) are also feasible. The mobile device is queried by another mobile device regarding a social connection with a second user in close proximity of the first user, the second user typically having line of sight to the first user. The processor performs the query by accessing memory and/or via the network interface in order to access social connection information residing in a server or cloud storage. The mobile device acts as a gatekeeper to determine whether or not the first user's information is made available to other individuals (e.g., the second user). Relationships set up through a social networking service such as Facebook, Google+, Instagram, Snapchat, Linkedln, and the like are used to see if two people (the first and second users) have a relationship, e.g., that they are "friends" or have "friends" that are in common. Such relationships can be stored in the memory accessible to the mobile device, allowing the relationships to be stored in local memory, or on a remote server (e.g, "in the cloud"). This enables the mobile device to perform a lookup to determine whether a relationship exists between the first user and the second user. Social networking sites can be checked periodically by the mobile device to maintain a list of contacts that have a relationship with the first user. This information can be downloaded to a mobile device of the first user and can be referenced when a second user approaches or queries the mobile device through a short range wireless technology. Alternatively, the queried mobile device can verify a social relationship with a user identifying information that corresponds to the second user. The relationship of the users is verified by using a data communication channel to query a social networking service about whether or not the user identifying information corresponding to the second user is associated with the account corresponding to the first user on the social network service. The result of the lookup indicates whether there is a relationship between the two users, and if so, may further specify the nature of that relationship.

The mobile device determines what information can be disclosed and transmitted to the second mobile device (that of the second user), based on the results of the lookup. If it is affirmed that there is a social relationship between the first and second users (as verified by the social information), a determination is made as to what information is to be represented, that is, is allowed to be disclosed or transmitted to the second user. This determination may depend upon the nature of the relationship between the first and second users (for example, the degrees of separation between the two users: Friends (1 degree), friend of a friend (2 degrees) etc.) and what information about the articles presently with the user are to be communicated to the other users based on the nature of the relationship between them. What is to be shared given a particular nature of relationship can be defined explicitly per article in the inventory or can be more generally represented as policies (e.g., only share pricing information about articles with friends, but not friends of friends).

The appropriate information of what can or should be disclosed (that is, returned) is then transmitted from the first user's mobile device to that of the second user via the network interface. The mobile device of the second user may perform a lookup and reference more information about articles in the inventory of the first user, including a name of the article, a location of the article on or about the first user's body (e.g., a shirt is worn on the upper body), any historical purchase information about the article, and information about where the article can be purchased now or was purchased. This information can be wholly or partially determined from information received from the first user's mobile device. Alternatively, from some of the information received from the first user's mobile device, the AR-capable mobile device of the second user can request additional information about such items from a store or other online resource over the Internet using a data connection. In an embodiment, information can be searched against a commercial database such as an Amazon et al., to determine whether alternatives for (i.e., products similar to) an identified article are available. This can include alternatives that are less expensive, similar in size, color, shape, material, and the like. It is even possible to offer alternatives that are the "most selected" by others or most popular as selected via by ratings, review sites, or other crowdsourced data.

Memory 645 is accessible to processor 620 and can act as a repository for memory related to any of the methods that incorporate the functionality of the media device. Memory 645 can provide the repository for storage of information such as program memory, downloads, uploads, or scratchpad calculations. The inventory of articles is stored along with the associated description information. The inventory and corresponding descriptions can be stored within the memory accessible to the mobile device so that the information can be made available to other individuals using short range wireless technology. One or more external databases can be accessed via the database interface 635. Those of skill in the art will recognize that memory 645 may be incorporated all or in part of processor 620. Portions of memory 645 may be remote from the mobile device (not shown) such that access by processor 620 is wireless. Network interface 605 has both receiver and transmitter elements for communication as known to those of skill in the art. User interface and display 610 is driven by interface circuit 615. The interface 610 is used as a multimedia interface having both audio and video capability to display streamed and/or downloaded audio and/or video obtained via network interface 605 connected to a network. In some embodiments, user interface and display 610 includes a head-mounted display, including the sensors as previously discussed. It is also possible for the mobile device, at this point, to accept a manually entered (typed in or, with voice recognition, dictated) description of the articles through a UI. Alternatively, the UI can accept a manual selection by the first user of the articles from a list of articles historically entered and available via the memory described above. The first user can also specify where an article is being worn or carried (pants on the lower body, T-shirt on an upper body, etc.). Optionally, the first user can also define where particular articles can be (or were) purchased. For example, when defining the inventory of what the first user is wearing on a particular day, the first user can also define that this item is available from a specific retailer for purchase. Such associations (between an article and information about the article's availability) can be made through a user interface where the first user makes such associations by inputting information into a form or other UI. As a further option, an image recognition module can be used to recognize different articles (e.g., each specific item of clothing) that a user may wear or carry today. For example, Amazon has a service where a customer can take a picture of an article and information about the article is returned. Whether done through a live person via the Amazon Mechanical Turk service or through machine vision techniques, the Amazon service resolves the picture to determine the identity and attributes of the article(s). When such a service returns multiple alternative articles as matching the image, the user can select which of the alternatives is correct. Optionally, a mobile device can be configured to remember where the first user has purchased specific items, e.g., through an online retailer such as Amazon, or based on automatic or manual entry of a shopping receipt. Hence, when the first user inputs that they are wearing or carrying a specific article, the device can reference memory to see if that article is listed on a purchase history log that resides within the mobile device, or in remote storage accessible by the mobile device (e.g., server, gateway or cloud storage). If the mobile device recognizes that the article was previously purchased from a retailer, the mobile device can automatically import, or offer to the first user the option of importing, some or all of this information to be made available to others through the present invention, using the short range wireless technology referenced above.

Database interface 635 is used to connect the mobile device to a database of information about known articles (e.g., to retailers of clothing items for clothing item descriptions, costs, and any other information, for example to facilitate comparison shopping by the second user).

In alternate embodiments, the databases can be accessed via a public or private network and the database interface 635 represents the network interface connecting to the databases. In another embodiment, the database interface 635 is used to connect to a local database (not shown). User interface and display 610 and its respective display driver 615 may be used to manage local data in memory 645, in particular the inventory created as described in conjunction with step 505.

Camera 655 is configured to scan the user's articles and the corresponding description tags. . This scanning of the first user's articles defines an inventory of articles to be worn or carried by the first user and is the preferred method by which the mobile device identifies the articles. For example, the smartphone can utilize QR codes that are on such articles (e.g., the first user's clothing) or UPC codes associated with the articles. The processor 620 is shown to have access to the camera 655 via the communication bus 650, though communication with the camera may incorporate a wireless link (not shown), for example if the camera 655 is located on an HMD remote from the processor 620.

Fig. 7 is a flowchart of the operation of a device of the second user. At 705 the device of the second user receives a request from the second user for information about an article of a first user. In one example UI, this can be achieved by the second user centering his field of view in the AR device on the article of the second user and speaking a trigger phrase such as "What is that?" which is recognized by a speech detection module, where both gaze direction determination and speech detection modules are components of the user interface 610. Identification of the first user as being in or near the second user's field- of-view can be achieved as described above.

At 710, the device of the second user sends a query to the device of the first user regarding social connection information, the query including identifying information about the second user. Information about the article should not be returned to the device of the second user until the degrees of relationship (social connection) between the first user and the second user is determined, if any. The social connection (relationship) determination is performed by the device of the first user using either locally stored relationship information, or by accessing social networking sites such as Facebook, Linkedln etc.

At this point, the device of the first user performs the steps 515, 520, 525, and 530, as described in conjunction with Fig. 5.

At 730 the selected information about the article is received from the device of the first user by the device of the second user. In some embodiments, the selected information about the article may be enhanced by the second device by a query to additional resources (e.g., the store server of Fig. 5).

At 735 the selected information about the article is displayed for the second user by the second user's device. The article information includes availability of the article including information about the retailer and pricing information. It should be noted that the device of the second user may be a smartphone or AR device or a combination of the smartphone and the AR device.

The mobile device of the second user includes components similar to those of the first user described above and shown in Fig. 6. The processor 620 of the mobile device (e.g., AR-capable device) requests information about an article of a first user via the network interface 605. The processor 620 of the mobile device of the second user receives the information about the article of the first user via the network interface 605. The received information about the article of the first user is forwarded to the interface and display driver 615 which forwards the information about the article of the first user to the user interface and display 610.

Although specific architectures are shown for the proposed implementation, one of skill in the art will recognize that implementation options exist such as distributed functionality of components and consolidation of components. Such options are equivalent to the functionality and structure of the depicted and described arrangements. It is to be understood that the proposed method and apparatus may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof. Special purpose processors may include application specific integrated circuits (ASICs), reduced instruction set computers (RISCs) and/or field programmable gate arrays (FPGAs). Preferably, the proposed method and apparatus is implemented as a combination of hardware and software. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage device. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (CPU), a random access memory (RAM), and input/output (I/O) interface(s). The computer platform also includes an operating system and microinstruction code. The various processes and functions described herein may either be part of the microinstruction code or part of the application program (or a combination thereof), which is executed via the operating system. In addition, various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device. It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input/output interfaces. Herein, the phrase "coupled" is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software based components.

It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures are preferably implemented in software, the actual connections between the system components (or the process steps) may differ depending upon the manner in which the proposed method and apparatus is programmed. Given the teachings herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the proposed method and apparatus.

For purposes of this application and the claims, using the exemplary phrase "at least one of A, B and C," the phrase means "only A, or only B, or only C, or any combination of A, B and C."