text:
US9594427B2 - Finger tracking - Google Patents
Finger tracking Download PDF
Nissanka Arachchige Bodhi Priyantha
Jie Liu
2014-05-23 Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
2014-05-23 Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PRIYANTHA, NISSANKA ARACHCHIGE BODHI, LIU, JIE
210000002435 Tendons Anatomy 0 abstract claims description 22
238000004458 analytical methods Methods 0 claims description 10
239000004984 smart glasses Substances 0 claims description 10
230000036410 touch Effects 0 description 35
230000003993 interaction Effects 0 description 11
230000001133 acceleration Effects 0 description 8
230000001939 inductive effects Effects 0 description 5
238000003306 harvesting Methods 0 description 4
238000010079 rubber tapping Methods 0 description 3
239000008264 clouds Substances 0 description 2
229920002574 CR-39 Polymers 0 description 1
241000854491 Delta Species 0 description 1
210000004247 Hand Anatomy 0 description 1
210000001503 Joints Anatomy 0 description 1
210000000707 Wrist Anatomy 0 description 1
230000003139 buffering Effects 0 description 1
239000011162 core materials Substances 0 description 1
239000010432 diamond Substances 0 description 1
239000010437 gem Substances 0 description 1
239000010977 jade Substances 0 description 1
230000003278 mimic Effects 0 description 1
239000011707 mineral Substances 0 description 1
230000000704 physical effects Effects 0 description 1
230000011218 segmentation Effects 0 description 1
230000007958 sleep Effects 0 description 1
210000001519 tissues Anatomy 0 description 1
238000007514 turning Methods 0 description 1
239000011089 white board Substances 0 description 1
238000004804 winding Methods 0 description 1
G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
G06F1/00—Details not covered by groups G06F3/00 – G06F13/00 and G06F21/00
G06F1/16—Constructional details or arrangements
G06F1/1613—Constructional details or arrangements for portable computers
G06F1/163—Wearable computers, e.g. on a belt
G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
G06F1/1698—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone
G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
G06F2203/033—Indexing scheme relating to G06F3/033
G06F2203/0331—Finger worn pointing device
The description relates to smart rings. One example can include a finger band configured to accommodate a user's finger. This example can also include a set of pressure sensors positioned on an inner surface of the finger band and configured to sense changes to tendons of the user's finger as pressure differentials and to output associated signals. The example can further include a gesture component configured to interpret the signals from the set of pressure sensors to identify individual actions performed by the user's finger.
Increasingly users interact with their mobile devices on the go. Many users continuously utilize various device applications. For instance, users listen to music on mobile devices while traveling, and constantly check E-mails. Shoppers browse through their shopping lists and do price comparisons while shopping, among others.
Interacting with mobile devices on-the-go tends to require the user to enter different gestures to scroll, zoom, flip, and/or enter text on graphical user interface (GUI) elements. The smart phone with its relatively large display has provided a unified and convenient platform for such interactions. However, more recent trends in wearable devices such as glasses, wrist bands, and watches have made such interactions limited and awkward due to the lack of touch real estate and the positioning of the device itself.
While mobile device interfaces continue to shrink, interfaces of remote display devices such as TVs and game consoles are becoming even more complex, requiring extensive maneuvering via simple remote controllers or requiring remote control with full keyboard-like capability. For example, with a conventional remote control, a simple task such as entering text to search for a movie title becomes a monumental task leading to a poor user experience.
The above listed example is intended to provide a quick reference to aid the reader and is not intended to define the scope of the concepts described herein.
The accompanying drawings illustrate implementations of the concepts conveyed in the present document. Features of the illustrated implementations can be more readily understood by reference to the following description taken in conjunction with the accompanying drawings. Like reference numbers in the various drawings are used wherever feasible to indicate like elements. Further, the left-most numeral of each reference number conveys the FIG. and associated discussion where the reference number is first introduced.
FIGS. 1 and 2 collectively show example smart ring applications in accordance with some implementations of the present concepts.
FIGS. 3-5 show example smart ring use case scenarios in accordance with some implementations of the present concepts.
FIG. 6 shows a system example in accordance with some implementations of the present concepts.
FIG. 7 shows stroke gesture primitives that can be utilized in accordance with some implementations of the present concepts.
FIG. 8 shows a state diagram that can be utilized in accordance with some implementations of the present concepts.
FIG. 9 shows an example flow chart in accordance with some implementations of the present concepts.
The present concepts relate to allowing a user to use his/her fingers to control a device. The implementations relate to a wearable ring platform (e.g., smart ring) worn on one or more of the user's fingers. (As used herein, the term 'finger' can include the 'thumb'). The smart ring can detect and interpret various control gestures or actions performed by the user. The smart ring can wirelessly transmit the control gestures to the device. As used herein any action that can be performed by one or more fingers can be thought of as a gesture. Thus, the action of touching a surface with the finger can be an example of a gesture. Similarly, touching and sliding the finger on a surface can be a gesture. Moving the finger in the air or bending the finger can be a gesture. Further, the action of waving with one or more fingers can be a gesture. Actions performed by both hands can also be a gesture. In summary, actions can include touch gestures and non-touch gestures and/or single finger gestures and multi-finger gestures. Further examples of actions (e.g., gestures) are described below.
Introductory FIGS. 1-2 show a human hand 100. Most of the hand is covered with skin 102. However, for purposes of explanation, the skin is not shown for a portion of the hand relating to the index finger 104 to allow visualization of the underlying tissues. More specifically, tendons 106 are visible on the index finger. These tendons can move when the finger moves or when the finger imparts a force on a surface. FIG. 2 introduces a pressure sensitive smart ring (or 'smart ring device' or 'smart ring') 202 positioned on the user's index finger. The pressure sensitive smart ring can include elements for detecting changes to the tendons. Examples of these elements are discussed in more detail below relative to FIG. 6. (Note that to avoid clutter on the drawing page, on FIG. 2, the tendon 106 lead line only goes to one of the tendons rather than two in FIG. 1).
FIG. 3 shows a first use case scenario 300 involving a user 302 wearing an example of pressure sensitive smart ring 202. In this case the user is engaging a digital display device 304. In this example, the digital display device includes a depth sensor 306. In some implementations the depth sensor can be manifest as a red, green, blue, plus depth (RGBD) camera. Various types of visible light, non-visible light, and/or sonic depth sensors, among others can be employed.
The depth sensor 306 can be useful in detecting the presence of user 302 in front of the digital display device 304 and detecting gestures performed by the user in front of the digital display device. However, the depth sensor may not be able to accurately detect user actions close to the digital display device. For instance, the depth sensor may not be able to distinguish whether the user is pointing at a location on the digital display device or touching the location on the digital display device. The pressure sensitive smart ring 202 can determine whether the user's finger is touching a surface as indicated by 'starburst' 308. Further, the pressure sensitive smart ring 202 can determine that while touching the surface the finger is moving in a downward direction as indicated by arrow 310. Data from the depth sensor 306 and the pressure sensitive smart ring 202 can detect user gestures in front of the digital display device as well as user contact of the digital display device. Thus, the pressure sensitive smart ring 202 and the depth sensor 306 can collectively provide information that can allow the digital display device to function in a manner similar to a touch sensitive display device without actually being 'touch sensitive.'
FIG. 4 shows another use case scenario 400 that is similar to the use case scenario of FIG. 3. In this case, the pressure sensitive smart ring 202 can operate cooperatively with smart glasses 402 to provide information about user 302. For instance, the smart glasses can capture information about what is in front of the user. In this example, the smart glasses can 'see' content on the digital display device 304. The smart glasses may be less effective at determining whether the user 302 is touching the digital display device 304. However, as explained above relative to FIG. 3, the pressure sensitive smart ring 202 can accurately determine whether the user is touching the digital display device. The pressure sensitive smart ring 202 and the smart glasses 402 can communicate with the digital display device 304 to detect user gestures (both touch and non-touch) to provide an enhanced interactive experience to the user.
FIG. 5 shows another use case scenario 500 at instance one, instance two, and instance three. Scenario 500 involves pressure sensitive smart rings 202(1) and 202(2) and a smart watch 502. Pressure sensitive smart ring 202(1) is positioned on the user's index finger 104 and pressure sensitive smart ring 202(2) is positioned on the user's middle finger. Instance one shows the user touching the smart watch with his index finger 104. This touch can be detected by pressure sensitive smart ring 202(1). This touch can be interpreted as a first user command, such as to select an application to run on the smart watch.
Instance two shows the user performing a non-touch command by bending the index finger 104 at the middle joint. This non-touch command can be detected by pressure sensitive smart ring 202(1) and distinguished by the smart ring from the command of instance one by a pressure profile (e.g., which pressure sensors detect pressure). This command could be interpreted as a scroll down command by the smart watch, for example.
Instance three shows a similar non-touch command to instance two except that it is performed by both the index finger and the middle finger (e.g. a multi-finger gesture) and detected by pressure sensitive smart ring 202(1) and pressure sensitive smart ring 202(2). This command could be interpreted as a 'select' command by the smart watch, for example. Mechanisms for accomplishing this functionality are described below relative to FIG. 6.
FIG. 6 shows a gesture detection system 600. For purposes of explanation, system 600 includes pressure sensitive smart ring 202, digital display device 304, depth sensor 306, smart glasses 402, smart watch 502, and a smart phone 602. Any of these devices can communicate over one or more networks 604.
Two configurations 606(1) and 606(2) are illustrated for pressure sensitive smart ring 202. Briefly, configuration 606(1) represents an operating system centric configuration and configuration 606(2) represents a system on a chip configuration. Configuration 606(1) is organized into one or more applications 610, operating system 612, and hardware 614. Configuration 606(2) is organized into shared resources 616, dedicated resources 618, and an interface 620 there between.
In either configuration, the pressure sensitive smart ring 202 can include storage 622, a processor 624, a battery 626 (or other power source), a battery charger 628, sensors 630, a communication component 632, and/or a gesture component 634. These elements can be positioned in/on or otherwise associated with a physical finger band 636. For instance, the elements can be positioned within the finger band 636 so that the pressure sensitive smart ring 202 has the general appearance of a traditional 'jewelry' ring. The finger band 636 can be formed of various materials such as plastics, polymers, and/or natural materials such as jade or other minerals. The pressure sensitive smart ring 202 can also include ornamental aspects such as precious stones to mimic a traditional jewelry ring.
From one perspective, any of pressure sensitive smart ring 202, digital display device 304, depth sensor 306, smart glasses 402, smart watch 502, and/or smart phone 602 can be thought of as computers.
The term "device," "computer," or "computing device" as used herein can mean any type of device that has some amount of processing capability and/or storage capability. Processing capability can be provided by one or more processors that can execute data in the form of computer-readable instructions to provide a functionality. Data, such as computer-readable instructions and/or user-related data, can be stored on storage, such as storage that can be internal or external to the computer. The storage can include any one or more of volatile or non-volatile memory, hard drives, flash storage devices, and/or optical storage devices (e.g., CDs, DVDs etc.), remote storage (e.g., cloud-based storage), among others. As used herein, the term "computer-readable media" can include signals. In contrast, the term "computer-readable storage media" excludes signals. Computer-readable storage media includes "computer-readable storage devices." Examples of computer-readable storage devices include volatile storage media, such as RAM, and non-volatile storage media, such as hard drives, optical discs, and/or flash memory, among others.
As mentioned above, configuration 606(2) can be thought of as a system on a chip (SOC) type design. In such a case, functionality provided by the device can be integrated on a single SOC or multiple coupled SOCs. One or more processors can be configured to coordinate with shared resources, such as memory, storage, etc., and/or one or more dedicated resources, such as hardware blocks configured to perform certain specific functionality. Thus, the term "processor" as used herein can also refer to central processing units (CPU), graphical processing units (CPUs), controllers, microcontrollers, processor cores, or other types of processing devices.
Generally, any of the functions described herein can be implemented using software, firmware, hardware (e.g., fixed-logic circuitry), manual processing, or a combination of these implementations. The term "component" as used herein generally represents software, firmware, hardware, whole devices or networks, or a combination thereof. In the case of a software implementation, for instance, these may represent program code that performs specified tasks when executed on a processor (e.g., CPU or CPUs). The program code can be stored in one or more computer-readable memory devices, such as computer-readable storage media. The features and techniques of the component are platform-independent, meaning that they may be implemented on a variety of commercial computing platforms having a variety of processing configurations.
The battery 626 can be charged in various ways by the battery charger 628. In one instance, the battery charger is manifest as a wireless inductive charger. The wireless inductive charger can include multiple conductive coils wrapping around the pressure sensitive smart ring 202, such as following the ring shape of the finger band 636. In other configurations the coils can be associated with the ring but not coiled around the ring. For instance, small coils can be contained within the finger band or otherwise associated with the finger band. The wireless inductive charger can capture magnetic energy from wireless chargers. The wireless chargers can be included in any of digital display device 304, depth sensor 306, smart glasses 402, smart watch 502, smart phone 602, and/or other devices. Beyond the charging facet, the wireless inductive charging feature can provide a proximity sensing function for the two devices. For instance, when the user places his hand with the pressure sensitive smart ring 202 near the smart phone 602, the wireless inductive charger can detect the magnetic field of the smart phone and thus indicate to the pressure sensitive smart ring 202 that it is proximate to the smart phone. Similarly, the entry of the pressure sensitive smart ring 202 into the magnetic field generated by the smart phone can indicate to the smart phone that the pressure sensitive smart ring 202 is proximate to the smart phone. In summary, some pressure sensitive smart ring implementations can opportunistically harvest energy from an NFC-enabled companion device, such as devices 304, 306, 402, 502, and/or 602, among others for perpetual operation without explicit charging.
Further, powering sensors and analyzing sensor data consumes power. The gesture component 634 can manage the sensors 630, battery 626, processor 624, and/or other components to conserve resources. In other configurations a power management controller (not shown) may manage the components. Several techniques for conserving power usage are described below.
Multiple types of sensors 630 can be included in the pressure sensitive smart ring 202. Of course, the sensors can include pressure sensors 638. In this example, three pressure sensors are illustrated positioned on an inner surface 640 (e.g., inwardly facing surface) of the finger band 636. In this case, when the pressure sensitive smart ring 202 is worn by the user, pressure sensor 638(1) is configured to be positioned at the 8-9 o'clock position (with the diamond at the top of the ring acting as 12 o'clock). Pressure sensor 638(2) is configured to be positioned at the 6 o'clock position and pressure sensor 638(3) is configured to be positioned at the 3-4 o'clock position. Of course, other numbers and/or positions of pressure sensors can be employed. For example, a radial array of pressure sensors could be positioned on the inside surface of the ring so that even if the ring spins on the user's finger, pressure detected by generally opposing pressure sensors can be indicative of a touch gesture. Examples of pressure sensors can include force sensitive resistors and/or piezoelectric sensors, among others. In some cases the pressure sensor can be coupled to a low power comparator and controlled by power gating as determined by the gesture component 634.
Sensors 630 can also include accelerometers, gyroscopes, magnetometers, and/or microphones, among others. For instance, the implementation of FIG. 4 can include pressure sensors to detect contact with the surface and accelerometers to detect the downward movement. While not shown in FIG. 4, the gyros could further detect 'twisting' of the pressure sensitive smart ring 202 to allow a straight down movement to be distinguished from a down and to the left or a down and to the right gesture. In another configuration, motion can be detected via the microphone as audio signals. Such a configuration can use less power than may be consumed by accelerometers.
In one such implementation an audio based motion detector can have multiple sub modules, such as a microphone with a built-in amplifier and high pass filter, a high-pass analog filter to filter out typical environmental noise, an amplifier, a low-pass filter, and an envelope detector.
The gesture component 634 can receive input from the sensors 630. The gesture component 634 can identify user actions from the sensor input. For instance, assume that the user performs the touch action of instance one of FIG. 5. In that case, pressure sensors 638(1) and 638(3) of FIG. 6 would detect pressure created by the lateral tendons and pressure sensor 638(2) would not detect pressure. The gesture component can interpret the output of the three sensors 638(1)-638(3) as a touch gesture. Now assume that the user performs the finger bend gesture of FIG. 5 instance two. In that case, each of sensors, 638(1), 638(2), and 638(3) would detect pressure. The gesture component can interpret the output of the three sensors 638(1)-638(3) as a non-touch finger bend gesture.
Other action detection (e.g., gesture detection) scenarios that involve identification of tapping, swiping, scrolling, and stroking for hand written text entry are described below.
The communication component 632 can allow the pressure sensitive smart ring 202 to communicate with various devices, such as the illustrated companion devices. The communication component can include a receiver and a transmitter and/or other radio frequency circuitry for communicating with various technologies, such as cellular, Wi-Fi (IEEE 802.xx), Bluetooth, etc.
Note that in some cases the gesture component 634 on the pressure sensitive ring 202 can be relatively robust and perform analysis on signals received from sensors 630 to identify a user action. The gesture component could then send an indication of the identified user action to a companion device (e.g., any of devices 304, 306, 402, 502, and/or 602) that the pressure sensitive smart ring is operating in cooperation with. In other cases, the pressure sensitive smart ring could send the signals to the companion device. A gesture component (not shown) on the companion device could then analyze the sensor signals to identify the user action. In a similar scenario the pressure sensitive smart ring could send information to a remote resource, such as cloud based resources for processing.
The discussion below provides further details on specific implementations of the pressure sensitive smart ring 202. For purposes of explanation, in this discussion smart phone 602 operates as a companion device to the pressure sensitive smart ring 202. In this case, sensors 630 include force sensitive resistors, accelerometers, and an audio based sensor configured to detect motion of the user's finger on a surface. In this discussion, only a single pressure sensitive smart ring 202 is employed, but as noted above, the user could wear anywhere from one to ten pressure sensitive smart rings 202 (e.g., one on each finger including thumbs).
To conserve energy, the gesture detection component 634 can place the pressure sensitive smart ring 202 in an inactive sleep state to prevent accidental interpretation of day-to-day user activities as gestures. In this state, the ring's accelerometer can be in a low-power autonomous motion-detection state.
When the user is ready to enter gestures on an available surface, the pressure sensitive smart ring 202 can be brought into an active state by tapping the surface four times (or some other unique action). Once in the active state, the pressure sensitive smart ring 202 can enter a touch-detect active state by turning on the touch detection sensor. The 1st tap can help trigger the accelerometer motion detector, while the rest of the taps can be used to reduce accidental triggering of the active state. The gesture component 634 can collect raw accelerometer readings which can also be buffered during the touch-detect state.
When the user touches the surface to enter a gesture, the touch is detected by the pressure sensors, and the pressure sensitive smart ring 202 can enter motion-detect state. In the motion-detect state, the audio-based motion detector can be turned on to detect the motion of the finger along the surface. During this state, processed accelerometer components along a plane of the ring can be stored (e.g., yz plane). At the end of motion or the touch, the gesture detection component 634 can employ a classifier to the accelerometer data to identify the gesture.
This implementation of the gesture component 634 can implement the following four groups of gestures including, tap, swipe, scroll, and written text, which are summarized in Table 1. Of course other gestures are contemplated, especially when the user wears pressure sensitive smart rings on multiple fingers.
Tap. The tap gesture can satisfy a similar functionality to the typical tap gesture on a touch sensitive surface, or the left mouse click. Similar to touch surfaces and mice, multiple, closely spaced, tap actions can define multiple gestures. For instance, two double-taps can be used to transition between "active" and "inactive" modes as described above.
Swipe. The swipe gesture can be used with primitive touch interfaces for quickly scanning through multiple windows or a collection of items such as photos. This implementation of the pressure sensitive smart ring 202 can support four different swipe actions which include: swipe-up, swipe-down, swipe-left, and swipe-right.
TABLE 1 Different gestures, including different modes, and number of classification primitives can be implemented on the pressure sensitive smart ring 202. Gesture Modes # primitives Tap down 1 Swipe up, down, left, right 4 Scroll up, down, left, right 6 right-up, left-down Written Text English characters 12
Scroll. The scroll action can be similar to the touch surface or mouse-based scroll. The gesture component 634 can identify six scroll actions: scroll-left, scroll-right, scroll-up, scroll-down, scroll-right-up, and scroll-left-down.
Writing-based text entry. The pressure sensitive smart ring 202 can also implement a writing-based English text entry, where the user can simply write the characters on a surface using the index finger.
The gesture component 634 can treat each character as a combination of multiple primitive shapes called "strokes". FIG. 7 shows examples of stroke gesture primitives 700. In this example, 12 strokes are used for text entry in this implementation. The strokes can include directional and/or rotational movements. The gesture component can identify these strokes, and can also measure inter-stoke latencies which can help identify strokes belonging to a single character. The stroke identifiers and inter-stroke latencies can be used to identify the individual characters at a remote device.
In this implementation, size and weight constraints limit the size of the battery used to power the pressure sensitive smart ring 202. In one case, a 10 mAh battery capacity can be used to approximate a traditional jewelry type ring. This battery can fit in the crown or the finger band of the pressure sensitive smart ring 202.
This relatively small 10 mAh battery capacity may cause the user to recharge the battery several times within a day, which would potentially be a large obstacle to the usability of the platform. To overcome this limitation, the pressure sensitive smart ring 202 can use the subcarrier based NFC energy harvesting approach to passively recharge the ring battery while the user is holding the smart phone 602 next to the ring.
To harvest energy from an NFC enabled antenna, the smart ring has a coil loop wound around the ring finger band. Winding the coil around the ring finger band can enable a large (and potentially maximum) loop size to achieve better energy harvesting.
Surface Detection
This implementation can use a combination of two techniques to detect the motion of the user's finger on a surface. As mentioned above, the pressure sensor can detect the finger touching a surface by sensing the pressure imparted by a lateral tendon of the finger; while an energy-efficient audio sensor can detect the movement of the finger on the surface by listening to the audio signal generated by the friction between the finger and the surface.
More specifically, the gesture component 634 can interpret the acoustic signals generated due to the friction when the finger moves across the surface to detect finger motion on the surface.
The amount of audio energy emitted from the finger/surface interaction is a function of the speed of finger motion and the physical properties of the surface.
Some implementations can offer enhanced surface detection using pressure and audio-based techniques. For instance, a combination of tendon pressure and audio based techniques can be analyzed to provide efficient and reliable surface detection due to the complementary characteristics of these techniques.
The tendon pressure-based technique can consume an order of magnitude less power than the audio-based solution, however, unlike the audio based solution, the pressure-based technique tends not to be able to identify finger motion directly. Accelerometers offer another solution, but at the potential cost of extra processing overhead and segmentation errors that can severely affect the system performance.
The audio-solution can detect both motion and touch during motion. However, the audio-based solution can be affected by ambient audio noise-induced errors. Band-pass filtering can remove a significant portion of these errors. The touch detector can prevent the accidental triggering of motion sensing due to errors that are not filtered. The touch detector can also prevent cascading failures by touch-based gating of the gesture detection in the presence of noise.
Gesture Classification
In some implementations the gesture component 634 can utilize classifiers for identifying the gesture primitives. For instance, gesture primitives can be categorized based on starting dynamics, real-timeliness, and context dependency of interpretation.
Hard landing gestures start with the user's finger landing on the surface at a relatively high velocity. Both taps and swipes can be hard landing gestures. Soft landing gestures can start with the finger meeting the surface at a low velocity. Both scroll gestures and strokes for text entry can be soft landing gestures.
The scroll gestures can entail real-time identification, since the user tends to desire continuous feedback on the current scroll position. Due to short interaction time, both tap and swipe gestures can be identified in non-real-time after the gesture has been completed. The strokes for text entry can also be identified in non-real-time, at the end of each stroke to improve classification accuracy. This is acceptable since the high-level symbols, such as characters, can only be interpreted after collecting all strokes that make up the character, due to lack of absolute position.
Context free gestures are gestures that can be interpreted on the pressure sensitive smart ring 202 without the knowledge of the user interface (UI) element the user is interacting with. Both tapping and swiping gestures belong to this category.
Context dependent gestures can utilize knowledge of the current UI element type for correct interpretation by the pressure sensitive smart ring 202. Some of the stroke and scroll gestures that look identical can be interpreted differently due to different real-time needs. To enable proper interpretation of these, the gesture component 634 can operate on the presumption that the remote device informs the pressure sensitive smart ring 202 when the user starts and stops interacting with a text entry area.
Resolving Angular Ambiguity of Accelerometer Data
The pressure sensitive smart ring 202's gesture component 634 can use the accelerometer data to identify different gestures on a surface. When gestures are performed on a surface, the signature of the gesture tends to be captured in the acceleration components that are parallel to the surface of interaction. Since the pressure sensitive smart ring 202, when worn on the finger, does not stay parallel to the interaction surface, and since individual users' frames of reference will differ from one another, the gesture component can convert the (x,y,z) components of the accelerometer to the (x,y,z) components with respect to the interacting plane. Some implementations can assume a horizontal plane of interaction, which is true for most of the interaction surfaces available in the environment. Other implementations can handle the interaction on an arbitrarily inclined plane.
For computing the components along the interacting plane, the gesture component 634 can use the gravity acceleration vector just prior to movement of the finger to determine the inclination of the finger to the plane, since this angle can vary across users and instances.
To perform this normalization, the gesture component 634 can compute two angles: pitch (8) and roll (0). These angles can be computed according to the following equations:
θ xyz = tan - 1  ( - G px G pz ) ( 1 ) ϕ xyz = tan - 1  ( G py G px 2 + G pz 2 ) ( 2 )
After computing these angles while the finger is stationary, they can be applied to subsequent accelerometer samples to compensate for finger orientation while the finger is moving:
x normal =−x·cos(−φ)+z·sin(−φ) (3)
y normal =y·cos(θ)−z·sin(θ) (4)
There are two limitations to correcting for the finger angle in this way. First, some implementations can operate on the assumption that these angles do not change during a gesture—if this assumption is violated, the gravity vector may pollute the x and y components, and a fraction of the x and y accelerations will be falsely attributed to the z axis. Second, some implementations do not correct for a third orientation angle, yaw (ψ), if gyros are not utilized (gravity vector provides no information as to how the finger twists in a plane perpendicular to gravity). Thus, these implementations can leverage the assumption that the user's finger motion will be perpendicular to their body and the impact of ψ is negligible. If this assumption is violated, the x and y acceleration components may not be properly separated. While worth noting, neither of these limitations tends to have a significant impact on classification accuracy.
Gesture Classification Overview
Some implementations of the gesture component 634 can use a two-level classification scheme for gesture classification. When the pressure sensors indicate a touch event, the top-level classifier, called the landing classifier, can be invoked to classify the event as either a soft landing event or a hard landing event. The gesture component 634 can evaluate the output of the landing classifier and the UI context reported by the companion device (e.g., the device the user is engaging). The gesture component 634 can then invoke one of the low level classifiers—swipe-tap, stroke, or scroll.
The gesture component 634 can also invoke the swipe-tap classifier to handle a hard landing. This classifier can use the surface touch duration—determined by the length of the audio envelope, and the accelerometer samples to classify the event as either a tap gesture or one of the four swipe gestures.
For a soft landing, if the context reports text input, the gesture component 634 can invoke the stroke classifier. Since there can be multiple consecutive strokes during a single touch, for example when writing L character, accelerometer data collected after the soft landing can be segmented based on the audio envelope. These segmented data can be fed to the stroke classifier to classify the stroke to one of the 12 possible strokes. This process can continue until the finger leaves the surface.
For a soft landing with a non-text input context, the gesture component 634 can invoke the scroll classifier. The scroll classifier can first detect a short 'nudge' at the start of the scroll based on the audio envelope, and can classify the data collected between the touch event and the end of the nudge to determine the type of scroll event. After this classification stage, the smart ring periodically can transmit 'in-scroll' messages to the companion device to provide real-time information on the continued scroll event until the finger stops moving and leaves the surface.
FIG. 8 shows a state diagram 800 of how different classification stages can be invoked in some implementations. The rest of this section describes these classification stages in more detail.
Landing Classifier
While the finger is moving on a surface, negligible activity on an accelerometer's z-axis is expected. However, when the finger initially lands on the surface, the finger can abruptly stop, causing large magnitude, short-lived spikes induced by the finger's sudden deceleration. The gesture component 634 (FIG. 6) can classify hard landings versus soft landings by exploiting this activity on the accelerometer z-axis at the beginning of a gesture entry. When a surface is initially detected at 802, n samples can be evaluated surrounding the start of motion. These n samples may be used as the input to a heuristic, threshold-based classification or as the input to an SVM classifier. The intuition behind the efficacy of each of these approaches is the existence of large, observable deltas in z-axis acceleration. To summarize, the z-axis magnitude can be compared to a threshold. A magnitude below the threshold can indicate a soft landing 804. A magnitude that is equal to or above the threshold can indicate a hard landing 806.
Swipe-Tap classifier
The swipe-tap classifier can be invoked after detecting a hard landing 806. A duration envelope of the contact of the finger across the surface can be compared to a duration threshold. A duration envelope that is less than the duration threshold can indicate a tap event 808. A duration that is longer than the duration threshold can indicate a swipe event 810.
A tap gesture can be identified solely from the envelope information, while identification of a swipe can entail further classification.
Stroke Classifier
Returning to soft landing 804, the soft landing can be evaluated to distinguish a scroll 812 from a stroke 814. An SVM classifier can be utilized to classify strokes. The SVM classifier can use the x, y, and z axis accelerometer readings as the feature vector. Since the stroke duration can vary at different instances as well as across users, a sample, such as 100 points can be linearly interpolated across each x and y axis sample for a given gesture. A fixed number of averages can be computed across these interpolated points for each axis. This set of averages can then be passed to the SVM classifier.
Scroll Classifier
Scroll classification can happen after a stroke is detected based on the context of the remote user interface (e.g., the companion device that the user is engaging). A short gesture, called a 'nudge', can be detected and classified using the SVM classifier to determine the start of one of the six possible scroll gestures. Some implementations can utilize a relatively small number of (such as six) different scroll gestures to ease classification of the scroll action with only a small number of samples. After the user performs the nudge, the length of the envelope can provide real-time information to the remote device on the progress of the scroll action.
Operating Modes
When the pressure sensitive smart ring 202 is in the active mode (touch-detect, or touch-process), the activity of the platform can be driven by the accelerometer. The accelerometer can autonomously collect samples at ≃400 Hz (for example) and can interrupt the microcontroller when a new sample is available.
During touch-detect, the sampled data can be buffered for landing type estimation, the microcontroller interrupt service routine (ISR) also can check the status of the touch detector.
If touch is active, the pressure sensitive smart ring 202 can transition to touch-process mode. In this mode, the audio-based motion detector output can be sampled at ≃100 Hz with processor ADC (at every 4th ISR). Once motion is detected—determined by the change of audio envelope by a preset threshold—the pressure sensitive smart ring 202 can start accumulating coordinate-aligned accelerometer data. At the end of the motion, the appropriate classifier can be invoked, and the results can be transmitted over the radio (e.g., communication component 632), such as by using a best effort transmission scheme.
SVM Classifier Implementation
As introduced above, an SVM classifier can be employed for gesture classification. The SVM classifier can be a multi-class linear-kernel SVM classifier. The SVM classifier can use pair-wise classification, entailing n(n−1)/pair-wise classifiers to classify n classes.
Some implementations can use four features for each x and y axis, resulting in eight SVM features. Each (F(i)X, F(i)Y) feature tuple can be calculated by first breaking all the (x,y) acceleration data in to four buckets, and then averaging the data in each bucket. The x and y acceleration component of each data sample can be computed immediately after reading (x,y,z) acceleration data sample from the accelerometer.
Due to limited RAM size, instead of buffering data at 400 Hz until the end of a gesture, some implementations can employ running averages of each n samples of data to be calculated. Value n can be selected such that there are at most 60 computed averages for the worst case duration of each gesture class. Once the gesture ends, these pre-computed averages can be broken into four equal-size buckets to compute the four feature tuples.
NFC Energy Harvesting
As mentioned above, wireless charging can be achieved via magnetic fields generated by a companion device, such as smart phone 602. Some implementations can be configured so that the charging occurs efficiently (potentially maximally) when the user holds the smart phone in a natural manner with the ring bearing hand.
In summary, at least some of the present implementations can offer a low-power wearable smart ring that can enable users to enter gestures by interacting on arbitrary surfaces. The smart ring can use energy efficient finger-tendon based touch detection and audio-based motion detection to capture user interaction instances. A light-weight mufti-classifier solution accurately classifies different gesture primitives. In one case, using a 10 mAh battery powered by energy harvested from an NFC-enabled smart phone, the ring can support more than 10 hours of active user interactions.
Example Methods
FIG. 9 illustrates a flowchart of a method or technique 900 that is consistent with at least some implementations of the present concepts.
At block 902, the method can obtain signals reflecting pressure of a tendon of a user's finger on a ring positioned on the finger.
At block 904, the method can interpret the signals to identify an action performed by the finger.
In some cases, the method can be performed on the ring by a processor or microcontroller. In other cases, the signals can be sent from the ring to another device, such as a companion device that is proximate to the ring and is working cooperatively with the ring. The companion device can then perform the interpreting. Some implementations can involve a single user wearing multiple rings. In such a case, each ring can communicate its signals to the companion device (e.g., with a different ring identifier with the signals). The companion device can then interpret actions relating to a single finger or multiple fingers (e.g., multi-finger gestures). For instance, the user may have waved a single finger to invoke a specific user command or the user could have waved all the fingers of the hand to invoke a different user command. In some cases, the smart ring may contain other sensors that sense other parameters. Signals from the other sensors can be interpreted in combination with signals from the pressure sensors to identify the user action. In still other cases, sensors on other devices may provide signals that can be utilized in combination with the sensed signals to identify the user action. Examples of such scenarios are illustrated relative to FIGS. 3-5 where the digital display 304, the depth sensor 306, the smart glasses 402, and/or the smart watch 502 may sense the user and provide data that is useful in combination with the pressure data. Also, in some cases, such as a digital whiteboard scenario, multiple users may be performing actions and each user may be wearing one or more smart rings.
The order in which the above methods are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement the method, or an alternate method. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof, such that a computing device can implement the method (e.g., computer-implemented method). In one case, the method is stored on a computer-readable storage media as a set of instructions such that execution by a processor of a computing device causes the computing device to perform the method.
In summary, the present implementations can derive useful information about user actions from pressure sensors on a smart ring worn on a user's finger.
The described methods or processes can be performed by the systems and/or devices described above, and/or by other devices and/or systems. The order in which the methods are described is not intended to be construed as a limitation, and any number of the described acts can be combined in any order to implement the method, or an alternate method. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof, such that a device can implement the method. In one case, the method is stored on computer-readable storage media as a set of instructions such that execution by a processor of a computing device causes the computing device to perform the method.
Although techniques, methods, devices, systems, etc., pertaining to detecting user actions are described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed methods, devices, systems, etc.
The invention claimed is:
1. A smart ring device, comprising:
a finger band configured to accommodate a user's finger;
at least one pressure sensor secured to the finger band in a manner that can detect pressure imparted by a tendon passing through the finger band during performance of actions by the user's finger; and,
a gesture component configured to:
perform an analysis of pressure signals from the at least one pressure sensor indicating that the tendon has imparted pressure on the at least one pressure sensor; and
based at least on the analysis of the pressure signals indicating that the tendon has imparted pressure on the at least one pressure sensor, detect that the user's finger is touching a surface of another device.
2. The smart ring device of claim 1, further comprising a battery and a wireless charger for the battery contained in the finger band.
3. The smart ring device of claim 2, wherein the wireless charger comprises conductive coils, wherein the conductive coils are wrapped around the finger band or wherein the conductive coils are associated with the finger band but do not wrap around the finger band.
4. The smart ring device of claim 1, wherein the at least one pressure sensor comprises an array of pressure sensors radially distributed around an inner surface of the finger band.
5. The smart ring device of claim 1, wherein the pressure sensor comprises a force sensitive resistor or a piezoelectric sensor.
6. The smart ring device of claim 1, further comprising motion detectors.
7. The smart ring device of claim 6, wherein the motion detectors comprise audio sensors or accelerometers.
8. The smart ring device of claim 1, further comprising a communication component configured to communicate an indication of the user's finger touching the surface to a companion device.
9. The smart ring device of claim 8, wherein the communication component comprises a Bluetooth transmitter.
10. The smart ring device of claim 1, wherein the gesture component is further configured to:
in a first instance:
receive an indication that the another device is in a text input context; and
responsive to the indication that the another device is in the text input context, interpret an action performed by the user's finger using a stroke classifier; and
in a second instance:
receive an indication that the another device is in a non-text input context; and
responsive to the indication that the another device is in the non-text input context, interpret the action using a scroll classifier.
11. A smart wearable device, comprising:
a set of pressure sensors positioned on an inner surface of the finger band and configured to sense changes to tendons of the user's finger as pressure differentials and to output associated pressure signals indicating pressure imparted by the tendons on the set of pressure sensors; and
a processor configured via computer-readable instructions that, when executed by the processor, cause the processor to:
perform an analysis of the pressure signals indicating the pressure imparted by the tendons on the set of pressure sensors; and
based at least on the analysis of the pressure signals indicatinq the pressure imparted by the tendons on the set of pressure sensors, detect that the user's finger is touching a surface of another device.
12. The smart wearable device of claim 11, wherein the computer-readable instructions, when executed by the processor, cause the processor to:
distinguish an individual touch action performed by the touching from another individual action where the user performs a non-touch gesture without touching the surface of the another device.
13. The smart wearable device of claim 12, wherein the non-touch gesture comprises a finger bend.
14. The smart wearable device of claim 11, further comprising another sensor and wherein the computer-readable instructions, when executed by the processor, cause the processor to:
determine direction of movement of the smart wearable device from output of the another sensor.
15. The smart wearable device of claim 14, wherein the another sensor comprises a microphone.
16. A method performed by one or more processors, the method comprising:
obtaining pressure signals reflecting pressure imparted by tendon of a user's finger on a pressure sensor positioned on the user's finger;
performing an analysis of the pressure signals indicating the pressure imparted by the tendon on the pressure sensor; and
based at least on the analysis, interpreting the pressure signals reflecting the pressure of the tendon as an indication that the user's finger has contacted a surface of another device.
17. The method of claim 16, wherein the pressure sensor is positioned on a ring and the analysis is performed by the ring.
18. The method of claim 16, wherein the pressure sensor is positioned on a wearable device, the obtaining the pressure signals is performed by a companion device other than the wearable device, and wherein the companion device performs the analysis.
19. The method of claim 18, further comprising:
obtaining, by the companion device, further pressure signals from other pressure sensors positioned on other fingers of the user, and wherein the analysis comprises identifying a multi-finger gesture performed by the user based at least on the pressure signals and the further pressure signals.
20. The method of claim 18, wherein the companion device comprises a smart phone, a smart watch, smart glasses, or a display device.
obtaining additional signals relating to movement of the user's finger while in contact with the surface of the another device and wherein the interpreting comprises interpreting the pressure signals and the additional signals to identify an action performed while the user's finger is in contact with the surface of the another device.
US14/286,910 2014-05-23 2014-05-23 Finger tracking Active US9594427B2 (en)
US14/286,910 US9594427B2 (en) 2014-05-23 2014-05-23 Finger tracking
PCT/US2015/031290 WO2015179262A1 (en) 2014-05-23 2015-05-18 Finger tracking
CN201580024018.8A CN106462239B (en) 2014-05-23 2015-05-18 Finger trail
EP15728263.3A EP3146410B1 (en) 2014-05-23 2015-05-18 Finger tracking
KR1020167036032A KR20170008854A (en) 2014-05-23 2015-05-18 Finger tracking
US15/223,887 US10191543B2 (en) 2014-05-23 2016-07-29 Wearable device touch detection
US16/005,056 US10359846B2 (en) 2014-05-23 2018-06-11 Wearable device gesture detection
US15/223,887 Continuation US10191543B2 (en) 2014-05-23 2016-07-29 Wearable device touch detection
US14/286,910 Active US9594427B2 (en) 2014-05-23 2014-05-23 Finger tracking
US15/223,887 Active US10191543B2 (en) 2014-05-23 2016-07-29 Wearable device touch detection
US16/005,056 Active US10359846B2 (en) 2014-05-23 2018-06-11 Wearable device gesture detection
KR (1) KR20170008854A (en)
US10459495B2 (en) * 2017-12-19 2019-10-29 North Inc. Wearable electronic devices having an inward facing input device and methods of use thereof
US9594427B2 (en) 2014-05-23 2017-03-14 Microsoft Technology Licensing, Llc Finger tracking
US9582076B2 (en) 2014-09-17 2017-02-28 Microsoft Technology Licensing, Llc Smart ring
US10474191B2 (en) 2014-10-15 2019-11-12 Motionvirtual, Inc. Wearable device
US9619049B2 (en) * 2015-02-18 2017-04-11 International Business Machines Corporation One-handed operation of mobile electronic devices
US9891718B2 (en) 2015-04-22 2018-02-13 Medibotics Llc Devices for measuring finger motion and recognizing hand gestures
US9921651B2 (en) * 2015-04-29 2018-03-20 International Business Machines Company Video display for visually impaired people
US9887591B2 (en) 2015-05-13 2018-02-06 International Business Machines Corporation Electric ring with external power source
KR20170028130A (en) 2015-09-03 2017-03-13 박준호 Wearable device
WO2018006291A1 (en) * 2016-07-06 2018-01-11 华为技术有限公司 Interaction device, system and method
US10251056B2 (en) * 2016-07-13 2019-04-02 Motorola Mobility Llc Electronic device with gesture actuation of companion devices, and corresponding systems and methods
US10372892B2 (en) 2016-07-13 2019-08-06 Motorola Mobility Llc Electronic device with gesture actuation of companion devices, and corresponding systems and methods
US9716779B1 (en) * 2016-08-31 2017-07-25 Maxine Nicodemus Wireless communication system
US10095342B2 (en) 2016-11-14 2018-10-09 Google Llc Apparatus for sensing user input
EP3340031A1 (en) * 2016-12-21 2018-06-27 Nxp B.V. Motion detection circuits, and associated methods
US10001808B1 (en) * 2017-03-29 2018-06-19 Google Llc Mobile device accessory equipped to communicate with mobile device
US10013081B1 (en) 2017-04-04 2018-07-03 Google Llc Electronic circuit and method to account for strain gauge variation
WO2018187557A1 (en) * 2017-04-05 2018-10-11 Microsensor Labs, LLC System and method for proximity sensing of movable portions of a premises
US10514797B2 (en) 2017-04-18 2019-12-24 Google Llc Force-sensitive user input interface for an electronic device
CN107329574A (en) * 2017-06-30 2017-11-07 联想(北京)有限公司 Input method and system for electronic equipment
US20190347479A1 (en) * 2018-05-10 2019-11-14 International Business Machines Corporation Writing recognition using wearable pressure sensing device
US20030142065A1 (en) 2002-01-28 2003-07-31 Kourosh Pahlavan Ring pointer device with inertial sensors
US20030214481A1 (en) 2002-05-14 2003-11-20 Yongming Xiong Finger worn and operated input device and method of use
US20040012559A1 (en) 2002-07-17 2004-01-22 Kanazawa University Input device
US20040032346A1 (en) 2002-08-19 2004-02-19 Samsung Electro-Mechanics Co., Ltd. Information input device, information processing device and information input method
US6882128B1 (en) 2000-09-27 2005-04-19 Science Applications International Corporation Method and system for energy reclamation and reuse
US20060001646A1 (en) 2004-07-02 2006-01-05 Wei Hai Finger worn and operated input device
EP1785808A1 (en) 2005-11-10 2007-05-16 Research In Motion Limited System and method for activating an electronic device
US20080014989A1 (en) 2006-07-13 2008-01-17 Sony Ericsson Mobile Communications Ab Conveying commands to a mobile terminal through body actions
US20090058830A1 (en) * 2007-01-07 2009-03-05 Scott Herz Portable multifunction device, method, and graphical user interface for interpreting a finger gesture
US20090096746A1 (en) 2007-10-12 2009-04-16 Immersion Corp., A Delaware Corporation Method and Apparatus for Wearable Remote Interface Device
US20090278798A1 (en) 2006-07-26 2009-11-12 The Research Foundation Of The State University Of New York Active Fingertip-Mounted Object Digitizer
US20090295758A1 (en) 2008-05-27 2009-12-03 Industrial Technology Research Institute Method for writing motion and trajectory recognition and writing apparatus and recognizing system
US20100007511A1 (en) 2008-07-14 2010-01-14 Sony Ericsson Mobile Communications Ab Touchless control of a control device
US20100030695A1 (en) 2008-02-08 2010-02-04 Microsoft Corporation Mobile device security using wearable security tokens
US20100302137A1 (en) 2009-05-27 2010-12-02 Microsoft Corporation Touch Sensitive Display Apparatus using sensor input
US20110007035A1 (en) * 2007-08-19 2011-01-13 Saar Shai Finger-worn devices and related methods of use
EP2302882A1 (en) 2009-09-24 2011-03-30 Research In Motion Limited Communication device and method for initiating NFC communication
WO2011055326A1 (en) 2009-11-04 2011-05-12 Igal Firsov Universal input/output human user interface
US20110212687A1 (en) 2010-02-26 2011-09-01 Research In Motion Limited Communications device responsive to near field communications (nfc) to enter locked or unlocked operating modes
US8033916B2 (en) 2007-05-04 2011-10-11 Theodore Caldwell Grip pressure sensor
US20110260839A1 (en) 2010-04-27 2011-10-27 Passif Semiconductor Corp Autonomous battery-free microwave frequency communication system
US20110313922A1 (en) 2009-06-22 2011-12-22 Mourad Ben Ayed System For NFC Authentication Based on BLUETOOTH Proximity
US20120024947A1 (en) 2010-07-29 2012-02-02 Bank Of America Corporation Wearable article having point of sale payment functionality
US20120038652A1 (en) 2010-08-12 2012-02-16 Palm, Inc. Accepting motion-based character input on mobile computing devices
US20120075173A1 (en) 2010-09-23 2012-03-29 Nokia Corporation Apparatus and method for user input
US20120083205A1 (en) 2010-10-04 2012-04-05 Qualcomm Incorporated Nfc device having a differential input envelope detector
US8179604B1 (en) 2011-07-13 2012-05-15 Google Inc. Wearable marker for passive interaction
US20120139708A1 (en) * 2010-12-06 2012-06-07 Massachusetts Institute Of Technology Wireless Hand Gesture Capture
US20120178365A1 (en) 2011-01-07 2012-07-12 Ran Katz Non-Volatile Memory for Contactless Systems
US20120206485A1 (en) 2010-02-28 2012-08-16 Osterhout Group, Inc. Ar glasses with event and sensor triggered user movement control of ar eyepiece facilities
US20120256860A1 (en) * 2011-04-05 2012-10-11 James Robert Justice Directional Finger Recognition Authoring
US8292833B2 (en) 2009-08-27 2012-10-23 Electronics And Telecommunications Research Institute Finger motion detecting apparatus and method
US20120309354A1 (en) 2011-06-06 2012-12-06 Syracuse University Situation aware security system and method for mobile devices
US20120315843A1 (en) 2011-06-10 2012-12-13 Broadcom Corporation Communications Device for Intelligently Routing Information Among Multiple User Interfaces
US20130005241A1 (en) 2011-06-28 2013-01-03 Broadcom Corporation Memory arbitrator for electronics communications devices
US20130022214A1 (en) 2011-07-19 2013-01-24 Dolby International Ab Method and System for Touch Gesture Detection in Response to Microphone Output
US20130027341A1 (en) 2010-04-16 2013-01-31 Mastandrea Nicholas J Wearable motion sensing computing interface
US20130050087A1 (en) 2011-08-31 2013-02-28 Hon Hai Precision Industry Co., Ltd. Multi-functional mouse assembly and input operation method thereof
US20130100044A1 (en) 2011-10-24 2013-04-25 Motorola Mobility, Inc. Method for Detecting Wake Conditions of a Portable Electronic Device
US8447704B2 (en) 2008-06-26 2013-05-21 Microsoft Corporation Recognizing gestures from forearm EMG signals
US20130127980A1 (en) 2010-02-28 2013-05-23 Osterhout Group, Inc. Video display modification based on sensor input for a see-through near-to-eye display
US20130135223A1 (en) * 2009-12-13 2013-05-30 Ringbow Ltd. Finger-worn input devices and methods of use
US20130157565A1 (en) 2011-12-14 2013-06-20 Marvell World Trade Ltd. Method and apparatus for charging a battery in a mobile device through a near field communication (nfc) antenna
US8478195B1 (en) 2012-02-17 2013-07-02 Google Inc. Two-factor user authentication using near field communication
US20130169560A1 (en) 2012-01-04 2013-07-04 Tobii Technology Ab System for gaze interaction
US20130203345A1 (en) 2005-12-31 2013-08-08 Blaze Mobile Wireless Bidirectional Communications between a Mobile Device and Associated Secure Element using Inaudible Sound Waves
US20130229508A1 (en) 2012-03-01 2013-09-05 Qualcomm Incorporated Gesture Detection Based on Information from Multiple Types of Sensors
US20130260689A1 (en) 2012-03-30 2013-10-03 Qualcomm Incorporated Wirelessly Powered Input Device
US8559987B1 (en) 2005-12-31 2013-10-15 Blaze Mobile, Inc. Wireless bidirectional communications between a mobile device and associated secure element
US20130290522A1 (en) 2012-04-11 2013-10-31 Edward F. Behm, Jr. Engine, System and Method of Locating a Mobile Device and Reporting on Other Devices Proximately Located Thereto
US20130285969A1 (en) 2011-09-30 2013-10-31 Giuseppe Raffa Detection of gesture data segmentation in mobile devices
US8572764B2 (en) 2010-12-09 2013-11-05 Dieter Thellmann Exercising glove
US8615374B1 (en) 2006-06-09 2013-12-24 Rockwell Automation Technologies, Inc. Modular, configurable, intelligent sensor system
US20140028547A1 (en) 2012-07-26 2014-01-30 Stmicroelectronics, Inc. Simple user interface device and chipset implementation combination for consumer interaction with any screen based interface
US20140085177A1 (en) * 2012-09-21 2014-03-27 Nokia Corporation Method and apparatus for responding to input based upon relative finger position
US20140101755A1 (en) 2012-10-10 2014-04-10 Research In Motion Limited Mobile wireless communications device providing security features based upon wearable near field communication (nfc) device and related methods
US20140143737A1 (en) 2012-11-20 2014-05-22 Samsung Electronics Company, Ltd. Transition and Interaction Model for Wearable Electronic Device
US20140139422A1 (en) 2012-11-20 2014-05-22 Samsung Electronics Company, Ltd. User Gesture Input to Wearable Electronic Device Involving Outward-Facing Sensor of Device
US20140139637A1 (en) * 2012-11-20 2014-05-22 Samsung Electronics Company, Ltd. Wearable Electronic Device
US20140138447A1 (en) 2012-11-20 2014-05-22 Omne Mobile Payments LLC Electronic card with a programmable magnetic stripe
US20140139486A1 (en) 2012-11-20 2014-05-22 Samsung Electronics Company, Ltd. Placement of Optical Sensor on Wearable Electronic Device
US20140143784A1 (en) 2012-11-20 2014-05-22 Samsung Electronics Company, Ltd. Controlling Remote Electronic Device with Wearable Electronic Device
US20140139454A1 (en) 2012-11-20 2014-05-22 Samsung Electronics Company, Ltd. User Gesture Input to Wearable Electronic Device Involving Movement of Device
US8742623B1 (en) 2013-09-16 2014-06-03 Google Inc. Device with dual power sources
US20140176809A1 (en) 2012-12-25 2014-06-26 Askey Computer Corp Ring-type remote control device, scaling control method and tap control method thereof
US20140187150A1 (en) 2012-12-27 2014-07-03 Cambridge Silicon Radio Limited Near Field Communication Devices and Methods
US20140285416A1 (en) 2013-03-20 2014-09-25 Microsoft Corporation Short Range Wireless Powered Ring for User Interaction and Sensing
US20140308930A1 (en) 2013-04-12 2014-10-16 Bao Tran Timely, glanceable information on a wearable device
US20150078586A1 (en) 2013-09-16 2015-03-19 Amazon Technologies, Inc. User input with fingerprint sensor
US20150133193A1 (en) 2013-03-15 2015-05-14 Smart Patents L.L.C Wearable devices and associated systems
US20150220158A1 (en) 2014-01-07 2015-08-06 Nod Inc. Methods and Apparatus for Mapping of Arbitrary Human Motion Within an Arbitrary Space Bounded by a User's Range of Motion
US20150326985A1 (en) 2014-05-08 2015-11-12 Microsoft Corporation Hand-worn device for surface gesture input
US20160077587A1 (en) 2014-09-17 2016-03-17 Microsoft Corporation Smart ring
US20160334870A1 (en) 2014-05-23 2016-11-17 Microsoft Technology Licensing, Llc Wearable device touch detection
US5832296A (en) * 1995-04-26 1998-11-03 Interval Research Corp. Wearable context sensitive user interface for interacting with plurality of electronic devices of interest to the user
US7042438B2 (en) 2003-09-06 2006-05-09 Mcrae Michael William Hand manipulated data apparatus for computers and video games
KR101299682B1 (en) 2006-10-16 2013-08-22 삼성전자주식회사 Universal input device
US20090251407A1 (en) * 2008-04-03 2009-10-08 Microsoft Corporation Device interaction with combination of rings
KR100920252B1 (en) 2008-11-07 2009-10-05 서창수 The mouse controlled by movements of finger
US8717291B2 (en) 2009-10-07 2014-05-06 AFA Micro Co. Motion sensitive gesture device
WO2011106797A1 (en) 2010-02-28 2011-09-01 Osterhout Group, Inc. Projection triggering through an external marker in an augmented reality eyepiece
US20140063055A1 (en) * 2010-02-28 2014-03-06 Osterhout Group, Inc. Ar glasses specific user interface and control interface based on a connected external device type
US20120075196A1 (en) * 2010-09-23 2012-03-29 Nokia Corporation Apparatus and method for user input
US10061387B2 (en) * 2011-03-31 2018-08-28 Nokia Technologies Oy Method and apparatus for providing user interfaces
USD705228S1 (en) 2012-02-23 2014-05-20 Marni Peters Actuating ring
US9235241B2 (en) 2012-07-29 2016-01-12 Qualcomm Incorporated Anatomical gestures detection system using radio signals
US8743052B1 (en) 2012-11-24 2014-06-03 Eric Jeffrey Keller Computing interface system
WO2014144015A2 (en) 2013-03-15 2014-09-18 Keller Eric Jeffrey Computing interface system
CN103777752A (en) * 2013-11-02 2014-05-07 上海威璞电子科技有限公司 Gesture recognition device based on arm muscle current detection and motion sensor
US9965761B2 (en) 2014-01-07 2018-05-08 Nod, Inc. Methods and apparatus for providing secure identification, payment processing and/or signing using a gesture-based input device
KR101933289B1 (en) 2014-04-01 2018-12-27 애플 인크. Devices and methods for a ring computing device
KR20160015050A (en) 2014-07-30 2016-02-12 엘지전자 주식회사 Mobile terminal
2015-05-18 CN CN201580024018.8A patent/CN106462239B/en active IP Right Grant
2015-05-18 KR KR1020167036032A patent/KR20170008854A/en unknown
US20050186994A1 (en) 2000-09-27 2005-08-25 Science Applications International Corporation Method and system for energy reclamation and reuse
US6861945B2 (en) 2002-08-19 2005-03-01 Samsung Electro-Mechanics Co., Ltd. Information input device, information processing device and information input method
US20140180851A1 (en) 2005-12-31 2014-06-26 Blaze Mobile, Inc. Using Harvested Power for NFC Transactions
US8879986B2 (en) 2005-12-31 2014-11-04 Michelle Fisher Wireless bidirectional communications between a mobile device and associated secure element using inaudible sound waves
US7702282B2 (en) 2006-07-13 2010-04-20 Sony Ericsoon Mobile Communications Ab Conveying commands to a mobile terminal through body actions
US8405612B2 (en) * 2007-10-12 2013-03-26 Immersion Corporation Method and apparatus for wearable remote interface device
US8106749B2 (en) 2008-07-14 2012-01-31 Sony Ericsson Mobile Communications Ab Touchless control of a control device
US8112066B2 (en) 2009-06-22 2012-02-07 Mourad Ben Ayed System for NFC authentication based on BLUETOOTH proximity
US8526880B2 (en) 2010-02-26 2013-09-03 Blackberry Limited Communications device responsive to near field communications (NFC) to enter locked or unlocked operating modes
US8500031B2 (en) 2010-07-29 2013-08-06 Bank Of America Corporation Wearable article having point of sale payment functionality
US8820649B2 (en) 2012-11-20 2014-09-02 Omne Mobile Payments, Inc. Electronic card with a programmable magnetic stripe
US9360946B2 (en) 2014-05-08 2016-06-07 Microsoft Technology Licensing, Llc Hand-worn device for surface gesture input
"Lightglove", retrieved at <<http://lightglove.com/>> on Apr. 28, 2014, 1 page.
"Lightglove", retrieved at > on Apr. 28, 2014, 1 page.
"Logisys", retrieved at <<http://www.logisyscomputer.com/viewsku.asp?SKUID=MS601BK&DID=KEYBOARD>>, published on Jul. 3, 2004, 1 page.
"Logisys", retrieved at >, published on Jul. 3, 2004, 1 page.
"Ring: Shortcut Everything", retrieved at <<https://www.kickstarter.com/projects/1761670738/ring-shortcut-everything>> on Apr. 28, 2014, 27 pages.
"Ring: Shortcut Everything", retrieved at > on Apr. 28, 2014, 27 pages.
"Wearfin", retrieved at <<http://wearfin.com>>on Apr. 28, 2014, 9 pages.
"Wearfin", retrieved at >on Apr. 28, 2014, 9 pages.
Agrawal et al., "Using Mobile Phones to Write in Air," MobiSys '11, Jun. 28-Jul. 1, 2011, Bethesda, Maryland, 14 pages.
Ahmad et al., "UbiHand: A Wearable Input Device for 3D Interaction", ACM SIGGRAPH 2006 Research posters, Jul. 30-Aug. 3, 2006, 1 page.
Akay, A., "Acoustics of friction," The Journal of the Acoustical Society of America, 111 (4), Apr. 2002, pp. 1525-1548, 24 pages.
Amendent/Response filed Sep. 22, 2014 to Non-Final Office Action mailed Jun. 24, 2014 from U.S. Appl. No. 13/847,900, 11 pages.
Amendment filed Jun. 14, 2016 to the Notice of Allowance mailed Mar. 22, 2016 from U.S. Appl. No. 14/489,385, 7 pages.
Amendment/Response filed Jan. 28, 2015 to Final Office Action mailed Oct. 29, 2014 from U.S. Appl. No. 13/847,900, 12 pages.
Amendment/Response filed May 11, 2015 to Non-Final Office Action mailed Feb. 18, 2015 from U.S. Appl. No. 13/847,900, 10 pages.
Amft, et al., "Sensing Muscle Activities with Body-Worn Sensors", In Proceedings of the International Workshop on Wearable and Implantable Body Sensor Networks, Apr. 3, 2006, 4 pages.
Amma et al., "Airwriting: Hands-free Mobile Text Input by Spotting and Continuous Recognition of 3d-Space Handwriting with Inertial Sensors", 16th International Symposium on Wearable Computers, Jun. 18, 2012, 8 pages.
Article 34 Demand mailed Nov. 10, 2015 from PCT Patent Application PCT/US2015/028682, 10 pages.
Chan et al., "FingerPad: Private and Subtle Interaction Using Fingertips", Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, Oct. 8-11, 2013, pp. 255-260, 6 pages.
Chang et al., "LIBSVM: A Library for Support Vector Machines," ACM Transactions on Intelligent Systems and Technology, vol. 2, No. 3, Apr. 2011, 27 pages.
Chang, Janie, "Intuitive Uls Featured During UIST 2012", Oct. 8, 2012, retrieved at <<http://research.microsoft.com/en-us/news/features/uist2012-100812.aspx>>on Apr. 28, 2014, 4 pages.
Chang, Janie, "Intuitive Uls Featured During UIST 2012", Oct. 8, 2012, retrieved at >on Apr. 28, 2014, 4 pages.
Chen et al., "uTrack: 3D Input Using Two Magnetic Sensors", Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, Oct. 8-11, 2013, pp. 237-244, 8 pages.
Communication pursuant to Rule 161(1) and 162 EPC mailed Dec. 15, 2016 from European Patent Application No. 15722634.1, 2 pages.
Cypress Semiconductor, "CY3290-TMA400EVK TrueTouch Gen4 Evaluation Test Kit," retrieved at <<http://www.cyrpress.com//documentation/development-kitsboards/cy2390-tma400evk-truetouch-gen4-evalutation-test-kit>>on Sep. 11, 2015, 2 pp.
Dunlop et al., "The challenge of Mobile Devices for Human Computer Interaction," Personal Ubiquitous Computing, Jan. 2002, pp. 6:235-236, 2 pages.
Eronen et al., "User Interfaces for Digital Television: A navigator Case Study," AVI 2000,ACM, pp. 276-279, 4 pages.
Final Office Action mailed Dec. 22, 2016 from U.S. Appl. No. 13/847,900, 15 pages.
Final Office Action mailed Jun. 19, 2015 from U.S. Appl. No. 13/847,900, 13 pages.
Final Office action mailed Mar. 11, 2016 from U.S. Appl. No. 13/847,900, 19 pages.
Final Office Action mailed Nov. 5, 2015 from U.S. Appl. No. 14/489,385, 16 pages.
Final Office Action mailed Oct. 29, 2014 from U.S. Appl. No. 13/847,900, 13 pages.
Genius Kye Systems Group, "Wireless Thumb Cursor Controller," retrieved on Sep. 11, 2015 from <<http://www.genuisnet.com/wSite/ct?xltem=51880&ctNode=3619&mp=1, 1 page.
Goel et al., "Surfacelink: Using Inertial and Acoustic Sensing to Enable Multi-Device Interaction on a Surface," Chi 14', Apr. 26-May 1, 2014, Toronto, Ontario, Canada, 10 pages.
Gummeson et al., "An Energy Harvesting Wearable Ring Platform for Gesture Input on Surfaces," MobiSys '14, Jun. 16-19, 2014, Bretton Woods, New Hampshire, 14 pages.
Gummeson et al., "EnGarde: Protecting the Mobile Phone from Malicious NFC Interactions," MobiSys '13, Jun. 25-28, 2013, Taipei, Taiwan, 14 pages.
Gustafson et al., "Imaginary Interfaces: Spatial Interaction with Empty Hands and without Visual Feedback", Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, Oct. 3-6, 2010, pp. 3-12, 10 pages.
Harrison et al., "Skinput: Appropriating the Body as an Input Surface," Proceedings of the ACM International Conference of Human-Computer Interaction, Apr. 10, 2010, 10 pages.
Harrison, et al., "Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces", In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology, Oct. 19, 2008, pp. 205-208.
Harrison, et al., "Tapsense: Enhancing Finger Interaction on Touch Surfaces", In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, Oct. 16, 2011, 8 pages.
Howard et al., "Lightglove: Wrist-Worn Virtual Typing and Pointing", Proceedings of the 5th IEEE International Symposium on Wearable Computers, Oct. 8-9, 2001, pp. 172-173, 2 pages.
International Preliminary Report on Patentability mailed Aug. 1, 2016 from PCT Patent Application No. PCT/US2015/031290, 8 pages.
International Preliminary Report on Patentability mailed Aug. 17, 2016 from PCT Patent Application No. PCT/US2015/028682, 8 pages.
International Preliminary Report on Patentability mailed Oct. 1, 2015 from PCT Patent Application No. PCT/US2014/031145, 7 pages.
International Preliminary Report on Patentability mailed Sep. 22, 2015 from PCT Patent Application PCT/US2014/031145, 7 pages.
International Search Report & Written Opinion mailed Jun. 25, 2014 from PCT Patent Application No. PCT/US/2014/031145, 9 pages.
International Search Report and Written Opinion mailed Aug. 4, 2015 from PCT Patent Application No. PCT/US2015/031290, 10 pages.
International Search Report and Written Opinion mailed Jul. 28, 2015 from PCT Patent Application No. PCT/US2015/028682, 12 pages.
International Search Report and Written Opinion mailed Oct. 26, 2015 from PCT Application No. PCT/US2015/049272, 13 pages.
Kienzle et al., "LightRing: Always-Available 2D Input on Any Surface", Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, Oct. 5-8, 2014, pp. 157-160, 4 pages.
Kim et al., "Digits: Freehand 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor", Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, Oct. 7-10, 2012, pp. 167-176, 10 pages.
Kye Systems Corp., "Ring mouse," retrieved Jan. 29, 2013 from <<http://www.geniusnet.com/wSite/public/Mmo/4/Ring-Mousei.jpg, 1 page.
Lam et al., "MIDS: Micro Input Devices System Using MEMS Sensors", IEEE Conference on Intelligent Robots and Systems, Oct. 2002, pp. 1184-1189, 6 pages.
Liu et al., "uWave: Accelerometer-based personalized gesture recognition and its applications," Pervasive and Mobile Computing 5, 2009, pp. 657-675, 19 pages.
Lopes, et al., "Augmenting Touch Interaction through Acoustic Sensing", In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, Nov. 13, 2011.
Lukowicz, et al., "Detecting and Interpreting Muscle Activity with Wearable Force Sensors", In Pervasive Computing Lecture Notes in Computer Science vol. 3968, May 7, 2006, 4 pages.
Mackenzie, Scott, "Fitts' Law as a Research and Design Tool in Human-Computer Interaction", Human-Computer Interaction, Mar. 1992, vol. 7, No. 1, pp. 91-139, 49 pages.
Morris et al., "Emerging Input Technologies for Always-Available Mobile Interaction", Foundations and Trends in Human-Computer Interaction, Apr. 2011, vol. 4, No. 4, pp. 245-316, 74 pages.
Murray-Smith et al., "Stane: Synthesized Surfaces for Tactile Input," Chi 2008, Apr. 5-10, 2008, Florence, Italy, 4 pages.
Nanayakkara et al., "EyeRing: A Finger-worn Assistant", Proceedings of Extended Abstracts on Human Factors in Computing Systems, CHI'12, May 5-10, 2012, 6 pages.
Nilkura et al., "Anywhere Surface Touch: Utilizing any surface as an input area", Proceedings of the 5th Augmented Human International Conference, AH'14, Mar. 7-9, 2014, pp. 1-8, 8 pages.
Non-Final Office Action mailed Feb. 18, 2015 from U.S. Appl. No. 13/847,900, 13 pages.
Non-Final Office Action mailed Jun. 17, 2015 from U.S. Appl. No. 14/489,385, 15 pages.
Non-Final Office Action mailed Nov. 18, 2016 from U.S. Appl. No. 15/287,500, 11 pages.
Non-Final Office action mailed Oct. 5, 2015 from U.S Appl. No. 13/847,900, 13 pages.
Notice of Allowability mailed Oct. 24, 2016 from U.S. Appl. No. 14/489,385, 38 pages.
Notice of Allowance mailed Feb. 16, 2016 from U.S. Appl. No. 14/987,526, 8 pages.
Notice of Allowance mailed Jul. 6, 2016 from US. Appl. No. 14/489,385, 8 pages.
Notice of Allowance mailed Mar. 22, 2016 from U.S. Appl. No. 14/489,385, 12 pages.
Notice of Allowance mailed Sep. 9, 2015 from U.S. Appl. No. 14/273,238, 8 pages.
Ogata et al., "iRing: Intelligent Ring Using Infrared Reflection," UIST '12, Oct. 7-10, 2012, Cambridge, MA, 6 pages.
Ogris, et al., "Using FSR based Muscle Activity Monitoring to Recognize Manipulative Arm", In Proceedings of the 11th IEEE International Symposium on Wearable Computers, Jan. 2007, 4 pages.
Pedley, Mark, "Tilt Sensing Using a Three-Axis Accelerometer," Freescale Semiconductor, Doc. No. AN3461, Rev. 6, Mar. 2013, 22 pages.
Response and PCT Demand filed Nov. 18, 2015 to International Search Report and Written Opinion mailed Aug. 4, 2015 from PCT Patent Application No. PCT/US2015/031290, 8 pages.
Response filed Feb. 22, 2016 to the Final Office Action mailed Nov. 5, 2015 from U.S. Patent Appl. No. 14/489,385, 11 pages.
Response filed Jan. 5, 2016 to the Non-Final Office action mailed Oct. 5, 2015 from U.S. Appl. No. 13/847,900, 10 pages.
Response filed May 17, 2016 to the Final Office Action mailed Mar. 11, 2016 from U.S. Appl. No. 13/847,900, 11 pages.
Response filed May 17, 2016 to the Final Office Action mailed Mar. 11, 2016 to U.S. Appl. No. 13/847,900, 11 pages.
Response filed Sep. 14, 2015 to the Non-Final Office Action mailed Jun. 17, 2015 from U.S Appl. No. 14/489,385, 10 pages.
Response filed Sep. 16, 2015 to Final Office Action mailed Jun. 19, 2015 from U.S. Appl. No. 13/847,900, 13 pages.
Response filed Sep. 25, 2016 to the Non-Final Office Action mailed Jun. 24, 2016 from U.S Patent Appl. No. 13/847,900, 12 pages.
Saponas et al., "PocketTouch: Through-Fabric Capacitive Touch Input", Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, Oct. 14, 2011, 1 page.
Schlomer et al., "Gesture Recognition with a Wii Controller," TEI '08, Feb. 18-20, 2008, Bonn, Germany, pp. 11-14, 4 pages.
Second Written Opinion mailed Apr. 28, 2016 from PCT Application No. PCT/US2015/028682, 7 pages.
Shepard et al., "LiveLab: Measuring Wireless Networks and Smartphone Users in the Field," ACM SIGMETRICS Performance Evaluation Review, 2011, 38(3), pp. 15-20, 6 pages.
Shilkrot et al., "FingerReader: A Wearable Device to Support Text Reading on the Go", Proceedings of Conference on Human Factors in Computing Systems, Apr. 26-May 1, 2014, 6 pages.
Sturman et al., "A Survey of Glove-based Input", IEEE Computer Graphics and Applications, Jan. 1994, vol. 14, No. 1, pp. 30-39, 10 pages.
Texas Instruments, "HF Antenna Design Notes," Technical Report, SCBA034, Sep. 2003, 51 pages.
U.S. Appl. No. 13/847,900, filed Mar. 20, 2013 Priyantha et al.
U.S. Appl. No. 14/273,238, filed May 8, 2014 Priyantha et al.
Wolf et al., "Tickle: A Surface-independent Interaction Technique for Grasp Interfaces", Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, TEI 2013, Feb. 10-13, 2013, 8 pages.
Yang, et al., "Magic Finger: Always-Available Input through Finger Instrumentation", In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, Oct. 7, 2012, pp. 147-156.
Zhang et al., "A Ring-shaped Interactive Device for Large Remote Display and Mobile Device Control," Proceedings of the 13th International Conference on Ubiquitous Computing (UbiComp '11), Sep. 17, 2011, Beijing, China, 2 pages.
Zhang, et al., "A Ring-Shaped Interactive Device for Large Remote Display and Mobile Device Control", In Proceedings of the 13th International Conference on Ubiquitous Computing, Jul. 17, 2011, pp. 473-474.
US10101887B2 (en) 2018-10-16 Device, method, and graphical user interface for navigating user interface hierarchies
EP3170062B1 (en) 2019-08-21 Raise gesture detection in a device
US10162512B2 (en) 2018-12-25 Mobile terminal and method for detecting a gesture to control functions
US10126826B2 (en) 2018-11-13 System and method for interaction with digital devices
US9081571B2 (en) 2015-07-14 Gesture detection management for an electronic device
US20130271350A1 (en) 2013-10-17 Multi-segment wearable accessory
US9535506B2 (en) 2017-01-03 Efficient gesture processing
EP3108351B1 (en) 2019-05-08 Activity continuation between electronic devices
JP6509486B2 (en) 2019-05-08 Wearable electronic device
US9791957B2 (en) 2017-10-17 Dynamic input at a touch-based interface based on pressure
CN100445937C (en) 2008-12-24 Handwriting path identifying system and method
US10139915B1 (en) 2018-11-27 Gesture-based small device input
AU2013260682B2 (en) 2019-05-16 Transition and interaction model for wearable electronic device
AU2013260681B2 (en) 2019-08-15 Controlling remote electronic device with wearable electronic device
US9405379B2 (en) 2016-08-02 Classification of user input
KR20140064688A (en) 2014-05-28 Placement of optical sensor on wearable electronic device
US9229489B2 (en) 2016-01-05 Adjusting mobile device state based on user intentions and/or identity
KR20140064693A (en) 2014-05-28 User gesture input to wearable electronic device involving movement of device
EP2733598B1 (en) 2019-04-03 GUI transitions on wearable electronic device
US9880620B2 (en) 2018-01-30 Smart ring
KR20140064694A (en) 2014-05-28 User gesture input to wearable electronic device involving outward-facing sensor of device
CN103793075B (en) 2017-02-15 Recognition method applied to intelligent wrist watch
US10423214B2 (en) 2019-09-24 Delegating processing from wearable electronic device
KR20160128378A (en) 2016-11-07 Text input on an interactive display
Feldman et al. 2005 ReachMedia: On-the-move interaction with everyday objects
Owner name: MICROSOFT CORPORATION, UNITED STATES
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PRIYANTHA, NISSANKA ARACHCHIGE BODHI;LIU, JIE;SIGNING DATES FROM 20140507 TO 20140522;REEL/FRAME:032960/0096


text:
Q: DataTable and ASP.NET Repeater I am binding a DataTable to a repeater. However when I run my aspx page it is not evaluating the Eval statements and displaying them on the page as
<<%# Eval("Name").ToString() %> Posted by: <<%# Eval("UserName").ToString() %>
<<%# Eval("Name").ToString() %> Posted by: <<%# Eval("UserName").ToString() %>
<<%# Eval("Name").ToString() %> Posted by: <<%# Eval("UserName").ToString() %>

It is also breaking it down to 3 rows. Which is also not what I had expected. I thought it would display as spans normally would
I have taken out the .ToString to check if it was that causing the issue. But it was not.
Below is my Repeater
<asp:Repeater ID="First" runat="server">
        <ItemTemplate>

                    <asp:Label ID="Name" runat="server" Text='<<%# Eval("Name").ToString() %>'></asp:Label>

                    <asp:Label ID="UserName" runat="server" Text='<<%# Eval("UserName").ToString() %>'></asp:Label>

        </ItemTemplate>
    </asp:Repeater>

The code to bind is the following:
public void dataLoad()
{
    DataTable Data = loadData(1,1,1);

    if (Data.Rows.Count < 10)
    {
        // Only populate the data from datatable
        First.DataSource = Data;
        First.DataBind();
        Second.Visible = false;
        noData.Visible = true;
    }

I have changed some of the names in the above example
In the Debug information I can see that my DataTable is using the correct column names, and that these match my eval statements.
I know I have created this in a rather strange way, from what I have read, and researched online it is more common to use a table rather than Divs/or Spans for this. So I am aware that that may be the problem at its heart.
I cannot see why it would be any different.

A: Remove the << from your eval statements. It should be:

        <asp:Label ID="Name" runat="server" Text='<%# Eval("Name").ToString() %>'></asp:Label>

        <asp:Label ID="UserName" runat="server" Text='<%# Eval("UserName").ToString() %>'></asp:Label>

redpajama_set_name: RedPajamaStackExchange


This is a ritual that can be used to provide clarity in your life. It allows the divine to communicate directly to us through vivid dreams. This ritual can be used to find discernment in specific situations or to simply have and recall vivid dreams. In this ritual we will be completely surrendering to the will of the divine and letting go of what we think...to discover the absolute reality of what is, in this now.
Begin by praying. Asking for the presence, protection and guidance of your Ancestors, Angels, The Orishas, The Neteru and The Divine LOVING spirit of The Most Eternal Source of all Creation. Making these acknowledgements are like a fail safe if things ever get out of hand. These beings that serve the highest good will be that for you the entire way through.
Start by smudging yourself and all ingredients with sage and/or Palo Santo.
If you are not at peace then sit in stillness until you are. Breathe in LOVE while exhaling what does not serve you until peace sets in. Do not enter ritual with a flustered mind or wavering heart.
Begin by preparing a Mugwort Tea. Pour 3 tablespoons of dried Mugwort into 1.5 cup of spring or distilled water. Add heat. Take an infinite breathe, visualizing and inhaling a tube of white light from the heavens and the core of mother earth into your heart. Send this energy into the water through your hands and begin to stir.
While stirring, call on the whale totem animal asking to discover the depths and mysteries of what is hidden beneath the shallow surface of your situation of focus. Let cool. Drink one half cup.
Prepare a Mojo bag filled with 1 tablespoon of Mugwort and the following crystals. | Prehnite , Moldavite, Amethyst, Tourmaline, Tiger's Eye, and Celestite | Inhale an infinite breathe and bless the mojo bag with sight beyond sight. Place it under the pillow of where you will sleep over night or inside the pillow case. Drink second half cup of tea.
Burn a few sour sop leaves in a fire resistant bowl. Do not evade the smoke, but instead inhale it as regular breath. Do not woft the smoke. Let it happen organically. Allow the leaves to burn while praying to Olokun of the ocean, asking him to reveal the deep mysteries of the situation clearly in your dreams. Drink third half cup of tea.
Prepare a pencil and paper next to your bed. Lay down to sleep. Take slow long breaths in and out until you fall asleep.
When you wake, record what you recall of your dreams with as much detail as possible. Read it back out loud and reflect on how you feel about it in your heart. Ask your Ancestors, Angels, the Orishas, the Neteru and The Divine LOVING spirit of the most Eternal Source of all Creation for clear understanding of the dream you just had.
Then conclude with some automatic writing. Allow the words to come and the pencil to go with as much gusto that arises. Stop when you are at peace in your heart.
Return to center using the infinite breathe. Visualize a golden egg surrounding your body. This egg is filled with the divine healing energy of the universe. Inhale this energy into your lungs and body. Exhale what any stresses or lingering thoughts. Continue this until you are at peace. Then inhale LOVE and exhale GRATITUDE until your vibration raises to your hearts content.


Last weekend I was kindly invited down to the Vurger Co. in Shoreditch to give their all-vegan menu a whirl. As a head's up, I'm not actually a vegan myself, I'm a pescatarian but I would absolutely love to go full veggie one day. The idea of vegan food has kind of a reputation of being bland and no match for the original, and I've definitely experienced this before in restaurants, so I was intrigued to see how the Vurger Co. took on fast food while still keeping it yum.