By way of general introduction, touch screens implementing tactile sensor arrays have recently received tremendous attention with the addition multi-touch sensing, metaphors, and gestures. After an initial commercial appearance in the products of FingerWorks™, such advanced touch screen technologies have received great commercial success from their defining role in the iPhone™ and subsequent adaptations in PDAs and other types of cell phones and hand-held devices. Despite this popular notoriety and the many associated patent filings, tactile array sensors implemented as transparent touchscreens were taught in the 1999 filings of issued U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. No. 11/761,978.
Despite the many popular touch interfaces and gestures, there remains a wide range of additional control capabilities that can yet be provided by further enhanced user interface technologies. A number of enhanced touch user interface features are described in U.S. Pat. Nos. 6,570,078 and 8,169,414 as well as, pending U.S. patent application Ser. Nos. 11/761,978, 12/418,605, 12/502,230, 12/541,948, and a significant number of related pending U.S. patent applications by the present and associated inventors. These patents and patent applications also address popular contemporary gesture and touch features. The enhanced user interface features taught in these patents and patent applications, together with popular contemporary gesture and touch features, can be rendered by the “High Dimensional Touch Pad” (HDTP) technology taught in those patents and patent applications. Implementations of the HTDP provide advanced multi-touch capabilities far more sophisticated that those popularized by FingerWorks™, Apple™, NYU, Microsoft™, Gesturetek™, and others.
Example Devices and Configurations Employing a Touchpad or Touchscreen
FIGS. 1a-1g (adapted from U.S. patent application Ser. No. 12/418,605) and 2a-2e (adapted from U.S. Pat. No. 7,557,797) depict a number of arrangements and embodiments employing a touch-based user interface. FIG. 1a illustrates a touch-based user interface as a peripheral that can be used with a desktop computer (shown) or laptop) not shown). FIG. 1 b depicts a touch-based user interface integrated into a laptop in place of the traditional touchpad pointing device. In FIGS. 1a-1b a touch-based user interface tactile sensor can be a stand-alone component or can be integrated over a display so as to form a touchscreen. FIG. 1 c depicts a touch-based user interface integrated into a desktop computer display so as to form a touchscreen. FIG. 1d shows a touch-based user interface integrated into a laptop computer display so as to form a touchscreen.
FIG. 1e depicts a touch-based user interface integrated into a cell phone, smartphone, PDA, or other hand-held consumer device. FIG. 1f shows a touch-based user interface integrated into a test instrument, portable service-tracking device, portable service-entry device, field instrument, or other hand-held industrial device. In FIGS. 1e-1f a touch-based user interface tactile sensor can be a stand-alone component or can be integrated over a display so as to form a touchscreen. FIG. 1g depicts a user interface touchscreen configuration that can be used in a tablet computer, wall-mount computer monitor, digital television, video conferencing screen, kiosk, etc. In at least the arrangements of FIGS. 1a, 1c, 1d, and 1g, or other sufficiently large tactile sensor implementation of a touch-based user interface, more than one hand can be used an individually recognized as such.
FIGS. 2a-2e and FIGS. 3a-3b (these adapted from U.S. Pat. No. 7,557,797) depict various integrations of a touch-based user interface into the back of a conventional computer mouse. Any of these arrangements can employ a connecting cable, or the device can be wireless.
In the integrations depicted in FIGS. 2a-2d a touch-based user interface tactile sensor can be a stand-alone component or can be integrated over a display so as to form a touchscreen. Such configurations have very recently become popularized by the product release of Apple™ “Magic Mouse™” although such combinations of a mouse with a tactile sensor array on its back responsive to multitouch and gestures were taught earlier in pending U.S. patent application Ser. No. 12/619,678 (priority date Feb. 12, 2004) entitled “User Interface Mouse with Touchpad Responsive to Gestures and Multi-Touch.”
In another embodiment taught in the specification of issued U.S. Pat. No. 7,557,797 and associated pending continuation applications more than two touchpads can be included in the advance mouse embodiment, for example as suggested in the arrangement of FIG. 2e. As with the arrangements of FIGS. 2a-2d, one or more of the plurality of touch-based user interface tactile sensors or exposed sensor areas of arrangements such as that of FIG. 2e can be integrated over a display so as to form a touchscreen. Other advance mouse arrangements include the integrated trackball/touchpad/mouse combinations of FIGS. 3a-3b taught in U.S. Pat. No. 7,557,797.
Overview of Touch-Based User Interface Sensor Technology
The information in this section provides an overview of HDTP user interface technology as described in U.S. Pat. Nos. 6,570,078 and 8,169,414 as well as pending U.S. patent application Ser. Nos. 11/761,978, 12/418,605, 12/502,230, 12/541,948, and related pending U.S. patent applications.
As an example, a touchpad used as a pointing and data entry device can comprise an array of sensors. The array of sensors is used to create a tactile image of a type associated with the type of sensor and method of contact by the human hand. The individual sensors in the sensor array can be pressure sensors and a direct pressure-sensing tactile image is generated by the sensor array. Alternatively, the individual sensors in the sensor array can be proximity sensors and a direct proximity tactile image is generated by the sensor array. Since the contacting surfaces of the finger or hand tissue contacting a surface typically increasingly deforms as pressure is applied, the sensor array comprised of proximity sensors also provides an indirect pressure-sensing tactile image. Alternatively, the individual sensors in the sensor array can be optical sensors. In one variation of this, an optical image is generated and an indirect proximity tactile image is generated by the sensor array. In another variation, the optical image can be observed through a transparent or translucent rigid material and, as the contacting surfaces of the finger or hand tissue contacting a surface typically increasingly deforms as pressure is applied, the optical sensor array also provides an indirect pressure-sensing tactile image.
Further, the array of sensors can be transparent or translucent and can be provided with an underlying visual display element such as an alphanumeric, graphics, or image display. The underlying visual display can comprise, for example, an LED array display, a backlit LCD, etc. Such an underlying display can be used to render geometric boundaries or labels for soft-key functionality implemented with the tactile sensor array, to display status information, etc. Tactile array sensors implemented as transparent touchscreens are taught in the 1999 filings of issued U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. No. 11/761,978.
In some implementations, the touchpad or touchscreen can comprise a tactile sensor array obtains or provides individual measurements in every enabled cell in the sensor array that provides these as numerical values. The numerical values can be communicated in a numerical data array, as a sequential data stream, or in other ways. When regarded as a numerical data array with row and column ordering that can be associated with the geometric layout of the individual cells of the sensor array, the numerical data array can be regarded as representing a tactile image. The only tactile sensor array requirement to obtain the full functionality of a touch-based user interface is that the tactile sensor array produce a multi-level gradient measurement image as a finger, part of hand, or other pliable object varies is proximity in the immediate area of the sensor surface.
Such a tactile sensor array should not be confused with the “null/contact” touchpad which, in normal operation, acts as a pair of orthogonally responsive potentiometers. These “null/contact” touchpads do not produce pressure images, proximity images, or other image data but rather, in normal operation, two voltages linearly corresponding to the location of a left-right edge and forward-back edge of a single area of contact. Such “null/contact” touchpads, which are universally found in existing laptop computers, are discussed and differentiated from tactile sensor arrays in issued U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. No. 11/761,978. Before leaving this topic, it is pointed out that these the “null/contact” touchpads nonetheless can be inexpensively adapted with simple analog electronics to provide at least primitive multi-touch capabilities as taught in issued U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. No. 11/761,978 (pre-grant publication U.S. 2007/0229477 and therein, paragraphs [0022]-[0029], for example).
More specifically, FIG. 4 (adapted from U.S. patent application Ser. No. 12/418,605) illustrates the side view of a finger 401 lightly touching the surface 402 of a tactile sensor array. In this example, the finger 401 contacts the tactile sensor surface in a relatively small area 403. In this situation, on either side the finger curves away from the region of contact 403, where the non-contacting yet proximate portions of the finger grow increasingly far 404a, 405a, 404b, 405b from the surface of the sensor 402. These variations in physical proximity of portions of the finger with respect to the sensor surface should cause each sensor element in the tactile proximity sensor array to provide a corresponding proximity measurement varying responsively to the proximity, separation distance, etc. The tactile proximity sensor array advantageously comprises enough spatial resolution to provide a plurality of sensors within the area occupied by the finger (for example, the area comprising width 406). In this case, as the finger is pressed down, the region of contact 403 grows as the more and more of the pliable surface of the finger conforms to the tactile sensor array surface 402, and the distances 404a, 405a, 404b, 405b contract. If the finger is tilted, for example by rolling in the user viewpoint counterclockwise (which in the depicted end-of-finger viewpoint clockwise 407a) the separation distances on one side of the finger 404a, 405a will contract while the separation distances on one side of the finger 404b, 405b will lengthen. Similarly if the finger is tilted, for example by rolling in the user viewpoint clockwise (which in the depicted end-of-finger viewpoint counterclockwise 407b) the separation distances on the side of the finger 404b, 405b will contract while the separation distances on the side of the finger 404a, 405a will lengthen.
In many various implementations, the tactile sensor array can be connected to interface hardware that sends numerical data responsive to tactile information captured by the tactile sensor array to a processor. In various implementations, this processor will process the data captured by the tactile sensor array and transform it various ways, for example into a collection of simplified data, or into a sequence of tactile image “frames” (this sequence akin to a video stream), or into highly refined information responsive to the position and movement of one or more fingers and other parts of the hand.
As to further detail of the latter example, a “frame” can refer to a 2-dimensional list, number of rows by number of columns, of tactile measurement value of every pixel in a tactile sensor array at a given instance. The time interval between one frame and the next one depends on the frame rate of the system and the number of frames in a unit time (usually frames per second). However, these features are and are not firmly required. For example, in some implementations a tactile sensor array can not be structured as a 2-dimensional array but rather as row-aggregate and column-aggregate measurements (for example row sums and columns sums as in the tactile sensor of year 2003-2006 Apple™ Powerbooks™, row and column interference measurement data as can be provided by a surface acoustic wave or optical transmission modulation sensor as discussed later in the context of FIG. 13, etc.). Additionally, the frame rate can be adaptively-variable rather than fixed, or the frame can be segregated into a plurality regions each of which are scanned in parallel or conditionally (as taught in U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. No. 12/418,605), etc.
FIG. 5a (adapted from U.S. patent application Ser. No. 12/418,605) depicts a graphical representation of a tactile image produced by contact with the bottom surface of the most outward section (between the end of the finger and the most nearby joint) of a human finger on a tactile sensor array. In this example tactile array, there are 24 rows and 24 columns; other realizations can have fewer, more, or significantly more (hundreds or thousands) of rows and columns. Tactile measurement values of each cell are indicated by the numbers and shading in each cell. Darker cells represent cells with higher tactile measurement values. Similarly, FIG. 5b (also adapted from U.S. patent application Ser. No. 12/418,605) provides a graphical representation of a tactile image produced by contact with multiple human fingers on a tactile sensor array. In other implementations, there can be a larger or smaller number of pixels for a given images size, resulting in varying resolution. Additionally, there can be larger or smaller area with respect to the image size resulting in a greater or lesser potential measurement area for the region of contact to be located in or move about.
FIG. 6 (adapted from U.S. patent application Ser. No. 12/418,605) depicts a realization wherein a tactile sensor array is provided with real-time or near-real-time data acquisition capabilities. The captured data reflects spatially distributed tactile measurements (such as pressure, proximity, etc.). The tactile sensory array and data acquisition stage provides this real-time or near-real-time tactile measurement data to a specialized image processing arrangement for the production of parameters, rates of change of those parameters, and symbols responsive to aspects of the hand's relationship with the tactile or other type of sensor array. In some applications, these measurements can be used directly. In other situations, the real-time or near-real-time derived parameters can be directed to mathematical mappings (such as scaling, offset, and nonlinear warpings) in real-time or near-real-time into real-time or near-real-time application-specific parameters or other representations useful for applications. In some implementations, general purpose outputs can be assigned to variables defined or expected by the application.
The tactile sensor array employed by touch-based user interface technologies can be implemented by a wide variety of means, for example:                Pressure sensor arrays (implemented by for example—although not limited to—one or more of resistive, capacitive, piezo, optical, acoustic, or other sensing elements);        Pressure sensor arrays (implemented by for example—although not limited to—one or more of resistive, capacitive, piezo, optical, acoustic, or other sensing elements);        Proximity sensor arrays (implemented by for example—although not limited to—one or more of capacitive, optical, acoustic, or other sensing elements);        Surface-contact sensor arrays (implemented by for example—although not limited to—one or more of resistive, capacitive, piezo, optical, acoustic, or other sensing elements).        
Below a few specific examples of the above are provided by way of illustration; however these are by no means limiting. The examples include:                Pressure sensor arrays comprising arrays of isolated sensors (FIG. 7);        Capacitive proximity sensors (FIG. 8);        Multiplexed LED optical reflective proximity sensors (FIG. 9);        Video camera optical reflective sensing (as taught in U.S. Pat. No. 6,570,078 and U.S. patent application Ser. Nos. 10/683,915 and 11/761,978):                    direct image of hand (FIG. 10a-10c);            image of deformation of material (FIG. 11);                        Surface contract refraction/absorption (FIG. 12).        
An example implementation of a tactile sensor array is a pressure sensor array. Pressure sensor arrays discussed in U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. No. 11/761,978. FIG. 7 depicts a pressure sensor array arrangement comprising a rectangular array of isolated individual two-terminal pressure sensor elements. Such two-terminal pressure sensor elements typically operate by measuring changes in electrical (resistive, capacitive) or optical properties of an elastic material as the material is compressed. In typical implementation, each sensor element in the sensor array can be individually accessed via multiplexing arrangement, for example as shown in FIG. 7, although other arrangements are possible and provided for by the invention. Examples of prominent manufacturers and suppliers of pressure sensor arrays include Tekscan™, Inc. (307 West First Street, South Boston, Mass., 02127, www.tekscan.com), Pressure Profile Systems™ (5757 Century Boulevard, Suite 600, Los Angeles, Calif. 90045, www.pressureprofile.com), Sensor Products™, Inc. (300 Madison Avenue, Madison, N.J. 07940 USA, www.sensorprod.com), and Xsensor™ Technology Corporation (Suite 111, 319-2nd Ave SW, Calgary, Alberta T2P 005, Canada, www.xsensor.com).
The capacitive touch sensors described above involve a capacitance change due to spatial compression of capacitive elements; there is no direct RF or electrostatic sensing of the finger itself, and the result is typically pressure sensing. Most capacitive touch sensors, however, do involve direct RF or electrostatic sensing of the finger itself, typically resulting in proximity sensing. It is also possible to create capacitive sensor arrays responsive to both proximity and pressure, for example such as the capacitive sensor arrays taught in U.S. Pat. No. 6,323,846 by Westerman.
Capacitive proximity sensors can be used in various handheld devices with touch interfaces (see for example, among many, http://electronics.howstuffworks.com/iphone2.htm, http://www.veritasetvisus.com/VVTP-12,%20Walker.pdf). Prominent manufacturers and suppliers of such sensors, both in the form of opaque touchpads and transparent touch screens, include Balda AG (Bergkirchener Str. 228, 32549 Bad Oeynhausen, DE, www.balda.de), Cypress™ (198 Champion Ct., San Jose, Calif. 95134, www.cypress.com), and Synaptics™ (2381 Bering Dr., San Jose, Calif. 95131, www.synaptics.com). In such sensors, the region of finger contact is detected by variations in localized capacitance resulting from capacitive proximity effects induced by an overlapping or otherwise nearly-adjacent finger. More specifically, the electrical field at the intersection of orthogonally-aligned conductive buses is influenced by the vertical distance or gap between the surface of the sensor array and the skin surface of the finger. Such capacitive proximity sensor technology is low-cost, reliable, long-life, stable, and can readily be made transparent. FIG. 8 (adapted from http://www.veritasetvisus.com/VVTP-12,%20Walker.pdf with slightly more functional detail added) shows a popularly accepted view of a typical cell phone or PDA capacitive proximity sensor implementation. Capacitive sensor arrays of this type can be highly susceptible to noise and various shielding and noise-suppression electronics and systems techniques can need to be employed for adequate stability, reliability, and performance in various electric field and electromagnetically-noisy environments. In some implementations of a touch-based user interface, the present invention can use the same spatial resolution as current capacitive proximity touchscreen sensor arrays. In other implementations, a higher spatial resolution is advantageous.
Forrest M. Mims is credited as showing that an LED can be used as a light detector as well as a light emitter. Recently, light-emitting diodes have been used as a tactile proximity sensor array (for example, as taught in U.S. Pat. No. 7,598,949 by Han and depicted in the associated video available at http://cs.nyu.edu/˜jhan/ledtouch/index.html). Such tactile proximity array implementations typically need to be operated in a darkened environment (as seen in the video in the above web link). In one implementation, each LED in an array of LEDs can be used as a photodetector as well as a light emitter, although a single LED can either transmit or receive information at one time. Each LED in the array can sequentially be selected to be set to be in receiving mode while others adjacent to it are placed in light emitting mode. A particular LED in receiving mode can pick up reflected light from the finger, provided by said neighboring illuminating-mode LEDs. FIG. 9 depicts an implementation. The invention provides for additional systems and methods for not requiring darkness in the user environment in order to operate the LED array as a tactile proximity sensor. In one implementation, potential interference from ambient light in the surrounding user environment can be limited by using an opaque pliable or elastically deformable surface covering the LED array that is appropriately reflective (directionally, amorphously, etc. as can be advantageous in a particular design) on the side facing the LED array. Such a system and method can be readily implemented in a wide variety of ways as is clear to one skilled in the art. In another implementation, potential interference from ambient light in the surrounding user environment can be limited by employing amplitude, phase, or pulse width modulated circuitry or software to control the underlying light emission and receiving process. For example, in an implementation the LED array can be configured to emit modulated light modulated at a particular carrier frequency or variational waveform and respond to only modulated light signal components extracted from the received light signals comprising that same carrier frequency or variational waveform. Such a system and method can be readily implemented in a wide variety of ways as is clear to one skilled in the art.
An important special case of this is the use of OLED arrays such as those used in OLED displays increasingly deployed in cellphones, smartphones, and Personal Digital Assistants (“PDAs”) manufactured by Samsung, Nokia, LG, HTC, Phillips, Sony and others. As taught in pending U.S. patent application Ser. Nos. 13/452,461, 13/180,345 and 13/547,024, such an arrangement can be implemented in a number of ways to provide a high-resolution optical tactile sensor for touch-based user interfaces. Color OLED array displays are of particular interest, in general and as pertaining to the present invention, because:                They can be fabricated (along with associated electrical wiring conductors) via printed electronics on a wide variety of surfaces such as glass, Mylar, plastics, paper, etc.;        Leveraging some such surface materials, they can be readily bent, printed on curved surfaces, etc.;        They can be transparent (and be interconnected with transparent conductors);        Leveraging such transparency, they can be:                    Stacked vertically,            Used as an overlay element atop an LCD or other display,            Used as an underlay element between an LCD and its associated backlight.As taught in U.S. Pat. No. 8,125,559 and pending U.S. patent application Ser. Nos. 13/452,461, 13/180,345 and 13/547,024—leveraging this in various ways, in accordance with implementations, array of inorganic-LEDs, OLEDs, or related optoelectronic devices is configured to perform functions of two or more of:                        a visual image display (graphics, image, video, GUI, etc.),        a (lensless imaging) camera (as taught in U.S. Pat. Nos. 8,284,290 and 8,305,480,        a tactile user interface (touch screen),        a proximate gesture user interface.As taught in pending U.S. patent application Ser. Nos. 13/452,461, 13/180,345 and 13/547,024, such arrangements further advantageously allow for a common processor to be used for both a display and a touch-based user interface. Further, the now widely-popular RF capacitive matrix arrangements used in contemporary multi-touch touchscreen is fully replaced with an arrangement involving far fewer electronic components.        
Another type of optical tactile sensor approach arranged to serve as both a display and a tactile sensor is taught in U.S. Pat. No. 8,049,739 by Wu et al., which uses a deformable back-lit LCD display comprising internally reflective elements and photosensitive elements associated with the LCD display responsive to the reflective light.
Use of video cameras for gathering control information from the human hand in various ways is discussed in U.S. Pat. No. 6,570,078 and Pending U.S. patent application Ser. No. 10/683,915. Here the camera image array is employed as a touch-based user interface tactile sensor array. Images of the human hand as captured by video cameras can be used as an enhanced multiple-parameter interface responsive to hand positions and gestures, for example as taught in U.S. patent application Ser. No. 10/683,915 Pre-Grant-Publication 2004/0118268 (paragraphs [314], [321]-[332], [411], [653], both stand-alone and in view of [325], as well as [241]-[263]). FIGS. 10a and 10b depict single camera implementations. As taught in section 2.1.7.2 of U.S. Pat. No. 6,570,078, pending U.S. patent application Ser. No. 10/683,915, U.S. patent application Ser. No. 13/706,214, two or more video cameras can be used in orthogonal or stereoscopic arrangements to capture hand expressions within 3-space regions. FIG. 10c depicts a two camera implementation. As taught in the aforementioned references, a wide range of relative camera sizes and positions with respect to the hand are provided for, considerably generalizing the arrangements shown in FIG. 10a-10c. 
In another video camera tactile controller implementation, a flat or curved transparent or translucent surface or panel can be used as sensor surface. When a finger is placed on the transparent or translucent surface or panel, light applied to the opposite side of the surface or panel reflects light in a distinctly different manner than in other regions where there is no finger or other tactile contact. The image captured by an associated video camera will provide gradient information responsive to the contact and proximity of the finger with respect to the surface of the translucent panel. For example, the parts of the finger that are in contact with the surface will provide the greatest degree of reflection while parts of the finger that curve away from the surface of the sensor provide less reflection of the light. Gradients of the reflected light captured by the video camera can be arranged to produce a gradient image that appears similar to the multilevel quantized image captured by a pressure sensor. By comparing changes in gradient, changes in the position of the finger and pressure applied by the finger can be detected. FIG. 11 depicts an implementation.
FIGS. 12a-12b depict an implementation of an arrangement comprising a video camera capturing the image of a deformable material whose image varies according to applied pressure. In the example of FIG. 12a, the deformable material serving as a touch interface surface can be such that its intrinsic optical properties change in response to deformations, for example by changing color, index of refraction, degree of reflectivity, etc. In another approach, the deformable material can be such that exogenous optic phenomena are modulated in response to the deformation. As an example, the arrangement of FIG. 12b is such that the opposite side of the deformable material serving as a touch interface surface comprises deformable bumps which flatten out against the rigid surface of a transparent or translucent surface or panel. The diameter of the image as seen from the opposite side of the transparent or translucent surface or panel increases as the localized pressure from the region of hand contact increases. Such an approach was created by Professor Richard M. White at U.C. Berkeley in the 1980's.
FIG. 13 depicts an optical or acoustic diffraction or absorption arrangement that can be used for contact or pressure sensing of tactile contact. Such a system can employ, for example light or acoustic waves. In this class of methods and systems, contact with or pressure applied onto the touch surface causes disturbances (diffraction, absorption, reflection, etc.) that can be sensed in various ways. The light or acoustic waves can travel within a medium comprised by or in mechanical communication with the touch surface. A slight variation of this is where surface acoustic waves travel along the surface of, or interface with, a medium comprised by or in mechanical communication with the touch surface.
Compensation for Non-Ideal Behavior of Tactile Sensor Arrays
Individual sensor elements in a tactile sensor array produce measurements that vary sensor-by-sensor when presented with the same stimulus. Inherent statistical averaging of the algorithmic mathematics can damp out much of this, but for small image sizes (for example, as rendered by a small finger or light contact), as well as in cases where there are extremely large variances in sensor element behavior from sensor to sensor, the invention provides for each sensor to be individually calibrated in implementations where that can be advantageous. Sensor-by-sensor measurement value scaling, offset, and nonlinear warpings can be invoked for all or selected sensor elements during data acquisition scans. Similarly, the invention provides for individual noisy or defective sensors can be tagged for omission during data acquisition scans.
FIG. 14 shows a finger image wherein rather than a smooth gradient in pressure or proximity values there is radical variation due to non-uniformities in offset and scaling terms among the sensors.
FIG. 15 shows a sensor-by-sensor compensation arrangement for such a situation. A structured measurement process applies a series of known mechanical stimulus values (for example uniform applied pressure, uniform simulated proximity, etc.) to the tactile sensor array and measurements are made for each sensor. Each measurement data point for each sensor is compared to what the sensor should read and a piecewise-linear correction is computed. In an implementation, the coefficients of a piecewise-linear correction operation for each sensor element are stored in a file. As the raw data stream is acquired from the tactile sensor array, sensor-by-sensor the corresponding piecewise-linear correction coefficients are obtained from the file and used to invoke a piecewise-linear correction operation for each sensor measurement. The value resulting from this time-multiplexed series of piecewise-linear correction operations forms an outgoing “compensated” measurement data stream. Such an arrangement is employed, for example, as part of the aforementioned Tekscan™ resistive pressure sensor array products.
Additionally, the macroscopic arrangement of sensor elements can introduce nonlinear spatial warping effects. As an example, various manufacturer implementations of capacitive proximity sensor arrays and associated interface electronics are known to comprise often dramatic nonlinear spatial warping effects. FIG. 16 (adapted from http://labs.moto.com/diy-touchscreen-analysis/) depicts the comparative performance of a group of contemporary handheld devices wherein straight lines were entered using the surface of the respective touchscreens. A common drawing program was used on each device, with widely-varying type and degrees of nonlinear spatial warping effects clearly resulting. For simple gestures such as selections, finger-flicks, drags, spreads, etc., such nonlinear spatial warping effects introduce little consequence. For more precision applications, such nonlinear spatial warping effects introduce unacceptable performance. Close study of FIG. 16 shows different types of responses to tactile stimulus in the direct neighborhood of the relatively widely-spaced capacitive sensing nodes versus tactile stimulus in the boundary regions between capacitive sensing nodes. Increasing the number of capacitive sensing nodes per unit area can reduce this, as can adjustments to the geometry of the capacitive sensing node conductors. In many cases improved performance can be obtained by introducing or more carefully implementing interpolation mathematics.
Overview of 3D, 6D, and Related Capabilities of HDTP Technology User Interface Technology
Some implementations of HDTP technology is provided. This will be followed by a summarizing overview of HDTP technology. With the exception of a few minor variations and examples, the material presented in this overview section is draw from U.S. Pat. No. 6,570,078, pending U.S. patent application Ser. Nos. 11/761,978, 12/418,605, 12/502,230, 12/541,948, 12/724,413, 13/026,248, and related pending U.S. patent applications and is accordingly attributed to the associated inventors.
FIGS. 17a-17f (adapted from U.S. patent application Ser. No. 12/418,605 and described in U.S. Pat. No. 6,570,078) illustrate six independently adjustable degrees of freedom of touch from a single finger that can be simultaneously measured by the HDTP technology. The depiction in these figures is from the side of the touchpad. FIGS. 17a-17c show actions of positional change (amounting to applied pressure in the case of FIG. 17c) while FIGS. 17d-17f show actions of angular change. Each of these can be used to control a user interface parameter, allowing the touch of a single fingertip to control up to six simultaneously-adjustable quantities in an interactive user interface as shown in FIG. 18.
Each of the six parameters listed above can be obtained from operations on a collection of sums involving the geometric location and tactile measurement value of each tactile measurement sensor. Of the six parameters, the left-right geometric center, forward-back geometric center, and clockwise-counterclockwise yaw rotation can be obtained from binary threshold image data. The average downward pressure, roll, and pitch parameters are in some implementations beneficially calculated from gradient (multi-level) image data. One remark is that because binary threshold image data is sufficient for the left-right geometric center, forward-back geometric center, and clockwise-counterclockwise yaw rotation parameters, these also can be discerned for flat regions of rigid non-pliable objects, and thus the HDTP technology thus can be adapted to discern these three parameters from flat regions with striations or indentations of rigid non-pliable objects.
Additionally, as taught in U.S. Pat. No. 6,570,078 and pending U.S. patent application Ser. Nos. 11/761,978 and 12/418,605, a wide range of richly-parameterized multi-touch configurations are supported by the HDTP technology. FIG. 19 depicts example multi-touch positions and gestures involving two fingers that are supported by the HDTP technology, and FIG. 20 depicts various individual and compound images associated with touch by various portions of the human hand whose recognition and classification are supported by the HDTP technology.