extfieldsofstudy:
Computer Science
Medicine

text:
A Frequency Modulation-Based Taxel Array: A Bio-Inspired Architecture for Large-Scale Artificial Skin

This work introduces an array prototype based on a Frequency Modulation (FM) encoding architecture to transfer multiple sensor signals on a single wire. The use case presented adopts Hall-effect sensors as an example to represent a much larger range of sensor types (e.g., proximity and temperature). This work aims to contribute to large area artificial skin systems which are a key element to enhance robotic platforms. Artificial skin will allow robotic platforms to have spatial awareness which will make interaction with objects and users safe. The FM-based architecture has been developed to address limitations in large-scale artificial skin scalability. Scalability issues include power requirements; number of wires needed; as well as frequency, density, and sensitivity bottlenecks. In this work, eight sensor signals are simultaneously acquired, transferred on a single wire and decoded in real-time. The overall taxel array current consumption is 36 mA. The work experimentally validates and demonstrates that different input signals can be effectively transferred using this approach minimizing wiring and power consumption of the taxel array. Four different tests using single as well as multiple stimuli are presented. Observations on performances, noise, and taxel array behaviour are reported. The results show that the taxel array is reliable and effective in detecting the applied stimuli.

Introduction
The importance and transformative impact that spatial awareness in robotic platforms would have been well known since the concept of human like robots came to be. This critical need is recognized and well described in a work of two decades ago [1]. Lumelshy et al. acknowledges the fundamental need for robots to possess bodies covered with a sensitive layer as a means to be more efficient, spatially aware, and allow safe interaction with inanimate objects and people. The only solution envisaged, at the time, was to constrain the robot in a structured environment.
This design paradigm is still valid today for the majority of robotic and artificial platforms. With the exclusion of small robots deemed low-risk of causing harm due to low mass or speed, other platforms require specific training and/or a controlled environment. Therefore, when robotic platforms are designed to act or interact in the human space performance is severely limited resulting in reduced capabilities to make the interactions safe or safer.
More recently, the emergence and growth of care robotics, advancements in prosthetics limbs and associated research [2,3] renewed attention on this topic due to the need for closing the distance between platforms and users. New materials and advancements in tactile systems contributed to the flourishing of new studies, areas of interests, and applications, for example, the rising need for robots to take over complex and risky [4,5] tasks in hazardous environments to safeguard human lives. Tasks in radioactive or corrosive environments require spatial awareness and sensory feedback to be performed effectively.
Artificial touch has various challenges which include the area covered, number of sensors, and processing power. Even in localized solutions such as with touch pads, compromises have to be made. Studies on localized fingertip and tactile pads are the most common focusing on improving and providing fine manipulation and/or dexterity to robotic hands. Many examples of commercialized products can be found, and new ones are continuously developed and investigated.
Within the artificial skin field, studies can either focus on sensory systems to be applied on human skin or a sensitive coverage to envelop robots and platforms. This work and reviewed literature focuses on the latter which can be further subdivided into two classes: one aims to provide interactive interfaces to enhance human engagement by using soft stuffed "animals" and platforms [6][7][8] for haptic, interaction, and social studies; the other, more relevant work, focuses on providing sensory coverage to platforms. The coverage architectures can be either mainly centralized or mainly decentralized. The first is ideal for a limited number of sensors; while it easily reaches connection limits, it allows for direct and easy access to the sensor values. The decentralized approach delegates some computation to distributed electronics and collects the data into digital buses, which increases the potential number of sensors but is limited by the bandwidth of the bus selected. Despite the approach used, the ideal goal for large-scale sensory solutions is to create a system that can be part of a robotic platform. If the platform is stationary, power and processing can be externally provided through, for example, a tether. However, to be part of an autonomous platform, a system needs to be able to exploit the local resources (battery power, on-board processing units) or additional resources need to be added. If the resources are available or it is possible to add them, the system is feasible. However, depending on the demand, this can limit the speed or battery life. In this context, artificial skin needs to keep power consumption and processing to a minimum, while allowing for a large number of sensors. These are key characteristics for allowing deployment on autonomous platforms enhancing spatial awareness. This is highly challenging, and each approach mentioned is designed around priorities and compromises. For these reasons, while examples of systems exist, their number is limited due to intrinsic constraints and limitations due to scalability, and applications on experimental platforms can be found within laboratory settings.
The work presented here has characteristics of both approaches. It lays at the intersection of various disciplines, including electronics, bio-inspiration, communication, and signal processing. Therefore, the background and context will be presented from different points of view. This work has two main contributions to studies of large-scale artificial skin: first, a real-life use case scenario with evaluation of low-frequency stimuli and real-time sampling and reconstruction, and second, the work showcases a completely new and different approach from current mainstream solutions and represents a unique design for internal robotic communication systems applied to sensory arrays.
The taxel, used in this work, exploits the Frequency Modulation (FM) technique to transfer the sensor signals (constituting the taxel array) over a single wire with extremely low power demand. The architecture working principle is described in the materials and methods section. A collection of alternative architectures and solutions are presented in the background section to provide a reference point.
The taxel array unit used in this work aims to showcase a single use case scenario and evaluate the performance of both the taxel array and architecture. In this example, eight Hall-effect sensors are used. One of the key features of the architecture is that any other sensor that outputs a voltage signal could alternatively be used. Other types of sensors can be made compatible after appropriate signal conditioning to generate proportional voltage variations.
The FM technique is not new, and it has been used for decades in radio and wireless communication background. However, the architecture shows intriguing and unique characteristics by repurposing this wireless methodology into a wired unidirectional robotic internal sensory bus/network. The taxel system has a layered structure inspired by natural skin. These two characteristics are inspired by natural tactile sensory systems.
As visible in Figure 1, both the skin and the prototype used are built in distinct layers. From top to bottom, there is a protective layer, sensors, encoding and communication, a mixer, and transmission to a central system. The design takes its inspiration from the neuromorphic approach of biological tactile systems, which integrates sensing, filtering, and preprocessing. In the design, this translates to sensing, filtering, and encoding, while the preprocessing is a desirable feature, it has not been implemented due to the higher priority of power and footprint reduction. The paper is structured as follows. Section 1.1 presents a brief overview of various key state-of-the-art elements and topics relevant to this work. Section 2 presents the materials and methods used in this work. Section 3 presents the results and observations. The performance of the taxel array is discussed in Section 4. Last, Section 5 summarizes the conclusions of the study.

Background
Artificial skin-like systems have seen a growing interest in different fields (e.g., wearable, thin, soft artificial skin for human-oriented applications) thanks to novel materials and techniques. However, large-area skin-like systems for robot spatial awareness are fewer in number and mainly limited by the issues of scalability that make deployment and use very challenging.
According to the literature, unresolved technical open issues [10] remain related to scalability and deployment. Scalability has a series of related issues which include power, costs, signal reconstruction, design, and maintenance complexity. Deployment includes difficulties in creating and integrating a unique design of conformable and compliant coverage for different platforms.
The different subsections aim to contextualize the three main components and concepts that enable the presented taxel action: the FM technique, artificial tactile systems, and proposed architectures for large-scale skin deployment.

FM Modulation Technique
A 2016 work shows a FM modulation-demodulation approach used to acquire and transfer an ECG signal in real-time that could be easily connected and decoded using an audio PC card [11]. This work [11] uses relatively low frequencies (20 Hz to 20 kHz) and transfers/decodes a single signal.
FM radio (8 MHz to 108 MHz), on the other hand, is capable of transmitting high number of signals at high frequencies, over great distances. The FM radio technique, introduced many decades ago, is therefore reliable and well known, for example, it has good noise rejection. The radio receivers have been designed and perfected to single out a "station" or signal.
The architecture and transmission strategy used in this work falls between these two extremes. The main differences include the modulation of a large number of signals as well as the demodulation of all signals at once. Two characteristics add higher complexity with respect to the other two applications. The single-wire transmission approach reduces power demand and looming within the system. Last, the frequency range used is mediumlow (8 kHz to 1 MHz), which facilitates the development and design of electronics while avoiding some of the constraints and issues caused by operation at radio frequencies. The system also benefits from the deep understanding of potential interference and noise related to FM transmission for which techniques to avoid or fix them have been developed and proved valid in radio. Literature and resources on channel intermodulation interference are not limited to FM radio techniques but also include wireless and frequency modulation communication in general. These can be affected by noise caused by a variety of sources including components, number of transmitters, or harmonics [12][13][14][15]. Due to the nature of the application and signals this effect has limited impact on the presented work.

Artificial Touch
Tactile systems are many and exhaustive reviews can be found in the literature. An example is [16] in which Lucarotti et al. reviews synthetic and bio-artificial tactile sensory systems. In other examples [17][18][19][20][21] the topic is approached from different points of view. All these works showcase new technologies and sensory systems based on rigid, semi-rigid, and soft materials aimed to emulate natural systems.
In most cases of tactile systems, regardless of the sensor typology used, the sensors are strictly interlinked with the system architecture. This is needed because different sensors often require specific hardware to be properly acquired and most importantly sent to the processing unit. A recent example is Skin-On interfaces [22]. Teyssier et al. present a self contained soft touch patch pad and envisions primarily the use in common interactive devices such as PC, smartphones, and smartwatches but also suggests the use to interact with robots and other objects.

Large Scale Tactile Architectures
Existing artificial skin for robotic platforms is the last element required to fully contextualize this work. Two types of architectures and proposed solutions to skin scalability are considered. The first collates solutions that minimize the distributed electronics by delegating the computationally intensive tasks of discriminating and reconstructing the data to neural networks and learning algorithms. The second collates those solutions in which the distributed electronics have higher functions and complexity. These are used to acquire, process, and serialize on digital buses the sensory data to a central control unit.
Although both types are very different from the underpinning principles used, the first will be briefly outlined for completeness, and the second will be used as a point of reference because it is closer in terms of distributed architecture layout to the one proposed.

• Neural network-based systems • Distributed electronics and digital bus
Neural network-based systems aim to reduce power and looming via simplifying electronics as an artificial version of mechanoreceptors. The downside is that the signal is highly complex and the processing requires a higher level centralized approach normally based on neural network-based algorithms [23][24][25]. This approach, although valid, might create a situation where transparency or modifications might be complex. Within these systems the signals are acquired and mixed in a single conductor. The signals can be action potentials-like signals as in [24,25] or multiple resistive sources as in [23]. The resulting composite signal requires neural networks to decouple the various sources and identify specific patterns. Without the level of abstraction and computation of a neural network the original signals would be difficult, if not impossible, to retrieve.
Distributed electronics and digital bus are considered representative of the mainstream approach and three architectures are selected to present alternative solutions as a comparison: iCub artificial skin, Cut-and-paste tactile sensors, and Hex-o-skin.
In the iCub artificial skin [26][27][28][29][30], the skin comprises triangular units, with each unit housing 12 capacitive sensors. An ideal 192 taxels can be connected using the CAN bus protocol and subnetworks can be used to increase the number of sensors. The units have been used on the iCub platform to envelop arms, legs, and torso.
In the Cut-and-paste tactile sensors [31] the skin features decentralized microcontrollers. Scaling power consumption for this architecture is difficult and needs to be managed applying time-sharing control over the sensor elements (e.g., 50 A to supply 1000 tactile elements). The skin related to this architecture is sensitive to both contact and pressure through LED light variations. This architecture is built on a flexible PCB and it can be cut and tailored to fit a specific layout.
In the Hex-o-skin [32][33][34], single hexagonal cells have a dedicated microcontroller and several sensors (7) including temperature, acceleration and proximity. In this case, the power consumption can be high and power management strategies to mitigate the energy used by the LEDs is needed.
All these architectures use digital communication systems and buses. Microcontrollers are used as an intermediary between sensors and the processing unit to encode and transmit the information digitally.
In contrast, in this work a low-power integrated circuit (IC) is used to achieve analogue FM encoding on a single wire. The digitalization is deferred to the central processing unit. Table 1 presents a brief summary of the key criteria compared with the transmission strategy used in this work.  These solutions have different advantages and disadvantages, and the choice of using one solution or another is strictly related to the task and desired specifications. For example, CAN bus solutions or microcontroller solutions, in general, have advantages during development and configuration, e.g., many changes can be made via soft-or firmware level. Buses also allow for bidirectional communication which is lost in FM. However, apart for debug/development and configuration purposes the loss of downstream communication is both consistent with natural systems and largely compensated by reduced power re-quirements. The FM encoding can be configured but physical changes (e.g., resistors) are required making it less "flexible", however once the skin is tailored to a robot it should not need changes. Last, the CAN bus, taken as example, has physical limitation by design which limits the number of sensors and/or update rates. Other faster buses can be used but the electronics to handle such speeds requires higher power and therefore a trade-off needs to be reached. The FM architecture has limits, but the bands can be simply changed and adapted to the specific task. For these reasons, the FM architecture has potential advantages when a large number or variety of sensors need to be used.
Compared to the two aforementioned classes of approaches, the proposed solution sits in the middle bridging the hard choices between IC based distributed systems to collect and stream the data, and high complex decoding learning-based strategies which hold great relevance but sometimes can lead to reduced transparency, constraints and complexity if changes are needed.
The proposed solution sees a reduced complexity of the distributed system with competitive reduction of power demand by maintaining a relatively easy access to the information encoded in the FM signal which can be retried numerically without the need of neural network and learning providing in effect a direct parallel measurement of the signals. Of course higher level computation, e.g., neural networks can be applied to these signals if needed afterwards for high level control purposes but the data are also available to adopt other control strategies.

Materials and Methods
The setup is visible in Figure 2 and comprises 4 elements: the supply (5V) (Figure 2A), the prototype unit ( Figure 2B), the acquisition and processing system based on a Compact Rio (CRio) 9045 from National Instruments (NI) (Figure 2C), and the single wire that connects the taxel array to a C Series digitalizer module NI-9775 ( Figure 2D).
The experiments have been sampled at 250 kHz, but NI-9775 can sample up to 20 Mega samples per second (20 Msps). At the moment, the processing is executed on the CRio real-time embedded system. The system has an FPGA on board which is not currently used. This choice, although limiting the number of sensors that can be used, is favored to allow for higher flexibility and transparency in this phase of the investigation and assessment. In Figure 3A the active sensors are shown in the circle and numbered from 0 to 7. The sensors are also identified by their channel also indexed 0 to 7 (channel carrier frequencies are 8,11,14,17,20,23,26, and 29 kHz, respectively). A transparent polycarbonate (Figure 3Ba) screen simulates the skin protective layers ensuring that the magnets used to trigger the Hall-effect sensor response could move freely without damaging the sensors and maintain the same distance which allows for consistent and repeatable stimuli. Halleffect sensors are selected because taxels using this technology exist [35]. Although the taxel in this work is highly simplified it also allows both the testing of the architecture as well as testing more realistic scenarios in sensing magnets at varying distance through the observation of the signal from neighboring sensors. Evidence is presented in the result section, Section 3.4 (Magnetic gradient and sensors response). Each sensor is connected to an FM encoder board (Figure 3Bc) with its own carrier frequency. The carrier frequencies are spaced at 3 kHz intervals. Each channel has a bandwidth of 3 kHz comprised of ±1 kHz channel deviation +500 Hz spacing each side.
Any sensor that outputs (conditioned or native) a voltage signal can be used with this architecture. The different layers are visible in Figure 3B. Figure 3Bb is a connection board that allows the interface between the sensors and the encoding units, Figure 3Bc is the encoding unit, and Figure 3Bd is a second connection board that mixes all the channels into a single FM mixed stream and sends it via a single coaxial wire to the acquisition and processing unit. The mixing board also hosts a voltage reference source at 2.5 V for the level shift conditioning stage (see Figure 4). The boards and prototype have been configured for a signal of ±7.5 V to enable a wide range of tests. To achieve this the conditioning stage has a 2.5 V level shifter with a 1/3 gain amplifier configuration that scales the inputs voltages to the IC range (0-5 V). The boards are not configured or optimized for the specific sensors. For these Hall-effect sensors conditioning would not be needed (the output ranges from around 1 to 4 V). This does not affect the end result; it would just change the ratio between signal and frequency deviation. The prototype is configured for a more general multipurpose application. The Hall-effect sensors are tested in these conditions to prove that the system works regardless, highlighting its flexibility and robustness.  Figure 5 presents two examples of input signal and output reconstruction. The two channels selected are Ch0 and Ch5. Ch0 is taken as an example for all the channels except Ch5. Ch5 would have similar output but it is affected by periodic noise which will be discussed in the following. The offset is due to the slight difference between the carrier frequency (tuned using the trimmer on the board) and the frequency used within the demodulation stages. Table 2 reports the signal amplitude for both channels. The amplitude of the noise directly measured from the sensor (black lines) is~23 mV and the one from the demodulated signal is~70 mV for those behaving like Ch0, taken as an ideal example. These amplitude variations are quite small 0.8% or 2.3% over the 3 V sensor range. The comparison of Ch0 input and reconstructed signal show that the quality of the signal is not particularly affected by the FM modulation/demodulation.
In Figure 4 a block diagram of the system is presented. Each encoding board can be divided into three stages: conditioning, encoding, and filtering. The conditioning stage can be tuned to accommodate a variety of sensors. The IC used for FM encoding produces a modulated square-wave which could be used but it would generate too much interference and undesired harmonics. Therefore, a 3rd-order passive filter is used to smooth the output FM signal and obtain a FM sinusoidal output.
After mixing, the FM signal is sent to the acquisition system (CRio Real-time core) which uses a real-time engine to perform software-based filtering and demodulation of all 8 sensors simultaneously.  The bandpass filter extracts each channel band. The singled FM waveforms are processed by NI labview signal processing blocks (e.g., downconvert) which extracts the quadrature (I/Q) complex components. These blocks shift the singled FM centered to the specific channel to the base band. The complex signal is then fed into Equation (1) [36] which can extract the modulating signal (input signal). A simple decimation is used to reduce the output sample rate and memory usage: where f sig is the demodulated signal and SF is a constant scaling factor. In this work, −7.5 was used (to remap the ±7.5 V configured range). The negative sign is used to compensate for VCO Inverted logic. Re f m and Im f m are the real and imaginary part of the FM I/Q signal, respectively. The taxels' signals are then converted into a 2D 4 × 4 gradient map. The real-time calculated values are also logged for further analysis and replay of the experiment. The data log has been used to playback the taxel array activity. In order to generate a 2D 4 × 4 map of the taxel array a further 8 locations have been generated in real-time as the average of neighboring sensor signals ( Figure 3A * locations). The 2D reconstruction has been adopted in this work because it is easier to visualize. However replay of the data using a 3D surface is also possible.
The magnet was moved manually to simulate excitation events to evaluate the taxel array and architecture response.

Results
This section presents the results and observations of the taxel array tests.
Five groups of data are presented using data acquired in four tests. The following subsections present: • Taxel reconstruction-the output of the taxel array is overlaid with video recorded during the experiment. • Single sensor evaluation-the complete output of two experiments in which each signal or channel is presented. • Ch5 (signal at 23 kHz) noise-data collection for which a post filter on channel 5 is turned ON and OFF. • Magnetic gradient and sensors response-correlation between signal intensity and sensor distance from magnet. • FFT mixed FM signal analysis-two mixed FM signals, from two experiments, for which frequency components have been analyzed using a Fast Fourier Transform (FFT). The mixed FM spectrum is analyzed and the various frequency components clearly visible. Figure 6 shows two snapshot sequences. The two movement patterns, (A) circular and (B) diagonal, are correctly reproduced by the taxel array. A video was captured during the experiments and manually synchronized. In the figure, the taxel array is reconstructed, using MATLAB, from the data logged during the experiment.       Figure 10 presents the acquisition during a single experiment where a low pass filter was applied to the decoded channel 5 signal. Figure 10A highlights a section from 15 to 30 s without the filter and Figure 10B from 30 to 45 s where the filter was toggled ON and OFF. The filter effect on Channel 5 is clear and improves the output considerably. There are still small aberrations, as shown at the point Figure 10B-1 but the result is satisfactory. The noise is caused by harmonics induced by the lower frequency channels, which was expected. Encoding board refinements and optimizations of the output filter stage should reduce these effects eliminating the need for the post-filter; however, even in this early prototype stage the output is sufficiently clear to detect the stimuli changes.

Magnetic Gradient and Sensors Response
Another interesting observation is the capability of the system to correctly recognize magnetic gradient related to the distance of the magnet. Figure 11 presents a portion of Figure 10B and the sensors layout. Due to the sensor layout and simple use case scenario selected to assess the system performance, the magnet maintains a constant vertical distance between magnet and sensor, which saturates the sensor output of those directly underneath the magnet. However, the neighboring sensors show signals correlated to the distance and influence of the magnetic field. Figure 11. The image on the right shows the sensor layout. The plot shows a portion of the recorded signals during which the magnet is present and then removed. In the instant (vertical line) around second 37 the magnet is hovering over Ch5 and Ch0 (e.g., around the circled position) as shown by the maximum signal on these channels in the plot. The other neighboring sensors show decreasing signal correlated to the distance to the magnet and magnetic field intensity. Figures 12 and 13 show FFT spectra for two different experiments. These plots show that the various frequencies are clearly visible and the center frequency (black line) represents 0 V signal (it is absent due to sensor working output ranges). In Figure 12, the predominant components at 2.5 V and~4 V (which are the base signal and max positive Hall-effect values) lay as expected around −333 Hz and −500 Hz off the center frequency, respectively. In Figure 13, the predominant components are 2.5 V,~4 V as before but a third peak at −100 Hz off the center frequency identifies the~1 V output, resulting in a negative value when the magnet polarity is changed.

FFT Mixed FM Signal Analysis
These values can be linearly calculated as 1000 Hz deviation/7.5 V * voltage measured, which was expected. The negative sign is due to the IC inverted logic (higher the voltage lower the frequency). This result, although expected, shows the repeatability among the 8 channels. Figures 12B and 13B show the FFT spectra of the mixed FM signal in a linear and log scale, respectively. Each channel, shown in Figure 12B, has been singled, in individual plots, to highlight the bands and show that in Ch5 there is evidence of a Ch0 third harmonic and other unwanted peaks ( Figure 12B Ch5 (*)) which are causing the noise (~250-300 mV) on Ch5 as shown in Figure 5. These peaks are also present in Figure 13B but harder to spot due to the overlap with Ch5 valid frequencies.  (1)) shows the carrier frequency position. The two thinner red lines (2) identify the peaks of the frequency deviations at −333 and −500 Hz from the carrier frequency that correspond to the sensor outputs 2.5 and 4 V respectively. Figure 13 presents another FFT spectrum for the experiment in which the magnet was used with both polarities. Similarly to Figure 12B the channels are singled out and another peak is visible representing the inverted polarity.  (1)) shows the carrier frequency position. The three thinner red lines (2) identify the peaks of the frequency deviations at −100, −333, and −500 Hz from the carrier frequency that correspond to the sensor outputs 1, 2.5, and 4 V respectively.

Discussion
This work showcases a use case application of a taxel based on FM encoding capable of providing real-time data acquisition. The FM architecture allows for compact communication to a central processing unit over a single wire.
One of the key points of reducing wire number in artificial skin structures is that for large numbers of sensors wire management (e.g., number, connections, mechanical fatigue, and space usage) becomes paramount. This applies to (i) the encoding units' connection to the central control, which in the FM based architecture is one coaxial wire and current systems using digital protocols have 2 to 4 wires. Multiple subunits can be used and therefore the wiring of FM-based architectures scale at a slower pace; (ii) the connection of sensors to the encoding units and although linked to the overall design and task, minimization of footprint and connections are ideally the goal (e.g., sensor close or on the same board). Some solutions present instead several sensors connected to a single microcontroller via leads which is an acceptable solution that provides advantages such as reach and reduction of encoding elements. However, for large numbers these cannot scale as well due to encumbrance, power and costs. Scalability potential is therefore skewed towards FM in terms of power, wire reduction and reduced board footprint. Where the FM system falls short is that its benefits only become apparent in the context of a large number of sensors. Considering that this is the end goal of artificial skin (collecting information from many thousands of sensors), this was expected and accepted during design.
The five sets of results presented highlight the good quality of signal reconstruction, and the feasibility and robustness of the system.
The taxel array sequences (presented in Section 3.1) show how the experiments were performed. The reconstruction examples provided as support material relate to the sequence in Figure 6A,B. At the moment the 2D 4 × 4 taxel array is built using sensory data and a simple neighboring average. A larger array and more advanced techniques would enhance the resolution of detection.
Comparing all channels (Section 3.2), the data acquisition quality and effective simultaneous parallel processing is visible. In these plots, the signal to noise ratio of all channels can be appreciated and Ch5 is visibly affected by higher noise. This noise is expected and well documented in FM. For this application mitigations are not required, however, to prove the quality and effectiveness of the proposed system a post filter was used to clean the signal. The result (Section 3.3) of the filtered/unfiltered signal shows how a simple filter can improve the output. However, it is also evident that even without a post filter the profile and rate of change is clearly detectable. Initial observations of the signal noise ratio show that it is very low, with the exception of Ch5, but still sufficiently low to discriminate sensor activity. The same filter could be applied to all signals but it is not necessary. Possible sources contributing to noise on the reconstructed signals include (i) the analogue nature of the sensors which introduce noise and (ii) the computation noise/errors introduced during the FM demodulation. Other sources of noise can be excluded because, as previously mentioned, FM is inherently resistant to noise and removing Ch0 causes the noise on Ch5 to almost disappear.
The observation in Section 3.4 about the signal correlation between distance and Halleffect sensor output highlights the sensitivity and encoding accuracy of the system. The current X-Y sensitivity demonstrated would extend to Z-axis. Whilst no further processing has been performed, it is evident that a wider range of information is present within the signals including direction of movement and speed.
Finally, the inspection (Section 3.5) of the overall mixed FM frequency spectra is compatible with the expected theoretical shape. Calculated main frequencies are present and the sources of noise on Ch5 (23 kHz) are visible. In mission-critical designs, this channel would be not used, reserved for low priority sensors, or assigned to low-priority and low-frequency signals. Improvement and optimization of the encoding boards such as the filtering stage should reduce this noise further, e.g., increasing the filtering order on Ch0 and Ch1 which are the most likely source of noise.
The repeatability and robustness of the signals, the low power consumption proves that tactile systems based on FM are feasible.

Conclusions
This work introduces the first example of a taxel array based on the Frequency Modulation technique. The results show the reliability and effective capability of the architecture in this use case. The transmission and reconstruction of the signal increases the amplitude of the noise from~1% to~2% over the 3 V sensor range which is quite small and allows for clear identification of the input signals. This working prototype presents encouraging features to be able to address scalability in artificial skin systems.
The overall current consumption of the taxel array used in this study is 36 mA at 5 V. This suggests that a 30 × 30 matrix or 900 sensors would consume 4.05 A. With the current prototype size and sensor density of 2 sensors per cm 2 , this would cover an area of 30 × 30 centimeters. By lowering the density, larger areas could be covered with the same power. The current encoding board consumption is currently around 1.5 mA per channel. The sensors power demand (the Hall-effect sensors used requires~4 mA) in this and other architectures is of critical importance and needs to be as low as possible to maximize effectiveness and scalability of artificial skins. The current prototype and design version has a theoretical limit of 300 simultaneous channels per single wire, whilst maintaining the 3 kHz band spacing over the 1 MHz bandwidth. 3 kHz is the smallest frequency band possible to allow up to 100-200 Hz signal to be encoded and at the same time to retain low filter orders. A way to increase the number of sensors would be to increase the bandwidth of the IC used. Multiple subunits are envisioned to increase the number of sensors. The use of multiple subunits is a strategy to increase number of sensors accepted in literature.
The current consumption for each element is very competitive being significantly lower than two of the examples presented in the background. The flexibility to encode different sensors also makes it competitive with respect to the lower power solution, e.g., iCub skin. The reduction in wire connection number and simultaneous signal communication are important points of strength of the proposed solution in the context of scalability. A benchmark comparison between key features of the proposed architecture and existing example solutions is presented in Table 1 (Background subsection). The literature solutions use digital communication buses which require decentralized dedicated microcontrollers to digitalize and send the data. On the other hand, the proposed solution requires a more complex centralized high-speed and high-process power unit intermediary between the sensors and PC (currently CRio and FPGA or similar in the future). This unit is expected to handle the decoding of a large number of signals. The proposed solution's ratio between sensors and decoding systems is however expected to be advantageous because the computation will grow at a significant slower pace with respect to number of sensors. This implies that the average cost per sensor will still be competitive, even if more powerful computation is required with consequent higher power demand and higher costs per single unit.
The results obtained in this research will make possible the development of larger arrays and the development of a standalone solution to enable multiple sensors to be tested and processed.


s2fieldsofstudy:
Computer Science
Biology

text:
Deadeye: A Novel Preattentive Visualization Technique Based on Dichoptic Presentation

Preattentive visual features such as hue or flickering can effectively draw attention to an object of interest -- for instance, an important feature in a scientific visualization. These features appear to pop out and can be recognized by our visual system, independently from the number of distractors. Most cues do not take advantage of the fact that most humans have two eyes. In cases where binocular vision is applied, it is almost exclusively used to convey depth by exposing stereo pairs. We present Deadeye, a novel preattentive visualization technique based on presenting different stimuli to each eye. The target object is rendered for one eye only and is instantly detected by our visual system. In contrast to existing cues, Deadeye does not modify any visual properties of the target and, thus, is particularly suited for visualization applications. Our evaluation confirms that Deadeye is indeed perceived preattentively. We also explore a conjunction search based on our technique and show that, in contrast to 3D depth, the task cannot be processed in parallel.

INTRODUCTION
Designing comprehensive visualizations requires a deep understanding of how perception actually works. Therefore, the visual system is one of our most important tools for acquiring and parsing information that surrounds us. Making efficient use of certain visual characteristics helps us to create visualizations that excel in their usability and user performance.
In particular, drawing the attention of users to certain elements is a research subject that is continuously being worked on in various fields, including psychology, computer science, psychophysics, and biology. Researchers have discovered several visual cues that can capture and guide our attendance to the object(s) of interest. The most prominent example is probably the search function of a PDF viewer, web browser, or text editor: it uses color to highlight the occurrence of a query, which allows us to instantly locate the results. A more sophisticated example is a medical visualization that utilizes flickering to highlight suspicious cells or tissue and helps doctors with exploring the data. Cues such as color, flickering, shape, size, and motion are examples of so-called preattentive visual features. Such features make an object pop out and allow us to grasp the object's presence in usually less than 250 ms. One important property of preattentive cues is that they perform equally well with an increasing number of distractors -the search time for our visual system remains constant. In other words, those features are processed in parallel by our visual system and are not searched in a serial fashion. That property is crucial when we revise our example with a full-page text search or the exploration of a huge medical dataset.
This paper summarizes Deadeye, a novel preattentive visualization technique that is based on the dichoptic presentation phenomenon. We render two different images for the two eyes. The images have one difference. The object that should pop out is presented to only one eye. Surprisingly, previous research stated that binocular rivalry, an effect based on showing two different images, is not processed in parallel, with the exception of luster effects. Since then, those dichoptic techniques have lost popularity as possible popout effects. The target object is a red circle among blue distractors and can be recognized preattentively. (b) Conjunction of color and shape: The target object is either a blue square or a red circle. We have to search each object in a seral fashion to find the target. (c) Conjunction of stereo and color: The target object is either in the back plane and red or in the front plane and blue. Images redrawn from [20] and [37].
Our contribution is twofold. First, our finding opens the way for further research on that technique and encourages derived applications in visualization (cf. Figure 1). Second, Deadeye is the first preattentive technique that does not modify any visual properties of the target object. All existing cues have to alter the target in one way or another -be it reshaping, recoloring or introducing motion. These changes can result in data misinterpretation, usually a highly undesired side effect. Furthermore, especially in visualization, most properties have a certain purpose and meaning, and reserving a whole dimension such as color or position for the visual popout is an expensive tradeoff. On the contrary, Deadeye solves these issues by preserving all visual properties of the target.
The paper shows that Deadeye is indeed perceived preattentively by conducting a state-of-the-art study that is common for such cues. Our evaluation also underlines the general applicability of the technique, as not a single participant reported headache or other similar physical strains. Three additional explorative experiments illustrate the usage of our technique in different real-world visualization scenarios.
In addition, we examine the phenomenon of a conjunction search based on Deadeye. Although preattentive features are processed in parallel, a conjunction of multiple cues usually leads to a serial search (cf. Figure 2). In most cases, we have to inspect each object individually and check for its properties. Hence, the time needed for the task increases linearly with the number of distractors. However, there exist a few exceptions, one of which is 3D depth. Previous research shows that the depth cue can be combined with, e.g., color or motion, and still be processed in parallel. As our technique also makes use of the binocular visual system, one might assume that Deadeye also shares that property. To investigate that claim, we conduct a study by combining our technique with hue variations. The results clearly show a significant increase in time needed to accomplish the task and the overall accuracy, exposing the serial character of that task.

Preattentive Visual Features
The human visual system works in a rather dynamic way. Our eyes are able to perceive detailed information in a very limited area only. That area is determined by the fovea, the central part of our retina. The fovea provides high-resolution images but covers only between one and two degrees of vision. In order to gather all the data, our eyes perform rapid saccadic movements between the states of fixed steadiness, i.e., our eyes scan for interesting objects and determine the next location for gathering a high-resolution fragment. Humans are not aware of the process, since the interchange between saccades and fixations happens three to four times in a second. For a deeper insight into the basics of the human vision, please refer to the research by Yanbus [57,58], Noton et al. [38] and Itti et al. [24].
Early research on the interplay between eye movement and data processing has discovered several visual features that can be detected in a split second. That speed led to the assumption that these features can be detected preattentively. As pointed out by Healey et al. [20], the term preattentive is not entirely correct, since a brief period of focus is still required to perceive these cues. However, we will stick to that term as it is used throughout the literature.
Such preattentive visual features can be detected within one focus period before the saccadic eye movement is triggered. Since the saccade has an initiation time of 200-250 ms [20], researchers apply that time threshold to verify if a visual variable is preattentive. In general, experiments are conducted as follows: an image with a number of distractors and possibly a target object is exposed to the participants, as shown in Figure 2. The participants have to decide whether a target object was present. The image is shown for either less than 250 ms (e.g., [16]), or until the participants make the decision via a button press (e.g., [5]). In the first case, the error rate is considered as the primary indicator, whereas in the second case, one takes both the error rate and the reaction time into count. The experiments are conducted with varying set sizes in order to prove that the error rate and/or reaction time remain constant with an increasing number of distractor objects.
The work by Healey et al. [20] summarizes and provides references for the following 16 preattentive visual cues: orientation, length, closure, size, curvature, density, number, hue, luminance, intersections, terminators, 3D depth, flicker, direction of motion, velocity of motion, and lighting direction. A prominent example is hue: consider a red dot that pops out in a set of blue distractor dots. For instance, Nagy et al. [35] conducted experiments to show how differences in color between distractors and the target object affect our search efficiency. As expected, a rather small color difference leads to an increased search time and diminishes our ability to process that visual feature in a preattentive way.
A more recent summary suited for a broader audience is exposed by Wolfe et al. [56]. Amongst other things, the article includes a classification of discovered features based on their likeliness to perform as guidance attributes. In addition, the authors summarize how the prior history of an observer influences the guidance of attention.
The most related visual feature to our research is the luster effect discovered by Wolfe et al. [55]. Hereby, the target is rendered dimmer than the background for one eye and brighter than the background for the other eye. The result is often said to pop out because of a metallic sheen. The authors also experimented with other dichoptic presentation techniques and concluded that they cannot be perceived preattentively and that luster is an exception. Our paper partially supports these assumptions, as Deadeye also works preattentively.
Another binocularity-based and closely related visual feature is 3D depth. Although previous research suggested that preattentive detection is solely based on 2D image features, Enns et al. [12,13] found that our visual system can also access visual features related to the 3D scene information. The authors concluded that both the lighting direction and three-dimensionality can be detected preattentively.
Different theories and models aim to explain the underlying nature of preattentive processing. Among the prominent ones is the feature integration theory by Treisman [50]. A detailed description of such models is behind the scope of our paper. We point to the work by Healey et al. [20] for a detailed state-of-the-art summary.
The model decomposes the low-level visual system in feature maps for each specific visual cue. Each map is encoded in parallel, which leads to an almost instantaneous detection of the feature. Other examples of such models include the texton theory [27] and the Boolean map theory [22].
Clearly, each variable has its advantages and drawbacks. Certain features, such as shape or 3D depth, have to alter the underlying visualization and can lead to misinterpretation. Other features suffer in terms of accuracy when the target object is located in the peripheral area. A detailed study was conducted by Gutwin et al. [16] to determine how different variables behave with varying distance from the focus point. This is a rather critical attribute, as multi-monitor setups gain more and more popularity. For these use-cases, Gutwin et al. suggest applying motion or flickering, as these variables perform well even at large angles. In the process, the authors applied the NASA-TLX survey, establishing comparisons among several variables. Therefore, we decided to rely on the same questionnaire to compare our outcome to the well-established visual cues.

Application Areas
Preattentive visual cues play an important role in visualization (e.g., [53]) and other areas such as human-computer interaction (e.g., [29]). One prominent use-case is directing the attraction of users to a certain object of interest. Several emphasis methods have recently been summarized by Hall et al. [17] and methods for attention modeling in general can be found in the state-of-the-art report by Borji et al. [7] and in the summary paper by Healey et al. [20]. Not surprisingly, none of these summaries mention binocular disparities as a means for guiding attention or annotating objects.
In the following paragraphs, we present a brief selection of established attention-guidance techniques. Hoffmann et al. [21] utilized visual cues in order to highlight the active window in a multi-monitor setup. The authors evaluated five types of window frames and masks and finally suggested a combination of color and tapered trails to guide the users' attention. Cole et al. [10] introduced the Stylized Focus, a variation of shading effects in order to draw the viewers' attention to a specific 3D object. The Popout Prism presented by Suh et al. [45] is an overview+detail-based system that enhances the representation of documents. The authors used the two popout effects color and size in order to visually emphasize elements of interest and also reduce the cognitive load.
Alper et al. [2] utilized the stereoscopic depth cue for highlighting purposes in 2D and 3D graph visualizations by projecting objects of interest onto a plane closer to the user. Additionally, the authors introduced a juxtaposing mechanism to allow focus and context views. The authors also emphasized the benefits of using highlighing mechanisms that do not reserve important visual attributes such as motion or color.
Possible applications and design guidelines for rather dynamic popout techniques flicker, direction, and velocity were studied by Huber et al. [23]. The adoption of flickering for dynamic narrative visualizations, the so-called Attractive Flicker, was discussed in detail by Waldner et al. [52]. In a first stage, user attention is attracted by a short and intensive flickering of the target object. The engagement stage relies on a less disturbing luminance oscillation and helps to keep track of the target. In contrast to cues such as color and size, the flickering does not distort the scene elements. Our technique goes a step further, since we do not alter any visible property of the object of interest.

Dichoptic Presentation
The term dichoptic presentation is applied when each eye is exposed to different stimuli, i.e., two different images. One of the resulting phenomena is called binocular rivalry (e.g., [1,6,15,28,41]). Instead of a stable single image, our vision switches to a mode with alternating periods of monocular dominance. One explanation is that the monocular neurons compete in the primary visual cortex and lead to the mentioned rivalry with regard to the interpretation of the image. Interestingly, as shown by Logothetis et al. [28], the rivalry does not depend on the eye, i.e., the dominant eye does not behave differently. One reason is that, in most cases, we are not able to determine which eye has detected a certain distinct stimulus. The research by Baker [4] addresses that possible loss of information by utilizing EEG and multivariate pattern analysis. Baker concludes that such eye-of-origin knowledge is lost due to our perception and consciousness. We also evaluate our technique for both eyes separately to explore the interplay between factors such as eye dominance and the preattentive processing of Deadeye.
Wolfe et al. [55] claimed that binocular rivalry cannot be processed preattentively. The only exception is the luster effect that we mentioned previously. Zou et al. [62] revisited that research and concluded that though interocular differences can guide attention in some cases (e.g., luster), the effects are rather weak and overridden by stronger features such as orientation or luminance. However, we show that Deadeye is another valid exception and propose to reconsider binocular rivalry in terms of its preattentiveness. Similarly to our suggestion, Friedenberg [15] also pointed out that it is still not clear whether binocular rivalry falls under late-stage voluntary attentional control or is processed by preattentive mechanisms. Another evidence to reconsider rivalry for attention guidance can be found in the work of Paffen et al. [40]. The authors conducted a study with ten participants to see how transparent, monocular, and binocular changes in images are perceived. They concluded that change blindness is attenuated in cases where the change is monocular. The work by Zhaoping [61] also aligns with our findings by indicating that ocular discontinuities have the potential to automatically capture our attention. In her studies, Zhaoping compared ocular singletons to orientation singletons and focused on the role of our primary visual cortex during the creation of bottom-up saliency maps for attention guidance.
Dichoptic presentation was extensively studied regarding luminance variation, i.e., exposing an object with different brightness to the left and right eye. How we perceive such a phenomenon was described in the work by de Weert et al. [11] and Teller et al. [46]. Anstis et al. [3] explored luminance effects in a more diversified way by adding other presentation techniques such as flicker to their comparisons. The research by Formankiewicz et al. [14] also contributed to that field by examining the ways how such luminance disparities are detected and revealed similarities to the detection of surface properties.
The use of dichoptic presentation for creating novel visual experiences has gained little attention in science. Most closely related to our proposed method is the work by Zhang et al. [60]. The authors briefly discuss several ideas for "Unconventional Binocular Presentation", with highlighting as one of those proposed applications. Unfortunately, the short note does not evaluate any of those thoughts in more detail. Later, Zhang [59] revisited some of the ideas and focused on the luster effects.
Apart from rivalry, dichoptic presentation also takes place in the case of binocular disparity. The term is associated with our ability to extract depth information based on the fact that our eyes have a slight horizontal offset and perceive two slightly different images. For further details on the basics of our stereo vision, we refer to the work of Julesz et al. [25,26] and Marr et al. [30,31]. Caziot and Backus [8] performed a thorough study on the effects and parameters of stereoscopic offsets to improve object recognition. We, however, avoid spatial offsets as the position of a data point cannot be changed in general without changing its value, e.g., in plots or graphs. For an overview and classification of the numerous other uses of stereoscopic 3D, we refer the reader to the formalization paper by Schild et al. [44] and Schild's dissertation [43].
In contrast to those rather common depth perception approaches, the so-called da Vinci stereopsis [36] refers to cases where we are able to extract depth information based on monocular occlusions. We point the readers to the state-of-the-art report by Harris et al. [18] for an overview. In addition, the more recent findings by Tsirlin et al. [51] indicate that depth perception in such cases is most likely due to occlusion geometry. In three trials of our prestudy, we enhanced one or multiple lines using Deadeye and asked the subjects to name the corresponding data.

THE DEADEYE EFFECT
We make use of dichoptic presentation and render the target object for one eye only. The other eye sees the plain background at the target location. In contrast to most binocular rivalry experiments, we do not show different objects at the target location. We simply create one image with and one image without the object. Hence, we claim that the conflict produced by the monocular neurons can be easily resolved. Our high-level visual system does not have to decide between two different objects. Instead, we get an image that is not unusual in our daily life. Think of a distant object that you look at through a small hole -the visual system does not report any conflicts, although the object is seen with one eye only (the dominant one).
Despite that naturalness, we can recognize the object enhanced with Deadeye in a split second. The object is best described as "eye-catching" or "somehow wrong", although we can focus on it and perceive all details without trouble. We attribute that popping out to our stereo vision ability. Extracting depth information and fusing two slightly offset images happens nearly instantly. However, depth calculations for the target object result in an error, i.e., that a single scene element cannot be put in a depth relative to other objects and the visual system preattentively recognizes that something went wrong.
An additional explanation to be considered is self-preservation mechanisms: We instantly react to a visual stimulus that is placed right in front of our eye, e.g., by closing our lid and moving back from the possible danger. In such cases, the stimulus is too near and visible with one eye only. Hence, we encourage more in-depth research including brain monitoring via fMRI to determine the exact cause of the observed effect. Our contribution focuses in the first place on establishing Deadeye as a preattentive visualization technique.

Enhancing Visualizations With Deadeye
Though Deadeye can be applied to a broad variety of visualization purposes, we cover a few basic examples to motivate the research on that technique. One possible scenario is the visualization of highdimensional data via line charts where we want to attract the observers' attention to one or multiple lines of interest (cf. Figure 3).
Applying Deadeye offers a unique advantage that cannot be achieved with any of the existing popout cues: highlighting the line without "wasting" any visual dimensions such as color or motion. The technique can be regarded as an additional degree of freedom for visual encoding that is orthogonal to the existing methods. This is essential, as realworld data visualizations tend to utilize as many visual properties (color, stroke width, etc.) as available to cover all data dimensions. Deadeye-enhanced line charts do not have to reserve a visual attribute for highlighting and, thus, can include more data dimensions.
The same argumentation is also valid for other representations such as bar charts, pie charts, or spider diagrams. Note that applying the "default" stereo vision to make a line to stand out from the chart is often not an option for such cases, as depth alters the coordinates of the underlying data and might result in wrong interpretations.
Another mentionable area is comparative visualization. Hereby, one of the common methods is to expose side-by-side views of the data. Such comparison tasks can be quite difficult and are even used as a challenge in so-called spot-the-difference puzzles (cf. Figure 1).
Our research on Deadeye demonstrates that there could be a much more effective way: the two images can be jointly exposed to our left and right eye. Hereby, the Deadeye effect allows a rapid recognition and localization of missing or differing objects, be it for scientific or entertaining purposes.
Note that though Deadeye requires stereo equipment, the corresponding hardware became commodity in recent years. 3D glasses, stereo projectors, and 3D TVs are wide spread but rarely used in day-to-day visualization applications. Hence, exploiting the devices for information highlighting is an alternative way of "recycling" such barely used equipment.

Prestudies
Prior to our main study, we conducted three exploratory experiments regarding the general applicability of Deadeye in different visualization scenarios. Our goals were twofold. First, we verified that subjects are able to exactly locate the popout targets and not only perceive a certain visual mismatch. Second, we determined whether the presence of other cues, such as color, are limiting the applicability of our technique.
We executed the following three prestudies: naming the Deadeyeenhanced line in a line chart (seven subjects, three trials, task cf. Figure 3), counting popping out atoms in a 3D visualization (four subjects, two trials, task cf. Figure 4), and naming the eye-catching image elements in a spot-the-difference puzzle scenario (five subjects, one trial, task cf. Figure 1). We did not impose any time constraints and only recorded the correctness of the answers.
In all cases, participants were able to locate and report the targets without any side effects such as headache. Additionally, in the line chart case, we verified that subjects were able to read the y-axis values aloud for several data points to make sure that information extraction is not affected. We conclude that Deadeye-enhanced objects can indeed be spotted and followed in a variety of visualization scenarios. Moreover, the technique was not affected by the presence of different colors or in a 3D-context (atom counting). Note, however, that the exact interplay of the technique with other preattentive cues is subject of future research.

EVALUATION
We conducted a user study to examine whether Deadeye can be perceived preattentively. Our experiment design is based on existing best practices for determining such effects. The participants are exposed to series of images composed of distractors and possibly a target object as shown in Figure 5. The participants have to decide whether the target object is present or not. As described in the Related Work section, two Our experimental setup. We equipped participants with active shutter glasses and presented a series of images on a 3D TV. After each image, the participants decided whether there was a target object or not. The images contained a varying number of circles jittered on a 5x6 grid. The two screenshots are a stereo pair example from our largest set with 30 objects. Readers could try to stereo fuse the images to perceive the popout effect.
approaches exist. One either displays the images for a fixed amount of time (100-250 ms) and measures the error rate, or the image is shown until the participants make a decision. In that case, one also has to consider the reaction time. In our case, we applied the fixed time option and displayed the image for 250 ms.
To prove the preattentive nature of Deadeye, the setup has to be repeated with varying set sizes. The error rate must remain nearly constant, no matter how many distractor objects are presented. Therefore, we considered four sets with the following sizes: 4, 8, 16, and 30. Additionally, we evaluated the conjunction search property of Deadeye by combining our technique with color. Since the common 3D depth is a visual cue that can be combined in parallel with other variables, one might assume the same property for our technique.

Hypotheses
Our main hypothesis is H1 is that Deadeye is indeed a popout technique that can be perceived preattentively. Therefore, the error rate has to be sufficiently low and remain constant among the different set sizes. Most of the related works consider an error rate below 10% as optimal.
Readers might suppose that Deadeye feels uncomfortable for our visual system. Therefore, we formulate a second hypothesis H2 as follows: Deadeye does not lead to headache or other physical strain. Otherwise, the application possibilities of that visual cue would be rather limited.

Procedure and Applied Measures
The study took place in a virtual reality laboratory at our university. After informing participants about the study's procedure, we administered a first questionnaire to assess the general demographic data. In addition, we asked whether the participants have any visual impairment and conducted a hole-in-the-card-test for eye dominance (Dolman method, e.g., [9,42]). Participants had to hold a DIN A4 sheet of paper at arm's length and fixate a distant object through a hole in the middle of the paper. They closed the left and right eye in turn and reported whether they still saw the object. If the object disappeared, the closed eye was marked as the dominant eye.
The main part of our study took place in front of a 3D TV (W / H / D 122,40 x 74,10 x 30,60 cm, 1080p) with active shutter glasses and a refresh rate of 60 Hz. The room light was switched off and the curtains were shut in order to minimize the flickering of the shutter glasses. The participants were placed on a chair 280 cm in front of the screen, as depicted in Figure 5. The distance resulted in a horizontal viewing angle of 12.63 • from the focus point (vertical: 7.54 • ). We gave the participants a keyboard with explicitly marked yes and no keys and told them to use their thumbs or index fingers for executing the input. The keys were chosen to have the maximum possible distance.
We told the participants that a series of images would be presented. Each image has a blue background and contains a number of yellow circles. Possibly, one of the circles pops out. The image remains visible for a split second. After the inspection, the participants have to press yes if they think that the target object was present, and no otherwise.
Between the images, a white crosshair on the same blue background was displayed in the middle. We explicitly advised the participants to focus on the crosshair. This is important as it prevents saccadic eye movements, i.e., the participants' eyes remain in the focus stage when they see the image, because the saccade has an initiation time of approximately 200-250 ms. The crosshair image was always presented for 2500 ms and disappeared when the screen switched to the actual image. We also informed the participants that they did not have to rush with their answer. During that period, the same blue background without the crosshair was exposed.
In addition, we explained that there would be a training stage before each set and that an audio feedback would indicate whether the given answer was correct or not. During the real experiment, the two sounds would be replaced with a third, rather neutral sound. This sound would prevent participants from becoming distracted by thinking about wrong answers and, thus, making subsequent errors due to the lack of concentration.
Overall, that part of the experiment consisted of four set sizes: 4, 8, 16, and 30 circles. For each set, participants had the chance to practice until they felt comfortable and told the examiner to begin with the real test. After each set, the participants were asked if they needed a pause or wanted to continue. Each set consisted of 48 images, half of them with a target object in a randomized order. Each participant experienced the same configuration. For the 24 images with a target object, 12 were rendered for the right eye and 12 for the left eye. The order was again randomly chosen. We applied that variation mainly to see whether there is a dependency between the error rate for an eye regarding its dominance.
The positions for the circles were randomly generated on a 5x6 grid with a jittering/offset function, as can be seen in Figure 5. We also left a vertical margin of 11,48 cm and a horizontal margin of 17,44 cm, limiting the overall horizontal viewing angle to about 8.88 • from the focus point (vertical: 5.22 • ). Each circle had a size of 4,59 cm or approximately 0.94 • . This setup led to the image being located in the focal, paracentral, and near-peripheral vision areas.
After the four sets were completed, we administered a web-based effort questionnaire mainly based on the NASA-TLX survey [19]. The main reason for choosing NASA-TLX is that the work by Gutwin et al. [16] used the same questionnaire for a variety of preattentive cues. Hence, we strived to create a meaningful comparison to other techniques.
The NASA-TLX survey contains six subscales, each scale ranges from 0 to 100 in increments of 5. Following aspects are measured: mental demand (low/high), physical demand (low/high), temporal demand (low/high), performance (good/poor), effort (low/high), and frustration level (low/high).
We included three additional questions on a seven-point Likert scale ranging from 0 to 6 with larger numbers indicating a more positive outcome: how well have you perceived the popout object?, how sure were you that you made the right decisions?, and how well were you able to focus the crosshair?. We will relate to these custom questions as clearness, decision-making, and focus, respectively. We also included the binary question about whether the participants experienced any headache or related physical strains.

Performance Results
In sum, 21 persons (9 females, 12 males), aged 18 to 42 (M = 25.29, SD = 6.21), participated in the study. Most of them were students (N = 15) or employees (N = 5). All participants reported normal or corrected to normal acuity and no defects of vision. Most participants had a right dominant eye (N = 16), only few had a left dominant eye (N = 5).
All performance-based computations were based on detailed automated logging of our experiment application. We measured the accuracy rate for each set. All variables were normally distributed according to Kolmogorov-Smirnov tests.
The results are depicted in Figure 6. As we have the detailed data for each image trial of each participant, we also investigated the accuracies based on the object location. The resulting accuracy matrix is presented in Figure 8. Hereby, we only considered images with Deadeye-enhanced objects. Each cell of the 5x6 grid contains the successful recognition rate of a target object at that screen position. We applied a color encoding (red means lower accuracy) to facilitate the visual analysis. The findings indicate that target objects further away from the focus point were harder to recognize.
In addition, we evaluated the relation between the dominant eye of the subject and the target eye of our technique. We had only 5 participants with a left dominant eye, hence we only considered subjects with the right dominant eye for that analysis. We compared the accuracies when the target object was exposed to the dominant eye (M = 0.84, SD = 0.13) and to the non-dominant eye (M = 0.86, SD = 0.10). A paired-samples t-test (t(15) = −1.22, p = .242) did not expose any significant differences.

Questionnaire Evaluation
After the preattentive experiment, we administered a questionnaire based on the NASA-TLX survey (see Figure 7) and our four custom questions. All variables were normally distributed according to Kolmogorov-Smirnov tests. The custom questions regarding clearness (M = 4.10, SD = 0.99) and focus (M = 4.14, SD = 1.49) are rather above average, whereas decision-making (M = 3.10, SD = 1.53) is slightly below, i.e., the participants were not very sure whether they made the correct decisions. All participants gave a negative answer to the question regarding headache or similar strains.

Conjunction Experiment
An important question for preattentive processing is the possibility of combining multiple features. In their manifold research Triesman et al. [48][49][50] pointed out that searching for a single visual stimulus can be conducted in parallel, whereas we can perform only a serial search for an item defined by the conjunction of two visual variables. An example conjunction of hue and form is depicted in Figure 2: the target object could be either a red circle among blue circles or a blue square among red squares. Wolfe et al. [54] experimented with different conjunctions of color, motion, size, and orientation, even considering a combination of three cues.
The assumptions of Triesman were partially disproved by McLeod et al. [32], and later, Müller et al. [34]: the authors reported that a conjunction of form and motion can be processed in parallel. Furthermore, as pointed out by Townsend [47], the general question whether a vision process is serial or parallel is still widely discussed.
Our second experiment was inspired by the findings on a conjunction search by Nakayama et al. [37]. The authors discovered that depth can be combined in parallel with other variables such as color and motion. The conducted experiments grouped objects in a front and a back plane. In the case of color, objects in the front plane were red and objects in the back plane were blue. Hence, a target object was either a blue one in the front or a red one in the back, as depicted in Figure 2. The authors claim that our visual system processes the planes in an alternating way, and that the search in each plane is executed in parallel. Since stereo vision is also based on binocular disparities, one might assume that our technique could lead to similar effects with regard to a conjunction search. Therefore, we conduct a similar experiment by combining Deadeye with color to evaluate that assumption.

Hypotheses
Our technique does not produce 3D depth, nevertheless Deadeye also relies on a binocular disparity. Hence, the similarity in the internal processing, i.e., the merging of two images, might lead to a similar property for our method. Therefore, we suggest that Deadeye can be combined with color as a second visual cue in parallel as our last hypothesis H3.

Procedure and Applied Measures
Subsequent to the questionnaire of the preattentive experiment, we explained our conjunction search experiment to the participants: Now, half of the circles would be magenta. All yellow circles are popping out, i.e., with Deadeye applied, all magenta circles are not popping out. A target object would have one of the two properties: either it is a yellow circle that does not pop out, or it is a magenta circle that pops out. The main difference from the first experiment is that the image would be displayed until the participants make a decision. We asked the subjects to perform as quickly and as accurately as possible. The reason to change the timing strategy was its more explorative fashion, as we would gather more information about the behavior of our technique for such a conjunction search.
Since the time was not fixed for this experiment, we decided to limit it to three set sizes: 4, 8, and 16 circles. Again, we offered a training phase for each set size and each set was composed of 48 images. Also, 24 images had a target object, 12 with a popping out magenta circle and 12 with a unmodified yellow circle. In each subgroup, 6 of the target objects were rendered for the left eye and 6 for the right eye. The ordering was randomized in the same fashion as in the first experiment.
After completion of the three sets, we requested the participants fill out the same questionnaire as after the first experiment. Overall, the study took about one hour.

Performance Results
For the f experiment, we analyzed the accuracy and the reaction time. All variables were normally distributed according to Kolmogorov-Smirnov tests. The average accuracies decrease with the increased object count (4 objects: M = 0.87, SD = 0.09; 8 objects: M = 0.81, SD = 0.12; 16 objects: M = 0.77, SD = 0.13). Our ANOVA result exposes a significant difference in accuracy between the sets (F(2, 19) = 6.81, p = .003). Post hoc Bonferroni tests reveal the details: there is a significant difference in accuracy between 4 and 16 objects (p = .020) and no significant difference between 4 and 8 objects (p = .053) or between 8 and 16 objects (p = .352).
In addition to accuracy, we also measured the reaction time, i.e., the time how long the image was shown until the participants made a decision. A summary is presented in Figure 6 We also compared the average accuracies for the popping out magenta target (M = 0.74, SD = 0.24) and the non-modified yellow target (M = 0.77, SD = 0.10). The result of a paired-samples t-test (t(20) = 0.87, p = .393) did not reveal any significant differences.

Questionnaire Evaluation
Again, we administered a questionnaire based on the NASA-TLX survey (see Figure 7) and our four custom questions. All variables were normally distributed according to Kolmogorov-Smirnov tests. The

DISCUSSION
The performance evaluation of the first experiment supports our main hypothesis H1 that Deadeye is indeed a preattentive technique. The participants recognized the feature in a 250 ms time frame with an average accuracy of ∼ 90 % independently of the number of distractors. The accuracy is similar to the outcome of most preattentive experiments described in the Related Work section and is considered as a completely sufficient proof.
With regard to wrong answers, ∼ 70 % of the errors were false negatives. In other words, it is more likely to overlook a popout target rather than wrongly imagine its presence, which is also a common behavior for preattentive techniques.
We can also confirm our second hypothesis H2 that our technique does not lead to physical strains such as headache. All participants replied with no to the corresponding question, both for the preattentive and for the conjunction experiments.
Hence, our finding is indeed an important contribution to the fundamental research, since Deadeye offers several advantages over other preattentive techniques. It is simple to implement (the object has to be removed for one eye), and, in comparison to most other methods, it does not alter any visible properties of the object such as color, position, size, or motion.
Our third hypothesis H3 stated that Deadeye can be processed in parallel when combined with color as a second visual cue. Clearly, this is not the case and the hypothesis has to be rejected. The accuracy and the reaction time are both significantly worse when the set size increases. The reaction time for only 4 objects is already greater than 2 seconds (i.e., not preattentive) and increases up to 3.5 seconds for 16 objects. Similarly, such an addition of 12 distractors results in an accuracy drop from 87 % to 77 %. Nevertheless, the accuracies show that a conjunction is still possible, although the search has to be executed serially.
Hence, Deadeye does not share similar properties with 3D depth as visual cue regarding conjunction search. Nakayama et al. [37] suggest that 3D depth works in parallel because our visual system is able separate the objects in the near and the far plane and to analyze each plane in turn. Hereby, the objects on each plane are processed in parallel. Although Deadeye eliminates any depth information of the target objects and puts them into a zero-depth plane, our experiment resulted in serial processing.
We hypothesize that the distance between the two planes does matter. Deadeye enhancements are rather subtle compared to two planes with a significant depth offset. Hence, we propose to repeat the 3D depth experiment and to gradually reduce the distance between the planes. We assume that there is a certain minimum threshold for parallel processing. If this is not the case, either the two-plane explanation is not fully correct, or our visual system is not able to group Deadeye-enhanced objects into a single depth plane. More aligned with our results, the work by O'Toole et al. [39] also reported that the question of parallel vs. serial search has several nuances when it comes to 3D depth as a cue. In particular, it does matter whether targets and distractors have crossed or uncrossed disparities, and whether targets are behind or in front of distractors.
To conclude, if a conjunction search is required, the 3D depth has an advantage. However, preattentive cues are rarely used for conjunctions. For the majority of popout applications, 3D depth has the drawback on modifying the position of the target object, which is often a significant issue. Deadeye, in contrast, preserves the correct object position.
An interesting aspect for preattentive cues is the subjective user perception. The first thing we notice is that users tend to underestimate their performance. This can be seen in the rather average results on the performance scale of the NASA-TLX (how successful do you think you were in accomplishing the goals?) and our custom question on decisionmaking (how sure were you that you made the right decisions?). Both results contradict the achieved accuracy of ∼ 90 %. On the other hand, this is not surprising when we recall that the preattentive analysis happens unconsciously. The images were shown for a split second, and subjects had to rely on a rather unchecked, intuitive result provided by the visual system.
Our second experiment was rated significantly worse regarding mental and physical demand, performance, and clearness. Overall, we can conclude that the task was more challenging and less straightforward compared to the first experiment. The target object could be only discovered by a conscious and serial search, which explains these scores. The participants also reported a significant decrease in their ability to focus on the crosshair before each image. We attribute that outcome to the fact that a serial search does not benefit from a stationary period, and an immediate saccadic eye movement appeared more efficient to the most subjects.
When comparing the NASA-TLX results of our first experiment to the outcomes reported by Gutwin et al. [16], we can observe that Deadeye performed slightly better than existing cues (note: be aware of different scales). Especially the subscales performance and effort appear to be significantly better and indicate that Deadeye can compete even against canonical cues such as color.
However, that comparison should be considered with a grain of salt. The authors explored the cues with a significantly larger viewing angle to study the behavior regarding the peripheral vision. Hence, the difference in results might be due to the viewing angle. Unfortunately, there is not much work on the comparison of different features, and we encourage to undertake further steps towards a comprehensive overview.
Our analysis shows an additional relation to the peripheral experiments of Gutwin et al., as the authors have discovered that certain cues were less efficient when applied far from the focus point. Similarly, our accuracy matrix in Figure 8 exposes a drop-off in performance for the outer columns. This indicates that Deadeye suffers under the same reduced peripheral efficiency as, e.g., shape and color. In our experiment, the image area covered a total visual angle of ∼ 19 • and included the central, paracentral, and the near-peripheral regions. Our depth perception decreases with the distance from the focus point (e.g., [33]) and nearly disappears in the peripheral region. Since we assume that the depth computation process is at least partially responsible for discovering Deadeye-enhanced objects, such a drop-off or even a complete inapplicability in far-peripheral regions appears rather comprehensible.
Another observation from our experiment is that it does not matter whether Deadeye is rendered on the dominant or the non-dominant eye. That behavior is rather in line with the findings of, e.g., Logothetis et al. [28], stating that binocular rivalry is not eye-dependent.
Certainly, Deadeye has limitations that need to be discussed. First of all, Deadeye requires a stereoscopic environment, since a different image needs to be exposed to each eye. This fact renders the technique less convenient for everyday usage and requires additional hardware.
In addition, we suggest to evaluate Deadeye in a 3D environment such as virtual reality. Our technique eliminates the depth cues for the target object. Thus, we assume that Deadeye might perform differently in such a scenario. Another disadvantage is that Deadeye is not applicable for one-eyed users. On the other hand, this kind of visual restriction is significantly less frequent compared to, e.g., color vision deficiency.
A mentionable difference between Deadeye and most other preattentive cues is its binary character. Our current method has no graduation, i.e., the object is either popping out or not. In contrast, cues such as hue can be applied in varying intensity levels, which is an additional degree of freedom and, e.g., allows a differentiation between target objects.

CONCLUSION AND FUTURE WORK
Our contribution is Deadeye, a novel preattentive visualization technique. Preattentive cues are used in a plethora of visualization approaches and interaction paradigms and allow us to enhance objects of interest such that they pop out independently from the number of distractors. Our technique contributes to the fundamental research in visualization in two ways. First, a discovery of a preattentive cue opens the door for further research and several possible applications. Second, Deadeye is the first preattentive technique that does not alter any visible properties of the target object. In contrast to existing cues, our method does not displace, recolor, reshape or animate the target. Hence, the probability of misinterpreting the data is minimized, which is a significant benefit compared to existing methods.
Deadeye is based on presenting two slightly different images to the human visual system. Hereby, the target object that should pop out is rendered for one eye only. We evaluated the method by a stateof-the-art study being commonly applied for popout variables and demonstrated that Deadeye can indeed be perceived preattentively. Three smaller, explorative experiments illustrate real-world applications of our technique in different visualization setups. In addition, we conducted a conjunction search experiment by combining Deadeye with color as a second cue. Our results showed that, in contrast to common 3D depth, Deadeye conjunctions cannot be processed in parallel.
Our initial findings encourage additional research questions that we will address in future work. Our preliminary lab experiments indicate that Deadeye delivers robust performance even if distractors of different kind are present. Our first tests with objects of different shape and color support that assumption and will be further extended in the future. This would be a significant advantage over many of other preattentive cues, since the data to be visualized is often composed of heterogeneous objects.
Another interesting experiment would be to apply our technique to moving objects to evaluate how the effect performs in dynamic scenes. Furthermore, we suggest integrating and evaluating Deadeye in sophisticated attention guidance setups and existing applicationlevel tools that already rely on popout techniques. In summary, we believe that dichoptic presentation has the potential to become a useful ingredient to the visualization toolbox beyond just stereoscopic 3D.