Place recognition using batlike sonar
One possible mechanism for sonar based recognition of places is to localize and classify the individual objects in the scene observed from that place is composed of ( Lewicki et al . , 2014 ) . Under this assumption , bats would reconstruct a ( presumably , 3D ) model of the layout of a given place and match this with a set of previously stored representations , e . g . ( Barchi et al . , 2013; Moss and Surlykke , 2001; Schnitzler et al . , 2003 ) . In other words , this mechanism implies that bats reconstruct a mental image of the environment from the echoes and use this to recognize a previously visited place . Under favourable conditions , extracting a description of a set of localized objects from the echoes is possible as the position , shape , size and texture of objects are all encoded in the binaural spectra of the echoes , e . g . ( Reijniers et al . , 2010; Schmidt , 1988; Von Helversen and Von Helversen , 2003; Peremans et al . , 2012; Wotton and Simmons , 2000; Simon et al . , 2011; Genzel and Wiegrebe , 2013 ) . However , a number of limitations of bio-sonar render it uncertain whether bats are able to apply such a place recognition strategy . Low signal-to-noise ratio obscures echo spectra ( Reijniers et al . , 2010 ) . Moreover , ambiguity of cues is introduced by concurrently encoding in the spectral cues both location and other properties of objects . For example , a spectral notch could be introduced by either the head related transfer function ( encoding the object’s location ) or the transfer function of the reflector ( encoding the object’s shape ) . This complicates extracting both object properties and object positions . Reconstructing the 3D position of objects is further complicated by the fact that , compared to vision , sonar has a limited field of view ( Surlykke et al . , 2009 ) , a low update rate ( Kleeman and Kuc , 2008; Peremans et al . , 2012 ) and a limited range ( Stilz and Schnitzler , 2012 ) . While most of these limitations to reconstructing the layout of objects could potentially be addressed by integrating information across calls , this would entail both time and complexity penalties . In addition , the capacity for reconstructing the 3D layout of objects is limited by the finite temporal resolution of the bat’s auditory system . The temporal integration in the auditory system severely limits the spatial resolution of biosonar and introduces interference between echoes . In a psychophysical experiment , Wiegrebe and Schmidt ( 1996 ) observed a temporal integration constant of about 200 μs in Megaderma lyra . Simmons and colleagues ( Simmons et al . , 1989 ) derived an integration constant of about 200–400 μs for the bat Eptesicus fuscus . Echoes from objects separated in time by less than the response time of the auditory filters will be integrated ( Simmons et al . , 1989 ) , thereby limiting bats’ ability to resolve the objects’ locations and properties . The limitations imposed by temporal integration can be appreciated by considering the volume of space across which echoes are integrated by the sonar system . This volume can be appropriately described as a section of a spherical shell . The thickness of the shell is determined by the temporal integration of the hearing apparatus . The opening angle of the section is determined by the directionality of the sonar system . Hence , the integrated volume increases rapidly for larger distances from the bat ( See Figure 1 and equation therein ) . For example , at a distance of 7 . 8 meters , an echolocation system with a beamwidth of 45 degrees is estimated to integrate the echoes originating from a volume of over 1 m3 . At the same distance , a system with a functional beamwidth of 60 degrees integrates echoes from a volume of about 2 m3 . Echoes originating from reflectors within this volume will be integrated and it is not clear that they will be perceived as individual echoes ( Geberl , 2013 ) . Indeed , in general , this volume will contain multiple reflectors . This is especially true when the scene contains complex objects such as leafy trees or bushes ( See Yovel et al . , 2008 for examples of echo trains returning from vegetation ) . 10 . 7554/eLife . 14188 . 003Figure 1 . Integration volume as a function of distance . The integration volume calculated for different beamwidths ϕ as a function of distance from the bat . The integration volume is approximated as a section of a spherical shell . The opening angle of the section is given by the beam width ϕ . The thickness of the shell Δ⁢r is given by the temporal integration of the sonar system . The temporal integration of the sonar system has been assumed to be 200 μs yielding an integration distance ( i . e . thickness of the shell ) of about 0 . 034 m . DOI: http://dx . doi . org/10 . 7554/eLife . 14188 . 003 Finally , it should be noted that the integration constant used to compute the volume of the sphere represents a very conservative upper boundary for the temporal resolution of the auditory system . Indeed , non-simultaneous masking effects have been found to extend for substantially longer intervals than the integration time ( both in bats and humans [Moore , 2012] ) . For example , Geberl ( 2013 ) found non-simultaneous masking in Phyllostomus discolor for temporal separation between echoes up to 6 ms . Some theoretical mechanisms , e . g . ( Fontaine and Peremans , 2009; Matsuo et al . , 2004; Saillant et al . , 1993 ) , have been proposed that would allow bats to extract environment impulse responses from signals composed of multiple closely spaced echo signals . However , none of them have been unequivocally proven to be exploited by bats . Moreover , so far , these mechanisms have only been tested using simple artificially generated echo trains . To the best of our knowledge , no algorithm has proven to be able to reconstruct the impulse response of a realistically complex reflector such as vegetation . Grunwald et al . ( 2004 ) reported that bats can discriminate between echoes with different impulse response statistics . This shows that the auditory periphery of the bat retains sufficient information to allow responding to the stochastic properties of the underlying impulse responses . However , this does not necessarily imply that bats can reconstruct the impulse response from the echoes , which is required to be able to reconstruct the 3D layout of a scene from the echoes . In summary , inherent limitations of biosonar , including ( 1 ) the dependence on ambiguous spectral cues for both object localization and classification and ( 2 ) the finite temporal resolution of the auditory system , render it unlikely that an explicit 3D representation of a complex scene is available to a cruising bat – and that such a representation is used to recognize places . As the capacity for recognizing places is nevertheless a necessary requirement for any navigation strategy , we propose an alternative mechanism , taking into account these limitations , to explain how echolocating bats might navigate . Instead of attempting to reconstruct the position , shape and identity of objects from the cochlear output , we propose the cochlear output to be used directly , extending the template based approach described by Wiegrebe ( 2008 ) . Under this hypothesis , bats are assumed to match the output of the cochlea to a set of stored templates , i . e . , classifying the echo signature as one of a set of memorized echo signatures each one recorded at a previously visited place . This approach obviates the need for complex reconstruction algorithms extracting 3D spatial information from the echo signals at the two ears . As it uses the sensory input directly , our approach to place recognition is analogous to the view-based place recognition that is thought to underlie visual navigation in insects ( Zeil et al . , 2003; Franz and Mallot , 2000 ) . In addition to circumventing the computationally hard problem of deriving a 3D spatial representation from complex echo signals , echo templates have been shown to be very discriminative . Kuc ( 1997 ) showed a bio-mimetic sonar device to be capable of detecting which side of a coin was up using very simple echo templates . Their computational simplicity and discriminative power have led sonar based templates being used to recognize places in robotic navigation algorithms before , e . g . ( Mataric and Brooks , 1990; Steckel and Peremans , 2013; Kuipers , 2000 ) . However , to the best of our knowledge , no study has looked at the properties of sonar templates accessible to bats operating in complex habitats . Indeed , robotic studies using sonar templates have all been carried out in artificial man-made environments . Moreover , these studies have used emitters , receivers and processing methods that are not necessarily biologically plausible . For example , robots typically ( Kleeman and Kuc , 2008 ) use a ring of sonar ranging devices ( Mataric and Brooks , 1990; Kuipers , 2000 ) , each with a limited field of view ( Steckel and Peremans , 2012 ) , that extract the delay of the first echo . In this paper , using ensonification data , we derive templates from many echo trains from natural habitats . In contrast to most robotic studies , we collect and process the echo data in a biologically plausible way . Next , we assess the viability of our hypothesis by evaluating whether the derived templates are sufficient to support place recognition . Indeed , the viability of a template-based approach to place recognition depends on the following two properties of the templates: In this paper , we test whether these conditions are satisfied by collecting many cochlear templates as they could be perceived by bats . In the discussion , we argue that , by satisfying these two criteria , the templates can support navigation . In addition , we discuss how the templates could support the acquisition of a cognitive map .
A custom ensonification device consisting of 31 Knowles FG series microphones ( Steckel and Peremans , 2012 ) and a Senscomp ( http://www . senscomp . com/ultrasonic-sensors/series-7000-sensors . php ) Series 7000 ultrasonic speaker was used ( see Figure 2 ) . The speaker produces about 106 d⁢Bs⁢p⁢l at 50 kHz and 1 meter distance . The device was mounted on a pan tilt system ( PTU-E46 , FLIR , Goleta , CA ) allowing us to rotate the device from −150 to 150 degrees in azimuth and from −25 to 40 degrees in elevation . The pan tilt system moved the ensonification device through the full extent of its mechanical range in 31 steps of 10 degrees in azimuth and 7 steps of 10 . 8 degrees in elevation The positional error of the pan tilt system is less than 0 . 1 degrees . 10 . 7554/eLife . 14188 . 004Figure 2 . Close up of the ensonification device . At the front of the device the array of 31 microphones ( Knowles FG series ) can be seen . The Senscomp Series 7000 ultrasonic speaker is located above this array . DOI: http://dx . doi . org/10 . 7554/eLife . 14188 . 004 At each azimuth and elevation direction , three measurements were gathered . The data capture and storage was handled by a single-board computer integrated into the ensonification device . The single board computer controlling the data collection also controlled the pan tilt system through a serial interface cable . The single-board computer was wirelessly connected to a laptop enabling us to start and monitor the data collection . A hyperbolic frequency modulated pulse sweeping from 100 to 40 kHz in 1 ms was emitted . This frequency range was limited by the frequency response of the emitter and the electronics . The recording of the echoes was started at the onset of the emission and ends 34 ms later . The duration of the recording time window was limited by the on-board memory of the data acquisition system . The sampling rate was 219 kSamples/s . Data was collected at three different sites ( see Figure 3 ) . The structure of the data is depicted in Figure 3 . First , twelve positions in St . Andrews Park ( Bristol , UK , 51:27:15 . 772N , 2:36:52 . 996W ) were selected for ensonification . To ensure the data represented different densities of clutter bats might operate in , four open , four semi-cluttered and four cluttered locations were selected . The data collected at St . Andrews Park consisted of 7812 ( 12 positions × 217 directions × 3 repeats ) echo trains at each of the 31 microphones . Second , data was collected in a park in Midreshet Ben Gurion , Israel ( 30:50:53 . 318N , 34:46:54 . 174E ) . At this site , bats use an artificial corridor lined with boulders as part of their commuting route . In this corridor , the ensonification device was placed at 50 positions along a straight line spaced 20 cm apart . The line approximately ran along the centre of the corridor . At each of the 50 positions , three measurements for each of the 217 directions were collected yielding 32 , 550 echo trains ( 50 positions × 217 directions × 3 repeats ) at each of the 31 microphones . Finally , data was collected at Royal Fort Gardens ( Bristol , UK , 51:27:26 . 417N , 2:36:4 . 619W ) . At this third site , 40 positions spaced 25 cm apart along a 10 meter line were sampled yielding 26 , 040 echo trains ( 40 positions × 217 directions × 3 repeats ) at each of the 31 microphones . In total , 66 , 402 echo trains for each of the 31 microphones were used in this paper , resulting in a data set consisting of a total of 2 , 058 , 462 echo trains . 10 . 7554/eLife . 14188 . 005Figure 3 . Illustration of the data collected . Three data sets were collected ( corresponding with three field sites ) . First , at St Andrews park , the ensonification device was placed at twelve different locations in habitats with varying levels of clutter ( Open , Semi Cluttered , and Cluttered ) . At each of the twelve positions , echo trains from 217 azimuth and elevation directions were collected . At the Israel and Royal Fort Gardens site , the ensonification device was placed at 50 and 40 positions along a straight line , respectively . At each of the 50 ( spaced 20 cm apart ) or 40 ( spaced 25 cm apart ) positions , echo trains from 217 azimuth and elevation directions were collected . DOI: http://dx . doi . org/10 . 7554/eLife . 14188 . 005 At both St . Andrews park and the Israel site data was collected at an approximate height of about 3 meters . In the case of St . Andrews park , overhanging vegetation sometimes restricted the device to be raised to this level . If so , the device was raised as high as possible . In the Royal Fort Park , all data was collected at a height of about 2 meters . Templates were derived from the data collected at each of the three sites . The data for each of the 12 , 50 and 40 positions in each data set were processed in the same manner . The method used in constructing the templates is summarized in algorithm 1 . The classification performance was determined for templates obtained at the St Andrews site . Estimating the probability of correct classification requires the introduction of a noise model . We introduced both a noise floor and stochastic noise on the templates .
The probability of correct classification for each of the 217 directions for the 12 positions in the St . Andrews data set is plotted in Figure 6 . For the Open environments , the probability of correct classification was very low ( i . e . Pc∼0 ) . In contrast , for the Semi cluttered and the Cluttered environments the number of templates that could be recognized with a high probability increased . The fourth Semi cluttered environment was an outlier . The classification probabilities for this position were very low . Physically , this environment might have resembled an Open environment . The branches of the large coniferous tree , the major feature at this position , did not reach low enough for them to generate echoes . Figure 6 also depicts the angular error ei for each of the 217 directions in the 12 positions . These results mirror those for the probability of classification showing large errors in the open conditions and small errors in the cluttered positions . 10 . 7554/eLife . 14188 . 008Figure 6 . Template classification performance . Left: Panoramic views taken at the 12 positions . Middle: Probability ( Pc ) for each of the 12 St Andrews positions as a function of azimuth and elevation . right: Angular error ( ei ) for each of the 12 St Andrews positions as a function of azimuth and elevation . DOI: http://dx . doi . org/10 . 7554/eLife . 14188 . 008 In an additional analysis , we confirmed that only few confusions occurred between templates from different positions . This is , the probability Pc for a given template did not decrease markedly ( ΔPc¯∼0 . 01 ) when allowing for confusion between a given template ( corresponding to one position at one azimuth and elevation direction ) and templates taken from other positions ( at the same or at a different angle ) .
In this paper , we propose template based place recognition might underlie sonar-based navigation in bats . Under this hypothesis , bats would recognize places by remembering their echo signature - rather than their 3D layout . Using ensonification data , we assessed the viability of this alternative hypothesis regarding bat navigation by assessing two critical properties of the templates: ( 1 ) unique classification of templates and ( 2 ) template continuity . In the following , we discuss our findings regarding these two properties . The Royal Fort Gardens data set resulted in significantly smaller linear catchment distances ( Figures 8 , 9 , 11 ) . On the other hand , the average probability of correct classification was significantly higher in the Royal Fort Gardens data set ( Figure 1 ) . The difference in layout between the two sites can be appreciated from inspecting Figure 3 . The Israel site was a more open habitat than the Royal fort site , where the reflecting foliage was closer to the ensonification device than the rocks at the Royal fort site . In addition , the rocks at the Israel site ( in spite of being farther away ) resulted in stronger echoes than the foliage at the Royal fort site ( compare panels a & d in Figures 8 , 9 ) . The larger linear catchment distances for the Israel data suggests that navigation by means of templates is facilitated by ( strong ) echoes originating from distant reflectors . Distant reflectors move more slowly through the bat’s ’field of view’ , i . e . , they undergo a smaller degree of motion parallax . As our data shows , this results in templates with larger catchment distances . This might facilitate navigation . On the other hand , the Royal Fort Gardens data suggest that the higher motion parallax for closer reflectors results in a better template classification . Indeed , closer reflectors change more abruptly when moving through space . This seems to result in templates that are easier to classify – at the cost of smaller linear catchment distances . In short , our data reveals a trade-off between the linear catchment distance size and the probability of correct classification of a template . The existence of this trade-off is confirmed by the negative correlation we found between the linear catchment distance size and the probability of correct classification . Hence , our results suggest that navigating bats should maintain a preferred distance to foliage and other reflectors . Indeed , keeping a larger distance would reduce the average object motion parallax and result in larger linear catchment distances . On the other hand , keeping a larger distance would reduce the probability of correctly classifying templates . In addition , flying further away from objects will , in general , result in weaker echoes ( in our Israel data this effect has been offset by the strong echoes returned by the rocks ) . Therefore , our template theory of navigation predicts navigation would be facilitated by keeping a distance from reflectors that balances the opposing effects of motion parallax on the templates’ linear catchment distance and probability of correct classification . Whilst successful navigation does not require a map-like representation , e . g . ( Cruse and Wehner , 2011; Cheung et al . , 2014; Hesslow , 2012; Collett et al . , 2013 ) , it is likely that bats use some sort of cognitive map ( Geva-Sagiv et al . , 2015 ) . However , to the best of our knowledge , the use of such a map has not been demonstrated experimentally ( Holland , 2007 ) . Nevertheless , indirect evidence for the existence of a cognitive map , integrating echolocation information if available , is the finding of grid and place cells in bats ( Yartsev et al . , 2011; Yartsev and Ulanovsky , 2013 ) and bats’ ability to remember flight routes in a completely dark room ( Barchi et al . , 2013 ) . Building and maintaining a cognitive map based on sonar templates would require the successful completion of three subtasks: ( 1 ) exploring the environment and building a library of templates , ( 2 ) integrating the templates into a map-like representation , ( 3 ) deriving motor plans from this representation that take the bat from its current location to a desired position . So far , knowledge about how bats could address these challenges is very sparse . However , work has begun on each of these components . Below , we review this work and outline a potential strategy for constructing a cognitive map using the templates as proposed in this paper . In this paper , we have proposed that bats recognize places by remembering their echo signature - rather than their 3D layout . It could be argued that , since bats have been shown to be capable of recognizing objects , they could very well use that same capability to recognize scenes based on the 3D object layout . However , we would like to point out that the behaviour of bats in recognition experiments differs from that of cruising/navigating bats . In recognition experiments , bats typically ensonify the same object from different directions as part of an active object-centred exploration process , e . g . ( Genzel et al . , 2012; Geipel et al . , 2013; Simon et al . , 2006 ) . Cruising/navigating bats , on the other hand , fly by objects along their flight path resulting in a more accidental , i . e . less object-centred , and less extensive series of observations of those objects . Such a fly-by mode of echolocation is well suited to the proposed template-based strategy as it requires local ’snapshots’ only . Also , we would like to argue that even in the object recognition experiments bats might not be building a 3D reconstruction of the object . They might instead be looking for diagnostic acoustic cues , e . g . spectral cues as hypothesized in the study by Simon et al . ( 2006 ) . Such an account of object recognition would be easily integrated with our template-based account of place recognition . One possible way the two might interact is that the template-based place descriptors would be used to build a map first and then later ( or possibly in parallel ) ’landmarks’ , i . e . uniquely identified objects , would be associated with places on the map . Hence , while we agree that a 3D place description -were it to exist- would be more general and easier to change , we believe it still remains to be proven that such a description is actually built by bats in a navigation context . Indeed , it is our hypothesis that bats would be driven towards an alternative , template based , mechanism either by the inability of their sonar systems to reconstruct a 3D layout of the environment or , if a 3D layout of the environment could be inferred from the echoes , by the associated much higher computational burden placed upon their limited cognitive resources . To produce direct evidence that would allow choosing between the two alternatives , we suggest future experiments might be set up to test the prediction by the template based place recognition mechanism advocated here that bats would not be able to distinguish between scenes if their 3D layout is different while their resulting template is similar . Therefore , behavioural evidence in favour of our approach would consist of bats failing to distinguish complex scenes giving rise to many echoes , i . e . with a similar complexity of their natural habitats , that result in the same template , essentially a low-pass filtered range-intensity profile . To test this , one could generate echoes from a complex virtual scene and its mirror image . These should result in the same template whereas a 3D model account predicts bats should be able to distinguish between the two scenes . Alternatively and possibly more straightforward to test , the 3D model account predicts bats would be able to recognize echoes coming from rotated versions of the same underlying scene . The template approach predicts that bats would be unable to make such generalizations for larger rotations , i . e . once the measured template would fall outside the catchment area of the stored template .
Echolocating bats have excellent spatial memory and are able to navigate to salient locations using bio-sonar . Navigating and route-following require animals to recognize places . Currently , it is mostly unknown how bats recognize places using echolocation . In this paper , we propose template based place recognition might underlie sonar-based navigation in bats . Under this hypothesis , bats recognize places by remembering their echo signature - rather than their 3D layout . Using a large body of ensonification data collected in three different habitats , we test the viability of this hypothesis assessing two critical properties of the proposed echo signatures: ( 1 ) they can be uniquely classified and ( 2 ) they vary continuously across space . Based on the results presented , we conclude that the proposed echo signatures satisfy both criteria . We discuss how these two properties of the echo signatures can support navigation and building a cognitive map .
Bats produce loud calls and listen to the returning echoes to find their way around . This process , known as echolocation , is sometimes described as 'seeing with sound' . The way bats perceive the world through echolocation , however , is fundamentally different from how we experience it through vision . Echolocation provides much less information about the world than vision does , but despite this , bats are agile navigators and hunters . It is not clear how bats navigate so well without much information . In particular , researchers would like to know how echolocating bats recognize the places that they regularly visit while foraging and navigating . When we visually recognize places , we identify and localize the various objects making up the scene . But echolocation is unlikely to provide enough information to allow bats to identify and localize the objects in a particular place . To investigate how bats recognize places , Vanderelst et al . built an artificial bat: a device that contained ultrasonic microphones to act like the bat’s ears and an ultrasonic speaker to act like the bat’s mouth . The artificial bat device was then used to collect echoes from different locations in real bat habitats . Processing the echoes using machine-learning techniques showed that the echoes that returned from each location were different enough for a computer to recognize the location . By using a simplified version of the echoes , Vanderelst et al . also showed that the locations could be recognized even if there was not enough information to identify specific objects or vegetation at the site . This suggests that bats do not simply use echolocation to recreate the three-dimensional layout of a location , as some researchers have proposed . While much remains to be learned about how bats use echolocation for navigation , future work that teases out bat navigation strategies might help us to build robots that can navigate using similar tactics .