Answers.Body: <p>A general 3D localization problem with distributed sensors could be approached like this:</p>
  <p>with 5 or more microphones that surround the insect(s) you can write 4 linear equations that relate insect location to time delay between microphones, that can be solved for 3D insect location + distance to reference location. If you put a microphone on each corner of the cube you get 7 linear equations that you use a least-mean-square solution.</p>
  <p>Trick is to estimate together with 3D location also the (redundant) distance to reference location.</p>
  <p>you should, with any decent micro controller, be able to estimate location in real time (&lt;0.1s) as required.</p>
  <p>In order not to clutter with unreadable maths the formula is easily derived from</p>
  <p>R_ni^2 = (x_n - x_i)^2 + (y_n - y_i)^2 + (z_n - z_i)^2   where  i: insect; n: n-th sensor</p>
  <p>if you expand all equations and use R_ni = R_0i + c * (t_n - t_i)</p>
  <p>then you should arrive to a set of linear equations that depend on R_0i, x_i, y_i, z_i.</p>
  <p>see also Zimmer (2011) for this and other localization and tracking algorithms, that are written in terms of underwater application but could easily be translated to in-air as long as wind-effects can be ignored.</p>
  <p>Caveat: sound of insect should be such that you can either determine the arrival on each microphone, or can estimate the arrival difference by cross- correlation.</p>
  <p>With your laboratory environment, multi-path should not be an issue, so I would try.</p>
  <p>The only issue I can see is that sound is such (continuous buzz) that time delay estimation or cross-correlation is not unique. In such cases you would need &gt;3 microphone beamformer, where the microphone spacings of the sub arrays are sufficiently close to guarantee unique directions. Then you are ending up in classical triangulations.</p>
  <p>Ref:</p>
  <p>Zimmer W., 2011, Passive Acoustic Monitoring of Cetaceans, Cambridge University Press, pp 198 ff</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-27T16:19:34.767Z
Answers.LastActivityDate: 2022-06-28T04:57:31.667Z
Answers.LastEditDate: 2022-06-28T04:57:31.667Z
Answers.UserDisplayName: WMXZ
Body: <p>I'm working with noisy-flying insects and <strong>I would like to track their position with a very low latency (&lt; 0.1s) under laboratory conditions</strong>.</p>
  <p>There are a couple of solutions based on 2 cameras and a tracking algorithm (e.g. <a href="https://doi.org/10.1109/IEMBS.2011.6090163" rel="nofollow noreferrer">doi</a>), however, even if some systems are supposed to be &quot;real-time&quot; (e.g. <a href="http://www.scitracks.com/trackit-3d-fly/" rel="nofollow noreferrer">here</a>), I have not seen practically any systems where the XYZ position are retrieved very quickly for realtime applications.</p>
  <p>Tracking the flight-tone looks a good idea for real time as it is less demanding to deal with sounds (1D) than with images (2D) and sample frequency is more easily higher in sound than in video. Theoretically, this could be done by measuring time and amplitude differences at each microphone of an array.</p>
  <p><strong>--&gt; QUESTION: Do you know any very fast localization algorithms using a microphone array, which have already been proved to be efficient to track a single flying insects producing a loud flight-tone (~ wingbeat frequency + overtones)?</strong></p>
  <p>PS: My question is related to another <a href="https://bioacoustics.stackexchange.com/questions/32/packages-to-acoustically-localise-animals-with-microphone-arrays">question</a> on this site, but there are 2 main differences: 1) Real-time is a requirement and 2) animals to be tracked are inside the space formed by the microphone array (laboratory condition), which facilitates the localization.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-27T12:54:26.167Z
Comments.Text: there are some phrases which could be made more specific '..with flying insects..' and '...context of noisy flying insects'. It'd be great to get the following details for a specific answer: 1) Will there be multiple insects flying and 2) what sounds or 'noise' of the insect will be used for tracking (the buzz from the wings, or some other sounds) and 3) if multiple insects will there be temporal/spectral overlap in the sounds produced by the insect/s?
Comments.UserDisplayName: Thejasvi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-27T13:31:14.597Z
Comments.Text: Thanks @Thejasvi, I've just edited my question to make it more specific and clearer.
Comments.UserDisplayName: Noil
Tags: <localization><tracking><insect><realtime>
Title: Real-time 3D tracking of flying insects using a microphone array
ThreadText: # Real-time 3D tracking of flying insects using a microphone array
  
  (Asked by: username_0 on 2022-06-27)
  
  I'm working with noisy-flying insects and **I would like to track their position with a very low latency (< 0.1s) under laboratory conditions**.
  
  There are a couple of solutions based on 2 cameras and a tracking algorithm (e.g. [doi](https://doi.org/10.1109/IEMBS.2011.6090163)), however, even if some systems are supposed to be "real-time" (e.g. [here](http://www.scitracks.com/trackit-3d-fly/)), I have not seen practically any systems where the XYZ position are retrieved very quickly for realtime applications.
  
  Tracking the flight-tone looks a good idea for real time as it is less demanding to deal with sounds (1D) than with images (2D) and sample frequency is more easily higher in sound than in video. Theoretically, this could be done by measuring time and amplitude differences at each microphone of an array.
  
  **--> QUESTION: Do you know any very fast localization algorithms using a microphone array, which have already been proved to be efficient to track a single flying insects producing a loud flight-tone (~ wingbeat frequency + overtones)?**
  
  PS: My question is related to another [question](https://bioacoustics.stackexchange.com/questions/32/packages-to-acoustically-localise-animals-with-microphone-arrays) on this site, but there are 2 main differences: 1) Real-time is a requirement and 2) animals to be tracked are inside the space formed by the microphone array (laboratory condition), which facilitates the localization.
  
  **Tags:** localization, tracking, insect, realtime
  
  > **Comments:**
  > **username_1**: there are some phrases which could be made more specific '..with flying insects..' and '...context of noisy flying insects'. It'd be great to get the following details for a specific answer: 1) Will there be multiple insects flying and 2) what sounds or 'noise' of the insect will be used for tracking (the buzz from the wings, or some other sounds) and 3) if multiple insects will there be temporal/spectral overlap in the sounds produced by the insect/s?  
  > **username_0**: Thanks @username_1, I've just edited my question to make it more specific and clearer.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-06-27. Score: 3 | Accepted answer
  
  A general 3D localization problem with distributed sensors could be approached like this:
  
  with 5 or more microphones that surround the insect(s) you can write 4 linear equations that relate insect location to time delay between microphones, that can be solved for 3D insect location + distance to reference location. If you put a microphone on each corner of the cube you get 7 linear equations that you use a least-mean-square solution.
  
  Trick is to estimate together with 3D location also the (redundant) distance to reference location.
  
  you should, with any decent micro controller, be able to estimate location in real time (<0.1s) as required.
  
  In order not to clutter with unreadable maths the formula is easily derived from
  
  R\_ni^2 = (x\_n - x\_i)^2 + (y\_n - y\_i)^2 + (z\_n - z\_i)^2 where i: insect; n: n-th sensor
  
  if you expand all equations and use R\_ni = R\_0i + c \* (t\_n - t\_i)
  
  then you should arrive to a set of linear equations that depend on R\_0i, x\_i, y\_i, z\_i.
  
  see also Zimmer (2011) for this and other localization and tracking algorithms, that are written in terms of underwater application but could easily be translated to in-air as long as wind-effects can be ignored.
  
  Caveat: sound of insect should be such that you can either determine the arrival on each microphone, or can estimate the arrival difference by cross- correlation.
  
  With your laboratory environment, multi-path should not be an issue, so I would try.
  
  The only issue I can see is that sound is such (continuous buzz) that time delay estimation or cross-correlation is not unique. In such cases you would need >3 microphone beamformer, where the microphone spacings of the sub arrays are sufficiently close to guarantee unique directions. Then you are ending up in classical triangulations.
  
  Ref:
  
  Zimmer W., 2011, Passive Acoustic Monitoring of Cetaceans, Cambridge University Press, pp 198 ff
  
  
  ---
  
  