Answers.Body: <p>You can also find a suite of Matlab functions on <a href="https://www.pamguard.org/48_MATLABRcode.html" rel="nofollow noreferrer">http://www.pamguard.org/48_MATLABRcode.html</a>. Many of the developers use Matlab for their fine-scale analysis.</p>
  <p>The PAMGuard Matlab interface is now on GitHub at <a href="https://github.com/PAMGuard" rel="nofollow noreferrer">github.com/PAMGuard</a>. It allows you to read detailed data into Matlab from the PAMGuard binary files. If you want data from the database, then open it with the sqlitedatabase function, however, this does need the Matlab database toolbox to be installed. If you have this toolbox, then you'll need to learn a couple of lines of SQL (lots of resources online to help with that) and you can read from the database straight into Matlab. This is a lot more efficient and a lot more reliable than going via CSV files.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-04T02:55:50.263Z
Answers.Text: @DougGillespie if you’ve got time, would you mind updating Emily’s answer with your comment using the Edit functionality? Comments can get sort of hidden in the interface over time and do best to have your valuable info in the answer itself. I can edit if you prefer.
Answers.UserDisplayName: selene
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-03T16:26:35.553Z
Answers.LastActivityDate: 2023-01-17T23:13:59.987Z
Answers.LastEditDate: 2023-01-17T23:13:59.987Z
Answers.UserDisplayName: etgriffiths
Answers.Body: <p>Here are 3 options in R:</p>
  <p><strong>(1) Use the RSQLite package</strong></p>
  <p>With the <code>RSQLite</code> package it is pretty easy to read a database table into R, your table name may be different if you changed it within PAMGuard. You would then need to repeat this for every database (or loop / <code>lapply</code> through a vector of databases). You'll also probably need to convert the time columns to POSIXct depending on what you plan to do with this data.</p>
  <pre><code>library(RSQLite)
  con &lt;- dbConnect('MyDb.sqlite3', drv=SQLite())
  # Your table name may be different than &quot;Spectrogram_Annotation&quot;
  sa &lt;- dbReadTable(con, 'Spectrogram_Annotation')
  dbDisconnect(con)
  </code></pre>
  <p><strong>(2) Use the PAMmisc package</strong></p>
  <p>This is actually something I've needed to do fairly often, so I wrote a function that does all this for you! <code>readSpecAnno</code> does all the steps in <strong>(1)</strong>, renames some of the columns to be more descriptive (ie. <code>f1</code>/<code>f2</code> -&gt; <code>fmin</code>/<code>fmax</code>), and converts date columns to <code>POSIXct</code> as well as adding an <code>end</code> for each row instead of <code>UTC</code> and <code>Duration</code>. The nice part of having this all in a single function is that it is really easy run it on multiple databases and combine the result.</p>
  <pre><code>library(PAMmisc)
  # Again, your table name may be different
  sa &lt;- readSpecAnno('MyDb.sqlite3', table='Spectrogram_Annotation')
  # Easy to run on lots and combine the results
  library(dplyr)
  dbList &lt;- c('MyDb1.sqlite3', 'MyDb2.sqlite3')
  allSa &lt;- bind_rows(lapply(dbList, function(x) {
                     readSpecAnno(x, table='Spectrogram_Annotation')
  }))
  </code></pre>
  <p><strong>(3) Use the PAMpal package for even more info</strong></p>
  <p>The above options will only give you information contained within the spectrogram annotation table, but nothing related to the whistle contours from the detector. The <code>PAMpal</code> package can read in those whistle contours and give you additional information (min/max frequency, contour slope, and many more!). Using <code>PAMpal</code> is more involved than the other two options, but depending on your use case it can definitely be worth it.</p>
  <pre><code>library(PAMpal)
  # PAMpal needs your PAMGuard database and binary files
  bin &lt;- './Binaries/'
  db &lt;- 'MyDb.sqlite3'
  # Tell it which files to process, this will trigger a couple popup questions in the console
  pps &lt;- PAMpalSettings(db, bin)
  # We'll use the spectrogram annotations to tell PAMpal which times we want processed
  library(PAMmisc)
  sa &lt;- readSpecAnno(db, table='Spectrogram_Annotation')
  data &lt;- processPgDetections(pps, mode='time', grouping=sa, id='SpecAnnoWhistles')
  </code></pre>
  <p>Here <code>data</code> will have <em>all</em> detections between the start and end times of the spectrogram annotations, so to remove the harmonics we can use the <code>filter</code> function to get only frequencies within that range</p>
  <pre><code>for(i in seq_along(events(data))) {
      # get grouping info
      thisGroup &lt;- ancillary(data[[i]])$grouping
      # Do filtering. Note fmin/max are in Hz, convert where appropriate
      data[[i]] &lt;- filter(data[[i]],
                              freqBeg &gt; thisGroup$fmin,
                              freqBeg &lt; thisGroup$fmax,
                              freqEnd &gt; thisGroup$fmin,
                              freqEnd &lt; thisGroup$fmax,
                              UTC + duration &lt; thisGroup$end)
  }
  </code></pre>
  <p>More information on <code>PAMpal</code> can be found <a href="https://taikisan21.github.io/PAMpal/" rel="noreferrer">here</a>, and a more detailed write-up with additional comments on specifically using <code>PAMpal</code> with spectrogram annotation tables can be found <a href="https://taikisan21.github.io/PAMpal/CaseStudies.html" rel="noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-29T20:46:36.927Z
Answers.LastActivityDate: 2022-06-29T20:46:36.927Z
Answers.UserDisplayName: Taiki Sakai
Body: <p>I am using the Whistle and Moan Detector (WMD) in PAMGuard to count whistles in recordings of common dolphins to quantify the whistle activity in response to simulated sonar sounds. The whistle detector is also picking up harmonics for the simulated sonar playback, but I don't want to count those in my whistle counts.</p>
  <p>I went through and used the Spectrogram Annotation tool in PAMGuard to draw boxes around each of the sonar harmonics so that I can remove them. But now, I cannot figure out how to easily access the &quot;list&quot; of those annotations. I can open the PG database in a SQliteStudio and copy and paste it into a csv but I need to do this for 20+ playbacks, each with three recorders per playback so that seems inefficient.</p>
  <p>Does anyone know of an easy way to extract these manual annotations for further processing in R or Matlab?</p>
  
Tags: <whistles><pamguard><whistle-moan-detector><spectrogram-annotation>
Title: How to extract the timing and received level of spectrogram annotations manually marked in PAMguard?
ThreadText: # How to extract the timing and received level of spectrogram annotations manually marked in PAMguard?
  
  (Asked by: username_0 on 2022-06-29)
  
  I am using the Whistle and Moan Detector (WMD) in PAMGuard to count whistles in recordings of common dolphins to quantify the whistle activity in response to simulated sonar sounds. The whistle detector is also picking up harmonics for the simulated sonar playback, but I don't want to count those in my whistle counts.
  
  I went through and used the Spectrogram Annotation tool in PAMGuard to draw boxes around each of the sonar harmonics so that I can remove them. But now, I cannot figure out how to easily access the "list" of those annotations. I can open the PG database in a SQliteStudio and copy and paste it into a csv but I need to do this for 20+ playbacks, each with three recorders per playback so that seems inefficient.
  
  Does anyone know of an easy way to extract these manual annotations for further processing in R or Matlab?
  
  **Tags:** whistles, pamguard, whistle-moan-detector, spectrogram-annotation
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2022-06-29. Score: 5 | Accepted answer
  
  Here are 3 options in R:
  (1) Use the RSQLite package
  With the RSQLite package it is pretty easy to read a database table into R, your table name may be different if you changed it within PAMGuard. You would then need to repeat this for every database (or loop / lapply through a vector of databases). You'll also probably need to convert the time columns to POSIXct depending on what you plan to do with this data.
  library(RSQLite)
  con <- dbConnect('MyDb.sqlite3', drv=SQLite())
  # Your table name may be different than "Spectrogram_Annotation"
  sa <- dbReadTable(con, 'Spectrogram_Annotation')
  dbDisconnect(con)
  
  (2) Use the PAMmisc package
  This is actually something I've needed to do fairly often, so I wrote a function that does all this for you! readSpecAnno does all the steps in (1), renames some of the columns to be more descriptive (ie. f1/f2 -> fmin/fmax), and converts date columns to POSIXct as well as adding an end for each row instead of UTC and Duration. The nice part of having this all in a single function is that it is really easy run it on multiple databases and combine the result.
  library(PAMmisc)
  # Again, your table name may be different
  sa <- readSpecAnno('MyDb.sqlite3', table='Spectrogram_Annotation')
  # Easy to run on lots and combine the results
  library(dplyr)
  dbList <- c('MyDb1.sqlite3', 'MyDb2.sqlite3')
  allSa <- bind_rows(lapply(dbList, function(x) {
                     readSpecAnno(x, table='Spectrogram_Annotation')
  }))
  
  (3) Use the PAMpal package for even more info
  The above options will only give you information contained within the spectrogram annotation table, but nothing related to the whistle contours from the detector. The PAMpal package can read in those whistle contours and give you additional information (min/max frequency, contour slope, and many more!). Using PAMpal is more involved than the other two options, but depending on your use case it can definitely be worth it.
  library(PAMpal)
  # PAMpal needs your PAMGuard database and binary files
  bin <- './Binaries/'
  db <- 'MyDb.sqlite3'
  # Tell it which files to process, this will trigger a couple popup questions in the console
  pps <- PAMpalSettings(db, bin)
  # We'll use the spectrogram annotations to tell PAMpal which times we want processed
  library(PAMmisc)
  sa <- readSpecAnno(db, table='Spectrogram_Annotation')
  data <- processPgDetections(pps, mode='time', grouping=sa, id='SpecAnnoWhistles')
  
  Here data will have all detections between the start and end times of the spectrogram annotations, so to remove the harmonics we can use the filter function to get only frequencies within that range
  for(i in seq_along(events(data))) {
      # get grouping info
      thisGroup <- ancillary(data[[i]])$grouping
      # Do filtering. Note fmin/max are in Hz, convert where appropriate
      data[[i]] <- filter(data[[i]],
                              freqBeg > thisGroup$fmin,
                              freqBeg < thisGroup$fmax,
                              freqEnd > thisGroup$fmin,
                              freqEnd < thisGroup$fmax,
                              UTC + duration < thisGroup$end)
  }
  
  More information on PAMpal can be found here, and a more detailed write-up with additional comments on specifically using PAMpal with spectrogram annotation tables can be found here.
  
  
  ---
  
  ### Answer by username_1 on 2022-07-03. Score: 4
  
  You can also find a suite of Matlab functions on [http://www.pamguard.org/48\_MATLABRcode.html](https://www.pamguard.org/48_MATLABRcode.html). Many of the developers use Matlab for their fine-scale analysis.
  
  The PAMGuard Matlab interface is now on GitHub at [github.com/PAMGuard](https://github.com/PAMGuard). It allows you to read detailed data into Matlab from the PAMGuard binary files. If you want data from the database, then open it with the sqlitedatabase function, however, this does need the Matlab database toolbox to be installed. If you have this toolbox, then you'll need to learn a couple of lines of SQL (lots of resources online to help with that) and you can read from the database straight into Matlab. This is a lot more efficient and a lot more reliable than going via CSV files.
  
  > **Comments:**
  > **username_0**: @DougGillespie if you’ve got time, would you mind updating Emily’s answer with your comment using the Edit functionality? Comments can get sort of hidden in the interface over time and do best to have your valuable info in the answer itself. I can edit if you prefer.  
  
  ---
  
  