Answers.Body: <p>There seems to be a problem with using the <code>[Title/Abstract]</code> field. What you have looks correct and should produce the expected result, since it should capture articles where the search term appears in either the title or in the abstract, or in both. But it does not. Strangely, you get the expected result using <code>{keyword}[Abstract]</code>.</p>
  <p>A workaround could be to <code>OR</code> the fields with:</p>
  <pre><code>f'(&quot;{keyword}&quot;[Title] OR &quot;{keyword}&quot;[Abstract])'
  </code></pre>
  <p>Perhaps something like:</p>
  <pre><code>search_term = ' AND '.join([
      f'(&quot;{start_date_str}&quot;[Publication Date] : &quot;{end_date_str}&quot;[Publication Date])',
      f'(&quot;{keyword}&quot;[Title] OR &quot;{keyword}&quot;[Abstract])',
      f'(open_access[filter])',
  ])
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T05:05:56.330Z
Answers.LastActivityDate: 2023-03-23T05:05:56.330Z
Answers.UserDisplayName: Steve
Body: <p>I am using the following code to download all pubmed central article ids that have a keyword appearing in their titles/abstracts and also falling between two dates. The code seems to work when I select database = &quot;pubmed&quot; but when I switch to database = &quot;pmc&quot; I get empty list. For example, using the start and end dates as mentioned in the code I get 4 PubMed ids: ['36902449', '36880658', '36583446', '35239250'] and when I check manually, the first one among them (which is '36902449') is also present in PMC database, of course, the id is different (10003631) but same article. The code is expected to fetch that id. I'd appreciate it if someone could point out what I'm missing here.</p>
  <pre class="lang-py prettyprint-override"><code>import os
  import datetime
  from Bio import Entrez
  
  # Set the email address to identify yourself to NCBI
  Entrez.email = &quot;example@gmail.com&quot;
  
  # Set the date range for the search
  start_date = datetime.datetime(2023, 3, 1)
  end_date = datetime.datetime(2023, 3, 22)
  
  # Format the dates as strings in the correct format
  start_date_str = start_date.strftime(&quot;%Y/%m/%d&quot;)
  end_date_str = end_date.strftime(&quot;%Y/%m/%d&quot;)
  
  # Build the search query with the date range
  keyword = &quot;dosage compensation&quot;
  # database = &quot;pubmed&quot;
  database = &quot;pmc&quot;
  
  # Use esearch to fetch the list of matching article IDs
  if database == &quot;pubmed&quot;:
      search_term = f'(&quot;{start_date_str}&quot;[Date - Publication] : &quot;{end_date_str}&quot;[Date - Publication]) AND {keyword}[Title/Abstract]'
      handle = Entrez.esearch(db=database,
                              term=search_term,
                              retmax=100,
                              api_key = os.environ['ENTREZ_API_KEY'])
  elif database == &quot;pmc&quot;:
      search_term = f'(&quot;{start_date_str}&quot;[Publication Date] : &quot;{end_date_str}&quot;[Publication Date]) AND {keyword}[Title/Abstract] AND open_access[filter]'
      handle = Entrez.esearch(db=database,
                          term=search_term,
                          retmax=1000,
                          api_key = os.environ['ENTREZ_API_KEY'])
  
  record = Entrez.read(handle)
  id_list = record['IdList']
  handle.close()
  
  # Print the number of matching articles
  print(f&quot;Found {len(id_list)} articles published between {start_date_str} and {end_date_str} in {database}&quot;)
  # print(id_list)
  </code></pre>
  
Tags: <python><biopython>
Title: Download all pubmed central article ids that have a keyword appearing in their titles/abstracts and also falling between two dates
ThreadText: # Download all pubmed central article ids that have a keyword appearing in their titles/abstracts and also falling between two dates
  
  (Asked by: username_0 on 2023-03-22)
  
  I am using the following code to download all pubmed central article ids that have a keyword appearing in their titles/abstracts and also falling between two dates. The code seems to work when I select database = "pubmed" but when I switch to database = "pmc" I get empty list. For example, using the start and end dates as mentioned in the code I get 4 PubMed ids: ['36902449', '36880658', '36583446', '35239250'] and when I check manually, the first one among them (which is '36902449') is also present in PMC database, of course, the id is different (10003631) but same article. The code is expected to fetch that id. I'd appreciate it if someone could point out what I'm missing here.
  
  ```py
  import os
  import datetime
  from Bio import Entrez
  
  # Set the email address to identify yourself to NCBI
  Entrez.email = "example@gmail.com"
  
  # Set the date range for the search
  start_date = datetime.datetime(2023, 3, 1)
  end_date = datetime.datetime(2023, 3, 22)
  
  # Format the dates as strings in the correct format
  start_date_str = start_date.strftime("%Y/%m/%d")
  end_date_str = end_date.strftime("%Y/%m/%d")
  
  # Build the search query with the date range
  keyword = "dosage compensation"
  # database = "pubmed"
  database = "pmc"
  
  # Use esearch to fetch the list of matching article IDs
  if database == "pubmed":
      search_term = f'("{start_date_str}"[Date - Publication] : "{end_date_str}"[Date - Publication]) AND {keyword}[Title/Abstract]'
      handle = Entrez.esearch(db=database,
                              term=search_term,
                              retmax=100,
                              api_key = os.environ['ENTREZ_API_KEY'])
  elif database == "pmc":
      search_term = f'("{start_date_str}"[Publication Date] : "{end_date_str}"[Publication Date]) AND {keyword}[Title/Abstract] AND open_access[filter]'
      handle = Entrez.esearch(db=database,
                          term=search_term,
                          retmax=1000,
                          api_key = os.environ['ENTREZ_API_KEY'])
  
  record = Entrez.read(handle)
  id_list = record['IdList']
  handle.close()
  
  # Print the number of matching articles
  print(f"Found {len(id_list)} articles published between {start_date_str} and {end_date_str} in {database}")
  # print(id_list)
  
  ```
  
  **Tags:** python, biopython
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-23. Score: 1 | Accepted answer
  
  There seems to be a problem with using the `[Title/Abstract]` field. What you have looks correct and should produce the expected result, since it should capture articles where the search term appears in either the title or in the abstract, or in both. But it does not. Strangely, you get the expected result using `{keyword}[Abstract]`.
  
  A workaround could be to `OR` the fields with:
  
  ```
  f'("{keyword}"[Title] OR "{keyword}"[Abstract])'
  
  ```
  
  Perhaps something like:
  
  ```
  search_term = ' AND '.join([
      f'("{start_date_str}"[Publication Date] : "{end_date_str}"[Publication Date])',
      f'("{keyword}"[Title] OR "{keyword}"[Abstract])',
      f'(open_access[filter])',
  ])
  
  ```
  
  
  ---
  
  