title: How does excluding strings using regex actually works
body: <p>I'm unclear how negative regular expressions work. I followed several posts (<a href="https://stackoverflow.com/questions/406230/regular-expression-to-match-a-line-that-doesnt-contain-a-word?rq=1">post 1</a>, <a href="https://stackoverflow.com/questions/6851921/negative-lookahead-regular-expression">post 2</a>, ) and I used their patterns and they work, but their explanations do not make sense to me. I tried several regex tester sites like <a href="https://regex101.com/" rel="nofollow noreferrer">regex101</a> etc, but they fail to process patterns that appear to work in Python as per posts 1 &amp; 2. </p>
  
  <p>My preferred approach is that regex would behave for negative logic the same way it behaves for positive logic. However, it appears to me that as soon as negative logic is used, it starts a whole new way of processing that is hard to follow. I know there are workarounds for this, but I'm interested in understanding it via regex.</p>
  
  <p>Objective for the below examples: Say I have a list of commodities, from which I would like to get a list that are not "gas" as defined by variable <code>except</code>. In other words, I need a list of products that do not "contain" word "gas" in their name. </p>
  
  <p>Here is a helper code to try different ideas: </p>
  
  <pre><code>import re
  cmdty = ['natural gas', 'Henry Hub Natural Gas Contract', 'Crude Oil', 'Brent', 'WTI']
  expect= cmdty[-3:] # i.e. ['Crude Oil', 'Brent', 'WTI']
  print(f'Starting list: {cmdty}. Would like to get: {expect}')
  def check (pattern,cmdty=cmdty, expect=expect, comment=""): 
      out = [c for c in cmdty if re.search(pattern,c)]
      good = "yes" if set(out) == set(expect) else "no"
      print(f'pattern={pattern:20}: worked: {good:&gt;3}. output={out}. comment: {comment}')
  </code></pre>
  
  <p>Various attempts at regular expressions to make it work:</p>
  
  <pre><code>check(pattern='(?i)(?=gas)',comment="This one works, but requires negating the results")
  check(pattern='(?i)(?!gas)',comment="My hope was that this would work")
  check('(?i)(?:!gas)',comment="")
  check('(?i)\s(?!gas)',comment="strange outcome")
  check('(?i).*(?!gas).*')
  check('^(?i)(?!.*gas).*$', comment='works')
  check('^(?i)((?!gas).)*$', comment='not sure this one works')
  check('(?i)^.*(?!gas).*$',comment="I'd expect this one to work, but does not")
  check('(?i)^(?!.*gas).*$', comment='works')
  check('(?i)nat(?!gas)', comment='makes sense, but super odd')
  </code></pre>
  
  <p>Initial list and the objective:</p>
  
  <pre><code>Starting list: ['natural gas', 'Henry Hub Natural Gas Contract', 'Crude Oil', 'Brent', 'WTI'].
  
  Would like to get: ['Crude Oil', 'Brent', 'WTI']
  </code></pre>
  
  <p>Here are output results using various attempts to get it to work. What is the way to think about this, so it makes sense. </p>
  
  <pre><code>pattern=(?i)(?=gas)         : worked:  no. output=['natural gas', 'Henry Hub Natural Gas Contract']. comment: This one works, but requires negating the results
  pattern=(?i)(?!gas)         : worked:  no. output=['natural gas', 'Henry Hub Natural Gas Contract', 'Crude Oil', 'Brent', 'WTI']. comment: My hope was that this would work
  pattern=(?i)(?:!gas)        : worked:  no. output=[]. comment: 
  pattern=(?i)\s(?!gas)       : worked:  no. output=['Henry Hub Natural Gas Contract', 'Crude Oil']. comment: strange outcome
  pattern=(?i).*(?!gas).*     : worked:  no. output=['natural gas', 'Henry Hub Natural Gas Contract', 'Crude Oil', 'Brent', 'WTI']. comment: 
  pattern=^(?i)(?!.*gas).*$   : worked: yes. output=['Crude Oil', 'Brent', 'WTI']. comment: works
  pattern=^(?i)((?!gas).)*$   : worked: yes. output=['Crude Oil', 'Brent', 'WTI']. comment: not sure this one works
  pattern=(?i)^.*(?!gas).*$   : worked:  no. output=['natural gas', 'Henry Hub Natural Gas Contract', 'Crude Oil', 'Brent', 'WTI']. comment: I'd expect this one to work, but does not
  pattern=(?i)^(?!.*gas).*$   : worked: yes. output=['Crude Oil', 'Brent', 'WTI']. comment: works
  pattern=(?i)nat(?!gas)      : worked:  no. output=['natural gas', 'Henry Hub Natural Gas Contract']. comment: makes sense, but super odd`
  </code></pre>