title: Matching Unicode word boundaries in Python
body: <p>In order to match the Unicode word boundaries [as defined in the <a href="http://unicode.org/reports/tr29/#Word_Boundary_Rules">Annex #29</a>] in Python, I have been using the <code>regex</code> package with flags <code>regex.WORD | regex.V1</code> (<code>regex.UNICODE</code> should be default since the pattern is a Unicode string) in the following way:</p>
  
  <pre><code>&gt;&gt;&gt; s="here are some words"
  &gt;&gt;&gt; regex.findall(r'\w(?:\B\S)*', s, flags = regex.V1 | regex.WORD)
  ['here', 'are', 'some', 'words']
  </code></pre>
  
  <p>It works well in this rather simple cases. However, I was wondering what is the expected behavior in case the input string contains certain punctuation. It seems to me that <a href="http://unicode.org/reports/tr29/#WB7">WB7</a> says that for example the apostrophe in <code>x'z</code> does not qualify as a word boundary which seems to be indeed the case:</p>
  
  <pre><code>&gt;&gt;&gt; regex.findall(r'\w(?:\B\S)*', "x'z", flags = regex.V1 | regex.WORD)
  ["x'z"]
  </code></pre>
  
  <p>However, if there is a vowel, the situation changes:</p>
  
  <pre><code>&gt;&gt;&gt; regex.findall(r'\w(?:\B\S)*', "l'avion", flags = regex.V1 | regex.WORD)
  ["l'", 'avion']
  </code></pre>
  
  <p>This would suggest that the regex module implements the rule <code>WB5a</code> mentioned in the standard in the <em>Notes</em> section. However, this rule also says that the behavior should be the same with <code>\u2019</code> (right single quotation mark) which I can't reproduce:</p>
  
  <pre><code>&gt;&gt;&gt; regex.findall(r'\w(?:\B\S)*', "l\u2019avion", flags = regex.V1 | regex.WORD)
  ['l’avion']
  </code></pre>
  
  <p>Moreover, even with "normal" apostrophe, a ligature (or <code>y</code>) seems to behave as a "non-vowel":</p>
  
  <pre><code>&gt;&gt;&gt; regex.findall(r'\w(?:\B\S)*', "l'œil", flags = regex.V1 | regex.WORD)
  ["l'œil"]
  &gt;&gt;&gt; regex.findall(r'\w(?:\B\S)*', "J'y suis", flags = regex.V1 | regex.WORD)
  ["J'y", 'suis']
  </code></pre>
  
  <p>Is this the expected behavior? (all examples above were executed with regex 2.4.106 and Python 3.5.2)</p>