Question: I have a web scraper that takes forum questions, splits them into individual words and writes it to the text file. The words are stored in a list of tuples. Each tuple contains the word and its frequency. Like so...

```
[(u'move', 3), (u'exploration', 4), (u'prediction', 21),
 (u'find', 5), (u'user', 2), (u'interface', 2), (u'pleasant', 2),
 (u'am', 11), (u'puzzled', 2), (u'find', 5), (u'way', 5),
 (u'prediction', 21), (u'mode', 2), (u'have', 21),
 (u'explored', 2), (u'file', 9), (u'Can', 7), (u'help', 6),
 (u'Possible', 1), (u'bug', 2), (u'data', 31), (u'is', 17)

```

however, some person on the forum used the character \u200b which breaks all my code because that character is no longer a Unicode whitespace.

```
(u'used\u200b', 1)

```

Printing it out does not produce an error, but writing to a text file does. I have found that `string.strip()` and `string.replace()` do not help, so I was wondering how to use the regex library to get rid of that character. I plan on parsing through the entire list of tuples to find it.

Answer: 
I tested that with python 2.7. `replace` works as expected:

```
>>> u'used\u200b'.replace(u'\u200b', '*')
u'used*'

```

and so does strip:

```
>>> u'used\u200b'.strip(u'\u200b')
u'used'

```

Just remember that the arguments to those functions have to be Unicode literals. It should be `u'\u200b'`, not `'\u200b'`. Notice the `u` in the beginning.

And actually, writing that character to a file works just fine.

```
>>> import codecs
>>> f = codecs.open('a.txt', encoding='utf-8', mode='w')
>>> f.write(u'used\u200bZero')

```

See resources:  

* [The python 2 Unicode howto](https://docs.python.org/2/howto/unicode.html)
* [The python 3 Unicode howto](https://docs.python.org/3/howto/unicode.html)
* [The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)](http://www.joelonsoftware.com/articles/Unicode.html)
you can open the file C:\Users\SOMEN\AppData\Local\Programs\Python\Python37-32\lib\encodings\**cp1252.py*\* in my case but it should be the same.

```
decoding_table = (
'\x00'     #  0x00 -> NULL
'\x01'     #  0x01 -> START OF HEADING
'\x02'     #  0x02 -> START OF TEXT
'\x03'     #  0x03 -> END OF TEXT
'\x04'     #  0x04 -> END OF TRANSMISSION
'\x05'     #  0x05 -> ENQUIRY
'\x06'     #  0x06 -> ACKNOWLEDGE
'\x07'     #  0x07 -> BELL
'\x08'     #  0x08 -> BACKSPACE
'\t'       #  0x09 -> HORIZONTAL TABULATION
'\n'       #  0x0A -> LINE FEED
'\x0b'     #  0x0B -> VERTICAL TABULATION
'\x0c'     #  0x0C -> FORM FEED
'\r'       #  0x0D -> CARRIAGE RETURN
'\x0e'     #  0x0E -> SHIFT OUT
'\x0f'     #  0x0F -> SHIFT IN
'\x10'     #  0x10 -> DATA LINK ESCAPE
'\x11'     #  0x11 -> DE
#add the character code here
'\u200b' #add this in the file and save it.

```