Time Machine started to backup files, it is still working and data is larger than actual disk data I cleaned backup disk as it was full and started a clean (empty) Time Machine backup.
On a disk of 465Gb my data occupies 321Gb. Time Machine has started backuping files and I found interesting that estimated backup size was increasing during the day. Left it to work overnight, and in the morning at time of writing it has backuped 485Gb out of estimated 533Gb. The difference between backuped files and estimated full size is increasing as I see.

How is this possible? What went wrong? Should I exclude some folder?

The Mac is OS X Yosemite. Time Machine worked correctly before upgrade.
This procedure helped to resolve the issue:


Turn off Time Machine and wait to completely stop
Erase Time Machine folder with all data (in my case a folder on NAS server but can be an USB disk).
Erase Time Machine configuration file on Mac: /Library/Preferences/com.apple.TimeMachine.plist
Setup Time Machine options. E.g. set exclude folders
Turn On Time Machine


I'm not sure about the conclusion, but suspect that .plist file was somehow corrupted.