title: get error when extracting feature of one Chinese sentense using sklearn
body: <p>I want to extract feature of a sentence in Chinese using a limited vocabulary,but I get no feature for Chinese word,only get feature for a '2015',it's quite confusing.</p>
  
  <p>this is my code</p>
  
  <pre><code># -*- coding:utf-8 -*-  
  
  from sklearn.feature_extraction.text import CountVectorizer
  
  #load dictionary
  dic_file = open("/home/xcy/graduation-design/YangBen/dict.txt")
  pn_dict = [ line.strip()for line in dic_file]
  
  vectorizer = CountVectorizer(tokenizer=lambda x: x.split(), vocabulary=pn_dict, binary=True)
  
  #load training data
  train_file = open("/home/xcy/graduation-design/YangBen/1k.txt")
  train_sentence = [ line.strip()  for line in train_file  if line.strip() != "" ]
  
  X = vectorizer.transform(train_sentence)
  
  print(X.toarray())
  
  #print(train_sentence[0].split()[0])
  
  #print(vectorizer.vocabulary_.get('下降'))
  
  #for i in vectorizer.get_feature_names():
  #       print(i)
  </code></pre>
  
  <p>/home/xcy/graduation-design/YangBen/dict.txt</p>
  
  <pre><code>下降
  公司
  2015
  </code></pre>
  
  <p>/home/xcy/graduation-design/YangBen/1k.txt</p>
  
  <pre><code>2015 年 三 季度 报 发现 ， 虽然 华策 影视 等 公司 的 净利润 同比 有所 下降
  </code></pre>
  
  <p>the output of my code is</p>
  
  <pre><code>[[0 0 1]]
  </code></pre>
  
  <p>while following output is expected, the first '1' for '下降' , second '1' for '公司'</p>
  
  <pre><code>[[1 1 1]]
  </code></pre>
  
  <p>all three files above is utf-8, watched in vim (:set fileencoding)</p>
  
  <p>Env:</p>
  
  <p>Ubuntu16.04,</p>
  
  <p>Python 2.7.12,</p>
  
  <p>virtualenv 15.1.0,</p>
  
  <p>just installed scikit-learn(pip install -U scikit-learn
  ) yeserday</p>