Question: I am searching way to use stanford word tokenizer in nltk, I want to use because when I compare results of stanford and nltk word tokenizer, they both are different. I know there might be way to use stanford tokenizer, like we can stanford POS Tagger and NER in NLTK. 

Is it possible to do use stanford tokenizer without running server? 

Thanks

Answer: 
Note: This solution would only work for:

* NLTK v3.2.5 (v3.2.6 would have an even simpler interface)
* Stanford CoreNLP (version >= 2016-10-31)

First you have to get Java 8 properly installed first and if Stanford CoreNLP works on command line, the Stanford CoreNLP API in NLTK v3.2.5 is as follows.

**Note:** You have to start the CoreNLP server in terminal BEFORE using the new CoreNLP API in NLTK.

On the terminal:

```
wget http://nlp.stanford.edu/software/stanford-corenlp-full-2016-10-31.zip
unzip stanford-corenlp-full-2016-10-31.zip && cd stanford-corenlp-full-2016-10-31

java -mx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer \
-preload tokenize,ssplit,pos,lemma,parse,depparse \
-status_port 9000 -port 9000 -timeout 15000

```

In Python:

```
>>> from nltk.parse.corenlp import CoreNLPParser
>>> st = CoreNLPParser()
>>> tokenized_sent = list(st.tokenize('What is the airspeed of an unladen swallow ?'))
>>> tokenized_sent
['What', 'is', 'the', 'airspeed', 'of', 'an', 'unladen', 'swallow', '?']

```
Outside of NLTK, you can use the [*official* Python interface that's recently release by Stanford NLP](https://github.com/stanfordnlp/python-stanford-corenlp):

Install
=======

```
cd ~
wget http://nlp.stanford.edu/software/stanford-corenlp-full-2016-10-31.zip
unzip stanford-corenlp-full-2016-10-31.zip && cd stanford-corenlp-full-2016-10-31
pip3 install -U https://github.com/stanfordnlp/python-stanford-corenlp/archive/master.zip

```

Setup Environment
=================

```
# On Mac
export CORENLP_HOME=/Users/<username>/stanford-corenlp-full-2016-10-31/

# On linux
export CORENLP_HOME=/home/<username>/stanford-corenlp-full-2016-10-31/

```

In Python
=========

```
>>> import corenlp
>>> with corenlp.client.CoreNLPClient(annotators="tokenize ssplit".split()) as client:
...     ann = client.annotate(text)
... 
[pool-1-thread-4] INFO CoreNLP - [/0:0:0:0:0:0:0:1:55475] API call w/annotators tokenize,ssplit
Chris wrote a simple sentence that he parsed with Stanford CoreNLP.
>>> sentence = ann.sentence[0]
>>> 
>>> [token.word for token in sentence.token]
['Chris', 'wrote', 'a', 'simple', 'sentence', 'that', 'he', 'parsed', 'with', 'Stanford', 'CoreNLP', '.']

```