Question: I'm having difficulties dealing with character encoding. I'm trying to scrape the following url:

```
http://www.google.com/movies?near=Montreal&date=0

```

My code looks like this:

```
var http = require('http');
var url = require('url');
var Iconv  = require('iconv').Iconv;

var location = 'montreal';

var googleMovies = url.parse("http://www.google.com/movies?near=" + location);

var req = http.request(googleMovies, function(response) {
    var str = '';
    response.on('data', function(chunk) {
        str += chunk;
    });
    response.on('end', function() {

        var iconv = new Iconv('latin1', 'UTF-8');
        str = iconv.convert(str).toString();

        console.log(str);
    });
});
req.end()

```

I've first tried without:

```
    var iconv = new Iconv('latin1', 'UTF-8');
    str = iconv.convert(str).toString();

```

but that was causing the � characters. 

I've tested the source listed above on this page:

<http://nlp.fi.muni.cz/projects/chared/>

and it seem to detect it as latin1, but things could be wrong.

Answer: 
The � characters come from the concatenation:

```
response.on('data', function(chunk) {
    str += chunk;
});

```

This converts each `chunk` to a `String` with the [default `encoding` of `utf8`](http://nodejs.org/api/buffer.html#buffer_buf_tostring_encoding_start_end). Any sequences in the `Buffer`s that aren't valid as UTF-8 will be lost and replaced by � at this point.

You'll want to leave the `chunk`s as `Buffer`s until after the `convert()`. They can be collected in an `Array` and combined with [`Buffer.concat()`](http://nodejs.org/api/buffer.html#buffer_class_method_buffer_concat_list_totallength).

```
var chunks = [];

response.on('data', function (chunk) {
    chunks.push(chunk);
});

response.on('end', function () {
    var iconv = new Iconv('latin1', 'UTF-8');
    var str = iconv.convert(Buffer.concat(chunks)).toString();
    console.log(str);
});

```
A `Buffer`'s default encoding is UTF-8, which is a variable-width encoding system. Characters after the ASCII range are encoded using multiple bytes. If you're receiving latin1-specific characters (codepoints > 127), they'll have the first bit set, which a UTF-8 decoder will consider as a multi-byte character, eventually resulting in an unmapped codepoint (displayed as �). 

`iconv` has a streaming decoder that you can pipe your response stream to.

```
http.request(googleMovies, function (response) {
    var iconv = new Iconv('latin1', 'UTF-8');
    response.pipe(iconv).pipe(process.stdout);
    //or response.pipe(iconv).on('data', console.log);
}).end();

```