title: R on Windows: character encoding hell
body: <p>I am trying to import a CSV encoded as OEM-866 (Cyrillic charset) into R on Windows. I also have a copy that has been converted into UTF-8 w/o BOM. Both of these files are readable by all other applications on my system, once the encoding is specified.</p>
  
  <p>Furthermore, on Linux, R can read these particular files with the specified encodings just fine. I can also read the CSV  on Windows IF I do not specify the "fileEncoding" parameter, but this results in unreadable text. When I specify the file encoding on Windows, I always get the following errors, for both the OEM and the Unicode file:</p>
  
  <p>Original OEM file import:</p>
  
  <pre><code>&gt; oem.csv &lt;- read.table("~/csv1.csv", sep=";", dec=",", quote="",fileEncoding="cp866")   #result:  failure to import all rows
  Warning messages:
  1: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  :
    invalid input found on input connection '~/Revolution/RProject1/csv1.csv'
  2: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  :
    number of items read is not a multiple of the number of columns
  </code></pre>
  
  <p>UTF-8 w/o BOM file import:</p>
  
  <pre><code>&gt; unicode.csv &lt;- read.table("~/csv1a.csv", sep=";", dec=",", quote="",fileEncoding="UTF-8") #result:    failure to import all row
  Warning messages:
  1: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  :
    invalid input found on input connection '~/Revolution/RProject1/csv1a.csv'
  2: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  :
    number of items read is not a multiple of the number of columns
  </code></pre>
  
  <p>Locale info:</p>
  
  <pre><code>&gt; Sys.getlocale()
     [1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"
  </code></pre>
  
  <p>What is it about R on Windows that is responsible for this? I've pretty much tried everything I could by this point, besides ditching windows.</p>
  
  <p>Thank  You</p>
  
  <p>(Additional failed attempts):</p>
  
  <pre><code>&gt;Sys.setlocale("LC_ALL", "en_US.UTF-8") #OS reports request to set locale to "en_US.UTF-8" cannot be honored
  &gt;options(encoding="UTF-8") #now nothing can be imported  
  &gt; noarg.unicode.csv &lt;- read.table("~/Revolution/RProject1/csv1a.csv", sep=";", dec=",", quote="")   #result: mangled cyrillic
  &gt; encarg.unicode.csv &lt;- read.table("~/Revolution/RProject1/csv1a.csv", sep=";", dec=",", quote="",encoding="UTF-8") #result: mangled cyrillic
  </code></pre>