Why is sed outputting no text? I have a text file that I need to remove double quotes from. I found this question that suggested using the following:

sed 's/"//g' &lt; infile &gt; outfile


I understand what it's doing as I'm familiar with Vim. But the output file has zero bytes in it after the command is run.

I did try the other suggestion from that question:

tr -d \" &lt; infile &gt; outfile


This results in an error:

tr: Illegal byte sequence


Note that when I open the file in MacVim and use

:%s /"//g


it works perfectly.

Is there some difference in macOS's sed? Any other suggestion on how I can do this?
Apparently, the file is in UTF-16 format, but most of the text-manipulation tools in macOS don't know how to work with that and will garble files if they try. UTF-16 uses two bytes per character; for plain-ASCII characters like ", it's the single-byte ASCII code followed by a null byte. Something like tr -d \" will delete the first byte, leaving a stray null byte and throwing the entire rest of the file off by one byte.

I think you can get perl to work in UTF-16 (see here), but I'm not sure of the ins and outs. This might work (based on this):

perl -pe 'BEGIN { binmode $_, ":raw:encoding(UTF-16LE)" for *STDIN, *STDOUT }; s/"//g;' infile &gt;outfile


You could also convert it to UTF-8 and back with 

iconv -f UTF-16LE -t UTF-8 infile | tr -d \" | iconv -f UTF-8 -t UTF-16LE &gt;outfile


If the output doesn't need to be in UTF-16, you can leave the last iconv out.