Writing to file

I have a program which generates many lines of data where each line
includes an integer and two floats. I want to know the fastest way to
write these generated lines to a file. Currently, I am writing to a
binary file as follows:
FileOutputStream fos = new FileOutputStream(fileName);
DataOutputStream dos = new DataOutputStream(fos);
for (n : N) {
dos.writeInt(n.id);
dos.writeFloat(x.floatValue());
dos.writeFloat(y.floatValue());
}

However it seems that this approach is very slow. Can I use any kind
of buffering technique to speed up writing to binary file?

Advertisements

On Wednesday, November 30, 2011 11:31:33 AM UTC-8, sara wrote:
> Hi All,
>
> I have a program which generates many lines of data where each line
> includes an integer and two floats. I want to know the fastest way to
> write these generated lines to a file. Currently, I am writing to a
> binary file as follows:
> FileOutputStream fos = new FileOutputStream(fileName);
> DataOutputStream dos = new DataOutputStream(fos);
> for (n : N) {
> dos.writeInt(n.id);
> dos.writeFloat(x.floatValue());
> dos.writeFloat(y.floatValue());
> }
>
> However it seems that this approach is very slow. Can I use any kind
> of buffering technique to speed up writing to binary file?

Have you considered reading the API docs?
<http://docs.oracle.com/javase/7/docs/api/java/io/BufferedOutputStream.html>

Note that you will need to build the DataOutputStream on top of the BufferedOutputStream rather than the other way around.

As to whether this helps performance, what measurements have you done, and how do you know that the result is "very slow"? Compared to what?

What were the conditions of the measurements (other load on the system, hard drive configuration, etc.)?

It will generate you code to do this. Make sure you ask for buffered
if you want speed.

Get rid of Float objects and just use pure float primitives if you
want speed.

There is not much overhead, just flipping the little endian Intel uses
internally to big endian.

--
Roedy Green Canadian Mind Productshttp://mindprod.com
For me, the appeal of computer programming is that
even though I am quite a klutz,
I can still produce something, in a sense
perfect, because the computer gives me as many
chances as I please to get it right.

On 11/30/2011 2:31 PM, sara wrote:
> I have a program which generates many lines of data where each line
> includes an integer and two floats. I want to know the fastest way to
> write these generated lines to a file. Currently, I am writing to a
> binary file as follows:
> FileOutputStream fos = new FileOutputStream(fileName);
> DataOutputStream dos = new DataOutputStream(fos);
> for (n : N) {
> dos.writeInt(n.id);
> dos.writeFloat(x.floatValue());
> dos.writeFloat(y.floatValue());
> }
>
> However it seems that this approach is very slow. Can I use any kind
> of buffering technique to speed up writing to binary file?

You can use a BufferedOutputStream between the DataOutputStream
and the FileOutputStream.

But I am a bit skeptical about the choice of format for
persisting data.

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!