Products and Services
Downloads
Store
Support
Education
Partners
About
Oracle Technology Network
Name: boT120536 Date: 02/26/2001
Development, JDK: "java version JDK.1.6_Borland"
Runtime, JRE: 1.1.6_008 (downloaded from your site - SUN )
the problem exist on version 1.2 and 1.3 too.
The problems are:
- it is impossible to unzip a "zip" file that contains a file with a real
size over 2Gb
- it is impossible to continue with the next "zip" entry if there is a
problem with the current entry.
- note: I know that it is impossible to do a zip file over 4Gb (but it
is possible to compress more than 4Gb of data if the result zip file
has a size less than 4Gb...). It is the norm for eip... But I'm not
sure this norm says that it is impossible to compress a file between
2Gb and 4Gb...
The test is:
1) - 6Gb of free space on your disk is needed
- 3Gb of used space on your disk is needed
2) Do a big zip file "big.zip" with 3 files a.dat, b.dat and c.dat :
- a.dat and c.dat must have a small size (less than 1Mb; 1Kb is fine)
- b.dat must have a big size (between 2Gb and 4Gb; 2.5Gb is fine. Note:
1Gb = 1024Mb = 1,048,576Kb = 1073741824 bytes)
- the name of the 3 files are important (there must be a file before
and after the big file b.dat in the zip file).
- for zipping operation, I have download the last version of the
shareware tools "Winzip" (another zip tools can be used if it is
compatible with your zip package - I have not try to zip the 3
files with the zip package in "java.utils" - I'm not sure it would
work...). Also, I have used this tools to create the b.dat (all my disk
with no compression).
3) Unzip "big.zip" with the zip package in "java utils".
The results are :
- a.dat is correctly unzipped (if it is not a.dat then it is c.dat; it
depends on how the file are stored in the "zip" file; so, it depends
on the tools used for zipping)
- b.dat is not unzipped (a ZipException "invalid entry size" is thrown)
- c.dat can not be unzipped since there was a problem with the previous
entry b.dat (if it is not c.dat then it is a.dat; ...)
(Review ID: 116880)
======================================================================
Name: boT120536 Date: 04/22/2001
C:\data\JavaInvestigations\zip>c:\jbuilder3\java\bin\java -version
java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0-C)
Java HotSpot(TM) Server VM (build 2.0fcs-E, mixed mode)
I can't seem to write large zip files.
The code below _should_ generate a zip file
with 100,000 entries.
=============================================
import java.util.zip.*;
import java.io.*;
public class ZipTest
{
private ZipOutputStream zip;
public static void main (String[] args) throws Exception
{
ZipTest t = new ZipTest ();
t.doIt();
}
public void doIt () throws Exception
{
zip = new ZipOutputStream (new FileOutputStream ("D:\\temp\\temp.zip"));
zip.setMethod(ZipOutputStream.STORED);
for (int i = 0; i < 100000; ++i)
{
byte[] data = new byte[1000];
String entryName = "Fred" + i;
CRC32 crc32 = new CRC32();
crc32.update(data, 0, data.length);
ZipEntry zipentry = new ZipEntry (entryName);
zipentry.setSize(data.length);
zipentry.setMethod(ZipEntry.STORED);
// zipentry.setTime();
zipentry.setCrc(crc32.getValue());
zip.putNextEntry (zipentry);
zip.write (data, 0, data.length);
zip.closeEntry();
}
zip.close();
}
}
========================================================
When I compile it, it runs to completion with no reported
errors. It generates a file that is _about_ the correct
size (106 MB). But, the file only reports 34464 entries
when opened. The last 63000+ entries are not accessible.
This is also a problem with JDK1.2.2
(Review ID: 121141)
======================================================================
EVALUATION
File sizes in zip files are generally stored as unsigned 32-bit quantities.
which do not fit into a java `int'. A `long' should have been used instead.
For example,
public synchronized int getTotalOut()
should have been
public synchronized long getTotalOut()
However, it is too late to change the API, especially for such a "small"
fix as to increase the size from 2GB to 4GB.
Long term, we want to support file sizes larger than 4GB, and this will
require major changes to the Zip File format. PKZIP implements such
an extension, and we should investigate their file format.
However, we _can_ get support for sizes up to 4GB.
Although the return type of getTotalOut and getTotalIn are semantically
incorrect, there is no data loss. We simply have to reinterpret
the value as unsigned. So it looks like we can fix this without making
any API changes. Most of the code in the zip package already uses "long"
to represent unsigned 32-bit values, so few changes are necessary.
One very good reason to fix this is that currently a user can use the "jar"
command to create a jar file that cannot be read by "jar".
###@###.### 2003-04-29
PUBLIC COMMENTS
Add support for zip files and jar files between 2GB and 4GB in size.
WORK AROUND
Name: boT120536 Date: 02/26/2001
There is no way to bypass this problem whitout rebuilding the zip package
in the "java.utils".
the problem is in the class "ZipInputStream":
private void readEnd(ZipEntry e) throws IOException {
...
if (e.size != inf.getTotalOut()) {
throw new ZipException(
"invalid entry size (expected " + e.size + " but got " +
inf.getTotalOut() + " bytes)");
...
}
--> e.size is a long on 64 bits (so accepting size over 2Gb)
inf.getTotalOut() returns an int on 32 bits (so not
accepting size over over 2Gb)
--> if all data for the current entry have been read and if
an error occured, the attribute "entry" is not set to
"null" so it's impossible to get the next zip entry
("getNextEntry" call "closeEntry" that call "read" that call
"ReadEnd" that threws an exception for the previous zip entry
that causes the error... But if I call "getNextEntry", it is
because I want to continue with the next zip entry, and I hope
this zip entry is valid...)
also :
- the native method "GetTotalOut" used in class "Inflater"
- the native method "InflatesBytes" used in class "Inflater"
They return a signed int on 32 bits (so a value between -2G
and +2G). I think they should return a long on 64 bits...
======================================================================
CONVERTED DATA
BugTraq+ Release Management Values
COMMIT TO FIX:
tiger
FIXED IN:
tiger
INTEGRATED IN:
tiger
tiger-b20