php unzipping huge files

Question

I'm trying to unzip a huge file (400+M compressed well over 4G unzipped) using php zip archive. I'm only unzipping one csv file in the zipped file.. the file I am interested in, unzipped, is over 4G.. I get as far as 3 records from the end of the file and the process takes off into lala land.. I mean the process simply goes on.. no output.. no errors no looping it just goes on... I haven't a clue as to what its doing.. my code is simple:

$zip = new ZipArchive;
$res = $zip->open($file);
if ($res === TRUE) 
{
   $num = $zip->numFiles;
   for($i = 0; $i < $zip->numFiles; $i++) 
   {
      $filename = $zip->getNameIndex($i);
  // if its the file I want then...
      $content = '';
  // my output file  .. yes I've already checked to make sure the dir exists
      $unzipped = fopen($dir ."/2" . $filename  , 'wb');         
      $fp = $zip->getStream($filename);
      if(!$fp) exit("failed\n");
      while (!feof($fp))
      {
    $chunkSize = 10240;
    $contents = fread($fp, $chunkSize);
        $fwrite = fwrite($unzipped, $contents);
      }
      fclose($fp);
      fclose($unzipped);
    }

    $zip->close();
    fclose($filename);

}

I've removed the write statements that go to another file to track progress. I get most of the file out.. (as I said 3 records short of the entire file)... but the process seems to go off somewhere.. it gets happens on the fread I just can't figure out what is happening.. It hasn't reached the eof.. the source is intact (checked with is_source($fp) just before the fread.. it throws no errors.. closing the browser doesn't stop it.. can't even stop apache.. have to shut down to end it...

any ideas??

thanks

goat · Accepted Answer · 2012-05-20 03:23:05Z

1

sounds like a bug to be honest(in php).

you might want to try outputting calls to memory_get_usage() to help you debug. But, also, see stream_copy_to_stream() because you can get rid of all that looping cruft. It might also be intresteting to keep a running total of bytes written to see if the number where things go awry look suspicious.

edited May 20, 2012 at 3:23

answered May 20, 2012 at 3:16

goat

32k7 gold badges76 silver badges98 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Blizz Over a year ago

According to someone at the documentation page the function appears to use quite a bit of memory on its own. So if that is true and memory is the issue here, this might not be the solution.

Bridget Over a year ago

Hi Chris, tried it with interesting results, it didn't copy the entire file, but it didn't go off into lala land either, which kind of leads me to believe that the first problem may be the initial zip stream.. It may not be getting everything.. but that's a different question and I need to so some research before posting.. thanks again.

Nick Clark · Accepted Answer · 2012-05-20 06:13:13Z

1

This is a stab in the dark, but try increasing the amount of memory the script can allocate. I've had similar problems when using the gzip functions and I had to make this change.

ini_set('memory_limit', '512M');

answered May 20, 2012 at 6:13

Nick Clark

4,4674 gold badges25 silver badges25 bronze badges

1 Comment

Bridget Over a year ago

Hi Nick, Thanks for the suggestion, tried it but it made no dif.. but thanks anyway

elad · Accepted Answer · 2014-07-06 23:32:57Z

0

$filename = '/media/file.gz';

$unzipped_content = '';   
$zd = gzopen($filename, "r");
while ($zip_file = gzread($zd, 10000000)){
    $unzipped_content.= $zip_file;
}
gzclose($zd);

echo $unzipped_content;

answered Jul 6, 2014 at 23:32

elad

211 bronze badge

Collectives™ on Stack Overflow

php unzipping huge files

3 Answers 3

2 Comments

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

1 Comment

Comments

Related