fpassthru broken on OS X
Update: bug is now fixed. See [my new post][3]
I’ve gotten a few strange bug reports about broken downloads in SabreDAV over a certain size. I was admitedly a bit sceptical, because this always worked pretty well.
I’ve unable so far to confirm this on other systems, but I’m very curious what other people are seeing.
To test, create a large file. The following statement creates a 5GB file:
dd if=/dev/zero of=5gigs bs=1024 count=5242880
Next, we want to stream this with fpassthru()
, the following script should
do this:
<?php
$file = __DIR__ . '/5gigs';
$h = fopen($file, 'r');
fpassthru($h);
I’ve tried this at a few sizes:
- At 1.5 GB the download succeeds
- At 3 GB and 6GB the script crashes and I’m not getting an error message.
- At 5GB I’m consistently only receiving 1 GB worth of data, before the script dies without warning.
This is tested on PHP 5.5.7 on OS X 10.9, and the behavior is the same through apache, or on the command line. If the previous file was called ‘foo.php’, you can easily test by running:
php foo.php > output
Output buffering and max execution time is off. Before I’m reporting a bug to bugs.php.net, I would love to know if I’m the only one out there seeing this.
So unless I’m making a mistake somewhere, else:
Do not use fpassthru on OS X if you want to support large files!
The following two workarounds always workthough:
<?php
$file = __DIR__ . '/5gigs';
$h = fopen($file, 'r');
while(!feof($h)) {
echo fread($h, 4096);
}
And my personal preference:
<?php
$file = __DIR__ . '/5gigs';
$h = fopen($file, 'r');
file_put_contents('php://output', $h);
PHP Bug
Now confirmed on 5.5.9 and opened a php bug report: https://bugs.php.net/bug.php?id=66736
Comments
SM •
fpassthru (eventually) uses size_t as file size if stream is mmap-able (which regular files usually are). size_it is 32-bit in 32-bit binaries, so if your PHP build is 32-bit, it's up to 4G. Or, more precisely, you get 5G number stuffed into 4G variable, which gives you ~1G. fread, etc. don't care too much about overall file size as they just try to read in small chunks. File offset is probably wrong somewhere but fread doesn't care.
Evert •
My PHP_INT_SIZE = 8, so I'm definitely good in this regard. I don't think there's any reason these days to compile to 32-bit.
BenBen •
Did you try readfile($file)? Its my favorite way of tackling this sort of things and according to http://www.garfieldtech.com... it should be working and be performant
Evert •
In my specific usecase, there is no 'filename', there's just a stream that can come from anywhere. See here:
https://github.com/fruux/sa...
Guest •
Aha, that makes sense :)
Daniel Lowrey •
If you're serving files of this size you had better be using the X-Sendfile header (or equivalent). Passing large files through userland like this is a serious performance WTF. No one should be doing this in the first place.
Evert •
Streams can originate from everywhere, and I don't discriminate where. I agree that it would be a good bonus feature to also allow files to be served using X-Sendfile *if* the stream represents a file on a filesystem, but this is not the assumption.
Daniel Lowrey •
Ah, makes sense. I was initially horrified by the thought of someone streaming multi-Gb filesystem resources in chunks through userland :)