Page MenuHomePhabricator

HHVM does not recognize compress.bzip2 streams
Closed, ResolvedPublic

Description

It seems that our HHVM was built without support for compress.bzip2 streams:

php

<?php
print_r(stream_get_wrappers());
?>
Array
(

[0] => compress.zlib
[1] => data
[2] => file
[3] => glob
[4] => http
[5] => https
[6] => phar
[7] => php
[8] => zip

)

missing is compress.bzip2, needed for dumps to run on HHVM.

Event Timeline

ArielGlenn raised the priority of this task from to Needs Triage.
ArielGlenn updated the task description. (Show Details)
ArielGlenn added projects: HHVM, Dumps-Generation.
ArielGlenn added a subscriber: ArielGlenn.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 28 2015, 9:46 AM

this is with version 3.6.5+dfsg1-1+wm6 on trusty.

Joe added a subscriber: Joe.EditedSep 28 2015, 10:47 AM

This is true with FB's official package as well:

joe@gizmo:~$ hhvm --version
HipHop VM 3.9.1 (rel)
Compiler: tags/HHVM-3.9.1-0-g0f72cfc2f0a01fdfeb72fbcfeb247b72998a66db
Repo schema: 6416960c150a4a4726fda74b659105ded8819d9c
joe@gizmo:~$ hhvm test.php
Array
(
    [0] => compress.zlib
    [1] => data
    [2] => file
    [3] => glob
    [4] => http
    [5] => https
    [6] => phar
    [7] => php
    [8] => zip
)
bd808 added a subscriber: bd808.

@Joe, is this something that could/should be backported for us? Or would you rather we wait and pick up the release that has this fix in it?

Change 242773 had a related patch set uploaded (by BryanDavis):
Backport of D2486378: Implement compress.bzip2:// stream wrapper

https://gerrit.wikimedia.org/r/242773

Change 242773 had a related patch set uploaded (by BryanDavis):
Backport of D2486378: Implement compress.bzip2:// stream wrapper
https://gerrit.wikimedia.org/r/242773

Patch applies cleanly but I haven't tested it in our build.

ArielGlenn moved this task from Backlog to Active on the Dumps-Generation board.Oct 1 2015, 8:29 AM
Reedy triaged this task as Normal priority.Oct 8 2015, 3:01 PM
Reedy set Security to None.

Change 242773 merged by Ori.livneh:
Backport of D2486378: Implement compress.bzip2:// stream wrapper

https://gerrit.wikimedia.org/r/242773

ori added a subscriber: ori.Oct 26 2015, 10:32 AM

@bd808:

home/ori/build-new/hhvm-3.6.5+dfsg1/hphp/runtime/ext/bz2/ext_bz2.cpp:40:11: error: ‘req’ does not name a type
   virtual req::ptr<File> open(const String& filename,
           ^
/home/ori/build-new/hhvm-3.6.5+dfsg1/hphp/runtime/ext/bz2/ext_bz2.cpp:70:3: error: cannot declare variable ‘HPHP::{anony
mous}::s_bzip2_stream_wrapper’ to be of abstract type ‘HPHP::{anonymous}::BZ2StreamWrapper’
 } s_bzip2_stream_wrapper;
   ^
/home/ori/build-new/hhvm-3.6.5+dfsg1/hphp/runtime/ext/bz2/ext_bz2.cpp:39:15: note:   because the following virtual funct
ions are pure within ‘HPHP::{anonymous}::BZ2StreamWrapper’:
 static struct BZ2StreamWrapper : Stream::Wrapper {
               ^
In file included from /home/ori/build-new/hhvm-3.6.5+dfsg1/hphp/runtime/ext/bz2/ext_bz2.cpp:20:0:
/home/ori/build-new/hhvm-3.6.5+dfsg1/hphp/runtime/base/stream-wrapper.h:40:26: note:    virtual HPHP::SmartPtr<HPHP::Fil
e> HPHP::Stream::Wrapper::open(const HPHP::String&, const HPHP::String&, int, const HPHP::SmartPtr<HPHP::StreamContext>&
)
   virtual SmartPtr<File> open(const String& filename,
                          ^
/usr/bin/cmake -E cmake_progress_report /home/ori/build-new/hhvm-3.6.5+dfsg1/CMakeFiles

ori, you said in irc that the new package would be available on the snapshots last night, but I don't see it, or at least, if the new package is 3.6.5+dfsg1-1+wm7 it shows the same behavior as reported initially, i.e. no compress.bzip2. Am I doing something wrong? This is on snapshot1004.

ori added a comment.Oct 27 2015, 6:10 AM

ori, you said in irc that the new package would be available on the snapshots last night, but I don't see it, or at least, if the new package is 3.6.5+dfsg1-1+wm7 it shows the same behavior as reported initially, i.e. no compress.bzip2. Am I doing something wrong? This is on snapshot1004.

Yeah, I hit another snag. I'm working on it at the moment, will have an update shortly.

ori added a comment.Oct 27 2015, 6:57 AM

I am stuck on this error:

/home/ori/build-old/hhvm-3.6.5+dfsg1/hphp/runtime/vm/jit/type.cpp: In member function ‘std::string HPHP::jit::Type::toString() const’:
/home/ori/build-old/hhvm-3.6.5+dfsg1/hphp/runtime/vm/jit/type.cpp:253:13: note: variable tracking size limit exceeded with -fvar-tracking-assignments, retryin
g without
 std::string Type::toString() const {
             ^
mark raised the priority of this task from Normal to High.Oct 27 2015, 2:19 PM
mark added a subscriber: mark.

Ori, any luck on this?

So the word is that the backport of the patch is essentially not doable and so this must wait on the new version of HHVM, 3.9, being packaged for WMF.

Rumour has it that there are 3.11 packages someplace, but a quick check shows they are not yet in the repo, so must still be undergoing testing.

Krinkle moved this task from Backlog to Defect on the HHVM board.Feb 21 2016, 5:54 PM
Krinkle moved this task from Defect to Blocked on the HHVM board.
Krinkle moved this task from Blocked to Defect on the HHVM board.Feb 21 2016, 5:57 PM
ori closed this task as Resolved.Mar 11 2016, 7:51 PM
ori claimed this task.

@ArielGlenn, 3.12.1 is now deployed everywhere, and:

[mw1099:~] $ hhvm --php -r 'var_dump( in_array( "compress.bzip2", stream_get_wrappers() ) );'
bool(true)