Page MenuHomePhabricator

Integrate a modern php REPL shell with MediaWiki
Open, Needs TriagePublic

Description

Currently MediaWiki provides a custom console via eval.php, which is integrated with the wiki's environment but otherwise rather poor (no proper readline support, no fatal error handling, the P part of REPL is missing, no tab autocompletion, no reflection...). Vagrant provides phpsh / hhvmsh which is somewhat better but still mediocre. It would be nice to replace these with a great REPL like PsySH or Boris.

Landed as maintenance/shell.php.

Things to maybe improve listed by @Tgr:

Details

Related Gerrit Patches:
mediawiki/vagrant : masterDownload PHP docs for PsySH
mediawiki/core : masterDrop in replacement of eval.php based on psysh

Event Timeline

Tgr created this task.Nov 4 2015, 8:10 AM
Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added a project: MediaWiki-Vagrant.
Tgr added a subscriber: Tgr.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 4 2015, 8:10 AM
Tgr renamed this task from Integrate a modern php REPL shell with mediawiki to Integrate a modern php REPL shell with MediaWiki.Nov 4 2015, 8:10 AM
Tgr set Security to None.
ori awarded a token.Nov 4 2015, 9:06 AM
Tgr added a comment.Aug 24 2016, 10:08 PM

https://gerrit.wikimedia.org/r/#/c/306487 is about integrating psysh with vagrant.

Change 334217 had a related patch set uploaded (by Hashar):
(WIP) Drop in replacement of eval.php based on psysh

https://gerrit.wikimedia.org/r/334217

hashar added a subscriber: hashar.Jan 25 2017, 11:01 PM

A very rough basic evening hack to wrap psysh as a maintenance script. The trick is that it supports crafting a scope and hence just passing $GLOBALS seems to work for me.

Lets follow up on Gerrit change.

Tgr added subscribers: srishakatux, Anomie, Elitre and 2 others.

Merged T69924. We should make sure that the new shell is Unicode-friendly.

hashar added a comment.Feb 3 2017, 9:47 AM

eval.php with some version of readline did not accept unicode. By @Nikerabbit on T69805#730015 and spawned later as T69924 . Anomie had a repro case which is to spawn readline to read some entry, trying to enter unicode such as or é fails either with PHP or HHVM (on terbium which is Jessie using php -a and hhvm -a).

Both maintenance/eval.php and psysh use readline() and can't handle unicode characters.

That works fine on my Mac laptop, with PHP compiled with --with-libedit. My home Jessie fails. Might be worth keeping T69924 open.

Merged T69924. We should make sure that the new shell is Unicode-friendly.

Unless fixing eval.php Unicode support is impossible/disproportionately hard, having a specific bug open is IMO preferable. hashar's test indicate the workaround may be achievable, no?

This proposal is selected for the Developer-Wishlist voting round and will be added to a MediaWiki page very soon. To the subscribers, or proposer of this task: please help modify the task description: add a brief summary (10-12 lines) of the problem that this proposal raises, topics discussed in the comments, and a proposed solution (if there is any yet). Remember to add a header with a title "Description," to your content. Please do so before February 5th, 12:00 pm UTC.

Tgr added a comment.Feb 4 2017, 10:46 PM

hashar's test indicate the workaround may be achievable, no?

Workarounds on the level of "compile a different version of PHP/readline", but I doubt we want to go there. PsySH can use other input handlers than readline (meant for Windows shells and might be worse than even non-Unicode-aware readline, but maybe worth a try).

Anyway, feel free to reopen. IMO there is always time to do that if PsySH does not work out.

Change 334217 merged by jenkins-bot:
Drop in replacement of eval.php based on psysh

https://gerrit.wikimedia.org/r/334217

Tgr added a comment.EditedFeb 24 2017, 1:28 AM

(deleted, wrong task)

Tgr added a comment.Feb 24 2017, 9:18 PM

Things to maybe improve:

  • Unicode support (T69924 - seems to be a HHVM bug)
  • get the dependencies to WMF production (https://gerrit.wikimedia.org/r/#/c/339584/)
  • use personal readline configuration
  • preserve readline history between sessions (per Configuration::getReadline() this should happen but does not)
  • install the PHP docs (so that help and reflection in PsySH works)
  • integrate with PSR-3 logging (add a flag to show what has been logged during a command)
  • add __debugInfo() to MediaWiki classes (used by VarDumper which is used by PsySH)
  • add a custom tab completion matcher, to support __get-based stuff. (Not sure if there is a good use case for that now. In the future, maybe services defined by extensions?)
Tgr added a comment.Feb 24 2017, 9:19 PM

@hashar do you want to demo this at the next CREDIT?

@hashar do you want to demo this at the next CREDIT?

That is a very good idea! Please step in, I just send a basic patch and you did all the work of reaching out to upstream and polishing up the draft patch I sent in a hurry. You are as legitimate (if not more) than me to talk about psysh :-] Moreover I can't attend CREDIT, my Wednesday evenings are busy.

Thanks for the summary of things to improve. Looks like most of them are related to readline/libedit. I get the history saved in ~/.config/psysh/psysh_history based on XDG recommendation, we should probably tweak the name :-}

Should we spam some new sub tasks or keep a single one to polish things up?

Tgr added a comment.Feb 28 2017, 12:16 AM

Most of these are too small for a separate task IMO. I'll file one for the PSR-3 stuff.
The history might just be a vagrant permission issue.

hashar updated the task description. (Show Details)Feb 28 2017, 8:41 PM

Another thing I noticed this morning is that shell.php runs a few things once the session is completed:

  • DeferredUpdates::doUpdates();
  • $lbFactory->commitMasterChanges( 'doMaintenance' );

That is because it is a regular maintenance script and that might lead to surprising effects.

Tgr added a comment.EditedMar 1 2017, 9:44 AM

Other small stuff to fix:

  • throws P5001 on vagrant on start (www-data needs to be able to write /home/vagrant/.config/psysh/psysh_history)
  • throws P5002 on vagrant on exit if redis was used (can use new Message('').'' to test)

These should be fixed in https://gerrit.wikimedia.org/r/#/c/306487/ probably.

@EBernhardson pointed out that mwrepl can set breakpoints which is a cool feature. mwrepl starts the HHVM debugger, which normally comes with its own shell, so not sure if it's possible to interface with that, but it's something to look into.

Tgr moved this task from Backlog to Pending on the User-Tgr board.
Tgr updated the task description. (Show Details)Aug 21 2017, 11:47 PM
Tgr moved this task from Pending to Next on the User-Tgr board.Aug 21 2017, 11:51 PM
Tgr moved this task from Next to Pending on the User-Tgr board.
Tgr updated the task description. (Show Details)Oct 20 2017, 5:11 AM
Tgr updated the task description. (Show Details)

Change 385317 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/vagrant@master] Download PHP docs for PsySH

https://gerrit.wikimedia.org/r/385317

Change 385317 merged by jenkins-bot:
[mediawiki/vagrant@master] Download PHP docs for PsySH

https://gerrit.wikimedia.org/r/385317

Tgr updated the task description. (Show Details)Nov 13 2017, 8:03 AM
Tgr updated the task description. (Show Details)
Tgr added a comment.Nov 14 2017, 12:52 AM

PsySH is now available on production servers but is somewhat problematic to use:

  • it uses ~/.local/share/psysh to store configuration, readline data etc. Since the expected way to execute it is mwscript shell.php which runs as www-data, www-data needs to write that directory. That seems problematic: making it world-writeable seems insecure (there is probably some way to trick PsySH into executing code via config settings), transferring ownership to www-data is also somewhat insecure plus leaves you with a file in your homedir that you can't write anymore.
  • php5-sqlite is needed to read PHP documentation, but is not available on our servers.
  • PHP documentation has to be downloaded manually (not problematic, just an annoyance).
Tgr updated the task description. (Show Details)May 28 2018, 2:57 PM
Tgr updated the task description. (Show Details)