Page MenuHomePhabricator

Add support for Pygments ran as CGI application via HTTP
Open, Needs TriagePublic

Description

Hello! On some shared hostings Pygments can't be run because the shell execution was limited by "security" reasons and I can't enable it. However, I have built my own workaround which turns Pygments into CGI-BIN application I can run via HTTP.

It's the patch to SyntaxHighlight.php which adds the flag $wgPygmentsUseCgiProxy. When you'll set it into 1, the wgPygmentizePath will be interpreted as HTTP link and will be executed via cURL.

It's the CGI script which will execute Pygments and will forward input data into it, and will return back the result:

Event Timeline

Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptNov 16 2018, 1:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Wohlstand updated the task description. (Show Details)Nov 16 2018, 1:40 PM

Hi @Wohlstand, thanks for taking a look at the code!

You are very welcome to use developer access to submit the proposed code changes as a Git branch directly into Gerrit which makes it easier to review them quickly and provide feedback.
If you don't want to set up Git/Gerrit, you can also use the Gerrit Patch Uploader. Thanks again!

Oh, nice! I have sent the raw patch because I hadn't any sort of understanding where to submit this, I'll try out this...

Wohlstand added a comment.EditedNov 17 2018, 11:59 PM

Sent just now:
https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/SyntaxHighlight_GeSHi/+/474518/

Note: this https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/SyntaxHighlight_GeSHi/+/474517/ that I have sent previously, is junk, contains a typo in README I have fixed quickly and I have sent an update. Discard this, please.

Change 474518 had a related patch set uploaded (by TheDJ; owner: Wohlstand):
[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Allow Pygments to be run via CGI

https://gerrit.wikimedia.org/r/474518

Wohlstand added a comment.EditedJan 15 2019, 3:04 PM

Hello! I want to ask you how are you? I have seen your Jenkins bot said "Verify -1" because of some weird "quibble-vendor-mysql-hhvm-docker" failure... Can anyone poke that bot to re-check the patch?

Hello!
Almost half of the year has been passed. Are you too busy and can't review this patch and give a rate on it?

Change 474518 had a related patch set uploaded (by Aklapper; owner: Wohlstand):
[mediawiki/extensions/SyntaxHighlight_GeSHi@master] Allow Pygments to be run via CGI

https://gerrit.wikimedia.org/r/474518

I rebased the patch, that's why there was another notification here for https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/SyntaxHighlight_GeSHi/+/474518/

Hi @Wohlstand, sorry that this has not seen a review yet. In theory, see https://www.mediawiki.org/wiki/Gerrit/Code_review/Getting_reviews for more information.
According to https://www.mediawiki.org/wiki/Developers/Maintainers , "SyntaxHighlight_GeSHi" is stewarded by the Editing Team so they'd be the one to ask if your proposed patch is in scope and could be accepted.

Wohlstand added a comment.EditedJul 17 2019, 2:03 PM

The reason for that V-1 was NOT my issue, it happened because of another issue that was in the state that was in a moment where I have submitted my patch. I think I'll try to re-poke it to trigger the re-check...

I see here some sort of code style / static analysis failure... Okay, will re-check this and will submit an update...

Wohlstand added a comment.EditedJul 30 2019, 10:31 AM

Sorry for waiting, I did the change, however, I can't submit it with no way:

via git review -R I getting next reply:

$ git review -R
remote: 
remote: Processing changes: refs: 1
remote: Processing changes: refs: 1, done            
remote: Pushing to refs/publish/* is deprecated, use refs/for/* instead.        
To ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/SyntaxHighlight_GeSHi.git
 ! [remote rejected] HEAD -> refs/publish/master/474518 (cannot add patch set to 474518.)
error: failed to push some refs to 'ssh://wohlstand@gerrit.wikimedia.org:29418/mediawiki/extensions/SyntaxHighlight_GeSHi.git'

// Idk how to fix the "remote: Pushing to refs/publish/* is deprecated, use refs/for/* instead." warning, is this a reason of failure?

I have checked everything, but I can't find a reason for this failure... :

git version 2.17.1
git-review version 1.26.0

Gerrit Patch uploader also fails, it won't log in and drops a 500 error on attempt to login:
https://tools.wmflabs.org/gerrit-patch-uploader/

So, I posting my updated patch here as a file:

Okay, I have installed git-review 1.28.0 by pip, however, it still fails:

$ git review -R
remote: 
remote: Processing changes: refs: 1
remote: Processing changes: refs: 1, done            
To ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/SyntaxHighlight_GeSHi.git
 ! [remote rejected] HEAD -> refs/for/master (cannot add patch set to 474518.)
error: failed to push some refs to 'ssh://wohlstand@gerrit.wikimedia.org:29418/mediawiki/extensions/SyntaxHighlight_GeSHi.git'
$ git branch
  master
* review/gerrit_patch_uploader/474518

Hi @Wohlstand, sorry that you run into problems. ;( I don't know which exact steps you performed before running git review -R or what created 474518, so it's hard to debug without a full list of steps. As this problem is unrelated to the topic that this task is about, I recommend to ask in a Wikimedia developer support forum, for example https://discourse-mediawiki.wmflabs.org/

Hi, @Aklapper , I did next:

# re-clonned repo from Gerrit
git clone ssh://wohlstand@gerrit.wikimedia.org:29418/mediawiki/extensions/SyntaxHighlight_GeSHi.git
cd SyntaxHighlight_GeSHi
# applied my patch with a change (added a new commit)
git am --signoff < 0001-Allow-Pygments-to-be-run-via-CGI.patch
# Then
git review -r origin -s
# and finally
git review -r origin -R

I'll try to ask the question on the forum, but this whole thing looks like a weird mess...

@Aklapper I think it could be due to security measures: the patch was authored by the uploader tool, and only people in the trusted contributors group can amend others' patches.

Thanks, @Daimona for help!
Anyway, I see it does a test of old errors I have fixed in an updated patch:

TheDJ added a subscriber: TheDJ.Jul 31 2019, 11:31 AM

made some of those changes for you, but notice the raw curl usage, which instead should be https://doc.wikimedia.org/mediawiki-core/master/php/classMWHttpRequest.html

Ok, @TheDJ, will try to change the CURL with MWHttpRequest thing and then, will post an updated patch. I have pulled your minor fixes, thanks for them!

@Wohlstand if you're familiar with CLI git, I suggest you to upload a brand new patch owned by yourself, so that you can later amend it. I believe all you have to do is:

cd path/to/SyntaxHighlight
git review -d 474518
git commit

Then delete the Change Id line so that it'll be recognized as a new patch, and then git review -R. If the upload is successful, you can then abandon https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/SyntaxHighlight_GeSHi/+/474518/ using the "Abandon" button.

Ok, thanks for a tip, @Daimona. I'm pretty familiar with CLI git as I using it for a long time (since 2014) for my own projects. Once I'll make a proper thing, I'll try to upload it again.