We are approaching the Outreachy'11 application deadline, and if you want to have your proposal considered to be part of this round, do sign up and add your proposal at https://outreachy.gnome.org/ before November 02 2015, 07:00 pm UTC. You can copy-paste the above proposal to the Outreachy application system, and keep on polishing it over here. Keep in mind that your mentors and the organization team will be evaluating your proposal here in Phabricator, and you are free to ask and get more reviews complying https://www.mediawiki.org/wiki/Outreach_programs/Life_of_a_successful_project#Answering_your_questions

Congrats on writing a proposal!

Quick comment on tag syntax: the commitID need not be a requirement, it should default to HEAD. Likewise, repo can be optional and default to MediaWiki core. Also the repo should not be a URL but a project specifier within the repo. Take a look at mediawiki.org's Template:Git_file for intelligent defaults; this things should be like that template, only it actually transcludes the file contents rather than link to it.. It's a good question whether editors should be able to say "Transclude this file from a GitHub/Gitlab/random repo" (instead of from gerrit.wikimedia.org or phabricator.wikimedia.org/diffusion) in the parser tag, or whether it should only be a config option, see Extension:GitHub's $egSomethingUrl variable.

Likewise, repo can be optional and default to MediaWiki core.

I don't think it should. Extensions should not hardcode Wikimedia usecases. For convenient defaults, you can always use templates.

Profile:

• Please clarify the time zone for the hours in the profile.

Synopsis:

• It might be worth mentioning the structured data use case (e.g. pulling extension.json and providing the data therein to the wiki template that creates the extension infobox) since it might take some dedicated functionality to do that. Of course if you decide that does not fit into the roadmap that's fine.

Step 1:

• You should give some more thought to how the repo parameter will work. You can use a git URL to specify the repo, every git repo will have that. There is no straightforward way to turn that into a HTTP request though. You can use git to get the content, but that's not trivial so you should give it some thoughts how exactly you will do that if you take that route. Alternatively, you can use a HTTP request to fetch the file, but there is no uniform way of putting a base URL + branch name + file name together into a full URL. (Have a look at how GitHub and git.wikimedia.org do it. You could check some other large git hosts, too.) Also if you go the HTTP route you might try end up with something that's not specific at all to git.
• there is very little difference between commit IDs and branch names in git, they are both ways to refer to a specific commit. There are other ways (tags are the most important ones, but see this explanation of a commitish for the general case). No point in having separate parameters for those.
• startline/endline is not that useful (compared to e.g. a regex). It might be handy in a few cases, but there should be better ways. (Again if you think those alternatives do not fit into your schedule that's fine, but should be mentioned explicitly.) I see that's mentioned later in some places, but it's a bit confusing.
• it's unclear what the code parameter does.
• if you request a file over HTTP, how do you determine the commit ID (to use as a caching key)? Do you require wiki editors to always provide one?
• how does the extension choose which formatting to use? Guess based on file extension?
• "Mediawiki files and Wiki files ... These files will be rendered as it is." - you probably mean their rendering will be left to the normal parser (ie. the extension will just pass them as wikitext).
• return array( "<syntaxhighlight>$text</syntaxhighlight>", 'nowiki' => true, 'noparse' => true, 'isHTML' => true ); will result in a <syntaxhighlight> tag in the HTML code (which does not do anything). Invoking another tag extension is a bit more troublesome than that, and IMO there is no reason to do it at all. Whoever invokes your extension can always do <syntaxhighlight>{{#GitAtWiki|...}</syntaxhighlight> on their side (or something like that. The actual syntax is a bit more complicated.)

Step 2:

• As said above, getting the commit id is not trivial. If the editor specifies something like "master", a HTTP request won't give it to you. If the editor specifies a commit id, there is no need for cache invalidation at all.
• Not sure you should bother with line numbers in the cache - if the file changes, just update. The wikitext parser has its own cache; if the text provided by your parser function is the same, the update will be cheap.
• It's a bit unclear what you expect the git hook to do. (Looking at the GitHub docs might help to give you an idea of how they work.

Step 3:

• You probably want to do the processing in PHP and just have a lua module for interfacing with it. Doing things in Lua is hard. Look at e.g. the Site lua library - it's just a bunch of wrappers for PHP functionality.
• You can also make the whole Lua thing into a stretch goal that you attempt if you progress faster than planned but don't put it on the schedule.

Deliverables/Timeline:

• I don't think it's realistic to split "write code for feature X" and "test it / get it reviewed". Generally the process is smoother when you write code in small steps and publish them as soon as possible ("commit early, commit often") and then work on the next small patch while waiting for reviews.
• An MVP is not an MVP unless it can be deployed right there and then (and we should deploy it right then, at least to the beta site). You might want to set apart some time for that after reaching the MVP stage (getting a security review, setting up the configuration etc - see Writing an extension for deployment).

Participation:

• it's a good idea to commit to a specific update schedule (say, weekly reports).

@Tgr :

Please clarify the time zone for the hours in the profile.

I guess you mean this :

Time Zone: UTC+5:30
Typical working hours: (Indian Standard Time)

right?

It might be worth mentioning the structured data use case (e.g. pulling extension.json and providing the data therein to the wiki template that creates the extension infobox) since it might take some dedicated functionality to do that. Of course if you decide that does not fit into the roadmap that's fine.

What about composer.json for the case specific in which extension.json doesn't exist and for the cases where extension.json and composer.json both exist.

It's a bit unclear what you expect the git hook to do. (Looking at the GitHub docs might help to give you an idea of how they work.

I want to add a post commit hook which triggers the update remotely from Jenkins. I know Jenkins-fu might get complicated but for now, I can't think a better solution to trigger the update. comments?

how does the extension choose which formatting to use? Guess based on file extension?

yes.

"Mediawiki files and Wiki files ... These files will be rendered as it is." - you probably mean their rendering will be left to the normal parser (ie. the extension will just pass them as wikitext).

yes

I disagree with @Tgr, I think line numbers are a useful way to specify a snippet. The wiki editor should use them with an unchanging commit (otherwise the file contents change), but its her choice

Your step 2.

Idea is to maintain a mapping/database of source (git-doc) and destination (wiki-page) and adding a post commit hook which triggers the update remotely from Jenkins.

is interesting. If the wikipage is pulling in HEAD, then depending on caching the page will remain the same even when git updates. I don't know how much caching the parser cache and page cache do. If it's only cached for a day or so and editors can force new contents with ?action=purge, then maybe we don't need this complicated dependency handling.

In T117167#1794796, @Spage wrote:

1 .

I think line numbers are a useful way to specify a snippet. The wiki editor should use them with an unchanging commit (otherwise the file contents change)

First case: One idea is that we don't update these snippets as they are coming from a fixed commit (not from master) hence we don't need any update mechanism for these types of cases.
Second case: If we need to update them as this is the motto of the project then I am not sure how to find a simple and safe solution to update such snippets. I tried to use one in my previous project proposal as:

Idea is to maintain a mapping/database of source (git-doc), destination (wiki-page), commit-id (from which the content is taken), start line number (The start line number from where the content was pulled) and end line number (The end line number till where content was pulled). After any commit in the corresponding file, check the difference between the content of git-doc between the start line and end line of commit-id (saved in mapping) and the content of git-doc between start line and end line patterns in the most recent commit.
( Why patterns? : It might be the case that start line and end line numbers get modified after the most recent commit. By comparing the content using the line numbers might give a false information. Hence, the solution is to search for the pattern, which are present at git-doc' start line and at end line (for the saved commit-id in mapping), in the most recent commit.)
If a difference is reported by above comparison then previously saved commit-id, start line and end line will be replaced by present commit-id, start line and end line number.

But it uses a database and a complex approach.
Which one is good to proceed with?
2 .

Your step 2.

Idea is to maintain a mapping/database of source (git-doc) and destination (wiki-page) and adding a post commit hook which triggers the update remotely from Jenkins.

is interesting. If the wikipage is pulling in HEAD, then depending on caching the page will remain the same even when git updates. I don't know how much caching the parser cache and page cache do. If it's only cached for a day or so and editors can force new contents with ?action=purge, then maybe we don't need this complicated dependency handling.

In T116277#1794773, @Spage wrote:

I believe to start this extension should just rely on the page cache and maybe the parser cache.

• Is this good to make editors to use ?action=purge to see the updated snippets each and every time?
• To automate the above process, we need to trigger the update (delete the cache etc.) after each commit in the corresponding file and to know which page' cache needs to be deleted, we need to have a database/mapping. comments?

If you can avoid having a database in the MVP version, you definitely should. It makes things a lot simpler. Maybe you find later you really need to add it, maybe other tasks prove more important. The wikitext parser has a cache, expiration of that cache is a dumb but very easy to use update mechanism; just set it to something short, like one day.