Page MenuHomePhabricator

Split out the 3 standalone tools from the DMCA toolbox and deploy them on tool labs
Closed, ResolvedPublic5 Estimated Story Points

Description

The code is at https://github.com/jamesryanalexander/tools-standalone/tree/776e429770615ee9f2bb6f1e39186af57fe6c42e

The 3 tools are:

  • globalsearch.php - Tool for searching for a phrase on all wikimedia wikis, currently requires and uses mediawiki OAuth.
  • globallinks.php - Tool for searching for a specific url linked from any wiki.
  • globalpagetext.php - Tool to show the text on a specific page given across all wikis (usually used to compare the status of a mediaWiki interface page).
What this task includes:
  • Splitting out the requisite code (but not doing any refactoring unless absolutely necessary, we can do that separately if needed)
  • Adding some barebones bootstrap UI for them
  • Deploying on tool labs (all 3 as part of the same tool)

Note: This task does not involve adding OAuth support for using the tools.

Event Timeline

DannyH triaged this task as Medium priority.May 31 2017, 5:11 PM
DannyH set the point value for this task to 5.

Also, there is no license in that code repository, so as far as I know, we don't have any right to modify it. :(

I've asked for a license to be added:
https://github.com/jamesryanalexander/tools-standalone/issues/1

Where should the code live? Should I request a new repository on https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests ?

I don't think you should modify it for this task. You can create a new repo under your account in Github and use the code in the standalone-tools folder and start from there.

For T167187: Epic ⚡️ : Implement DMCA and CP takedown report tools you can fork the current repo and work on it under your account.

You don't need to request Gerrit repositories for either, I think. You can keep working on it on Github.

You do need to request a tool-labs tool for the current task. You can read more about it here: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs

What should we call the "Service Group"?

Also, what user should I authenticate as (which seems to be an environment variable)? And where would I get a "consumer key"?

What should we call the "Service Group"?

I'd vote for "globalsearch" since that is what the three endpoints attempt to do.

Also, what user should I authenticate as (which seems to be an environment variable)? And where would I get a "consumer key"?

I think this can be done without auth at all. I don't think the Action API actions being used return anything different based on user permissions.

+1 to globalsearch and not using Oauth.

Also, is it within this ticket to remove the dependency on https://github.com/jamesryanalexander/tools-coreinclude ? (Or whichever other code this tool uses.) If not, that dependency should be refactored and loaded with Composer, I reckon.

So it looks like it uses OAuth in order to get a list of Wikimedia sites. Then once it has the list, it performs the search against each one of the site's API's and then merges the results. I'll see if I can track down where it's getting this list and if OAuth is really required or not.

I added Composer to the library so now it can be loaded (and autoloaded!)
https://github.com/davidbarratt/tools-coreinclude

Welp! doesn't need OAuth. https://meta.wikimedia.org/w/api.php?action=sitematrix&format=json

Honestly I think this should be rewritten as a SPA in JavaScript. :)

@dbarratt: Looks good. A few loose ends to tie up:

  • Please add myself (kaldari) and Jalexander as maintainers of the Tool Labs project (via this page)
  • Please move the repo under https://github.com/wikimedia/ (this might involve folking the old repo and then deleting it)
  • Change the Tool Labs tool to pull from the new GitHub repo

Once that is done, feel free to mark as resolved and move to the Done column.

Welp! doesn't need OAuth. https://meta.wikimedia.org/w/api.php?action=sitematrix&format=json

Honestly I think this should be rewritten as a SPA in JavaScript. :)

Yeah, it should be, the ONLY reason it was using OAuth was because I could reuse shared code. It totally shouldn't be and was planned to be rewritten when moved out.

Done!

https://tools.wmflabs.org/globalsearch/

I also created T167544: Rewrite Global Search (globalsearch) in JavaScript because the tool should really be rewritten. :)

\o/

@dbarratt If you add me (https://github.com/bd808) as a "Collaborator" on the current GitHub repo you are using I can migrate it to the Wikimedia organization for you. The migration will leave a redirect for the current URLs that transparently forward to the new location. See https://www.mediawiki.org/wiki/Manual:Developing_libraries#Transferring_an_existing_GitHub_repo_to_Wikimedia for the procedure.

@dbarratt If you add me (https://github.com/bd808) as a "Collaborator" on the current GitHub repo you are using I can migrate it to the Wikimedia organization for you.

@bd808 I added you as a collaborator on https://github.com/davidbarratt/tools-standalone and https://github.com/davidbarratt/tools-coreinclude

@bd808: Can you change the name of the repos while you're at it, to globalsearch-standalone and globalsearch-coreinclude?

@dbarratt: Why are there 2 different repos?

@dbarratt: Why are there 2 different repos?

tools-standalone has a dependency on tools-coreinclude which is here:
https://github.com/davidbarratt/tools-standalone/blob/master/composer.json#L7

However, now that I've removed OAuth, I think it's a single function that it's depending on, which I could just copy over if you would prefer.

However, now that I've removed OAuth, I think it's a single function that it's depending on, which I could just copy over if you would prefer.

That would be awesome if we could consolidate them.

@bd808: Can you change the name of the repos while you're at it, to globalsearch-standalone and globalsearch-coreinclude?

The normal naming convention would be to call the consolidated repo "tools-globalsearch".

dbarratt subscribed.

@bd808 Let me know when it's moved over on GitHub and I'll change the origin on Tools.

https://github.com/wikimedia/tools-globalsearch

I gave the Community Tech team admin permission over the repo.

Only things left to do are update the README and change the tool instance to pull from the new repo.

Tool updated

tools.globalsearch@tools-bastion-03:~/public_html$ git remote -v
origin	https://github.com/wikimedia/tools-globalsearch.git (fetch)
origin	https://github.com/wikimedia/tools-globalsearch.git (push)