Page MenuHomePhabricator

Help CorenBot migrate to a new API
Closed, ResolvedPublic8 Estimated Story Points

Description

Now that we're losing access to Yahoo's index, we need to migrate existing copyvio tools to using a different API.

Event Timeline

DannyH created this task.Mar 29 2016, 5:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 29 2016, 5:16 PM
DannyH renamed this task from Help copyvio tools migrate to a new API to Help CorenBot migrate to a new API.Mar 29 2016, 5:26 PM
DannyH set the point value for this task to 8.
kaldari added a subscriber: coren.Apr 2 2016, 6:06 PM
kaldari added a subscriber: kaldari.

The authorization for the new API is done through a basic authorization header passed as part of the API request.

In PHP this would be implemented something like:

$accountKey = 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA';
$webSearchURL = 'https://api.datamarket.azure.com/Bing/Search/Web?$format=json&Query=';
$context = stream_context_create(array(
  'http' => array(
    'request_fulluri' => true,
    'header'  => "Authorization: Basic " . base64_encode($accountKey . ":" . $accountKey)
  )
));
$request = $webSearchURL . urlencode( '\'' . $_POST["searchText"] . '\'');
$response = file_get_contents($request, 0, $context);
$jsonobj = json_decode($response);

@coren: Let me know what I can do to help. Do you have the code in a version control system?

The new Google Search API is now available for use from Tool Labs.

API documentation:
https://developers.google.com/custom-search/json-api/v1/using_rest#making_a_request

Proxy documentation:
https://wikitech.wikimedia.org/wiki/Nova_Resource:Google-api-proxy

Ping me to get the key and cx value.

kaldari added a comment.EditedJun 3 2016, 11:48 PM

Sent the info to Coren by Wikipedia email.

CorenSearchBot is back up and running! Yay!

kaldari closed this task as Resolved.Jun 7 2016, 10:30 PM
kaldari claimed this task.
kaldari moved this task from Ready to Q1 2018-19 on the Community-Tech-Sprint board.
DannyH moved this task from Estimated to Archive on the Community-Tech board.Jun 20 2016, 8:20 PM