Page MenuHomePhabricator

Change http to https in Community Tech bot API usages (and other places)
Closed, ResolvedPublic0.5 Story Points

Description

Per https://lists.wikimedia.org/pipermail/wikitech-l/2016-May/085618.html we should update our API usages. Creating this ticket to keep a track of this.

We need to test, make sure nothing's broken.

Event Timeline

Niharika created this task.May 20 2016, 3:52 AM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptMay 20 2016, 3:52 AM

I checked all the repos on my machine and couldn't find any still using the http URL.

The bot was flagged by a script which monitors incoming bot traffic for http requests, so there's got to be something. I'm going to check if there's something in database-reports using http. Or something our bot relies on which uses that.

Niharika added a subscriber: bd808.EditedMay 24 2016, 2:37 PM

My best guess is it's mwclient doing it. Mwclient was updated to use https in February while our bot is using it since last year (which is inconsequential).
@bd808 does mwclient on labs need to be updated?

bd808 added a comment.May 24 2016, 4:03 PM

My best guess is it's mwclient doing it. Mwclient was updated to use https in February while our bot is using it since last year (which is inconsequential).
@bd808 does mwclient on labs need to be updated?

Looks like it:

tools-bastion-02.tools:~
bd808$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mwclient
>>> mwclient.__ver__
'0.8.0.dev1'
bd808 added a comment.May 24 2016, 4:07 PM

See also T105794#2314347 for a note about setting scheme='https' in config to deal with protocol relative URLs.

bd808 added a comment.May 24 2016, 4:20 PM

I filed T136106: Upgrade globally installed python-mwclient to 0.8.1 to get the global package upgraded. The quick fix for any given tool would be to switch to using a virtualenv that installs the newer package locally.

DannyH triaged this task as Normal priority.May 24 2016, 5:32 PM
DannyH updated the task description. (Show Details)
DannyH set the point value for this task to 0.5.
BBlack added a subscriber: BBlack.EditedMay 25 2016, 2:03 PM

Now that the package is updated, is there a process we can go through to get individual bots to use the new software (as in, do we need restarts of long-running daemons)? Also, the scheme='http' issue in mwclient seems like a bug. Shouldn't it be taking the scheme from the configured base URI if it encounters proto-rel links while parsing things?

Now that the package is updated, is there a process we can go through to get individual bots to use the new software (as in, do we need restarts of long-running daemons)?

I sent a quick announcement to labs-l about restarting long running jobs using mwclient.

Also, the scheme='http' issue in mwclient seems like a bug. Shouldn't it be taking the scheme from the configured base URI if it encounters proto-rel links while parsing things?

That does seem like something that could be fixed upstream. @Steinsplitter could you file a bug/feature request with mwclient? I'd file one but I'm not sure how to describe triggering this issue.

I sent a quick announcement to labs-l about restarting long running jobs using mwclient.

Thanks!

@Niharika @bd808: I assume this bug is resolved now?

kaldari assigned this task to bd808.Jun 2 2016, 4:49 PM
Restricted Application added a project: User-bd808. · View Herald TranscriptJun 2 2016, 4:49 PM
bd808 closed this task as Resolved.Jun 2 2016, 5:54 PM

Current report at https://logstash.wikimedia.org/#/dashboard/elasticsearch/api-feature-usage-http looks clean for "Community Tech bot".

bd808 moved this task from To Do to Done on the User-bd808 board.Jun 2 2016, 8:21 PM
bd808 moved this task from Done to Archive on the User-bd808 board.Jun 3 2016, 11:01 PM
DannyH moved this task from Untriaged to Archive on the Community-Tech board.Jun 6 2016, 11:57 PM