Page MenuHomePhabricator
Create Task
Maniphest T140925

merlbot cron jobs disabled due to HTTP POST errors
Closed, ResolvedPublic
Actions

Description

The conclusion of T121279: Figure out a way to keep MerlBot running when the HTTP POST loophole is closed was that the bot could not be updated to use HTTPS for all POST requests to the Wikimedia production Action API. Fixing this requires source code changes to the Java applications that the merlbot tool runs.

Per T121279#2480749 the job entries for the merlbot tool have been commented out in the tool's crontab file. A backup was made as crontab-201607201730.txt and the entries commented out were marked with ## bd808 disabled for T121279 # to make finding them easier in the file.

These jobs are only disabled to keep them from consuming resources on the job grid while being ineffective in updating wikis due to the HTTPS-only server restriction. Any and all of them can and should be re-enabled as soon as the applications themselves are fixed.

Related Objects
Search...

Event Timeline

bd808 created this task.Jul 20 2016, 5:46 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 20 2016, 5:46 PM
bd808 mentioned this in T121279: Figure out a way to keep MerlBot running when the HTTP POST loophole is closed.Jul 20 2016, 5:49 PM
bd808 added a comment.Jul 20 2016, 5:57 PM

A typical error message from a failing job is:

WARNING: unexcpected Status code: 403 (Insecure Request Forbidden - use HTTPS - https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2016-May/000110.html):
Jul 20, 2016 4:39:28 AM de.mabomuja.mabomwframework.impl.cluster.mw.MwServerClusterImpl getSite0
WARNING: unexcpected Status code: 403 (Insecure Request Forbidden - use HTTPS - https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2016-May/000110.html):
Exception in thread "main" java.lang.NullPointerException
        at de.mabomuja.mabomwframework.impl.cluster.mw.MwServerClusterImpl.doRequest(MwServerClusterImpl.java:181)
        at de.mabomuja.mabomwframework.impl.cluster.mw.MwServerClusterImpl.getSiteMatrix0(MwServerClusterImpl.java:406)
        at de.mabomuja.mabomwframework.impl.cluster.mw.MwServerClusterImpl.getSiteMatrix(MwServerClusterImpl.java:393)
        at de.mabomuja.mabomwframework.impl.cluster.mw.MwServerClusterImpl.getSiteForId(MwServerClusterImpl.java:229)
        at de.mabomuja.mabomwframework.api.mw.MwInstanceManager.getSiteForId(MwInstanceManager.java:24)
        at de.mabomuja.mabomwframework.merlbot.lists.defects.W_WikidataProperties.main(W_WikidataProperties.java:91)

I think the jobs could be fixed by adding code to configure the Apache HttpCore library to install a [[https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e485|SystemDefaultRoutePlanner]]. The jobs are already configured to run java with -Dhttp.proxyHost=tools-merlbot-proxy.tools.eqiad.wmflabs -Dhttp.proxyPort=80 command line arguments that will proxy all requests through a http->https reverse proxy that was created just for merlbot (T137235).

bd808 raised the priority of this task from Lowest to High.Jul 20 2016, 5:57 PM
Luke081515 subscribed.Jul 20 2016, 5:57 PM
bd808 assigned this task to Merl.Jul 20 2016, 5:57 PM
Aklapper removed Merl as the assignee of this task.Mar 11 2020, 1:51 PM

@Merl was last active in 2016. Hence I am resetting the assignee of this task.
Resetting the assignee avoids the impression that somebody is already working on this task.

It is unclear to me if someone else could potentially work on fixing this task, as I do not if the codebase is public and where it is located.

bd808 added a comment.Mar 11 2020, 2:55 PM
In T140925#5960223, @Aklapper wrote:

It is unclear to me if someone else could potentially work on fixing this task, as I do not if the codebase is public and where it is located.

The code base is not public and source is not available on the Toolforge bastions. There are only binary jar files there.

bd808 closed this task as Resolved.Nov 2 2020, 6:19 PM
bd808 claimed this task.
Restricted Application added a project: User-bd808. · View Herald TranscriptNov 2 2020, 6:19 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL