Maniphest T219935

purgeUnusedProjects.php is causing fatals on Beta
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

Command: /srv/mediawiki/multiversion/MWScript.php extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki=enwikivoyage
Request URL: n/a (cron job)
Request ID: ddc0bb1c594cd3d41d5d2972

message

[ddc0bb1c594cd3d41d5d2972] [no req]   Wikimedia\Rdbms\DBQueryError from line 1585 of /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Query: DELETE FROM `page_assessments_projects` WHERE (pap_project_id NOT IN ())
Function: PurgeUnusedProjects::execute
Error: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '))' at line 1 (172.16.5.170:3306)

trace

#0 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1554): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#1 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1272): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#2 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(3028): Wikimedia\Rdbms\Database->query(string, string)
#3 /srv/mediawiki/php-master/extensions/PageAssessments/maintenance/purgeUnusedProjects.php(59): Wikimedia\Rdbms\Database->delete(string, string, string)
#4 /srv/mediawiki/php-master/maintenance/doMaintenance.php(96): PurgeUnusedProjects->execute()
#5 /srv/mediawiki/php-master/extensions/PageAssessments/maintenance/purgeUnusedProjects.php(71): include(string)
#6 /srv/mediawiki/multiversion/MWScript.php(100): include(string)
#7 {main}

Impact

N/A

Notes

MediaWiki: 1.33.0-alpha (2912916)
PageAssesments: 1.1.0 (4200d61)
PHP version: 5.6.99-hhvm
Logstash: https://kibana4.wmflabs.org/goto/2fb274a4a21d335361ab5a11a3232d79

Details

	Subject	Repo	Branch	Lines +/-
	Protect against empty array causing sql syntax error	mediawiki/extensions/PageAssessments	master	+6 -0

Customize query in gerrit

Related Objects

Mentioned In: T208231: Issues with purgeUnusedProjects.php cron job on mwmaint1002 (Fri Oct 26, 2018)
Mentioned Here: T208231: Issues with purgeUnusedProjects.php cron job on mwmaint1002 (Fri Oct 26, 2018)
rEPAS4200d61cf9ef: Use ::class for class name resolution
rMW29129160af3a: Merge "Style change tags correctly on log extracts"

Event Timeline

MarcoAurelio created this task.Apr 2 2019, 9:22 PM

Restricted Application added a project: Community-Tech. · View Herald TranscriptApr 2 2019, 9:22 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

MarcoAurelio renamed this task from xtensions/PageAssessments/maintenance/purgeUnusedProjects.php is causing fatals on Beta to extensions/PageAssessments/maintenance/purgeUnusedProjects.php is causing fatals on Beta.Apr 2 2019, 9:22 PM

MarcoAurelio mentioned this in T208231: Issues with purgeUnusedProjects.php cron job on mwmaint1002 (Fri Oct 26, 2018).

@MarcoAurelio I don't think this is the same problem as T208231: Issues with purgeUnusedProjects.php cron job on mwmaint1002 (Fri Oct 26, 2018).

Can we disable the cron job on beta?

https://phabricator.wikimedia.org/diffusion/EPAS/browse/master/maintenance/purgeUnusedProjects.php$58

$conds = [ 'pap_project_id NOT IN (' . $dbr->makeList( $usedProjectIds ) . ')' ];

should be, perhaps?

$conds = [ 'pap_project_id NOT IN '(' . $dbr->makeList( $usedProjectIds ) . ')' ];

I see traces of this error going back up to one year as of today in logstash-beta. I'm guessing PageAssessments isn't successfuly deleting unused projects since, at least, some months.

In T219935#5079562, @Niharika wrote:

@MarcoAurelio I don't think this is the same problem as T208231: Issues with purgeUnusedProjects.php cron job on mwmaint1002 (Fri Oct 26, 2018).

Can we disable the cron job on beta?

I think beta now uses the same file production uses to run their jobs. I'm not sure if it is possible to deactivate this cron job for Beta only; but if it is a syntax error it should be fixed.

In T219935#5079587, @MarcoAurelio wrote:
https://phabricator.wikimedia.org/diffusion/EPAS/browse/master/maintenance/purgeUnusedProjects.php$58
$conds = [ 'pap_project_id NOT IN (' . $dbr->makeList( $usedProjectIds ) . ')' ];

I think this syntax is correct. I think the issue is that the $usedProjectIds is empty so there's nothing in the SQL clause for the NOT IN () to filter by.

This code was added in this commit: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/PageAssessments/+/c3fcf6926f4c53fb4eb3e1f980f04a3292cf354e%5E%21/maintenance/purgeUnusedProjects.php

I'm taking a look now to see why that array might be empty.

@aezell Possibly because beta doesn't have any wikiprojects. The code should be able to handle this but I think we never ran into this before on other projects.
Not sure why we are only just now seeing this on beta.

@kaldari SOS.

I should have a patch shortly.

Change 500850 had a related patch set uploaded (by Aezell; owner: Aezell):
[mediawiki/extensions/PageAssessments@master] Protect against empty array causing sql syntax error

https://gerrit.wikimedia.org/r/500850

gerritbot added a project: Patch-For-Review.Apr 2 2019, 9:50 PM

Change 500850 merged by jenkins-bot:
[mediawiki/extensions/PageAssessments@master] Protect against empty array causing sql syntax error

https://gerrit.wikimedia.org/r/500850

Mentioned in SAL (#wikimedia-releng) [2019-04-02T22:24:24Z] <hauskatze> maurelio@deployment-deploy01:~$ mwscript extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki=enwikivoyage | T219935

maurelio@deployment-deploy01:~$ mwscript extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki=enwikivoyage
Projects before purge: 0
No projects found.
Done.

Looks good! Thanks!

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.25; 2019-04-09).Apr 2 2019, 11:01 PM

• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM

Maintenance_bot removed a project: Patch-For-Review.Aug 28 2019, 11:13 PM

kaldari moved this task from Ready to Done on the MediaWiki-extensions-PageAssessments board.Apr 7 2020, 8:06 PM

extensions/PageAssessments/maintenance/purgeUnusedProjects.php is causing fatals on BetaClosed, ResolvedPublicPRODUCTION ERRORActions